Coding method of three-dimensional grid, decoding method of three-dimensional grid and related devices

Abstract

The embodiment of the invention discloses a coding method of a three-dimensional grid, which can code the three-dimensional grid into a feature sequence, and since the number of repeated vertexes in a plurality of grid subgraphs corresponding to the feature sequence is small, resources for storing the vertexes can be saved, and effective compression of the three-dimensional grid can be realized. The encoding method of the three-dimensional grid comprises the steps that after the encoding device divides the three-dimensional grid into a plurality of grid subgraphs, a feature subsequence corresponding to each grid subgraph is generated, the feature subsequence comprises all vertex coordinates of the grid subgraphs and an adjacent matrix of the grid subgraphs, and then a feature sequence comprising all the feature subsequences is generated. The invention also provides a decoding method, a coding device and a decoding device of the three-dimensional grid.

Description

Technical Field

[0001] The present application relates to the field of cloud computing, and in particular to a three-dimensional grid encoding method, a three-dimensional grid decoding method and related devices. Background Art

[0002] A 3D mesh is a collection of polyhedral shapes, vertices, and polygons on the surface of a three-dimensional object.

[0003] Currently, there is a three-dimensional mesh encoding method comprising: dividing the three-dimensional mesh into N triangular meshes, each triangular mesh including 3 vertices, and composing the vertex coordinates of all triangular meshes into a sequence, wherein the sequence includes 3N vertex coordinates.

[0004] Such sequences require more storage resources. Summary of the invention

[0005] The present application provides a three-dimensional mesh encoding method, which can generate multiple mesh subgraphs, each of which includes at least two non-repeating vertices, thereby reducing the storage resources required to store the vertices, thereby effectively compressing the three-dimensional mesh. The present application also provides a three-dimensional mesh decoding method, encoding device, decoding device, computing device, computer-readable storage medium and computer program product.

[0006] In a first aspect, a method for encoding a three-dimensional grid is provided, the method comprising: an encoding device divides the three-dimensional grid into a plurality of grid subgraphs, generates a feature subsequence corresponding to each grid subgraph, and generates a feature sequence including all feature subsequences. The feature subsequence includes all vertex coordinates of the grid subgraph and an adjacency matrix of the grid subgraph.

[0007] The multiple grid subgraphs include a first type of grid subgraph, each grid subgraph in the first type of grid subgraph includes n target vertices and n adjacent points of the target vertices, and the target vertices of any two grid subgraphs in the first type of grid subgraph have no intersection. Therefore, there are fewer repeated vertices in the multiple grid subgraphs, and the total number of vertices of all grid subgraphs is less than the number of vertices of all triangular meshes. Therefore, the amount of vertex data of all grid subgraphs is smaller, and the characteristic subsequences including vertex coordinates require less storage resources, which can achieve effective compression of three-dimensional meshes.

[0008] In some possible implementations, the encoding device divides the three-dimensional grid into multiple grid subgraphs including steps A to C; wherein step A includes: the encoding device selects n target vertices from the three-dimensional grid to be processed, step B includes: the encoding device determines that the grid subgraph includes a first vertex set and a first adjacent edge set; step C includes: the encoding device updates the three-dimensional grid to be processed to the remaining part after removing the n target vertices and the first adjacent edge set in the three-dimensional grid to be processed; and steps A to C are executed repeatedly until the number of adjacent points of each vertex in the three-dimensional grid to be processed is less than n. Wherein, the initial data of the three-dimensional grid to be processed is a three-dimensional grid, the n target vertices include the first target vertex and the first n-1 vertices of the adjacent points of the first target vertex, the first vertex set includes the collection of adjacent points of the n target vertices and the n target vertices, and the first adjacent edge set is the collection of adjacent edges of the n target vertices. The grid subgraphs selected in this way are all first-class grid subgraphs, thereby all or most of the three-dimensional grid can be divided into first-class grid subgraphs. There are fewer repeated vertices in the first-class grid subgraphs, so the feature sequences corresponding to the first-class grid subgraphs occupy less storage resources, which can achieve effective compression of the three-dimensional grid.

[0009] In some possible implementations, the encoding device divides the three-dimensional grid into multiple grid subgraphs including steps A to F, step A includes: the encoding device selects n target vertices from the three-dimensional grid to be processed; step B includes: the encoding device determines that the grid subgraph includes a first vertex set and a first adjacent edge set; step C includes: the encoding device updates the three-dimensional grid to be processed to the remaining part after removing the n target vertices and the first adjacent edge set in the three-dimensional grid to be processed; steps A to C are looped until the number of adjacent points of each vertex in the three-dimensional grid to be processed is less than n; step D includes: the encoding device selects the first target vertex and the adjacent point set from the three-dimensional grid to be processed; step E includes: the encoding device determines that the grid subgraph includes a second vertex set and a second adjacent edge set; step F includes: the encoding device updates the three-dimensional grid to be processed to the remaining part after removing the first target vertex, the adjacent point set and the second adjacent edge set in the three-dimensional grid to be processed; steps D to F are looped until the three-dimensional grid to be processed is empty. The initial data of the three-dimensional mesh to be processed is a three-dimensional mesh, the n target vertices include the first target vertex and the first n-1 vertices of the adjacent points of the first target vertex, the first vertex set includes the collection of the adjacent points of the n target vertices and the n target vertices, the first adjacent edge set is the collection of the adjacent edges of the n target vertices, the second vertex set is the collection of the first target vertex, the adjacent point set and the adjacent points of all vertices in the adjacent point set, the adjacent point set includes all adjacent points of the first target vertex, and the second adjacent edge set is the collection of the adjacent edges of the first target vertex and the adjacent edges of all vertices in the adjacent point set. In this way, the three-dimensional mesh can be divided into a large number of first-type mesh subgraphs and a small number of second-type mesh subgraphs, the total number of target vertices in the second-type mesh subgraphs is less than n, so that all mesh subgraphs include all adjacent edges and vertices of the three-dimensional mesh, and all mesh subgraphs have fewer repeated vertices, so the feature sequences corresponding to all mesh subgraphs require less storage resources, and lossless compression of the three-dimensional mesh can be achieved.

[0010] In some possible implementations, the serial numbers of each layer of vertices in the three-dimensional grid are an arithmetic progression, the difference between the minimum serial number of the vertices in the i+1th layer and the maximum serial number of the vertices in the i-th layer is equal to the common difference of the arithmetic progression, L is the number of vertex layers in the three-dimensional grid, and i is a positive integer less than L.

[0011] In some possible implementations, the serial numbers of each layer of vertices in the three-dimensional grid are an arithmetic progression, the difference between the maximum serial number of the vertices in the i-1th layer and the minimum serial number of the vertices in the i-th layer is equal to the common difference of the arithmetic progression, L is the number of vertex layers in the three-dimensional grid, and i is a positive integer less than L.

[0012] In some possible implementations, the first target vertex is the vertex with the smallest sequence number in the grid subgraph.

[0013] In some possible implementations, the first target vertex is the vertex with the largest sequence number in the grid subgraph.

[0014] In some possible implementations, n is 2, 3 or 4. In this way, first-class grid subgraphs of different sizes can be obtained, that is, a variety of feasible solutions for segmenting three-dimensional grids are provided. It should be understood that the present application can set n to other integers greater than 1 according to actual needs.

[0015] In some possible implementations, the feature subsequence further includes a first separator and a second separator, the first separator is located between the coordinates of the last target vertex and the coordinates of the first adjacent point, and the second separator is located between the coordinates of the last adjacent point and the adjacency matrix. The first separator is used to separate the last target vertex and the adjacent point of the target vertex, and the second separator is used to separate the adjacent point of the target vertex and the adjacency matrix.

[0016] In some possible implementations, the adjacency matrix of the grid subgraph includes T values, where T is the total number of vertices in the grid subgraph, and the value is used to indicate the connection relationship between a vertex and all vertices in the grid subgraph. Using T values ​​to represent the connection relationship of T vertices requires less storage resources.

[0017] In some possible implementations, the encoding method further includes: the encoding device inputs the feature sequence into a variational quantized variational autoencoder (VQVAE), and outputs a latent vector through the variational quantized variational autoencoder. The length of the feature sequence is greater than the length of the latent vector. The variational quantized variational autoencoder can compress the feature sequence, so that the feature sequence composed of all feature subsequences can be further compressed, thereby improving the compression rate.

[0018] The second aspect provides a three-dimensional grid decoding method, the method comprising: a decoding device receives a feature sequence, and decodes the feature sequence into a three-dimensional grid. The feature sequence includes a plurality of feature subsequences, and the feature subsequences are related to the grid subsequences of the three-dimensional grid. Figure 1 One-to-one correspondence, the grid subgraphs of the three-dimensional grid include the first type of grid subgraphs, each grid subgraph in the first type of grid subgraphs includes n target vertices and n adjacent points of the target vertices, and the target vertices of any two grid subgraphs in the first type of grid subgraphs have no intersection, and n is an integer greater than 1. This provides a method for restoring a feature sequence to a three-dimensional grid.

[0019] In some possible implementations, the grid subgraph of the three-dimensional grid further includes a second type of grid subgraph, and the number of target vertices in the second type of grid subgraph is less than n.

[0020] In some possible implementations, the above decoding method further includes: after the decoding device receives the latent vector, the latent vector is input into a decoder of the variational quantization autoencoder, and the three-dimensional grid is output through the decoder.

[0021] In some possible implementations, the three-dimensional grid decoding method further includes: a decoding device receives an intended text input by a user; when the intended text includes a keyword corresponding to the three-dimensional grid, the decoding device inputs the intended text into an autoregressive model, outputs a latent vector corresponding to the intended text through the autoregressive model, and then inputs the latent vector corresponding to the intended text into a decoder of a variational quantization autoencoder, and outputs a three-dimensional grid through the decoder. This provides a method for generating a three-dimensional grid based on a user's intended text, which has the advantages of being simple and efficient.

[0022] The third aspect provides a method for training a variational quantization autoencoder, the method comprising: after a data training device obtains a three-dimensional grid set, the three-dimensional grids of the three-dimensional grid set are respectively encoded into feature sequences to obtain a feature sequence set, and then model training is performed based on the feature sequence set to obtain a variational quantization autoencoder.

[0023] In some possible implementations, the variational quantization autoencoder includes an encoder and a decoder, the encoder is used to convert a feature sequence into a latent vector, and the decoder is used to generate a target feature sequence similar to the feature sequence according to the latent vector from the encoder.

[0024] In some possible implementations, the encoder includes a convolutional neural network and a quantized neural network, the convolutional neural network is used to convolve the feature sequence, and the quantized neural network is used to generate a latent vector based on a vector obtained by convolution of the convolutional neural network.

[0025] In some possible implementations, the quantization neural network is a cluster quantization neural network, a finite scalar quantization (FSQ) neural network, or a look-up free quantization (LFQ) neural network.

[0026] A fourth aspect provides a method for training an autoregressive model, the method comprising: after a data training device obtains a training set, a model training is performed according to the training set to obtain an autoregressive model. The training set includes a plurality of samples, the samples include keywords corresponding to a three-dimensional grid and a potential vector corresponding to the three-dimensional grid, the keywords corresponding to the three-dimensional grid are used as input data of the autoregressive model, and the potential vector corresponding to the three-dimensional grid is used as a label of the autoregressive model.

[0027] In some possible implementations, the autoregressive model is a transformer model.

[0028] The fifth aspect provides an encoding device, which includes a segmentation module and a serialization module. The segmentation module is used to divide the three-dimensional grid into multiple grid sub-graphs, and the serialization module is used to generate a feature sub-sequence corresponding to each grid sub-graph, and generate a feature sequence including all feature sub-sequences.

[0029] In some possible implementations, the segmentation module is specifically used to execute step A to step C in a loop until the number of neighboring points of each vertex in the three-dimensional mesh to be processed is less than n.

[0030] In some possible implementations, the segmentation module is specifically used to loop through steps A to C until the number of neighboring points of each vertex in the three-dimensional mesh to be processed is less than n; and then loop through steps D to F until the three-dimensional mesh to be processed is empty.

[0031] In some possible implementations, the encoding device further includes an encoding module, which is specifically configured to input the feature sequence generated by the serialization module into a variational quantization autoencoder, and output a potential vector through the variational quantization autoencoder.

[0032] The explanation of terms in the fifth aspect, the steps performed by each module and the beneficial effects can be found in the corresponding description of the first aspect.

[0033] A sixth aspect provides a decoding device, which includes a receiving module and a decoding module, the receiving module is used to receive a feature sequence, and the decoding module is used to decode the feature sequence received by the receiving module into a three-dimensional grid.

[0034] In some possible implementations, the grid subgraph of the three-dimensional grid further includes a second type of grid subgraph, and the number of target vertices in the second type of grid subgraph is less than n.

[0035] In some possible implementations, the receiving module is further used to receive a latent vector, and the decoding module is further used to input the latent vector into a decoder of a variational quantization autoencoder, and output a three-dimensional grid through the decoder.

[0036] In some possible implementations, the receiving module is also used to receive the intended text input by the user; when the intended text includes keywords corresponding to the three-dimensional grid, the decoding module is also used to input the intended text into the autoregressive model, and output the latent vector corresponding to the intended text through the autoregressive model; the latent vector corresponding to the intended text is input into the decoder of the variational quantization autoencoder, and the three-dimensional grid is output through the decoder.

[0037] The explanation of terms in the sixth aspect, the steps performed by each module and the beneficial effects can be found in the corresponding description of the second aspect.

[0038] The seventh aspect provides a data training device, which includes a serialization module and a training module. The serialization module is used to obtain a three-dimensional grid set and then encode the three-dimensional grids of the three-dimensional grid set into feature sequences to obtain a feature sequence set. The training module is used to perform model training according to the feature sequence set to obtain a variational quantization autoencoder.

[0039] In some possible implementations, the variational quantization autoencoder includes an encoder and a decoder, the encoder is used to convert a feature sequence into a latent vector, and the decoder is used to generate a target feature sequence similar to the feature sequence according to the latent vector from the encoder.

[0040] In some possible implementations, the encoder includes a convolutional neural network and a quantized neural network, the convolutional neural network is used to convolve the feature sequence, and the quantized neural network is used to generate a latent vector based on a vector obtained by convolution of the convolutional neural network.

[0041] In some possible implementations, the quantized neural network is a clustered quantized neural network, a finite scalar quantized neural network, or a query-free quantized neural network.

[0042] In an eighth aspect, a data training device is provided, which includes an acquisition module and a training module, wherein the acquisition module is used to acquire a training set, and the training module is used to perform model training according to the training set to obtain an autoregressive model. The training set includes multiple samples, and the samples include keywords corresponding to a three-dimensional grid and a potential vector corresponding to the three-dimensional grid. The keywords corresponding to the three-dimensional grid are used as input data of the autoregressive model, and the potential vector corresponding to the three-dimensional grid is used as a label of the autoregressive model.

[0043] In some possible implementations, the autoregressive model is a transformer model.

[0044] A ninth aspect provides a computing device comprising a processor and a memory, wherein the memory stores computer-readable instructions, and the processor executes the computer-readable instructions so that the computing device performs a method as in the above aspect or any possible implementation of the above aspect.

[0045] The tenth aspect provides a computer-readable storage medium, which includes computer-readable instructions; the computer-readable instructions are used to implement the method in the above aspect or any possible implementation of the above aspect.

[0046] The eleventh aspect provides a computer program product, which includes computer-readable instructions; the computer-readable instructions are used to implement the method in the above aspect or any possible implementation of the above aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1A schematic diagram of encoding a three-dimensional grid into a feature sequence in an embodiment of the present application; Figure 2 A schematic diagram of decoding a feature sequence into a three-dimensional grid in an embodiment of the present application; Figure 3 A schematic diagram of encoding a three-dimensional grid into a potential vector in an embodiment of the present application; Figure 4 Another schematic diagram of encoding a three-dimensional grid into a potential vector in an embodiment of the present application; Figure 5 A schematic diagram of decoding a potential vector into a three-dimensional grid in an embodiment of the present application; Figure 6 Another schematic diagram of decoding a potential vector into a three-dimensional grid in an embodiment of the present application; Figure 7 Another schematic diagram of decoding the intended text into a three-dimensional grid in an embodiment of the present application; Figure 8 A schematic diagram of a data training device in an embodiment of the present application; Fig. 9 Another schematic diagram of the data training device in the embodiment of the present application; Fig.10 A flowchart of a three-dimensional grid encoding method in an embodiment of the present application; Fig.11 A schematic diagram of the corresponding relationship between the grid subgraph and the adjacency matrix in an embodiment of the present application; Fig.12 A flow chart of segmenting a three-dimensional grid in an embodiment of the present application; Fig.13 Another flowchart of segmenting a three-dimensional grid in an embodiment of the present application; Fig.14 A schematic diagram of a three-dimensional grid in an embodiment of the present application; Fig.15 A schematic diagram of a three-dimensional grid subgraph in an embodiment of the present application; Fig.16 A schematic diagram of a decoding method in an embodiment of the present application; Fig.17 A structural diagram of a computing device in an embodiment of the present application; Fig.18 A structural diagram of a computing device cluster in an embodiment of the present application; Fig.19 This is another structural diagram of a computing device cluster in an embodiment of the present application. DETAILED DESCRIPTION

[0048] The three-dimensional mesh encoding method and the three-dimensional mesh decoding method in the present application can be applied to image processing scenarios, such as games, digital humans or clothing design, etc. The encoding device of the present application can be an encoding device that provides cloud services or a local encoding device, and the decoding device can be a decoding device that provides cloud services or a local decoding device.

[0049] The following is an introduction to the process of encoding a three-dimensional grid into a feature sequence in conjunction with the encoding device of the present application. Figure 1 In one embodiment, the encoding device 100 includes a segmentation module 101 and a serialization module 102. After the user inputs a three-dimensional grid into the encoding device 100, the segmentation module 101 divides the three-dimensional grid into multiple grid sub-graphs, and the serialization module 102 generates a feature sub-sequence corresponding to each grid sub-graph, and then generates a feature sequence including all feature sub-sequences.

[0050] See also Figure 2 In one embodiment, the decoding device 200 includes a receiving module 201 and a decoding module 202, wherein the receiving module 201 is used to receive a feature sequence, and the decoding module 202 is used to decode the feature sequence into a three-dimensional grid. The three-dimensional grid in this application can be, but is not limited to, a character model, an object model, a scene model, an online digital human model, or a clothing model in a game, and can be specifically set according to actual conditions, and this application does not limit it.

[0051] See also Figure 3 In one embodiment, the encoding device 100 includes a segmentation module 101, a serialization module 102 and an encoding module 103. After the user inputs a three-dimensional grid into the encoding device 100, the segmentation module 101 divides the three-dimensional grid into multiple grid sub-graphs, the serialization module 102 generates a feature sub-sequence corresponding to each grid sub-graph, and then generates a feature sequence including all feature sub-sequences. The encoding module 103 is specifically used to input the feature sequence into a variational quantization auto-encoder, and output a potential vector through the variational quantization auto-encoder.

[0052] The following is an introduction to a three-dimensional grid encoding method in conjunction with an embodiment. Figure 4 In one embodiment, after the user inputs the three-dimensional model of the flower, the segmentation module 101 divides the three-dimensional mesh of the flower into multiple mesh subgraphs, the serialization module 102 generates a feature subsequence corresponding to each mesh subgraph, such as feature subsequence 1 to feature subsequence P, and generates a feature sequence including all feature subsequences, and the encoding module 103 is specifically used to input the feature sequence into the variational quantization autoencoder, and output the potential vector through the variational quantization autoencoder. It should be understood that P is an integer greater than 1.

[0053] See also Figure 5In some embodiments, the receiving module 201 is further used to receive a latent vector, and the decoding module 202 is specifically used to input the latent vector received by the receiving module 201 into a decoder of the variational quantization autoencoder, and output a three-dimensional grid through the decoder.

[0054] The decoding method is described below in conjunction with an embodiment. Figure 6 In one embodiment, the receiving module 201 is further used to receive a latent vector from a user, and the decoding module 202 is specifically used to input the latent vector received by the receiving module 201 into a decoder of the variational quantization autoencoder, and output a three-dimensional mesh of the flower through the decoder.

[0055] See also Figure 7 In another embodiment, the receiving module 201 is further used to receive the intended text (e.g., blooming flowers) input by the user; when the intended text includes flowers, the decoding module 202 is further used to input the intended text into the autoregressive model, and output the latent vector corresponding to the intended text through the autoregressive model; the latent vector corresponding to the intended text is input into the decoder of the variational quantization autoencoder, and the three-dimensional mesh of the flower is output through the decoder.

[0056] The present application includes a data training device for training a variational quantized autoencoder, see Figure 8 In another embodiment, the present application further provides a data training device 800, which includes an encoding module 801 and a training module 802. The encoding module 801 is used to encode the three-dimensional grids of the three-dimensional grid set into feature sequences after obtaining the three-dimensional grid set to obtain the feature sequence set. The training module 802 is used to perform model training according to the feature sequence set to obtain a variational quantization autoencoder. The specific method of encoding the three-dimensional grids of the three-dimensional grid set into feature sequences by the encoding module 801 can be referred to. The method of encoding the three-dimensional grids into feature sequences by the encoding device 100 is similar, and the specific method can be referred to the corresponding description above.

[0057] In some possible implementations, the variational quantization autoencoder includes an encoder and a decoder, the encoder is used to convert a feature sequence into a latent vector, and the decoder is used to generate a target feature sequence similar to the feature sequence according to the latent vector from the encoder.

[0058] In some possible implementations, the encoder includes a convolutional neural network and a quantized neural network, the convolutional neural network is used to convolve the feature sequence, and the quantized neural network is used to generate a latent vector based on a vector obtained by convolution of the convolutional neural network.

[0059] The following is an introduction to the process of training the variational quantization autoencoder by the training module 802. The training of the variational quantization autoencoder includes the process of training the encoder and training the decoder. In the process of training the encoder, the feature sequence is input into the encoder, processed by one or more convolutional neural networks of the encoder, and then processed by one or more quantization neural networks to obtain a potential vector z. The quantization neural network can be, but is not limited to, a clustered quantization neural network, a finite scalar quantization neural network, or a query-free quantization neural network.

[0060] In the process of training the decoder, the latent vector z is input into the decoder, and the three-dimensional grid is output by the decoder. The loss between the three-dimensional grid output by the decoder and the original three-dimensional grid corresponding to the feature sequence is calculated according to the loss function, and the gradient is calculated by the back propagation algorithm. The model parameters of the variational quantization autoencoder are updated according to the gradient and the learning rate. Among them, the loss function can be but is not limited to the mean square error loss function or the cross entropy loss function. The model parameters θ, the learning rate η and the gradient ▽L satisfy the following formula: θ=θ-η▽L.

[0061] The above steps of training the encoder and training the decoder are iterated until the loss function converges or a preset number of training rounds is reached. In the variational quantization autoencoder, the number of convolutional neural network layers of the encoder, the number of quantization neural network layers, the number of convolution kernels in the convolutional neural network, the size of the quantization neural network, and the learning rate can be adjusted according to actual conditions, and this application does not limit this.

[0062] The present application also includes a data training device for training an autoregressive model, see Fig. 9 In another embodiment, the data training device 900 includes an acquisition module 901 and a training module 902, wherein the acquisition module 901 is used to acquire a training set, and the training module 902 is used to perform model training according to the training set to obtain an autoregressive model. The training set includes multiple samples, and the samples include keywords corresponding to the three-dimensional grid and potential vectors corresponding to the three-dimensional grid. The keywords corresponding to the three-dimensional grid are used as input data of the autoregressive model, and the potential vectors corresponding to the three-dimensional grid are used as labels of the autoregressive model. Optionally, the autoregressive model is a transformer model. The transformer model includes an encoder and a decoder, and the encoder includes multiple encoding layers, each encoding layer includes a multi-head attention layer and a feedforward neural network layer. The decoder includes multiple decoding layers, each decoding layer includes a masked multi-head attention layer, a multi-head attention layer, and a feedforward neural network layer. The encoder can convert the input text into a vector, and the decoder can generate the next output data based on the output data of the encoder and the historical output data.

[0063] The following is an introduction to the process of training the autoregressive model by the training module 902. In one embodiment, the keywords corresponding to the three-dimensional grid are converted into vectors X, and the potential vectors corresponding to the three-dimensional grid are used as observation vectors Y, which satisfy the following formula: Y=Xβ+ε, where β is the parameter of the autoregressive model and ε is the error vector. β is estimated using the least squares method. It should be understood that the method of training the autoregressive model is not limited to the above examples, and other methods can also be used for training, which is not limited in this application.

[0064] Since the data volume of all triangular meshes in the prior art is large and requires a lot of storage resources, this application provides another method for saving three-dimensional mesh data as a feature sequence, which can reduce the storage resources of three-dimensional mesh data. Fig.10 In one embodiment, the three-dimensional grid encoding method in the present application includes the following steps: S1001. The encoding device divides a three-dimensional grid into multiple grid sub-graphs.

[0065] Among them, the three-dimensional grid can be considered as a three-dimensional grid model, the three-dimensional grid includes multiple triangular grids, and multiple grid subgraphs include a first type of grid subgraph. Each grid subgraph in the first type of grid subgraph includes n target vertices and n adjacent points of the target vertices. The vertices in the first type of grid subgraph can be considered as the collection of n target vertices and n adjacent points of the target vertices. The target vertices of any two grid subgraphs in the first type of grid subgraph have no intersection, and n is an integer greater than 1. In a single grid subgraph, the vertices are unique, and any two grid subgraphs in multiple grid subgraphs have no common edges. There are fewer repeated vertices in multiple grid subgraphs, so the resulting grid subgraphs require less storage resources. Multiple grid subgraphs can also include a second type of grid subgraph, and the number of target vertices in the second type of grid subgraph is less than n.

[0066] S1002: The encoding device generates a feature subsequence corresponding to each grid subgraph, where the feature subsequence includes all vertex coordinates of the grid subgraph and an adjacency matrix of the grid subgraph.

[0067] Taking the first type of grid subgraph as an example, the grid subgraph includes n target vertices and n adjacent points of the target vertices, then the characteristic subsequence of the grid subgraph includes {coordinates of target vertex 1, coordinates of target vertex 2, ..., coordinates of target vertex n, coordinates of adjacent point 1, coordinates of adjacent point 2, ..., coordinates of the last adjacent point, adjacency matrix}. Optionally, n is 2, 3 or 4. In this way, the three-dimensional grid can be divided into grid subgraphs of different sizes. Optionally, the characteristic subsequence also includes a first separator and a second separator, the first separator is located between the coordinates of the last target vertex and the coordinates of the first adjacent point, the first separator is used to separate the last target vertex and the adjacent points of the target vertex, the second separator is located between the coordinates of the last adjacent point and the adjacency matrix, and the second separator is used to separate the adjacent points of the target vertex and the adjacency matrix. For example, {coordinates of target vertex 1, coordinates of target vertex 2, ..., coordinates of target vertex n, first separator, coordinates of adjacent point 1, coordinates of adjacent point 2, ..., coordinates of the last adjacent point, second separator, adjacency matrix}.

[0068] The adjacency matrix of the grid subgraph is used to indicate the connection relationship of all vertices of the grid subgraph. In some embodiments, the adjacency matrix of the grid subgraph includes T values, T is the total number of vertices of the grid subgraph, and the i-th value is used to indicate the connection relationship between the i-th vertex and all vertices in the grid subgraph. This provides an adjacency matrix that occupies less storage resources.

[0069] See also Fig.11 In one embodiment, the grid subgraph 1101 includes 6 vertices, namely vertex 1, vertex 2, vertex 5, vertex 6, vertex 7 and vertex 11. If two vertices are connected, the connection relationship between the two vertices is in the original adjacency matrix A. ij The value in is 1. If the two vertices are not connected, the connection relationship between the two vertices is in the original adjacency matrix A. ij The value in is 0, so the original adjacency matrix A ij for:

[0070] a ij is the value of the i-th row and j-th column, a ji is the value of the jth row and i-th column. The value of the i-th row can represent the connection relationship between the i-th vertex and all vertices in the grid subgraph, and the value of the i-th column can represent the connection relationship between the i-th vertex and all vertices in the grid subgraph. If the i-th vertex is connected to the j-th vertex, then a ij and a ji are all 1. If the i-th vertex is not connected to the j-th vertex, then a ij and a jiare all 0. In some embodiments, the value of each row in the original adjacency matrix is ​​recorded as an integer, thereby obtaining an adjacency matrix including T integers. For example, 011100 is recorded as 28. In other embodiments, the value of each column in the original adjacency matrix is ​​recorded as an integer, thereby obtaining an adjacency matrix including T integers.

[0071] In some other embodiments, the original adjacency matrix A ij is an upper triangular matrix, which is:

[0072] a ij is the value of the i-th row and j-th column, and the value of the i-th row can represent the connection relationship between the i-th vertex and all vertices in the grid subgraph. In some embodiments, the value of each row in the original adjacency matrix is ​​recorded as an integer, thereby obtaining an adjacency matrix including T integers.

[0073] In some other embodiments, the original adjacency matrix A ij is a lower triangular matrix, which is:

[0074] a ij is the value of the i-th row and j-th column, and the value of the i-th column can represent the connection relationship between the i-th vertex and all vertices in the grid subgraph. In some embodiments, the value of each column in the original adjacency matrix is ​​recorded as an integer, thereby obtaining an adjacency matrix including T integers.

[0075] In other embodiments, the adjacency matrix of the grid subgraph includes T*T values, each of which is 0 or 1. In other embodiments, the adjacency matrix of the grid subgraph is an upper triangular matrix, which includes T*(T+1) / 2 values, each of which is 0 or 1. In other embodiments, the adjacency matrix of the grid subgraph is a lower triangular matrix, which includes T*(T+1) / 2 values, each of which is 0 or 1. It should be understood that the values ​​of T, i and j can be set according to actual conditions and are not limited in this application.

[0076] S1003. The encoding device generates a feature sequence including all feature subsequences.

[0077] In this embodiment, multiple grid subgraphs include a first type of grid subgraph, each grid subgraph in the first type of grid subgraph includes n target vertices and n adjacent points of the target vertices, and the target vertices of any two grid subgraphs in the first type of grid subgraph have no intersection. Therefore, there are fewer repeated vertices in the multiple grid subgraphs, and the total number of vertices of all grid subgraphs is less than the number of vertices of all triangular meshes. Therefore, the amount of vertex data of all grid subgraphs is smaller, and the characteristic subsequences including vertex coordinates require less storage resources, which can achieve effective compression of three-dimensional meshes.

[0078] This application can use a variety of methods to segment the 3D grid, which are described in detail below. Fig.12 In an optional embodiment, S1001 includes the following steps: Step A: The encoding device selects n target vertices from the three-dimensional mesh to be processed, where the n target vertices include a first target vertex and first n-1 vertices of adjacent points of the first target vertex.

[0079] Step B: The encoding device determines that the grid subgraph includes a first vertex set and a first adjacent edge set, the first vertex set is a collection of n target vertices and adjacent points of n target vertices, and the first adjacent edge set is a collection of adjacent edges of the n target vertices.

[0080] Step C: The encoding device updates the three-dimensional mesh to be processed to the remaining portion after removing the n target vertices and the first adjacent edge set from the three-dimensional mesh to be processed.

[0081] Step A to step C are executed repeatedly until the number of neighboring points of each vertex in the three-dimensional mesh to be processed is less than n.

[0082] In this embodiment, the initial data of the three-dimensional mesh to be processed is a three-dimensional mesh, and steps A to C are executed cyclically, so that all or most of the three-dimensional mesh can be divided into a plurality of first-type mesh subgraphs. Since the n target vertices of the previous mesh subgraph are removed when the mesh subgraph is divided, the target vertices of the first-type mesh subgraph are repeated less times, and compared with the existing triangular mesh including a large number of repeated vertices, the repeated vertices can be reduced, thereby saving storage resources.

[0083] Among them, n can be but not limited to 2, 3 or 4. In this way, first-class grid subgraphs of different sizes can be obtained, and feature sequences of different sizes can be generated according to first-class grid subgraphs of different sizes. The present application can select the feature sequence with the smallest data volume for storage, thereby saving the most storage resources. It should be noted that when n=3, the feature sequence generated based on the grid subgraph of this size has a good compression rate, and the error between the three-dimensional grid reconstructed based on the feature sequence and the original three-dimensional grid is small.

[0084] In some embodiments, the sequence number of each layer of vertices in the three-dimensional grid is an arithmetic progression, the difference between the minimum sequence number of the vertices in the i+1th layer and the maximum sequence number of the vertices in the ith layer is equal to the common difference of the arithmetic progression, L is the number of vertex layers of the three-dimensional grid, and i is a positive integer less than L. In other embodiments, the sequence number of each layer of vertices in the three-dimensional grid is an arithmetic progression, the difference between the maximum sequence number of the vertices in the i-1th layer and the minimum sequence number of the vertices in the ith layer is equal to the common difference of the arithmetic progression, L is the number of vertex layers of the three-dimensional grid, and i is a positive integer less than L. In some embodiments, the first target vertex is the vertex with the smallest sequence number in the grid subgraph. In some embodiments, the first target vertex is the vertex with the largest sequence number in the grid subgraph.

[0085] In practical applications, the grid subgraph of the three-dimensional grid may also include the second type of grid subgraph. The following is an introduction to the method of segmenting the second type of grid subgraph from the three-dimensional grid. Fig.13 In an optional embodiment, S1001 includes the following steps: Step A: The encoding device selects n target vertices from the three-dimensional mesh to be processed, where the n target vertices include a first target vertex and first n-1 vertices of adjacent points of the first target vertex.

[0086] Step B: The encoding device determines that the grid subgraph includes a first vertex set and a first adjacent edge set, the first vertex set is a collection of n target vertices and adjacent points of n target vertices, and the first adjacent edge set is a collection of adjacent edges of the n target vertices.

[0087] Step C: The encoding device updates the three-dimensional mesh to be processed to the remaining portion after removing the n target vertices and the first adjacent edge set from the three-dimensional mesh to be processed.

[0088] Step A to step C are executed repeatedly until the number of neighboring points of each vertex in the three-dimensional mesh to be processed is less than n.

[0089] Step D: The encoding device selects a first target vertex and an adjacent point set from the three-dimensional mesh to be processed, and the adjacent point set includes all adjacent points of the first target vertex.

[0090] Step E: The encoding device determines that the grid subgraph includes a second vertex set and a second adjacent edge set, the second vertex set is the first target vertex, the adjacent point set and the collection of adjacent points of all vertices in the adjacent point set, and the second adjacent edge set is the collection of adjacent edges of the first target vertex and adjacent edges of all vertices in the adjacent point set.

[0091] Step F: The encoding device updates the three-dimensional mesh to be processed to the remaining portion after removing the first target vertex, the adjacent point set and the second adjacent edge set from the three-dimensional mesh to be processed.

[0092] Steps D to F are executed repeatedly until the three-dimensional grid to be processed is empty.

[0093] In this embodiment, the initial data of the three-dimensional mesh to be processed is a three-dimensional mesh, and steps A to C are executed cyclically, so that all or most of the three-dimensional mesh can be divided into a plurality of first-type mesh subgraphs. Since the n target vertices of the previous mesh subgraph are removed when the mesh subgraph is divided, the target vertices of the first-type mesh subgraph are repeated less times, and compared with the existing triangular mesh including a large number of repeated vertices, the repeated vertices can be reduced, thereby saving storage resources.

[0094] After looping through step A to step C, if there are still adjacent edges left, then looping through step D to step F can be performed to segment the remaining three-dimensional grid into a plurality of second-type grid subgraphs, where the number of target fixed points of the second-type grid subgraphs is less than n. When segmenting the second-type grid subgraphs from the three-dimensional grid to be processed, the vertices of the second-type grid subgraphs are also removed, so the second-type grid subgraphs have fewer duplicate vertices and require fewer storage resources.

[0095] It should be noted that since the neighboring edges of the n target vertices will be removed when selecting a grid subgraph, the first type of grid subgraph has no common neighboring edges. Similarly, the second type of grid subgraph has no common neighboring edges. When all neighboring edges are added to the grid subgraph, the three-dimensional grid to be processed is empty.

[0096] For ease of understanding, the following is an introduction to the method for segmenting a three-dimensional grid in the present application using a specific example. Fig.14 In one embodiment, the unfolded three-dimensional grid 1400 is as follows Fig.14 As shown, the vertices are numbered from left to right and from bottom to top. The first layer includes vertices 1 to 4, the second layer includes vertices 5 to 10, the third layer includes vertices 11 to 15, and the fourth layer includes vertices 16 to 17.

[0097] See also Fig.15, the first grid subgraph is recorded as grid subgraph 1401, first select vertex 1, the adjacent points of vertex 1 include vertex 2, vertex 5 and vertex 6, where the vertex with the smallest sequence number is vertex 2, the vertex with the second smallest sequence number is vertex 5, the adjacent points of vertex 2 include vertex 1, vertex 6 and vertex 7, and the adjacent points of vertex 5 include vertex 1, vertex 6 and vertex 11, then the set of vertex 1, vertex 2, vertex 5, the adjacent points of vertex 1, the adjacent points of vertex 2 and the adjacent points of vertex 5 is {1,2,5,6,7,11}. In this application, the adjacent edge from vertex i to vertex j is recorded as edge ij. Since the three-dimensional grid is an undirected graph, edge ij and edge ji can be considered to be the same edge. The adjacent edges of vertex 1 include edge 12, edge 15, edge 16, the adjacent edges of vertex 2 include edge 21, edge 26, edge 27, and the adjacent edges of vertex 5 include edge 51, edge 56, and edge 511. The set of the adjacent edges of vertex 1, vertex 2, and vertex 5 is {edge 12, edge 15, edge 16, edge 26, edge 27, edge 56, edge 511}, and the resulting grid subgraph 1401 includes {1,2,5,6,7,11} and {edge 12, edge 15, edge 16, edge 26, edge 27, edge 56, edge 511}. Remove {1,2,5} and {edge 12, edge 15, edge 16, edge 26, edge 27, edge 56, edge 511} from the three-dimensional grid, and use the remaining part as the three-dimensional grid to be processed for selecting the second grid subgraph.

[0098] The second grid subgraph is denoted as grid subgraph 1402. Vertex 3 is selected from the remaining part. The adjacent points of vertex 3 include vertex 4, vertex 8 and vertex 9, among which the vertex with the smallest sequence number is vertex 4, and the vertex with the second smallest sequence number is vertex 8. The adjacent points of vertex 4 include vertex 3, vertex 9 and vertex 10, and the adjacent points of vertex 8 include vertex 7, vertex 9, vertex 13 and vertex 14. Then the set of vertex 3, vertex 4, vertex 8, the adjacent points of vertex 3, the adjacent points of vertex 4 and the adjacent points of vertex 8 is {3,4,8,7,9,10,13,14}. The neighboring edges of vertex 3 include edge 34, edge 38, and edge 39; the neighboring edges of vertex 4 include edge 43, edge 49, and edge 410; the neighboring edges of vertex 8 include edge 83, edge 87, edge 89, edge 813, and edge 814. The collection of the neighboring edges of vertex 3, vertex 4, and vertex 8 is {edge 34, edge 38, edge 39, edge 49, edge 410, edge 87, edge 89, edge 813, and edge 814}. The resulting grid subgraph 1402 includes {3, 4, 8, 7, 9, 10, 13, 14} and {edge 34, edge 38, edge 39, edge 49, edge 410, edge 87, edge 89, edge 813, and edge 814}. Remove {3, 4, 8} and {edge 34, edge 38, edge 39, edge 49, edge 410, edge 87, edge 89, edge 813, edge 814} from the three-dimensional grid, and use the remaining part as the three-dimensional grid to be processed for selecting the third grid sub-graph.

[0099] The third grid subgraph is denoted as grid subgraph 1403. Vertex 6 is selected from the remaining part. The adjacent points of vertex 6 include vertex 7, vertex 11 and vertex 12, where the vertex with the smallest sequence number is vertex 7, and the vertex with the second smallest sequence number is vertex 11. The adjacent points of vertex 7 include vertex 6, vertex 12 and vertex 13, and the adjacent points of vertex 11 include vertex 6 and vertex 12. Then the set of vertex 6, vertex 7, vertex 11, the adjacent points of vertex 6, the adjacent points of vertex 7 and the adjacent points of vertex 11 is {6, 7, 11, 12, 13}. The neighboring edges of vertex 6 include edge 67, edge 611, and edge 612; the neighboring edges of vertex 7 include edge 76, edge 712, and edge 713; the neighboring edges of vertex 11 include edge 116 and edge 1112. The collection of the neighboring edges of vertex 6, vertex 7, and vertex 11 is {edge 67, edge 611, edge 612, edge 712, edge 713, edge 1112}. The resulting grid subgraph 1403 includes {6, 7, 11, 12, 13} and {edge 67, edge 611, edge 612, edge 712, edge 713, edge 1112}. Remove {6, 7, 11} and {edge 67, edge 611, edge 612, edge 712, edge 713, edge 1112} from the three-dimensional grid, and use the remaining part as the three-dimensional grid to be processed for selecting the fourth grid sub-graph.

[0100] The fourth grid subgraph is denoted as grid subgraph 1404. Vertex 9 is selected from the remaining part. The adjacent points of vertex 9 include vertex 10, vertex 14 and vertex 15, where the vertex with the smallest sequence number is vertex 10 and the vertex with the second smallest sequence number is vertex 14. The adjacent points of vertex 9 include vertex 10, vertex 14 and vertex 15. The adjacent points of vertex 14 include vertex 9, vertex 13, vertex 15 and vertex 17. Then the set of vertex 9, vertex 10, vertex 14, the adjacent points of vertex 9, the adjacent points of vertex 10 and the adjacent points of vertex 14 is {9, 10, 14, 13, 15, 17}. The neighboring edges of vertex 9 include edge 910, edge 914, and edge 915; the neighboring edges of vertex 10 include edge 109 and edge 1015; the neighboring edges of vertex 14 include edge 149, edge 1413, edge 1415, and edge 1417. The collection of the neighboring edges of vertex 9, vertex 10, and vertex 14 is {edge 910, edge 914, edge 915, edge 1015, edge 1413, edge 1415, edge 1417}. The resulting grid subgraph 1404 includes {9, 10, 14, 13, 15, 17} and {edge 910, edge 914, edge 915, edge 1015, edge 1413, edge 1415, edge 1417}. Remove {9, 10, 14} and {edge 910, edge 914, edge 915, edge 1015, edge 1413, edge 1415, edge 1417} from the three-dimensional grid, and use the remaining part as the three-dimensional grid to be processed for selecting the fifth grid sub-graph.

[0101] The fifth grid subgraph is denoted as grid subgraph 1405. Vertex 12 is selected from the remaining part. The adjacent points of vertex 12 include vertex 13, vertex 16 and vertex 17, where the vertex with the smallest sequence number is vertex 13, and the vertex with the second smallest sequence number is vertex 16. The adjacent points of vertex 13 include vertex 12 and vertex 17, and the adjacent points of vertex 16 include vertex 12 and vertex 17. Then the set of vertex 12, vertex 13, vertex 16, the adjacent points of vertex 12, the adjacent points of vertex 13 and the adjacent points of vertex 16 is {12, 13, 16, 17}. The neighboring edges of vertex 12 include edge 1213 and edge 1216, the neighboring edges of vertex 13 include edge 1312, edge 1316, and edge 1317, and the neighboring edges of vertex 16 include edge 1612, edge 1613, and edge 1617. Then the collection of the neighboring edges of vertex 12, vertex 13, and vertex 17 is {edge 1213, edge 1216, edge 1316, edge 1317, edge 1617}, and the resulting grid subgraph 1405 includes {12, 13, 16, 17} and {edge 1213, edge 1216, edge 1316, edge 1317, edge 1617}. Remove {12, 13, 16} and {edge 1213, edge 1216, edge 1316, edge 1317, edge 1617} from the three-dimensional mesh. Since the remaining vertices include vertex 15 and vertex 17, the number of adjacent points of vertex 15 and the number of adjacent points of vertex 17 are both 0, so the segmentation of the three-dimensional mesh can be stopped.

[0102] The feature subsequence corresponding to the grid subgraph 1401 is {coordinates of vertex 1 coordinates of vertex 2 coordinates of vertex 5 coordinates | coordinates of vertex 6 coordinates of vertex 7 coordinates of vertex 11 coordinates | 28 38 37 56 16 8}, the coordinates of each vertex include X-axis coordinates, Y-axis coordinates, and Z-axis coordinates, and the length of the adjacency matrix is ​​equal to the number of vertices in the grid subgraph, so the length of the feature subsequence of the grid subgraph 1401 is 6*3+6. It can be inferred that the length of the feature subsequence of the grid subgraph is equal to the number of vertices*4.

[0103] The number of vertices of mesh subgraphs 1401 to 1405 is 6, 8, 5, 6, and 4, respectively. It can be seen that the length of the characteristic sequence of the three-dimensional mesh 1400 = 6*4+8*4+5*4+6*4+4*4=116. When the three-dimensional mesh is stored in a triangular mesh, the three-dimensional mesh 1400 includes 18 triangles, and 18 triangle vertex coordinates are required, so the sequence length = 9*18=162. It can be seen that the sequence length of the present application is smaller, so storage resources can be saved and the compression rate can be improved.

[0104] In an optional embodiment, the encoding method of the three-dimensional grid in the present application further includes: the encoding device inputs the feature sequence into a variational quantization autoencoder, and outputs a potential vector through the variational quantization autoencoder. The variational quantization autoencoder can further compress the feature sequence, thereby improving the compression rate of the compressed three-dimensional grid. In the present application, the compression rate refers to the percentage of the difference between the original data size and the compressed data size and the original data size.

[0105] The decoding method in this application is introduced below. Fig.16 In one embodiment, the three-dimensional grid decoding method in the present application includes the following steps: S1601. The decoding device receives a feature sequence.

[0106] The feature sequence includes multiple feature subsequences, and the feature subsequences are related to the grid subsequences of the three-dimensional grid. Figure 1 One-to-one correspondence, the feature subsequence includes all vertex coordinates of the grid subgraph and the adjacency matrix of the grid subgraph. The grid subgraph of the three-dimensional grid includes a first type of grid subgraph, each grid subgraph in the first type of grid subgraph includes n target vertices and n adjacent points of the target vertices, and the target vertices of any two grid subgraphs in the first type of grid subgraph have no intersection. In some possible embodiments, the grid subgraph of the three-dimensional grid also includes a second type of grid subgraph, and the number of target vertices in the second type of grid subgraph is less than n. n is an integer greater than 1, such as 2, 3, 4, and the specific value is not limited.

[0107] S1602: The decoding device decodes the feature sequence into a three-dimensional grid.

[0108] The decoding device reconstructs the network subgraph according to the characteristic subsequence and composes the network subgraph into a three-dimensional grid.

[0109] In this embodiment, for a feature sequence with a high compression rate, the present application can restore the feature sequence into a three-dimensional grid, thereby providing an effective decoding method.

[0110] In some possible embodiments, the above decoding method further includes: a decoding device receives a latent vector, inputs the latent vector into a decoder of a variational quantization autoencoder, and outputs a three-dimensional grid through the decoder.

[0111] In this embodiment, for the latent vector obtained by processing the feature sequence using the variational quantization autoencoder, the decoding device can input it into the decoder of the variational quantization autoencoder, and output its corresponding three-dimensional grid through the decoder. It should be understood that the latent vector is also called a hidden vector, which is shorter than the length of the feature sequence.

[0112] In some possible embodiments, the above-mentioned decoding method also includes: the decoding device receives the intended text input by the user; when the intended text includes keywords corresponding to the three-dimensional grid, the decoding device inputs the intended text into the autoregressive model, outputs the latent vector corresponding to the intended text through the autoregressive model, and then inputs the latent vector corresponding to the intended text into the decoder of the variational quantization autoencoder, and outputs the three-dimensional grid through the decoder.

[0113] In this embodiment, the user can input the intended text, and the decoding device outputs the three-dimensional grid according to the intended text, and can efficiently draw the corresponding three-dimensional grid according to the user's needs.

[0114] The following is an introduction to the hardware devices in this application. Figure 3 , an encoding device 100 in the present application includes a segmentation module 101, a serialization module 102 and an encoding module 103, the segmentation module 101 is used to divide a three-dimensional grid into multiple grid subgraphs, the three-dimensional grid includes multiple triangle grids, the multiple grid subgraphs include a first type of grid subgraph, each grid subgraph in the first type of grid subgraph includes n target vertices and n adjacent points of the target vertices, and the target vertices of any two grid subgraphs in the first type of grid subgraph have no intersection; The serialization module 102 is used to generate a feature subsequence corresponding to each grid subgraph, the feature subsequence including all vertex coordinates of the grid subgraph and the adjacency matrix of the grid subgraph; and generate a feature sequence including all feature subsequences.

[0115] As an example of a software functional unit, the segmentation module 101 may include code running on a computing instance. The computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Furthermore, the computing instance may be one or more. For example, the segmentation module 101 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code may be distributed in the same region or in different regions. Furthermore, the multiple hosts / virtual machines / containers used to run the code may be distributed in the same availability zone (AZ) or in different AZs, each AZ including one data center or multiple data centers with similar geographical locations. Generally, a region may include multiple AZs.

[0116] Similarly, multiple hosts / virtual machines / containers used to run the code can be distributed in the same virtual private cloud (VPC) or in multiple VPCs. Usually, a VPC is set up in a region. For cross-region communication between two VPCs in the same region and between VPCs in different regions, a communication gateway needs to be set up in each VPC to achieve interconnection between VPCs through the communication gateway.

[0117] As an example of a hardware functional unit, the segmentation module 101 may include at least one computing device, such as a server, etc. Alternatively, the segmentation module 101 may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0118] The multiple computing devices included in the segmentation module 101 can be distributed in the same region or in different regions. The multiple computing devices included in the segmentation module 101 can be distributed in the same AZ or in different AZs. Similarly, the multiple computing devices included in the segmentation module 101 can be distributed in the same VPC or in multiple VPCs. The multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0119] In other embodiments, the segmentation module 101 may be used to perform Fig.10 , Fig.12 or Fig.13 The serialization module 102 may be used to perform any step performed by the encoding device in the encoding method of the three-dimensional mesh shown in FIG. Fig.10 , Fig.12 or Fig.13 The encoding module 103 may be used to perform any step performed by the encoding device in the three-dimensional mesh encoding method shown in FIG. Fig.10 , Fig.12 or Fig.13In the coding method of the three-dimensional mesh shown in the figure, any step performed by the coding device, the steps implemented by the segmentation module 101, the serialization module 102 and the coding module 103 can be specified as needed, and the segmentation module 101, the serialization module 102 and the coding module 103 can be respectively implemented Fig.10 , Fig.12 or Fig.13 The different steps in the three-dimensional mesh encoding method shown are used to implement the full functionality of the encoding device. The encoding module 103 is an optional module.

[0120] In some embodiments, the segmentation module 101 is specifically configured to execute step A to step C in a loop until the number of neighboring points of each vertex in the three-dimensional mesh to be processed is less than n.

[0121] In some embodiments, the segmentation module 101 is specifically used to loop through steps A to C until the number of neighboring points of each vertex in the three-dimensional mesh to be processed is less than n; and then loop through steps D to F until the three-dimensional mesh to be processed is empty.

[0122] In some embodiments, the encoding device 100 further includes an encoding module 103, and the encoding module 103 is used to input the feature sequence into a variational quantization autoencoder, and output a potential vector through the variational quantization autoencoder.

[0123] See also Figure 5 In the present application, a decoding device 200 includes: a receiving module 201 for receiving a feature sequence; a decoding module 202 for decoding the feature sequence into a three-dimensional grid.

[0124] As an example of a software functional unit, the decoding module 202 may include code running on a computing instance. The computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Furthermore, the computing instance may be one or more. For example, the decoding module 202 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code may be distributed in the same region or in different regions. Furthermore, the multiple hosts / virtual machines / containers used to run the code may be distributed in the same availability zone (AZ) or in different AZs, each AZ including one data center or multiple data centers with similar geographical locations. Generally, a region may include multiple AZs.

[0125] Similarly, multiple hosts / virtual machines / containers used to run the code can be distributed in the same virtual private cloud (VPC) or in multiple VPCs. Usually, a VPC is set up in a region. For cross-region communication between two VPCs in the same region and between VPCs in different regions, a communication gateway needs to be set up in each VPC to achieve interconnection between VPCs through the communication gateway.

[0126] As an example of a hardware functional unit, the decoding module 202 may include at least one computing device, such as a server, etc. Alternatively, the decoding module 202 may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0127] The multiple computing devices included in the decoding module 202 can be distributed in the same region or in different regions. The multiple computing devices included in the decoding module 202 can be distributed in the same AZ or in different AZs. Similarly, the multiple computing devices included in the decoding module 202 can be distributed in the same VPC or in multiple VPCs. The multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0128] In other embodiments, the decoding module 202 may be used to perform Fig.16 The receiving module 201 may be used to perform any step performed by the decoding device in the decoding method of the three-dimensional grid shown in FIG. Fig.16 In the decoding method of the three-dimensional mesh shown in FIG. 1 , any step performed by the decoding device, the steps implemented by the receiving module 201 and the decoding module 202 can be specified as needed, and the receiving module 201 and the decoding module 202 can respectively implement Fig.16 The different steps in the three-dimensional grid decoding method shown implement the entire function of the decoding device.

[0129] In some embodiments, the receiving module 201 is further used to receive a latent vector, which is obtained by processing a feature sequence using a variational quantization autoencoder; the decoding module 202 is further used to input the latent vector into a decoder of the variational quantization autoencoder, and output a three-dimensional grid through the decoder.

[0130] In some embodiments, the receiving module 201 is also used to receive the intended text input by the user; when the intended text includes keywords corresponding to the three-dimensional grid, the decoding module 202 is also used to input the intended text into the autoregressive model, and output the latent vector corresponding to the intended text through the autoregressive model; the latent vector corresponding to the intended text is input into the decoder of the variational quantization autoencoder, and the three-dimensional grid is output through the decoder.

[0131] The present application also provides a computing device 1700. Fig.17 As shown, in one embodiment, the computing device 1700 includes: a bus 1702, a processor 1704, a memory 1706, and a communication interface 1708. The processor 1704, the memory 1706, and the communication interface 1708 communicate with each other via the bus 1702. It should be understood that the present application does not limit the number of processors and memories in the computing device 1700.

[0132] The bus 1702 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.17 The bus 1702 may include a path for transmitting information between various components of the computing device 1700 (eg, the memory 1706, the processor 1704, and the communication interface 1708).

[0133] The processor 1704 may include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP). The processor includes multiple processing cores.

[0134] The memory 1706 may include a volatile memory, such as a random access memory (RAM). The memory 1706 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD) or a solid state drive (SSD). In some embodiments, the memory 1706 stores an executable program code, and the processor 1704 executes the executable program code to implement the functions of the aforementioned segmentation module 101, the serialization module 102 and the encoding module 103, thereby implementing the above-mentioned three-dimensional mesh encoding method. In other embodiments, the memory 1706 stores an executable program code, and the processor 1704 executes the executable program code to implement the functions of the aforementioned receiving module 201 and the decoding module 202, thereby implementing the above-mentioned three-dimensional mesh decoding method.

[0135] The communication interface 1708 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 1700 and other devices or communication networks.

[0136] The embodiment of the present application also provides a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smart phone.

[0137] like Fig.18 As shown, the computing device cluster includes at least one computing device 1700. The memory 1706 in one or more computing devices 1700 in the computing device cluster may store the same instructions for executing the three-dimensional mesh encoding method.

[0138] In some possible implementations, the memory 1706 of one or more computing devices 1700 in the computing device cluster may also store partial instructions for executing the three-dimensional mesh encoding method. In other words, the combination of one or more computing devices 1700 may jointly execute the instructions for executing the three-dimensional mesh encoding method.

[0139] It should be noted that the memory 1706 in different computing devices 1700 in the computing device cluster can store different instructions, which are respectively used to execute part of the functions of the encoding device. That is, the instructions stored in the memory 1706 in different computing devices 1700 can implement the functions of the segmentation module 101, the serialization module 102 and the encoding module 103. Fig.19 As shown, in the computing device 1700A, the memory 1706 stores executable program code, and the processor 1704 executes the executable program code to implement the functions of the aforementioned segmentation module 101 and serialization module 102. In the computing device 1700B, the memory 1706 stores executable program code, and the processor 1704 executes the executable program code to implement the functions of the aforementioned encoding module 103.

[0140] The memory 1706 in one or more computing devices 1700 in the computing device cluster may also store the same instructions for executing the three-dimensional mesh decoding method.

[0141] In some possible implementations, the memory 1706 of one or more computing devices 1700 in the computing device cluster may also store partial instructions for executing the decoding method of the three-dimensional mesh. In other words, the combination of one or more computing devices 1700 may jointly execute the instructions for executing the decoding method of the three-dimensional mesh.

[0142] It should be noted that the memory 1706 in different computing devices 1700 in the computing device cluster can store different instructions, which are respectively used to execute part of the functions of the decoding device. That is, the instructions stored in the memory 1706 in different computing devices 1700 can implement the functions of the receiving module 201 and the decoding module 202.

[0143] The embodiment of the present application also provides a computer program product including instructions. The computer program product may be software or a program product including instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on at least one computing device, the at least one computing device executes the three-dimensional mesh encoding method or the three-dimensional mesh decoding method of the present application.

[0144] The embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that can be stored by a computing device or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk). The computer-readable storage medium includes instructions that instruct the computing device to execute the three-dimensional mesh encoding method or the three-dimensional mesh decoding method of the present application.

[0145] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0146] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the protection scope of the technical solutions of the embodiments of the present invention.

Claims

1. A three-dimensional mesh encoding method, characterized in that: The method is applied to an encoding device, and the method comprises: The encoding device divides a three-dimensional grid into a plurality of grid subgraphs, the three-dimensional grid includes a plurality of triangular grids, the plurality of grid subgraphs include a first type of grid subgraph, each grid subgraph in the first type of grid subgraph includes n target vertices and adjacent points of the n target vertices, the target vertices of any two grid subgraphs in the first type of grid subgraph have no intersection, and n is an integer greater than 1; The encoding device generates a feature subsequence corresponding to each of the grid subgraphs, wherein the feature subsequence includes all vertex coordinates of the grid subgraph and an adjacency matrix of the grid subgraph; The encoding device generates a feature sequence including all of the feature subsequences.

2. The method according to claim 1, characterized in that The encoding device divides the three-dimensional grid into a plurality of grid sub-graphs, including: Step A: the encoding device selects n target vertices from a three-dimensional mesh to be processed, the n target vertices include a first target vertex and first n-1 vertices of adjacent points of the first target vertex, and the initial data of the three-dimensional mesh to be processed is the three-dimensional mesh; Step B: the encoding device determines that the grid subgraph includes a first vertex set and a first adjacent edge set, the first vertex set includes the collection of adjacent points of the n target vertices and the n target vertices, and the first adjacent edge set is the collection of adjacent edges of the n target vertices; Step C: the encoding device updates the to-be-processed three-dimensional mesh to the remaining portion after removing the n target vertices and the first adjacent edge set from the to-be-processed three-dimensional mesh; Steps A to C are executed in a loop until the number of neighboring points of each vertex in the three-dimensional mesh to be processed is less than n.

3. The method according to claim 1, characterized in that The encoding device divides the three-dimensional grid into a plurality of grid sub-graphs, including: Step A: the encoding device selects n target vertices from a three-dimensional mesh to be processed, the n target vertices include a first target vertex and first n-1 vertices of adjacent points of the first target vertex, and the initial data of the three-dimensional mesh to be processed is the three-dimensional mesh; Step B: The encoding device determines that the grid subgraph includes a first vertex set and a first adjacent edge set, wherein the first vertex set includes a collection of n target vertices and adjacent points of the n target vertices, and the first adjacent edge set is a collection of adjacent edges of the n target vertices; Step C: the encoding device updates the to-be-processed three-dimensional mesh to the remaining portion after removing the n target vertices and the first adjacent edge set from the to-be-processed three-dimensional mesh; Execute step A to step C repeatedly until the number of neighboring points of each vertex in the three-dimensional mesh to be processed is less than n; Step D: the encoding device selects a first target vertex and an adjacent point set from the three-dimensional mesh to be processed, wherein the adjacent point set includes all adjacent points of the first target vertex; Step E: the encoding device determines that the grid subgraph includes a second vertex set and a second adjacent edge set, the second vertex set is the first target vertex, the adjacent point set and the collection of adjacent points of all vertices in the adjacent point set, and the second adjacent edge set is the collection of adjacent edges of the first target vertex and adjacent edges of all vertices in the adjacent point set; Step F: the encoding device updates the to-be-processed three-dimensional mesh to the remaining portion after removing the first target vertex, the adjacent point set, and the second adjacent edge set from the to-be-processed three-dimensional mesh; Steps D to F are executed in a loop until the three-dimensional grid to be processed is empty.

4. The method according to any one of claims 2 to 3, characterized in that The first target vertex is the vertex with the smallest sequence number in the grid subgraph, or the first target vertex is the vertex with the largest sequence number in the grid subgraph.

5. The method according to any one of claims 1 to 3, characterized in that The n is 2, 3 or 4.

6. The method according to any one of claims 1 to 3, characterized in that The characteristic subsequence also includes a first separator and a second separator, wherein the first separator is located between the coordinates of the last target vertex and the coordinates of the first adjacent point, and the second separator is located between the coordinates of the last adjacent point and the adjacency matrix.

7. The method according to any one of claims 1 to 3, characterized in that The method further comprises: The encoding device inputs the feature sequence into a variational quantization autoencoder, and outputs a potential vector through the variational quantization autoencoder.

8. A three-dimensional grid decoding method, characterized in that: The method is applied to a decoding device, and the method comprises: The decoding device receives a feature sequence, the feature sequence includes a plurality of feature subsequences, the feature subsequences correspond to the grid subgraphs of the three-dimensional grid one by one, the grid subgraphs of the three-dimensional grid include a first type of grid subgraphs, each grid subgraph in the first type of grid subgraphs includes n target vertices and adjacent points of the n target vertices, the target vertices of any two grid subgraphs in the first type of grid subgraphs have no intersection, and n is an integer greater than 1; The decoding device decodes the feature sequence into a three-dimensional grid.

9. The method according to claim 8, characterized in that The grid subgraph of the three-dimensional grid also includes a second type of grid subgraph, and the number of target vertices in the second type of grid subgraph is less than n.

10. The method according to claim 8 or 9, characterized in that: The method further comprises: The decoding device receives a latent vector, wherein the latent vector is obtained by processing the feature sequence using a variational quantization autoencoder; The decoding device inputs the latent vector into a decoder of a variational quantization autoencoder, and outputs the three-dimensional grid through the decoder.

11. The method according to any one of claims 8 to 9, characterized in that The method further comprises: The decoding device receives the intended text input by the user; When the intended text includes the keyword corresponding to the three-dimensional grid, the decoding device inputs the intended text into an autoregressive model, and outputs a latent vector corresponding to the intended text through the autoregressive model; The decoding device inputs the potential vector corresponding to the intended text into a decoder of a variational quantization autoencoder, and outputs the three-dimensional grid through the decoder.

12. A coding device, characterized in that: include: a segmentation module, configured to divide a three-dimensional grid into a plurality of grid subgraphs, wherein the three-dimensional grid includes a plurality of triangular grids, and the plurality of grid subgraphs include a first type of grid subgraphs, wherein each grid subgraph in the first type of grid subgraphs includes n target vertices and adjacent points of the n target vertices, and the target vertices of any two grid subgraphs in the first type of grid subgraphs have no intersection, and n is an integer greater than 1; A serialization module is used to generate a feature subsequence corresponding to each of the grid subgraphs, wherein the feature subsequence includes all vertex coordinates of the grid subgraph and an adjacency matrix of the grid subgraph; and to generate a feature sequence including all of the feature subsequences.

13. The device according to claim 12, characterized in that The serialization module is specifically used to perform the following steps: Step A: selecting n target vertices from a three-dimensional mesh to be processed, wherein the n target vertices include a first target vertex and first n-1 vertices of adjacent points of the first target vertex, and the initial data of the three-dimensional mesh to be processed is the three-dimensional mesh; Step B: Determine that the grid subgraph includes a first vertex set and a first adjacent edge set, wherein the first vertex set includes a collection of adjacent points of the n target vertices and the n target vertices, and the first adjacent edge set is a collection of adjacent edges of the n target vertices; Step C: updating the to-be-processed three-dimensional mesh to the remaining portion after removing the n target vertices and the first adjacent edge set from the to-be-processed three-dimensional mesh; Steps A to C are executed in a loop until the number of neighboring points of each vertex in the three-dimensional mesh to be processed is less than n.

14. The device according to claim 12, characterized in that The serialization module is specifically used to perform the following steps: Step A: selecting n target vertices from a three-dimensional mesh to be processed, wherein the n target vertices include a first target vertex and first n-1 vertices of adjacent points of the first target vertex, and the initial data of the three-dimensional mesh to be processed is the three-dimensional mesh; Step B: Determine that the grid subgraph includes a first vertex set and a first adjacent edge set, wherein the first vertex set includes a collection of n target vertices and adjacent points of the n target vertices, and the first adjacent edge set is a collection of adjacent edges of the n target vertices; Step C: updating the to-be-processed three-dimensional mesh to the remaining portion after removing the n target vertices and the first adjacent edge set from the to-be-processed three-dimensional mesh; Execute step A to step C repeatedly until the number of neighboring points of each vertex in the three-dimensional mesh to be processed is less than n; Step D: selecting a first target vertex and an adjacent point set from the three-dimensional mesh to be processed, wherein the adjacent point set includes all adjacent points of the first target vertex; Step E: Determine that the grid subgraph includes a second vertex set and a second adjacent edge set, wherein the second vertex set includes the first target vertex, the adjacent point set, and the collection of adjacent points of all vertices in the adjacent point set, and the second adjacent edge set is the collection of adjacent edges of the first target vertex and adjacent edges of all vertices in the adjacent point set; Step F: updating the to-be-processed three-dimensional mesh to the remaining portion after removing the first target vertex, the adjacent point set, and the second adjacent edge set from the to-be-processed three-dimensional mesh; Steps D to F are executed in a loop until the three-dimensional grid to be processed is empty.

15. The device according to any one of claims 13 to 14, characterized in that The first target vertex is the vertex with the smallest sequence number in the grid subgraph, or the first target vertex is the vertex with the largest sequence number in the grid subgraph.

16. The device according to any one of claims 12 to 14, characterized in that The n is 2, 3 or 4.

17. The device according to any one of claims 12 to 14, characterized in that The characteristic subsequence also includes a first separator and a second separator, wherein the first separator is located between the coordinates of the last target vertex and the coordinates of the first adjacent point, and the second separator is located between the coordinates of the last adjacent point and the adjacency matrix.

18. The device according to any one of claims 12 to 14, characterized in that The device also includes: The encoding module is used to input the feature sequence into a variational quantization autoencoder, and output a potential vector through the variational quantization autoencoder.

19. A decoding device, characterized in that: include: A receiving module, configured to receive a feature sequence, wherein the feature sequence includes a plurality of feature subsequences, the feature subsequences correspond one-to-one to a grid subgraph of a three-dimensional grid, the grid subgraph of the three-dimensional grid includes a first type of grid subgraph, each grid subgraph in the first type of grid subgraph includes n target vertices and adjacent points of the n target vertices, and the target vertices of any two grid subgraphs in the first type of grid subgraph have no intersection, and n is an integer greater than 1; A decoding module is used to decode the feature sequence into a three-dimensional grid.

20. The device according to claim 19, characterized in that The grid subgraph of the three-dimensional grid also includes a second type of grid subgraph, and the number of target vertices in the second type of grid subgraph is less than n.

21. The device according to claim 19 or 20, characterized in that The receiving module is further used to receive a potential vector, where the potential vector is obtained by processing the feature sequence using a variational quantization autoencoder; The decoding module is further used to input the potential vector into a decoder of a variational quantization autoencoder, and output the three-dimensional grid through the decoder.

22. The device according to any one of claims 19 to 20, characterized in that The receiving module is also used to receive the intended text input by the user; When the intended text includes keywords corresponding to the three-dimensional grid, the decoding module is also used to input the intended text into an autoregressive model, and output a latent vector corresponding to the intended text through the autoregressive model; input the latent vector corresponding to the intended text into a decoder of a variational quantization autoencoder, and output the three-dimensional grid through the decoder.

23. A computing device cluster, characterized in that: It includes at least one computing device, each computing device includes a processor and a memory; the processor of the at least one computing device is used to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the method as described in any one of claims 1 to 7.

24. A computing device cluster, comprising a processor and a memory, characterized in that: It includes at least one computing device, each computing device includes a processor and a memory; the processor of the at least one computing device is used to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the method as described in any one of claims 8 to 11.

25. A computer-readable storage medium, characterized in that: The method comprises computer program instructions, and when the computer program instructions are executed by a computing device cluster, the computing device cluster performs the method according to any one of claims 1 to 11.

26. A computer program product comprising instructions, characterized in that When the instructions are executed by a computing device cluster, the computing device cluster is caused to perform the method according to any one of claims 1 to 11.

Images