A human body mesh decoupling representation method based on mesh autoencoder

Abstract

The present invention discloses a human body mesh decoupling representation method based on a mesh autoencoder, comprising: 1) constructing a training data set and a mesh autoencoder, the mesh autoencoder is composed of an encoder and a decoder, the encoder is composed of a shape encoder, a posture encoder and a rotation and translation encoder, taking a human body mesh as input, the shape encoder outputs a shape coding representation, the posture encoder outputs a posture coding representation, the rotation and translation encoder outputs a rotation matrix and a translation vector, the decoder takes a combined encoding of the shape coding representation and the posture coding representation as input, and outputs a human body mesh; 2) training the mesh autoencoder with the training data set, and inputting the human body mesh into the trained mesh autoencoder to obtain its decoupling representation. The present invention can obtain a shape and posture representation of a human body mesh that is rotation and translation invariant and independently decoupled from each other, so that more three-dimensional human body meshes of different shapes and postures can be edited and synthesized using existing human body mesh data.

Description

Technical Field

[0001] The invention relates to the technical field of computer graphics and computer animation, and in particular to a human body mesh decoupling representation method based on mesh autoencoder. Background Art

[0002] With the development of three-dimensional scanning and reconstruction technology, three-dimensional human body mesh data has become more and more common, and learning low-dimensional representation of human body mesh has been widely used in computer graphics and computer animation. Traditional control structure-based representation methods are only applicable to the mesh sequence representation of a single human body. Existing human body parameterized representation models such as SMPL have achieved good results in many applications, but it is not easy to use them and requires professional knowledge and manual intervention. In recent years, deep learning has achieved great success in processing two-dimensional images. Deep geometric learning for three-dimensional data has also received more and more attention, and many deep neural mesh autoencoders for three-dimensional mesh data have been proposed. Most of the existing methods based on mesh autoencoders obtain a single global encoding of the three-dimensional mesh. A few methods realize the encoding representation of the decoupled shape and posture of the three-dimensional mesh, but the obtained encoding representation does not have rotation invariance. When performing some tasks, such as interpolating two human body meshes with different orientations, such methods cannot obtain a reasonable human body mesh. The present invention proposes a human body mesh decoupling representation method based on a mesh autoencoder, which can obtain a shape and posture representation of the human body mesh that is rotationally and translationally invariant and independently decoupled from each other, so that more three-dimensional human body meshes with different shapes and postures can be edited and synthesized using existing data. Summary of the invention

[0003] The purpose of the present invention is to overcome the shortcomings and deficiencies of the prior art and to propose an effective, scientific and reasonable human body mesh decoupling representation method based on a mesh autoencoder.

[0004] To achieve the above object, the technical solution provided by the present invention is: a human body mesh decoupling representation method based on a mesh autoencoder, comprising the following steps:

[0005] 1) constructing a training data set and a grid autoencoder; wherein the grid autoencoder consists of an encoder and a decoder, wherein the encoder consists of three branches: a shape encoder, a posture encoder, and a rotation and translation encoder, wherein the three branches take a human body mesh as input, the shape encoder outputs a shape coding representation, the posture encoder outputs a posture coding representation, the rotation and translation encoder outputs a rotation matrix and a translation vector, and the decoder takes a combined coding of the shape coding representation and the posture coding representation as input, and outputs a human body mesh;

[0006] 2) Use the established training dataset to train the constructed mesh autoencoder, and input the human body mesh into the trained mesh autoencoder to obtain its decoupled representation.

[0007] Further, in step 1), a training data set is constructed using a public 3D human motion sequence database. The public 3D human motion sequence database refers to the AMASS 3D human motion sequence database, which integrates other 3D human motion sequence data sets using the SMPL model. There are currently 24 sub-datasets, and the recommended data usage scheme includes 9 sub-datasets for training, which contain 313 different human bodies. Constructing a training data set refers to performing interval sampling on the action sequences in the 9 sub-datasets for training in the AMASS 3D human motion sequence database to form a human mesh training data set containing different postures of different human bodies. The specific process is as follows: The process is as follows: first, start from the action frame at 10% of each action sequence, sample at an interval of 500 frames, and end at the action frame at 90% of the action sequence; when the number of frames sampled in an action sequence is less than 2, first reduce the sampling interval by a step of 50 frames, and re-sample the action sequence; when the sampling interval is reduced to 100 frames, if the number of frames sampled is still less than 2, then reduce the sampling interval by a step of 10 frames; when the sampling interval is reduced to 10 frames, if the number of frames sampled is still less than 2, then skip the action sequence; finally, 33451 human body mesh data of 313 different human bodies are obtained, and each human body mesh contains 6890 three-dimensional vertices.

[0008] Further, in step 1), when a human body mesh X is input to the three branches of the encoder, the shape encoder outputs the shape code representation β of X, the posture encoder outputs the posture code representation θ of X, and the rotation and translation encoder outputs the rotation matrix rot and translation vector tran of X. The combined code (β, θ) is input to the decoder, and the decoder outputs a human body mesh Finally, use rot and tran Perform rotation and translation to obtain the reconstructed human body mesh of X The details of the various branches of the encoder and decoder of the grid autoencoder are as follows:

[0009] Shape encoder: It takes a vector of size (6890×3) representing a human body mesh as input and outputs a shape encoding representation of size (7×3). It consists of 4 consecutive downsampling modules, each of which consists of 1 mesh convolution layer and 1 mesh pooling layer. That is, the shape encoder has a total of 8 layers, and the output feature sizes of each layer are (6890×16), (1925×16), (1925×32), (400×32), (400×64), (54×64), (54×3), (7×3).

[0010] Pose encoder: It takes a vector of human body mesh of size (6890×3) as input and outputs a pose encoding representation of size (7×9). It consists of 4 consecutive downsampling modules, each of which consists of 1 mesh convolution layer and 1 mesh pooling layer. That is, the pose encoder has a total of 8 layers, and the output feature sizes of each layer are (6890×16), (1925×16), (1925×32), (400×32), (400×64), (54×64), (54×9), (7×9).

[0011] Rotation-translation encoder: It takes a vector of size (6890×3) representing the human body mesh as input, and outputs a rotation matrix of size (3×3) and a translation vector of size (1×3). The rotation-translation encoder is composed of 4 consecutive downsampling modules, 1 fully connected module, and 1 rotation-translation mapping module connected in sequence; each downsampling module consists of 1 grid convolution layer and 1 grid pooling layer. The output feature size of each layer of the 8-layer network in the 4 downsampling modules is (6890×8), (1925×8), and (1925×8). ×8), (1925×16), (400×16), (400×32), (54×32), (54×64), (7×64); the fully connected module consists of 3 stacked fully connected layers, and the feature sizes of each network layer are (1×128), (1×128), (1×6) respectively; the rotation translation mapping module takes the first 3-dimensional vector output by the fully connected module and uses the Rodriguez rotation formula to calculate a (3×3) rotation matrix, and takes the last 3-dimensional vector output by the fully connected module as the translation vector;

[0012] Decoder: It takes the synthetic vector of shape encoding representation and posture encoding representation of size (7×12) as input, and outputs a vector of size (6890×3) representing the human body mesh. The decoder contains 4 consecutive upsampling modules; each upsampling module consists of 1 grid anti-pooling layer and 1 grid deconvolution layer, that is, the decoder has a total of 8 layers, and the output feature sizes of each layer are (54×12), (54×64), (400×64), (400×32), (1925×32), (1925×16), (6890×16), (6890×3).

[0013] Further, the step 2) comprises the following steps:

[0014] 2.1) Construct a training data tuple. First, select a human body mesh X1 from the training data set. Then select a human body mesh X2 with the same shape as X1 but different posture from the training data set. Then select a human body mesh X3 with different shape and posture from X1 from the training data set. Finally, perform random rotation and translation operations on X1 to obtain a human body mesh X4. These four human body meshes constitute a training data tuple (X1, X2, X3, X4).

[0015] 2.2) Design the training process: First, input the training data tuple (X1, X2, X3, X4) obtained in step 2.1) into the three branches of the encoder respectively to obtain their shape encoding representation (β1, β2, β3, β4), posture encoding representation (θ1, θ2, θ3, θ4), rotation matrix (rot1, rot2, rot3, rot4) and translation vector (tran1, tran2, tran3, tran4); then input the combined code (β2, θ1) into the decoder to obtain the human body mesh Use rot4 and tran4 to Perform rotation and translation to obtain the reconstructed human body mesh of X4 Input the combined code (β2,θ4) into the decoder to get the human body mesh Use rot1 and tran1 to Perform rotation and translation to obtain the reconstructed human body mesh of X1 Input the combined code (β3,θ2) into the decoder to get the human body mesh Then put The posture encoder input to the encoder gets Gesture encoding Then the combination code (β1, ) is input to the decoder to obtain the human body mesh Use rot2 and tran2 to Perform rotation and translation to obtain the reconstructed human body mesh of X2 Finally, the combined code (β4, ) is input to the decoder to obtain the human body mesh Use rot2 and tran2 to Perform rotation and translation to obtain the reconstructed human body mesh of X2

[0016] 2.3) Design loss function L:

[0017] L=λ cross L cross +λ self L self +λ rt L rt

[0018] The loss function L contains the exchange consistency loss L cross , self-consistency loss L self and the rotation-translation consistency loss L rt Three items of content;

[0019] L cross Used to constrain the independent decoupling between the shape encoder, attitude encoder and rotation and translation encoder:

[0020]

[0021] λ cross For L cross Weight;

[0022] L self Used to enhance the decoupling degree of shape encoder and attitude encoder:

[0023]

[0024] λ self For L self Weight;

[0025] L rt Used to enhance the decoupling degree of the rotary-translational encoder:

[0026]

[0027] λ rt For L rt Weight;

[0028] 2.4) Train the grid autoencoder, set the training data batch size to 48, the initial learning rate to 0.001, and use the ADAM optimizer to train the grid autoencoder; after 1000 iterations, the grid autoencoder training is completed;

[0029] 2.5) The human body mesh is input into the trained mesh autoencoder, and the three branches of the encoder output the shape encoding representation, posture encoding representation, rotation matrix and translation vector decoupled from the human body mesh.

[0030] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0031] 1. The training data set constructed by the present invention reduces similar data as much as possible while ensuring the diversity of shapes and postures, thereby accelerating the training efficiency of the network.

[0032] 2. The grid autoencoder architecture and loss function proposed in the present invention can obtain shape and posture representations of the human body mesh that are independently decoupled from each other.

[0033] 3. The human body mesh coding representation obtained by the present invention is rotationally invariant and can also be used for tasks such as interpolation of human body meshes with inconsistent orientations, and has a rich editing mechanism. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 Schematic diagram of human body meshes of various shapes and postures included in the training data set.

[0035] Figure 2 Architecture diagram of the grid autoencoder.

[0036] Figure 3 This is the result of interpolating two human body meshes with different shapes and postures using the method of the present invention. DETAILED DESCRIPTION

[0037] The present invention will be further described below in conjunction with specific embodiments.

[0038] This embodiment provides a human body mesh decoupling representation method based on a mesh autoencoder, comprising the following steps:

[0039] 1) Use a public 3D human motion sequence database to construct a training dataset and a grid autoencoder.

[0040] The use of the public 3D human motion sequence database specifically involves selecting the AMASS 3D human motion sequence database as the basic data. This database integrates other 3D human motion sequence data sets using the SMPL model. There are currently 24 sub-datasets, and the recommended data usage plan includes 9 sub-datasets for training, containing 313 different human bodies. Constructing a training data set refers to performing interval sampling on the action sequences in the 9 sub-datasets used for training in the AMASS 3D human motion sequence database to form a human mesh training data set containing different postures of different human bodies. The specific process is: first, from each action sequence The method starts from the action frame at 10% of the action sequence, samples are taken at intervals of 500 frames, and ends at the action frame at 90% of the action sequence; when the number of frames sampled from an action sequence is less than 2, the sampling interval is first reduced by a step of 50 frames, and the action sequence is sampled again. When the sampling interval is reduced to 100 frames, if the number of frames sampled is still less than 2, the sampling interval is reduced by a step of 10 frames. When the sampling interval is reduced to 10 frames, if the number of frames sampled is still less than 2, the action sequence is skipped; finally, 33451 human body mesh data of 313 different human bodies are obtained, and each human body mesh contains 6890 three-dimensional vertices, such as Figure 1 As shown, the constructed training dataset includes human body meshes of various shapes and postures.

[0041] like Figure 2As shown in the figure, the constructed grid autoencoder consists of two parts: an encoder and a decoder. The encoder consists of three branches: a shape encoder, a posture encoder, and a rotation and translation encoder. When a human body mesh X is input to the three branches of the encoder, the shape encoder outputs the shape encoding representation β of X, the posture encoder outputs the posture encoding representation θ of X, and the rotation and translation encoder outputs the rotation matrix rot and translation vector tran of X. The combined code (β, θ) is input to the decoder, and the decoder outputs a human body mesh Finally, use rot and tran Perform rotation and translation to obtain the reconstructed human body mesh of X The details of the various branches of the encoder and decoder of the grid autoencoder are as follows:

[0042] Shape encoder: It takes a vector of size (6890×3) representing a human body mesh as input and outputs a shape encoding representation of size (7×3). It consists of 4 consecutive downsampling modules, each of which consists of 1 mesh convolution layer and 1 mesh pooling layer. That is, the shape encoder has a total of 8 layers, and the output feature sizes of each layer are (6890×16), (1925×16), (1925×32), (400×32), (400×64), (54×64), (54×3), (7×3).

[0043] Pose encoder: It takes a vector of human body mesh of size (6890×3) as input and outputs a pose encoding representation of size (7×9). It consists of 4 consecutive downsampling modules, each of which consists of 1 mesh convolution layer and 1 mesh pooling layer. That is, the pose encoder has a total of 8 layers, and the output feature sizes of each layer are (6890×16), (1925×16), (1925×32), (400×32), (400×64), (54×64), (54×9), (7×9).

[0044] Rotation-translation encoder: It takes a vector of size (6890×3) representing the human body mesh as input, and outputs a rotation matrix of size (3×3) and a translation vector of size (1×3). The rotation-translation encoder is composed of 4 consecutive downsampling modules, 1 fully connected module, and 1 rotation-translation mapping module connected in sequence; each downsampling module consists of 1 grid convolution layer and 1 grid pooling layer. The output feature size of each layer of the 8-layer network in the 4 downsampling modules is (6890×8), (1925×8), and (1925×8). ×8), (1925×16), (400×16), (400×32), (54×32), (54×64), (7×64); the fully connected module consists of 3 stacked fully connected layers, and the feature sizes of each network layer are (1×128), (1×128), (1×6) respectively; the rotation translation mapping module takes the first 3-dimensional vector output by the fully connected module and uses the Rodriguez rotation formula to calculate a (3×3) rotation matrix, and takes the last 3-dimensional vector output by the fully connected module as the translation vector;

[0045] Decoder: It takes the synthetic vector of shape encoding representation and posture encoding representation of size (7×12) as input, and outputs a vector of size (6890×3) representing the human body mesh. The decoder contains 4 consecutive upsampling modules; each upsampling module consists of 1 grid anti-pooling layer and 1 grid deconvolution layer, that is, the decoder has a total of 8 layers, and the output feature sizes of each layer are (54×12), (54×64), (400×64), (400×32), (1925×32), (1925×16), (6890×16), (6890×3).

[0046] 2) Using the established training data set to train the constructed mesh autoencoder, inputting the human body mesh into the trained mesh autoencoder to obtain its decoupled representation, including the following steps:

[0047] 2.1) Construct a training data tuple. First, select a human body mesh X1 from the training data set. Then select a human body mesh X2 with the same shape as X1 but different posture from the training data set. Then select a human body mesh X3 with different shape and posture from X1 from the training data set. Finally, perform random rotation and translation operations on X1 to obtain a human body mesh X4. These four human body meshes constitute a training data tuple (X1, X2, X3, X4).

[0048] 2.2) Design the training process: First, input the training data tuple (X1, X2, X3, X4) obtained in step 2.1) into the three branches of the encoder respectively to obtain their shape encoding representation (β1, β2, β3, β4), posture encoding representation (θ1, θ2, θ3, θ4), rotation matrix (rot1, rot2, rot3, rot4) and translation vector (tran1, tran2, tran3, tran4); then input the combined code (β2, θ1) into the decoder to obtain the human body mesh Use rot4 and tran4 to Perform rotation and translation to obtain the reconstructed human body mesh of X4 Input the combined code (β2,θ4) into the decoder to get the human body mesh Use rot1 and tran1 to Perform rotation and translation to obtain the reconstructed human body mesh of X1 Input the combined code (β3,θ2) into the decoder to get the human body mesh Then put The posture encoder input to the encoder gets Gesture encoding Then the combination code (β1, ) is input to the decoder to obtain the human body mesh Use rot2 and tran2 to Perform rotation and translation to obtain the reconstructed human body mesh of X2 Finally, the combined code (β4, ) is input to the decoder to obtain the human body mesh Use rot2 and tran2 to Perform rotation and translation to obtain the reconstructed human body mesh of X2

[0049] 2.3) Design loss function L:

[0050] L=λ cross L cross +λ self L self +λ rt L rt

[0051] The loss function L contains the exchange consistency loss L cross , self-consistency loss L self and the rotation-translation consistency loss L rt Three items of content;

[0052] L cross Used to constrain the independent decoupling between the shape encoder, attitude encoder and rotation and translation encoder:

[0053]

[0054] λ cross For L cross Weight, set to 0.4;

[0055] L self Used to enhance the decoupling degree of shape encoder and attitude encoder:

[0056]

[0057] λ self For L self Weight, set to 0.4;

[0058] L rt Used to enhance the decoupling degree of the rotary-translational encoder:

[0059]

[0060] λ rt For L rt Weight, set to 0.2;

[0061] 2.4) Train the grid autoencoder, set the training data batch size to 48, the initial learning rate to 0.001, and use the ADAM optimizer to train the grid autoencoder; after 1000 iterations, the grid autoencoder training is completed;

[0062] 2.5) Input the human body mesh into the trained mesh autoencoder, and the three branches of the encoder output the shape encoding representation, posture encoding representation, rotation matrix and translation vector decoupled from the human body mesh; Figure 3 As shown in FIG. 1 , the result of shape interpolation of two human body meshes using their obtained decoupled representations is shown. The leftmost and rightmost parts of the figure are two human body meshes with different shapes and postures. They are input into the trained mesh autoencoder to obtain their decoupled representations. While keeping the posture encoding representation of the left human body mesh unchanged, linear interpolation is performed between its shape encoding representation and the shape encoding representation of the right human body mesh, and the new human body mesh is obtained by inputting it into the decoder. The obtained human body mesh is then rotated and translated using the rotation matrix and translation vector of the left human body mesh, and finally a new human body mesh with the same posture and orientation as the left human body mesh but different shape is obtained.

[0063] The feasibility of the above method of the present invention has been proved through experiments. The proposed human body mesh decoupling representation method based on grid autoencoder can obtain the shape and posture representation of the human body mesh decoupling, and has rotation invariance. The obtained coding representation can be used to synthesize human body meshes of different shapes and postures, which has practical promotion value and is worthy of promotion.

[0064] The above-described embodiments are only preferred embodiments of the present invention and are not intended to limit the scope of implementation of the present invention. Therefore, all changes made according to the shape and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A human body mesh decoupling representation method based on mesh autoencoder, characterized in that: The following steps are involved: 1) constructing a training data set and a grid autoencoder; wherein the grid autoencoder consists of an encoder and a decoder, wherein the encoder consists of three branches: a shape encoder, a posture encoder, and a rotation and translation encoder, wherein the three branches take a human body mesh as input, the shape encoder outputs a shape coding representation, the posture encoder outputs a posture coding representation, the rotation and translation encoder outputs a rotation matrix and a translation vector, and the decoder takes a combined coding of the shape coding representation and the posture coding representation as input, and outputs a human body mesh; When a human body mesh X is input to the three branches of the encoder, the shape encoder outputs the shape code representation β of X, the posture encoder outputs the posture code representation θ of X, and the rotation and translation encoder outputs the rotation matrix rot and translation vector tran of X. The combined code (β,θ) is input to the decoder, and the decoder outputs a human body mesh Finally, use rot and tran Perform rotation and translation to obtain the reconstructed human body mesh of X The details of the various branches of the encoder and decoder of the grid autoencoder are as follows: Shape encoder: It takes a vector of human body mesh of size 6890×3 as input and outputs a shape encoding representation of size 7×3. It consists of 4 consecutive downsampling modules, each of which consists of 1 mesh convolution layer and 1 mesh pooling layer. That is, the shape encoder has a total of 8 layers, and the output feature sizes of each layer are 6890×16, 1925×16, 1925×32, 400×32, 400×64, 54×64, 54×3, and 7×3 respectively. Pose encoder: It takes a vector representing a human body mesh of size 6890×3 as input and outputs a pose encoding representation of size 7×9. It consists of 4 consecutive downsampling modules, each of which consists of 1 mesh convolution layer and 1 mesh pooling layer. That is, the pose encoder has a total of 8 layers, and the output feature sizes of each layer are 6890×16, 1925×16, 1925×32, 400×32, 400×64, 54×64, 54×9, and 7×9 respectively. Rotation-translation encoder: It takes a vector of size 6890×3 representing the human body mesh as input, outputs a rotation matrix of size 3×3 and a translation vector of size 1×3. The rotation-translation encoder is composed of 4 consecutive downsampling modules, 1 fully connected module and 1 rotation-translation mapping module connected in sequence; each downsampling module consists of 1 grid convolution layer and 1 grid pooling layer. The output feature size of each network layer of the 8-layer network in the 4 downsampling modules is 6890×8, 1925×8, 1925×16, 400×16, 400×32, 54×32, 54×64, and 7×64 respectively; the fully connected module is composed of 3 stacked fully connected layers, and the output feature size of each network layer is 1×128, 1×128, and 1×6 respectively; the rotation-translation mapping module takes the first 3-dimensional vector output by the fully connected module and uses the Rodriguez rotation formula to calculate a 3×3 rotation matrix, and takes the last 3-dimensional vector output by the fully connected module as the translation vector; Decoder: It takes the synthetic vector of shape encoding representation and posture encoding representation of size 7×12 as input, and outputs a vector of size 6890×3 representing the human body mesh. The decoder contains 4 consecutive upsampling modules; each upsampling module consists of 1 mesh anti-pooling layer and 1 mesh deconvolution layer, that is, the decoder has a total of 8 layers of networks, and the output feature sizes of each layer are 54×12, 54×64, 400×64, 400×32, 1925×32, 1925×16, 6890×16, 6890×3 respectively; 2) Use the established training dataset to train the constructed mesh autoencoder, and input the human body mesh into the trained mesh autoencoder to obtain its decoupled representation.

2. The human body mesh decoupling representation method based on mesh autoencoder according to claim 1, characterized in that: In step 1), a training data set is constructed using a public 3D human motion sequence database. The public 3D human motion sequence database refers to the AMASS 3D human motion sequence database, which integrates other 3D human motion sequence data sets using the SMPL model. There are currently 24 sub-datasets, and the recommended data usage plan includes 9 sub-datasets for training, which contain 313 different human bodies. Constructing a training data set refers to performing interval sampling on the action sequences in the 9 sub-datasets used for training in the AMASS 3D human motion sequence database to form a human mesh training data set containing different postures of different human bodies. The specific process is as follows: : First, start from the action frame at 10% of each action sequence, sample at an interval of 500 frames, and end at the action frame at 90% of the action sequence; when the number of frames sampled in an action sequence is less than 2, first reduce the sampling interval by a step of 50 frames, and re-sample the action sequence. When the sampling interval is reduced to 100 frames, if the number of frames sampled is still less than 2, then reduce the sampling interval by a step of 10 frames. When the sampling interval is reduced to 10 frames, if the number of frames sampled is still less than 2, skip the action sequence; finally, 33451 human body mesh data of 313 different human bodies are obtained, and each human body mesh contains 6890 three-dimensional vertices.

3. The human body mesh decoupling representation method based on mesh autoencoder according to claim 1, characterized in that: The step 2) comprises the following steps: 2.1) Construct a training data tuple. First, select a human body mesh X1 from the training data set. Then select a human body mesh X2 with the same shape as X1 but different posture from the training data set. Then select a human body mesh X3 with different shape and posture from X1 from the training data set. Finally, perform random rotation and translation operations on X1 to obtain a human body mesh X4. These four human body meshes constitute a training data tuple (X1, X2, X3, X4). 2.2) Design the training process: First, input the training data tuple (X1, X2, X3, X4) obtained in step 2.1) into the three branches of the encoder respectively to obtain their shape encoding representation (β1, β2, β3, β4), posture encoding representation (θ1, θ2, θ3, θ4), rotation matrix (rot1, rot2, rot3, rot4) and translation vector (tran1, tran2, tran3, tran4); then input the combined code (β2, θ1) into the decoder to obtain the human body mesh Use rot4 and tran4 to Perform rotation and translation to obtain the reconstructed human body mesh of X4 Input the combined code (β2,θ4) into the decoder to get the human body mesh Use rot1 and tran1 to Perform rotation and translation to obtain the reconstructed human body mesh of X1 Input the combined code (β3, λ2) into the decoder to get the human body mesh Then put The posture encoder input to the encoder gets Gesture encoding Then encode the combination Input to the decoder to get the human body mesh Use rot2 and tran2 to Perform rotation and translation to obtain the reconstructed human body mesh of X2 Finally, encode the combination Input to the decoder to get the human body mesh Use rot2 and tran2 to Perform rotation and translation to obtain the reconstructed human body mesh of X2 2.3) Design loss function L: L=λ cross L cross +λ self L self +λ rt L rt The loss function L contains the exchange consistency loss L cross , self-consistency loss L self and the rotation-translation consistency loss L rt Three items of content; L cross Used to constrain the independent decoupling between the shape encoder, attitude encoder and rotation and translation encoder: λ cross For L cross Weight; L self Used to enhance the decoupling degree of shape encoder and attitude encoder: λ self For L self Weight; L rt Used to enhance the decoupling degree of the rotary-translational encoder: λ rt For L rt Weight; 2.4) Train the grid autoencoder, set the training data batch size to 48, the initial learning rate to 0.001, and use the ADAM optimizer to train the grid autoencoder; after 1000 iterations, the grid autoencoder training is completed; 2.5) The human body mesh is input into the trained mesh autoencoder, and the three branches of the encoder output the shape encoding representation, posture encoding representation, rotation matrix and translation vector decoupled from the human body mesh.

Images