A new method for multi-pose generation of human characters

Through grid reconstruction and deep image processing of video data, UNet and SMPL-X models are used to generate character reconstruction models, which solves the problem of incoherent clothing changes caused by character posture adjustment in comic creation, realizes automatic generation, and improves creative efficiency and quality.

CN118736072BActive Publication Date: 2024-11-08HANGZHOU JELLYFISH INTELLIGENT TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411215034.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-09-02
Publication Date
2024-11-08
Estimated Expiration
2044-09-02

AI Technical Summary

Technical Problem

During the comic creation process, designers need to frequently manually adjust the character's posture, resulting in incoherent changes in clothing, low creative efficiency and poor quality.

Method used

By acquiring video data, grid reconstruction and deep image processing, the character reconstruction model is generated using UNet network and SMPL-X model to realize the automatic generation of multi-pose characters and ensure the consistency of clothing and clothing.

Benefits of technology

It simplifies the comic creation process, improves the creative efficiency and work quality, reduces the designer's repetitive labor, ensures the consistency of clothing, and improves the creative level in the field of visual arts.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118736072B_ABST
    Figure CN118736072B_ABST
Patent Text Reader

Abstract

The present invention discloses a new method for generating multiple poses of a human character, and belongs to the technical field of computer image recognition. The method for generating multiple poses of a human character comprises obtaining video data containing a specific character; reconstructing a mesh of the character in each single-frame image in the video data to obtain a mesh of the human body; obtaining camera parameters corresponding to the single-frame image, and projecting the mesh of the human body according to the camera parameters to obtain a depth image; training a UNet network using the depth image and the single-frame image to obtain a reconstruction model of the character; obtaining an expected depth image according to expected pose parameters, and processing the expected depth image using the character reconstruction model to obtain an expected pose image of the character. After inputting the depth image, the character reconstruction model constructed by the present invention can output an avatar image matching the depth image, and the avatar image is consistent with the clothing of the character in the original single-frame image, which simplifies the complex links of comic creation and improves the creation efficiency and the quality of the work.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the technical field of computer image recognition, and in particular to a new method for generating multiple postures of a human character. Background Art

[0002] In the comic creation process, it is often necessary to make a character take a new pose, such as a standing person sitting on a chair in the next shot, and the character's clothing needs to be consistent in the previous and next shots. Currently, this requires a lot of manual repetitive work by designers, which not only has low creation efficiency, but also multiple manual modifications will lead to inconsistent clothing changes, resulting in poor comic quality and poor viewing experience. Summary of the invention

[0003] Purpose of the invention: To provide a new method for generating multiple poses of a human character, so as to solve the above-mentioned problems existing in the prior art.

[0004] Technical solution: A new method for generating multiple poses of human characters, comprising the following steps:

[0005] Step 1: Obtain video data containing a specific person;

[0006] Step 2: Reconstruct the mesh of the person in each single frame image in the video data to obtain the mesh of the human body;

[0007] Step 3: Get the camera parameters corresponding to the single frame image, and project the human body mesh according to the camera parameters to obtain the depth image;

[0008] Step 4: Use the depth image and single-frame image to train the UNet network to obtain the character reconstruction model;

[0009] Step 5: Obtain the expected depth image according to the expected posture parameters, and process the expected depth image using the character reconstruction model to obtain the expected posture image of the character.

[0010] Furthermore, the step of obtaining the mesh of the character includes the following steps:

[0011] Step 21: Encode the single frame image based on the Vision Transformer model to obtain the encoding features of the segmented image;

[0012] Step 22: Use a multi-layer perceptron to obtain confidence information of the segmented image, and filter the segmented image according to the score threshold to obtain a confidence image;

[0013] Step 23: Fusing the features of the confidence image, and using the fused features to obtain human body parameters;

[0014] Step 24: Input the human body parameters into the SMPL-X model to obtain the mesh of the human body;

[0015] The confidence information includes the confidence score and the offset value of the coordinates of the key points of the human head relative to the coordinates of the center point of the segmented image. The segmented image with a confidence score greater than or equal to the score threshold is a confidence image.

[0016] Furthermore, the method of obtaining the confidence information of the segmented image by using a multi-layer perceptron specifically includes the following steps:

[0017] Step 221: Input the encoded features of the segmented image into the hidden layer of the multi-layer perceptron;

[0018] Step 222: each node in the hidden layer performs a weighted sum operation and introduces nonlinear characteristics through the activation function ReLU;

[0019] Step 223: The multi-layer perceptron output layer has two nodes, one for the confidence score and the other for the offset value. The confidence score and the offset value are recorded as confidence information.

[0020] Furthermore, the step of fusing the features of the confidence image and obtaining the human body parameters using the fused features includes the following steps:

[0021] Step 231: combining the coding features and position coding of the confidence image into fusion features;

[0022] Step 232: concatenate the fused features with the human mean parameters defined in the SMPL-X model to obtain the expanded features;

[0023] Step 233: Use the extended feature as the query, the encoded feature as the key and value, and use block processing to update the query value;

[0024] Step 234: input the updated query value into the multi-layer perceptron again, and use the multi-layer perceptron to obtain the human body parameters;

[0025] Among them, the human body mean parameters include shape parameters, posture parameters and depth parameters.

[0026] Furthermore, in the process of obtaining the mesh of the human body, the training of the model is optimized using a loss function 1, and the loss function 1 is expressed as: ,in,

[0027] To detect the loss, , used to measure the loss of the multilayer perceptron in step 22 when detecting the human body from the segmented image, s is the confidence score, is the true value of the annotation in the training data, i and j are the row and column indexes of the segmented image;

[0028] is the regression loss, , used to measure the loss of human key point regression and human parameter regression, n is the sample size, c is the offset value predicted in step 234, x and d represent the human parameters and depth regressed in step 24, respectively. They correspond to the true values ​​of the annotations in the c, x, and d training data respectively.

[0029] is the grid loss, , used to measure the loss of the mesh of the human body regressed in step 24 and the mesh of the human body in a single frame image, n is the sample size, M is the mesh predicted by the model, is the true value of the annotation in the training data.

[0030] is the projection loss, , πK is the camera projection operator matrix, t n = (tx, ty, tz), t n are the x, y, and z axis coordinate values ​​of the key points of the human head predicted in step 22, is the labeled true value in the training data;

[0031] λ1 is a hyperparameter used to balance between several losses.

[0032] Furthermore, in the mesh reconstruction process, a full-body dataset of subject close-up frames is also used to train the model. The subject in the full-body dataset of subject close-up frames approaches the camera and presents full-body close-ups of hands and facial details in various postures.

[0033] Furthermore, projecting the mesh of the human body according to the camera parameters comprises the following steps:

[0034] Step 31: Using the formula Get the coordinate values ​​of u, v and d;

[0035] Step 32: Using the formula Concatenate the d value corresponding to the coordinates obtained in step 31 in the new view and the r, g, and b values ​​at the corresponding grid points to obtain a depth image;

[0036] Among them, x is a 3D point cloud coordinate of the mesh under the SMPL model definition, K, R, t, theta and beta are the parameters of the camera intrinsic parameters and camera extrinsic parameters, which are used to describe various transformation relationships in camera imaging and imaging reconstruction.

[0037] Furthermore, the UNet network in step 4 includes an input layer, an intermediate layer and an output layer, the intermediate layer includes convolution, deconvolution and pooling neural networks, the kernel size of the convolution layer is set to 3, and the kernel size of the maximum pooling layer is set to 2.

[0038] Furthermore, the character reconstruction model obtained in step 4 is iterated until convergence using the second loss function, and the second loss function is expressed as:

[0039] ,

[0040] in, is the VGG perceptual loss, , I gt is a single frame image, I X is the reconstructed image output by the character reconstruction model, i is the activation value of the i-th layer of the UNet network, and the multi-scale discrimination loss and feature matching loss are expressed as , λ2 is a hyperparameter of multi-scale discriminant loss and feature matching loss, and λ2 is selected as 0.1.

[0041] Furthermore, the expected posture parameters are obtained by identifying in a motion capture system or a video sequence.

[0042] Beneficial effects: The present invention first reconstructs the mesh of the character in a single-frame image in the video, and then processes the reconstructed human mesh information into a depth image, and uses the depth image and the original single-frame image to train the UNet network, and finally obtains a character reconstruction model that inputs a depth image and outputs a character reconstruction image containing a clothing-consistent clone image. When creating an animation, by changing the posture parameters, the depth image in the desired posture can be obtained, and the character reconstruction model can be used to quickly obtain a clothing-consistent clone image that matches the depth image. This solution not only simplifies the complex links in the comics creation process, but also improves the creation efficiency and the quality of the work, and is of great significance for improving the creation level of comics and other visual arts. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a flow chart of the multi-posture generation method in the invention. DETAILED DESCRIPTION

[0044] In the following description, a large number of specific details are provided to provide a more thorough understanding of the present invention. However, it is apparent to those skilled in the art that the present invention can be implemented without one or more of these details. In other examples, in order to avoid confusion with the present invention, some technical features well known in the art are not described.

[0045] like Figure 1As shown, a new method for generating multiple poses of a human character includes the following steps:

[0046] Step 1: Obtain video data containing a specific person;

[0047] Step 2: Reconstruct the mesh of the person in each single frame image in the video data to obtain the mesh of the human body;

[0048] Step 3: Get the camera parameters corresponding to the single frame image, and project the human body mesh according to the camera parameters to obtain the depth image;

[0049] Step 4: Use the depth image and single-frame image to train the UNet network to obtain the character reconstruction model;

[0050] Step 5: Obtain the expected depth image according to the expected posture parameters, and process the expected depth image using the character reconstruction model to obtain the expected posture image of the character.

[0051] The present invention first reconstructs the mesh of the character in a single-frame image in the video, and then processes the reconstructed human mesh information into a depth image, and uses the depth image and the original single-frame image to train the UNet network, and finally obtains a character reconstruction model that inputs a depth image and outputs a clone image with consistent clothing. When creating an animation, by changing the posture parameters, the depth image in the desired posture can be obtained, and the clone image with consistent clothing that matches the depth image can be quickly obtained using the character reconstruction model. This two-stage fully automatic workflow reduces the need for manual intervention and the repetitive work required by designers when creating different postures. It also ensures the consistency of the clothing of the previous and subsequent characters, and solves the problem of incoherent clothing changes caused by multiple modifications in traditional methods, which helps to improve the quality and appearance of the final work. It is of great significance to improve the level of creation in comics and other visual arts.

[0052] The process of obtaining the character's mesh in step 2 includes the following steps:

[0053] Step 21: Encode the single frame image based on the Vision Transformer model to obtain the encoding features of the segmented image.

[0054] Specifically, when using Vision Transformer to encode an input single-frame image, the single-frame image is first divided into a series of segmented images of the same size. Each segmented image is linearly projected and then additionally positionally encoded to form a token sequence. These token sequences are sent to the Transformer model for processing to obtain the encoded features of each segmented image.

[0055] Step 22: Use a multi-layer perceptron to obtain the confidence information of the segmented image, and filter the segmented image according to a score threshold to obtain a confidence image; wherein the confidence information includes a confidence score and an offset value of the coordinates of the key points of the human head relative to the coordinates of the center point of the segmented image, and a segmented image with a confidence score greater than or equal to the score threshold is a confidence image.

[0056] Specifically, the multilayer perceptron is a feedforward neural network, including an input layer, multiple hidden layers, and an output layer. The input layer receives the encoded features obtained from step 21; the hidden layer contains multiple nodes, each of which performs a weighted sum operation and outputs the result through an activation function; the output layer generates the final output, that is, the output confidence score and the offset value of the head key point.

[0057] The confidence score is a probability value between 0 and 1, indicating the probability that the segmented image contains a human body. This score can be used to evaluate the model's degree of certainty about the segmented region. By setting a score threshold, the regions most likely to contain a human body can be screened out, thereby improving the accuracy of subsequent processing. The threshold is usually 0.5 or higher. If the confidence score is higher than the set threshold, the segmented region is considered to be a reliable detection result and is marked as a confident image.

[0058] The offset value refers to the offset distance of the key points of the human head relative to the coordinates of the center point of the segmented image. These key points may include feature points such as eyes, noses, and ears. The offset value can help locate the exact position of these key points, which is crucial for subsequent human mesh reconstruction. In addition, the offset value can also be used as an additional source of information when calculating the confidence score. That is, if a segmented image contains a human head, then its offset value should point to a reasonable head position, which can be used to enhance the reliability of the confidence score.

[0059] Step 23: Fusing the features of the confidence image, and using the fused features to obtain human body parameters;

[0060] Specifically, by fusing the features of the confidence image, combining position encoding and human mean parameters, using the attention mechanism and multi-layer perceptron, accurate human parameters can be finally obtained. These parameters are then input into the SMPL-X model to generate the human body mesh. This process ensures the accuracy and fidelity of the human body mesh, thereby ensuring the quality of the final generated human pose image.

[0061] Step 24: Input the human body parameters into the SMPL-X model to obtain the mesh of the human body.

[0062] Specifically, the SMPL-X model is a statistical model widely used in human posture and shape modeling. Based on the human body parameters (shape parameters, posture parameters, and depth parameters) obtained in step 23, the height and body shape of the person are first determined according to the shape parameters, then the person is determined to be in that specific state according to the posture parameters, and finally the mesh is ensured to be at the correct depth position according to the depth parameters. The SMPL-X model combines these parameters to generate a 3D mesh of a specific human posture.

[0063] In particular, because the details of the characters are mainly concentrated on the face and hands, this solution specially trains the model used in step 2. The full-body dataset of the subject close-up frames is a full-body close-up of the subject in various poses with hands and facial details close to the camera. This helps the model learn the details of the face and hands, making the generated human 3D mesh more detailed and realistic.

[0064] In step 22, “obtaining confidence information of the segmented image using a multi-layer perceptron” specifically includes the following steps:

[0065] Step 221: Input the encoded features of the segmented image into the hidden layer of the multi-layer perceptron;

[0066] Step 222: each node in the hidden layer performs a weighted sum operation and introduces nonlinear characteristics through the activation function ReLU;

[0067] Step 223: The multi-layer perceptron output layer has two nodes, one for the confidence score and the other for the offset value. The confidence score and the offset value are recorded as confidence information.

[0068] The training process of the multi-layer perceptron to obtain the confidence information of the segmented image requires initializing weights, forward propagation, loss calculation, back propagation, and iterative training.

[0069] In step 23, “fusing the features of the confidence image and obtaining human body parameters using the fused features” specifically includes the following steps:

[0070] Step 231: combining the coding features and position coding of the confidence image into fusion features;

[0071] Specifically, the features of the confidence image include encoding features (representing some features of the person in a single frame image, namely, the person's shape, posture and other information) and position encoding (used to add spatial position information to the features at each position to help the model understand the specific position of the features in the image). This method is very important for reconstructing the human body mesh because it can help the model better understand the relative position relationship of different parts of the human body in the image, thereby more accurately estimating human body parameters.

[0072] Step 232: concatenate the fused features with the human body mean parameters defined in the SMPL-X model to obtain extended features; wherein the human body mean parameters include shape parameters, posture parameters, and depth parameters.

[0073] Specifically, the extended features are formed by splicing the fused features with the mean human body parameters, and their purpose is to provide more comprehensive information in the subsequent steps so as to better estimate the human body parameters. The mean human body parameters are a human body model customized in the SMPL-X model that represents a general sense and is suitable for most people, but it is not targeted at any specific individual. The relationship between the mean human body parameters and the human body parameters is that the mean human body parameters serve as a reference benchmark to help the model better understand the basic structure of the human body. The extended features help the model learn how to infer more specific human body parameters from the image. In particular, in step 234, the extended features are processed by the attention mechanism and input into the multi-layer perceptron to finally obtain the human body parameters. The human body parameters are calculated based on the single-frame image information in the video data and are closer to the characteristics of the actual person.

[0074] Step 233: Use the extended feature as the query, the encoded feature as the key and value, and use block processing to update the query value;

[0075] Specifically, the block consists of cross attention, self-attention, and multi-layer perceptron in sequence, and the number of blocks is multiple. The attention mechanism is used to help the model better focus on features related to human body parameter estimation. Among them, cross attention interacts with the expanded features (query) and the encoded features (keys and values), and determines which encoded features are most important to the expanded features by calculating similarity scores. These scores are used to weight the encoded features to produce a weighted sum, which is the updated query value. Self-attention interacts with the encoded features and itself, and strengthens the feature representation by calculating the similarity scores between each feature. This process helps capture long-term dependencies between features. The features processed by cross attention and self-attention are input into the multi-layer perceptron to generate human body parameter estimation.

[0076] The number of blocks can be multiple, which means that the above process will be repeated multiple times. Each block will further optimize the query value until the best human body parameter estimation is achieved.

[0077] Step 234: input the updated query value into the multi-layer perceptron again, and use the multi-layer perceptron to obtain the human body parameters;

[0078] The updated query value is input into the multi-layer perceptron again to further optimize the feature representation and ensure that the model can make full use of the information processed by the attention mechanism to more accurately estimate the human body parameters. This practice helps to improve the accuracy and quality of the final generated human mesh.

[0079] In the process of obtaining the mesh of the human body, the training of the model is optimized using a loss function 1, which is expressed as: ,in,

[0080] To detect the loss, , used to measure the loss of the multilayer perceptron in step 22 when detecting the human body from the segmented image, s is the confidence score, is the true value of the annotation in the training data, i and j are the row and column indexes of the segmented image;

[0081] is the regression loss, , used to measure the loss of human key point regression and human parameter regression, n is the sample size, c is the offset value predicted in step 234, x and d represent the human parameters and depth regressed in step 24, respectively. They correspond to the true values ​​of the annotations in the c, x, and d training data respectively.

[0082] is the grid loss, , used to measure the loss of the mesh of the human body regressed in step 24 and the mesh of the human body in a single frame image, n is the sample size, M is the mesh predicted by the model, is the true value of the annotation in the training data.

[0083] is the projection loss, , πK is the camera projection operator matrix, t n = (tx, ty, tz), t n are the x, y, and z axis coordinate values ​​of the key points of the human head predicted in step 22, is the labeled true value in the training data;

[0084] λ1 is a hyperparameter used to balance between several losses.

[0085] Projecting the human body mesh according to the camera parameters includes the following steps:

[0086] Step 31: Using the formula Get the coordinate values ​​of u, v and d;

[0087] Among them, x is the coordinate of a 3D cloud point of the mesh under the SMPL model definition; K is the transformation matrix, the camera internal parameter; R is the rotation matrix or quaternion, the camera external parameter; t is the translation matrix or x / y / z, the camera external parameter; theta is the pose matrix, the dimension is equal to the number of joint points, which is 3, and beta is the 10-dimensional vector obtained after PCA dimensionality reduction. The above parameters are used to describe various transformation relationships in camera imaging and imaging reconstruction.

[0088] Step 32: Using the formula Concatenate the d value corresponding to the coordinates obtained in step 31 in the new view and the r, g, and b values ​​at the corresponding grid points to obtain a depth image;

[0089] Among them, r, g, and b represent the values ​​of the red, green, and blue color channels respectively. The depth image is formed by concatenating the d value of the coordinates in the new view with the corresponding r, g, and b values. The d value represents the depth information, while the r, g, and b values ​​retain the color information, so that the generated depth image contains not only the depth data but also the color information, which is very important for the subsequent reconstruction of the clone image containing the consistency of clothing.

[0090] The UNet network in step 4 includes an input layer, an intermediate layer and an output layer. The intermediate layer includes convolution, deconvolution and pooling neural networks. The kernel size of the convolution layer is set to 3, and the kernel size of the maximum pooling layer is set to 2.

[0091] The character reconstruction model obtained in step 4 is iterated until convergence using the second loss function, and the second loss function is expressed as:

[0092] ,

[0093] in, is the VGG perceptual loss, , I gt is a single frame image, I X is the reconstructed image output by the character reconstruction model, i is the activation value of the i-th layer of the UNet network; the multi-scale discrimination loss and feature matching loss are expressed as , represents the objective function of minimizing the generator G, where the maximum is the discriminator D k The objective function of

[0094] is the adversarial loss between the generator and the kth discriminator, which measures whether the generator can deceive the discriminator into thinking that the generated data is real; is the feature matching loss, which measures the similarity between the generated data and the real data in low-level features; λ2 is a hyperparameter of the multi-scale discriminant loss and feature matching loss, and λ2 is selected as 0.1.

[0095] The second loss function is a key component of the training process, which is used to guide the learning process of the character reconstruction model and ensure that the model can accurately reconstruct the expected pose image of the character from the input depth image, which is consistent with the clothing of the character in the original single frame image. The first loss function optimizes the model parameters by minimizing the gap between the predicted results and the actual results, thereby improving the prediction performance of the model.

[0096] Preferably, the expected posture parameters can be extracted from a posture library, or obtained through a motion capture system, or recognized from a video sequence. Because the expected posture parameters are a matrix, they can also be manually adjusted according to the existing posture parameters.

[0097] The preferred embodiments of the invention are described in detail above in conjunction with the accompanying drawings; however, the invention is not limited to the specific details in the above embodiments; within the technical concept of the invention, various equivalent transformations may be made to the technical solutions of the invention, and all of these equivalent transformations fall within the protection scope of the invention.

Claims

1. A new method for multi-pose generation of human characters, It is characterized in that The following steps are involved: Step 1: Obtain video data containing a specific person; Step 2: Reconstruct the mesh of the person in each single frame image in the video data to obtain the mesh of the human body; Wherein, the step 2: reconstructing the mesh of the person in each single frame image in the video data to obtain the mesh of the human body includes the following steps: Step 21: Encode the single frame image based on the Vision Transformer model to obtain the encoding features of the segmented image; Step 22: Use a multi-layer perceptron to obtain confidence information of the segmented image, and filter the segmented image according to the score threshold to obtain a confidence image; Step 23: Fusing the features of the confidence image, and using the fused features to obtain human body parameters; Wherein, the step 23: fusing the features of the confidence image and obtaining the human body parameters using the fused features, comprises the following steps: Step 231: combining the coding features and position coding of the confidence image into fusion features; Step 232: concatenate the fused features with the human mean parameters defined in the SMPL-X model to obtain the expanded features; Step 233: Use the extended feature as the query, the encoded feature as the key and value, and use block processing to update the query value; Step 234: input the updated query value into the multi-layer perceptron again, and use the multi-layer perceptron to obtain the human body parameters; Among them, human body parameters include shape parameters, posture parameters and depth parameters; Step 24: Input the human body parameters into the SMPL-X model to obtain the mesh of the human body; The confidence information includes a confidence score and an offset value of the coordinates of the key points of the human head relative to the coordinates of the center point of the segmented image. The segmented image with a confidence score greater than or equal to a score threshold is a confidence image. In the process of obtaining the mesh of the human body, the training of the model is optimized using a loss function 1, and the loss function 1 is expressed as: ,in, To detect the loss, , used to measure the loss of the multilayer perceptron in step 22 when detecting the human body from the segmented image, s is the confidence score, is the true value of the annotation in the training data, i and j are the row and column indexes of the segmented image; is the regression loss, , used to measure the loss of human key point regression and human parameter regression, n is the sample size, c is the offset value predicted in step 22, x and d represent the human parameters regressed in step 234, Corresponding to the true value of the annotation in the c, x, d training data respectively; is the grid loss, , used to measure the loss of the mesh of the human body regressed in step 24 and the mesh of the human body in a single frame image, n is the sample size, M is the mesh predicted by the model, is the labeled true value in the training data; is the projection loss, , πK is the camera projection operator matrix, t n = (tx, ty, tz), t n are the x-, y-, and z-axis coordinate values ​​of the key points of the human head obtained according to the offset values ​​predicted in step 22, is the labeled true value in the training data; λ 1 is a hyperparameter used to balance between several losses; Step 3: Get the camera parameters corresponding to the single frame image, and project the human body mesh according to the camera parameters to obtain the depth image; Step 4: Use the depth image and single-frame image to train the UNet network to obtain the character reconstruction model; Step 5: Obtain the expected depth image according to the expected posture parameters, and process the expected depth image using the character reconstruction model to obtain the expected posture image of the character.

2. A new method for generating multiple poses of a human character according to claim 1, It is characterized in that The method of obtaining the confidence information of the segmented image by using a multi-layer perceptron specifically includes the following steps: Step 221: Input the encoded features of the segmented image into the hidden layer of the multi-layer perceptron; Step 222: each node in the hidden layer performs a weighted sum operation and introduces nonlinear characteristics through the activation function ReLU; Step 223: The multi-layer perceptron output layer has two nodes, one for the confidence score and the other for the offset value. The confidence score and the offset value are recorded as confidence information.

3. A new method for generating multiple poses of a human character according to claim 1, It is characterized in that During the mesh reconstruction process, a full-body dataset of subject close-up frames is also used to train the model. The subject of the full-body dataset of subject close-up frames approaches the camera and presents full-body close-ups of hands and facial details in various postures.

4. A new method for generating multiple poses of a human character according to claim 1, It is characterized in that The method of projecting the mesh of the human body according to the camera parameters comprises the following steps: Step 31: Using the formula Get the coordinate values ​​of u, v and d; Step 32: Using the formula Concatenate the d value corresponding to the coordinates obtained in step 31 in the new view and the r, g, and b values ​​at the corresponding grid points to obtain a depth image; Among them, x is a 3D point cloud coordinate of the mesh under the SMPL model definition, K, R, and t are camera intrinsic parameters, which are used to describe various transformation relationships in camera imaging and imaging reconstruction.

5. A new method for generating multiple poses of a human character according to claim 4, It is characterized in that The UNet network in step 4 includes an input layer, an intermediate layer and an output layer, the intermediate layer includes convolution, deconvolution and pooling neural networks, the kernel size of the convolution layer is set to 3, and the kernel size of the maximum pooling layer is set to 2.

6. A new method for generating multiple poses of a human character according to claim 5, It is characterized in that The character reconstruction model obtained in step 4 uses the loss function 2 to iterate the model until convergence, and the loss function 2 is expressed as: , Among them, G is the generator of the adversarial network, D k The discriminator used in the adversarial network, is the adversarial loss between the generator and the k-th discriminator, is the feature matching loss, is the VGG perceptual loss, , I gt is a single frame image, I X The reconstructed image output by the character reconstruction model, is the activation value of the i-th layer of the UNet network, and the multi-scale discrimination loss and feature matching loss are expressed as ,λ 2 is a hyperparameter for multi-scale discriminant loss and feature matching loss, and λ is selected 2 is 0.

1.

7. A new method for generating multiple poses of a human character according to claim 6, It is characterized in that The expected posture parameters are obtained by identifying in a motion capture system or a video sequence.