3D modeling method and control system based on animation design

Abstract

The invention belongs to the technical field of 3D modeling based on image processing, and particularly relates to a 3D modeling method based on animation design and a control system.The animation modeling subsystem completes automatic three-dimensional modeling by obtaining a front image of a single character to obtain a character three-dimensional model; the animation action driving subsystem firstly outputs final motion parameters according to the input motion data, and then adjusts a role three-dimensional model; finally, after the output motion parameters and the role three-dimensional model are obtained, frame-by-frame motion parameter adaptation is carried out, layer-by-layer driving skeleton operation is carried out, and preliminary role motion roaming is generated; and the rendering display module is used for rendering and displaying the initial role motion roaming to obtain final role motion roaming. According to the invention, rapid role three-dimensional modeling can be carried out according to a single front image, the motion parameters are obtained according to the motion data, and a plurality of role three-dimensional models and a plurality of motion parameters are randomly matched to directly and rapidly obtain three-dimensional animation roaming of a plurality of roles in different motion states.

Description

Technical Field

[0001] The invention belongs to the technical field of 3D modeling based on image processing, and in particular relates to a 3D modeling method and a control system based on animation design. Background Art

[0002] Animation design belongs to the CG (abbreviation of Computer Graphics) industry. It mainly forms a unique visual art creation mode through the combination of comics and animation with storylines, and uses related expression methods such as two-dimensional and three-dimensional animations, and animation special effects. When conducting animation design, 3D modeling is required, especially character modeling.

[0003] Existing character modeling methods, such as Chinese patent number CN105261062B, disclose a character segmentation modeling method, including using a 3D scanner to collect cloud data of character head points to construct a character head model; preprocessing the constructed character head model to obtain a preprocessed character head model; using an edge folding simplification method based on edge curvature and area error to simplify the preprocessed character head model, and using an interactive method to retain the detailed features of the character head model; matching and merging the obtained character head model with the character torso model in the character torso database to finally obtain a complete character model. In addition, in the existing animation roaming production process, the character model is first produced, and then the action control is completed according to the character model, and the animation roaming is finally output. For some animation design needs, such as different characters in the game need to repeat the same action roaming, it is cumbersome to produce them separately. For this reason, we propose a 3D modeling method and control system based on animation design, which can quickly perform character 3D modeling according to a single front image, and at the same time obtain motion parameters according to motion data, and randomly match several character 3D models with several motion parameters to directly and quickly obtain 3D animation roaming of multiple characters in different motion states. Summary of the invention

[0004] The purpose of the present invention is to provide a 3D modeling method and control system based on animation design, which can quickly perform three-dimensional character modeling based on a single frontal image, and at the same time obtain motion parameters based on motion data, and randomly match several character three-dimensional models with several motion parameters to directly and quickly obtain three-dimensional animation roaming of multiple characters in different motion states.

[0005] The technical solution adopted by the present invention is as follows: A 3D modeling method based on animation design includes the following steps: Step 1: Build an animation modeling subsystem, obtain the front image of the character, then remove the background to obtain the front image of the character without background, send the front image of the character without background to the back convolution prediction generation network, and predict the back image of the character without background; Step 2: construct a 3D model, and then use the 3D model to generate an SMPL model based on the front image of the character without background, and adapt the front image of the character without background and the back image of the character without background based on the SMPL model to generate an SMPL front normal map and an SMPL back normal map; Step 3: Generate the clothing SMPL front normal map and the clothing SMPL back normal map through the convolutional neural network respectively according to the SMPL front normal map and the SMPL back normal map, and then reversely adjust the SMPL model through the clothing SMPL front normal map and the clothing SMPL back normal map; Step 4: The SMPL model, the clothing SMPL front normal map, and the clothing SMPL back normal map are predicted by a convolutional neural network to output the character's three-dimensional model; Step 5: construct an animation action driving subsystem, which includes a three-dimensional action data processing module, a model adjustment module, a skeleton skinning animation driving module, and a rendering display module; Step 6: The three-dimensional motion data processing module uses the OpenPose joint point detection algorithm to obtain the two-dimensional joint point data of the character according to the input motion data, and uses the obtained two-dimensional joint point data as the training data of the 3D character posture estimation neural network system to obtain the 3D character motion data stream, and finally outputs the motion parameters, and sends the motion parameters to the skeleton skinning animation driving module; Step 7: the model adjustment module obtains the character 3D model outputted in step 4, completes preprocessing of the character 3D model, and obtains the bone structure features of the character 3D model; Step 8: After the skeleton skinning animation driving module obtains the motion parameters outputted in step 6, the character 3D model outputted in step 4, and the skeleton structure features of the character 3D model, it performs frame-by-frame motion parameter adaptation, calculates driving data, and performs layer-by-layer driving skeleton operation to generate preliminary character motion roaming; Step 9: The rendering display module performs cloud rendering on the preliminary motion video to obtain the final character motion roaming.

[0006] Preferably, in step 1, the background is removed by a digital image segmentation processing method, and the back convolutional generation network adopts a character back translation based on Pix2PixHD. Specifically, the back generator network uses a global generator G, and the global generator G first downsamples the input image, extracts image features in the convolution layer, obtains global information, and then inputs the obtained features into the residual network, and then restores the features to the same original size as the input background-free character front image through the upsampling convolution layer. Then the back generator network adopts a multi-scale discriminator to expand the receptive field of the image. Finally, the back generator network is optimized by a loss function to predict the background-free character back image.

[0007] Preferably, in step 2, the SMPL naked character model is used to reduce spatial structure ambiguity and standardize the prediction of the character surface range; and the PyMAFI61 method is used to estimate the SMPL model, and the SMPL front normal map and the SMPL back normal map are directly obtained based on the SMPL model.

[0008] Preferably, in step 3, the clothing front normal map and the clothing back normal map are trained based on the SMPL front normal map, the SMPL back normal map and the background-free character front image, and the background-free character front image and the background-free character back image are respectively combined with images of two corresponding angles of the SMPL model to obtain the clothing front normal map and the clothing back normal map, and the prediction process involves an optimization process of the normal map and the SMPL model.

[0009] Preferably, the convolutional neural network used in step 4 is a trained implicit expression network.

[0010] Preferably, the motion data input in step 6 is the character motion roaming, the three-dimensional motion data processing module is provided with a motion data storage library, and the motion data storage library stores a number of motion data. In the process of outputting frame-by-frame motion parameters according to the motion data in step 6, the OpenPose joint point detection algorithm is first used to obtain the character motion roaming, and the 2D position pixels of the character joints in each frame image are extracted and saved in JSON format; the character posture recognition based on the OpenPose architecture uses the VGG network model; then, the OpenPose architecture uses a convolutional neural network to obtain the degree of association between different character parts and the confidence map of different key points of the character; the 2D key points of the character existing in the image are obtained by parsing the relevant key points and confidence, and finally the motion parameters containing motion skeleton information are generated according to the 2D key point information of each frame.

[0011] Preferably, in step 7, the model adjustment module performs quantitative and qualitative adjustments on the character's three-dimensional model. Quantitatively adjust the character's 3D model to directly control the shape of the entire body of the character's 3D model through numerical parameters, and control the corresponding shape through specific numerical values; quantitatively adjust the character's 3D model so that the modeled character's 3D model fits the model in the motion parameters; In the qualitative adjustment of the character's three-dimensional model, the parameters correspond to the specific features of the model and are observable; qualitative adjustment is the final adjustment of the character's three-dimensional model based on quantitative adjustment, so that the virtual character obtained by modeling coincides with the model in the motion parameters.

[0012] Preferably, the step 8 specifically includes the following steps: Step 801: Calculate driving data; The first step of skeletal animation control is data preparation and calculation, including initial frame pose data and current frame transformation data; the rotation and translation transformation of the current frame are superimposed on the initial pose to generate the driving data of the current frame pose; Step 802: driving the skeleton layer by layer; In each frame, when the data drives the bones to rotate and translate, the tree-like hierarchical relationship of the bones is utilized, and a layer-by-layer driving method is adopted to ensure that all parent node joints are driven before child node joints; each transformation corresponds to the control of a joint, and the combination of transformations of a single part completes the posture transformation of the whole body; when the actions of all frames are played continuously, a dynamic three-dimensional animation can be reconstructed.

[0013] Preferably, the step 9 specifically includes adjusting the observation angle and the playback speed; Observation angle adjustment: Use the adjustment button to adjust the position and angle of the space camera, thereby adjusting the space observation angle and adjusting the observation angle of the reconstructed animation; Playback speed adjustment: Adjust the frame-by-frame rendering speed through the buttons, control the frequency of frame updates, and adjust the playback speed.

[0014] A 3D modeling control system based on animation design, characterized by: comprising an animation modeling subsystem and an animation action driving subsystem; The animation modeling subsystem is provided with a three-dimensional modeling model, which completes automatic three-dimensional modeling by acquiring a single front image of a character to obtain a three-dimensional model of the character, and sends the three-dimensional model of the character to the animation action driving subsystem; The animation modeling subsystem is provided with a character three-dimensional model database, and the character three-dimensional model database is used to store an initial single character frontal image and a corresponding character three-dimensional model; The animation action driving subsystem includes a three-dimensional action data processing module, a model adjustment module, a skeleton skinning animation driving module, and a rendering and display module. The three-dimensional action data processing module is used to output the final motion parameters according to the input motion data. The model adjustment module obtains the three-dimensional model of the character generated by the animation modeling subsystem, and makes quantitative and qualitative adjustments to the three-dimensional model of the character, so that the three-dimensional model of the character coincides with the model in the motion parameters; the skeleton skinning animation driving module obtains the motion parameters output by the three-dimensional action data processing module, the three-dimensional model of the character output by the animation modeling subsystem, and the skeleton structure characteristics of the three-dimensional model of the character, and then performs frame-by-frame motion parameter adaptation, calculates driving data, and performs layer-by-layer driving skeleton operation to generate a preliminary character motion roaming; the rendering and display module is used to render and display the preliminary character motion roaming to obtain the final character motion roaming; A motion data storage library is provided inside the three-dimensional motion data processing module, and the motion data storage library stores a number of motion data and corresponding motion parameters including motion skeleton information.

[0015] The technical effects achieved by the present invention are: In the present invention, in step 1 to step 4, the input RGB image of the front side of the character is used to predict the RGB image of the back side of the character through the image translation network, which is used as the prior knowledge of the normal prediction network to optimize the accuracy of the predicted normal image; the detail effect of the back side of the model is optimized; the present invention uses the RGB image of the front side and the positive normal image of the SMPL-mesh, and the predicted RGB image of the back side and the back normal image of the SMPL-mesh as two input pairs to obtain a normal image generation network for a dressed human body at two corresponding angles; the predicted normal image is used to extract features to add details such as clothing wrinkles to reconstruct the back side of the human body, so that the final obtained three-dimensional model of the character is more accurate, and the output of the three-dimensional model of the character is achieved according to a single front image, with faster calculation speed and more accurate output results.

[0016] In step 6 of the present invention, OpenPose is an open source processing framework for real-time multi-person 2D joint detection, which is written in C++ programming language, uses OpenCV and Cafe, has high accessibility, and can be freely adapted to various operating systems; OpenPose is a real-time deep neural network model that can jointly detect human hands, facial points, body and foot key points in a single image or video frame, and ultimately makes the obtained motion parameters more accurate.

[0017] In the present invention, by constructing an animation action driving subsystem, it is possible to extract the two-dimensional joint data features according to the input motion video to obtain motion parameters, and at the same time adjust the character three-dimensional model through the model adjustment module so that the two-dimensional joint data features in the motion parameters can be adapted, and then the skeleton skinning animation driving module is used to adapt the motion parameters frame by frame, calculate the driving data, and drive the skeleton operation layer by layer to generate a preliminary character motion roaming, which is rendered and obtained to be displayed. Therefore, the present invention can perform rapid character three-dimensional modeling according to a single frontal image, and at the same time obtain motion parameters according to the motion data, and randomly match a number of character three-dimensional models with a number of motion parameters to directly and quickly obtain a three-dimensional animation roaming of multiple characters in different motion states. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a flow chart of a 3D modeling method based on animation design of the present invention; Figure 2 It is a system block diagram of a 3D modeling control system based on animation design in the present invention; Figure 3 The present invention is a flow chart of a system for generating a three-dimensional model of a character in a 3D modeling method based on animation design. DETAILED DESCRIPTION

[0019] In order to make the purpose and advantages of the present invention more clearly understood, the present invention is specifically described below in conjunction with embodiments. It should be understood that the following text is only used to describe one or several specific embodiments of the present invention, and does not strictly limit the scope of protection of the specific claims of the present invention.

[0020] like Figure 1-Figure 3 As shown, a 3D modeling method based on animation design includes the following steps: Step 1: Build an animation modeling subsystem, obtain the front image of the character, then remove the background to obtain the front image of the character without background, send the front image of the character without background to the back convolution prediction generation network, and predict the back image of the character without background; Step 2: construct a 3D model, and then use the 3D model to generate an SMPL model based on the front image of the character without background, and adapt the front image of the character without background and the back image of the character without background based on the SMPL model to generate an SMPL front normal map and an SMPL back normal map; Step 3: Generate the clothing SMPL front normal map and the clothing SMPL back normal map through the convolutional neural network respectively according to the SMPL front normal map and the SMPL back normal map, and then reversely adjust the SMPL model through the clothing SMPL front normal map and the clothing SMPL back normal map; Step 4: The SMPL model, the clothing SMPL front normal map, and the clothing SMPL back normal map are predicted by a convolutional neural network to output the character's three-dimensional model; Step 5: construct an animation action driving subsystem, which includes a three-dimensional action data processing module, a model adjustment module, a skeleton skinning animation driving module, and a rendering display module; Step 6: The three-dimensional motion data processing module uses the OpenPose joint point detection algorithm to obtain the two-dimensional joint point data of the character according to the input motion data, and uses the obtained two-dimensional joint point data as the training data of the 3D character posture estimation neural network system to obtain the 3D character motion data stream, and finally outputs the motion parameters, and sends the motion parameters to the skeleton skinning animation driving module; Step 7: The model adjustment module obtains the character 3D model outputted in step 4, completes preprocessing of the character 3D model, and obtains the bone structure features of the character 3D model; Step 8: After the skeleton skinning animation driving module obtains the motion parameters outputted in step 6, the character 3D model outputted in step 4, and the skeleton structure features of the character 3D model, it performs frame-by-frame motion parameter adaptation, calculates driving data, and performs layer-by-layer driving skeleton operation to generate preliminary character motion roaming; Step 9: The rendering display module performs cloud rendering on the preliminary motion video to obtain the final character motion roaming.

[0021] In the present invention, in step 1 to step 4, the input RGB image of the front side of the character is used to predict the RGB image of the back side of the character through the image translation network, which is used as the prior knowledge of the normal prediction network to optimize the accuracy of the predicted normal image; the detail effect of the back side of the model is optimized; the present invention uses the RGB image of the front side and the positive normal image of the SMPL-mesh, and the predicted RGB image of the back side and the back normal image of the SMPL-mesh as two input pairs to obtain a normal image generation network for a dressed human body at two corresponding angles; the predicted normal image is used to extract features to add details such as clothing wrinkles to reconstruct the back side of the human body, so that the final obtained three-dimensional model of the character is more accurate, and the output of the three-dimensional model of the character is achieved according to a single front image, with faster calculation speed and more accurate output results.

[0022] In step 6 of the present invention, OpenPose is an open source processing framework for real-time multi-person 2D joint detection, released by the 201 research group of Carnegie Mellon University; the framework is written in C++ programming language, using OpenCV and Cafe, with high accessibility and can be freely adapted to various operating systems; OpenPose is a real-time deep neural network model that can jointly detect human hands, facial points, body and foot key points in a single image or video frame, ultimately making the obtained motion parameters more accurate.

[0023] In the present invention, by constructing an animation action driving subsystem, it is possible to extract the two-dimensional joint data features according to the input motion video to obtain motion parameters, and at the same time adjust the character three-dimensional model through the model adjustment module so that the two-dimensional joint data features in the motion parameters can be adapted, and then the skeleton skinning animation driving module is used to adapt the motion parameters frame by frame, calculate the driving data, and drive the skeleton operation layer by layer to generate a preliminary character motion roaming, which is rendered and obtained to be displayed. This allows the present invention to quickly perform three-dimensional character modeling based on a single frontal image, and at the same time obtain motion parameters based on the motion data, and randomly match a number of character three-dimensional models with a number of motion parameters to directly and quickly obtain three-dimensional animation roaming of multiple characters in different motion states.

[0024] Preferably, in step 1, the background is removed by a digital image segmentation processing method, and the back convolutional generation network adopts a character back translation based on Pix2PixHD. Specifically, the back generator network uses a global generator G, and the global generator G first downsamples the input image, extracts image features in the convolution layer, obtains global information, and then inputs the obtained features into the residual network, and then restores the features to the same original size as the input background-free character front image through the upsampling convolution layer. Then the back generator network adopts a multi-scale discriminator to expand the receptive field of the image. Finally, the back generator network is optimized by a loss function to predict the background-free character back image.

[0025] In the present invention, the back generator network then adopts a multi-scale discriminator; it is convenient to expand the receptive field of the image; the real pictures of the back of the characters in the training set and the synthetic pictures generated by the generator are down-sampled accordingly, and two discriminants will be matched respectively for two images with different resolutions; according to the size of the input picture, the receptive field is reduced from large to small, among which the generator with the largest scale has the largest receptive field for the image, which is intended to provide the generator with a global perspective, and the generator with a smaller scale is used to add more details to the image; the output effect of the generator's back image of the character can be optimized without consuming a lot of training time.

[0026] Preferably, in step 2, since there is no constraint in estimating the spatial pose of the character topological structure through a neural network, the present invention uses the SMPL naked character model to reduce spatial structure ambiguity and standardize the prediction of the character surface range; and uses the PyMAFI61 method to estimate the SMPL model, which can complete parameterized naked character surface reconstruction from a single image and accurately restore the character posture in the input image; the normal map reflects the light and shadow changes brought about by the global illumination of the space on the model, and its sharpening indicates that the detail information in the picture is magnified, so introducing the normal map corresponding to the input image helps to supplement the details in the reconstruction process; obtaining the normal map of the dressed character corresponding to the input image with the help of an image translation network is a more intuitive idea for generating the normal map, but this method is to learn features pixel by pixel through the image, so a large amount of data is required for training and learning, and the high-resolution image generation effect is better; and the SMPL front normal map and the SMPL back normal map are directly obtained by the SMPL model; Preferably, in step 3, the SMPL front normal map, the SMPL back normal map and the background-free character front image are used for training the clothing front normal map and the clothing back normal map, so as to provide the image translation network with a priori knowledge of the character's limb posture structure and reduce the burden of pixel-by-pixel learning of the training network; the background-free character front image and the background-free character back image are respectively combined with images of two corresponding angles of the SMPL model to obtain the clothing front normal map and the clothing back normal map, and the prediction process involves an optimization process of the normal map and the SMPL model.

[0027] In the present invention, the normal map reflects the light and shadow changes brought about by the global illumination of the space on the model, and its sharpening indicates that the detail information in the picture is magnified. Therefore, introducing the normal map corresponding to the input image helps to supplement the details in the reconstruction process; using the image translation network to obtain the normal map of the clothed human body corresponding to the input image is a more intuitive idea for generating the normal map, but this method learns features pixel by pixel through the image, so a large amount of data is required for training and learning, and the effect of generating high-resolution images is better; using the SMPL-mesh naked human body normal map and RGB clothed human body map obtained in this experiment for the training of the clothed normal map, the image translation network is provided with a priori of the posture structure of the human limbs, reducing the burden of pixel-by-pixel learning of the training network.

[0028] Preferably, the convolutional neural network used in step 4 is a trained implicit expression network. In the present invention, the implicit expression network inputs seven-dimensional features based on the front and back sides. The present invention uses the SMPL model obtained above and the front and back normal maps of the clothed body to reconstruct the implicit surface of the clothed body based on the image features. Finally, an accurate three-dimensional model of the character is output.

[0029] Preferably, the motion data input in step 6 is the character motion roaming, the three-dimensional motion data processing module is provided with a motion data storage library, and the motion data storage library stores a number of motion data. In the process of outputting frame-by-frame motion parameters according to the motion data in step 6, the OpenPose joint point detection algorithm is first used to obtain the character motion roaming, and the 2D position pixels of the character joints in each frame image are extracted and saved in JSON format; the character posture recognition based on the OpenPose architecture uses the VGG network model; the VGG network model is a convolutional neural network of OpenPose, which has multiple layers and is also a popular network architecture, and many of its variants have excellent performance. The model uses the first 10 layers to process the input image or video stream to obtain features; then, the OpenPose architecture uses a convolutional neural network to obtain the degree of association between different character parts and the confidence map of different key points of the character; the 2D key points of the character in the image are obtained by parsing the relevant key points and confidence, and finally the motion parameters containing motion skeleton information are generated according to the 2D key point information of each frame, and each frame contains a character skeleton feature information.

[0030] Preferably, in step 7, the model adjustment module performs quantitative and qualitative adjustments on the character's three-dimensional model. Quantitatively adjusting the character's three-dimensional model directly controls the shape of the entire body of the character's three-dimensional model through numerical parameters, and controls the corresponding shape through specific numerical values; quantitatively adjusting the character's three-dimensional model makes the modeled character's three-dimensional model fit the model in the motion parameters.

[0031] In quantitative adjustment, this paper uses deep learning to obtain SMPL body data for character feature description, and uses a small number of parameters to control the body shape of the character model. There is no one-to-one mapping relationship between the parameters and a specific feature of the character. The SMPL parameters can be obtained by means of deep learning, mainly output by the previous character morphology parameter prediction subsystem. The parameters are abstractly described based on the whole body posture, and are no longer specific to a single clear feature such as height or weight.

[0032] In the qualitative adjustment of the character's three-dimensional model, the parameters correspond to the specific features of the model and are observable; qualitative adjustment is the final adjustment of the character's three-dimensional model based on quantitative adjustment, so that the virtual character obtained by modeling coincides with the model in the motion parameters.

[0033] In qualitative adjustment, this paper designs an interface for manual adjustment and setting. On the one hand, by visually dragging and adjusting a certain body parameter of the character, increase or decrease operations are performed to qualitatively adjust the virtual character's body shape, including adjusting the height of the virtual character model with the parameter Height, and adjusting the weight of the virtual character model with the parameter Weight. On the other hand, the characteristic parameter value is also used to adjust whether the virtual character model has significant personal features such as "general belly" and "protruding ears", so that the virtual character can be portrayed more accurately through significant features.

[0034] Preferably, the step 8 specifically includes the following steps: Step 801: Calculate driving data; The mathematical principle of skeletal drive is coordinate transformation. The drive of bones in space includes rotating bones and translating bones, so that the essence of its movement is the rotation and translation transformation of the object coordinate system, which is based on the mathematical operation of the transformation matrix; therefore, the first step of skeletal animation control is data preparation and calculation, including initial frame posture data and current frame transformation data. The rotation and translation transformation of the current frame is superimposed on the initial initial posture to generate the driving data of the current frame posture; Step 802: driving the skeleton layer by layer; In each frame, when the data drives the bones to rotate and translate, the tree-like hierarchical relationship of the bones is utilized, and a layer-by-layer driving method is adopted to ensure that all parent node joints are driven before child node joints; each transformation corresponds to the control of a joint, and the combination of transformations of a single part completes the posture transformation of the whole body; when the actions of all frames are played continuously, a dynamic three-dimensional animation can be reconstructed.

[0035] Preferably, the step 9 specifically includes adjusting the observation angle and the playback speed; Observation angle adjustment: The position and angle of the space camera can be adjusted through the adjustment buttons on the UI interface, thereby adjusting the space observation angle and adjusting the observation angle of the reconstructed animation; the variable observation angle enables multi-angle observation of the virtual character; and the fixed step size adjustment enables all-round observation of the virtual character from the front, back, left and right. Playback speed adjustment: Different playback speeds can achieve low-speed, medium-speed and high-speed playback of reconstructed videos to meet the needs of different motion analysis scenarios; the frame-by-frame rendering speed can be adjusted by buttons, the frame update frequency can be controlled, and the playback speed can be adjusted to ensure variable-speed playback of reconstructed roaming.

[0036] A 3D modeling control system based on animation design, including an animation modeling subsystem and an animation action driving subsystem; The animation modeling subsystem is provided with a three-dimensional modeling model, which completes automatic three-dimensional modeling by acquiring a single front image of a character to obtain a three-dimensional model of the character, and sends the three-dimensional model of the character to the animation action driving subsystem; The animation modeling subsystem is provided with a character three-dimensional model database, and the character three-dimensional model database is used to store an initial single character frontal image and a corresponding character three-dimensional model; The animation action driving subsystem includes a three-dimensional action data processing module, a model adjustment module, a skeleton skinning animation driving module, and a rendering and display module. The three-dimensional action data processing module is used to output the final motion parameters according to the input motion data. The model adjustment module obtains the three-dimensional model of the character generated by the animation modeling subsystem, and makes quantitative and qualitative adjustments to the three-dimensional model of the character, so that the three-dimensional model of the character coincides with the model in the motion parameters; the skeleton skinning animation driving module obtains the motion parameters output by the three-dimensional action data processing module, the three-dimensional model of the character output by the animation modeling subsystem, and the skeleton structure characteristics of the three-dimensional model of the character, and then performs frame-by-frame motion parameter adaptation, calculates driving data, and performs layer-by-layer driving skeleton operation to generate a preliminary character motion roaming; the rendering and display module is used to render and display the preliminary character motion roaming to obtain the final character motion roaming; A motion data storage library is provided inside the three-dimensional motion data processing module, and the motion data storage library stores a number of motion data and corresponding motion parameters including motion skeleton information.

[0037] In the present invention, by constructing an animation action driving subsystem, the present invention can quickly perform three-dimensional character modeling based on a single frontal image, and simultaneously obtain motion parameters based on motion data, and randomly match several character three-dimensional models with several motion parameters to directly and quickly obtain three-dimensional animation roaming of multiple characters in different motion states.

[0038] The above is only a preferred embodiment of the present invention. It should be noted that, for those skilled in the art, several improvements and modifications can be made without departing from the principles of the present invention, and these improvements and modifications should also be considered as the protection scope of the present invention. The structures, devices and operating methods not specifically described and explained in the present invention shall be implemented according to the conventional means in the art unless otherwise specified and limited.

Claims

1. A 3D modeling method based on animation design, characterized in that: The following steps are involved: Step 1: Build an animation modeling subsystem, obtain the front image of the character, then remove the background to obtain the front image of the character without background, send the front image of the character without background to the back convolution prediction generation network, and predict the back image of the character without background; Step 2: construct a 3D model, and then use the 3D model to generate an SMPL model based on the front image of the character without background, and adapt the front image of the character without background and the back image of the character without background based on the SMPL model to generate an SMPL front normal map and an SMPL back normal map; Step 3: Generate the clothing SMPL front normal map and the clothing SMPL back normal map through the convolutional neural network respectively according to the SMPL front normal map and the SMPL back normal map, and then reversely adjust the SMPL model through the clothing SMPL front normal map and the clothing SMPL back normal map; Step 4: The SMPL model, the clothing SMPL front normal map, and the clothing SMPL back normal map are predicted by a convolutional neural network to output the character's three-dimensional model; Step 5: construct an animation action driving subsystem, which includes a three-dimensional action data processing module, a model adjustment module, a skeleton skinning animation driving module, and a rendering display module; Step 6: The three-dimensional motion data processing module uses the OpenPose joint point detection algorithm to obtain the two-dimensional joint point data of the character according to the input motion data, and uses the obtained two-dimensional joint point data as the training data of the 3D character posture estimation neural network system to obtain the 3D character motion data stream, and finally outputs the motion parameters, and sends the motion parameters to the skeleton skinning animation driving module; Step 7: The model adjustment module obtains the character 3D model outputted in step 4, completes preprocessing of the character 3D model, and obtains the bone structure features of the character 3D model; Step 8: After the skeleton skinning animation driving module obtains the motion parameters outputted in step 6, the character 3D model outputted in step 4, and the skeleton structure features of the character 3D model, it performs frame-by-frame motion parameter adaptation, calculates driving data, and performs layer-by-layer driving skeleton operation to generate preliminary character motion roaming; Step 9: The rendering display module performs cloud rendering on the preliminary motion video to obtain the final character motion roaming.

2. A 3D modeling method based on animation design according to claim 1, characterized in that: In the step 1, the background is removed by a digital image segmentation processing method, and the back convolutional generation network adopts a character back translation based on Pix2PixHD. Specifically, the back generator network uses a global generator G, and the global generator G first downsamples the input image, extracts image features in the convolution layer, obtains global information, and then inputs the obtained features into the residual network, and then restores the features to the same original size as the input background-free character front image through the upsampling convolution layer. Then the back generator network adopts a multi-scale discriminator to expand the receptive field of the image. Finally, the back generator network is optimized by a loss function to predict the background-free character back image.

3. The 3D modeling method based on animation design according to claim 1, characterized in that: In step 2, the SMPL naked character model is used to reduce spatial structure ambiguity and standardize the prediction of the character surface range; The PyMAFI61 method is used to estimate the SMPL model, and the SMPL front normal map and the SMPL back normal map are directly obtained based on the SMPL model.

4. The 3D modeling method based on animation design according to claim 2, characterized in that: In the step 3, the clothing front normal map and the clothing back normal map are trained based on the SMPL front normal map, the SMPL back normal map and the background-free character front image. The background-free character front image and the background-free character back image are respectively combined with images of two corresponding angles of the SMPL model to obtain the clothing front normal map and the clothing back normal map. The prediction process involves an optimization process of the normal map and the SMPL model.

5. The 3D modeling method based on animation design according to claim 1, characterized in that: The convolutional neural network used in step 4 is a trained implicit expression network.

6. The 3D modeling method based on animation design according to claim 1, characterized in that: The motion data input in step 6 is the character motion roaming, and the three-dimensional motion data processing module is provided with a motion data storage library, in which a number of motion data are stored. In the process of outputting frame-by-frame motion parameters according to the motion data in step 6, the OpenPose joint point detection algorithm is firstly used to obtain the character motion roaming, and the 2D position pixels of the character joints in each frame image are extracted and saved in JSON format; the character posture recognition based on the OpenPose architecture uses the VGG network model; then, the OpenPose architecture uses a convolutional neural network to obtain the degree of association between different character parts and the confidence map of different key points of the character; the 2D key points of the character existing in the image are obtained by parsing the relevant key points and confidence, and finally the motion parameters containing the motion skeleton information are generated according to the 2D key point information of each frame.

7. The 3D modeling method based on animation design according to claim 1, characterized in that: In step 7, the model adjustment module performs quantitative and qualitative adjustments on the character's three-dimensional model. Quantitatively adjust the character's 3D model to directly control the shape of the entire character's 3D model through numerical parameters, and control the corresponding shape through specific numerical values; Quantitatively adjust the character 3D model so that the character 3D model obtained by modeling fits the model in the motion parameters; In the qualitative adjustment of the character's three-dimensional model, the parameters correspond to the specific features of the model and are observable; qualitative adjustment is the final adjustment of the character's three-dimensional model based on quantitative adjustment, so that the virtual character obtained by modeling coincides with the model in the motion parameters.

8. The 3D modeling method based on animation design according to claim 1, characterized in that: The step 8 specifically includes the following steps: Step 801: Calculate driving data; The first step of skeletal animation control is data preparation and calculation, including initial frame pose data and current frame transformation data; the rotation and translation transformation of the current frame are superimposed on the initial pose to generate the driving data of the current frame pose; Step 802: driving the skeleton layer by layer; In each frame, when the data drives the bones to rotate and translate, the tree-like hierarchical relationship of the bones is utilized, and a layer-by-layer driving method is adopted to ensure that all parent node joints are driven before child node joints; each transformation corresponds to the control of a joint, and the combination of transformations of a single part completes the posture transformation of the whole body; when the actions of all frames are played continuously, a dynamic three-dimensional animation can be reconstructed.

9. The 3D modeling method based on animation design according to claim 1, characterized in that: The step 9 specifically includes adjusting the observation angle and the playback speed; Observation angle adjustment: Use the adjustment button to adjust the position and angle of the space camera, thereby adjusting the space observation angle and adjusting the observation angle of the reconstructed animation; Playback speed adjustment: Adjust the frame-by-frame rendering speed through the buttons, control the frequency of frame updates, and adjust the playback speed.

10. A 3D modeling control system based on animation design, characterized in that: Including animation modeling subsystem and animation action driving subsystem; The animation modeling subsystem is provided with a three-dimensional modeling model, which completes automatic three-dimensional modeling by acquiring a single front image of a character to obtain a three-dimensional model of the character, and sends the three-dimensional model of the character to the animation action driving subsystem; The animation modeling subsystem is provided with a character three-dimensional model database, and the character three-dimensional model database is used to store an initial single character frontal image and a corresponding character three-dimensional model; The animation action driving subsystem includes a three-dimensional action data processing module, a model adjustment module, a skeleton skinning animation driving module, and a rendering and display module. The three-dimensional action data processing module is used to output the final motion parameters according to the input motion data. The model adjustment module obtains the three-dimensional model of the character generated by the animation modeling subsystem, and makes quantitative and qualitative adjustments to the three-dimensional model of the character, so that the three-dimensional model of the character coincides with the model in the motion parameters; the skeleton skinning animation driving module obtains the motion parameters output by the three-dimensional action data processing module, the three-dimensional model of the character output by the animation modeling subsystem, and the skeleton structure characteristics of the three-dimensional model of the character, and then performs frame-by-frame motion parameter adaptation, calculates driving data, and performs layer-by-layer driving skeleton operation to generate a preliminary character motion roaming; the rendering and display module is used to render and display the preliminary character motion roaming to obtain the final character motion roaming; A motion data storage library is provided inside the three-dimensional motion data processing module, and the motion data storage library stores a number of motion data and corresponding motion parameters including motion skeleton information.

Images