Cartoon role model construction method

Abstract

The invention relates to the technical field of computer vision, and discloses a cartoon role model construction method which is used for extracting and modeling the shape and posture of a role. According to the method, the problems of low precision of constructing the cartoon role 3D model and the like in the prior art are solved.

Description

Technical Field

[0001] The invention relates to the technical field of computer vision, and in particular to a method for constructing a cartoon character model. Background Art

[0002] Creating 3D models of cartoon characters is widely used in many fields, including video games, interactive media and animation production. Significant progress has been made in the field of digital modeling, especially in the direction of human body modeling. Researchers have explored digital models of the human body, which can be used to quickly model different human characters through parameterization. A large amount of data has been obtained through virtual environment modeling and real human body scanning, which can be used for training deep models. Current technical solutions mainly focus on two aspects: parametric modeling of the human body and modeling with clothing. The former realizes modeling by estimating the shape and posture parameters of the digital model of the human body, while the latter focuses on modeling the human body with clothing.

[0003] However, fast 3D modeling of cartoon characters has not been fully studied. Existing high-quality 3D models rely heavily on manual work by skilled character designers, a process that is time-consuming and labor-intensive. Deep learning-related technologies have good generalization capabilities after learning massive amounts of data, which helps speed up this process. However, cartoon characters are rich in diversity compared to real people, which has a significant impact on related tasks, making the learning process difficult. Summary of the invention

[0004] In order to overcome the deficiencies of the prior art, the present invention provides a method for constructing a cartoon character model, which solves the problems of low precision in constructing a 3D model of a cartoon character in the prior art.

[0005] The technical solution adopted by the present invention to solve the above problems is:

[0006] A method for constructing a cartoon character model extracts and models the shape and posture of the character respectively.

[0007] As a preferred technical solution, the following steps are included:

[0008] S1, feature extraction: extracting the shape and posture features of the character;

[0009] S2, feature consistency constraint: feature consistency constraints are imposed on shape features and posture features;

[0010] S3, feature fusion and loop feedback: feature fusion of shape features and posture features, and the use of multiple iterations to gradually fit the initial posture features to the target posture features to achieve geometrically guided loop feedback;

[0011] S4, decoding: decoding the shape features and posture features after feature fusion, and decoding the posture features after geometrically guided loop feedback.

[0012] As a preferred technical solution, in step S2, the calculation formula of the feature consistency constraint is:

[0013]

[0014] In the formula, I represents the input image, I l represents the image obtained after data augmentation, B(·) represents the extraction operation of the encoder on the image features, L represents the number of different postures or different shapes, l represents the number of different postures or different shapes, and ||·||2 represents the L2 distance calculation.

[0015] As a preferred technical solution, in step S3, when fusing features, the orthogonal decoupled features are concatenated with the global features, and the fusion of the orthogonal decoupled features and the global features is achieved through a multi-layer perceptron.

[0016] As a preferred technical solution, when obtaining orthogonal decoupled features, the self-attention mechanism is used to model the association between different regional blocks of local features. The formula is:

[0017] F l ′ =σ(f(F l )·F l )

[0018] In the formula, σ(·) represents the Softplus activation function, f(·) represents the convolution calculation with a convolution kernel of 1, and F l Represents the local features of the input, F l ′ Represents the local features after transformation.

[0019] As a preferred technical solution, the expression of the orthogonal decoupling feature of the local feature relative to the global feature is:

[0020]

[0021] In the formula, F g represents the global feature, F l represents local features, A(·) represents average pooling calculation, Represents the orthogonal decoupling feature.

[0022] As a preferred technical solution, in step S4, the decoder models the relationship between geometric features and image features, and achieves semantic alignment of image features and geometric features by adjusting parameters of the cartoon character model.

[0023] As a preferred technical solution, in step S4, the modeling is implemented as follows:

[0024] θ i+1 =θ i +D(F img ,F geo )

[0025] In the formula, θ i+1 represents the iteration result of the i+1th model parameter, θ i represents the iteration result of the i-th model parameter, D represents the decoder input, and F img represents the image features, F geo Represents geometric features.

[0026] As a preferred technical solution, in step S1, the order of the jigsaw pieces in the input image is disrupted and then the shape features and posture features of the character are extracted.

[0027] As a preferred technical solution, the method further comprises the following steps:

[0028] S5, 3D supervision: Use supervision information to supervise the extraction of shape features and posture features.

[0029] Compared with the prior art, the present invention has the following beneficial effects:

[0030] (1) The present invention proposes a 3D modeling scheme for cartoon characters based on a dual-branch structure, which uses a feature decoupled modeling method to improve the consistent expression of the modeling results and the input image;

[0031] (2) The present invention can achieve rapid reconstruction of images into 3D models by regressing the shape and posture parameters of cartoon character models, and improve the distinction between different character types by combining modeling of local detail features;

[0032] (3) The present invention proposes a geometric information-guided loop feedback strategy, which can effectively align the modeling results with the 2D input image and improve the accuracy of character posture modeling. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 A schematic diagram of the modeling process framework. DETAILED DESCRIPTION

[0034] The present invention will be further described in detail below in conjunction with embodiments and drawings, but the embodiments of the present invention are not limited thereto.

[0035] Example 1

[0036] like Figure 1As shown, in order to realize the rapid reconstruction of the 3D model of cartoon characters, the present invention proposes a dual-branch structure neural network model. This architecture separates the modeling process of posture and shape, and combines feature consistency constraints to enhance the backbone network (i.e., encoder) to capture the expression ability of the corresponding task-related features.

[0037] In order to solve the impact of significant changes in the appearance of cartoon characters, the present invention proposes a fusion mechanism to generate a comprehensive expression of the input object, including combining the local features extracted after disrupting the input image blocks with the global features extracted from the original image.

[0038] In order to solve the complexity of cartoon character posture changes, the present invention proposes a geometry-guided feedback loop strategy. This strategy iteratively compares the 3D modeling results with the 2D input image in the feature dimension, reducing the difference between the two to achieve semantic alignment.

[0039] The present invention proposes a novel cartoon character mesh modeling scheme, which independently learns the features corresponding to different tasks by means of feature decoupling. The method uses two sets of encoders and decoders to model the posture parameters and shape parameters respectively. Considering that the differences of cartoon characters mainly exist in the ears, mouths, hands and feet. A jigsaw enhancement technique is proposed, which makes the model focus on the local details of the character by disrupting the order of jigsaw pieces in the image. In addition, a hybrid fusion module is proposed to combine the global features extracted from the original image with the local features extracted from the jigsaw processed image. Considering the significant differences in posture in cartoon images, a feedback loop strategy guided by geometric information is designed. The difference between geometric information and image information in the feature space is gradually reduced in an iterative manner, and the consistency between modeling and image in posture expression is achieved. In addition, a feature consistency loss is introduced to improve the ability of the model to express the features of related tasks.

[0040] The present invention proposes a cartoon character 3D modeling scheme based on a dual-branch structure, which utilizes a feature decoupled modeling method to improve the consistent expression of the modeling result and the input image.

[0041] The present invention can realize the rapid reconstruction of the image to the 3D model by regressing the shape and posture parameters of the cartoon character model, and improve the distinction between different character types by combining the modeling of local detail features.

[0042] The present invention proposes a geometric information-guided loop feedback strategy, which can effectively align the modeling results with the 2D input image and improve the accuracy of character posture modeling.

[0043] Example 2

[0044] like Figure 1As shown, as a further optimization of Example 1, based on Example 1, this embodiment also includes the following technical features:

[0045] The goal of this solution is to model the shape and posture of a given cartoon animal character. Figure 1 shown.

[0046] 1. Feature Decoupling Implementation

[0047] The present invention defines that given an input image, the model outputs rotation information of 23 skeleton key points and shape information with a vector dimension of 100.

[0048] The main network is a dual-branch structure, and different branches contain a set of corresponding encoders and decoders to model the shape and posture of the character respectively. The supervision information of different tasks is used at the end of the structure to guide the learning of the features of each branch. In order to further expand the difference between the two, a feature consistency constraint is proposed. The posture of the input model is randomly changed to produce different postures of the same character. The posture data can be obtained from the human body model data. At the same time, the posture of the input model is retained, and random shape parameters are obtained from the data set to obtain different character expressions with the same posture. Then, through the feature extraction of the backbone network, the features of different postures and different shapes are obtained respectively.

[0049] The feature consistency loss calculates the Euclidean distance between different features to constrain the feature extractors of different branches to extract task-related features. The calculation process is expressed as:

[0050]

[0051] in, is the feature consistency constraint loss, I is the input image, and I l is the image obtained after data augmentation, B(·) is the image feature extracted by the backbone network. L represents the number of different postures or shapes, l represents the number of different postures or shapes, and ||·||2 calculates the L2 distance between the two.

[0052] 2. Hybrid Feature Fusion Module

[0053] Cartoon characters are distinguished mainly by local details, such as ears, mouths, and feet. The present invention will simultaneously model the global and local features of the target character. The input image is divided into n×n blocks, and the order of the blocks is randomly disrupted before feature extraction. Since the overall information of the target is destroyed, the network is forced to obtain the representation of the corresponding task through the local details of the input.

[0054] A feature fusion method of orthogonal decomposition is proposed to filter redundant information in local features. First, the self-attention mechanism is used to model the association between different regional blocks of local features:

[0055] F l ′=σ(f(F l )·F l )

[0056] Where σ(·) is the Softplus activation function, f(·) is the convolution calculation with a convolution kernel of 1, and F l Represents the local features of the input, F l ′ is the local feature after transformation.

[0057] The orthogonal decomposition features of local features relative to global features can be expressed as:

[0058]

[0059] Among them, F g and F l Represent global and local features respectively, and A(·) is the average pooling calculation. Orthogonal decoupled features The concatenated features are then concatenated with the global features and passed through a multi-layer perceptron (MLP) to fuse the two.

[0060] 3. Geometry-guided feedback loop module

[0061] Compared with humans, cartoon characters have more complex and diverse movements, so a multiple iteration method is designed to make an initial pose model gradually fit the target pose. The present invention considers representing the degree of fit from the feature dimension, including image feature space and geometric feature space.

[0062] Formally, the model parameters of the i-th modeling are input into RaBit (digital unified model) to obtain the grid model expressed by its corresponding parameters. The number of grids exceeds 30,000. It is downsampled to obtain 606 3D key points, and the geometric features are extracted through PointNet (3D backbone network). The decoder models the relationship between geometric features and image features to adjust the parameters of the model and achieve semantic alignment of image features and geometric features. Its implementation method is:

[0063] θ i+1 =θ i +D(F img , F geo )

[0064] where θ i is the iteration result of the i-th model parameter, D is the decoder input including the image feature F img and geometric features Fgeo .

[0065] By adopting the technical solution of the present invention, the following technical effects are achieved:

[0066] We validated the proposed model on the public 3DBiCar dataset. The proposed model uniformly crops the input image to a resolution of 224 and uses ResNet-50 as the backbone network. A total of 250 rounds of training were performed, and the learning rate was set to 5*e -5 , the number of loop iterations is set to 3. From the comparison results in Table 1, it can be found that our proposed scheme surpasses all previous methods and obtains the best performance, which proves the effectiveness of the proposed scheme in solving the task of single image modeling cartoon 3D model.

[0067] Table 1 Comparison of experimental results of 3DBiCar dataset

[0068] Comparison Method MPVE↓(mm) MPJPE↓(mm) PA-MPJPE↓(mm) DecoMR 85.7 81.2 47.2 Mesh-Transformer 62.5 49.6 30.7 HMR 65.4 51.3 30.9 Our Approach 61.6 48.1 30.3

[0069] As described above, the present invention can be preferably implemented.

[0070] All features disclosed in all embodiments in this specification, or steps in all methods or processes implicitly disclosed, except for mutually exclusive features and / or steps, can be combined and / or expanded or replaced in any manner.

[0071] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. According to the technical essence of the present invention, within the spirit and principles of the present invention, any simple modification, equivalent replacement and improvement made to the above embodiment still falls within the protection scope of the technical solution of the present invention.

Claims

1. A method for constructing a cartoon character model, characterized in that the shape and posture of the character are extracted and modeled respectively.

2. A method for constructing a cartoon character model according to claim 1, characterized in that: The following steps are involved: S1, feature extraction: extracting the shape and posture features of the character; S2, feature consistency constraint: feature consistency constraints are imposed on shape features and posture features; S3, feature fusion and loop feedback: feature fusion of shape features and posture features, and the use of multiple iterations to gradually fit the initial posture features to the target posture features to achieve geometrically guided loop feedback; S4, decoding: decoding the shape features and posture features after feature fusion, and decoding the posture features after geometrically guided loop feedback.

3. A method for constructing a cartoon character model according to claim 2, characterized in that: In step S2, the calculation formula of the feature consistency constraint is: In the formula, I represents the input image, I l represents the image obtained after data augmentation, B(·) represents the extraction operation of the encoder on the image features, L represents the number of different postures or different shapes, l represents the number of different postures or different shapes, and ||·||2 represents the L2 distance calculation.

4. A method for constructing a cartoon character model according to claim 3, characterized in that: In step S3, when the features are fused, the orthogonal decoupled features are concatenated with the global features, and the fusion of the orthogonal decoupled features and the global features is achieved through a multi-layer perceptron.

5. A method for constructing a cartoon character model according to claim 4, characterized in that: When obtaining orthogonal decoupled features, the self-attention mechanism is used to model the association between different regional blocks of local features. The formula is: F′ l =σ(f(F l )·F l ) In the formula, σ(·) represents the Softplus activation function, f(·) represents the convolution calculation with a convolution kernel of 1, and F l represents the local features of the input, F l ′ Represents the local features after transformation.

6. A method for constructing a cartoon character model according to claim 5, characterized in that: The expression of the orthogonal decoupling feature of local features relative to global features is: In the formula, F g represents the global feature, F l represents local features, A(·) represents average pooling calculation, Represents the orthogonal decoupling feature.

7. A method for constructing a cartoon character model according to claim 2, characterized in that: In step S4, the decoder models the relationship between geometric features and image features, and achieves semantic alignment of image features and geometric features by adjusting parameters of the cartoon character model.

8. A method for constructing a cartoon character model according to claim 7, characterized in that: In step S4, the modeling is implemented as follows: i i+1 =θ i +D(F img ,F geo ) In the formula, θ i+1 represents the iteration result of the i+1th model parameter, θ i represents the iteration result of the i-th model parameter, D represents the decoder input, and F img represents the image features, F geo Represents geometric features.

9. A method for constructing a cartoon character model according to claim 2, characterized in that: In step S1, the order of the jigsaw pieces in the input image is disrupted and then the shape features and posture features of the character are extracted.

10. A method for constructing a cartoon character model according to any one of claims 2 to 9, characterized in that: The following steps are also included: S5, 3D supervision: Use supervision information to supervise the extraction of shape features and posture features.

Images