A method, apparatus, device, storage medium, and product for processing the face of a virtual character.

Abstract

This application provides a method, apparatus, device, storage medium, and product for processing virtual character faces. The technical solution provided in this application acquires a two-dimensional expressionless face image, determines the facial deformation parameters of the two-dimensional expressionless face image using a deformation parameter model, and generates a virtual face model based on the facial deformation parameters and a set standard face model. Users only need to upload a two-dimensional expressionless face image, and the three-dimensional deformable standard face model can be adjusted according to the facial deformation parameters to obtain a virtual face model reflecting the facial features in the two-dimensional expressionless face image. This simplifies the virtual character face-shaping operation while improving the virtual character face processing effect.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method, apparatus, device, storage medium, and product for processing virtual character faces. Background Technology

[0002] With the development of image processing technology, users often use virtual characters with facial features similar to themselves to interact with other users. Users generally generate virtual characters with facial features similar to themselves by sculpting a standard face model.

[0003] The traditional method of character creation involves using drag-and-drop controls on a standard human face model to adjust its features, generating the desired virtual avatar. However, this process is complex and cumbersome, generally requiring skilled technicians to achieve satisfactory results. Ordinary users often struggle to create the desired virtual avatars, resulting in poor facial rendering. Summary of the Invention

[0004] This application provides a method, apparatus, device, storage medium, and product for processing virtual character faces, in order to solve the technical problems of complex and cumbersome operation of virtual character face processing and poor face processing effect in related technologies, simplify face processing operation, and improve the face processing effect of virtual characters.

[0005] In a first aspect, embodiments of this application provide a method for processing the face of a virtual character, including:

[0006] Obtain a two-dimensional, expressionless human face image;

[0007] The two-dimensional expressionless face image is input into a trained deformation parameter model. The deformation parameter model is used to analyze and process the two-dimensional expressionless face image to obtain facial deformation parameters that reflect the facial features in the two-dimensional expressionless face image.

[0008] A virtual face model is generated based on the face deformation parameters and the set standard face model, wherein the standard face model is a three-dimensional deformable face model.

[0009] In a second aspect, embodiments of this application provide a virtual character face processing device, including an image acquisition module, a parameter acquisition module, and a face generation module, wherein:

[0010] The image acquisition module is configured to acquire a two-dimensional expressionless human face image;

[0011] The parameter acquisition module is configured to input the two-dimensional expressionless face image into a trained deformation parameter model, and analyze and process the two-dimensional expressionless face image through the deformation parameter model to obtain face deformation parameters that reflect the facial features in the two-dimensional expressionless face image.

[0012] The face generation module is configured to generate a virtual face model based on the face deformation parameters and a set standard face model, wherein the standard face model is a three-dimensional deformable face model.

[0013] In a third aspect, embodiments of this application provide a virtual character face processing device, including: a memory and one or more processors;

[0014] The memory is used to store one or more programs;

[0015] When the one or more programs are executed by the one or more processors, the one or more processors implement the virtual character face processing method as described in the first aspect.

[0016] In a fourth aspect, embodiments of this application provide a non-volatile storage medium for storing computer-executable instructions, which, when executed by a computer processor, are used to perform the virtual character face processing method as described in the first aspect.

[0017] In a fifth aspect, embodiments of this application provide a computer program product comprising a computer program stored in a computer-readable storage medium, wherein at least one processor of the device reads from the computer-readable storage medium and executes the computer program, causing the device to perform the virtual character face processing method as described in the first aspect.

[0018] This application embodiment acquires a two-dimensional expressionless face image, uses a deformation parameter model to determine the facial deformation parameters of the two-dimensional expressionless face image, and generates a virtual face model based on the facial deformation parameters and a set standard face model. Users only need to upload a two-dimensional expressionless face image, and the three-dimensional deformable standard face model can be adjusted according to the facial deformation parameters to obtain a virtual face model that reflects the facial features in the two-dimensional expressionless face image. This simplifies the virtual character face shaping operation and improves the virtual character face processing effect. Attached Figure Description

[0019] Figure 1 This is a flowchart of a virtual character face processing method provided in an embodiment of this application;

[0020] Figure 2 This is a schematic diagram of a registration process for a set of key points provided in an embodiment of this application;

[0021] Figure 3 This is a schematic diagram of a registration result provided in an embodiment of this application;

[0022] Figure 4 This is a schematic diagram of the structure of a virtual character face processing device provided in an embodiment of this application;

[0023] Figure 5 This is a schematic diagram of the structure of a virtual character face processing device provided in an embodiment of this application. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this application clearer, specific embodiments of this application will be described in further detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are merely for explaining this application and not for limiting it. It should also be noted that, for ease of description, only the parts relevant to this application are shown in the drawings, not all of them. Before discussing exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe operations (or steps) as sequential processes, many of these operations can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the operations can be rearranged. The process can be terminated when its operation is completed, but additional steps not included in the drawings may also be present. The above processes can correspond to methods, functions, procedures, subroutines, subroutines, etc.

[0025] The virtual character face processing method provided in this application can be applied to face-shaping scenarios for virtual characters in live streaming, games, etc. For example, in a live streaming scenario, when a streamer wants to create a virtual character with their own characteristics to interact with the audience, the streamer can automatically generate a virtual face model by providing a two-dimensional expressionless face image. The aim is to determine the facial deformation parameters of the two-dimensional expressionless face image through a deformation parameter model, and generate a virtual face model based on the facial deformation parameters and a set standard face model, simplifying face processing operations and improving the face processing effect of the virtual character. Traditional virtual character face processing solutions generally involve users adjusting the facial features of a standard face model using drag bars on a face-shaping control panel to generate the user's desired virtual character, or professionals (such as professional face sculptors) generating the virtual character through lengthy face-shaping operations. These methods suffer from complex and cumbersome face processing operations and poor face processing effects. Therefore, this application provides a virtual character face processing method to solve the technical problems of complex and cumbersome operations and poor face processing effects in existing virtual character face processing solutions.

[0026] Figure 1A flowchart of a virtual character face processing method provided in an embodiment of this application is given. The virtual character face processing method provided in this embodiment of the application can be executed by a virtual character face processing device, which can be implemented by hardware and / or software and integrated into a virtual character face processing device.

[0027] The following description uses a virtual character face processing device to perform a virtual character face processing method as an example. (Reference) Figure 1 The method for processing the virtual character's face includes:

[0028] S101: Obtain a two-dimensional, expressionless human face image.

[0029] The two-dimensional expressionless face image provided by this solution is a frontal, expressionless two-dimensional face image provided to users who need to create virtual character faces. The two-dimensional expressionless face image can be obtained by taking a picture with a camera device, or by downloading it from a download address specified by the user, or by obtaining it from a local storage location (such as a photo album).

[0030] For example, a two-dimensional, expressionless facial image provided by the user can be obtained. For instance, when a broadcaster needs to create a virtual face model, they can use a virtual character face processing device (such as a mobile phone or computer) to capture a frontal, expressionless image of their own face as a two-dimensional, expressionless facial image. Subsequently, a virtual face model reflecting the facial features in the user-provided two-dimensional, expressionless facial image will be generated.

[0031] S102: Input the two-dimensional expressionless face image into the trained deformation parameter model, and analyze and process the two-dimensional expressionless face image through the deformation parameter model to obtain the face deformation parameters that reflect the facial features in the two-dimensional expressionless face image.

[0032] This solution pre-configures a trained deformation parameter model in the virtual character face processing device. This deformation parameter model can be used to analyze and process the provided two-dimensional expressionless face image and output face deformation parameters that reflect the facial features in the two-dimensional expressionless face image. The face deformation parameters can be used to define the shape basis vectors of a three-dimensional deformable face model (3DMM).

[0033] For example, the acquired two-dimensional expressionless face image is input into the deformation parameter model. After receiving the two-dimensional expressionless face image, the deformation parameter model will analyze and process the two-dimensional expressionless face image and output the corresponding face deformation parameters.

[0034] In one possible embodiment, the deformation parameter model provided by this solution can be trained based on two-dimensional expressionless face sample images and face sample deformation parameters, wherein the face sample deformation parameters are determined based on the registration results of the three-dimensional key point set of the face sample and the key point set of the virtual character.

[0035] The deformation parameter model provided in this solution can be built based on a convolutional neural network. The backbone network of the deformation parameter model can be built based on a ResNet18 network or a MobileNet network. At the same time, the last layer of the deformation parameter model is set to a sigmoid layer to ensure that the element values ​​of the face deformation parameters output by the deformation parameter model are between 0 and 1.

[0036] The face sample deformation parameters reflect the facial features of the sample user's face in a two-dimensional, expressionless face sample image. These parameters are determined based on the registration results of the corresponding sample user's three-dimensional keypoint set and the sample user's virtual character keypoint set. Optionally, the sample user's virtual character keypoint set is a set of three-dimensional keypoints from the face sample on the virtual character. The virtual character can be created by a professional face sculptor based on the sample user's facial features.

[0037] For example, multiple two-dimensional expressionless face sample images and three-dimensional key point sets of face samples are acquired using an image acquisition device. For each sample user, multiple two-dimensional expressionless face sample images are acquired under different shooting environments. Optionally, different shooting environments can be different combinations of lighting conditions, indoor and outdoor scenes, and shooting distances to improve the sample diversity of the two-dimensional expressionless face sample images. For example, 100 two-dimensional expressionless face sample images covering different lighting conditions, outdoor and indoor scenes, and different shooting distances are acquired for each sample user.

[0038] Each sample user corresponds to a set of three-dimensional key points for their face. This set of three-dimensional key points records a series of points on the sample user's face image, consisting of three-dimensional coordinates (i.e., the three-dimensional key points of the face sample). The set of three-dimensional key points for the face sample can be represented as: Where m is the number of key points in the set of 3D key points of the face sample, which can be determined based on the acquisition accuracy of the image acquisition device. It is understandable that... This can be understood as the 3D coordinates of the 3D key point of face sample number 1 in the set of 3D key points of face samples. .

[0039] In one embodiment, when acquiring two-dimensional expressionless facial sample images of multiple sample users using an image acquisition device, it is necessary to ensure that the sample users' faces are in an upright posture, their faces are not making any expressions, and that the image acquisition device captures the sample users' facial images from the front to obtain the two-dimensional expressionless facial sample images. Simultaneously, when capturing the two-dimensional expressionless facial sample images of the sample users, it is necessary to ensure that the sample users are within the maximum shooting range of the image acquisition device to guarantee the accuracy of the acquired three-dimensional facial key points.

[0040] Furthermore, the keypoint set of the virtual character for each sample user is obtained. It's understandable that the number of keypoints in the keypoint set of the virtual character generated by a professional face sculptor based on face-shaping operations may differ from the number of keypoints in the 3D keypoint set of the human face sample acquired by the image acquisition device. Therefore, registration processing is required between the 3D keypoint set of the human face sample and the keypoint set of the virtual character. Let's assume the keypoint set of the virtual character is represented as... , where n is the number of key points for the virtual character. It should be explained that the set of 3D key points for a human face sample acquired by the image acquisition device is relatively sparse compared to the set of key points for a virtual character. The number of key points in the set of 3D key points for a human face sample is on the order of hundreds or thousands, while the number of key points in the set of key points for a virtual character is generally over ten thousand, meaning n is greater than m.

[0041] The registration of the keypoint set can be understood as finding a transformation parameter T such that the keypoint set of the virtual character... After transformation With the set of 3D key points of face samples The overlap should be as high as possible, and the key points of the virtual character should be grouped together. Find the set of 3D key points that can match the face sample A set of key point numbers that correspond one-to-one .at this time, correspond , correspond , correspond The transformation parameter T corresponding to the registration result can include a scaling factor, a rotation matrix, and a translation vector.

[0042] For example, after registering the set of 3D key points of each sample user's face sample and the set of key points of the virtual character to obtain the registration result, the deformation parameters of each sample user's face sample are determined based on the registration result. In one embodiment, the deformation parameters of the face sample can be determined based on the corresponding transformation parameters and the set of 3D key points of the face sample. By changing the deformation parameters of the face sample, the virtual face key points in the final generated virtual face model, which correspond to the key point number set, can coincide with the 3D key points of the face sample in 3D space.

[0043] Furthermore, using collected 2D expressionless face sample images and face deformation parameters from multiple user samples as training data, a deformation parameter model is trained, and the trained model is then configured into the virtual character face processing device. For example, using 2D expressionless face sample images as input and the corresponding face deformation parameters as output, the deformation parameter model is trained until the set loss function requirements are met. This scheme determines the face sample deformation parameters based on the registration results of the 3D keypoint set of the face sample and the keypoint set of the virtual character, and trains the deformation parameter model using the collected 2D expressionless face sample images and face deformation parameters. The deformation parameter model can accurately analyze the face deformation parameters corresponding to the input 2D image, improving the face processing effect and efficiency of virtual character face sculpting.

[0044] In one embodiment, the loss function for training the deformation parameter model can be expressed as:

[0045]

[0046] in, This paper analyzes and processes two-dimensional expressionless face sample images using a deformation parameter model to predict the facial deformation parameters. These are the deformation parameters of the face sample corresponding to the two-dimensional expressionless face sample image. Optionally, the deformation parameter model can be continuously adjusted using the gradient descent method until the value of the loss function is lower than the set loss function threshold. After the loss function converges and stops, the deformation parameter model is saved and configured into the application corresponding to the virtual character face processing device.

[0047] In one possible embodiment, when registering the set of 3D key points of a face sample and the set of key points of a virtual character, this solution can be based on the following registration formula:

[0048]

[0049] in, The scaling factor that needs to be solved. Let be the rotation matrix that needs to be solved. Let be the translation vector that needs to be solved. This is the scaling factor for the tuning process. For the rotation matrix in the tuning process, This is the translation vector for the tuning process. A collection of key points for a virtual character. Set of 3D key points for face samples The 3D key points of the i-th face sample A collection of key points for virtual characters The i-th virtual character is key.

[0050] For example, in the process of registering the 3D keypoint set of a human face sample and the keypoint set of a virtual character, the combination of the scaling factor s, rotation matrix R, and translation vector t that minimizes the value on the right side of the above formula is determined by continuously adjusting the scaling factor s, rotation matrix R, and translation vector t. Partially, this can be understood as the process of transforming the set of key points for a virtual character based on transformation parameters T, where the scaling factor s is a scalar, the rotation matrix R is a 3x3 orthogonal matrix, and the translation vector t is a 3D translation vector. Optionally, the registration and optimization process of the 3D key point set of the face sample and the key point set of the virtual character based on the registration formula can be solved using the Iterative Closest Point (ICP) algorithm. This scheme registers the 3D key point set of the face sample and the key point set of the virtual character based on the registration formula, obtaining accurate registration results, thereby obtaining more accurate face sample deformation parameters and ensuring the face processing effect.

[0051] In one possible embodiment, such as Figure 2 As shown in the schematic diagram of the registration process for a set of key points, this scheme may include steps S1001-S1003 when registering a set of 3D key points for a face sample and a set of key points for a virtual character:

[0052] S1001: Based on the number of first key points, register the set of 3D key points of the face sample and the set of key points of the virtual character to obtain the first scaling factor, the first rotation matrix and the first translation vector.

[0053] S1002: Based on the number of second key points, the 3D key point set of the face sample and the key point set of the virtual character are registered to obtain the second scaling factor, the second rotation matrix and the second translation vector. The number of second key points is greater than the number of first key points.

[0054] S1003: Based on the second scaling factor, the second rotation matrix, and the second translation vector, determine the set of key point numbers that match the set of 3D key points of the face sample in the set of key points of the virtual character.

[0055] This scheme performs two steps for registering the 3D keypoint sets of face samples and the keypoint sets of virtual characters: coarse registration and fine registration. Coarse registration is based on a first set of keypoints, while fine registration is based on a second set of keypoints, with the second set having a greater number than the first. Coarse registration can be understood as a relatively rough registration process performed when the transformation between the two point clouds is completely unknown, aiming to provide better initial transformation values ​​for fine registration.

[0056] For example, based on a first number of key points (e.g., the 68 facial key points commonly used in the field of face recognition), a first number of 3D key points for the face sample are determined from the set of 3D key points for the face sample. Registration processing is then performed on the set of 3D key points for the face sample and the set of key points for the virtual character to obtain a coarse registration result, i.e., determining the first scaling factor, the first rotation matrix, and the first translation vector. The coarse registration process can be based on the registration formula provided above.

[0057] Furthermore, after determining the first scaling factor, the first rotation matrix, and the first translation vector, based on the second number of keypoints (e.g., the number of keypoints corresponding to the 3D keypoint set of the face sample), a second number of 3D keypoints for the face sample are determined from the 3D keypoint set of the face sample. Based on the aforementioned determined first scaling factor, first rotation matrix, and first translation vector, registration processing is performed on the 3D keypoint set of the face sample and the keypoint set of the virtual character to obtain a fine registration result, namely the second scaling factor, the second rotation matrix, and the second translation vector. Optionally, when performing fine registration processing on the 3D keypoint set of the face sample and the keypoint set of the virtual character, the first scaling factor can be fixed (i.e., the second scaling factor determined by fine registration is the same as the first scaling factor). The first rotation matrix and the first translation vector are used as the initial values ​​of the second rotation matrix and the second translation vector. The second rotation matrix and the second translation vector are continuously optimized based on the registration formula to obtain the final registration result. In this case, the second scaling factor, the second rotation matrix, and the second translation vector are the scaling factor, rotation matrix, and translation vector corresponding to the registration result.

[0058] Furthermore, based on the determined second scaling factor, second rotation matrix, and second translation vector, a set of keypoint numbers matching the set of 3D keypoints of the face sample is determined within the set of keypoints of the virtual character. For example, by traversing each 3D keypoint of the face sample in the set of 3D keypoints of the face sample, the virtual character keypoints in the set of keypoints that are closest to the 3D keypoints of the face sample (e.g., Euclidean distance, Hammington distance, etc.) are identified, and their keypoint numbers within the set of keypoints are determined. Based on these keypoint numbers, a set of keypoint numbers can be obtained. This scheme, by performing coarse and fine registration on the set of 3D keypoints of the face sample and the set of keypoints of the virtual character, improves both registration efficiency and registration effect, obtaining more accurate scaling factors, rotation matrices, and translation vectors, and determining an accurate set of keypoint numbers, thus effectively improving the face processing effect for virtual characters.

[0059] like Figure 3 The provided diagram illustrates a registration result, showing the relative positions of the 3D keypoint sets of the face sample and the keypoint sets of the virtual character after registration. The denser keypoints are those of the virtual character, while the sparser keypoints are those of the face sample (for ease of display, the face sample's 3D keypoints are thicker than the virtual character's keypoints). After registration, the face sample's 3D keypoints can basically overlap with the virtual character's keypoints, and the face sample's 3D keypoints can find a one-to-one correspondence with the closest keypoint in the virtual character's keypoints (this can be recorded through a keypoint number set).

[0060] In one possible embodiment, the face sample deformation parameters provided by this solution can be determined based on the following deformation parameter determination formula:

[0061]

[0062] in, For the face sample deformation parameters that need to be solved, For the face sample deformation parameters in the tuning process, The second scaling factor, This is the second rotation matrix. The second translation vector, Set of key point numbers Based on the deformation parameters of face samples Determined key point numbering, The set of 3D key points for the face sample is used. Optionally, the solution process for the deformation parameter determination formula is a least squares optimization problem, and the final face sample deformation parameters can be solved using gradient descent or Newton's iteration method. Optionally, the 2D expressionless face sample image and the solved face sample deformation parameters can be saved as a binary data file, which will be used as sample data to train the deformation parameter model. This scheme calculates the face sample deformation parameters more accurately through the deformation parameter determination formula, effectively improving the face processing effect for virtual characters.

[0063] S103: Generate a virtual face model based on the face deformation parameters and the set standard face model. The standard face model is a three-dimensional deformable face model.

[0064] This solution pre-configures a standard face model in the virtual character face processing device. The standard face model provided by this solution is a three-dimensional deformable face model. The three-dimensional deformable face model can be adjusted according to the given face deformation parameters to obtain a virtual face model with face features reflected by the given face deformation parameters.

[0065] For example, after determining the facial deformation parameters corresponding to a two-dimensional expressionless face image, a standard face model can be adjusted according to the facial deformation parameters to generate a virtual face model. The generation of the virtual face model based on the facial deformation parameters and the standard face model can be performed using a set image processing engine (e.g., Unreal Engine), and the virtual face model can be rendered by the image processing engine.

[0066] In one possible embodiment, when generating a virtual face model based on face deformation parameters and a set standard face model, this solution can be based on face deformation parameters, and linearly weighted processing is performed on the set standard face model and the face shape vector of the standard face model to obtain the virtual face model.

[0067] The standard face model provided in this solution is a three-dimensional deformable face model, which can be obtained by linearly weighting a standard face and face deformation parameters (shape basis vectors). For example, after determining the face deformation parameters corresponding to a two-dimensional expressionless face image, the two-dimensional expressionless face image and the face deformation parameters can be submitted to a designated image processing engine. The image processing engine then performs linear weighting processing on the standard face model and the face shape vector of the standard face model based on the face deformation parameters to obtain and render a virtual face model. This virtual face model records the user's facial features. This solution obtains a virtual face model reflecting the user's facial features by linearly weighting the standard face model and face shape vector based on face deformation parameters. The generated virtual face model is closer to the user's appearance, simplifying face processing operations while improving the face processing effect. Users only need to upload a two-dimensional expressionless face image to generate a virtual face model, effectively lowering the threshold for face processing.

[0068] In one possible embodiment, the virtual face model provided by this solution can be determined based on the following face model generation formula:

[0069]

[0070] in, For virtual face models, For standard face models, Let i be the i-th dimension face shape vector in the standard face model. Let i be the i-th dimension of the face deformation parameters. These are 106-dimensional facial deformation parameters. Standard face model. and the i-th dimension face shape vector All The vector matrix is ​​defined by n, where n is the number of facial key points in the virtual face model, 3 is the coordinate dimension, and the i-th dimension, the face deformation parameter, ranges from 0 to 1, representing the semantic features of the face. The standard face model is also defined. and the i-th dimension face shape vector The i-th dimension of the face deformation parameter is a constant. The virtual face model is the independent variable. As the dependent variable, what needs to be explained is the virtual face model. Although it is The matrix, but its key point set of the virtual character The meanings are the same; both refer to the 3-dimensional coordinate information of n facial key points. This solution uses a facial model generation formula to linearly weight the standard facial model and the facial shape vector to obtain a virtual facial model that better matches the user's facial features, effectively improving the facial processing effect for virtual characters.

[0071] The above describes a method that acquires a two-dimensional expressionless face image, uses a deformation parameter model to determine the facial deformation parameters of the two-dimensional expressionless face image, and generates a virtual face model based on the facial deformation parameters and a set standard face model. Users only need to upload a two-dimensional expressionless face image, and the three-dimensional deformable standard face model can be adjusted according to the facial deformation parameters to obtain a virtual face model that reflects the facial features in the two-dimensional expressionless face image. This simplifies the virtual character face creation process and improves the virtual character face processing effect.

[0072] Figure 4 This is a schematic diagram of the structure of a virtual character face processing device provided in an embodiment of this application. (Reference) Figure 4 The virtual character face processing device includes an image acquisition module 41, a parameter acquisition module 42, and a face generation module 43.

[0073] The image acquisition module 41 is configured to acquire a two-dimensional expressionless face image; the parameter acquisition module 42 is configured to input the two-dimensional expressionless face image into a trained deformation parameter model, and analyze and process the two-dimensional expressionless face image through the deformation parameter model to obtain face deformation parameters that reflect the facial features in the two-dimensional expressionless face image; the face generation module 43 is configured to generate a virtual face model based on the face deformation parameters and a set standard face model, wherein the standard face model is a three-dimensional deformable face model.

[0074] The above describes a method that acquires a two-dimensional expressionless face image, uses a deformation parameter model to determine the facial deformation parameters of the two-dimensional expressionless face image, and generates a virtual face model based on the facial deformation parameters and a set standard face model. Users only need to upload a two-dimensional expressionless face image, and the three-dimensional deformable standard face model can be adjusted according to the facial deformation parameters to obtain a virtual face model that reflects the facial features in the two-dimensional expressionless face image. This simplifies the virtual character face creation process and improves the virtual character face processing effect.

[0075] Based on the above embodiments, the deformation parameter model is trained on two-dimensional expressionless face sample images and face sample deformation parameters, and the face sample deformation parameters are determined based on the registration results of the three-dimensional key point set of the face sample and the key point set of the virtual character.

[0076] Based on the above embodiments, the registration of the 3D key point set of the face sample and the key point set of the virtual character includes:

[0077] Based on the number of first key points, the 3D key point set of the face sample and the key point set of the virtual character are registered to obtain the first scaling factor, the first rotation matrix and the first translation vector.

[0078] Based on the number of second key points, the 3D key point set of the face sample and the key point set of the virtual character are registered to obtain the second scaling factor, the second rotation matrix and the second translation vector. The number of second key points is greater than the number of first key points.

[0079] Based on the second scaling factor, the second rotation matrix, and the second translation vector, determine the set of key point numbers that match the set of 3D key points of the face sample in the set of key points of the virtual character.

[0080] Based on the above embodiments, the registration of the 3D key point set of the face sample and the key point set of the virtual character is performed based on the following formula:

[0081]

[0082] in, The scaling factor that needs to be solved. Let be the rotation matrix that needs to be solved. Let be the translation vector that needs to be solved. This is the scaling factor for the tuning process. For the rotation matrix in the tuning process, This is the translation vector for the tuning process. A collection of key points for a virtual character. Set of 3D key points for face samples The 3D key points of the i-th face sample A collection of key points for virtual characters The key point of the i-th virtual character.

[0083] Based on the above embodiments, the face sample deformation parameters are determined using the following formula:

[0084]

[0085] in, For the face sample deformation parameters that need to be solved, For the face sample deformation parameters in the tuning process, The second scaling factor, This is the second rotation matrix. The second translation vector, Set of key point numbers Based on the deformation parameters of face samples Determined key point numbering, It is a set of 3D key points for a face sample.

[0086] Based on the above embodiments, when generating a virtual face model based on face deformation parameters and a set standard face model, the face generation module 43 is configured to perform linear weighting processing on the set standard face model and the face shape vector of the standard face model based on face deformation parameters to obtain a virtual face model.

[0087] Based on the above embodiments, the virtual face model is determined according to the following formula:

[0088]

[0089] in, For virtual face models, For standard face models, Let i be the i-th dimension face shape vector in the standard face model. Let be the i-th dimension face deformation parameters.

[0090] It is worth noting that in the embodiments of the virtual character face processing device described above, the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the protection scope of the embodiments of the present invention.

[0091] This application also provides a virtual character face processing device, which can integrate the virtual character face processing apparatus provided in this application. Figure 5 This is a schematic diagram of the structure of a virtual character face processing device provided in an embodiment of this application. (Reference) Figure 5 The virtual character face processing device includes: an input device 53, an output device 54, a memory 52, and one or more processors 51; the memory 52 is used to store one or more programs; when one or more programs are executed by one or more processors 51, the one or more processors 51 implement the virtual character face processing method provided in the above embodiments. The virtual character face processing device, equipment, and computer provided above can be used to execute the virtual character face processing method provided in any of the above embodiments, and have corresponding functions and beneficial effects.

[0092] This application also provides a non-volatile storage medium storing computer-executable instructions, which, when executed by a computer processor, are used to perform the virtual character face processing method provided in the above embodiments. Of course, the computer-executable instructions provided in this application are not limited to the virtual character face processing method provided above; they can also perform related operations in the virtual character face processing method provided in any embodiment of this application. The virtual character face processing apparatus, device, and storage medium provided in the above embodiments can execute the virtual character face processing method provided in any embodiment of this application. Technical details not described in detail in the above embodiments can be found in the virtual character face processing method provided in any embodiment of this application.

[0093] Based on the above embodiments, this application also provides a computer program product. The technical solution of this application, in essence or in other words, the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. The computer program product is stored in a storage medium and includes several instructions to cause a computer device, mobile terminal, or processor therein to execute all or part of the steps of the virtual character face processing method provided in the various embodiments of this application.

Claims

1. A method for processing the face of a virtual character, characterized in that, include: Obtain a two-dimensional, expressionless human face image; The two-dimensional expressionless face image is input into a trained deformation parameter model. The deformation parameter model is used to analyze and process the two-dimensional expressionless face image to obtain facial deformation parameters that reflect the facial features in the two-dimensional expressionless face image. The deformation parameter model is trained based on two-dimensional expressionless face sample images and face sample deformation parameters. The face sample deformation parameters are determined based on the registration results of the face sample three-dimensional key point set and the virtual character key point set. The face sample three-dimensional key point set is sparser than the virtual character key point set. A virtual face model is generated based on the face deformation parameters and the set standard face model, wherein the standard face model is a three-dimensional deformable face model.

2. The virtual character face processing method according to claim 1, characterized in that, The registration of the set of 3D key points of the face sample and the set of key points of the virtual character includes: Based on the first number of key points, the three-dimensional key point set of the face sample and the key point set of the virtual character are registered to obtain the first scaling factor, the first rotation matrix and the first translation vector. Based on the number of second key points, the three-dimensional key point set of the face sample and the key point set of the virtual character are registered to obtain a second scaling factor, a second rotation matrix and a second translation vector. The number of second key points is greater than the number of first key points. Based on the second scaling factor, the second rotation matrix, and the second translation vector, determine the set of key point numbers that match the set of 3D key points of the face sample in the set of key points of the virtual character.

3. The virtual character face processing method according to claim 2, characterized in that, The registration of the set of 3D key points of the face sample and the set of key points of the virtual character is based on the following formula: in, The scaling factor that needs to be solved. Let be the rotation matrix that needs to be solved. Let be the translation vector that needs to be solved. This is the scaling factor for the tuning process. For the rotation matrix in the tuning process, This is the translation vector for the tuning process. A collection of key points for a virtual character. Set of 3D key points for face samples The 3D key points of the i-th face sample A collection of key points for virtual characters The key point of the i-th virtual character.

4. The virtual character face processing method according to claim 2, characterized in that, The deformation parameters of the face sample are determined based on the following formula: in, For the face sample deformation parameters that need to be solved, For the face sample deformation parameters in the tuning process, The second scaling factor, This is the second rotation matrix. The second translation vector, Set of key point numbers Based on the deformation parameters of face samples Determined key point numbering, It is a set of 3D key points for a face sample.

5. The virtual character face processing method according to claim 1, characterized in that, The process of generating a virtual face model based on the facial deformation parameters and a set standard face model includes: Based on the facial deformation parameters, a virtual facial model is obtained by linearly weighting the set standard facial model and the facial shape vector of the standard facial model.

6. The virtual character face processing method according to claim 1, characterized in that, The virtual face model is determined based on the following formula: in, For virtual face models, For standard face models, Let i be the i-th dimension face shape vector in the standard face model. Let be the i-th dimension face deformation parameters.

7. A virtual character face processing device, characterized in that, It includes an image acquisition module, a parameter acquisition module, and a face generation module, wherein: The image acquisition module is configured to acquire a two-dimensional expressionless human face image; The parameter acquisition module is configured to input the two-dimensional expressionless face image into a trained deformation parameter model, and analyze and process the two-dimensional expressionless face image through the deformation parameter model to obtain face deformation parameters reflecting the face features in the two-dimensional expressionless face image; the deformation parameter model is trained based on two-dimensional expressionless face sample images and face sample deformation parameters, and the face sample deformation parameters are determined based on the registration results of the three-dimensional key point set of the face sample and the key point set of the virtual character, wherein the three-dimensional key point set of the face sample is sparser than the key point set of the virtual character; The face generation module is configured to generate a virtual face model based on the face deformation parameters and a set standard face model, wherein the standard face model is a three-dimensional deformable face model.

8. A virtual character face processing device, characterized in that, include: Memory and one or more processors; The memory is used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the virtual character face processing method as described in any one of claims 1-6.

9. A non-volatile storage medium for storing computer-executable instructions, characterized in that, The computer-executable instructions, when executed by a computer processor, are used to perform the virtual character face processing method as described in any one of claims 1-6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the virtual character face processing method according to any one of claims 1-6.

Images