A training method and system for user expression capture model

Abstract

The present invention discloses a training method and system for a user expression capture model, comprising: firstly, using common expression data in a first training expression data set to perform an initial training operation on an original user expression capture model to obtain an advanced user expression capture model; then obtaining an updated expression data set required for model update to filter out gradient vectors corresponding to updated expression data belonging to a third training expression data set and including common expressions; finally, using the gradient vectors corresponding to the remaining updated expression data in the updated expression data set to update the hyperparameters of the advanced user expression capture model until a loss function reaches a preset condition, thereby obtaining a user expression capture model for capturing common expression data and unconventional expression data. With such a design, a model capable of accurately capturing user expressions can be obtained to improve subsequent data processing efficiency.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular to a training method and system for a user expression capture model. Background Art

[0002] At present, with the development of concepts such as "metaverse", users demand a more realistic experience in the corresponding virtual environment. How to achieve a more realistic expression display of virtual characters in the virtual environment is a problem that technical personnel in this field need to solve. Summary of the invention

[0003] The purpose of the present invention is to provide a training method and system for a user expression capture model.

[0004] In a first aspect, an embodiment of the present invention provides a method for training a user expression capture model, comprising:

[0005] Determine a first training expression data set, a second training expression data set, and a third training expression data set, wherein the samples in the first training expression data set are common expression data, the samples in the second training expression data set are unconventional expression data, and the third training expression data set includes common expression data and unconventional expression data;

[0006] Using the common expression data in the first training expression data set to perform an initial training operation on the original user expression capture model to obtain an advanced user expression capture model;

[0007] An updated expression data set required for model updating is obtained from the first training expression data set, the second training expression data set and the third training expression data set by traversal polling. In the process of using the updated expression data in the updated expression data set to update the advanced user expression capture model, after filtering out the gradient vectors corresponding to the updated expression data that belong to the third training expression data set and include commonly used expressions, the gradient vectors corresponding to the remaining updated expression data in the updated expression data set are used to update the hyperparameters of the advanced user expression capture model until the loss function reaches a preset condition, thereby obtaining a user expression capture model for capturing commonly used expression data and unconventional expression data.

[0008] In a second aspect, an embodiment of the present invention provides a server system, including a server, where the server is used to execute the method of the first aspect.

[0009] Compared with the prior art, the beneficial effects provided by the present invention include: the present invention discloses a training method and system for a user expression capture model, including: first, using the common expression data in the first training expression data set to perform an initial training operation on the original user expression capture model to obtain an advanced user expression capture model; then obtaining an updated expression data set required for model updating to filter out the gradient vectors corresponding to the updated expression data belonging to the third training expression data set and including common expressions; finally, using the gradient vectors corresponding to the remaining updated expression data in the updated expression data set to update the hyperparameters of the advanced user expression capture model until the loss function reaches a preset condition, thereby obtaining a user expression capture model for capturing common expression data and unconventional expression data. With such a design, a model that can accurately capture user expressions can be obtained, thereby improving data processing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only illustrate certain embodiments of the present invention and should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can also be obtained based on these drawings without creative work.

[0011] Figure 1 A schematic flow chart of the steps of a method for training a user expression capture model provided by an embodiment of the present invention;

[0012] Figure 2 A schematic block diagram of the structure of a server provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0013] The specific implementation modes of the present invention are described in detail below in conjunction with the accompanying drawings.

[0014] Please refer to Figure 1 , Figure 1 A method for training a user expression capture model is provided in an embodiment of the present invention. The method for training the user expression capture model is described in detail below.

[0015] Step S202, determining a first training expression data set, a second training expression data set and a third training expression data set, wherein the samples in the first training expression data set are common expression data, the samples in the second training expression data set are unconventional expression data, and the third training expression data set includes common expression data and unconventional expression data.

[0016] The first training expression data set, the second training expression data set and the third training expression data set are all image sets used to train the user expression capture model. Common expression data refers to the type of expression in the image that belongs to common expressions, such as smile, cry, anger, etc., and unconventional expression data refers to the type of expression in the image that belongs to unconventional expressions, such as uncommon expressions such as crying and laughing in a specific scene, distortion, etc.

[0017] Assume that the user expression capture model is first trained with a small number of samples of unconventional expressions. The user expression capture model learns the features used to distinguish samples of unconventional expressions from the samples of unconventional expressions, and obtains the feature space of unconventional expressions. Assume that the user expression capture model is then trained with a large number of samples of common expressions. The user expression capture model learns the features used to distinguish samples of common expressions from the samples of common expressions, and obtains the feature space of common expressions. The feature space of common expressions compresses the feature space of unconventional expressions, shortening the distance between features in the feature space of unconventional expressions.

[0018] In the present invention, a first training expression data set, a second training expression data set and a third training expression data set are designed, the samples in the first training expression data set are common expression data, which can be used for a user expression capture model to learn features for distinguishing common expressions; the samples in the second training expression data set are unconventional expression data, which can be used for a user expression capture model to learn features for distinguishing unconventional expressions; the third training expression data set includes common expression data and unconventional expression data, which can be used for a user expression capture model to learn features for distinguishing common expressions and unconventional expressions, and the common expression data included in the third training expression data set can be used to intervene in the user expression capture model's learning of features of unconventional expressions, so as to expand the feature space of unconventional expressions.

[0019] In one embodiment, the computer device determines a first training expression data set and a second training expression data set, and uses the union of the first training expression data set and the second training expression data set as the third training expression data set. In other embodiments, the samples in the third training expression data set do not overlap or partially overlap with the samples in the first training expression data set and the second training expression data set, and the third training expression data set includes common expression data and unconventional expression data.

[0020] Step S204, using the common expression data in the first training expression data set to perform an initial training operation on the original user expression capture model to obtain an advanced user expression capture model.

[0021] The original user expression capture model may be a user expression capture model that has not been trained with samples in the first training expression dataset, the second training expression dataset, or the third training expression dataset. The advanced user expression capture model may be a user expression capture model that has been trained with samples in the first training expression dataset.

[0022] In the present invention, a computer device determines a first training expression data set, a second training expression data set, and a third training expression data set, and first uses the common expression data in the first training expression data set to perform an initial training operation on an original user expression capture model to obtain an advanced user expression capture model. Since the first training expression data set only includes common expression data, the common expression data in the first training expression data set is used to perform an initial training operation on the original user expression capture model, so that the user expression capture model learns essential features for distinguishing common expressions, thereby ensuring the recognition accuracy of the user expression capture model for the common expression data.

[0023] In one embodiment, the computer device performs an initial training operation on the original user expression capture model to update the hyperparameters of the expression feature recognition layer and the expression type classification layer.

[0024] Among them, the expression feature recognition layer is a model structure that has the ability to extract expression feature vectors through sample learning. The expression feature recognition layer can be a convolutional neural network structure, which can perform operations such as convolution calculation, nonlinear activation function calculation, and pooling calculation. The input end of the expression feature recognition layer is the expression image, and the output end is the expression feature vector. The expression feature vector can be a feature map, a feature vector, etc. The expression type classification layer is a model structure that has the ability to classify and regress through sample learning. The input end of the expression type classification layer is the expression feature vector, and the output end is the expression capture classification result. It can be understood that a general network structure with the ability to extract expression feature vectors and the ability to classify and regress can meet the requirements of the expression feature recognition layer and the expression type classification layer in the embodiment of the present invention. Therefore, the embodiment of the present invention can adopt a general network structure as the expression feature recognition layer and the expression type classification layer.

[0025] In one embodiment, the computer device determines more than one commonly used expression data from the first training expression data set, inputs more than one commonly used expression data into the original user expression capture model, extracts the expression feature vector corresponding to each commonly used expression data through the expression feature recognition layer in the original user expression capture model, and obtains the expression capture classification result corresponding to each commonly used expression data according to the expression feature vector through the expression type classification layer in the original user expression capture model. The computer device constructs an initial cost function according to the expression capture classification result corresponding to each commonly used expression data and the expression classification identifier, determines the gradient vector of this training according to the initial cost function, updates the hyperparameters of the expression feature recognition layer and the expression type classification layer according to the gradient vector, and then returns to the step of determining more than one commonly used expression data from the first training expression data set to continue training until the training is completed.

[0026] Step S206, traversing and polling to obtain an updated expression data set required for model updating from the first training expression data set, the second training expression data set and the third training expression data set, and in the process of using the updated expression data in the updated expression data set to update the advanced user expression capture model, after filtering out the gradient vectors corresponding to the updated expression data that belong to the third training expression data set and include commonly used expressions, use the gradient vectors corresponding to the remaining updated expression data in the updated expression data set to update the hyperparameters of the advanced user expression capture model until the loss function reaches a preset condition, thereby obtaining a user expression capture model for capturing commonly used expression data and unconventional expression data.

[0027] In the present invention, a computer device determines a first training expression data set, a second training expression data set, and a third training expression data set, first uses the common expression data in the first training expression data set to perform an initial training operation on the original user expression capture model to obtain an advanced user expression capture model, then traverses and polls to obtain an updated expression data set from the first training expression data set, the second training expression data set, and the third training expression data set, and uses the updated expression data in the updated expression data set to perform model update on the advanced user expression capture model. The advanced user expression capture model learns features for distinguishing common expressions based on the common expression data in the first training expression data set, learns features for distinguishing unconventional expressions based on the unconventional expression data in the second training expression data set, and learns features for distinguishing common expressions from unconventional expressions based on the common expression data and unconventional expression data in the third training expression data set, and the common expression data included in the third training expression data set can be used to intervene in the learning of features of unconventional expressions by the advanced user expression capture model to expand the feature space of unconventional expressions.

[0028] In one embodiment, the computer device updates the advanced user expression capture model and updates the hyperparameters of the expression feature recognition layer and the expression type classification layer.

[0029] In one embodiment, a computer device performs a data extraction operation on a first training expression data set, a second training expression data set, and a third training expression data set to obtain an updated expression data set, inputs the updated expression data in the updated expression data set into an advanced user expression capture model, extracts expression feature vectors corresponding to each updated expression data through an expression feature recognition layer in the advanced user expression capture model, and obtains expression capture classification results corresponding to each updated expression data according to the expression feature vectors through an expression type classification layer in the advanced user expression capture model. The computer device constructs a model update cost function according to the expression capture classification results corresponding to the updated expression data and the expression classification identifier, determines the gradient vector of this training according to the model update cost function, screens out the gradient vectors corresponding to the updated expression data belonging to the third training expression data set and including common expressions from the gradient vectors, uses the remaining gradient vectors corresponding to the updated expression data in the updated expression data set to update the hyperparameters of the advanced user expression capture model, and then returns to the step of performing data extraction operations on the first training expression data set, the second training expression data set, and the third training expression data set to continue training until the training is completed. Since the gradient vectors corresponding to the third training expression data set and the updated expression data including common expressions are screened out, the third training expression data set and the updated expression data including common expressions are only used to expand the feature space of unconventional expressions without affecting the update of hyperparameters.

[0030] In one embodiment, a computer device determines the gradient vector of this training based on a model update cost function, and performs mask processing on the gradient vector corresponding to the third training expression data set and the updated expression data including commonly used expressions, so that the gradient vector corresponding to the third training expression data set and the updated expression data including commonly used expressions does not participate in the update of hyperparameters.

[0031] The method provided by the embodiment of the present invention can improve the recognition accuracy of the user expression capture model for commonly used expression data, and also greatly improve the recognition accuracy of the user expression capture model for unconventional expression data.

[0032] In the above-mentioned processing method of the user expression capture model, the common expression data in the first training expression data set is first used to perform an initial training operation on the original user expression capture model to obtain an advanced user expression capture model, and the recognition accuracy of the user expression capture model for the common expression data is improved; then the updated expression data in the first training expression data set, the second training expression data set and the third training expression data set are used to fine-tune the advanced user expression capture model, and the advanced user expression capture model learns features for distinguishing common expressions based on the common expression data in the first training expression data set, learns features for distinguishing unconventional expressions based on the unconventional expression data in the second training expression data set, and learns features for distinguishing common expressions and unconventional expressions based on the common expression data and unconventional expression data in the third training expression data set. The common expression data included in the third training expression data set can be used to intervene in the learning of the features of unconventional expressions by the advanced user expression capture model to expand the feature space of unconventional expressions, reduce the vector distance between non-same-person unconventional expressions calculated by the user expression capture model, and improve the recognition accuracy of unconventional expression data; in this way, while improving the recognition accuracy of the user expression capture model for common expression data, the recognition accuracy of the user expression capture model for unconventional expression data is also greatly improved.

[0033] In one embodiment, using the common expression data in the first training expression data set to perform an initial training operation on the original user expression capture model to obtain an advanced user expression capture model includes:

[0034] Step S602: determining commonly used expression data and corresponding expression classification identifiers from the first training expression data set.

[0035] The expression classification identifier may be an expression label corresponding to the expression in the expression image. The expression label is used to uniquely distinguish the expression and may be composed of at least one of letters, numbers, and text.

[0036] In one embodiment, the computer device determines more than one commonly used expression data from the first training expression data set, and uses the more than one commonly used expression data as a group to perform pre-training on the original user expression capture model.

[0037] Step S604: input the common expression data determined from the first training expression data set into the original user expression capture model to obtain the expression capture classification result.

[0038] In one embodiment, a computer device extracts expression feature vectors corresponding to each commonly used expression data through an expression feature recognition layer in an original user expression capture model; and obtains expression capture classification results corresponding to each commonly used expression data based on the expression feature vectors through an expression type classification layer in the original user expression capture model.

[0039] The expression feature vector is data used to reflect the expression features in the expression image.

[0040] In one embodiment, the expression feature vector may specifically be one or a combination of position information, texture information, shape information, color information, etc., which are related to the expression feature and extracted from the expression image.

[0041] In one embodiment, the computer device extracts the expression feature vectors corresponding to each common expression data through the expression feature recognition layer in the original user expression capture model, passes each expression feature vector to the expression type classification layer in the original user expression capture model, and performs classification and regression processing on each expression feature vector through the expression type classification layer to obtain the expression capture classification result corresponding to each common expression data. The expression capture classification result can be a probability vector whose dimension matches the number of expression labels in the training sample, and the value of each dimension represents the probability that the expression in the expression image corresponds to one of the expression labels.

[0042] Step S606, constructing an initial cost function according to the expression classification identifier and the expression capture classification result.

[0043] In one embodiment, the computer device constructs an initial cost function according to the difference between the expression classification identifier corresponding to each common expression data and the expression capture classification result. The initial cost function may be a Softmax function.

[0044] Step S608, after using the hyperparameters when the initial cost function is minimized as the hyperparameters for updating the original user expression capture model, return to the step of determining the common expression data from the first training expression data set to continue training until the training is completed.

[0045] In one embodiment, the computer device obtains the gradient vector corresponding to the current training according to the gradient descent algorithm in the direction of minimizing the initial cost function, and updates the hyper parameters of the expression feature recognition layer and the expression type classification layer according to the gradient vector. The gradient descent algorithm can be a stochastic gradient descent algorithm, or an algorithm optimized according to the stochastic gradient descent algorithm, such as a stochastic gradient descent algorithm with a momentum term.

[0046] In one embodiment, the training end condition may be that the number of training times reaches a preset number, or the loss value calculated by the initial cost function is less than a preset value, etc.

[0047] In this embodiment, the common expression data in the first training expression data set is used to perform initial training operations on the original user expression capture model, so that the user expression capture model learns the essential features for distinguishing common expressions, thereby ensuring the recognition accuracy of the common expression data by the user expression capture model.

[0048] In one embodiment, obtaining an updated expression data set required for model updating from a first training expression data set, a second training expression data set, and a third training expression data set includes: determining a data extraction rule set in advance; performing data extraction operations on the first training expression data set, the second training expression data set, and the third training expression data set according to the data extraction rule, respectively, to obtain the updated expression data set required for model updating.

[0049] In one embodiment, the computer device performs data extraction operations on the first training expression data set, the second training expression data set, and the third training expression data set according to the data extraction rules set in advance, takes the common expression data sampled from the first training expression data set as subset A, takes the unconventional expression data sampled from the second training expression data set as subset B, takes the common expression data and / or unconventional expression data sampled from the third training expression data set as subset C, and merges subsets A, B, and C into an updated expression data set. The computer device inputs the updated expression data in the updated expression data set into the advanced user expression capture model to perform multi-task model update on the advanced user expression capture model.

[0050] In one embodiment, in the process of updating the advanced user expression capture model using the updated expression data in the updated expression data set, after filtering out the gradient vectors corresponding to the updated expression data belonging to the third training expression data set and including the commonly used expressions, the hyperparameters of the advanced user expression capture model are updated using the gradient vectors corresponding to the remaining updated expression data in the updated expression data set, including:

[0051] Step S702, using each updated expression data in the updated expression data set to perform model update on the advanced user expression capture model, and determining a gradient vector corresponding to each updated expression data during the model update process.

[0052] In one embodiment, an advanced user expression capture model is updated using each updated expression data in an updated expression data set, and a gradient vector corresponding to each updated expression data is determined during the model update process, including: inputting each updated expression data in the updated expression data set into the advanced user expression capture model to obtain an expression capture classification result corresponding to each updated expression data; constructing a model update cost function based on the expression classification identifier and the expression capture classification result corresponding to each updated expression data; and obtaining a gradient vector corresponding to each updated expression data when the model update cost function is minimized.

[0053] In one embodiment, each updated expression data in the updated expression data set is input into an advanced user expression capture model to obtain an expression capture classification result corresponding to each updated expression data, including: extracting an expression feature vector corresponding to each updated expression data through an expression feature recognition layer in the advanced user expression capture model; obtaining an expression capture classification result corresponding to each updated expression data according to the expression feature vector through an expression type classification layer in the advanced user expression capture model.

[0054] In one embodiment, the computer device constructs a model update cost function according to the difference between the expression classification identifiers corresponding to each updated expression data and the expression capture classification results. The model update cost function may be a Softmax function.

[0055] In one embodiment, a model update cost function is constructed based on the expression classification identifiers and expression capture classification results corresponding to each updated expression data, including: constructing a common cost function based on the expression classification identifiers and expression capture classification results corresponding to the updated expression data belonging to the first training expression data set in the updated expression data set; constructing an unconventional cost function based on the expression classification identifiers and expression capture classification results corresponding to the updated expression data belonging to the second training expression data set in the updated expression data set; constructing a comprehensive cost function based on the expression classification identifiers and expression capture classification results corresponding to the updated expression data belonging to the third training expression data set in the updated expression data set; determining the cost allocation weights set in advance; and summing the common cost function, the unconventional cost function and the comprehensive cost function according to the cost allocation weights to obtain the model update cost function.

[0056] Among them, the common cost function, the unconventional cost function and the comprehensive cost function can be loss functions, such as the Softmax function.

[0057] In one embodiment, the updated expression data in the updated expression data set that belongs to the first training expression data set (referred to as subset A for short), the updated expression data in the updated expression data set that belongs to the second training expression data set (referred to as subset B for short), and the updated expression data in the updated expression data set that belongs to the third training expression data set (referred to as subset C for short) can be set with cost allocation weights respectively.

[0058] In one embodiment, the computer device obtains the gradient vector corresponding to the current training according to the gradient descent algorithm in the direction of minimizing the model update cost function, and determines the gradient vector corresponding to each updated expression data from the gradient vector. The gradient descent algorithm can be a stochastic gradient descent algorithm, or an algorithm optimized according to the stochastic gradient descent algorithm, such as a stochastic gradient descent algorithm with a momentum term.

[0059] Step S704, after filtering out the gradient vectors corresponding to the updated expression data that belong to the third training expression data set and include commonly used expressions from the gradient vectors corresponding to each updated expression data, use the gradient vectors corresponding to the remaining updated expression data in the updated expression data set to update the hyperparameters of the advanced user expression capture model.

[0060] Assume that the updated expression data 4 and the updated expression data 5 are the updated expression data that belong to the third training expression data set and include commonly used expressions in the updated expression data set. It can be seen that the computer device extracts the expression feature vector corresponding to each updated expression data, and obtains the expression capture classification result corresponding to each updated expression data according to the expression feature vector. According to the expression classification identification and expression capture classification result corresponding to the updated expression data of subset A, a common cost function is constructed, according to the expression classification identification and expression capture classification result corresponding to the updated expression data of subset B, an unconventional cost function is constructed, according to the expression classification identification and expression capture classification result corresponding to the updated expression data of subset C, a comprehensive cost function is constructed, and the common cost function, the unconventional cost function and the comprehensive cost function are summed according to the cost distribution weight to obtain the model update cost function. According to the direction of minimizing the model update cost function, the gradient vector corresponding to this training is obtained according to the gradient descent algorithm, and the gradient vector corresponding to each updated expression data is determined from the gradient vector. From the gradient vectors corresponding to each updated expression data, after filtering out the gradient vectors corresponding to updated expression data 4 and updated expression data 5, use the gradient vectors corresponding to updated expression data 1, updated expression data 2, updated expression data 3 and updated expression data 6 to update the hyperparameters of the advanced user expression capture model.

[0061] In this embodiment, the advanced user expression capture model is fine-tuned using the updated expression data in the first training expression data set, the second training expression data set, and the third training expression data set. The advanced user expression capture model learns features for distinguishing common expressions based on the common expression data in the first training expression data set, learns features for distinguishing unconventional expressions based on the unconventional expression data in the second training expression data set, and learns features for distinguishing common expressions and unconventional expressions based on the common expression data and unconventional expression data in the third training expression data set. The common expression data included in the third training expression data set can be used to intervene in the advanced user expression capture model's learning of features of unconventional expressions, so as to expand the feature space of unconventional expressions, reduce the vector distance between non-same-person unconventional expressions calculated by the user expression capture model, and improve the recognition accuracy of unconventional expression data.

[0062] In one embodiment, the method further includes: fixing the hyperparameters of the expression feature recognition layer of the advanced user expression capture model, traversing and polling to obtain the updated expression data set required for model update from the first training expression data set, the second training expression data set and the third training expression data set, and using the updated expression data in the updated expression data set required for model update to perform model update on the expression type classification layer of the advanced user expression capture model; in the process of model update, after filtering out the gradient vectors corresponding to the updated expression data that belong to the third training expression data set and include commonly used expressions, the hyperparameters of the expression type classification layer are updated using the gradient vectors corresponding to the remaining updated expression data in the updated expression data set; until the loss function reaches a preset condition, fixing the hyperparameters of the expression type classification layer of the advanced user expression capture model, traversing and polling to obtain the updated expression data set required for model update from the first training expression data set, the second training expression data set and the third training expression data set, and using the updated expression data in the updated expression data set required for model update to perform model update on the expression feature recognition layer of the advanced user expression capture model; in the process of model update, after filtering out the gradient vectors corresponding to the updated expression data that belong to the third training expression data set and include commonly used expressions, the hyperparameters of the expression feature recognition layer are updated using the gradient vectors corresponding to the remaining updated expression data in the updated expression data set.

[0063] In this embodiment, after the hyperparameters of the expression type classification layer are updated, the hyperparameters of the expression feature recognition layer may be updated to improve the training effect and training efficiency.

[0064] In one embodiment, in the process of updating the advanced user expression capture model using the updated expression data in the updated expression data set, after filtering out the gradient vectors corresponding to the updated expression data that belong to the third training expression data set and include commonly used expressions, the gradient vectors corresponding to the remaining updated expression data in the updated expression data set are used to update the hyperparameters of the advanced user expression capture model, including: when in the updated expression data set, the updated expression data sampled from the third training expression data set includes unconventional expression data, then the gradient vectors corresponding to the updated expression data that belong to the third training expression data set and include unconventional expressions, the updated expression data that belong to the first training expression data set, and the updated expression data that belong to the second training expression data set are used to update the hyperparameters of the advanced user expression capture model; when in the updated expression data set, the updated expression data sampled from the third training expression data set does not include unconventional expression data, then the gradient vectors corresponding to the updated expression data that belong to the first training expression data set and the updated expression data that belong to the second training expression data set are used to update the hyperparameters of the advanced user expression capture model.

[0065] Specifically, when the computer device performs data extraction operations on the first training expression data set, the second training expression data set, and the third training expression data set, since the third training expression data set includes common expression data and unconventional expression data, the updated expression data sampled by the computer device from the third training expression data set has the following three situations: (1) The computer device only samples common expression data from the third training expression data set. (2) The computer device only samples unconventional expression data from the third training expression data set. (3) The computer device samples common expression data and unconventional expression data from the third training expression data set. In response to different sampling situations, the computer device adopts different hyperparameter updating measures. For the first situation (1), the computer device uses the gradient vectors corresponding to the updated expression data belonging to the first training expression data set and the updated expression data belonging to the second training expression data set in the updated expression data set to update the hyperparameters of the advanced user expression capture model. For the second and third cases, the computer device uses the gradient vectors corresponding to the updated expression data belonging to the third training expression data set and including the unconventional expressions, the updated expression data belonging to the first training expression data set, and the updated expression data belonging to the second training expression data set in the updated expression data set to update the hyperparameters of the advanced user expression capture model. In this way, the updated expression data of the third training expression data set and including the commonly used expressions are only used to expand the feature space of the unconventional expressions, and do not affect the update of the hyperparameters.

[0066] In one embodiment, the method also includes: determining test expression data and a test expression classification corresponding to the test expression data, the test expression data including common expressions or unconventional expressions; determining a test expression feature vector corresponding to the test expression data through a user expression capture model obtained after model updating; determining a comparison expression feature vector corresponding to the test expression classification; when a vector distance between the test expression feature vector and the comparison expression feature vector is lower than a preset distance threshold, determining that the test expression data is captured as the expression classification corresponding to the comparison expression feature vector.

[0067] The test expression data is an image of the expression to be captured. Expression capture is to verify whether the expression in the test expression data matches the test expression classification. The test expression feature vector is data used to reflect the expression features in the test expression data. The comparison expression feature vector is data used to reflect the expression features in the reference expression image. The reference expression image in this application scenario is a pre-stored expression image that matches the test expression classification.

[0068] Specifically, the computer device determines the test expression feature vector corresponding to the test expression data and the comparison expression feature vector corresponding to the test expression classification through the trained user expression capture model. When the vector distance between the test expression feature vector and the comparison expression feature vector is lower than a preset distance threshold, it is determined that the test expression data is captured as the expression classification corresponding to the comparison expression feature vector.

[0069] In an embodiment of the present invention, the following example is also provided to better utilize the user expression capture model.

[0070] Step S101, completing the expression capture of the target user based on the user expression capture model to obtain the target expression data.

[0071] Step S102, determining a preset expression area in the virtual environment corresponding to the target expression data.

[0072] Step S103, using the expression optimization model trained according to the target expression data, controlling the virtual character corresponding to the preset expression area to display expression.

[0073] In the embodiment of the present invention, the user expression capture model can be used to capture the expression of the target user. The target user can refer to a player, an ordinary user, or an actor who provides an expression source for a specific virtual character, etc., without limitation. Then, the preset expression area in the virtual environment corresponding to the target expression data can be determined, and then the expression optimization model trained according to the target expression data can be used to control the virtual character corresponding to the preset expression area to display the expression, so that the virtual character can display the expression realistically and accurately.

[0074] In order to clearly describe the aforementioned step S103, it can be implemented through the following detailed steps.

[0075] Step S901: The computer device obtains a plurality of accuracy indication marks of preset expression regions, wherein the plurality of accuracy indication marks are used to represent different accuracy indications of a plurality of facial regions of the preset expression regions, and one accuracy indication mark corresponds to at least one facial region.

[0076] The preset expression area refers to the part of the subject that can perform a certain expression. For example, the preset expression area is the face of a virtual character in a game application; for another example, the preset expression area is the display part of the screen or other virtual characters in a virtual environment for interacting with the user. The embodiments of the present invention do not limit the preset expression area.

[0077] In some embodiments, when the preset expression area displays an expression, the accuracy indication of different facial areas is different. Multiple accuracy indication marks are used to indicate different accuracy indications of different facial areas. In an embodiment of the present invention, an accuracy indication mark corresponds to at least one facial area, and the accuracy indication mark is used to indicate the level of the accuracy indication of the at least one facial area.

[0078] Taking the game scene as an example, the preset expression area is the virtual character in the game, and the virtual character needs to show various and flexible expressions, such as talking with a partner, releasing skills, and communicating with the enemy. In the actual application of the game, the game producer has different accuracy indications for different facial areas of the virtual character. Some facial areas have higher accuracy indications, such as the mouth, eyebrows, eyes, etc., and some facial areas have lower accuracy indications, such as the chin, etc. The computer device obtains multiple accuracy indication marks of the preset expression areas, that is, obtains the accuracy indication marks of different facial areas of the virtual character.

[0079] It should be noted that the accuracy indicator may be in any form, and the embodiment of the present invention does not limit the form of the accuracy indicator.

[0080] In some embodiments, the accuracy indicator is a character, and different characters are used to represent different accuracy indicator values.

[0081] Step S902: The computer device obtains multiple demand calculation difficulty coefficients according to multiple accuracy indication identifiers, where one demand calculation difficulty coefficient corresponds to one accuracy indication identifier, and the demand calculation difficulty coefficient is positively correlated with the accuracy indication represented by the corresponding accuracy indication identifier.

[0082] The demand calculation difficulty coefficient is used to indicate the complexity of the model. Optionally, the demand calculation difficulty coefficient can be represented by characters, and different characters represent different levels of calculation difficulty coefficients. For example, the demand calculation difficulty coefficients are S, A and B, and the corresponding model complexity levels are high, medium and low. Optionally, the demand calculation difficulty coefficient can also be represented by a specific numerical value, for example, the demand calculation difficulty coefficient is 100%, 80% and 60%, etc. Among them, the demand calculation difficulty coefficient of 100% can refer to the network calculation difficulty coefficient of the expression optimization model used in the relevant technology. Optionally, the demand calculation difficulty coefficient can be represented by the number of hyperparameter network architecture data. Optionally, the demand calculation difficulty coefficient can be represented by the number of units in each layer in the model. The embodiment of the present invention does not limit the demand calculation difficulty coefficient.

[0083] Step S903: The computer device generates a plurality of first expression learning networks whose network calculation difficulty coefficients are respectively the plurality of required calculation difficulty coefficients, and the plurality of first expression learning networks are respectively used to learn expressions of facial areas indicated by different accuracy.

[0084] The network calculation difficulty coefficients of the multiple first expression learning networks are respectively the multiple requirement calculation difficulty coefficients. Since the multiple requirement calculation difficulty coefficients respectively correspond one-to-one with the multiple accuracy indication marks, the multiple first expression learning networks also correspond one-to-one with the multiple accuracy indication marks, and the network calculation difficulty coefficients of the first expression learning networks match the corresponding accuracy indication marks. The network calculation difficulty coefficient of the first expression learning network matches the accuracy indication mark corresponding to the first expression learning network, which means that: the higher the accuracy indication represented by the accuracy indication mark corresponding to the first expression learning network, the higher the network calculation difficulty coefficient corresponding to the first expression learning network; the lower the accuracy indication represented by the accuracy indication mark corresponding to the first expression learning network, the lower the network calculation difficulty coefficient corresponding to the first expression learning network.

[0085] Since the higher the network calculation difficulty coefficient of the first expression learning network, the first expression learning network can learn more accurate expression representations and generate more accurate expression parameters for the facial area, therefore, the higher the accuracy indication represented by the accuracy indicator corresponding to the first expression learning network, the higher the network calculation difficulty coefficient corresponding to the first expression learning network. Since the accuracy indication of some facial areas is low, even if the expression parameters generated for the facial areas are not accurate enough, it will not affect the expression control effect. Therefore, an expression learning network with a lower network calculation difficulty coefficient can be used, so that the size of the model can be reduced.

[0086] Step S904, the computer device determines, for any first expression learning network, from the training expression parameters of the preset expression area, the training expression parameters of the first facial area corresponding to the first accuracy indicator, the first accuracy indicator corresponding to the required calculation difficulty coefficient of the first expression learning network, and performs an iterative training operation on the first expression learning network according to the training expression parameters of the first facial area to obtain a second expression learning network, and the second expression learning network is used to generate expression parameters for the first facial area.

[0087] The training expression parameters of the preset expression area are expression parameters of the preset expression area when performing a certain expression. The expression parameters may be recorded by an expression capture device or determined by other means. For example, taking a game scene as an example, the training expression parameters of the preset expression area may be expression parameters generated for the preset expression area during the game operation. It should be noted that the embodiments of the present invention illustrate the content and source of the training expression parameters through the following embodiments, which will not be repeated here.

[0088] In an embodiment of the present invention, a first expression learning network corresponds to an accuracy indicator, and the accuracy indicator corresponds to at least one facial area. Therefore, the first expression learning network is used to generate expression parameters for the at least one facial area, so that the expression parameters generated by the first expression learning network meet the accuracy indications of different facial areas.

[0089] The first facial region may be one facial region or multiple facial regions, depending on whether the facial region corresponding to the accuracy indicator corresponding to the first expression learning network is one facial region or multiple facial regions.

[0090] It should be noted that the training methods of the multiple first expression learning networks are the same, and the embodiment of the present invention only exemplifies the training process of one first expression learning network.

[0091] Step S905: The computer device obtains an expression optimization model based on the multiple second expression learning networks obtained through training, and the expression optimization model is used to generate expression parameters with different precisions for multiple facial regions.

[0092] In one possible implementation, a computer device obtains an expression optimization model based on multiple second expression learning networks obtained through training, including: deploying the multiple second expression learning networks obtained through training into the same model file to obtain a model file of the expression optimization model. Since the multiple second expression learning networks are deployed into the same model file, when calling the model to generate expression parameters, the multiple second expression learning networks can be run simultaneously by calling the model file to obtain expression parameters of multiple facial areas.

[0093] The expression optimization model training method provided by the embodiment of the present invention takes into account the different accuracy indications of different facial regions. For facial regions with higher accuracy indications, an expression learning network with a higher network calculation difficulty coefficient is used for learning, and for facial regions with lower accuracy indications, an expression learning network with a lower network calculation difficulty coefficient is used for learning. Since the model size of the expression learning network with a lower network calculation difficulty coefficient is smaller, compared with the expression optimization model obtained by learning all facial regions using an expression learning network with a higher network calculation difficulty coefficient, the model size of the expression optimization model obtained by training multiple expression learning networks is reduced, thereby achieving compression of the expression optimization model and improving the overall fluency of the model.

[0094] An embodiment of the present invention provides an expression optimization model training method, which includes the following steps.

[0095] 301. A computer device obtains a plurality of accuracy indication marks of preset expression regions, wherein the plurality of accuracy indication marks are used to represent different accuracy indications of a plurality of facial regions of the preset expression regions, and one accuracy indication mark corresponds to at least one facial region.

[0096] In some embodiments, the sizes of the multiple facial regions in the preset expression area are different, that is, the occupancy rates of the multiple facial regions in the preset expression area are different. It can be understood that the larger the facial region, the higher the occupancy rate in the preset expression area, and the easier it is to focus on the facial region when the expression is displayed in the preset expression area, so the accuracy indication of these facial regions is higher. The smaller the facial region, the lower the occupancy rate in the preset expression area, and the less likely it is to focus on the facial region when the expression is displayed in the preset expression area, so the accuracy indication of these facial regions is lower.

[0097] In one possible implementation, a computer device obtains multiple accuracy indication marks for a preset expression area, including: the computer device obtains occupancy rates of multiple facial areas in the preset expression area, and obtains multiple accuracy indication marks corresponding to the multiple facial areas based on the occupancy rates of the multiple facial areas, wherein the accuracy indication represented by the accuracy indication mark corresponding to the facial area has a positive feedback relationship with the occupancy rate.

[0098] Optionally, the computer device obtains multiple accuracy indicators corresponding to the multiple facial regions based on the occupancy rates of the multiple facial regions, including: clustering the occupancy rates of the multiple facial regions to obtain multiple clusters, and obtaining an accuracy indicator for each cluster, the accuracy indicator being used to indicate the accuracy indication of the facial region corresponding to each occupancy rate in the cluster. Optionally, the computer device obtains multiple accuracy indicators corresponding to the multiple facial regions based on the occupancy rates of the multiple facial regions, including: obtaining the accuracy indicator corresponding to each facial region based on the association information between the occupancy rate range and the accuracy indicator and the occupancy rates of the multiple facial regions.

[0099] In one possible implementation, a computer device obtains multiple accuracy indication marks for preset expression areas, including: obtaining key indicator data for multiple facial areas in the preset expression area based on functional information of the preset expression area, and obtaining accuracy indication marks corresponding to the multiple facial areas based on the key indicator data of the multiple facial areas, wherein the accuracy indication represented by the accuracy indication mark corresponding to the facial area and the importance represented by the key indicator data of the facial area are in a positive feedback relationship.

[0100] Optionally, the computer device obtains the accuracy indicator corresponding to the multiple facial regions based on the key indicator data of the multiple facial regions, including: clustering the key indicator data of the multiple facial regions to obtain multiple clusters, and obtaining an accuracy indicator for each cluster, the accuracy indicator being used to indicate the accuracy indication of the facial region corresponding to each key indicator data in the cluster. Optionally, the computer device obtains the accuracy indicator corresponding to the multiple facial regions based on the key indicator data of the multiple facial regions, including: obtaining the accuracy indicator corresponding to each facial region based on the association information between the key indicator data range and the accuracy indicator and the key indicator data of the multiple facial regions.

[0101] In one possible implementation, a computer device obtains multiple accuracy indication marks for a preset expression area, including: obtaining occupancy rates of multiple facial areas in the preset expression area; obtaining key indicator data of multiple facial areas in the preset expression area based on functional information of the preset expression area; obtaining multiple accuracy indication marks corresponding to the multiple facial areas based on the occupancy rates and key indicator data of the multiple facial areas, the accuracy indication represented by the accuracy indication mark corresponding to the facial area is in a positive feedback relationship with the occupancy rate, and the accuracy indication represented by the accuracy indication mark corresponding to the facial area is a positive feedback relationship.

[0102] 302. The computer device obtains a plurality of demand calculation difficulty coefficients according to the plurality of accuracy indication identifiers, wherein one demand calculation difficulty coefficient corresponds to one accuracy indication identifier, and the demand calculation difficulty coefficient is positively correlated with the accuracy indication represented by the corresponding accuracy indication identifier.

[0103] The higher the accuracy indicator of the facial region, the more accurate the expression parameters generated by the model for the facial region. In order for the model to generate accurate expression parameters for the facial region, the model needs to learn the precise representation of each expression, for example, the model can learn more dimensional representations of expressions, etc. Therefore, the model needs to have more hyperparameter network architecture data.

[0104] In one possible implementation, a computer device obtains multiple requirement calculation difficulty coefficients based on the multiple accuracy indication identifiers, including: for any accuracy indication identifier, obtaining the accuracy indication value indicated by the accuracy indication identifier, and obtaining the requirement calculation difficulty coefficient based on the accuracy indication value, wherein the requirement calculation difficulty coefficient is positively correlated with the accuracy indication value.

[0105] For example, if the accuracy indicator value is S, the required calculation difficulty coefficient is 100%, and the required calculation difficulty coefficient of 100% indicates the calculation difficulty coefficient of the expression optimization model being used in the related technology; if the accuracy indicator value is A, the required calculation difficulty coefficient is 80%; if the accuracy indicator value is B, the required calculation difficulty coefficient is 50%, and so on.

[0106] Optionally, the computer device obtains the demand calculation difficulty coefficient according to the accuracy indication value, including: the computer device obtains the demand calculation difficulty coefficient corresponding to the accuracy indication value indicated by the accuracy indication identifier according to the association information between the accuracy indication value and the network calculation difficulty coefficient. Optionally, the computer device obtains the demand calculation difficulty coefficient according to the accuracy indication value, including: the computer device processes the accuracy indication value according to a first algorithm to obtain the demand calculation difficulty coefficient. The first algorithm can be any statistical algorithm, and the embodiment of the present invention does not limit the first algorithm.

[0107] In some embodiments, different accuracy indicators correspond to different numbers of facial regions. In an embodiment of the present invention, facial regions corresponding to the same accuracy indicator are learned through a first expression learning network. If a certain accuracy indicator corresponds to more facial regions, the requirements for the first expression learning network to learn the expressions of these facial regions are higher, and the network calculation difficulty coefficient of the first expression learning network is higher. Optionally, the computer device obtains multiple required calculation difficulty coefficients based on the multiple accuracy indicators, including: for any accuracy indicator, obtaining a first number of facial regions corresponding to the accuracy indicator, and obtaining the required calculation difficulty coefficient based on the first number, wherein the required calculation difficulty coefficient is positively correlated with the first number.

[0108] Optionally, the computer device obtains the demand calculation difficulty coefficient according to the first number, including: the computer device obtains the demand calculation difficulty coefficient corresponding to the first number according to the association information between the number of facial regions and the network calculation difficulty coefficient. Optionally, the computer device obtains the demand calculation difficulty coefficient according to the first number, including: the computer device processes the first number according to a second algorithm to obtain the demand calculation difficulty coefficient. The second algorithm can be any statistical algorithm, and the embodiment of the present invention does not limit the second algorithm.

[0109] In some embodiments, a computer device obtains multiple requirement calculation difficulty coefficients based on the multiple accuracy indication identifiers, including: for any accuracy indication identifier, obtaining the accuracy indication value indicated by the accuracy indication identifier and a first number of facial areas corresponding to the accuracy indication identifier; obtaining the requirement calculation difficulty coefficient based on the accuracy indication value and the first number, the requirement calculation difficulty coefficient is positively correlated with the accuracy indication value, and the requirement calculation difficulty coefficient is positively correlated with the first number.

[0110] 303. The computer device generates a plurality of first expression learning networks whose network calculation difficulty coefficients are respectively the plurality of required calculation difficulty coefficients, and the plurality of first expression learning networks are respectively used to learn expressions of facial regions indicated by different accuracy.

[0111] In some embodiments, the higher the network computing difficulty coefficient of the first expression learning network, the greater the number of hyperparameter network architecture data of the first expression learning network. Optionally, the computer device determines multiple first expression learning networks whose numbers of hyperparameter network architecture data respectively match multiple required computing difficulty coefficients, and the number of hyperparameter network architecture data of the first expression learning network and the network computing difficulty coefficient represented by the required computing difficulty coefficient are in a positive feedback relationship.

[0112] It should be noted that the first expression learning network in the embodiment of the present invention can be any deep learning model, and the embodiment of the present invention does not limit the first expression learning network. Optionally, the first expression learning network can be an expert expression learning network, which includes multiple first expression prediction architectures, each of which is responsible for learning a type of expression.

[0113] In some embodiments, the first expression learning network includes a plurality of first expression prediction architectures. The computer device generates a plurality of first expression learning networks whose network calculation difficulty coefficients are respectively a plurality of required calculation difficulty coefficients, including: for any required calculation difficulty coefficient, according to the number of expression types of a preset expression area, generating a first expression learning network whose number of first expression prediction architectures is the same as the number of expression types and the number of hyperparameter network architecture data of the first expression prediction architecture matches the required calculation difficulty coefficient, the first expression prediction architecture being used to learn the expressions of the facial area under different expression types.

[0114] 304. The computer device determines a first weight allocation network, where the first weight allocation network is used to obtain weights of expression parameters output by the expression learning network.

[0115] The expression learning network may be the first expression learning network or the third expression learning network. In the embodiment of the present invention, the weight allocation network is used to obtain the weight of the expression parameter output by the expression learning network. For example, after the expression learning network predicts the expression parameter of the facial area at the next moment, the expression parameter can be input into the weight allocation network, and the weight allocation network obtains the weight of the expression parameter, processes the expression parameter according to the weight, and then outputs the processed expression parameter.

[0116] In some embodiments, the weight distribution network is a gate network. It should be noted that the embodiment of the present invention only uses the gate network as an example to exemplify the weight distribution network, and does not limit the weight distribution network. The weight distribution network can also be other networks.

[0117] It should be noted that, in the embodiment of the present invention, the first weight allocation network is used to represent the weight allocation network obtained before training according to the training expression parameters, and the second weight allocation network is used to represent the weight allocation network obtained after training according to the training expression parameters.

[0118] It should be noted that the expression parameters output by each first expression learning network are processed by the acquisition weight allocation network. Among them, multiple first expression learning networks can correspond to one acquisition weight allocation network respectively, or can share one acquisition weight allocation network.

[0119] In one possible implementation, a computer device determines a first weight distribution network, including: determining first weight distribution networks corresponding to multiple first expression learning networks, respectively, and the first weight distribution network corresponding to any first expression learning network is used to obtain the weight of the expression parameters output by the first expression learning network; or, determining the first weight distribution network corresponding to the multiple first expression learning networks, and the first weight distribution network is used to obtain the weight of the expression parameters output by the multiple first expression learning networks.

[0120] 305. For any first expression learning network, the computer device determines, from the training expression parameters of the preset expression area, the training expression parameters of the first facial area corresponding to the first accuracy indicator, the first accuracy indicator corresponding to the required calculation difficulty coefficient of the first expression learning network, and performs an iterative training operation on the first expression learning network and the first weight allocation network according to the training expression parameters of the first facial area to obtain a third expression learning network and a second weight allocation network.

[0121] In an embodiment of the present invention, the first facial area is a facial area in the preset expression area that corresponds to the same accuracy indication identifier as the first expression learning network. Therefore, the learning effect of the first expression learning network on the expression matches the accuracy indication of the first facial area. Therefore, the first expression learning network can be iteratively trained based on the training expression parameters of the first facial area so that the first expression learning network learns the expression of the first facial area.

[0122] The training expression parameters of the first facial area are relatively high-precision expression parameters of the first facial area. It should be noted that the relatively high-precision expression parameters here refer to the accuracy of the expression parameters, rather than that the expression parameters include more types of expression parameters. In some embodiments, the training expression parameters of the first facial area are expression parameters recorded by the expression capture device when a professional expression actor wears an expression capture device and makes the expressions required in the game. In other embodiments, the training expression parameters of the first facial area are expression parameters of the first facial area of ​​a virtual character during the game. The embodiment of the present invention does not limit the method for determining the training expression parameters.

[0123] In some embodiments, the training expression parameters of the facial region include the training expression parameters of the facial region within a preset period range and the second training expression parameters of the facial region in the next period of the preset period range. The computer device performs an iterative training operation on the first expression learning network and the first weight allocation network according to the training expression parameters of the first facial region to obtain a third expression learning network and a second weight allocation network, including: processing the first training expression parameters of the first facial region through the first expression learning network to obtain the first expression parameters to be processed of the first facial region, the first expression parameters to be processed being the expression parameters of the next period of the preset period range obtained by the first expression learning network for the first facial region; processing the first expression parameters to be processed through the first weight allocation network to obtain the second expression parameters to be processed; performing an iterative training operation on the first expression learning network and the first weight allocation network according to the second expression parameters to be processed and the second training expression parameters of the first facial region to obtain the third expression learning network and the second weight allocation network.

[0124] Optionally, the computer device performs an iterative training operation on the first expression learning network and the second weight allocation network according to the second expression parameter to be processed and the second training expression parameter of the first facial area to obtain a third expression learning network and a second weight allocation network, including: the computer device performs an iterative training operation on the first expression learning network and the second weight allocation network according to the difference between the second expression parameter to be processed and the second training expression parameter to obtain a third expression learning network and a second weight allocation network, so as to converge the errors of the third expression learning network and the second weight allocation network.

[0125] In some embodiments, it is considered that changes in some facial regions are related to changes in other facial regions, for example, changes in the mouth and eye regions are related in a smiling expression scenario. In an embodiment of the present invention, different first expression learning networks are trained for different facial regions according to different accuracy indications of different facial regions. Therefore, since the mouth and eyes are not of the same accuracy indication, two first expression learning networks will be used to learn the mouth and eyes respectively. Taking the first expression learning network for learning mouth expressions as an example, the first expression learning network is trained using the training expression parameters of the mouth. In order to enable the first expression learning network to learn the expressions of facial regions more accurately, expression parameters of other facial regions may also be input when training the first expression learning network and using the first expression learning network.

[0126] Taking the training process as an example, in a possible implementation, the computer device performs an iterative training operation on the first expression learning network and the first weight allocation network according to the training expression parameters of the first facial area to obtain the third expression learning network and the second weight allocation network, including: the computer device performs an iterative training operation on the first expression learning network and the first weight allocation network according to the training expression parameters of the first facial area and the training expression parameters of the second facial area to obtain the third expression learning network and the second weight allocation network. Wherein, the second facial area is a facial area related to the expression of the first facial area, or the second facial area is a facial area that affects the expression of the first facial area.

[0127] In some embodiments, when a preset expression region is expressing an expression, the expression amplitudes of the multiple facial regions of the preset expression region are different. For some facial regions with larger expression amplitudes, since the expression amplitudes of the facial regions are larger, the expression of the facial regions is more likely to attract people's attention, or the expression of the facial regions is more important, and therefore, the accuracy indication of the facial regions is higher. For some facial regions with smaller expression amplitudes, since the expression amplitudes of the facial regions are smaller, the expression of the facial regions is not easy to attract people's attention, or the expression of the facial regions is not very important, and therefore, the accuracy indication of the facial regions is lower. Among them, the facial regions with larger expression amplitudes will affect the expression of the facial regions with smaller expression amplitudes. Therefore, when the computer device trains the expression learning network corresponding to the low-precision facial regions, it can refer to the high-precision facial regions.

[0128] In one possible implementation, a computer device performs an iterative training operation on a first expression learning network and a first weight allocation network according to training expression parameters of a first facial region to obtain a third expression learning network and a second weight allocation network, including: the computer device obtains a second facial region, which is a facial region other than the first facial region in preset expression regions; determines training expression parameters of the second facial region from the training expression parameters of the preset expression regions; performs an iterative training operation on the first expression learning network and the first weight allocation network according to the training expression parameters of the first facial region and the training expression parameters of the second facial region to obtain a third expression learning network and a second weight allocation network.

[0129] Optionally, the computer device acquires the second facial region, including: the computer device acquires the second facial region that affects the expression of the first facial region from the preset expression region according to a first association relationship, wherein the first association relationship is used to represent the association relationship of the expressions of the multiple facial regions. Optionally, the first association relationship records which facial regions have an association relationship.

[0130] Optionally, the computer device obtains the second facial region, including: the computer device obtains the occupancy rate of multiple facial regions in the preset expression region in the preset expression region, and obtains the facial region whose occupancy rate exceeds the first occupancy rate as the second facial region. The first occupancy rate may be any occupancy rate, for example, 5%, 10%, etc. Optionally, the first occupancy rate is an empirical value. Optionally, the first occupancy rate is a value set by a technician.

[0131] Optionally, the computer device obtains the second facial area, including: the computer device obtains at least one target accuracy indicator based on the accuracy indicator corresponding to the first facial area, the accuracy indication indicated by the target accuracy indicator is greater than the accuracy indication indicated by the accuracy indicator corresponding to the first facial area; and obtains the facial area corresponding to the at least one target accuracy indicator as the second facial area.

[0132] It should be noted that the computer device can perform an iterative training operation on each first expression learning network according to the training expression parameters of the first facial region and the training expression parameters of the second facial region; the computer device can also perform an iterative training operation on some first expression learning networks according to the training expression parameters of the first facial region and the training expression parameters of the second facial region according to actual needs. The embodiments of the present invention are not limited to this.

[0133] In some embodiments, the training expression parameters of the facial region include a first training expression parameter of the facial region within a preset period range and a second training expression parameter of the facial region in the next period of the preset period range. The computer device performs an iterative training operation on the first expression learning network and the first weight allocation network according to the training expression parameters of the first facial region and the training expression parameters of the second facial region to obtain a third expression learning network and a second weight allocation network, including: the computer device processes the first training expression parameter of the first facial region and the first training expression parameter of the second facial region through the first expression learning network to obtain a first expression parameter to be processed of the first facial region, the first expression parameter to be processed is the expression parameter of the next period of the preset period range obtained by the first expression learning network for the first facial region; the first expression parameter to be processed is processed through the first weight allocation network to obtain a second expression parameter to be processed; according to the second expression parameter to be processed and the second training expression parameter of the first facial region, the first expression learning network and the first weight allocation network are performed iterative training operations to obtain a third expression learning network and a second weight allocation network.

[0134] The preset period range and the next period of the preset period range may be periods of any length, for example, 1 second, 2 seconds, or any other length.

[0135] The computer device performs an iterative training operation on the first expression learning network according to the second expression parameter to be processed and the second training expression parameter of the first facial area to obtain a second expression learning network, including: the computer device performs an iterative training operation on the first expression learning network according to the difference between the second expression parameter to be processed and the second training expression parameter to obtain a second expression learning network.

[0136] 306. The computer device obtains a second expression learning network according to the third expression learning network and the second weight allocation network, where the second expression learning network is used to learn the expression of the first facial area.

[0137] In the embodiment of the present invention, the weight distribution network is used to obtain the weights of the expression parameters output by the expression learning network, that is, the expression parameters output by the expression learning network need to be processed by the weight distribution network before being output. Therefore, the expression learning network can be further processed according to the weight distribution network so that the expression parameters output by the expression learning network can be used directly.

[0138] In some embodiments, the computer device obtains the second expression learning network according to the third expression learning network and the second weight allocation network, including: performing an integration operation on the third expression learning network and the second weight allocation network to obtain the second expression learning network.

[0139] In some embodiments, the computer device acquires the second expression learning network according to the third expression learning network and the second weight allocation network, including: processing the third expression learning network according to the second weight allocation network to obtain the second expression learning network. For example, adjusting the hyperparameter network architecture data of the third expression learning network according to the second weight allocation network.

[0140] In some embodiments, the first expression learning network includes a plurality of first expression prediction frameworks, and the plurality of first expression prediction frameworks are used to learn expressions of different expression types of facial regions. The third expression learning network includes a plurality of second expression prediction frameworks, and the plurality of second expression prediction frameworks are obtained by training a plurality of first expression prediction frameworks. In one possible implementation, the computer device obtains the second expression learning network according to the third expression learning network and the second weight allocation network, including: the computer device obtains the weights of the plurality of second expression prediction frameworks according to the second weight allocation network, and according to the weights of the plurality of second expression prediction frameworks, performs an integration operation on the hyperparameter network framework data of the plurality of second expression prediction frameworks to obtain the second expression learning network.

[0141] 307. The computer device obtains an expression optimization model according to the multiple second expression learning networks obtained through training.

[0142] In an embodiment of the present invention, each second expression learning network is used to generate expression parameters for a facial region corresponding to the same accuracy indicator as the second expression learning network. The expression optimization model is used to generate expression parameters for multiple facial regions of the subject. In some embodiments, the computer device obtains the expression optimization model based on multiple second expression learning networks obtained through training, including: performing an integration operation on the multiple second expression learning networks to obtain the expression optimization model.

[0143] Among them, the integration operation of multiple second expression learning networks includes: deploying multiple second expression learning networks into the same model file, which is the model file of the expression optimization model, and the computer device simultaneously runs the multiple second expression learning networks according to the model file, thereby generating expression parameters for multiple facial areas.

[0144] The expression optimization model training method provided by the embodiment of the present invention takes into account the different accuracy indications of different facial regions. For facial regions with higher accuracy indications, an expression learning network with a higher network calculation difficulty coefficient is used for learning, and for facial regions with lower accuracy indications, an expression learning network with a lower network calculation difficulty coefficient is used for learning. Since the model size of the expression learning network with a lower network calculation difficulty coefficient is smaller, compared with the expression optimization model obtained by learning all facial regions using an expression learning network with a higher network calculation difficulty coefficient, the model size of the expression optimization model obtained by training multiple expression learning networks is reduced, thereby achieving compression of the expression optimization model and improving the overall fluency of the model.

[0145] In addition, in the process of training the expression learning network, the embodiment of the present invention introduces a weight distribution network. By obtaining the weight distribution network to perform weighted processing on the expression parameters output by the expression learning network, the transition between different expressions can be natural, thereby improving the expression effect of the facial area.

[0146] It should be noted that the embodiment of the present invention only takes the example of obtaining a weight distribution network to assist in training an expression learning network to exemplify the training process of the expression learning network. In another embodiment, the expression learning network may be trained without obtaining a weight distribution network. In one possible implementation, the computer device performs an iterative training operation on any first expression learning network according to the training expression parameters of the first facial region to obtain a second expression learning network, the second expression learning network being used to learn the expression of the first facial region, the first facial region being a facial region corresponding to the same accuracy indicator as the first expression learning network; and obtains an expression optimization model based on multiple second expression learning networks obtained through training.

[0147] In some embodiments, a computer device performs an iterative training operation on a first expression learning network according to training expression parameters of a first facial region to obtain a second expression learning network, including: obtaining a second facial region, where the second facial region is a facial region other than the first facial region in preset expression regions; determining training expression parameters of the second facial region from the training expression parameters of the preset expression regions; performing an iterative training operation on the first expression learning network according to the training expression parameters of the first facial region and the training expression parameters of the second facial region to obtain a second expression learning network.

[0148] In some embodiments, the first expression learning network includes multiple first expression prediction frameworks, and the multiple first expression prediction frameworks are used to learn expressions of different expression types. The following takes the first expression learning network including multiple first expression prediction frameworks as an example to exemplify the process of training the first expression learning network to obtain the second expression learning network.

[0149] In one possible implementation, the training expression parameters of the facial area include sample expression types, and the computer device performs an iterative training operation on the first expression learning network according to the training expression parameters of the first facial area to obtain a second expression learning network, including: the computer device inputs the training expression parameters of the first facial area into a first expression prediction architecture that matches the sample expression type in the training expression parameters of the first facial area, and the first expression prediction architecture is used to learn expressions of the expression type; according to the training expression parameters of the first facial area, an iterative training operation is performed on the first expression prediction architecture to obtain a second expression prediction architecture; and according to multiple trained second expression prediction architectures, a second expression learning network is obtained.

[0150] It should be noted that when training the first expression learning network, training expression parameters of different expression types will be used to perform iterative training operations on the first expression learning network so that the first expression learning network can learn expressions of different expression types in the first facial area. Therefore, multiple first expression prediction architectures in the first expression learning network can be trained.

[0151] It should be noted that when the computer device trains multiple first expression prediction architectures to obtain multiple second expression prediction architectures, the first expression learning network can be called a third expression learning network. That is, after the computer device performs iterative training operations on multiple first expression prediction architectures in the first expression learning network, a third expression learning network is obtained, and the third expression learning network includes the multiple second expression prediction architectures obtained through training.

[0152] In some embodiments, the computer device obtains the second expression learning network based on the third expression learning network. Since the third expression learning network includes multiple second expression prediction architectures, each second expression prediction architecture is used to generate an expression of one type of expression. Therefore, the number of hyperparameter network architecture data of the third expression learning network is large, and the model size is large. The third expression learning network can be compressed to obtain the second expression learning network. In a possible implementation, the computer device performs an integration operation on multiple second expression prediction architectures in the third expression learning network to obtain the second expression learning network. Optionally, the second expression learning network includes an expression prediction architecture, which is used to generate expression parameters of different expression types for the first facial area.

[0153] Optionally, the computer device performs an integration operation on the multiple second expression prediction architectures in the third expression learning network, including: weighting the corresponding hyperparameter network architecture data in the multiple second expression prediction architectures.

[0154] In some embodiments, the expression learning network is assisted in performing iterative training operations by acquiring a weight distribution network, so that when the multiple second expression prediction architectures in the third expression learning network are integrated, the weights of the multiple second expression prediction architectures can be acquired according to the weight distribution network. For example, the weights of the hyperparameter network architecture data in the multiple second expression prediction architectures are acquired.

[0155] In a possible implementation, the third expression learning network includes a plurality of second expression prediction frameworks, the plurality of second expression prediction frameworks are used to learn expressions of different expression types, and the second weight allocation network is used to obtain weights of expression parameters output by the plurality of second expression prediction frameworks. Optionally, the computer device obtains the second expression learning network according to the third expression learning network and the second weight allocation network, including: obtaining weights of the plurality of second expression prediction frameworks according to the second weight allocation network; performing an integration operation on the plurality of second expression prediction frameworks according to the weights of the plurality of second expression prediction frameworks to obtain the second expression learning network.

[0156] In some embodiments, multiple first expression learning networks correspond to a first weight allocation network respectively. The following takes "multiple first expression learning networks correspond to a first weight allocation network respectively" as an example to exemplify the process of obtaining a weight allocation network to assist expression learning network training.

[0157] In one possible implementation, a computer device determines first weight allocation networks corresponding to multiple first expression learning networks respectively; for any first expression learning network, an iterative training operation is performed on the first expression learning network and the first weight allocation network corresponding to the first expression learning network according to a training expression parameter of a first facial region, to obtain a third expression learning network and a second weight allocation network, wherein the first facial region is a facial region corresponding to the same accuracy indicator as the first expression learning network; a second expression learning network is obtained according to the third expression learning network and the second weight allocation network; and the expression optimization model is obtained according to the multiple second expression learning networks obtained through training.

[0158] Optionally, the training expression parameters of the facial area include a first training expression parameter of the facial area within a preset period range and a second training expression parameter of the facial area in the next period of the preset period range. The computer device performs an iterative training operation on the first expression learning network and the first weight allocation network corresponding to the first expression learning network according to the training expression parameters of the first facial area to obtain a third expression learning network and a second weight allocation network, including: processing the first training expression parameter of the first facial area through the first expression learning network to obtain a first expression parameter to be processed of the first facial area, the first expression parameter to be processed being the expression parameter of the next period of the preset period range obtained by the first expression learning network for the first facial area; processing the first expression parameter to be processed through the first weight allocation network to obtain a second expression parameter to be processed; performing an iterative training operation on the first expression learning network and the first weight allocation network according to the second expression parameter to be processed and the second training expression parameter to obtain a third expression learning network and a second weight allocation network.

[0159] The first weight distribution network is used to obtain the weight of the expression parameter output by the first expression learning network. In some embodiments, the computer device processes the first expression parameter to be processed through the first weight distribution network to obtain the second expression parameter to be processed, including: the computer device processes the first expression parameter to be processed through the first weight distribution network to obtain the weight of the first expression parameter to be processed; and performs weighted processing on the first expression parameter to be processed according to the weight of the first expression parameter to be processed to obtain the second expression parameter to be processed.

[0160] It should be noted that when the computer device processes the first expression parameter to be processed through the first weight distribution network and obtains the weight of the first expression parameter to be processed, the weight can be obtained based on part of the data in the first expression parameter to be processed, or the weight can be obtained based on all the data.

[0161] In some embodiments, the computer device processes the first expression parameter to be processed through a first weight distribution network to obtain the weight of the first expression parameter to be processed, including: the computer device processes the expression parameters of key points in the first expression parameter to be processed through the first weight distribution network to obtain the weight of the first expression parameter to be processed.

[0162] The key point may be the center point of the first facial region or any point of the first facial region. The embodiment of the present invention does not limit the key point.

[0163] In some embodiments, the third expression learning network includes multiple second expression prediction architectures, which are used to generate expressions for learning different types of expressions. The computer device obtains the second expression learning network based on the third expression learning network and the second weight allocation network, including: the computer device obtains the weights of the multiple second expression prediction architectures in the third expression learning network based on the second weight allocation network, and performs an integration operation on the multiple second expression prediction architectures based on the weights of the multiple second expression prediction architectures to obtain the second expression learning network.

[0164] That is to say, the computer device performs iterative training operations on multiple first expression prediction architectures in the first expression learning network through the training expression parameters of different expression types in the first facial area, and obtains multiple trained second expression prediction architectures, which is also the third expression learning network. Among them, when the computer device trains multiple first expression prediction architectures, it will input the training expression parameters into the corresponding first expression prediction architecture for processing according to the expression types of the training expression parameters, and the first weight allocation network will obtain weights for the expression parameters output by the first expression prediction architecture. It should be noted that in the embodiment of the present invention, the method for the first weight allocation network to obtain weights for the expression parameters output by different first expression prediction architectures is the same, which will not be repeated here.

[0165] When the computer device performs iterative training operations on any first expression learning network according to the training expression parameters and the first weight allocation network corresponding to the first expression learning network, it adopts the method shown in the current embodiment, which will not be repeated here.

[0166] In some embodiments, multiple first expression learning networks share a first weight distribution network. The following takes "multiple first expression learning networks share a first weight distribution network" as an example to exemplify the process of obtaining a weight distribution network to assist expression learning network training.

[0167] In a possible implementation, the training expression parameters of the facial region include a first training expression parameter of the facial region within a preset period range and a second training expression parameter of the facial region within a next period of the preset period range.

[0168] The computer device determines a first weight allocation network corresponding to a plurality of first expression learning networks; according to the association information between the facial regions and the first expression learning networks, the first training expression parameters of the plurality of facial regions are respectively input into the plurality of first expression learning networks, and the input first training expression parameters are respectively processed by the plurality of first expression learning networks to obtain first expression parameters to be processed of the plurality of facial regions output by the plurality of first expression learning networks, wherein the first expression parameters to be processed are expression parameters of the next cycle predicted by the first expression learning networks for the facial regions; the first expression parameters to be processed of the plurality of facial regions output by the plurality of first expression learning networks are processed by the first weight allocation network to obtain second expression parameters to be processed of the plurality of facial regions; for any first expression learning network, according to the second training expression parameters of the first facial region input into the first expression learning network and the second expression parameters to be processed of the first facial region, an iterative training operation is performed on the first expression learning network to obtain a third expression learning network, wherein the first facial region is a facial region corresponding to the same accuracy indicator as the first expression learning network. According to the second training expression parameters and the second expression parameters to be processed according to multiple facial areas, an iterative training operation is performed on the first weight allocation network to obtain a second weight allocation network; according to the multiple third expression learning networks and the second weight allocation network, multiple second expression learning networks are obtained; according to the multiple second expression learning networks, an expression optimization model is obtained.

[0169] For example, the plurality of facial regions include facial region 1, facial region 2, and facial region 3. The plurality of first expression learning networks include first expression learning network 1 and first expression learning network 2. Among them, facial region 1, facial region 2, and first expression learning network 1 correspond to the same accuracy indication mark, and facial region 3 and first expression learning network 2 correspond to the same accuracy indication mark.

[0170] When training multiple first expression learning networks according to the training expression parameters of the preset expression areas, the training expression parameters of the facial area 1 and the facial area 2 are input into the first expression learning network 1, and the training expression parameters of the facial area 3 are input into the first expression learning network 2. The first expression parameters to be processed of the multiple facial areas output by the first expression learning network 1 and the first expression learning network 2 are all input into the first weight allocation network, and the first weight allocation network processes the first expression parameters to be processed of the multiple facial areas to obtain the second expression parameters to be processed of the multiple facial areas; according to the training expression parameters and the second expression parameters to be processed according to the preset expression areas, the first weight allocation network is iteratively trained to obtain the second weight allocation network; according to the training expression parameters and the second expression parameters to be processed according to the facial area 1 and the training expression parameters and the second expression parameters to be processed according to the facial area 2, the first expression learning network 1 is iteratively trained to obtain the third expression learning network 1; according to the training expression parameters and the second expression parameters to be processed according to the facial area 3, the first expression learning network 2 is iteratively trained to obtain the third expression learning network 2.

[0171] Afterwards, according to the second weight allocation network, the weight of the second expression prediction architecture in each third expression learning network is obtained, and according to the weight obtained for the second expression prediction architecture according to the second weight allocation network, the second expression prediction architecture belonging to the same third expression learning network is integrated to obtain multiple second expression learning networks.

[0172] In an embodiment of the present invention, after obtaining an expression optimization model through model training, the expression optimization model can be deployed in a terminal. When the terminal runs a game application, the expression optimization model is called to generate expression parameters for a virtual character (such as a virtual character, a virtual animal, etc.) in the game application, and the virtual character is controlled to display an expression according to the expression parameters. The following is the terminal calling the expression optimization model to generate expression parameters for the virtual character in the game.

[0173] The embodiments of the present invention also provide the following examples.

[0174] 801. In response to a manipulation instruction for a virtual character in a virtual environment, a terminal determines a first expression parameter and a manipulation parameter of the virtual character, where the first expression parameter is an expression parameter of multiple facial regions of the virtual character at a current time node.

[0175] The current time node may be any frame in an expression segment. For example, in a real-time game scene, the current time node refers to the current frame.

[0176] 802. The terminal performs a matching operation on the manipulation parameter according to the first association information to obtain a second expression parameter, wherein the first association information is used to indicate a matching relationship between the manipulation parameter and the expression parameter.

[0177] 803. The terminal processes the first expression parameter and the second expression parameter through an expression optimization model to obtain a third expression parameter, where the third expression parameter is an expression parameter of multiple facial regions at a next time node of the current time node.

[0178] It should be noted that the expression optimization model in the above step 802 is trained through the aforementioned embodiment. Therefore, the model size of the expression optimization model is relatively small, and the computing resources consumed by the terminal to run the expression generation model are better, and will not cause application lag.

[0179] 804. The terminal controls the virtual character to display an expression according to the third expression parameter.

[0180] Since the third expression parameter is an expression parameter of multiple facial regions, the multiple facial regions of the virtual character can be controlled to express according to the corresponding expression parameters, so that the virtual character performs the corresponding expression.

[0181] The embodiment of the present invention provides a server 100, which includes a processor and a non-volatile memory storing computer instructions. When the computer instructions are executed by the processor, the server 100 executes the aforementioned training method for a user expression capture model. Figure 2 As shown, Figure 2 The server 100 is a block diagram of a structure of a server 100 provided in an embodiment of the present invention. The server 100 includes a memory 111, a processor 112 and a communication unit 113. To achieve data transmission or interaction, the memory 111, the processor 112 and the communication unit 113 are electrically connected to each other directly or indirectly.

[0182] For the purpose of illustration, the foregoing description is made with reference to specific embodiments. However, the above illustrative discussion is not intended to be exhaustive or to limit the present disclosure to the precise form disclosed. In accordance with the above teachings, numerous modifications and variations are possible. These embodiments are selected and described in order to best illustrate the principles of the present disclosure and its practical application, so that those skilled in the art can best utilize the present disclosure and utilize various embodiments with different modifications to suit the intended specific application. For the purpose of illustration, the foregoing description is made with reference to specific embodiments. However, the above illustrative discussion is not intended to be exhaustive or to limit the present disclosure to the precise form disclosed. In accordance with the above teachings, numerous modifications and variations are possible. These embodiments are selected and described in order to best illustrate the principles of the present disclosure and its practical application, so that those skilled in the art can best utilize the present disclosure and utilize various embodiments with different modifications to suit the intended specific application.

Claims

1. A method for training a user expression capture model, characterized in that: include: Determine a first training expression data set, a second training expression data set, and a third training expression data set, wherein samples in the first training expression data set are common expression data, samples in the second training expression data set are unconventional expression data, and the third training expression data set includes common expression data and unconventional expression data; Using the common expression data in the first training expression data set to perform an initial training operation on the original user expression capture model to obtain an advanced user expression capture model; An updated expression data set required for model updating is obtained from the first training expression data set, the second training expression data set and the third training expression data set by traversal polling. In the process of using the updated expression data in the updated expression data set to update the advanced user expression capture model, after filtering out the gradient vectors corresponding to the updated expression data that belong to the third training expression data set and include commonly used expressions, the gradient vectors corresponding to the remaining updated expression data in the updated expression data set are used to update the hyperparameters of the advanced user expression capture model until the loss function reaches a preset condition, thereby obtaining a user expression capture model for capturing commonly used expression data and unconventional expression data.

2. The method according to claim 1, characterized in that The method further comprises: Capturing the expression of a target user based on the user expression capture model to obtain target expression data; Determining a preset expression area in a virtual environment corresponding to the target expression data; The expression optimization model trained according to the target expression data is used to control the virtual character corresponding to the preset expression area to display an expression.

3. The method according to claim 2, characterized in that The method of controlling the virtual character corresponding to the preset expression area to display an expression by using the expression optimization model trained according to the target expression data comprises: Acquire multiple accuracy indication marks of preset expression regions, wherein the multiple accuracy indication marks are used to represent different accuracy indications of multiple facial regions of the preset expression region, and one accuracy indication mark corresponds to at least one facial region; According to the multiple accuracy indication identifiers, multiple demand calculation difficulty coefficients are obtained, one demand calculation difficulty coefficient corresponds to one accuracy indication identifier, and the demand calculation difficulty coefficient is positively correlated with the accuracy indication represented by the corresponding accuracy indication identifier; Generate a plurality of first expression learning networks whose network calculation difficulty coefficients are respectively the plurality of required calculation difficulty coefficients, and the plurality of first expression learning networks are respectively used to learn expressions of facial regions indicated by different accuracy levels; For any first expression learning network, determining a training expression parameter of a first facial area corresponding to a first accuracy indicator from the training expression parameters of the preset expression area, wherein the first accuracy indicator corresponds to a required calculation difficulty coefficient of the first expression learning network, and the training expression parameter is derived from the target expression data; performing an iterative training operation on the first expression learning network according to the training expression parameters of the first facial region to obtain a second expression learning network, wherein the second expression learning network is used to generate expression parameters for the first facial region; Acquire an expression optimization model according to the plurality of second expression learning networks obtained through training, wherein the expression optimization model is used to generate expression parameters of different precisions for the plurality of facial regions; In response to a manipulation instruction for a virtual character in a virtual environment, determining a first expression parameter and a manipulation parameter of the virtual character, wherein the first expression parameter is an expression parameter of multiple facial regions of the virtual character at a current time node; According to the first association information, a matching operation is performed on the manipulation parameter to obtain a second expression parameter, wherein the first association information is used to indicate a matching relationship between the manipulation parameter and the expression parameter; The first expression parameter and the second expression parameter are processed by the expression optimization model to obtain a third expression parameter, where the third expression parameter is an expression parameter of the multiple facial regions at a time node next to the current time node; According to the third expression parameter, the virtual character is controlled to display an expression.

4. The method according to claim 3, characterized in that The step of performing an iterative training operation on the first expression learning network according to the training expression parameter of the first facial region to obtain a second expression learning network comprises: Acquire a second facial region, where the second facial region is another facial region in the preset expression region except the first facial region; Determining the training expression parameters of the second facial area from the training expression parameters of the preset expression area; Performing an iterative training operation on the first expression learning network according to the training expression parameters of the first facial area and the training expression parameters of the second facial area to obtain a second expression learning network; The obtaining of the second facial region comprises: According to a first association relationship, obtaining a second facial region in the preset expression region that affects the expression of the first facial region, wherein the first association relationship is used to represent the association relationship between the expressions of the multiple facial regions; or Obtaining occupancy rates of a plurality of facial regions in the preset expression region, and obtaining a facial region whose occupancy rate exceeds the first occupancy rate as the second facial region; or, According to the accuracy indicator corresponding to the first facial region, obtaining at least one target accuracy indicator, the accuracy indication indicated by the target accuracy indicator being greater than the accuracy indication indicated by the accuracy indicator corresponding to the first facial region; obtaining the facial region corresponding to the at least one target accuracy indicator as the second facial region; The training expression parameters of the facial area include a first training expression parameter of the facial area within a preset period range and a second training expression parameter of the facial area in a next period of the preset period range; The method of performing an iterative training operation on the first expression learning network according to the training expression parameters of the first facial region and the training expression parameters of the second facial region to obtain a second expression learning network comprises: Processing the first training expression parameter of the first facial area and the first training expression parameter of the second facial area through the first expression learning network to obtain a first expression parameter to be processed of the first facial area, wherein the first expression parameter to be processed is an expression parameter of the next cycle of the preset cycle range obtained by the first expression learning network for the first facial area; According to the first expression parameter to be processed and the second training expression parameter of the first facial area, an iterative training operation is performed on the first expression learning network to obtain the second expression learning network.

5. The method according to claim 3, characterized in that: The obtaining, according to the multiple accuracy indication marks, multiple requirement calculation difficulty coefficients includes: For any accuracy indicator, obtain the accuracy indication value indicated by the accuracy indicator, and obtain the demand calculation difficulty coefficient according to the accuracy indication value, wherein the demand calculation difficulty coefficient is positively correlated with the accuracy indication value; or For any accuracy indicator, obtaining a first number of facial regions corresponding to the accuracy indicator, and obtaining the requirement calculation difficulty coefficient according to the first number, wherein the requirement calculation difficulty coefficient is positively correlated with the first number; The step of obtaining a plurality of accuracy indication marks of the preset expression area includes: Obtaining occupancy rates of multiple facial regions in the preset expression region, and obtaining multiple accuracy indication marks corresponding to the multiple facial regions according to the occupancy rates of the multiple facial regions, wherein the accuracy indications represented by the accuracy indication marks corresponding to the facial regions are in a positive feedback relationship with the occupancy rates; or According to the functional information of the preset expression area, key indicator data of multiple facial areas in the preset expression area are obtained, and according to the key indicator data of the multiple facial areas, accuracy indication marks corresponding to the multiple facial areas are obtained, and the accuracy indication represented by the accuracy indication mark corresponding to the facial area and the importance represented by the key indicator data of the facial area are in a positive feedback relationship.

6. The method according to claim 1, characterized in that The method of performing an initial training operation on the original user expression capture model using the common expression data in the first training expression data set to obtain an advanced user expression capture model includes: Determining commonly used expression data and corresponding expression classification identifiers from the first training expression data set; Extracting the expression feature vector corresponding to each of the commonly used expression data through the expression feature recognition layer in the original user expression capture model; Obtaining expression capture classification results corresponding to each of the commonly used expression data according to the expression feature vectors through the expression type classification layer in the original user expression capture model; Constructing an initial cost function according to the expression classification identifier and the expression capture classification result; After using the hyperparameters when the initial cost function is minimized as the hyperparameters for updating the original user expression capture model, return to the step of determining the common expression data from the first training expression data set to continue training until the training is completed.

7. The method according to claim 1, characterized in that The step of obtaining an updated expression data set required for model updating from the first training expression data set, the second training expression data set, and the third training expression data set comprises: Determine the data extraction rules set in advance; According to the data extraction rule, data extraction operations are performed on the first training expression data set, the second training expression data set and the third training expression data set respectively to obtain the updated expression data set required for model updating.

8. The method according to claim 1, characterized in that In the process of updating the advanced user expression capture model using the updated expression data in the updated expression data set, after filtering out the gradient vectors corresponding to the updated expression data belonging to the third training expression data set and including common expressions, the hyper parameters of the advanced user expression capture model are updated using the gradient vectors corresponding to the remaining updated expression data in the updated expression data set, including: Extracting the expression feature vector corresponding to each of the updated expression data through the expression feature recognition layer in the advanced user expression capture model; Obtaining expression capture classification results corresponding to each of the updated expression data according to the expression feature vectors through the expression type classification layer in the advanced user expression capture model; Constructing a common cost function according to the expression classification identifier corresponding to the updated expression data belonging to the first training expression data set in the updated expression data set and the expression capture classification result; constructing an unconventional cost function according to the expression classification identifier corresponding to the updated expression data belonging to the second training expression data set in the updated expression data set and the expression capture classification result; Constructing a comprehensive cost function according to the expression classification identifier corresponding to the updated expression data belonging to the third training expression data set in the updated expression data set and the expression capture classification result; Determine the cost allocation weights set in advance; Summing the common cost function, the unconventional cost function and the comprehensive cost function according to the cost distribution weight to obtain an updated cost function of the model; Obtaining a gradient vector corresponding to each of the updated expression data when the update cost function of the model is minimized; After filtering out the gradient vectors corresponding to the updated expression data that belong to the third training expression data set and include commonly used expressions from the gradient vectors corresponding to each of the updated expression data, the gradient vectors corresponding to the remaining updated expression data in the updated expression data set are used to update the hyperparameters of the advanced user expression capture model.

9. The method according to claim 1, characterized in that: In the process of updating the advanced user expression capture model using the updated expression data in the updated expression data set, after filtering out the gradient vectors corresponding to the updated expression data belonging to the third training expression data set and including common expressions, the hyper parameters of the advanced user expression capture model are updated using the gradient vectors corresponding to the remaining updated expression data in the updated expression data set, including: When the updated expression data sampled from the third training expression data set in the updated expression data set includes unconventional expression data, the updated expression data belonging to the third training expression data set and including unconventional expressions, the updated expression data belonging to the first training expression data set, and the gradient vectors corresponding to the updated expression data belonging to the second training expression data set are used to update the hyperparameters of the advanced user expression capture model; When the updated expression data sampled from the third training expression data set in the updated expression data set does not include unconventional expression data, the gradient vectors corresponding to the updated expression data belonging to the first training expression data set and the updated expression data belonging to the second training expression data set are used to update the hyperparameters of the advanced user expression capture model.

10. A server system, characterized in that: The method comprises a server, wherein the server is used to execute the method according to any one of claims 1 to 9.

Images