A face processing method, device, computer equipment and storage medium

Abstract

The present disclosure provides a facial processing method, device, computer equipment and storage medium, wherein the method comprises: inputting acquired facial image samples of real faces and screenshot samples of character faces corresponding to virtual characters into a facial processing model to be trained, respectively, to obtain first image features and first face pinching parameter information corresponding to the facial image samples, and second image features and second face pinching parameter information corresponding to the character face screenshot samples; based on the first image features and the second image features, determining image feature loss information; based on the first face pinching parameter information and the second face pinching parameter information, determining face pinching parameter loss information; based on the image feature loss information and the face pinching parameter loss information, obtaining a trained facial processing model. The disclosed embodiment uses the image feature loss information and the face pinching parameter loss information as domain loss, which can reduce the domain difference between the virtual character and the real facial image, and the virtual character has a higher similarity with the real facial image.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a face processing method, apparatus, computer equipment and storage medium. Background Art

[0002] With the rapid development of online games, more and more role playing games (RPGs) have emerged to meet users' demand for customized game characters. In RPGs, game characters are usually created through the face pinching function, which helps users get the corresponding game character appearance according to the real person's settings.

[0003] When creating corresponding game characters based on real face images, how to make the game characters created by the face-pinching function more similar to real faces is a problem worth studying. Summary of the invention

[0004] The embodiments of the present disclosure at least provide a face processing method, apparatus, computer equipment and storage medium.

[0005] In a first aspect, an embodiment of the present disclosure provides a face processing method, comprising:

[0006] Obtaining facial image samples of real human faces and screenshot samples of character faces corresponding to virtual characters;

[0007] Inputting the human face image sample and the character face screenshot sample into the face processing model to be trained respectively, obtaining the first image feature and the first face pinching parameter information corresponding to the human face image sample, and the second image feature and the second face pinching parameter information corresponding to the character face screenshot sample;

[0008] Determine image feature loss information based on the first image feature and the second image feature; and determine face pinching parameter loss information based on the first face pinching parameter information and the second face pinching parameter information;

[0009] Based on the image feature loss information and the face pinching parameter loss information, the weight parameter information in the face processing model to be trained is adjusted to obtain a trained face processing model; the trained face processing model is used to obtain the face pinching parameter information of the corresponding virtual character after inputting the face image to be processed of a real face.

[0010] In an optional implementation, after the human face image sample and the character face screenshot sample are respectively input into the face processing model to be trained and the first face pinching parameter information corresponding to the human face image sample is obtained, the method further includes:

[0011] Inputting the second face pinching parameter information of the character face screenshot sample into a pre-trained imitator to obtain a first face generated image;

[0012] Based on the first image information of the character face screenshot sample and the second image information of the first face generated image, obtaining face restoration loss information;

[0013] The step of adjusting the weight parameter information in the face processing model to be trained based on the image feature loss information and the face pinching parameter loss information to obtain a trained face processing model includes:

[0014] Based on the image feature loss information, the face pinching parameter loss information and the face restoration loss information, the weight parameter information in the face processing model to be trained is adjusted to obtain a trained face processing model.

[0015] In an optional implementation, after the human face image sample and the character face screenshot sample are respectively input into the face processing model to be trained and the first face pinching parameter information corresponding to the human face image sample is obtained, the method further includes:

[0016] Inputting the first face pinching parameter information of the face image sample into a pre-trained imitator to obtain a second face generated image;

[0017] Inputting the face image sample and the second face generated image into a pre-trained first face recognition model to obtain a third image feature of the face image sample and a fourth image feature of the second face generated image respectively; and inputting the face image sample and the second face generated image into a pre-trained second face recognition model to obtain a fifth image feature of the face image sample and a sixth image feature of the second face generated image respectively;

[0018] Determining first face consistency loss information based on the third image feature and the fourth image feature; and determining second face consistency loss information based on the fifth image feature and the sixth image feature;

[0019] The step of adjusting the weight parameter information in the face processing model to be trained based on the image feature loss information and the face pinching parameter loss information to obtain a trained face processing model includes:

[0020] Based on the image feature loss information, the face pinching parameter loss information, the first face consistency loss information and the second face consistency loss information, the weight parameter information in the face processing model to be trained is adjusted to obtain a trained face processing model.

[0021] In an optional implementation, the face image sample includes a plurality of face image pairs; each of the face image pairs includes two different face images of the same person;

[0022] For each face image pair, the first face pinching parameter information corresponding to the face image sample includes the third face pinching parameter information and the fourth face pinching parameter information corresponding to the two face images in the face image sample pair, and the fifth face pinching parameter information corresponding to any other face image;

[0023] After inputting the human face image sample and the character face screenshot sample into the face processing model to be trained respectively and obtaining the first face pinching parameter information corresponding to the human face image sample, the method further includes:

[0024] Determining face contrast loss information based on the third face pinching parameter information, the fourth face pinching parameter information, and the fifth face pinching parameter information;

[0025] The step of adjusting the weight parameter information in the face processing model to be trained based on the image feature loss information and the face pinching parameter loss information to obtain a trained face processing model includes:

[0026] Based on the image feature loss information, the face pinching parameter loss information and the face contrast loss information, the weight parameter information in the face processing model to be trained is adjusted to obtain a trained face processing model.

[0027] In an optional implementation, adjusting the weight parameter information in the face processing model to be trained based on the image feature loss information and the face pinching parameter loss information to obtain the trained face processing model includes:

[0028] Taking the derivative of the sum of the image feature loss information and the face pinching parameter loss information to obtain a gradient function of the weight parameters in the face processing model to be trained;

[0029] Substituting the current weight parameter information into the gradient function to obtain the gradient vector after iteration;

[0030] Based on the iterated gradient vector, the current weight parameter information is updated to obtain updated weight parameter information.

[0031] In an optional implementation, the human face image sample and the character face screenshot sample are respectively input into the face processing model to be trained to obtain the first image feature and the first face pinching parameter information corresponding to the human face image sample, and the second image feature and the second face pinching parameter information corresponding to the character face screenshot sample, including:

[0032] Inputting the human face image sample and the character face screenshot sample into the encoder of the face processing model to be trained respectively, downsampling the human face image sample and the character face screenshot sample respectively, and obtaining the seventh image feature and the eighth image feature of the human face image sample and the character face screenshot sample respectively under the first preset dimension;

[0033] Inputting the seventh image feature and the eighth image feature into the decoder of the face processing model to be trained respectively, upsampling the seventh image feature and the eighth image feature to obtain the ninth image feature and the tenth image feature of the face image sample and the character face screenshot sample respectively under the second preset dimension;

[0034] The seventh image feature and the ninth image feature, the eighth image feature and the tenth image feature are respectively input into the regressor of the face processing model to be trained to obtain the first image feature and the first face pinching parameter information corresponding to the face image sample, and the second image feature and the second face pinching parameter information corresponding to the character face screenshot sample.

[0035] In an optional implementation, before inputting the second face pinching parameter information of the character face screenshot sample into a pre-trained imitator to obtain the first face generated image, the method further includes:

[0036] Obtaining sixth face pinching parameter information of the character face screenshot sample;

[0037] The sixth face pinching parameter information of the character face screenshot sample is used as input and the character face screenshot sample is used as output to train the imitator to be trained, thereby obtaining a trained imitator.

[0038] In a second aspect, the present disclosure also provides a face processing method, including:

[0039] Obtaining a face image to be processed of a real face;

[0040] The facial image to be processed is input into a facial processing model obtained after training according to the first aspect, or the steps in any possible implementation manner of the first aspect, to obtain face pinching parameter information of the facial image to be processed; the face pinching parameter information is used to render a virtual character in a virtual scene.

[0041] In a third aspect, the present disclosure also provides a face processing device, including:

[0042] A first acquisition module is used to acquire facial image samples of real human faces and screenshot samples of character faces corresponding to virtual characters;

[0043] A first input module, used to input the human face image sample and the character face screenshot sample into the face processing model to be trained, respectively, to obtain the first image feature and first face pinching parameter information corresponding to the human face image sample, and the second image feature and second face pinching parameter information corresponding to the character face screenshot sample;

[0044] A first determining module, configured to determine image feature loss information based on the first image feature and the second image feature; and determine face pinching parameter loss information based on the first face pinching parameter information and the second face pinching parameter information;

[0045] The adjustment module is used to adjust the weight parameter information in the face processing model to be trained based on the image feature loss information and the face pinching parameter loss information to obtain a trained face processing model; the trained face processing model is used to obtain the face pinching parameter information of the corresponding virtual character after inputting the face image to be processed of a real face.

[0046] In a fourth aspect, the present disclosure also provides a face processing device, including:

[0047] An acquisition module, used for acquiring a face image to be processed of a real face;

[0048] An input module is used to input the facial image to be processed into a facial processing model obtained after training according to the first aspect or the steps in any possible implementation method of the first aspect, to obtain face pinching parameter information of the facial image to be processed; the face pinching parameter information is used to render a virtual character in a virtual scene.

[0049] In a fifth aspect, an embodiment of the present disclosure further provides a computer device, comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the computer device is running, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, the above-mentioned first aspect, or the steps in any possible implementation of the first aspect, or the steps in the above-mentioned second aspect are executed.

[0050] In a sixth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the computer program executes the above-mentioned first aspect, or the steps of any possible implementation of the first aspect, or executes the steps of the above-mentioned second aspect.

[0051] A face processing method provided by an embodiment of the present disclosure combines image feature loss information and face pinching parameter loss information to train a face processing model during a training process, wherein the image feature loss information is determined based on a first image feature from a real domain and a second image feature from a virtual domain, and the face pinching parameter loss information is determined based on first face pinching parameter information from the real domain and second face pinching parameter information from the virtual domain. The image feature loss information and the face pinching parameter loss information are used as domain losses, and the weight parameter information of the face processing model is adjusted based on the domain losses, which can reduce the domain difference between the rendered virtual character and the real face image to a certain extent. In this way, after the face image to be processed of the real face is input into the trained face processing model, the virtual character rendered based on the obtained face pinching parameter information can have a high similarity with the face image to be processed.

[0052] In order to make the above-mentioned objectives, features and advantages of the present disclosure more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following is a brief introduction to the drawings required for use in the embodiments. The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and are used together with the specification to illustrate the technical solutions of the present disclosure. It should be understood that the following drawings only illustrate certain embodiments of the present disclosure and should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can also be obtained based on these drawings without creative work.

[0054] Figure 1 A flow chart of a face processing method provided by an embodiment of the present disclosure is shown;

[0055] Figure 2 A flowchart of another face processing method provided by an embodiment of the present disclosure is shown;

[0056] Figure 3 A flowchart of another face processing method provided by an embodiment of the present disclosure is shown;

[0057] Figure 4 A schematic diagram of a face processing device provided by an embodiment of the present disclosure is shown;

[0058] Figure 5 A schematic diagram showing another face processing device provided by an embodiment of the present disclosure;

[0059] Figure 6 A schematic diagram of a computer device provided by an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0060] In order to make the purpose, technical scheme and advantages of the embodiments of the present disclosure clearer, the technical scheme in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. The components of the embodiments of the present disclosure generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure provided in the drawings is not intended to limit the scope of the present disclosure for protection, but merely represents the selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present disclosure.

[0061] In the schemes for creating game characters through face-pinching functions, real face images are usually input into pre-trained face recognition networks or face segmentation networks to obtain face-pinching parameters, and then the face-pinching parameters are adjusted to render the game characters. However, in the above schemes, the face recognition network or face segmentation network is not trained based on the difference between the real domain where the real face image is located and the virtual domain where the game character is located, resulting in obvious domain differences between the game characters rendered based on the obtained face-pinching parameters and the real face images.

[0062] Based on this, the present disclosure provides a face processing method. During the training process, the face processing model is trained by combining image feature loss information and face pinching parameter loss information, wherein the image feature loss information is determined based on a first image feature from a real domain and a second image feature from a virtual domain, and the face pinching parameter loss information is determined based on the first face pinching parameter information from the real domain and the second face pinching parameter information from the virtual domain. The image feature loss information and the face pinching parameter loss information are used as domain loss, and the weight parameter information of the face processing model is adjusted based on the domain loss. This can reduce the domain difference between the rendered virtual character and the real face image to a certain extent. In this way, after the face image to be processed of the real face is input into the trained face processing model, the virtual character rendered based on the obtained face pinching parameter information can have a high similarity with the face image to be processed.

[0063] The defects existing in the above schemes and the proposed solutions are the results obtained by the inventor after practice and careful research. Therefore, the discovery process of the above problems and the solutions proposed by the present disclosure for the above problems below should be the contributions made by the inventor to the present disclosure during the disclosure process.

[0064] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0065] To facilitate understanding of this embodiment, a face processing method disclosed in the embodiment of the present disclosure is first introduced in detail. The execution subject of the face processing method provided in the embodiment of the present disclosure is generally a computer device with certain computing capabilities.

[0066] The face processing method provided by the embodiment of the present disclosure is described below by taking the execution subject as a server as an example.

[0067] See also Figure 1 FIG. 1 is a flowchart of a face processing method provided by an embodiment of the present disclosure, the method comprising S101 to S104, wherein:

[0068] S101: Obtain facial image samples of real human faces and character face screenshot samples corresponding to virtual characters.

[0069] Here, before training the face processing model, accurate training samples are required, that is, facial image samples of real faces and character face screenshot samples corresponding to virtual characters. Among them, facial image samples of real faces can refer to facial photos, frame images, video screenshots, etc. of real people existing in the real world, such as user's own facial photos. Character face screenshot samples corresponding to virtual characters can refer to images obtained by taking screenshots of the faces of virtual characters in virtual scenes, such as facial screenshots of game characters.

[0070] S102: Input the human face image sample and the character face screenshot sample into the face processing model to be trained respectively, and obtain the first image feature and first face pinching parameter information corresponding to the human face image sample, and the second image feature and second face pinching parameter information corresponding to the character face screenshot sample.

[0071] The first image feature may refer to the feature of the face in the face image sample; the first face pinching parameter information may correspond to the multi-dimensional information of the face in the face image sample, which may specifically include facial information such as eyes, nose, mouth, eyebrows, etc. The second image feature may refer to the feature of the face in the character face screenshot sample; the second face pinching parameter information may correspond to the multi-dimensional information of the face in the character face screenshot sample, which may specifically include facial information such as eyes, nose, mouth, eyebrows, etc.

[0072] Inputting a human face image sample into the face processing model to be trained, the encoding model in the face processing model to be trained can extract a first image feature and a first face pinching parameter information corresponding to the human face image sample; and inputting a character face screenshot sample into the face processing model to be trained, the encoding model in the face processing model to be trained can extract a second image feature and a second face pinching parameter information corresponding to the character face screenshot sample.

[0073] In order to obtain the image features and face pinching parameter information in the face image samples and the character face screenshot samples more clearly and comprehensively, in one embodiment, the face image samples and the character face screenshot samples can be respectively input into the encoder of the face processing model to be trained, and the face image samples and the character face screenshot samples can be downsampled respectively to obtain the seventh image feature and the eighth image feature of the face image sample and the character face screenshot sample in the first preset dimension respectively; then, the seventh image feature and the eighth image feature are respectively input into the decoder of the face processing model to be trained, and the seventh image feature and the eighth image feature are upsampled to obtain the ninth image feature and the tenth image feature of the face image sample and the character face screenshot sample in the second preset dimension respectively; then, the seventh image feature and the ninth image feature, the eighth image feature and the tenth image feature are respectively input into the regressor of the face processing model to be trained to obtain the first image feature and the first face pinching parameter information corresponding to the face image sample, and the second image feature and the second face pinching parameter information corresponding to the character face screenshot sample.

[0074] The encoder-decoder structure composed of the encoder and the decoder can be used as a generator in the face processing model to be trained to obtain the first image features of the face image sample and the second image features of the character face screenshot sample. Exemplarily, the encoder-decoder structure can be a VQVAE or VQ-VAE-2 structure.

[0075] The encoder can downsample the face image samples and the character face screenshot samples respectively, perform dimensionality reduction processing on the face image samples or the character face screenshot samples, and obtain the seventh image feature of the face image samples under the first preset dimension and the eighth image feature of the character face screenshot samples under the first preset dimension. In one embodiment, the seventh image feature can also be quantized to obtain a discretized seventh image feature; at the same time, the eighth image feature can be quantized to obtain a discretized eighth image feature.

[0076] The decoder is a symmetrical structure similar to the encoder, and can upsample the face image samples and character face screenshot samples after dimensionality reduction processing, so that the face image samples and character face screenshot samples after dimensionality reduction processing return to their original size, that is, the ninth image feature and the tenth image feature of the face image sample and the character face screenshot sample under the second preset dimension, respectively. In one embodiment, the ninth image feature can also be quantized to obtain a discretized ninth image feature; the tenth image feature can also be quantized to obtain a discretized tenth image feature.

[0077] The seventh image feature and the ninth image feature are connected to obtain a first connected image feature, and the eighth image feature and the tenth image feature are connected to obtain a second connected image feature. Here, the first connected image feature and the second connected image feature can be three-dimensional multi-dimensional image features.

[0078] The regressor may be located at the last layer of the decoder, and the regressor may be composed of a preset number of convolutional layers. Based on the image features after the first connection, the first image features of the face image sample can be obtained in the second-to-last layer of the regressor, and the first face pinching parameter information corresponding to the face image sample can be obtained in the penultimate layer of the regressor. Based on the image features after the second connection, the second image features of the character face screenshot sample can be obtained in the second-to-last layer of the regressor, and the second face pinching parameter information corresponding to the character face screenshot sample can be obtained in the penultimate layer of the regressor.

[0079] S103: Determine image feature loss information based on the first image feature and the second image feature; and determine face pinching parameter loss information based on the first face pinching parameter information and the second face pinching parameter information.

[0080] Here, the maximum mean discrepancy (MMD) can be used to determine the difference between the vector mean of the first image feature and the vector mean of the second image feature, and the image feature loss information can be determined based on the difference. In addition, the MMD can be used to determine the difference between the vector mean of the first face pinching parameter information and the vector mean of the second face pinching parameter information, and the face pinching parameter loss information can be determined based on the difference.

[0081] S104: Based on the image feature loss information and the face pinching parameter loss information, adjust the weight parameter information in the face processing model to be trained to obtain a trained face processing model; the trained face processing model is used to obtain the face pinching parameter information of the corresponding virtual character after inputting the face image to be processed of a real face.

[0082] The weight parameter information in the embodiments of the present disclosure may refer to the weight parameters in the codec structure. The weight parameter information may be adjusted by minimizing the sum of the square of the image feature loss information and the square of the face pinching parameter loss information. Specifically, in one embodiment, the sum of the image feature loss information and the face pinching parameter loss information may be derived to obtain a gradient function of the weight parameter information in the face processing model to be trained. The current weight parameter information is then substituted into the gradient function to obtain an iterated gradient vector; wherein the gradient vector may indicate that the directional derivative of the loss information at this point has a maximum value along this direction. Finally, based on the iterated gradient vector, the current weight parameter information is updated to obtain updated weight parameter information.

[0083] The above implementation process is executed in each iteration, and so on, until the preset training cutoff condition is reached and the iteration is stopped to obtain the final updated weight parameter information, so that the distance between the first image feature and the second image feature is small enough, and the distance between the first face pinching parameter information and the second face pinching parameter information is small enough. The preset training cutoff condition can be a preset number of iterations, or the difference in loss information between two adjacent iterations is less than a set threshold, which is not specifically limited here.

[0084] When the preset training cutoff condition is reached, a trained face processing model can be obtained. After the face image to be processed of a real face is input into the trained face processing model, face pinching parameter information of the corresponding virtual character can be obtained. Based on the face pinching parameter information of the virtual character, the corresponding virtual character can be rendered.

[0085] By using the face pinching parameter information obtained by the trained face processing model in the above embodiment, the rendered virtual character is less different from the real face and has a higher similarity, which provides the user with a better gaming experience.

[0086] In order to ensure that the face pinching parameter information of the character face screenshot is obtained by using the trained face processing model, and the virtual character rendered by using the face pinching parameter information of the character face screenshot is indistinguishable from the input character face screenshot, in one embodiment, the weight parameter information in the face processing model to be trained can be adjusted by adding face restoration loss information during the training of the face processing model to be trained.

[0087] Specifically, after obtaining the second face pinching parameter information corresponding to the character face screenshot sample, the second face pinching parameter information of the character face screenshot sample can be input into a pre-trained imitator to obtain a first face generated image; then, based on the first image information of the character face screenshot sample and the second image information of the first face generated image, the face restoration loss information is obtained; finally, based on the image feature loss information, the face pinching parameter loss information and the face restoration loss information, the weight parameter information in the face processing model to be trained is adjusted to obtain a trained face processing model.

[0088] Among them, the pre-trained imitator can realize the function of the game engine, and render the first face generated image based on the second face pinching parameter information.

[0089] The first image information of the character face screenshot sample may refer to the color information of the character face screenshot sample in the three channels of red, green and blue; the second image information of the first face generated image may refer to the color information of the first face generated image in the three channels of red, green and blue.

[0090] Based on the first image information of the character face screenshot sample in vector form and the second image information of the first face generated image in vector form, the sum of squares of the difference between the two vectors, i.e., face restoration loss information, can be determined. By minimizing the sum of squares of the difference between the two vectors and adjusting the weight parameter information in the face processing model to be trained, the difference between the virtual character rendered by the face pinching parameter information of the character face screenshot and the input character face screenshot can be made smaller.

[0091] In one embodiment, the imitator can be trained by the following steps:

[0092] Obtain the sixth face pinching parameter information of the character face screenshot sample; use the sixth face pinching parameter information of the character face screenshot sample as input and the character face screenshot sample as output, train the imitator to be trained, and obtain a trained imitator.

[0093] The sixth face pinching parameter information may correspond to a multi-dimensional vector of a human face in the character face screenshot sample, and may specifically include facial information such as nose, eyes, and mouth. Here, the sixth face pinching parameter information may be derived from the client where the virtual character corresponding to the character face screenshot sample is located; it may also be obtained through the aforementioned face processing model to be trained, that is, the same as the second face pinching parameter information. Among them, the sixth face pinching parameter information obtained through the face processing model to be trained may have a problem of mismatching the sixth face pinching parameter information with the character face screenshot sample due to the imperfect face processing model to be trained, resulting in a poor training effect on the imitator.

[0094] In order to ensure that different facial image samples of the same real face, after obtaining the face pinching parameter information, the virtual characters rendered based on the face pinching parameter information are similar, in one embodiment, this can be achieved by adding face consistency loss information during the process of training the face processing model to be trained, and adjusting the weight parameter information in the face processing model to be trained.

[0095] Specifically, after obtaining the first face pinching parameter information corresponding to the character's facial image sample, the first face pinching parameter information of the facial image sample can be input into a pre-trained simulator to obtain a second face generated image; then, the facial image sample and the second face generated image are input into a pre-trained first face recognition model to obtain a third image feature of the facial image sample and a fourth image feature of the second face generated image, respectively; and the facial image sample and the second face generated image are input into a pre-trained second face recognition model to obtain a fifth image feature of the facial image sample and a sixth image feature of the second face generated image, respectively; next, based on the third image feature and the fourth image feature, determine the first face consistency loss information; and, based on the fifth image feature and the sixth image feature, determine the second face consistency loss information; finally, based on the image feature loss information, the face pinching parameter loss information, the first face consistency loss information and the second face consistency loss information, adjust the weight parameter information in the face processing model to be trained to obtain a trained face processing model.

[0096] Among them, the aforementioned imitators in the pre-trained imitators used here can be the same, so the training process of the imitator will not be described here in detail, and the specific training process can be found in the previous text.

[0097] Two face recognition models are used here, namely the first face recognition model and the second face recognition model. The two face recognition models can be different face recognition models. For example, the first face recognition model can be a mesh renderer, and the second face recognition model can be a face recognition network LightCNN-29v2.

[0098] The first face consistency loss information may represent the loss between the third image feature of the face image sample and the fourth image feature of the second face generated image. Based on the third image feature in vector form and the fourth image feature in vector form, the sum of the squares of the differences between the two vectors may be determined, i.e., the first face consistency loss information. The second face consistency loss information may represent the loss between the fifth image feature of the face image sample and the sixth image feature of the second face generated image. Based on the fifth image feature in vector form and the sixth image feature in vector form, the sum of the squares of the differences between the two vectors may be determined, i.e., the second face consistency loss information.

[0099] By minimizing the sum of the first face consistency loss information and the second face consistency loss information and adjusting the weight parameter information in the face processing model to be trained, the similarity between virtual characters rendered based on different face image samples of the same real face can be made higher.

[0100] In order to ensure that virtual characters rendered after obtaining face-pinching parameter information based on the same real face are more similar, and virtual characters rendered after obtaining face-pinching parameter information based on different real faces are less similar, in one embodiment, during the process of training the face processing model to be trained, face contrast loss information can be added to adjust the weight parameter information in the face processing model to be trained.

[0101] The face image samples may include multiple face image pairs; each face image pair may include two different face images of the same person.

[0102] For each face image pair, the first face pinching parameter information corresponding to the face image sample includes the third face pinching parameter information and the fourth face pinching parameter information corresponding to the two face images in the face image sample pair, and the fifth face pinching parameter information corresponding to any other face image.

[0103] After obtaining the first face pinching parameter information corresponding to the face image sample, the face contrast loss information can be determined based on the third face pinching parameter information, the fourth face pinching parameter information and the fifth face pinching parameter information; then, based on the image feature loss information, the face pinching parameter loss information and the face contrast loss information, the weight parameter information in the face processing model to be trained is adjusted to obtain the trained face processing model.

[0104] Here, the first product of the third face pinching parameter information and the fourth face pinching parameter information, and the second product of the third face pinching parameter information and the fifth face pinching parameter information can be calculated respectively; then the sum of the first product and the second product is calculated; then the ratio of the first product to the sum of the first product and the second product is calculated; finally, the negative value of the logarithm of the ratio is taken, that is, the face contrast loss information.

[0105] By minimizing the face contrast loss information and adjusting the weight parameter information in the face processing model to be trained, it can be achieved that the virtual characters rendered after obtaining the face pinching parameter information of the same real face are more similar, and the virtual characters rendered after obtaining the face pinching parameter information of different real faces are less similar.

[0106] Furthermore, in order to ensure that the virtual character rendered after obtaining the face pinching parameter information based on the real face is more similar to the real face, in one embodiment, in the process of training the face processing model to be trained, the image feature loss information, the face pinching parameter loss information, the face restoration loss information, the face consistency loss information and the face contrast loss information can be used simultaneously to adjust the weight parameter information in the face processing model to be trained.

[0107] See also Figure 2 FIG. 2 is a flowchart of another face processing method provided by an embodiment of the present disclosure, wherein the method includes S201 to S202, wherein:

[0108] S201: Obtain a face image to be processed of a real face.

[0109] S202: Inputting the face image to be processed into a face processing model trained according to the face processing method described in the above embodiment to obtain face pinching parameter information of the face image to be processed; the face pinching parameter information is used to render a virtual character in a virtual scene.

[0110] The training process of the face processing model may refer to the aforementioned embodiment and will not be described in detail here.

[0111] See also Figure 3 FIG. 1 is a flowchart of another face processing method provided by an embodiment of the present disclosure. Figure 3 In the face processing method shown, image feature loss information, face pinching parameter loss information, face restoration loss information, face consistency loss information and face contrast loss information are used to train the face processing model. The face processing model includes a codec, and the codec is described as a VQ-VAE-2 model. The codec includes an encoder and a decoder. The encoder (Encoder) can be defined as E, and the decoder (Decoder) can be defined as D.

[0112] Step 1: Obtain training samples.

[0113] The training samples include multiple face image pairs of real faces and character face screenshot samples corresponding to virtual characters. The multiple face image pairs of real faces include unlabeled images in the source domain (Source Domain) (i.e., the real domain). Each face image pair contains two different face image samples of the same person. The character face screenshot samples corresponding to the virtual character include labeled images in the target domain (Target Domain) (i.e., the virtual domain). In each round of training, any face image pair, any other face image, and any character face screenshot sample corresponding to any virtual character can be used as input. Here, any face image sample in the face image pair can be marked as The other face image sample in the face image pair is marked as Label any other face image as The character face screenshot sample corresponding to any virtual character is marked as I t .

[0114] Step 2: Input the training sample into the encoder to obtain the first image features corresponding to each image in the training sample.

[0115] Taking the original image features of 256×256 corresponding to each image in the training sample as an example, the encoder E can downsample each image to obtain a 64×64-dimensional bottom-layer latent map, and then further reduce the 64×64-dimensional bottom-layer latent map to obtain a quantized 32×32-dimensional top-layer latent map, i.e., the first image feature.

[0116] Through the above downsampling process, the 32×32-dimensional first image features corresponding to any face image pair, any other face image, and any character face screenshot sample corresponding to any virtual character can be obtained respectively.

[0117] Step 3: Input the first image features corresponding to each image in the training sample into the decoder to obtain the second image features corresponding to each image in the training sample.

[0118] The decoder D is a symmetrical structure similar to the encoder E. It can first upsample the first image features corresponding to each image to obtain the image features of the original dimension, that is, the image features of 256×256 dimensions. Then, it is quantized to obtain the quantized 64×64-dimensional top-layer latent map, that is, the second image feature, which has the same dimension as the previous 64×64-dimensional bottom-layer latent map.

[0119] Step 4: Input the second image feature into the regressor in the face processing model to obtain the third image feature corresponding to each image in the training sample and the face pinching parameter information corresponding to each image.

[0120] Here, the regressor is defined as R. The regressor R is set in the last layer of the decoder D. The regressor R is composed of 6 convolutional layers.

[0121] Here, the 32×32-dimensional top-level latent map obtained in step 2 is connected with the 64×64-dimensional top-level latent map obtained in step 3 to obtain a 64×64×128-dimensional stereoscopic latent map. As shown in Table 1, the 64×64×128-dimensional stereoscopic latent map is used as input to the regressor R. After the stereoscopic latent map is processed by each convolutional layer in the regressor R, a 2×2×256-dimensional third image feature can be obtained in the L5 layer of the regressor R, that is, the fifth layer of the regressor R, and then a 1×1×267-dimensional face pinching parameter information is obtained in the last layer of the regressor R.

[0122] Table 1

[0123]

[0124] Here, a sample face image The corresponding third image feature can be expressed as f S , a sample of character face screenshots corresponding to any virtual character I t The corresponding third image feature can be expressed as f t . Face image samples The corresponding face pinching parameter information can be expressed as P, face image sample The corresponding face pinching parameter information can be expressed as P + , any other face image The corresponding face pinching parameter information can be expressed as P - , a sample of character face screenshots corresponding to any virtual character I t The corresponding face pinching parameter information can be expressed as P t .

[0125] Step 5: Take the face image samples in the face image pair Corresponding face pinching parameter information P and character face screenshot sample I t Corresponding face pinching parameter information P t Input them into the pre-trained simulator to generate face image samples The corresponding first face generated image I s' And character face screenshot sample I t The corresponding second face generated image I t' .

[0126] Here, the simulator (Imitator) is defined as G. The second face generated image I is defined t' The image information is represented as G(P t ), and character face screenshot sample I t The image information is represented as I t The image information here may be red, green, and blue color channel information.

[0127] When training the simulator G, the second face generated image I can be calculated t' Image information G(P t ) and character face screenshot sample I t Image information I t The loss between , and then use the loss to train the simulator. Specifically, the second face generated image I can be calculated t' Image information G(P t ) and character face screenshot sample I t Image information I t The square of the difference.

[0128] Second face generated image I t' Image information G(P t ) and character face screenshot sample I t Image information I t The loss between imitation Can be:

[0129] By minimizing the loss L imitation The simulator G is trained, and the trained simulator G can generate the second face image I t' Screenshot sample I with character face t More similar.

[0130] Step 6: Based on face image samples The corresponding third image feature f S And character face screenshot sample I t The corresponding third image feature f t , determine the image feature loss information; based on the face image sample Corresponding face pinching parameter information P and character face screenshot sample I t Corresponding face pinching parameter information P t , determining face pinching parameter loss information; and, determining first loss information based on the image feature loss information and the face pinching parameter loss information.

[0131] Here we can use the maximum mean difference (MMD) to determine the face image samples Screenshot sample I with character face t The distance between them, here we get the first loss information as That is, the domain loses information.

[0132] in, Corresponding image feature loss information, Corresponding to the face pinching parameter loss information, H k is a kernel Hilbert space with a kernel function, which can be a Gaussian kernel function σ is the bandwidth, which is used to control the local range of the Gaussian kernel function.

[0133] Step 7: Sample screenshot based on character face I t Image information I t and the second face generated image I t' Image information I t' , determine the second loss information.

[0134] Based on character face screenshot sample I t The image information of the second face generates image I t' The image information can be determined by first determining the first sub-loss:

[0135] Here we use the loss in the VQ-VAE-2 model and define it as the second sub-loss, which can be: Among them, e is the character face screenshot sample I t quantization encoding; sg is an operator used to indicate the operation of stopping the gradient; α is a hyperparameter, here we can set α = 0.25.

[0136] The second loss information is: L prestored =L param +βL differ , that is, face restoration loss information; where β is the weight coefficient, here β = 0.25 can be set.

[0137] Step 8: Sample face images And the first face generated image I s' Input into the pre-trained face reconstruction model to obtain face image samples The fourth image feature and the first face generate image I s' The fifth image feature; and, the face image sample And the first face generated image I s' Input into the pre-trained face recognition model to obtain face image samples The sixth image feature and the first face generate image I s' seventh image feature; based on the fourth image feature and the fifth image feature, determining the first face consistency loss information; and, based on the sixth image feature and the seventh image feature, determining the second face consistency loss information; and, based on the first face consistency loss information and the second face consistency loss information, determining the third loss information.

[0138] The face reconstruction model can be marked as F 3d , the face recognition model can be marked as F id .

[0139] Face image samples The fourth image feature can be expressed as F 3d (I S ), the first face generated image I s' The fifth image feature can be expressed as F 3d (I S' ), the first face consistency loss information can be obtained as:

[0140] Face image samples The sixth image feature can be expressed as F id (I S ), the first face generated image I s' The seventh image feature can be expressed as F id (I S' ), the second face consistency loss information can be obtained as:

[0141] By adding the first face consistency loss information and the second face consistency loss information, the third loss information can be obtained: L consistency =L 3d +L id , that is, face consistency loss information.

[0142] Step 9: Based on face image samples Corresponding face pinching parameter information P + , face image samples Corresponding face pinching parameter information P - , Character face screenshot sample I t Corresponding face pinching parameter information P t , determine the fourth loss information.

[0143] Here, the fourth loss information may be:

[0144] That is, the face contrast loss information; where τ is the preset coefficient.

[0145] Step 10: Based on the first loss information, the second loss information, the third loss information, and the fourth loss information, the total loss information may be determined.

[0146] The total loss information is: L = λ 1 L domain +λ 2 L prestored +λ 3 L consistency +λ 4 L contrastive ; Among them, λ 1 =1,λ 2 =0.01,λ3 =0.02,λ 4 =0.02.

[0147] Step 11: Based on the total loss information, adjust the weight parameter information in the face processing model to be trained to obtain a trained face processing model.

[0148] The weight parameter information in the face processing model to be trained can be adjusted by referring to the aforementioned adjustment process, which will not be described in detail here. The trained face processing model can be used to obtain the face pinching parameter information of the corresponding virtual character after inputting the face image to be processed of the real face. The obtained face pinching parameter information of the virtual character can be rendered in the game engine to obtain the virtual character.

[0149] Those skilled in the art will appreciate that, in the above method of specific implementation, the order in which the steps are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of the steps should be determined by their functions and possible internal logic.

[0150] Based on the same inventive concept, a face processing device corresponding to the face processing method is also provided in the embodiment of the present disclosure. Since the principle of solving the problem by the device in the embodiment of the present disclosure is similar to the above-mentioned face processing method in the embodiment of the present disclosure, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0151] Reference Figure 4 , which is a schematic diagram of the architecture of a face processing device provided by an embodiment of the present disclosure, the device comprises: a first acquisition module 401, a first input module 402, a first determination module 403, and an adjustment module 404; wherein,

[0152] The first acquisition module 401 is used to acquire facial image samples of real human faces and character face screenshot samples corresponding to virtual characters;

[0153] A first input module 402 is used to input the human face image sample and the character face screenshot sample into the face processing model to be trained, respectively, to obtain the first image feature and first face pinching parameter information corresponding to the human face image sample, and the second image feature and second face pinching parameter information corresponding to the character face screenshot sample;

[0154] A first determining module 403 is configured to determine image feature loss information based on the first image feature and the second image feature; and determine face pinching parameter loss information based on the first face pinching parameter information and the second face pinching parameter information;

[0155] The adjustment module 404 is used to adjust the weight parameter information in the face processing model to be trained based on the image feature loss information and the face pinching parameter loss information to obtain a trained face processing model; the trained face processing model is used to obtain the face pinching parameter information of the corresponding virtual character after inputting the face image to be processed of a real face.

[0156] In an optional implementation, the device further includes:

[0157] A second input module is used to input the second face pinching parameter information of the character face screenshot sample into a pre-trained imitator to obtain a first face generated image;

[0158] A second determination module is used to obtain face restoration loss information based on the first image information of the character face screenshot sample and the second image information of the first face generated image;

[0159] The adjustment module 404 is specifically used for:

[0160] Based on the image feature loss information, the face pinching parameter loss information and the face restoration loss information, the weight parameter information in the face processing model to be trained is adjusted to obtain a trained face processing model.

[0161] In an optional implementation, the device further includes:

[0162] A third input module is used to input the first face pinching parameter information of the face image sample into a pre-trained imitator to obtain a second face generated image;

[0163] a third determination module, configured to input the face image sample and the second face generated image into a pre-trained first face recognition model, and respectively obtain a third image feature of the face image sample and a fourth image feature of the second face generated image; and input the face image sample and the second face generated image into a pre-trained second face recognition model, and respectively obtain a fifth image feature of the face image sample and a sixth image feature of the second face generated image;

[0164] a fourth determination module, configured to determine first face consistency loss information based on the third image feature and the fourth image feature; and to determine second face consistency loss information based on the fifth image feature and the sixth image feature;

[0165] The adjustment module 404 is specifically used for:

[0166] Based on the image feature loss information, the face pinching parameter loss information, the first face consistency loss information and the second face consistency loss information, the weight parameter information in the face processing model to be trained is adjusted to obtain a trained face processing model.

[0167] In an optional implementation, the face image sample includes a plurality of face image pairs; each of the face image pairs includes two different face images of the same person;

[0168] For each face image pair, the first face pinching parameter information corresponding to the face image sample includes the third face pinching parameter information and the fourth face pinching parameter information corresponding to the two face images in the face image sample pair, and the fifth face pinching parameter information corresponding to any other face image;

[0169] The device also includes:

[0170] a fifth determining module, configured to determine face contrast loss information based on the third face pinching parameter information, the fourth face pinching parameter information, and the fifth face pinching parameter information;

[0171] The adjustment module 404 is specifically used for:

[0172] Based on the image feature loss information, the face pinching parameter loss information and the face contrast loss information, the weight parameter information in the face processing model to be trained is adjusted to obtain a trained face processing model.

[0173] In an optional implementation manner, the adjustment module 404 is specifically configured to:

[0174] Taking the derivative of the sum of the image feature loss information and the face pinching parameter loss information to obtain a gradient function of the weight parameters in the face processing model to be trained;

[0175] Substituting the current weight parameter information into the gradient function to obtain the gradient vector after iteration;

[0176] Based on the iterated gradient vector, the current weight parameter information is updated to obtain updated weight parameter information.

[0177] In an optional implementation manner, the first input module 402 is specifically configured to:

[0178] Inputting the human face image sample and the character face screenshot sample into the encoder of the face processing model to be trained respectively, downsampling the human face image sample and the character face screenshot sample respectively, and obtaining the seventh image feature and the eighth image feature of the human face image sample and the character face screenshot sample respectively under the first preset dimension;

[0179] Inputting the seventh image feature and the eighth image feature into the decoder of the face processing model to be trained respectively, upsampling the seventh image feature and the eighth image feature to obtain the ninth image feature and the tenth image feature of the face image sample and the character face screenshot sample respectively under the second preset dimension;

[0180] The seventh image feature and the ninth image feature, the eighth image feature and the tenth image feature are respectively input into the regressor of the face processing model to be trained to obtain the first image feature and the first face pinching parameter information corresponding to the face image sample, and the second image feature and the second face pinching parameter information corresponding to the character face screenshot sample.

[0181] In an optional implementation, the device further includes:

[0182] A second acquisition module is used to obtain the sixth face pinching parameter information of the character face screenshot sample;

[0183] The training module is used to take the sixth face pinching parameter information of the character face screenshot sample as input and the character face screenshot sample as output, train the imitator to be trained, and obtain a trained imitator.

[0184] Reference Figure 5 FIG. 5 is a schematic diagram of the architecture of another face processing device provided by an embodiment of the present disclosure, wherein the device comprises: an acquisition module 501 and an input module 502; wherein:

[0185] An acquisition module 501 is used to acquire a face image to be processed of a real face;

[0186] Input module 502, used for inputting the face image to be processed into Figure 1 The face processing model obtained after training the face processing method obtains face pinching parameter information of the face image to be processed; the face pinching parameter information is used to render a virtual character in a virtual scene.

[0187] For descriptions of the processing flow of each module in the device and the interaction flow between each module, reference may be made to the relevant descriptions in the above method embodiment, which will not be described in detail here.

[0188] Based on the same technical concept, the embodiment of the present disclosure also provides a computer device. Figure 6As shown, it is a schematic diagram of the structure of a computer device 600 provided in an embodiment of the present disclosure, including a processor 601, a memory 602, and a bus 603. Among them, the memory 602 is used to store execution instructions, including a memory 6021 and an external memory 6022; the memory 6021 here is also called an internal memory, which is used to temporarily store the operation data in the processor 601, and the data exchanged with the external memory 6022 such as a hard disk. The processor 601 exchanges data with the external memory 6022 through the memory 6021. When the computer device 600 is running, the processor 601 communicates with the memory 602 through the bus 603, so that the processor 601 executes the following instructions:

[0189] Obtaining facial image samples of real human faces and screenshot samples of character faces corresponding to virtual characters;

[0190] Inputting the human face image sample and the character face screenshot sample into the face processing model to be trained respectively, obtaining the first image feature and the first face pinching parameter information corresponding to the human face image sample, and the second image feature and the second face pinching parameter information corresponding to the character face screenshot sample;

[0191] Determine image feature loss information based on the first image feature and the second image feature; and determine face pinching parameter loss information based on the first face pinching parameter information and the second face pinching parameter information;

[0192] Based on the image feature loss information and the face pinching parameter loss information, the weight parameter information in the face processing model to be trained is adjusted to obtain a trained face processing model; the trained face processing model is used to obtain the face pinching parameter information of the corresponding virtual character after inputting the face image to be processed of a real face.

[0193] Or, the processor 601 executes the following instructions:

[0194] Obtaining a face image to be processed of a real face;

[0195] The face image to be processed is input into the face processing model obtained after training according to the above face processing method to obtain face pinching parameter information of the face image to be processed; the face pinching parameter information is used to render a virtual character in a virtual scene.

[0196] The embodiment of the present disclosure also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the face processing method described in the above method embodiment are executed. The storage medium can be a volatile or non-volatile computer-readable storage medium.

[0197] The embodiment of the present disclosure further provides a computer program product, which carries a program code. The instructions included in the program code can be used to execute the steps of the face processing method described in the above method embodiment. For details, please refer to the above method embodiment, which will not be repeated here.

[0198] The computer program product may be implemented in hardware, software or a combination thereof. In one optional embodiment, the computer program product is implemented as a computer storage medium. In another optional embodiment, the computer program product is implemented as a software product, such as a software development kit (SDK).

[0199] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. In the several embodiments provided in the present disclosure, it should be understood that the disclosed device and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the device or unit can be electrical, mechanical or other forms.

[0200] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0201] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0202] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present disclosure, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0203] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present disclosure, which are used to illustrate the technical solutions of the present disclosure, rather than to limit them. The protection scope of the present disclosure is not limited thereto. Although the present disclosure is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed in the present disclosure, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure shall be based on the protection scope of the claims.

Claims

1. A facial processing method, characterized in that: include: Obtaining facial image samples of real human faces and screenshot samples of character faces corresponding to virtual characters; Inputting the human face image sample and the character face screenshot sample into the face processing model to be trained respectively, obtaining the first image feature and the first face pinching parameter information corresponding to the human face image sample, and the second image feature and the second face pinching parameter information corresponding to the character face screenshot sample; Determine image feature loss information based on the first image feature and the second image feature; and determine face pinching parameter loss information based on the first face pinching parameter information and the second face pinching parameter information; Based on the image feature loss information and the face pinching parameter loss information, the weight parameter information in the face processing model to be trained is adjusted to obtain a trained face processing model; the trained face processing model is used to obtain the face pinching parameter information of the corresponding virtual character after inputting the face image to be processed of a real face, Among them, the determining of image feature loss information based on the first image feature and the second image feature; and the determining of face pinching parameter loss information based on the first face pinching parameter information and the second face pinching parameter information include: determining a first difference between the vector mean of the first image feature and the vector mean of the second image feature using the maximum average difference, and determining the image feature loss information based on the first difference; and determining a second difference between the vector mean of the first face pinching parameter information and the vector mean of the second face pinching parameter information using the maximum average difference, and determining the face pinching parameter loss information based on the second difference.

2. The method according to claim 1, characterized in that After inputting the human face image sample and the character face screenshot sample into the face processing model to be trained respectively and obtaining the first face pinching parameter information corresponding to the human face image sample, the method further includes: Inputting the second face pinching parameter information of the character face screenshot sample into a pre-trained imitator to obtain a first face generated image; Based on the first image information of the character face screenshot sample and the second image information of the first face generated image, obtaining face restoration loss information; The step of adjusting the weight parameter information in the face processing model to be trained based on the image feature loss information and the face pinching parameter loss information to obtain a trained face processing model includes: Based on the image feature loss information, the face pinching parameter loss information and the face restoration loss information, the weight parameter information in the face processing model to be trained is adjusted to obtain a trained face processing model.

3. The method according to claim 1, characterized in that After inputting the human face image sample and the character face screenshot sample into the face processing model to be trained respectively and obtaining the first face pinching parameter information corresponding to the human face image sample, the method further includes: Inputting the first face pinching parameter information of the face image sample into a pre-trained imitator to obtain a second face generated image; Inputting the face image sample and the second face generated image into a pre-trained first face recognition model to obtain a third image feature of the face image sample and a fourth image feature of the second face generated image respectively; and inputting the face image sample and the second face generated image into a pre-trained second face recognition model to obtain a fifth image feature of the face image sample and a sixth image feature of the second face generated image respectively; Determining first face consistency loss information based on the third image feature and the fourth image feature; and determining second face consistency loss information based on the fifth image feature and the sixth image feature; The step of adjusting the weight parameter information in the face processing model to be trained based on the image feature loss information and the face pinching parameter loss information to obtain a trained face processing model includes: Based on the image feature loss information, the face pinching parameter loss information, the first face consistency loss information and the second face consistency loss information, the weight parameter information in the face processing model to be trained is adjusted to obtain a trained face processing model.

4. The method according to claim 1, characterized in that: The face image samples include a plurality of face image pairs; each of the face image pairs includes two different face images of the same person; For each face image pair, the first face pinching parameter information corresponding to the face image sample includes the third face pinching parameter information and the fourth face pinching parameter information corresponding to the two face images in the face image sample pair, and the fifth face pinching parameter information corresponding to any other face image; After inputting the human face image sample and the character face screenshot sample into the face processing model to be trained respectively and obtaining the first face pinching parameter information corresponding to the human face image sample, the method further includes: Determining face contrast loss information based on the third face pinching parameter information, the fourth face pinching parameter information, and the fifth face pinching parameter information; The step of adjusting the weight parameter information in the face processing model to be trained based on the image feature loss information and the face pinching parameter loss information to obtain a trained face processing model includes: Based on the image feature loss information, the face pinching parameter loss information and the face contrast loss information, the weight parameter information in the face processing model to be trained is adjusted to obtain a trained face processing model.

5. The method according to claim 1, characterized in that The step of adjusting the weight parameter information in the face processing model to be trained based on the image feature loss information and the face pinching parameter loss information to obtain a trained face processing model includes: Taking the derivative of the sum of the image feature loss information and the face pinching parameter loss information to obtain a gradient function of the weight parameters in the face processing model to be trained; Substituting the current weight parameter information into the gradient function to obtain the gradient vector after iteration; Based on the iterated gradient vector, the current weight parameter information is updated to obtain updated weight parameter information.

6. The method according to claim 1, characterized in that The step of inputting the human face image sample and the character face screenshot sample into the face processing model to be trained respectively, obtaining the first image feature and the first face pinching parameter information corresponding to the human face image sample, and the second image feature and the second face pinching parameter information corresponding to the character face screenshot sample, comprises: Inputting the human face image sample and the character face screenshot sample into the encoder of the face processing model to be trained respectively, downsampling the human face image sample and the character face screenshot sample respectively, and obtaining the seventh image feature and the eighth image feature of the human face image sample and the character face screenshot sample respectively under the first preset dimension; Inputting the seventh image feature and the eighth image feature into the decoder of the face processing model to be trained respectively, upsampling the seventh image feature and the eighth image feature to obtain the ninth image feature and the tenth image feature of the face image sample and the character face screenshot sample respectively under the second preset dimension; The seventh image feature and the ninth image feature, the eighth image feature and the tenth image feature are respectively input into the regressor of the face processing model to be trained to obtain the first image feature and the first face pinching parameter information corresponding to the face image sample, and the second image feature and the second face pinching parameter information corresponding to the character face screenshot sample.

7. The method according to claim 2, characterized in that Before inputting the second face pinching parameter information of the character face screenshot sample into a pre-trained imitator to obtain the first face generated image, the method further includes: Obtaining sixth face pinching parameter information of the character face screenshot sample; The sixth face pinching parameter information of the character face screenshot sample is used as input and the character face screenshot sample is used as output to train the imitator to be trained to obtain a trained imitator.

8. A facial processing method, characterized in that: include: Obtaining a face image to be processed of a real face; The face image to be processed is input into a face processing model obtained after training according to the face processing method according to any one of claims 1 to 7 to obtain face pinching parameter information of the face image to be processed; the face pinching parameter information is used to render a virtual character in a virtual scene.

9. A facial processing device, characterized in that: include: A first acquisition module is used to acquire facial image samples of real human faces and character face screenshot samples corresponding to virtual characters; A first input module, used to input the human face image sample and the character face screenshot sample into the face processing model to be trained, respectively, to obtain the first image feature and first face pinching parameter information corresponding to the human face image sample, and the second image feature and second face pinching parameter information corresponding to the character face screenshot sample; A first determining module, configured to determine image feature loss information based on the first image feature and the second image feature; and determine face pinching parameter loss information based on the first face pinching parameter information and the second face pinching parameter information; The adjustment module is used to adjust the weight parameter information in the face processing model to be trained based on the image feature loss information and the face pinching parameter loss information to obtain a trained face processing model; the trained face processing model is used to obtain the face pinching parameter information of the corresponding virtual character after inputting the face image to be processed of a real face, Among them, the determining of image feature loss information based on the first image feature and the second image feature; and the determining of face pinching parameter loss information based on the first face pinching parameter information and the second face pinching parameter information include: determining a first difference between the vector mean of the first image feature and the vector mean of the second image feature using the maximum average difference, and determining the image feature loss information based on the first difference; and determining a second difference between the vector mean of the first face pinching parameter information and the vector mean of the second face pinching parameter information using the maximum average difference, and determining the face pinching parameter loss information based on the second difference.

10. A facial processing device, characterized in that: include: An acquisition module, used for acquiring a face image to be processed of a real face; An input module is used to input the face image to be processed into a face processing model obtained after training according to the face processing method according to any one of claims 1 to 7, to obtain face pinching parameter information of the face image to be processed; the face pinching parameter information is used to render a virtual character in a virtual scene.

11. A computer device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the computer device is running, the processor and the memory communicate with each other through the bus, and when the machine-readable instructions are executed by the processor, the steps of the face processing method according to any one of claims 1 to 7 or the steps of the face processing method according to claim 8 are executed.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the face processing method according to any one of claims 1 to 7 or the steps of the face processing method according to claim 8 are executed.

Images