Methods, devices, electronic equipment and storage media for assessing facial expression transfer

Abstract

The application provides an expression migration evaluation method and device, electronic equipment and storage medium, and relates to the technical field of image processing. The method comprises the following steps: loading an expression migration test set, the expression migration test set comprising a plurality of test videos and standard mixed deformation parameters corresponding to each test video, and objects in each test video having different test expressions; running a first expression migration program to perform expression migration on each test video in the expression migration test set, and obtaining predicted mixed deformation parameters corresponding to each test video; and determining a migration evaluation result of the first expression migration program according to the predicted mixed deformation parameters corresponding to each test video and the standard mixed deformation parameters corresponding to each test video. In the method, the expression migration result of the first expression migration program is dataized, the expression migration effect is measured through the migration evaluation result, the evaluation of the migration effect of the expression migration program is objectified, and the credibility is higher.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and more specifically, to a method, apparatus, electronic device, and storage medium for evaluating facial expression transfer. Background Technology

[0002] With the widespread adoption of real-time video applications on mobile devices, more and more users are using these applications to interact with others. In scenarios such as social networking with strangers and other applications, users need a method for transferring facial expressions to a virtual model for display. This is typically achieved using facial expression transfer algorithms. Evaluating the effectiveness of facial expression transfer can, to some extent, help optimize these algorithms, thereby improving user satisfaction with the results.

[0003] Currently, most evaluations of facial expression transfer effectiveness are based on subjective assessments. Test subjects manually make a specified facial expression, then observe the expression of a virtual model in real time, and subjectively compare the two to determine the accuracy, sensitivity, and stability of the facial expression transfer.

[0004] However, the evaluation results obtained by the above methods are affected by various external factors, resulting in low accuracy and reliability. Summary of the Invention

[0005] The purpose of this application is to address the shortcomings of the prior art by providing a method, apparatus, electronic device, and storage medium for evaluating facial expression transfer, so as to solve the problems of poor accuracy and reliability of facial expression transfer evaluation results in the prior art.

[0006] To achieve the above objectives, the technical solutions adopted in the embodiments of this application are as follows:

[0007] In a first aspect, embodiments of this application provide a facial expression transfer evaluation method, applied in a computer device displaying a graphical user interface, the method comprising:

[0008] Load the facial expression transfer test set, which includes multiple test videos and standard hybrid deformation parameters corresponding to each test video. The objects in each test video have different test expressions.

[0009] Run the first expression transfer program to perform expression transfer on each test video in the expression transfer test set and obtain the prediction hybrid deformation parameters corresponding to each test video;

[0010] Based on the predicted hybrid deformation parameters corresponding to each test video and the standard hybrid deformation parameters corresponding to each test video, the migration evaluation result of the first expression migration program is determined.

[0011] Optionally, before loading the facial expression transfer test set, the method includes:

[0012] Run the first expression transfer program to perform expression transfer on each initial video in the initial set of expression transfer and obtain the initial blending deformation parameters corresponding to each initial video;

[0013] The system responds to the adjustment of the initial blending deformation parameters for each initial video in each frame until the expression of the object in the initial video and the expression of the virtual model after expression transfer meet the preset similarity. The current initial blending deformation parameters are then used as the standard blending deformation parameters for each initial video in each frame.

[0014] The standard hybrid deformation parameters of each initial video are obtained based on the standard hybrid deformation parameters of each initial video in each frame.

[0015] The expression transfer test set is obtained based on each initial video and the standard hybrid deformation parameters of each initial video.

[0016] Optionally, the predicted hybrid deformation parameters of each test video include: frame identifier, predicted motion parameters of different parts of the face in each frame, and weights of each motion parameter in each frame;

[0017] The standard hybrid deformation parameters for each test video include: frame identifier, standard motion parameters for different parts of the face in each frame, and weights for each motion parameter in each frame.

[0018] Optionally, determining the transfer evaluation result of the first expression transfer procedure based on the predicted hybrid deformation parameters corresponding to each test video and the standard hybrid deformation parameters corresponding to each test video includes:

[0019] Based on the predicted hybrid deformation parameters corresponding to each test video and the standard hybrid deformation parameters corresponding to each test video, the Euclidean distance is used to calculate the migration evaluation results of the first expression migration program.

[0020] Optionally, the step of calculating the transfer evaluation result of the first expression transfer procedure using Euclidean distance based on the predicted hybrid deformation parameters corresponding to each test video and the standard hybrid deformation parameters corresponding to each test video includes:

[0021] The predicted motion parameters of different parts of the face and the standard motion parameters of different parts of the face in each target frame of each test video are subtracted and the absolute value is taken. Then, the weight of each motion parameter in each target frame is multiplied to obtain the evaluation result of each test video in each target frame. The target frame is any frame in each frame.

[0022] The average value of the evaluation results of each test video under each target frame is taken to obtain the evaluation result corresponding to each test video.

[0023] Based on the evaluation results corresponding to each test video, the migration evaluation results of the first expression migration program are obtained.

[0024] Optionally, after determining the transfer evaluation result of the first expression transfer procedure based on the predicted hybrid deformation parameters corresponding to each test video and the standard hybrid deformation parameters corresponding to each test video, the method includes:

[0025] Run the second expression transfer program and determine the transfer evaluation result of the second expression transfer program based on the running results. The second expression transfer program is a different expression transfer program from the first expression transfer program.

[0026] The target expression transfer procedure is determined based on the transfer evaluation results of the second expression transfer procedure and the transfer evaluation results of the first expression transfer procedure.

[0027] Optionally, after determining the transfer evaluation result of the first expression transfer procedure based on the predicted hybrid deformation parameters corresponding to each test video and the standard hybrid deformation parameters corresponding to each test video, the method includes:

[0028] Optimize the parameters in the first expression transfer algorithm, obtain a new transfer evaluation result of the first expression transfer program, and iterate until the new transfer evaluation result and the transfer evaluation result of the first expression transfer program meet a preset condition. Then, use the current parameters of the first expression transfer algorithm as the target parameters of the first expression transfer algorithm.

[0029] Optionally, before running the first expression transfer procedure to perform expression transfer on each initial video in the initial set of expression transfer, the method includes:

[0030] Create multiple facial expression transfer test cases, each corresponding to a different facial expression action and / or facial part;

[0031] Based on each facial expression transfer test case, record the corresponding test video for each facial expression transfer test case in a standard recording environment.

[0032] Optionally, running the first expression transfer program to perform expression transfer on each test video in the expression transfer test set includes:

[0033] Run the first expression transfer program to perform expression transfer on each test video in the expression transfer test set, and assign the expression corresponding to each test video to the virtual model displayed by the graphical user interface. The expression corresponding to each test video is obtained according to the predicted hybrid deformation parameters corresponding to each test video.

[0034] Secondly, this application also provides an expression transfer evaluation device, which is applied in a computer device displaying a graphical user interface. The device includes: a loading module, a running module, and a determining module.

[0035] The loading module is used to load the expression transfer test set, which includes multiple test videos and standard hybrid deformation parameters corresponding to each test video. The objects in each test video have different test expressions.

[0036] The running module is used to run the first expression transfer program, perform expression transfer on each test video in the expression transfer test set, and obtain the prediction hybrid deformation parameters corresponding to each test video.

[0037] The determining module is used to determine the migration evaluation result of the first expression migration program based on the predicted hybrid deformation parameters corresponding to each test video and the standard hybrid deformation parameters corresponding to each test video.

[0038] Optionally, the running module is further configured to run a first expression transfer program to perform expression transfer on each initial video in the initial set of expression transfers and obtain the initial hybrid deformation parameters corresponding to each initial video.

[0039] The system responds to the adjustment of the initial blending deformation parameters for each initial video in each frame until the expression of the object in the initial video and the expression of the virtual model after expression transfer meet the preset similarity. The current initial blending deformation parameters are then used as the standard blending deformation parameters for each initial video in each frame.

[0040] The standard hybrid deformation parameters of each initial video are obtained based on the standard hybrid deformation parameters of each initial video in each frame.

[0041] The expression transfer test set is obtained based on each initial video and the standard hybrid deformation parameters of each initial video.

[0042] Optionally, the predicted hybrid deformation parameters of each test video include: frame identifier, predicted motion parameters of different parts of the face in each frame, and weights of each motion parameter in each frame;

[0043] The standard hybrid deformation parameters for each test video include: frame identifier, standard motion parameters for different parts of the face in each frame, and weights for each motion parameter in each frame.

[0044] Optionally, the determining module is specifically used to calculate the migration evaluation result of the first expression migration program by using Euclidean distance based on the predicted hybrid deformation parameters corresponding to each test video and the standard hybrid deformation parameters corresponding to each test video.

[0045] Optionally, the determining module is specifically used to calculate the difference and take the absolute value of the predicted motion parameters of different parts of the face and the standard motion parameters of different parts of the face in each test video in each target frame, and then multiply the weight of each motion parameter in each target frame to obtain the evaluation result of each test video in each target frame, where the target frame is any frame in each frame.

[0046] The average value of the evaluation results of each test video under each target frame is taken to obtain the evaluation result corresponding to each test video.

[0047] Based on the evaluation results corresponding to each test video, the migration evaluation results of the first expression migration program are obtained.

[0048] Optionally, the running module is further configured to run a second expression transfer program and determine the transfer evaluation result of the second expression transfer program based on the running result, wherein the second expression transfer program is a different expression transfer program from the first expression transfer program;

[0049] The determination module is further configured to determine the target expression transfer procedure based on the transfer evaluation results of the second expression transfer procedure and the transfer evaluation results of the first expression transfer procedure.

[0050] Optionally, the device further includes: an optimization module;

[0051] The optimization module is used to optimize the parameters in the first expression transfer algorithm, obtain a new transfer evaluation result of the first expression transfer program, and iteratively execute the algorithm until the new transfer evaluation result and the transfer evaluation result of the first expression transfer program meet a preset condition. Then, the current parameters of the first expression transfer algorithm are used as the target parameters of the first expression transfer algorithm.

[0052] Optionally, the apparatus further includes: a creation module;

[0053] The creation module is used to create multiple expression transfer test cases, each corresponding to different facial expressions and / or facial features;

[0054] Based on each facial expression transfer test case, record the corresponding test video for each facial expression transfer test case in a standard recording environment.

[0055] Optionally, the operating module is specifically used for

[0056] Run the first expression transfer program to perform expression transfer on each test video in the expression transfer test set, and assign the expression corresponding to each test video to the virtual model displayed by the graphical user interface. The expression corresponding to each test video is obtained according to the predicted hybrid deformation parameters corresponding to each test video.

[0057] Thirdly, embodiments of this application provide an electronic device, including: a processor, a storage medium, and a bus. The storage medium stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to perform the steps of the method provided in the first aspect.

[0058] Fourthly, embodiments of this application provide a storage medium storing a computer program that, when executed by a processor, performs the steps of the method provided in the first aspect.

[0059] The beneficial effects of this application are:

[0060] This application provides a method, apparatus, electronic device, and storage medium for evaluating facial expression transfer. The method includes: loading a facial expression transfer test set, which includes multiple test videos and standard hybrid deformation parameters corresponding to each test video, wherein the objects in each test video have different test facial expressions; running a first facial expression transfer program to perform facial expression transfer on each test video in the facial expression transfer test set, and obtaining the predicted hybrid deformation parameters corresponding to each test video; and determining the transfer evaluation result of the first facial expression transfer program based on the predicted hybrid deformation parameters corresponding to each test video and the standard hybrid deformation parameters corresponding to each test video. In this method, by introducing the standard hybrid deformation parameters of each test video, and calculating the transfer evaluation result of the first facial expression transfer program based on the standard hybrid deformation parameters of each test video and the predicted hybrid deformation parameters of each test video inferred by the first facial expression transfer program, the facial expression transfer result can be obtained. By digitizing the facial expression transfer result of the first facial expression transfer program, and measuring the facial expression transfer effect through the transfer evaluation result, the evaluation of the transfer effect of the facial expression transfer program is made more objective. Compared with the subjective evaluation of the transfer effect by humans, the evaluation result of the transfer effect obtained by this method has higher reliability and accuracy.

[0061] Furthermore, using pre-recorded test videos reduces hardware, projection, and recording time costs compared to real-time camera recording, saving resources and improving testing efficiency. Recording test videos against a standard green screen background also eliminates environmental interference and lighting issues, making facial expressions the sole variable, reducing complex problem-solving steps, and enabling more accurate and efficient testing. Attached Figure Description

[0062] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0063] Figure 1 A flowchart illustrating an expression transfer evaluation method provided in an embodiment of this application;

[0064] Figure 2 A flowchart illustrating another facial expression transfer evaluation method provided in this application embodiment;

[0065] Figure 3 A graphical user interface diagram provided for an embodiment of this application;

[0066] Figure 4 A flowchart illustrating another facial expression transfer evaluation method provided in this application embodiment;

[0067] Figure 5 A flowchart illustrating another facial expression transfer evaluation method provided in this application embodiment;

[0068] Figure 6 A flowchart illustrating another facial expression transfer evaluation method provided in this application embodiment;

[0069] Figure 7 This is a schematic diagram of an expression transfer test case provided in an embodiment of this application;

[0070] Figure 8 A schematic diagram of an expression transfer evaluation device provided in an embodiment of this application;

[0071] Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0072] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the accompanying drawings in this application are for illustrative and descriptive purposes only and are not intended to limit the scope of protection of this application. Furthermore, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate operations implemented according to some embodiments of this application. It should be understood that the operations in the flowcharts may not be implemented in sequence, and steps without logical contextual relationships may be reversed or implemented simultaneously. In addition, those skilled in the art, guided by the content of this application, may add one or more other operations to the flowcharts, or remove one or more operations from the flowcharts.

[0073] Furthermore, the described embodiments are merely some, not all, of the embodiments of this application. The components of the embodiments of this application described and illustrated herein can typically be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0074] It should be noted that the term "comprising" will be used in the embodiments of this application to indicate the presence of the features declared thereafter, but does not exclude the addition of other features.

[0075] First, we will explain the two commonly used facial expression transfer methods, which are mainly divided into two facial expression transfer techniques. The first is 2D facial expression transfer based entirely on video, often referred to as "AI face swapping." This approach has a high degree of realism, but it has high requirements for input data, and the output can only be a single-angle video, which has significant limitations. The second approach uses facial expression transfer algorithms to infer the blendshape parameters of the face in the input video, and then attaches these parameters to a virtual model in a 3D image engine to achieve the effect of 3D facial expression transfer. This approach has low requirements for input data, and the output facial expression transfer result can be from any angle, with any model, in any environment, and under any lighting conditions, offering greater flexibility.

[0076] This invention is based on a second expression transfer algorithm, which evaluates the expression transfer effect achieved by the expression transfer algorithm, thereby achieving an evaluation of the expression transfer algorithm.

[0077] Figure 1This is a flowchart illustrating a facial expression transfer evaluation method provided in an embodiment of this application. This method can be applied to computer devices displaying a graphical user interface (GUI). The GUI here can be a developer-oriented GUI, distinct from a conventional user-oriented GUI. This GUI may include parameter configuration controls to provide developers with a testing environment for the product, allowing developers to adjust product parameters in real time. Optionally, in this embodiment, the GUI can be a visual interface provided by the Unity editor.

[0078] like Figure 1 As shown, the method may include:

[0079] S101. Load the facial expression transfer test set. The facial expression transfer test set includes multiple test videos and standard hybrid deformation parameters corresponding to each test video. The objects in each test video have different test facial expressions.

[0080] Optionally, the facial expression migration test suite can be pre-created and saved in the Unity editor. When testing, the facial expression migration test suite can be loaded first. This loading can be done by retrieving the facial expression migration test suite from the Unity editor and opening it in the Unity editor's graphical user interface.

[0081] The expression transfer test set may include multiple test pairs. Each test pair may include a test video and the corresponding standard blendshape parameters for the test video. Blendshape parameters may refer to parameters that describe the movements of different parts of the face, such as parameters for blinking or raising eyebrows.

[0082] The subjects in each test video can have different test expressions. Here, "subjects" can refer to people, and "different test expressions" can refer to different facial features or facial movements that each test expression focuses on, in order to enrich the variety of test videos and make the test results more general.

[0083] S102. Run the first expression transfer program to perform expression transfer on each test video in the expression transfer test set and obtain the prediction hybrid deformation parameters corresponding to each test video.

[0084] In some embodiments, in response to a user’s command to run a first expression transfer program on a graphical user interface, the first expression transfer program can be run to perform expression transfer processing on each test video in the expression transfer test set and obtain the predicted hybrid deformation parameters corresponding to each test video.

[0085] The first facial migration program can be any facial migration algorithm to be tested. It can be pre-loaded into the Unity editor, or it can be selected and loaded online in real time from a large number of facial migration algorithms.

[0086] Standard hybrid deformation parameters refer to the more accurate hybrid deformation parameters corresponding to the test video, while predicted hybrid deformation parameters are only calculated by the first expression transfer program. Their accuracy will be affected by the first expression transfer program. When different expression transfer programs are used for expression transfer processing, the predicted hybrid deformation parameters obtained will be different.

[0087] S103. Based on the predicted hybrid deformation parameters corresponding to each test video and the standard hybrid deformation parameters corresponding to each test video, determine the migration evaluation results of the first expression migration procedure.

[0088] In this embodiment, the standard hybrid deformation parameters and predicted hybrid deformation parameters of each test video can be used to calculate the evaluation results of each test video when the first expression transfer program is applied to each test video. Based on the evaluation results of each test video, the transfer evaluation results of the first expression transfer program can be determined.

[0089] This method can obtain the transfer evaluation results of the first expression transfer procedure, and the transfer evaluation results can be used to evaluate the expression transfer effect of the first expression transfer procedure. By digitizing the expression transfer effect, the problem of poor accuracy of evaluation results caused by subjective evaluation can be avoided.

[0090] In summary, the facial expression transfer evaluation method provided in this embodiment includes: loading a facial expression transfer test set, which includes multiple test videos and standard hybrid deformation parameters corresponding to each test video, wherein the objects in each test video have different test facial expressions; running a first facial expression transfer program to perform facial expression transfer on each test video in the facial expression transfer test set, and obtaining the predicted hybrid deformation parameters corresponding to each test video; and determining the transfer evaluation result of the first facial expression transfer program based on the predicted hybrid deformation parameters corresponding to each test video and the standard hybrid deformation parameters corresponding to each test video. In this method, by introducing the standard hybrid deformation parameters of each test video, and based on the standard hybrid deformation parameters of each test video and the predicted hybrid deformation parameters of each test video inferred by the first facial expression transfer program, the transfer evaluation result of the first facial expression transfer program can be calculated. By digitizing the facial expression transfer result of the first facial expression transfer program, and using the transfer evaluation result to measure the facial expression transfer effect, the evaluation of the transfer effect of the facial expression transfer program is made more objective. Compared with the subjective evaluation of the transfer effect by humans, the evaluation result of the transfer effect obtained by this method has higher reliability and accuracy.

[0091] Figure 2 A flowchart illustrating another facial expression transfer evaluation method provided in this application embodiment; optionally, in step S101, before loading the facial expression transfer test set, this method may include:

[0092] S201. Run the first expression transfer program to perform expression transfer on each initial video in the initial set of expression transfer and obtain the initial blending deformation parameters corresponding to each initial video.

[0093] This embodiment provides a detailed explanation of how to obtain the standard hybrid deformation parameters for each test video in the facial expression transfer test set.

[0094] In one feasible approach, the first expression migration program can be run first to obtain the initial blending deformation parameters corresponding to each initial video. Based on the obtained initial blending deformation parameters, the user's parameters can be adjusted to obtain the standard blending deformation parameters for each initial video.

[0095] It is worth noting here that each initial video in the initial set of expression transfer refers to each test video in step S101 above, while the initial hybrid deformation parameters obtained after running the first expression transfer program can refer to the predicted hybrid deformation parameters of each initial video obtained when the first expression transfer program is used to perform expression transfer processing on each initial video.

[0096] The execution process of this step is similar to that of step S102. That is, the same steps are performed in the process of obtaining the standard hybrid deformation parameters of each test video and in the process of evaluating the migration effect of the first expression migration program by applying the test videos.

[0097] S202, respond to the adjustment operation of the initial blending deformation parameters of each initial video in each frame until the expression of the object in the initial video and the expression of the virtual model after expression transfer meet the preset similarity, and use the current initial blending deformation parameters as the standard blending deformation parameters of each initial video in each frame.

[0098] During the process of running the first expression transfer program, the initial video and the virtual model can be displayed simultaneously on the graphical user interface. The expression of the virtual model is obtained by transferring the expression of the object in the initial video. Users can watch the difference between the expression of the virtual model and the expression of the object in the initial video in real time.

[0099] During the expression transfer process, the graphical user interface can display the initial blending deformation parameters of each initial video in each video frame in real time. That is, it can display the initial blending deformation parameters of each initial video in each video frame after the first expression transfer program has performed the calculations.

[0100] Due to the accuracy issues of the first expression transfer procedure, the accuracy of the initial blending deformation parameters of the obtained initial video in each video frame may be poor. The quality of expression transfer depends to some extent on the accuracy of the initial blending deformation parameters of the obtained initial video. The expression of the virtual model obtained after expression transfer based on the initial blending deformation parameters may differ from the expression of the object in the initial video.

[0101] Optionally, the system can respond to user input regarding adjustments to the initial blending deformation parameters for each initial video in each frame. During the adjustment process, the virtual model is observed in real time until the similarity between the facial expressions of the objects in the initial video and the facial expressions of the virtual model in each frame meets a preset threshold. If this is the case, it can be considered that the facial expression transfer effect obtained after performing facial expression transfer on the initial video under the currently adjusted initial blending deformation parameters is good, and the current initial blending deformation parameters can be used as the standard blending deformation parameters for each video frame.

[0102] This can be done by adjusting the initial blending deformation parameters of each initial video at each frame, or by selecting only preset keyframes and adjusting only the initial blending deformation parameters of the keyframes to obtain the standard blending deformation parameters of each initial video at each keyframe in order to reduce the amount of computation.

[0103] In one feasible approach, a standard blending deformer parameter file script can be pre-created in the Unity editor. This script can be loaded into the Unity editor as a plugin, allowing the standard blending deformer parameters of each initial video to be stored in the created standard blending deformer parameter file. During testing, the steps described above are followed: first, the facial expression migration test set is loaded to load each test video and its standard blending deformer parameters.

[0104] S203. Obtain the standard hybrid deformation parameters of each initial video based on the standard hybrid deformation parameters of each initial video in each frame.

[0105] The above describes the operations performed on each frame of each initial video, resulting in the standard blending deformation parameters of each initial video in each frame. Combining the standard blending deformation parameters of each frame yields the standard blending deformation parameters of each initial video. In other words, the standard blending deformation parameters of each initial video include the standard blending deformation parameters of each initial video in each frame.

[0106] S204. Based on each initial video and the standard hybrid deformation parameters of each initial video, obtain the expression transfer test set.

[0107] The expression transfer test set can include each initial video and the standard hybrid deformation parameters of each initial video. As explained above, the initial video refers to the test video. In the obtained expression transfer test set, the initial video can be called the test video, and the standard hybrid deformation parameters of each initial video correspond to the standard hybrid deformation parameters of each test video.

[0108] Optionally, the predicted hybrid deformation parameters for each test video include: frame identifier, predicted motion parameters for different parts of the face in each frame, and weights for each motion parameter in each frame.

[0109] Since the predicted hybrid deformation parameters of each test video are obtained by combining the predicted hybrid deformation parameters of each test video in each frame, in order to facilitate the reading and calling of the predicted hybrid deformation parameters of each frame, in this embodiment, the predicted hybrid deformation parameters of each test video obtained above may include: frame identifier (i.e., frame number), predicted motion parameters of different parts of the lower part of each frame, and weights of each motion parameter in each frame.

[0110] By using frame identifiers, the predicted motion parameters and weights of different parts of the lower part of each frame can be mapped one-to-one. During the call, the predicted motion parameters and weights of different parts of the lower part of the frame can be obtained through the frame identifier.

[0111] In each frame of the test video, the face of the object will have different expressions, and the expressions are generated by the movements of each part of the face. For example, the expression of crying may correspond to the movement of the corners of the mouth drooping, and the eyes may correspond to the movement of slightly closing. Each movement corresponds to motion parameters, so the predicted motion parameters of different parts of the face in each frame can be obtained.

[0112] The weights of each motion parameter in each frame can be set by the user based on the actual situation. For actions that have a greater impact on the final expression transfer effect, the motion parameters can be set to higher values. For example, the closing and opening of the eyes has a significant impact on the overall expression, so the weight of the eye-closing or eye-opening actions can be set to higher values. Alternatively, the weight of the actions of the facial parts that are of interest in the test video can be set to higher values. For example, if the test video focuses on a single video of closed eyes, and the focus is on the eyes, while the nose, mouth, and other parts are not the focus, then the weight of the eye-closing and eye-opening actions can be set to higher values.

[0113] By setting the weights of each action parameter appropriately, the accuracy of the evaluation results can be improved.

[0114] Similarly, the standard hybrid deformation parameters for each test video include: frame identifier, standard motion parameters for different parts of the face in each frame, and weights for each motion parameter in each frame.

[0115] Here, the only difference between the standard hybrid deformation parameters and the predicted hybrid deformation parameters of each test video is that the motion parameters of different parts of the lower face in each frame of the standard hybrid deformation parameters are the pre-created standard motion parameters, while the motion parameters of different parts of the lower face in each frame of the predicted hybrid deformation parameters are the predicted motion parameters obtained by using the first expression transfer procedure.

[0116] Optionally, in step S103, determining the migration evaluation result of the first expression transfer procedure based on the predicted blending deformation parameters corresponding to each test video and the standard blending deformation parameters corresponding to each test video may include: calculating the migration evaluation result of the first expression transfer procedure using Euclidean distance based on the predicted blending deformation parameters corresponding to each test video and the standard blending deformation parameters corresponding to each test video.

[0117] In this embodiment, the process of obtaining the migration evaluation results of the first expression migration program can be automated. After loading the expression migration test set, the standard hybrid deformation parameters of each frame in each test video can be displayed in a list on the graphical user interface. Running the first expression migration program can execute each test video in sequence and obtain the predicted hybrid deformation parameters of each test video in each frame, which can also be displayed on the graphical user interface.

[0118] In response to user-input migration evaluation commands, the system can calculate the Euclidean distance based on the standard hybrid deformation parameters and the predicted hybrid deformation parameters of each test video in each frame, and display the calculation results on the graphical user interface.

[0119] Optionally, users can also modify the weights of the predicted motion parameters for different parts of the lower part of each test video in each frame, as displayed on the graphical user interface, to optimize them.

[0120] Alternatively, in one possible implementation, in response to a user-inputted evaluation command for the target frame, Euclidean distance can be calculated based on the standard hybrid deformation parameters and the predicted hybrid deformation parameters of the target frame in the test video to obtain the migration evaluation result when the first expression migration procedure is applied to the target frame.

[0121] In another possible approach, in response to a user-inputted evaluation instruction that does not specify any frame, Euclidean distance can be calculated based on the standard blending deformation parameters and the predicted blending deformation parameters of the test video to obtain the migration evaluation result when the first expression transfer procedure is applied to the test video.

[0122] Figure 3 This is a schematic diagram of a graphical user interface provided for an embodiment of this application. Figure 3As shown, the left side of the graphical user interface displays the test video and virtual model. The right side displays the control panel. When the user loads the test video onto the test video position on the left side of the graphical user interface, the standard hybrid deformation parameters of each frame in the test video can be displayed in the control panel on the right. Responding to evaluation commands, the evaluation process can be executed automatically, calculating the Euclidean distance between the standard and predicted hybrid deformation parameters of each frame in the test video to obtain the migration evaluation results for each frame. These migration evaluation results can also be displayed in the corresponding positions on the right side of the graphical user interface and can be stored in a pre-created evaluation result file for easy viewing by the user.

[0123] During the automated evaluation process, the virtual model on the left is given the same facial expression as the object in the test video. That is, the facial expression of the object in the test video is transferred to the virtual model, and the slight difference between the facial expression of the virtual model and the facial expression of the object in the test video is determined by the accuracy of the first facial expression transfer program.

[0124] The automated evaluation achieved by this method can improve the convenience of evaluating facial expression transfer procedures, while also ensuring high accuracy of the evaluation results.

[0125] Figure 4 This is a flowchart illustrating another expression transfer evaluation method provided in an embodiment of this application; optionally, in the above steps, the transfer evaluation result of the first expression transfer procedure is obtained by calculating the Euclidean distance based on the predicted hybrid deformation parameters corresponding to each test video and the standard hybrid deformation parameters corresponding to each test video, which may include:

[0126] S301. Calculate the difference between the predicted motion parameters of different parts of the face and the standard motion parameters of different parts of the face in each target frame of each test video and take the absolute value. Then multiply the absolute value by the weight of each motion parameter in each target frame to obtain the evaluation result of each test video in each target frame. The target frame is any frame in each frame.

[0127] Here, only the target frame can be selected for calculation to reduce the amount of computation. The target frame can be some key frames, or the calculation can be performed on each frame.

[0128] Optionally, the predicted motion parameters of different parts of the face and the standard motion parameters of different parts of the face in the target frame can be subtracted and the absolute values ​​can be taken. The absolute values ​​are then multiplied by the weights of the motion parameters to obtain the evaluation results of each test video in each target frame.

[0129] For example, the predicted motion parameters for different parts of the face in the test video at the target frame include: predicted eye motion parameter 1, predicted mouth motion parameter 2, and predicted eyebrow motion parameter 3. The standard motion parameters for different parts of the face in the test video at the target frame include: standard eye motion parameter 1, standard mouth motion parameter 2, and standard eyebrow motion parameter 3. The weight of the eye motion parameter is 'a', the weight of the mouth motion parameter is 'b', and the weight of the eyebrow motion parameter is 'c'. Then, the evaluation result of the test video at the target frame can be calculated as follows: |Standard eye motion parameter 1 - Predicted eye motion parameter 1|·a + |Standard mouth motion parameter 2 - Predicted mouth motion parameter 2|·b + |Standard eyebrow motion parameter 3 - Predicted eyebrow motion parameter 3|·c.

[0130] S302. Take the average value of the evaluation results of each test video under each target frame to obtain the evaluation result corresponding to each test video.

[0131] In one feasible approach, the average value of the evaluation results of each test video at each target frame can be calculated, and the calculated result can be used as the evaluation result for each test video.

[0132] S303. Based on the evaluation results corresponding to each test video, obtain the migration evaluation results of the first expression migration program.

[0133] Alternatively, the evaluation results for each test video can be averaged to obtain the transfer evaluation result of the first expression transfer procedure. In this process, weights can be assigned to the evaluation results of each test video; for test videos where the subject has expressions that significantly impact the expression transfer effect, a larger weight can be assigned to the evaluation result.

[0134] The average value obtained can be used as the overall transfer evaluation result for the first expression transfer procedure.

[0135] Of course, the evaluation result of any test video can also be used as the transfer evaluation result of the first expression transfer procedure. In this case, the transfer evaluation of the first expression transfer procedure can be an evaluation of the transfer effect of the facial movements corresponding to the test video. For example, if the test video is a video of a closed-eye movement, when the evaluation result of the test video is used as the transfer evaluation result of the first expression transfer procedure, it is the transfer evaluation result of the first expression transfer procedure when performing eye movement expression transfer.

[0136] Figure 5 A flowchart illustrating another facial expression transfer evaluation method provided in this application embodiment; optionally, in step S103, after determining the transfer evaluation result of the first facial expression transfer procedure based on the predicted hybrid deformation parameters corresponding to each test video and the standard hybrid deformation parameters corresponding to each test video, the method of this application may include:

[0137] S401. Run the second expression transfer program and determine the transfer evaluation result of the second expression transfer program based on the running results. The second expression transfer program is a different expression transfer program from the first expression transfer program.

[0138] The transfer evaluation results of the facial expression transfer procedure obtained by this method can be used to evaluate different facial expression transfer procedures in order to select the better facial expression transfer procedure.

[0139] Optionally, the first expression transfer program can be replaced by a second expression transfer program. By executing the steps of this method, the transfer evaluation result of the second expression transfer program can be obtained. The second expression transfer program is different from the first expression transfer program, but the expression transfer evaluation method executed is the same.

[0140] S402. Based on the migration evaluation results of the second expression transfer procedure and the migration evaluation results of the first expression transfer procedure, determine the target expression transfer procedure.

[0141] Then, the target expression transfer algorithm can be determined based on the transfer evaluation results of the second expression transfer procedure and the transfer evaluation results of the first expression transfer procedure.

[0142] In this case, the expression transfer program with the smaller transfer evaluation result among the second expression transfer program and the first expression transfer program can be used as the target expression transfer program.

[0143] Optionally, in step S103, after determining the migration evaluation result of the first expression migration program based on the predicted hybrid deformation parameters corresponding to each test video and the standard hybrid deformation parameters corresponding to each test video, the method of this application may further include: optimizing the parameters in the first expression migration algorithm, obtaining a new migration evaluation result of the first expression migration program, iteratively executing until the new migration evaluation result and the migration evaluation result of the first expression migration program meet a preset condition, and then using the current parameters of the first expression migration algorithm as the target parameters of the first expression migration algorithm.

[0144] The migration evaluation results of the expression transfer program obtained by this method can also be used to optimize the parameters of the currently running expression transfer program. The method of this application can be iteratively executed to obtain new migration evaluation results for the first expression transfer program. After each iteration, the new migration evaluation result is compared with the migration evaluation result of the first expression transfer program (which can be referred to as the initial migration evaluation result for distinction). If the new migration evaluation result is less than the initial migration evaluation result, and the difference between the new and initial migration evaluation results meets a preset accuracy threshold, then the parameters of the first expression transfer program run in this iteration can be used as the target parameters of the first expression transfer algorithm. If the new migration evaluation result does not meet the above conditions, the parameters of the first expression transfer program are further optimized, and the method of this application is executed.

[0145] Figure 6 A flowchart illustrating another expression transfer evaluation method provided in this application embodiment; optionally, in step S201, before running the first expression transfer program to perform expression transfer on each initial video in the initial expression transfer set, the method of this application may further include:

[0146] S501. Create multiple facial expression transfer test cases, each corresponding to different facial expressions and / or facial features.

[0147] Optionally, the expression transfer test cases may include multiple test cases, each of which may correspond to a facial expression and / or facial feature. Figure 7 This is a schematic diagram of an expression transfer test case provided in an embodiment of this application, which may include: mouth action test cases, eyebrow and eyelid action test cases, eyeball action test cases, and combined expression test cases. The eyebrow and eyelid action test cases may include: single / double eyelid closure, slow blinking (once per second), slow blinking while wearing glasses, frowning, and double eyelid raising.

[0148] Figure 7 The examples shown are just a few possible test cases. In actual applications, there are many more test cases. The test cases should include as many key facial expressions as possible, and they will not be listed one by one here.

[0149] S502. Based on each facial expression transfer test case, record the corresponding test video for each facial expression transfer test case in a standard recording environment.

[0150] Based on the created facial expression transfer test cases, test videos corresponding to each test case can be recorded. For example, for the frowning test case, the recorded test video shows the object performing the frowning action.

[0151] It should be noted that the test video can be recorded on the same person to avoid the impact of differences caused by different people making the same expression on the evaluation results.

[0152] Furthermore, test videos can be recorded against a standard green screen background to address environmental interference and lighting issues, making facial expressions the sole variable. This reduces the complexity of problem identification and enables more accurate and efficient testing.

[0153] Using pre-recorded test videos for testing reduces hardware costs, screen projection costs, and recording time costs compared to using a camera to record test videos in real time, thus saving resources and improving testing efficiency.

[0154] Optionally, in step S201, running the first expression transfer program to perform expression transfer on each test video in the expression transfer test set may include: running the first expression transfer program to perform expression transfer on each test video in the expression transfer test set, and assigning the virtual model displayed by the graphical user interface to the expression corresponding to each test video, wherein the expression corresponding to each test video is obtained according to the predicted hybrid deformation parameters corresponding to each test video.

[0155] Expression transfer processing involves transferring the facial expressions of objects in a video to a virtual model for display, thus synchronizing the virtual model with the expressions of the objects in the video. In this embodiment, a first expression transfer program is run to transfer expressions to each test video, thereby assigning the virtual model displayed on the graphical user interface with the corresponding facial expressions of the objects in the test videos. The facial expressions of the objects in the test videos can be obtained based on the predicted hybrid deformation parameters of the test videos calculated by the first expression transfer program.

[0156] It is worth noting that the virtual model used for facial expression transfer was the same in all test videos. This avoids the virtual model affecting the facial expression transfer effect due to different virtual models used, which could lead to poor accuracy in the facial expression transfer evaluation results.

[0157] Optionally, when the test video and virtual model change, the method of this application can be re-executed to regenerate the standard hybrid deformation parameters of the test video, so as to ensure the accuracy of the generated standard hybrid deformation parameters.

[0158] In summary, the facial expression transfer evaluation method provided in this application includes: loading a facial expression transfer test set, which includes multiple test videos and standard hybrid deformation parameters corresponding to each test video, wherein the objects in each test video have different test facial expressions; running a first facial expression transfer program to perform facial expression transfer on each test video in the facial expression transfer test set, and obtaining the predicted hybrid deformation parameters corresponding to each test video; and determining the transfer evaluation result of the first facial expression transfer program based on the predicted hybrid deformation parameters corresponding to each test video and the standard hybrid deformation parameters corresponding to each test video. In this method, by introducing the standard hybrid deformation parameters of each test video, and by calculating the transfer evaluation result of the first facial expression transfer program based on the standard hybrid deformation parameters of each test video and the predicted hybrid deformation parameters of each test video inferred using the first facial expression transfer program, the facial expression transfer evaluation result can be obtained. By digitizing the facial expression transfer result of the first facial expression transfer program and measuring the facial expression transfer effect through the transfer evaluation result, the evaluation of the transfer effect of the facial expression transfer program becomes more objective. Compared with the subjective evaluation of the transfer effect by humans, the evaluation result of the transfer effect obtained by this method has higher reliability and accuracy.

[0159] Furthermore, using pre-recorded test videos reduces hardware, projection, and recording time costs compared to real-time camera recording, saving resources and improving testing efficiency. Recording test videos against a standard green screen background also eliminates environmental interference and lighting issues, making facial expressions the sole variable, reducing complex problem-solving steps, and enabling more accurate and efficient testing.

[0160] The following describes the apparatus, equipment, and storage medium used to implement the facial expression transfer evaluation method provided in this application. The specific implementation process and technical effects are described above and will not be repeated below.

[0161] Figure 8 This is a schematic diagram of an expression transfer evaluation device provided in an embodiment of this application. The functions implemented by this expression transfer evaluation device correspond to the steps performed by the method described above. This device can be understood as the aforementioned computer equipment, such as... Figure 8 As shown, the device may include: a loading module 710, a running module 720, and a determining module 730;

[0162] Loading module 710 is used to load the expression transfer test set, which includes multiple test videos and standard hybrid deformation parameters corresponding to each test video. The objects in each test video have different test expressions.

[0163] The running module 720 is used to run the first expression transfer program, perform expression transfer on each test video in the expression transfer test set, and obtain the prediction hybrid deformation parameters corresponding to each test video.

[0164] The determination module 730 is used to determine the migration evaluation result of the first expression migration program based on the predicted hybrid deformation parameters corresponding to each test video and the standard hybrid deformation parameters corresponding to each test video.

[0165] Optionally, the running module 720 is also used to run the first expression transfer program to perform expression transfer on each initial video in the initial set of expression transfer and obtain the initial blending deformation parameters corresponding to each initial video.

[0166] The system responds to the adjustment of the initial blending deformation parameters for each initial video in each frame until the expression of the object in the initial video and the expression of the virtual model after expression transfer meet the preset similarity. The current initial blending deformation parameters are then used as the standard blending deformation parameters for each initial video in each frame.

[0167] The standard hybrid deformation parameters of each initial video are obtained based on the standard hybrid deformation parameters of each initial video in each frame.

[0168] Based on each initial video and the standard hybrid deformation parameters of each initial video, an expression transfer test set is obtained.

[0169] Optionally, the predicted hybrid deformation parameters for each test video include: frame identifier, predicted motion parameters for different parts of the face in each frame, and weights of each motion parameter in each frame;

[0170] The standard hybrid deformation parameters for each test video include: frame identifier, standard motion parameters for different parts of the face in each frame, and weights for each motion parameter in each frame.

[0171] Optionally, module 730 is specifically used to calculate the migration evaluation results of the first expression migration program by using Euclidean distance based on the predicted hybrid deformation parameters corresponding to each test video and the standard hybrid deformation parameters corresponding to each test video.

[0172] Optionally, the determination module 730 is specifically used to calculate the difference and take the absolute value of the predicted motion parameters of different parts of the face and the standard motion parameters of different parts of the face in each test video in each target frame, and then multiply the weight of each motion parameter in each target frame to obtain the evaluation result of each test video in each target frame. The target frame is any frame in each frame.

[0173] The average value of the evaluation results of each test video under each target frame is taken to obtain the evaluation result corresponding to each test video.

[0174] Based on the evaluation results corresponding to each test video, the migration evaluation results of the first expression transfer program are obtained.

[0175] Optionally, the running module 720 is also used to run a second expression transfer program and determine the transfer evaluation result of the second expression transfer program based on the running result. The second expression transfer program is a different expression transfer program from the first expression transfer program.

[0176] Indeed, module 730 is also used to determine the target expression transfer procedure based on the transfer evaluation results of the second expression transfer procedure and the transfer evaluation results of the first expression transfer procedure.

[0177] Optionally, the device further includes: an optimization module;

[0178] The optimization module is used to optimize the parameters in the first expression transfer algorithm, obtain the new transfer evaluation result of the first expression transfer program, and iteratively execute until the new transfer evaluation result and the transfer evaluation result of the first expression transfer program meet the preset conditions. Then, the current parameters of the first expression transfer algorithm are used as the target parameters of the first expression transfer algorithm.

[0179] Optionally, the apparatus further includes: a creation module;

[0180] The creation module is used to create multiple facial expression transfer test cases, each corresponding to different facial expressions and / or facial features;

[0181] Based on each facial expression transfer test case, record the corresponding test video for each facial expression transfer test case in a standard recording environment.

[0182] Optionally, module 720 is used specifically for

[0183] Run the first expression transfer program to transfer expressions to each test video in the expression transfer test set, and assign the corresponding expressions to the virtual model displayed in the graphical user interface. The expressions corresponding to each test video are obtained based on the predicted hybrid deformation parameters corresponding to each test video.

[0184] In this manner, after loading the expression transfer test set, the loading module runs the first expression transfer program to perform expression transfer on each test video in the test set, obtaining the predicted blending deformation parameters corresponding to each test video. The determining module then determines the transfer evaluation result of the first expression transfer program based on the predicted blending deformation parameters and the standard blending deformation parameters corresponding to each test video. Because the standard blending deformation parameters of each test video are introduced, the transfer evaluation result of the first expression transfer program can be calculated based on these parameters and the predicted blending deformation parameters inferred from the first expression transfer program. By digitizing the expression transfer results of the first expression transfer program and using the transfer evaluation results to measure the expression transfer effect, the evaluation of the expression transfer program's transfer effect becomes more objective. Compared to subjective human evaluation of the transfer effect, the evaluation results obtained by this method have higher reliability and accuracy.

[0185] These modules can be one or more integrated circuits configured to implement the above methods, such as one or more Application Specific Integrated Circuits (ASICs), one or more digital signal processors (DSPs), or one or more Field Programmable Gate Arrays (FPGAs). Alternatively, when a module is implemented using processing element scheduler code, the processing element can be a general-purpose processor, such as a Central Processing Unit (CPU) or other processor capable of calling program code. Furthermore, these modules can be integrated together as a system-on-a-chip (SOC).

[0186] The modules described above can be connected or communicate with each other via wired or wireless connections. Wired connections can include metal cables, optical fibers, hybrid cables, or any combination thereof. Wireless connections can include connections via LAN, WAN, Bluetooth, ZigBee, or NFC, or any combination thereof. Two or more modules can be combined into a single module, and any module can be divided into two or more units. Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems and devices described above can be referred to the corresponding processes in the method embodiments, and will not be repeated here.

[0187] Figure 9This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device can be the aforementioned computer device, including: a processor 801, a storage medium 802, and a bus 803. The storage medium 802 stores machine-readable instructions executable by the processor 801. When the electronic device runs an expression transfer evaluation method as described in the embodiment, the processor 801 communicates with the storage medium 802 via the bus 803. The processor 801 executes the machine-readable instructions to perform the following steps:

[0188] Load the facial expression transfer test set, which includes multiple test videos and standard hybrid deformation parameters corresponding to each test video. The objects in each test video have different test expressions.

[0189] Run the first expression transfer program to perform expression transfer on each test video in the expression transfer test set and obtain the prediction hybrid deformation parameters corresponding to each test video;

[0190] Based on the predicted hybrid deformation parameters corresponding to each test video and the standard hybrid deformation parameters corresponding to each test video, the migration evaluation result of the first expression migration program is determined.

[0191] In a feasible implementation, before executing the loading of the expression transfer test set, the processor 801 is also used to: run the first expression transfer program, perform expression transfer on each initial video in the initial expression transfer set, and obtain the initial blending deformation parameters corresponding to each initial video.

[0192] The system responds to the adjustment of the initial blending deformation parameters for each initial video in each frame until the expression of the object in the initial video and the expression of the virtual model after expression transfer meet the preset similarity. The current initial blending deformation parameters are then used as the standard blending deformation parameters for each initial video in each frame.

[0193] The standard hybrid deformation parameters of each initial video are obtained based on the standard hybrid deformation parameters of each initial video in each frame.

[0194] The expression transfer test set is obtained based on each initial video and the standard hybrid deformation parameters of each initial video.

[0195] In a feasible implementation, the predicted hybrid deformation parameters of each test video include: frame identifier, predicted motion parameters for different parts of the face under each frame, and weights of each motion parameter under each frame;

[0196] The standard hybrid deformation parameters for each test video include: frame identifier, standard motion parameters for different parts of the face in each frame, and weights for each motion parameter in each frame.

[0197] In a feasible implementation, when the processor 801 determines the migration evaluation result of the first expression transfer program based on the predicted blending deformation parameters corresponding to each test video and the standard blending deformation parameters corresponding to each test video, it specifically performs the following: based on the predicted blending deformation parameters corresponding to each test video and the standard blending deformation parameters corresponding to each test video, it uses Euclidean distance calculation to obtain the migration evaluation result of the first expression transfer program.

[0198] In a feasible implementation, when the processor 801 executes the Euclidean distance calculation based on the predicted hybrid deformation parameters corresponding to each test video and the standard hybrid deformation parameters corresponding to each test video to obtain the migration evaluation result of the first expression migration program, it specifically performs the subtraction and takes the absolute value of the predicted motion parameters of different parts of the face and the standard motion parameters of different parts of the face in each target frame of each test video, and then multiplies the weight of each motion parameter in each target frame to obtain the evaluation result of each test video in each target frame, where the target frame is any frame in each frame;

[0199] The average value of the evaluation results of each test video under each target frame is taken to obtain the evaluation result corresponding to each test video.

[0200] Based on the evaluation results corresponding to each test video, the migration evaluation results of the first expression migration program are obtained.

[0201] In one feasible implementation, after the processor 801 determines the migration evaluation result of the first expression migration program based on the predicted blending deformation parameters corresponding to each test video and the standard blending deformation parameters corresponding to each test video, it is also used to run a second expression migration program and determine the migration evaluation result of the second expression migration program based on the running result. The second expression migration program is a different expression migration program from the first expression migration program.

[0202] The target expression transfer procedure is determined based on the transfer evaluation results of the second expression transfer procedure and the transfer evaluation results of the first expression transfer procedure.

[0203] In a feasible implementation, after the processor 801 determines the migration evaluation result of the first expression migration program based on the predicted hybrid deformation parameters and the standard hybrid deformation parameters corresponding to each test video, it is further used to optimize the parameters in the first expression migration algorithm, obtain a new migration evaluation result of the first expression migration program, and iterate until the new migration evaluation result and the migration evaluation result of the first expression migration program meet a preset condition. Then, the current parameters of the first expression migration algorithm are used as the target parameters of the first expression migration algorithm.

[0204] In one feasible implementation, before executing the first expression transfer program to perform expression transfer on each initial video in the initial expression transfer set, the processor 801 is also used to create multiple expression transfer test cases, each expression transfer test case corresponding to different facial expressions and / or facial features.

[0205] Based on each facial expression transfer test case, record the corresponding test video for each facial expression transfer test case in a standard recording environment.

[0206] In one feasible implementation, when the processor 801 executes the first expression transfer program to perform expression transfer on each test video in the expression transfer test set, it specifically runs the first expression transfer program to perform expression transfer on each test video in the expression transfer test set and assigns the corresponding expression to the virtual model displayed by the graphical user interface. The expression corresponding to each test video is obtained based on the predicted blending deformation parameters corresponding to each test video. In this way, after the loading module loads the expression transfer test set, the running module runs the first expression transfer program to perform expression transfer on each test video in the expression transfer test set and obtains the predicted blending deformation parameters corresponding to each test video; and the determining module determines the transfer evaluation result of the first expression transfer program based on the predicted blending deformation parameters and the standard blending deformation parameters corresponding to each test video. By introducing standard hybrid deformation parameters for each test video, and based on these parameters and the predicted hybrid deformation parameters inferred from the first expression transfer procedure, the transfer evaluation results of the first expression transfer procedure can be calculated. By digitizing the expression transfer results of the first expression transfer procedure and using the transfer evaluation results to measure the expression transfer effect, the evaluation of the expression transfer effect is made more objective. Compared with the subjective evaluation of the transfer effect by humans, the evaluation results of the transfer effect obtained by this method have higher reliability and accuracy.

[0207] The storage medium 802 stores program code, which, when executed by the processor 801, causes the processor 801 to perform various steps in the methods according to various exemplary embodiments of this application described in the "Exemplary Methods" section above.

[0208] The processor 801 can be a general-purpose processor, such as a central processing unit (CPU), digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.

[0209] Storage medium 802, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. The memory can include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card-type memory, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic memory, magnetic disk, optical disk, etc. Memory is any other medium capable of carrying or storing desired program code having an instruction or data structure form and accessible by a computer, but is not limited thereto. Storage medium 802 in this embodiment can also be a circuit or any other device capable of implementing storage functions for storing program instructions and / or data.

[0210] Optionally, embodiments of this application also provide a computer-readable storage medium storing a computer program, which is executed by a processor, and the processor performs the following steps:

[0211] Load the facial expression transfer test set, which includes multiple test videos and standard hybrid deformation parameters corresponding to each test video. The objects in each test video have different test expressions.

[0212] Run the first expression transfer program to perform expression transfer on each test video in the expression transfer test set and obtain the prediction hybrid deformation parameters corresponding to each test video;

[0213] Based on the predicted hybrid deformation parameters corresponding to each test video and the standard hybrid deformation parameters corresponding to each test video, the migration evaluation result of the first expression migration program is determined.

[0214] In a feasible implementation, before executing the loading of the expression transfer test set, the processor 801 is also used to: run the first expression transfer program, perform expression transfer on each initial video in the initial expression transfer set, and obtain the initial blending deformation parameters corresponding to each initial video.

[0215] The system responds to the adjustment of the initial blending deformation parameters for each initial video in each frame until the expression of the object in the initial video and the expression of the virtual model after expression transfer meet the preset similarity. The current initial blending deformation parameters are then used as the standard blending deformation parameters for each initial video in each frame.

[0216] The standard hybrid deformation parameters of each initial video are obtained based on the standard hybrid deformation parameters of each initial video in each frame.

[0217] The expression transfer test set is obtained based on each initial video and the standard hybrid deformation parameters of each initial video.

[0218] In a feasible implementation, the predicted hybrid deformation parameters of each test video include: frame identifier, predicted motion parameters for different parts of the face under each frame, and weights of each motion parameter under each frame;

[0219] The standard hybrid deformation parameters for each test video include: frame identifier, standard motion parameters for different parts of the face in each frame, and weights for each motion parameter in each frame.

[0220] In a feasible implementation, when the processor 801 determines the migration evaluation result of the first expression transfer program based on the predicted blending deformation parameters corresponding to each test video and the standard blending deformation parameters corresponding to each test video, it specifically performs the following: based on the predicted blending deformation parameters corresponding to each test video and the standard blending deformation parameters corresponding to each test video, it uses Euclidean distance calculation to obtain the migration evaluation result of the first expression transfer program.

[0221] In a feasible implementation, when the processor 801 executes the Euclidean distance calculation based on the predicted hybrid deformation parameters corresponding to each test video and the standard hybrid deformation parameters corresponding to each test video to obtain the migration evaluation result of the first expression migration program, it specifically performs the subtraction and takes the absolute value of the predicted motion parameters of different parts of the face and the standard motion parameters of different parts of the face in each target frame of each test video, and then multiplies the weight of each motion parameter in each target frame to obtain the evaluation result of each test video in each target frame, where the target frame is any frame in each frame;

[0222] The average value of the evaluation results of each test video under each target frame is taken to obtain the evaluation result corresponding to each test video.

[0223] Based on the evaluation results corresponding to each test video, the migration evaluation results of the first expression migration program are obtained.

[0224] In one feasible implementation, after the processor 801 determines the migration evaluation result of the first expression migration program based on the predicted blending deformation parameters corresponding to each test video and the standard blending deformation parameters corresponding to each test video, it is also used to run a second expression migration program and determine the migration evaluation result of the second expression migration program based on the running result. The second expression migration program is a different expression migration program from the first expression migration program.

[0225] The target expression transfer procedure is determined based on the transfer evaluation results of the second expression transfer procedure and the transfer evaluation results of the first expression transfer procedure.

[0226] In a feasible implementation, after the processor 801 determines the migration evaluation result of the first expression migration program based on the predicted hybrid deformation parameters and the standard hybrid deformation parameters corresponding to each test video, it is further used to optimize the parameters in the first expression migration algorithm, obtain a new migration evaluation result of the first expression migration program, and iterate until the new migration evaluation result and the migration evaluation result of the first expression migration program meet a preset condition. Then, the current parameters of the first expression migration algorithm are used as the target parameters of the first expression migration algorithm.

[0227] In one feasible implementation, before executing the first expression transfer program to perform expression transfer on each initial video in the initial expression transfer set, the processor 801 is also used to create multiple expression transfer test cases, each expression transfer test case corresponding to different facial expressions and / or facial features.

[0228] Based on each facial expression transfer test case, record the corresponding test video for each facial expression transfer test case in a standard recording environment.

[0229] In one feasible implementation, when the processor 801 executes the first expression transfer program to perform expression transfer on each test video in the expression transfer test set, it is specifically used to run the first expression transfer program to perform expression transfer on each test video in the expression transfer test set, and assign the virtual model displayed by the graphical user interface to the expression corresponding to each test video. The expression corresponding to each test video is obtained according to the predicted hybrid deformation parameters corresponding to each test video.

[0230] In this manner, after loading the expression transfer test set, the loading module runs the first expression transfer program to perform expression transfer on each test video in the test set, obtaining the predicted blending deformation parameters corresponding to each test video. The determining module then determines the transfer evaluation result of the first expression transfer program based on the predicted blending deformation parameters and the standard blending deformation parameters corresponding to each test video. Because the standard blending deformation parameters of each test video are introduced, the transfer evaluation result of the first expression transfer program can be calculated based on these parameters and the predicted blending deformation parameters inferred from the first expression transfer program. By digitizing the expression transfer results of the first expression transfer program and using the transfer evaluation results to measure the expression transfer effect, the evaluation of the expression transfer program's transfer effect becomes more objective. Compared to subjective human evaluation of the transfer effect, the evaluation results obtained by this method have higher reliability and accuracy.

[0231] In this embodiment, the computer program, when run by the processor, can also execute other machine-readable instructions to perform other methods as described in the embodiments. For details on the specific execution steps and principles, please refer to the description of the embodiments, which will not be repeated here.

[0232] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0233] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0234] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0235] The integrated units implemented as software functional units described above can be stored in a computer-readable storage medium. These software functional units, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute some steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A method of expression migration assessment, the method comprising: Applied to a computer device displaying a graphical user interface, the method comprises: loading an expression migration test set, the expression migration test set comprising a plurality of test videos and standard mixed deformation parameters corresponding to each test video, an object in each test video having different test expressions; running a first expression migration program to perform expression migration on each test video in the expression migration test set to obtain predicted mixed deformation parameters corresponding to each test video; determining a migration evaluation result of the first expression migration program according to the predicted mixed deformation parameters corresponding to each test video and the standard mixed deformation parameters corresponding to each test video; the determination of the migration evaluation result of the first expression migration program according to the predicted mixed deformation parameters corresponding to each test video and the standard mixed deformation parameters corresponding to each test video comprises: performing difference and taking absolute value on the predicted motion parameters of different parts of the face under each target frame and the standard motion parameters of different parts of the face of each test video, and then point-multiplying the weights of each motion parameter under each target frame to obtain an evaluation result of each test video under each target frame, the target frame being any frame in each frame; averaging the evaluation results of each test video under each target frame to obtain an evaluation result corresponding to each test video; obtaining the migration evaluation result of the first expression migration program according to the evaluation result corresponding to each test video.

2. The method of claim 1, wherein, Before the loading of the expression migration test set, the method comprises: running a first expression migration program to perform expression migration on each initial video in an expression migration initial set to obtain initial mixed deformation parameters corresponding to each initial video; in response to an adjustment operation on the initial mixed deformation parameters of each initial video under each frame, until the expression of the object in the initial video and the expression of the virtual model after expression migration satisfy a preset similarity, taking the current initial mixed deformation parameters as the standard mixed deformation parameters of each initial video under each frame; obtaining the standard mixed deformation parameters of each initial video according to the standard mixed deformation parameters of each initial video under each frame; obtaining the expression migration test set according to each initial video and the standard mixed deformation parameters of each initial video.

3. The method according to claim 1 or 2, characterized in that, The predicted mixed deformation parameters of each test video comprise frame identifiers, predicted motion parameters of different parts of the face under each frame, and weights of each motion parameter under each frame; The standard mixed deformation parameters of each test video comprise frame identifiers, standard motion parameters of different parts of the face under each frame, and weights of each motion parameter under each frame.

4. The method of claim 1, wherein, After the determination of the migration evaluation result of the first expression migration program according to the predicted mixed deformation parameters corresponding to each test video and the standard mixed deformation parameters corresponding to each test video, the method comprises: running a second expression migration program and determining a migration evaluation result of the second expression migration program according to a running result, the second expression migration program being different from the first expression migration program; determining a target expression migration program according to the migration evaluation result of the second expression migration program and the migration evaluation result of the first expression migration program.

5. The method of claim 1, wherein, After determining the migration evaluation result of the first expression transfer procedure according to the predicted mixed deformation parameters corresponding to each test video and the standard mixed deformation parameters corresponding to each test video, the method comprises: Optimizing the parameters in the first expression transfer algorithm to obtain a new migration evaluation result of the first expression transfer procedure, and iteratively executing until the new migration evaluation result and the migration evaluation result of the first expression transfer procedure satisfy a preset condition, and then taking the current parameters of the first expression transfer algorithm as target parameters of the first expression transfer algorithm.

6. The method of claim 2, wherein, Before running the first expression transfer procedure to perform expression transfer on each initial video in the initial expression transfer set, the method comprises: Creating a plurality of expression transfer test cases, each expression transfer test case corresponding to different expression actions and / or facial parts; According to each expression transfer test case, recording a test video corresponding to each expression transfer test case in a standard recording environment.

7. The method of claim 2, wherein, The method comprises: Running the first expression transfer procedure to perform expression transfer on each test video in the expression transfer test set, and assigning each test video corresponding expression to the virtual model displayed on the graphical user interface, the expression corresponding to each test video being obtained according to the predicted mixed deformation parameters corresponding to each test video.

8. An expression migration assessment apparatus characterized by comprising: The device is applied to a computer device displaying a graphical user interface, and the device comprises a loading module, a running module and a determining module. The loading module is configured to load an expression transfer test set, the expression transfer test set comprising a plurality of test videos and standard mixed deformation parameters corresponding to each test video, the object in each test video having different test expressions. The running module is configured to run a first expression transfer procedure to perform expression transfer on each test video in the expression transfer test set, and obtain predicted mixed deformation parameters corresponding to each test video. The determining module is configured to determine a migration evaluation result of the first expression transfer procedure according to the predicted mixed deformation parameters corresponding to each test video and the standard mixed deformation parameters corresponding to each test video. The determining module is specifically configured to subtract the standard action parameters of different parts of the face from the predicted action parameters of different parts of the face of each test video at each target frame, take the absolute value, and then multiply the weight of each action parameter at each target frame to obtain an evaluation result of each test video at each target frame, the target frame being any frame in the frames. An average value of the evaluation results of each test video at each target frame is obtained to obtain an evaluation result corresponding to each test video. The migration evaluation result of the first expression transfer procedure is obtained according to the evaluation result corresponding to each test video.

9. An electronic device, comprising: The device comprises a processor, a storage medium and a bus, the storage medium storing program instructions executable by the processor, the processor and the storage medium communicating through the bus when the electronic device is running, and the processor executing the program instructions to execute the steps of the method according to any one of claims 1 to 7. ​ 10. A computer-readable storage medium, characterized in that, The storage medium has stored thereon a computer program, which, when executed by a processor, performs the steps of the method according to any one of claims 1 to 7.

Images