Evaluation method, device and equipment
By constructing an evaluation image or model of facial expression evaluation criteria, the quality of digital human expression-based binding can be objectively evaluated and adjusted, solving the problem that binding quality depends on experience, and improving the fidelity and production efficiency of digital human facial animation.
Patent Information
- Application Number
- CN202310934271.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2023-06-02
- Filing Date
- 2023-07-27
- Publication Date
- 2024-12-03
AI Technical Summary
In the prior art, the evaluation of the quality of digital human expression-based binding depends on the experience of the binder, and lacks objectivity and standardization, resulting in poor binding effect and affecting the efficiency of digital human production.
By constructing an evaluation image or model of facial expression evaluation standards, the computing device obtains the expression base to be tested and evaluates it, generates the expression base group, calls the evaluation image or model for evaluation, obtains the binding quality evaluation data, and adjusts the binding based on the evaluation data Define strategies to improve binding quality.
The objective and standardized evaluation of digital human expression-based binding has been realized, the binding quality has been improved, and the fidelity and production efficiency of digital human facial animation has been improved.
Smart Images

Figure CN119068291A_ABST
Abstract
Description
[0001] This application claims the priority of the Chinese patent application filed with the China Patent Office on June 2, 2023, with application number 202310651371.5 and application name “A Method, Device and Equipment for Evaluation”, all contents of which are incorporated by reference into this application. Technical Field
[0002] The present application relates to the field of computer vision technology, and in particular to an evaluation method, device and equipment. Background Art
[0003] Digital human is a digital human image that is close to a real person and is synthesized by computer means. The three-dimensional (3D) digital human produced by combining computer graphics, computer vision and other technologies has a more realistic display effect and has therefore been widely used.
[0004] In the process of making 3D digital human, a very important process is to bind the expression base of 3D digital human. Expression base is applied to the facial expression of animation character. The binding artist or automatic binding algorithm assigns the expression base to the appropriate points on the face of digital human, and then drives the algorithm to learn the weights of different expression bases, so as to make the face of digital human move. If the binding effect of expression base is good, the points on the face of digital human corresponding to expression base are suitable, and the final expression animation has little deviation from real person. On the contrary, if the binding effect of expression base is poor, there may be problems such as not conforming to the actual movement of real person's face.
[0005] However, the evaluation of expression-based binding effects (or quality) is usually based on the judgment of binding artists based on experience, which lacks objectivity and cannot achieve a standardized binding process, thus adversely affecting the efficiency of digital human production. Summary of the invention
[0006] The embodiments of the present application provide an evaluation method, apparatus, computing device, device cluster, computer storage medium and computer program product, which can objectively evaluate the quality of digital human expression base binding, thereby improving the production effect.
[0007] In a first aspect, an embodiment of the present application provides an evaluation method, the method comprising: obtaining a plurality of expression bases to be tested, the plurality of expression bases to be tested being respectively bound to different points on the face of a target object; determining at least one expression base group, each expression base group comprising at least one of the plurality of expression bases to be tested, and all expression bases to be tested in each expression base group being used to form a facial expression; calling an evaluation image or an evaluation model, evaluating each expression base group to obtain evaluation data, the evaluation image and the evaluation model being used to characterize a facial expression evaluation standard, and the evaluation data being used to characterize a gap between the facial expression formed by the expression base group and the facial expression evaluation standard.
[0008] In this embodiment, an evaluation image or evaluation model as a facial expression evaluation standard can be pre-constructed on the server. For example, the evaluation image can include a large number of photos recording real facial expressions, and the evaluation model can be a classification model for identifying facial expressions, etc. In this way, after the digital human (i.e., the target object) completes the binding of facial expression bases, all the bound expression bases to be tested can be uploaded to the server and grouped according to the facial expressions that can be formed by the expression bases to be tested. Then, the facial expressions that can be formed by the expression base group are measured using the evaluation image or the evaluation model, and it can be determined whether the facial expressions of each expression base group are close to the evaluation standard represented by the evaluation image or the evaluation model. In this way, the binding quality of all the expression bases to be tested in the expression base group can be determined. If they are close to the evaluation standard, the binding quality is high, otherwise, the binding quality is low. In this way, objective and standardized quality evaluation can be achieved without relying on manual experience.
[0009] In some possible implementations, the expression base to be tested is uploaded by a user terminal, and the method includes: matching a target adjustment strategy from an adjustment strategy library based on evaluation data, the adjustment strategy library including one or more adjustment strategies, the adjustment strategy being used to adjust the binding state of the expression base, and the target adjustment strategy being one of the adjustment strategies in the adjustment strategy library; sending the target adjustment strategy to the user terminal to adjust the binding state of the expression base to be tested on the face of the target object.
[0010] In this implementation, if the evaluation data indicates that the binding quality does not meet the preset standard (such as a threshold), the corresponding target adjustment strategy can be called from the adjustment strategy library and fed back to the user terminal to assist the user (binding engineer or evaluator) or instruct the automatic binding mechanism to adjust the binding state of the expression base to be tested according to the target adjustment strategy, such as adjusting the binding point of the expression base to be tested on the digital human face. In this way, it is beneficial to improve the binding quality of the digital human facial expression base, thereby improving the production effect of the digital human.
[0011] In some possible implementations, determining at least one expression base group includes: obtaining configuration information of the expression base to be tested, the configuration information being used to configure the expression base group and the emotion type associated with the expression base group; combining the expression bases to be tested according to the configuration information to obtain at least one expression base group, each expression base group being associated with an emotion type.
[0012] In this implementation, all expression bases to be tested can be configured into one or more combinations according to the configuration information input by the user, and each combination is associated with an emotion type. In this way, binding quality evaluation is performed based on the grouping selected by the user, which facilitates the completion of correct expression base grouping to meet user needs in more complex expression base binding scenarios (such as many expression bases to be tested, richer facial expressions formed by the expression bases, or a large number of micro-expressions, etc.).
[0013] In some possible implementations, each expression base group is associated with an emotion type; the evaluation image includes image expression images corresponding to different emotion types, each image expression image includes the facial expression of a character; the evaluation image or the evaluation model is called to evaluate each expression base group to obtain evaluation data, including: according to the emotion type associated with the target expression base group, calling a target image expression image with the same emotion type, the target expression base group being one of at least one expression base group; calculating the similarity between the facial expression formed by the target expression base group and the facial expression in the target image expression image to obtain a similarity value; comparing the similarity value with a similarity threshold to obtain evaluation data for the target expression base group.
[0014] In this implementation, variable control can be used, that is, each expression base group can be used as a target expression base group, and similarity calculations can be performed one by one with the corresponding evaluation images of the same emotion type, so that the similarity between each target expression base group and the corresponding facial expression evaluation standard can be more accurately determined to obtain the evaluation result.
[0015] In some possible implementations, each expression base group is associated with an emotion type; the evaluation model includes an emotion model, which is an artificial intelligence model used to classify emotion types; an evaluation image or an evaluation model is called to evaluate each expression base group to obtain evaluation data, including: inputting a target expression base group into the emotion model, the target expression base group being one of at least one expression base group; using the emotion model to classify and predict the emotion type of facial expressions formed by the target expression base group, and determining the predicted emotion type of the target expression base group; comparing the emotion type associated with the target expression base group with the predicted emotion type for consistency, and obtaining evaluation data of the target expression base group.
[0016] In this implementation, variable control can be used, that is, each expression base group is used as a target expression base group and input into the emotion model one by one for emotion classification, so that the consistency between each target expression base group and the corresponding facial expression evaluation standard can be obtained more accurately to obtain the evaluation result.
[0017] In a second aspect, an embodiment of the present application provides an evaluation device, which includes: an acquisition module, used to acquire multiple expression bases to be tested, and the multiple expression bases to be tested are respectively bound to different points on the face of the target object; a processing module, used to determine at least one expression base group, each expression base group includes at least one of the multiple expression bases to be tested, and all the expression bases to be tested in each expression base group are used to form a facial expression; the processing module is also used to call an evaluation image or an evaluation model, evaluate each expression base group, and obtain evaluation data, the evaluation image and the evaluation model are used to represent the image or model of the facial expression evaluation standard, and the evaluation data is used to characterize the gap between the facial expression formed by the expression base group and the facial expression evaluation standard.
[0018] In some possible implementations, the expression base to be tested is uploaded by a user terminal, and the processing module is also used to: match the target adjustment strategy from an adjustment strategy library based on the evaluation data, the adjustment strategy library includes one or more adjustment strategies, the adjustment strategy is used to adjust the binding state of the expression base, and the target adjustment strategy is one of the adjustment strategies in the adjustment strategy library; send the target adjustment strategy to the user terminal to adjust the binding state of the expression base to be tested on the face of the target object.
[0019] In some possible implementations, the acquisition module is also used to obtain configuration information of the expression base to be tested, and the configuration information is used to configure the expression base group and the emotion type associated with the expression base group; the processing module is also used to combine the expression base to be tested according to the configuration information to obtain at least one expression base group, and each expression base group is associated with an emotion type.
[0020] In some possible implementations, each expression base group is associated with an emotion type; the evaluation image includes image expression images corresponding to different emotion types, each image expression image includes the facial expression of a character; the processing module is specifically used to: call a target image expression image with the same emotion type according to the emotion type associated with the target expression base group, the target expression base group being one of at least one expression base group; calculate the similarity between the facial expression formed by the target expression base group and the facial expression in the target image expression image to obtain a similarity value; compare the similarity value with a similarity threshold to obtain evaluation data for the target expression base group.
[0021] In some possible implementations, each expression base group is associated with an emotion type; the evaluation model includes an emotion model, which is an artificial intelligence model used to classify emotion types; the processing module is specifically used to: input the target expression base group into the emotion model, the target expression base group is one of at least one expression base group; classify and predict the emotion type of the facial expression formed by the target expression base group through the emotion model, and determine the predicted emotion type of the target expression base group; compare the emotion type associated with the target expression base group with the predicted emotion type for consistency, and obtain evaluation data of the target expression base group.
[0022] In a third aspect, an embodiment of the present application provides a computing device, comprising: at least one memory for storing a program; and at least one processor for executing the program stored in the memory; wherein, when the program stored in the memory is executed, the processor is used to execute the method described in the first aspect or any possible implementation of the first aspect.
[0023] In a fourth aspect, an embodiment of the present application provides a computing device cluster, comprising at least one computing device, each computing device comprising a processor and a memory; the processor of at least one computing device is used to execute instructions stored in the memory of at least one computing device, so that the computing device cluster executes the method described in the first aspect or any possible implementation of the first aspect.
[0024] In a fifth aspect, an embodiment of the present application provides a computer program product comprising instructions, which, when executed by a computing device cluster, enables the computing device cluster to execute the method described in the first aspect or any possible implementation manner of the first aspect.
[0025] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium, including computer program instructions. When the computer program instructions are executed by a computing device cluster, the computing device cluster executes the method described in the first aspect or any possible implementation of the first aspect.
[0026] It can be understood that the beneficial effects of the second to sixth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS
[0027] Figure 1 It is a schematic diagram of an application scenario provided by an embodiment of the present application;
[0028] Figure 2 is a schematic diagram of an evaluation image provided in an embodiment of the present application;
[0029] Figure 3 is a schematic diagram of an evaluation model provided in an embodiment of the present application;
[0030] Figure 4 It is a flowchart of an evaluation method provided in an embodiment of the present application;
[0031] Figure 5 is a flow chart of an evaluation method in a specific embodiment of the present application;
[0032] Figure 6 is a schematic diagram of a mapping table provided in an embodiment of the present application;
[0033] Figure 7is a flow chart of an evaluation method in another specific embodiment of the present application;
[0034] Figure 8 is a structural schematic diagram of an evaluation device provided in an embodiment of the present application;
[0035] Fig. 9 is a schematic diagram of the structure of a computing device provided in an embodiment of the present application;
[0036] Fig.10 It is a schematic diagram of the structure of a device cluster provided in an embodiment of the present application;
[0037] Fig.11 It is a structural diagram of a device cluster provided in an embodiment of the present application. DETAILED DESCRIPTION
[0038] The term "and / or" in this article is a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The symbol " / " in this article indicates that the associated objects are in an or relationship, for example, A / B means A or B.
[0039] The terms "first" and "second" in the specification and claims herein are used to distinguish different objects rather than to describe a specific order of the objects. For example, a first response message and a second response message are used to distinguish different response messages rather than to describe a specific order of the response messages.
[0040] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.
[0041] To facilitate understanding of the technical solution of the present application, the terms involved in this document are explained below.
[0042] Digital human is a technology that uses computer technology to synthesize digital human beings, creating a digital human image that is close to real people. Digital human can be divided into two types: two-dimensional digital human and three-dimensional digital human.
[0043] Three-dimensional (3D) digital human, 3D digital human is an important form of digital human, which is mainly achieved through the combination of different technologies such as computer graphics and computer vision. The production process of 3D digital human includes several processes such as modeling, binding, driving, and rendering. Generally, two-dimensional (2D) digital human is saved in video format, while 3D digital human data is saved in animation format. The head of 3D digital human can be in the form of expression base or mesh vertex, etc. The data of 3D digital human is mostly modeled through such abstract data.
[0044] Rigging: Mapping key points to the digital human model through manual or automated algorithms. The number and position of key point bindings affect the movement effect of the digital human. Rigging is a key step in making the digital human move. A good rigging algorithm not only makes the digital human move more realistically and accurately, but also reduces the pressure of subsequent digital human processes. The rigging algorithm is very important for the final digital human drive. To put it simply, a puppet is a system that is manipulated like a digital human. The rigging step is to determine the position of the puppet string, and the subsequent drive is to control these positions to make the digital human move.
[0045] Blend shapes: is an animation production method, which is applied to the facial expressions of animated characters. It can deform the "base mesh" into the "target shape" through vertex interpolation operations. Blend shapes are a very important binding method for facial binding (controlling facial movements) in 3D digital humans. A good binding algorithm can correctly assign blend shapes to appropriate facial points based on a real person image. If the quality of blend shape binding is not good, the facial points of the modeled 3D digital human will deviate greatly from the real person image, which does not conform to the actual facial movement of the real person image. Therefore, it is necessary to evaluate the quality of the blend shape and assist the binder to make adjustments based on the evaluation results.
[0046] Digital human evaluation: Digital human evaluation is mainly used to evaluate the production quality of different modules in the digital human production process. Since the digital human process is long, evaluating different processes is conducive to analyzing the bottleneck of digital human production and improving the overall quality of digital human production.
[0047] For the digital human expression base binding process, the expectation is to make the 3D digital human model as similar as possible to the specified real person image, but the expression base in the expression base system is templated, while the faces of real people are different. Therefore, if the binder selects inaccurate facial points of the 3D digital human model during binding, or the selected expression base is not suitable, etc., it may cause the facial expressions presented by the bound 3D digital human model to be unvivid, unnatural, and unrealistic, and the animation effect is poor, that is, the binding quality is poor.
[0048] In the related art, the main way to evaluate the binding quality of expression bases is for the binder to evaluate the binding quality based on his / her experience after binding. This method relies too much on the subjective experience of the binder himself / herself and lacks objective standards to verify the binding quality. In addition, if a group of expression bases are not well bound, the binder still has to independently evaluate and adjust based on experience, and there is a lack of standardized processes to guide the binder to adjust the group of expression bases.
[0049] In order to achieve an objective evaluation of the binding quality of the expression base, so as to improve the binding effect of the digital human facial expression base, an evaluation method is provided in the embodiment of the present application. The method mainly uses a preset image or model to represent the evaluation standard, measures the binding quality of the expression base bound to the digital human model by the binding artist, and obtains an objective evaluation result, so as to facilitate the binding artist to effectively adjust the bound expression base according to the evaluation result, so as to improve the binding quality of the digital human expression base.
[0050] To facilitate understanding of the technical solution of the present application, at least one application scenario of an embodiment of the present application is introduced below.
[0051] For example, Figure 1 The diagram shows an application scenario of an embodiment of the present application. Figure 1 As shown, the binder can bind multiple expression bases (also referred to as "expression bases to be tested") 102 to the face of the 3D digital human model 101 on the user terminal 10. In this way, these expression bases 102 can be used to make the face of the 3D digital human model present corresponding facial expressions through the subsequent driving process. However, before driving, in order to evaluate the binding quality of the expression bases on the face of the 3D digital human model, the binder can upload the 3D digital human model 101 bound with the expression bases 102 to the server 20, and the server 20 will evaluate the binding quality of the expression bases of the 3D digital human model 101.
[0052] Specifically, in this example, the server 20 can be based on preset corresponding facial expression evaluation standards, which can be evaluation images (including static pictures, dynamic pictures, or videos, etc.) that represent facial expressions of characters (including anthropomorphic character images, etc.), or artificial intelligence models that can be used for emotion type classification and recognition (also referred to as "emotion models" herein). In this way, the binding quality of the expression base can be evaluated by calculating the similarity between the facial expression of the expression base and the evaluation image, or the binding quality of the expression base can be evaluated by calculating the accuracy of the facial expression of the expression base being recognized by the emotion model.
[0053] In this example, the server 20 uses the evaluation image or the evaluation model to perform calculations on the acquired 3D digital human model 101 to determine the difference between the expression base 102 bound to the 3D digital human model 101 and the facial expression evaluation standard. For example, the similarity between the expression base 102 bound to the face of the 3D digital human model 101 and the aforementioned evaluation image is calculated. The larger the similarity value, the more similar the facial expression presented by the expression base 102 is to the facial expression of the character recorded in the evaluation image, that is, the higher the binding quality of the expression base 102. On the contrary, the larger the similarity difference value, the worse the binding quality. Or, for example, through the emotion model, the emotion type of the expression presented by the expression base 102 bound to the face of the 3D digital human model 101 is identified. If the emotion type can be correctly identified, the binding quality is high. On the contrary, the binding quality is poor.
[0054] In this way, the server 20 outputs an evaluation score for the binding quality of the expression base 102 on the 3D digital human model 101 based on the gap between the expression base bound to the 3D digital human model 101 and the facial expression evaluation standard, and outputs a corresponding adjustment strategy when the evaluation score indicates that the binding quality is poor, thereby assisting the binding engineer to adjust the bound expression base, which not only realizes objective and standardized expression base binding quality evaluation, but also helps to improve the expression base binding effect.
[0055] It is understood that the user terminal pole 10 can be a smart phone, a personal computer, etc., but is not limited thereto. The server 20 can be a hardware server, a cloud server, or a virtualized device (such as a virtual machine, a container, etc.).
[0056] It can be understood that the above-mentioned evaluation method can be used in any scenario of evaluating the quality of facial expression-based binding of a virtual image model, and will not be elaborated on herein.
[0057] The following is an introduction to an implementation form of a facial expression evaluation standard provided in an embodiment of the present application. The facial expression evaluation standard in this article can be an evaluation image or an evaluation model used to characterize the facial expression evaluation standard to evaluate the quality of the expression base binding to be tested of the target object's facial binding.
[0058] In some possible implementations, the evaluation image may be an image. Figure 2 As shown, the image image X10 may include at least one image expression set (A, B, ...), each image expression set (A, B, ...) includes multiple image expression images X11, and each image expression image X11 includes the facial expression of a character (including an anthropomorphic image, the same below). Each image expression set (A, B, ...) is associated with an emotion type. Exemplarily, the emotion type may include neutral, pleasant, surprise, sadness, anger, disgust, fear, ..., etc., but is not limited to this.
[0059] Exemplarily, each image expression image X11 in each image expression set (A, B, ...) can respectively represent facial expressions of the same emotional type of different characters. The image expression image X11 can be in the form of a static picture, a dynamic picture or a video (ie, a continuous multi-frame image).
[0060] In some specific examples, the corresponding image expression images X11 can be obtained by shooting facial expressions of different emotional types of multiple real-life images. For example, 100 images of real-life expressions such as laughing, crying, angry, surprised, and disgusted are shot, and these image expression images are classified according to the emotional type. The image expression images of the same emotional type are divided into the same set to form an image expression set (A, B, ...). The image expression set (A, B, ...) is used as an evaluation standard data set to calculate the similarity between the expression base bound to the face of the 3D digital human model and the expression of the real-life image, and the binding quality of the facial expression base of the 3D digital human model can be evaluated. In this way, by using a large number of different real-life image expressions as references for evaluation, it is avoided to be affected by a single image, which is conducive to more accurately reflecting the expression characteristics presented by the facial expression base of the 3D digital human model. For example, the binder expects to present the facial expression of laughing through certain expression bases. After binding, the expressions presented by these expression bases are similar to the expressions of laughing of most real-life images, which means that the binding quality of these facial expression bases used to present the laughing expression is high, thereby improving the accuracy of the expression base binding quality evaluation.
[0061] In some possible implementations, the evaluation model may be a sentiment model, which may be an artificial intelligence model capable of classifying sentiment types.
[0062] For example, the sentiment model can be a classifier obtained by training a neural network. For example, but not limited to, the sentiment model can be a classifier based on a multi-layer perceptron (MLP), a classifier based on a support vector machine (SVM), a classifier based on a k-nearest neighbor (k-NN), etc. In this example, please refer to Figure 3 As shown, a large amount of emotion data X20 can be used for model training to obtain an emotion model X30, and the emotion data X20 can be facial expression images (can be pictures or videos) X21 of different emotion types of real people. In some examples, mature model training technology can be used to collect a large amount of emotion data X20 as samples for training to obtain the emotion model X30.
[0063] In this way, using the emotion model X30, the expression base bound to the face of the 3D digital human model can be used as input (the expression base can be converted into a corresponding facial expression image inside the emotion model X30, or the expression base can be converted into a facial expression image first and then input into the emotion model), and the facial expressions formed by the expression base are classified and predicted in the emotion model X30 to identify the emotion type to which the facial expressions of these expression bases belong (the emotion type can be reflected in the form of a probability value or a numerical value). If the emotion type to which the facial expression base (facial expression) of the 3D digital human model belongs is correctly identified, or the accuracy reaches a preset threshold, it can be indicated that the binding quality of the expression base bound to the face of the 3D digital human model is high, otherwise, the binding quality is poor.
[0064] Next, based on the above description, an evaluation method provided by an embodiment of the present application is introduced. It can be understood that the method is proposed based on the above description, and part or all of the contents of the method can be referred to the above description.
[0065] See also Figure 4 , Figure 4 is a flow chart of an evaluation method provided in an embodiment of the present application. It can be understood that the method can be executed by any device, equipment, platform, or device cluster with computing and processing capabilities. Figure 4 As shown, the evaluation method may include:
[0066] S410, the computing device obtains a plurality of expression bases to be tested, and the plurality of expression bases to be tested are respectively bound to different points on the face of the target object.
[0067] In this embodiment, after the binding engineer has bound the facial expression base to a 3D digital human model of a certain image on the user terminal, the 3D digital human model can be uploaded to the computing device as the target object to evaluate the binding quality of the expression base (i.e., the expression base to be tested). Exemplarily, the computing device can be a hardware server, a cloud server, or other virtualized devices, such as a virtual machine, a container, etc. As a specific example, the computing device can be Figure 1 The server 20 shown in FIG. 1 may be Figure 1 The user terminal 10 shown in FIG. 1 is also taken as an example below.
[0068] For example, the server 20 may obtain a 3D digital human model and monitor one or more expression bases to be tested bound to the face of the 3D digital human model. For example, the face of a 3D digital human model may be bound with an expression base of the left eye looking left, an expression base of the right eye blinking, an expression base of the chin down when opening the mouth, and so on.
[0069] Exemplarily, the 3D digital human model may be a model based on a human image (including anthropomorphic character images, real-life images, etc.), and the face of the 3D digital human model may have a number of points, each of which is controllable. The expression base to be tested of the 3D digital human model obtained in this step describes the binding relationship with the facial points, for example, the expression base to be tested for right eye blinking describes the binding relationship of n points related to the right eye on the face of the 3D digital human model.
[0070] It can be understood that the facial expressions of the 3D digital human model can be presented by changes in the eyebrows, eyes, nose, mouth, chin, cheeks and other parts. Each expression base to be tested bound to the face of the 3D digital human model corresponds to a part expression, such as blinking the left eye, pursing the mouth to the right, etc. The superposition of multiple expression bases to be tested can generate the facial expressions of the 3D digital human model. For example, the superposition of the expression base of blinking the left eye and the expression base of sticking out the tongue can show a facial expression of a happy (naughty) emotion. A single expression base to be tested can generate micro-expressions on the face of the 3D digital human model, such as the micro-expression of raising the left eyebrow on the face.
[0071] As a specific example, the expression base to be tested for the facial binding of the 3D digital human model can come from an expression base system, such as the 52 expression bases provided by ARKit (an application) (covering expressions of eyes, nose, mouth, chin, cheeks, etc.), but not limited to this.
[0072] In some possible examples, the expression bases to be tested bound to the face of the 3D digital human model can also be uploaded in the form of expression images, in which corresponding facial expressions are formed by one or more expression bases to be tested, and the points where each expression base is associated and bound on the face of the 3D digital human model are described. For example, if the expression bases of left eye blinking and tongue sticking out are bound to the face of the 3D digital human model, an expression image showing left eye blinking and tongue sticking out can be obtained, and the expression image also describes the points where these expression bases to be tested are associated and bound on the face of the 3D digital human model.
[0073] S420, determining at least one expression basis group.
[0074] In this embodiment, the server 20 can automatically or passively group all the expression bases to be tested bound to the face of the 3D digital human model according to the expression bases to be tested bound to the face of the 3D digital human model, and obtain one or more expression base groups, wherein an expression base group includes at least one expression base to be tested, and all the expression bases to be tested in each expression base group are used to form a facial expression.
[0075] Exemplarily, the automatic generation of the expression base group may be implemented based on a preset grouping mechanism, and the passive generation of the expression base group may be implemented through external input instructions.
[0076] S430, calling the evaluation image or the evaluation model, evaluating each expression basis group, and obtaining evaluation data.
[0077] In this embodiment, the server 20 calls an image or model for representing the facial expression evaluation standard, such as Figure 2 The image shown in image X10 can also be Figure 3 The emotional model X30 shown in the figure is also used as an example in the following text.
[0078] In this embodiment, by performing computational processing on each expression basis set and the evaluation image or the evaluation model, the gap between the facial expression formed by each expression basis set and the facial expression evaluation standard can be determined, that is, the evaluation data can be obtained.
[0079] Exemplarily, if the image image X10 is called, when calculating and processing each expression base group, each expression base group can be compared with the corresponding image image X10 for facial expression similarity. For example, an expression base group used to form a laughing expression is similar to an image expression image X11 with a laughing expression recorded in the image image X10; or an expression base group used to form a crying expression is similar to an image expression image X11 with a crying expression recorded in the image image X10. By comparing the similarity with the evaluation image, the gap between the expression base group and the evaluation image can be determined, wherein the larger the similarity value (i.e., evaluation data) between the two, the smaller the gap, the more similar the two are, and the higher the binding quality of the expression base group, and vice versa, the lower the binding quality.
[0080] Exemplarily, if the emotion model X30 is called, each expression base group can be input into the emotion model X30 as a set of input data, and the emotion type classification prediction is performed on the facial expression formed by each expression base group in the emotion model X30. For example, if an expression base group used to form a crying expression is classified and predicted by the emotion model X30 and belongs to the emotion type of "sadness", then the classification is accurate. If an expression base group used to form a laughing expression is classified and predicted by the emotion model X30, but belongs to the emotion type of "fear", then the classification is inaccurate, and the reason for the inaccurate classification is that the binding quality of the expression base group is poor (such as the binding point is not suitable). In this way, if the emotion type classification of each expression base group is accurate, or the emotion classification of most expression base groups is accurate, then the obtained accuracy rate (i.e., evaluation data) is high, indicating that the gap between each expression base group and the facial expression evaluation standard is small, and the binding quality is high. Otherwise, the binding quality is low.
[0081] In this way, the method of the embodiment of the present application measures the binding quality of the facial expression base of the 3D digital human model by setting an image or model that can represent the facial expression evaluation standard, thereby solving the problem of inconsistent binding quality evaluation caused by the uneven subjective experience of the binder, and achieving objective and standardized expression base binding quality evaluation, which is conducive to improving the production quality and efficiency of the digital human model.
[0082] An evaluation method provided in some specific embodiments of the present application is described in detail below with reference to the accompanying drawings.
[0083] For example, Figure 5 FIG. 1 is a flow chart of an evaluation method provided by a specific embodiment of the present application. Figure 5 As shown, the method may include:
[0084] S510: The computing device obtains an expression base to be tested bound to the face of the target object.
[0085] In this embodiment, the implementation principle of step S510 can refer to the relevant description of the above step S410, which will not be repeated here.
[0086] S520: Generate at least one expression base group according to the expression base to be tested.
[0087] In this embodiment, the server 20 may generate at least one expression group through an external input instruction. Specifically, step S520 may include:
[0088] S521, the computing device obtains configuration information of the expression base to be tested.
[0089] In this example, reference Figure 5 As shown, the server 20 can detect the expression base 102 to be tested bound to the face of the 3D digital human model 101 through step S5101, and then can interact with the user terminal 10 of the user (which can be a binder or a person responsible for evaluation, which is not limited in this example), and execute this step to obtain the configuration information X40 input by the user, and the configuration information X40 is used to configure the expression base group and the emotion type associated with the expression base group. Among them, configuring the expression base group is to divide one or more expression bases 102 to be tested that can form a facial expression into a combination.
[0090] In this example, different expression base groups 1, 2... are configured according to the user's configuration information X40, and a mapping relationship between each expression base group 1, 2... and a perceptible emotion type is established. For example, ARKit provides 52 expression bases, and daily emotion types can be roughly divided into 7 categories, including neutral (Neutral), happiness (Happiness), surprise (Surprise), sadness (Sadness), anger (Anger), disgust (Disgust), fear (Fear) and other emotion types. A certain emotion type may be related to some of the 52 expression bases. Therefore, by establishing a mapping between expression bases and different facial expressions, the correlation between emotion types and expression base combinations can be established. Among them, this configuration operation can be obtained through a public mapping table, or it can be customized for a specific expression base input.
[0091] For example Figure 6 As shown in , the first, third, and fifth expression bases 102a in the expression base system are mapped to the laughing expression F1, and the fourth and sixth expression bases 102a are mapped to the crying expression F2. Then, all expression base groups including the first, third, or fifth expression bases 102a can be correlated with the emotion type of "happy", and all expression base groups including the fourth or sixth expression base 102a can be correlated with the emotion type of "sad". By analogy, the mapping relationship between each expression base 102a and different facial expressions F0 in the expression base system, as well as the correlation between the emotion type and the expression base combination can be established, and these mapping relationships and correlations are described through the mapping table 201, so that the user can input configuration information based on the mapping table 201.
[0092] In this way, according to the configuration information X40 sent by the user, the server 20 combines and configures all the expression bases 102 to be tested that are bound to the face of the 3D digital human model 101. In the expression base binding scenarios of more complex facial expressions or expression base binding scenarios of micro-expressions, the correct division of expression base groups can also be achieved through user-independent configuration.
[0093] In this example, some other possible implementations are that after the server 20 detects the expression base 102 to be tested, it can call the internal preset configuration strategy to divide all the detected expression bases 102 to be tested into one or more expression base groups 1, 2, ..., and associate each expression base group 1, 2, ... with a corresponding emotion type. This method is more suitable for production scenarios with more patterned facial expressions. For example, the expression base is bound to the set standard digital human model through an algorithm or manually to form a patterned facial expression. In this scenario, the configuration information can be automatically output through the configuration strategy to realize the division of expression base groups.
[0094] Exemplarily, the configuration information X40 may include expression base identification information for generating the same expression base group, identification information of emotion types associated with each expression base group, etc., but is not limited thereto.
[0095] S522, generating one or more expression base groups according to the configuration information.
[0096] In this example, continue to refer to Figure 5 The server 20 combines all the expression bases 102 to be tested according to the configuration information X40 to generate one or more expression base groups 1, 2, ..., each expression base group 1, 2, ... is associated with an emotion type.
[0097] S530, calling the evaluation image or the evaluation model, performing calculation processing on each expression basis group, and obtaining evaluation data.
[0098] In this embodiment, the server 20 performs calculation processing on each expression basis group according to the preset facial expression evaluation standard and control variables, that is, a separate calculation processing flow is performed with each expression basis group 1, 2, ... as the target expression basis group, and the consistency of each expression basis group with the facial expression evaluation standard represented by the evaluation image or the evaluation model is calculated to obtain evaluation data.
[0099] In one possible implementation, Figure 5 As shown, if the evaluation image is an image set including image facial expressions, step S530 may specifically include:
[0100] S531a, according to the emotion type associated with the target expression base group, calling the target image expression image with the same emotion type.
[0101] In this step, when the target expression base group is evaluated, according to the emotion type associated with the target expression base group, an image expression set with the same emotion type as the target expression base group is searched from the image image X10, and all image expression images X11 in the image expression set are used as the target image expression image X11a for calling.
[0102] S532a, calculating the similarity between the facial expression formed by the target expression basis set and the facial expression in the target image expression image, and obtaining a similarity value.
[0103] In this step, the target expression base set can be converted into an image format, the image includes the facial expressions formed by the target expression base set, and then the image is similarity calculated with each target image expression image X11a one by one to obtain multiple similarity values X50. For example, the target expression base set is converted into an image of a corresponding crying expression, and similarity is calculated with 100 target image expression images X11a showing different real crying expressions, and 100 similarity values X50 can be obtained.
[0104] S533a, comparing the similarity value with the similarity threshold to obtain evaluation data for the target expression basis set.
[0105] In this step, all similarity values X50 calculated for the target expression base group can be averaged to obtain a similarity average value, which is between [0,1]. Then, the similarity average value is compared with the preset similarity threshold to obtain the difference between the two, that is, the evaluation data X60 of the target expression base group. If the similarity average value represented by the evaluation data X60 is lower than the similarity threshold, it indicates that the target expression base group is quite different from the real person's image expression image X11, that is, the facial expression formed by the target expression base group is quite different from the facial expression evaluation standard. On the contrary, if the similarity average value represented by the evaluation data X50 is higher than the similarity threshold, it indicates that the facial expression formed by the target expression base group is slightly different from the facial expression evaluation standard, the similarity is high, and the binding quality of the expression base group meets expectations. If the similarity average value is 1, it means that the bound expression base to be tested is perfectly consistent with the facial expression evaluation standard. If the similarity average value is 0, it means that the bound expression base to be tested is completely inconsistent with the facial expression evaluation standard, and the quality of the binding of the expression base to be tested is very poor.
[0106] In this example, the average value of the evaluation data of each expression base group can be used as the overall evaluation data of all the expression bases 102 to be tested bound to the face of the 3D digital human model 101 .
[0107] In another possible implementation, Figure 7 As shown, if the evaluation model is a sentiment model, step S530 may specifically include:
[0108] S531 b, input the target expression basis set into the emotion model.
[0109] In this step, the target expression basis set can be input into the preset emotion model X30 to perform classification prediction of the emotion type X30. The target expression basis set can be input into the emotion model X30 and then converted into a binary image including the facial expression formed by the target expression basis set, and then the image is classified and predicted by the emotion model X30. Alternatively, in some examples, the target expression basis set can also be first converted into a corresponding image format and then input into the emotion model X30 for classification prediction.
[0110] S532b, classify and predict the emotional type of the facial expressions formed by the target expression base group through the emotional model, and determine the predicted emotional type of the target expression base group.
[0111] In this step, the facial expression formed by the target expression base group is classified and predicted by the emotion model X30, and the emotion type (i.e., predicted emotion type) X70 to which the target expression base group belongs can be output. It can be understood that the predicted emotion type X70 can be in the form of a probability value or a numerical value.
[0112] S533b, comparing the consistency of the emotion type associated with the target expression base set with the predicted emotion type to obtain evaluation data of the target expression base set.
[0113] In this step, after obtaining the predicted emotion type X70 of the target expression basis set, the accuracy of the current classification prediction is verified, that is, the predicted emotion type X70 is compared with the emotion type associated with the target expression basis set for consistency, and the comparison result (i.e., the evaluation data of the target expression basis set) X80 is obtained. If the comparison result X80 indicates that the predicted emotion type is consistent with the emotion type associated with the target expression basis set, the classification is considered accurate; if not, the classification is considered inaccurate. In this step, when the classification is inaccurate, it can be considered that it is caused by the low binding quality of the target expression basis set.
[0114] Exemplarily, in this step, each expression basis group 1, 2, ... obtains the corresponding comparison result X80 according to the above S531 b to S533b, and then the classification accuracy of all expression basis groups 1, 2, ... can be calculated based on all comparison results X80, where the classification accuracy = accurately classified expression basis groups / all expression basis groups, and the value range of the classification accuracy is [0,1].
[0115] Next, the classification accuracy can be compared with the preset accuracy threshold, and the difference obtained by comparison is the overall evaluation data of all the expression bases 102 to be tested that are bound to the face of the 3D digital human model 101. Among them, after comparison, if the classification accuracy is higher than the accuracy threshold, it means that most of the expression bases 102 to be tested are similar to the facial expression evaluation standard, and these expression bases to be tested meet expectations. If the classification accuracy is lower than the accuracy threshold, it means that the binding quality of these expression bases 102 to be tested is poor. If the classification accuracy is 1, it means that the bound expression base to be tested is perfectly consistent with the facial expression evaluation standard. If the classification accuracy is 0, it means that the bound expression base is completely inconsistent with the facial expression evaluation standard, and the quality of the binding of the expression base to be tested is very poor.
[0116] In this way, the embodiment of the present application can achieve an objective and standardized evaluation of the facial expression-based binding quality of a 3D digital human model through the above steps S510 to S530.
[0117] In this embodiment, after obtaining the evaluation data through the above S530, if the evaluation data indicates that the quality of expression base binding is poor and does not meet expectations, the method may further include:
[0118] S540: When the evaluation data does not meet the preset binding quality condition, match the target adjustment strategy from the adjustment strategy library.
[0119] In this embodiment, reference Figure 5 or Figure 6 As shown, the server 20 may pre-create an adjustment strategy library X90, which may include one or more adjustment strategies X91, and the adjustment strategy X91 is used to adjust the binding state of the expression base to be tested. Exemplarily, the binding state may include the adjustment direction of the expression base to be tested (such as increasing or decreasing the expression amplitude), facial points, replaceable expression bases (such as replacing expression base b with expression base a), etc., but is not limited thereto.
[0120] Exemplarily, the preset binding quality condition may include at least one of the following:
[0121] The similarity value of the target expression base group is higher than the similarity threshold, the emotion type of the target expression base group is consistent with its predicted emotion type, and the classification accuracy of all expression base groups is higher than the accuracy threshold.
[0122] When the evaluation data (X60, X80) does not meet the binding quality condition, the server 20 can match the corresponding target adjustment strategy X92 from the adjustment strategy library X90 according to the expression base group corresponding to the evaluation data (X60, X80), and the target adjustment strategy X92 is one of the adjustment strategies X91 in the adjustment strategy library X90. For example, the similarity value of the target expression base group aa is 0.4, the similarity threshold is 0.8, and the evaluation data of the target expression base group aa does not meet the binding quality condition, then the server 20 determines that the expression amplitude formed by the target expression base group aa is small, and therefore matches the target adjustment strategy X91, which indicates to increase the expression amplitude of the target expression base group aa, or adjust the facial points bound to the target expression base group aa.
[0123] S550, sending the target adjustment strategy to the user terminal to adjust the expression base to be tested bound to the face of the target object.
[0124] In this step, the server 20 sends the matching target adjustment strategy X91 to the user terminal 10, so that the binding engineer (or other personnel) on the user terminal 10 side or the automatic binding algorithm can adjust the corresponding expression base 102 to be tested on the face of the 3D digital human model 101 according to the target adjustment strategy X91, and complete the correction of the expression base binding effect, which is conducive to achieving a more realistic, vivid and natural facial expression effect on the face of the 3D digital human model.
[0125] Next, an evaluation device provided in an embodiment of the present application is introduced.
[0126] Exemplarily, the present application also provides an evaluation device 800, such as Figure 8 As shown, the device 800 includes:
[0127] An acquisition module 801 is used to acquire an expression base to be tested bound to the face of the target object, and there is at least one expression base to be tested;
[0128] Processing module 802, used for generating at least one expression base group according to the expression base to be tested, each expression base group includes at least one expression base to be tested, and all the expression bases to be tested in each expression base group are used to form a facial expression;
[0129] The processing module 802 is also used to call the evaluation image or evaluation model, perform calculation processing on each expression basis group, and obtain evaluation data. The evaluation image and the evaluation model are images or models used to characterize the facial expression evaluation standard, and the evaluation data is used to characterize the gap between the facial expression formed by the expression basis group and the facial expression evaluation standard.
[0130] In some possible implementations, the processing module 802 may also be used to:
[0131] According to the evaluation data, a target adjustment strategy is matched from an adjustment strategy library, wherein the adjustment strategy library includes one or more adjustment strategies, and the adjustment strategy is used to adjust the binding state of the expression base, and the target adjustment strategy is one of the adjustment strategies in the adjustment strategy library;
[0132] The target adjustment strategy is sent to the user terminal to adjust the expression base to be tested bound to the face of the target object.
[0133] In some possible implementations, the acquisition module 801 is further used to acquire configuration information of the expression base to be tested, where the configuration information is used to configure the expression base group and the emotion type associated with the expression base group;
[0134] The processing module 802 is further used to combine the expression bases to be tested according to the configuration information to obtain at least one expression base group, each expression base group being associated with an emotion type.
[0135] In some possible implementations, the evaluation image includes image expression images corresponding to different emotion types, each image expression image includes a facial expression of a character. Then the processing module 802 can be specifically used to:
[0136] According to the emotion type associated with the target expression base group, calling the target image expression image with the same emotion type, the target expression base group being one of at least one expression base group;
[0137] Calculate the similarity between the facial expression formed by the target expression basis set and the facial expression in the target image expression image to obtain a similarity value;
[0138] The similarity value is compared with the similarity threshold to obtain evaluation data for the target expression basis set.
[0139] In some possible implementations, the evaluation model includes a sentiment model, which is an artificial intelligence model for sentiment type classification; then the processing module 802 may be specifically used to:
[0140] Inputting a target expression basis set into the emotion model, the target expression basis set being one of at least one expression basis set;
[0141] The emotional type of the facial expressions formed by the target expression base group is classified and predicted by the emotional model to determine the predicted emotional type of the target expression base group;
[0142] The emotion type associated with the target expression base set is compared with the predicted emotion type for consistency to obtain the evaluation data of the target expression base set.
[0143] Wherein, both the acquisition module 801 and the processing module 802 can be implemented by software or by hardware. Exemplarily, the implementation of the acquisition module 801 is described below by taking the acquisition module 801 as an example. Similarly, the implementation of the processing module 802 can refer to the implementation of the acquisition module 801.
[0144] As an example of a software functional unit, the acquisition module 801 may include code running on a computing instance. Among them, the computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Further, the above-mentioned computing instance may be one or more. For example, the acquisition module 801 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code may be distributed in the same region (region) or in different regions. Furthermore, the multiple hosts / virtual machines / containers used to run the code may be distributed in the same availability zone (AZ) or in different AZs, each AZ including a data center or multiple data centers with close geographical locations. Among them, usually a region may include multiple AZs.
[0145] Similarly, multiple hosts / virtual machines / containers used to run the code can be distributed in the same virtual private cloud (VPC) or in multiple VPCs. Usually, a VPC is set up in a region. For cross-region communication between two VPCs in the same region and between VPCs in different regions, a communication gateway needs to be set up in each VPC to achieve interconnection between VPCs through the communication gateway.
[0146] As an example of a hardware functional unit, the acquisition module 801 may include at least one computing device, such as a server, etc. Alternatively, the acquisition module 801 may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL) or any combination thereof.
[0147] The multiple computing devices included in the acquisition module 801 can be distributed in the same region or in different regions. The multiple computing devices included in the acquisition module 801 can be distributed in the same AZ or in different AZs. Similarly, the multiple computing devices included in the acquisition module 801 can be distributed in the same VPC or in multiple VPCs. The multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.
[0148] It should be noted that, in other embodiments, the acquisition module 801 can be used to execute any step in the evaluation method, and the processing module 802 can be used to execute any step in the evaluation method. The steps that the acquisition module 801 and the processing module 802 are responsible for implementing can be specified as needed. The full functions of the evaluation device can be realized by respectively implementing different steps in the evaluation method through the acquisition module 801 and the processing module 802.
[0149] The present application embodiment also provides a computing device 900. Fig. 9 As shown, the computing device 900 includes: a bus 902, a processor 904, a memory 906, and a communication interface 908. The processor 904, the memory 906, and the communication interface 908 communicate through the bus 902. The computing device 900 can be a server or a terminal device. It should be understood that the present application does not limit the number of processors and memories in the computing device 900.
[0150] The bus 902 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig. 9 The bus 904 is represented by only one line, but does not mean that there is only one bus or one type of bus. The bus 904 may include a path for transmitting information between various components of the computing device 900 (eg, the memory 906, the processor 904, and the communication interface 908).
[0151] The processor 904 may include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).
[0152] The memory 906 may include a volatile memory, such as a random access memory (RAM). The processor 904 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).
[0153] The memory 906 stores executable program codes, and the processor 904 executes the executable program codes to respectively implement the functions of the acquisition module 801 and the processing module 802, thereby implementing the evaluation method. That is, the memory 906 stores instructions for executing the evaluation method.
[0154] The communication interface 908 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 900 and other devices or communication networks.
[0155] The embodiment of the present application also provides a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smart phone.
[0156] like Fig.10 As shown, the computing device cluster includes at least one computing device 900. The memory 906 in one or more computing devices 900 in the computing device cluster may store the same instructions for executing the evaluation.
[0157] In some possible implementations, the memory 906 of one or more computing devices 900 in the computing device cluster may also store partial instructions for executing the evaluation method. In other words, the combination of one or more computing devices 900 may jointly execute instructions for executing the evaluation method.
[0158] It should be noted that the memory 906 in different computing devices 900 in the computing device cluster can store different instructions, which are respectively used to execute part of the functions of the evaluation device. That is, the instructions stored in the memory 906 in different computing devices 900 can implement the functions of one or more modules in the acquisition module 801 and the processing module 802.
[0159] In some possible implementations, one or more computing devices in the computing device cluster may be connected via a network, which may be a wide area network or a local area network. Fig.11 A possible implementation is shown. Fig.11 As shown, two computing devices 900A and 900B are connected via a network. Specifically, the network is connected via a communication interface in each computing device. In this type of possible implementation, the memory 906 in the computing device 900A stores instructions for executing the functions of the acquisition module. At the same time, the memory 906 in the computing device 900B stores instructions for executing the functions of the processing module 802.
[0160] Fig.11 The connection mode between the computing device clusters shown may be that considering that the evaluation method provided in the present application requires a large amount of data storage and data calculation, it is considered to hand over the functions implemented by the processing module 802 to the computing device 900B for execution.
[0161] It should be understood that Fig.11 The functions of the computing device 900A shown in FIG. 9A may also be completed by multiple computing devices 900. Similarly, the functions of the computing device 900B may also be completed by multiple computing devices 900.
[0162] In some possible implementations, the memory 906 of one or more computing devices 900 in the computing device cluster may also store partial instructions for executing the evaluation method. In other words, the combination of one or more computing devices 900 may jointly execute instructions for executing the evaluation method.
[0163] The embodiment of the present application also provides a computer program product including instructions. The computer program product may be software or a program product including instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on at least one computing device, the at least one computing device executes the evaluation method.
[0164] The embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that can be stored by the computing device or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk). The computer-readable storage medium includes instructions that instruct the computing device to execute the evaluation method, or instruct the computing device to execute the evaluation method.
[0165] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of the present invention.
Claims
1. An evaluation method, characterized in that: The method comprises: Acquire a plurality of expression bases to be tested, wherein the plurality of expression bases to be tested are respectively bound to different points on the face of the target object; Determine at least one expression base group, each of the expression base groups includes at least one of the multiple expression bases to be tested, and all the expression bases to be tested in each of the expression base groups are used to form a facial expression; An evaluation image or an evaluation model is called to evaluate each of the expression base groups to obtain evaluation data, wherein the evaluation image and the evaluation model are both used to characterize the facial expression evaluation standard, and the evaluation data is used to characterize the gap between the facial expression formed by the expression base group and the facial expression evaluation standard.
2. The method according to claim 1, characterized in that: The expression base to be tested is uploaded by a user terminal, and the method comprises: According to the evaluation data, a target adjustment strategy is matched from an adjustment strategy library, wherein the adjustment strategy library includes one or more adjustment strategies, wherein the adjustment strategy is used to adjust the binding state of the expression base, and the target adjustment strategy is one of the adjustment strategies in the adjustment strategy library; The target adjustment strategy is sent to the user terminal to adjust the binding state of the expression base to be tested on the face of the target object.
3. The method according to claim 1 or 2, characterized in that: The determining of at least one expression basis group comprises: Acquire configuration information of the expression base to be tested, wherein the configuration information is used to configure an expression base group and an emotion type associated with the expression base group; According to the configuration information, the expression bases to be tested are combined to obtain at least one expression base group, and each expression base group is associated with an emotion type.
4. The method according to any one of claims 1 to 3, characterized in that: Each of the expression base groups is associated with an emotion type; The evaluation images include image expression images corresponding to different emotion types, each of which includes a facial expression of a character; The calling of the evaluation image or the evaluation model to evaluate each of the expression basis groups to obtain evaluation data includes: According to the emotion type associated with the target expression base group, calling a target image expression image with the same emotion type, wherein the target expression base group is one of the at least one expression base group; Calculating the similarity between the facial expression formed by the target expression basis set and the facial expression in the target image expression image to obtain a similarity value; The similarity value is compared with a similarity threshold to obtain evaluation data for the target expression basis set.
5. The method according to any one of claims 1 to 3, characterized in that: Each of the expression base groups is associated with an emotion type; The evaluation model includes a sentiment model, which is an artificial intelligence model for sentiment type classification; The calling of the evaluation image or the evaluation model to evaluate each of the expression basis groups to obtain evaluation data includes: Inputting a target expression basis set into the emotion model, wherein the target expression basis set is one of the at least one expression basis set; Performing classification prediction of the emotion type of the facial expression formed by the target expression basis set by using the emotion model to determine the predicted emotion type of the target expression basis set; The emotion type associated with the target expression basis set is compared with the predicted emotion type for consistency to obtain evaluation data of the target expression basis set.
6. An evaluation device, characterized in that The device comprises: An acquisition module, used for acquiring a plurality of expression bases to be tested, wherein the plurality of expression bases to be tested are respectively bound to different points on the face of the target object; A processing module, used for determining at least one expression base group, each of which includes at least one of the plurality of expression bases to be tested, and all the expression bases to be tested in each of which are used to form a facial expression; The processing module is also used to call an evaluation image or an evaluation model to evaluate each of the expression base groups to obtain evaluation data. The evaluation image and the evaluation model are used to represent the image or model of the facial expression evaluation standard, and the evaluation data is used to represent the gap between the facial expression formed by the expression base group and the facial expression evaluation standard.
7. The device according to claim 6, characterized in that The expression base to be tested is uploaded by a user terminal, and the processing module is further used for: According to the evaluation data, a target adjustment strategy is matched from an adjustment strategy library, wherein the adjustment strategy library includes one or more adjustment strategies, wherein the adjustment strategy is used to adjust the binding state of the expression base, and the target adjustment strategy is one of the adjustment strategies in the adjustment strategy library; The target adjustment strategy is sent to the user terminal to adjust the binding state of the expression base to be tested on the face of the target object.
8. The device according to claim 6 or 7, characterized in that The acquisition module is further used to acquire configuration information of the expression base to be tested, wherein the configuration information is used to configure the expression base group and the emotion type associated with the expression base group; The processing module is further used to combine the expression bases to be tested according to the configuration information to obtain at least one expression base group, each of which is associated with an emotion type.
9. The device according to any one of claims 6-7, characterized in that: Each of the expression base groups is associated with an emotion type; The evaluation images include image expression images corresponding to different emotion types, each of which includes a facial expression of a character; The processing module is specifically used for: According to the emotion type associated with the target expression base group, calling a target image expression image with the same emotion type, wherein the target expression base group is one of the at least one expression base group; Calculating the similarity between the facial expression formed by the target expression basis set and the facial expression in the target image expression image to obtain a similarity value; The similarity value is compared with a similarity threshold to obtain evaluation data for the target expression basis set.
10. The device according to any one of claims 6-7, characterized in that: Each of the expression base groups is associated with an emotion type; The evaluation model includes a sentiment model, which is an artificial intelligence model for sentiment type classification; The processing module is specifically used for: Inputting a target expression basis set into the emotion model, wherein the target expression basis set is one of the at least one expression basis set; Performing classification prediction of the emotion type of the facial expression formed by the target expression basis set by using the emotion model to determine the predicted emotion type of the target expression basis set; The emotion type associated with the target expression basis set is compared with the predicted emotion type for consistency to obtain evaluation data of the target expression basis set.
11. A computing device cluster, characterized in that: comprising at least one computing device, each computing device comprising a processor and a memory; The processor of the at least one computing device is used to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the method according to any one of claims 1-5.
12. A computer program product comprising instructions, characterized in that When the instructions are executed by a computing device cluster, the computing device cluster executes the method according to any one of claims 1 to 5.
13. A computer-readable storage medium, characterized in that: The method comprises computer program instructions. When the computer program instructions are executed by a computing device cluster, the computing device cluster executes the method according to any one of claims 1 to 5.
