Digital human facial expression transition method, device, electronic device, and storage medium

Abstract

To provide a method for transferring facial expression of digital humans applied to scenes such as meta-universe and virtual digital humans, a device for transferring facial expression of digital humans, an electronic device, and a storage medium.SOLUTION: A method for transferring facial expression of digital humans includes a step of screening an identification of a target reference model matched with an object model from a preset reference model library. The reference model library includes a plurality of reference models. The method also includes a step of acquiring an expression library of the target reference model on the basis of the identification of the target reference model, and a step of acquiring a last frame of an expression of the object model by transferring the last frame of an expression in the expression library of the target reference model into the object model.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to computer technology and artificial intelligence technology fields, particularly to augmented reality, virtual reality, computer vision, deep learning, and other technical fields, and can be applied to scenes such as meta-universes and virtual digital humans. Specifically, the present disclosure relates to a method, device, electronic device, and storage medium for facial expression transition of a digital human. [Background technology]

[0002] Binding is a key part of the design of digital human image-driven models, enabling the addition of facial expressions to digital human models. In conventional technologies, binding is usually completed by professional designers. There are two types of binding: blendshape deformation and bone skin. Some scenes use a combination of the two to achieve optimization.

[0003] For different digital human images, the binding work also requires a certain amount of investment. However, it is important to note that the time period is not always short, usually more than one to two weeks. With the demand for high-quality facial expression-driven effects of ultra-realistic digital humans, this time cost will increase significantly. Summary of the Invention [Problem to be solved by the invention]

[0004] The present disclosure provides a method, device, electronic device, and storage medium for facial expression transition of a digital human. [Means for solving the problem]

[0005] According to one aspect of the present disclosure, there is provided a method for facial expression transition of a digital human, comprising: selecting an identifier of a target reference model that matches the object model from a predefined reference model library, the reference model library including a plurality of reference models; obtaining an expression library of the target reference model based on an identifier of the target reference model; and transferring a final frame of an expression in the expression library of the target reference model to the object model to obtain a final frame of an expression of the object model.

[0006] According to another aspect of the present disclosure, there is provided a facial expression transition device for a digital human, comprising: a selection module for selecting an identifier of a target reference model matching the object model from a predefined reference model library, the reference model library including a plurality of reference models; an acquisition module for acquiring a facial expression library of the target reference model based on an identifier of the target reference model; a final frame transition module for transitioning a final frame of an expression in the expression library of the target reference model to the object model to obtain a final frame of an expression of the object model.

[0007] According to another aspect of the present disclosure, there is provided an electronic device, comprising: at least one processor; a memory communicatively coupled to the at least one processor; The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enable the at least one processor to perform the method of the aspect and any possible implementation.

[0008] According to another aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium having stored thereon computer instructions that cause the computer to perform the method of the aspect and any possible implementation thereof.

[0009] According to yet another aspect of the present disclosure, there is provided a computer program product, comprising a computer program which, when executed by a processor, implements the method of the aspect and any possible implementation manner.

[0010] According to the technology disclosed herein, facial expression transitions of digital humans can be automatically realized. Compared with conventional technologies, the entire facial expression transition process can save time and effort, and the accuracy of the transitioned facial expressions can be effectively ensured. Furthermore, the efficiency of digital human facial expression transitions can be effectively improved, and the efficiency of generating digital human images can be improved.

[0011] It should be understood that the contents described herein are not intended to identify key or important features of the embodiments of the present disclosure, nor should they be used to limit the scope of the present disclosure. Other features of the present disclosure can be readily understood through the following specification. [Brief explanation of the drawings]

[0012] The drawings are for a better understanding of the present application and are not intended to limit the present application. [Figure 1] FIG. 1 is a schematic diagram according to a first embodiment of the present disclosure. [Figure 2] FIG. 10 is a schematic diagram according to a second embodiment of the present disclosure. [Figure 3] 1 is a schematic diagram of the structure of an object model eye provided by the present disclosure; FIG. [Figure 4] FIG. 10 is a schematic diagram according to a third embodiment of the present disclosure. [Figure 5] FIG. 10 is a schematic diagram according to a fourth embodiment of the present disclosure. [Figure 6]FIG. 1 is a block diagram of an electronic device for implementing a method according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, exemplary embodiments of the present application will be described with reference to the drawings. For ease of understanding, various details of the embodiments of the present application are included and should be considered as merely examples. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present application. Similarly, for the sake of clarity, the following description will omit descriptions of well-known functions and structures.

[0014] Obviously, the described embodiments are some of the embodiments of the present application, but not all of the embodiments, and all other embodiments that a person skilled in the art can obtain without creative effort according to the embodiments of the present application all belong to the scope of protection of the present application.

[0015] Note that terminal devices related to the embodiments of the present disclosure may include, but are not limited to, smart devices such as mobile phones, personal digital assistants (PDAs), wireless handheld devices, tablet computers, etc. Display devices may include, but are not limited to, devices with display capabilities such as personal computers and televisions.

[0016] Furthermore, the term "and / or" in this specification describes only the relation between related objects and indicates that three types of relations can exist, for example, A and / or B can indicate three cases: only A exists, A and B exist simultaneously, or only B exists. It should be understood that the symbol " / " generally indicates that the related objects before and after it are in an "or" relation.

[0017] In the prior art, binding facial expressions to digital humans is mainly completed manually by professional designers, making the entire process time-consuming and resulting in a significant decrease in the efficiency of facial expression binding. Based on this, the present disclosure provides a digital human facial expression transition method that can automatically transition the facial expressions of a reference model to a digital human object model, effectively improving the efficiency of binding facial expressions to digital human images.

[0018] 1 is a schematic diagram of a first embodiment of the present disclosure. As shown in FIG. 1, this embodiment provides a facial expression transition method for a digital human, which specifically includes the following steps: S101: selecting an identifier of a target reference model that matches an object model from a preset reference model library, the reference model library including a plurality of reference models; The execution entity of the digital human facial expression transition method of this embodiment may be a digital human facial expression transition device, which may be an electronic entity or a software integrated application, and the device can transition the facial expression of any reference model in the reference model library to any object model to realize automatic digital human facial expression transition.

[0019] The object model in this embodiment is a digital human model that uses facial expressions. Each pre-defined reference model in the reference model library is a pre-defined digital human model of various forms and styles, and may include, for example, a male model, a female model, a boy model, a middle-aged male model, an elderly male model, a girl model, a middle-aged female model, an elderly female model, etc. Specifically, various reference models may be pre-defined as needed, and the present invention is not limited thereto.

[0020] The identifier of the target reference model in this embodiment is any identifier that can identify the target reference model, and is not limited here.

[0021] S102, obtain an expression library of the target reference model according to the identifier of the target reference model; In this embodiment, facial expression libraries can be preset for various preset reference models, and each reference model's facial expression library can include multiple facial expressions. Specifically, various facial expressions can be stored in the form of point cloud data. For example, the natural state of the reference model is a state in which the digital human is not making any facial expressions, and the natural state can also be a standard state. After adding an expression to the reference model digital human, a positional change is always generated in the point cloud that executes the expression on the reference model, while no positional change occurs in the point cloud that does not execute the expression. Therefore, for the expression of the reference model, only point cloud data that changes when the expression is executed can be stored.

[0022] S103, the final frame of the facial expression in the facial expression library of the target reference model is transferred to the object model to obtain the final frame of the facial expression of the object model.

[0023] In practical application scenarios, the range of many facial expressions is small, and the facial expression library only contains the point cloud data of the final frame of the corresponding expression, that is, the facial expression library only records the final state of the expression. In this case, when performing facial expression transition, the final frame of the expression in the facial expression library of the target reference model can be transitioned to the object model to obtain the final frame of the expression of the object model.

[0024] The facial expression transition in this embodiment is to realize automatic, accurate, and efficient facial expression transition by transitioning the final frame of the facial expression of the target reference model that matches the object model in the reference model library to the object model.

[0025] By using the above technical solution, the digital human facial expression transition method of this embodiment can automatically transition facial expressions in the facial expression library of a target reference model that matches an object model to the object model. Compared with the prior art, this eliminates the need for professional designers to manually bind, saving time and effort. It not only effectively ensures the accuracy of the transitioned facial expressions, but also effectively improves the efficiency of digital human facial expression transition and the efficiency of generating digital human images.

[0026] FIG. 2 is a schematic diagram of a second embodiment of the present disclosure. This embodiment is based on the technical solution of the embodiment shown in FIG. 1 and further describes the technical solution of the present disclosure in more detail. As shown in FIG. 2, the facial expression transition method of a digital human of this embodiment may specifically include the following steps: S201, obtaining a first feature curve of a preset region from an object model; S202, obtaining a second feature curve of the preset region from each reference model; S203, calculating offset distances of preset locations between the object model and each reference model according to the first feature curve and each second feature curve; S204, selecting an identifier of a target reference model matching the object model from the reference model library according to the offset distance of the preset portion between the object model and each reference model; The above steps S201-S204 of this embodiment are a specific implementation of step S101 of the embodiment shown in FIG.

[0027] Since the distinction between different digital human images is primarily expressed in the face, the preset region in this embodiment can refer to one region of the face, such as the eyes or mouth. For example, the first feature curve may be at least one of six curves: the upper eyelid, the lower eyelid, the outer contour of the upper eyelid, the outer contour of the lower eyelid, the inner eye corner divergence, and the outer eye corner divergence. Alternatively, the first feature curve may be at least one of multiple curves, such as the inner upper lip line, the outer upper lip line, and the bite line. For digital human image models such as the object model and the reference model, the number and distribution of point clouds included in the same preset region are the same. That is, for the same preset region, corresponding points exist between two models.

[0028] In this embodiment, when steps S201-S204 are performed, only one preset region can be selected, or two or more preset regions can be selected, and the same preset region can obtain only one first feature curve from the object model and one corresponding second feature curve from each reference model, or can obtain two or more first feature curves from the object model and two or more corresponding second feature curves from each reference model.

[0029] When only one preset region is selected and one first characteristic curve and one second characteristic curve are selected, the distribution and number of points included in the first characteristic curve and the second characteristic curve are the same. In this case, when specifically implementing step S203, the following steps can be included: (a1) obtaining coordinates of each point on the first characteristic curve and each second characteristic curve; The coordinates of each point on the first characteristic curve and each second characteristic curve can be obtained from the model data of the object model and each reference model, respectively.

[0030] (b1) calculating point distances of the same point identifiers on the first feature curve and each of the second feature curves based on the coordinates of each point on the first feature curve and each of the second feature curves; The object model and the reference model may use the same point identifiers for points corresponding to the distribution. For example, a first feature curve of a preset region of the object model may include points identified as 1, 2, 3, 4, 5, and 6, and correspondingly, a second feature curve corresponding to the first feature curve of the same preset region of the reference model may include points identified as 1, 2, 3, 4, 5, and 6. Points with the same identifiers have the same distribution and corresponding positions. However, the coordinates of points with the same identifiers are not the same in two different models.

[0031] Specifically, the distance between points with the same point identifier is calculated based on the coordinates of two points with the same point identifier on the first characteristic curve and each of the second characteristic curves.

[0032] (c1) adding the point distances of each point identifier on the first feature curve and each second feature curve to obtain a sum of point distances; (d1) Obtaining offset distances of preset portions between the object model and each reference model based on the sum of the point distances and the number of points included in the first feature curve.

[0033] Specifically, the number of points included in the first characteristic curve is the same as the number of points included in each of the second characteristic curves.

[0034] For example, for a second feature curve of a given reference model, the point distances of each point identifier on the corresponding first feature curve and the corresponding second feature curve are added to obtain the sum of the point distances, and then the sum of the point distances is divided by the number of points included in the first feature curve to obtain the average distance of each point, which is the offset distance of the preset portion between the object model and the reference model. Using this method, the offset distance of the preset portion between the object model and the reference model can be accurately calculated.

[0035] In this case, when step S204 is specifically performed, the identifier of the reference model having the smallest offset distance of the preset portion from the plurality of reference models in the reference model library can be selected as the identifier of the target reference model to be matched with the object model, and this method can accurately and efficiently obtain the target reference model to be matched with the object model.

[0036] Optionally, in one embodiment of the present disclosure, for any reference model, point cloud registration of the first and second feature curves is required before performing step (a1) to accurately calculate the offset distance of the preset portion between the object model and the reference model. Without registration, if the distance between the first and second feature curves is large, even if the shapes and distributions of the first and second feature curves are completely identical, the calculated offset distance of the preset portion between the object model and the reference model will be large, resulting in a larger error. In this embodiment, the point cloud registration of the first and second feature curves also involves manipulating the second feature curve using at least one of the manipulation methods of displacement, rotation, and scaling to ensure that the first and second feature curves have the most common characteristics, such as having the same centroid and the most intersections. During the registration process, multiple manipulation attempts can be made to obtain the optimal registration position, i.e., the position at the sum of the point distances between the first and second feature curves, i.e., the registration position of the first and second feature curves.

[0037] In one embodiment of the present disclosure, if only one preset region is selected and two or more first feature curves and two or more second feature curves participate in the above-mentioned selection, for the first feature curve of any group and the corresponding second feature curve, the above-mentioned method can be used to calculate the sum of the point distances of each point identifier on the first feature curve and the second feature curve of the group. Then, the integrated value of the sum of the point distances of each point identifier on the first feature curve and the second feature curve of all groups can be obtained as the offset distance of the preset region between the object model and the corresponding reference model.

[0038] In one embodiment of the present disclosure, when two or more preset locations are selected to participate in the above-mentioned selection, for each preset location, the offset distance corresponding to one of the preset locations can be calculated according to the above-mentioned method. Then, further, the following steps are performed: (a2) calculating a total offset distance between the object model and each reference model based on the weight of each preset portion and the offset distance corresponding to each preset portion; The weight of each preset part in this embodiment can be set according to the actual situation. For example, the characteristic expressions of the eyes and mouth on the face are prominent, so the weight of the eyes and mouth can be set high.

[0039] (b2) Selecting an identifier of a target reference model that matches the object model from the reference model library based on the total offset distance between the object model and each reference model.

[0040] Specifically, the identifier of the reference model having the smallest overall offset distance from among the plurality of reference models in the reference model library can be selected as the identifier of the target reference model that matches the object model.

[0041] Using the above method, it is possible to select the target reference model that best matches the object model from the reference model library in a very efficient, accurate and rational manner.

[0042] Optionally, in one embodiment of the present disclosure, step S101 of the embodiment shown in FIG. 1 above can also be implemented using any of the following methods: A. Selecting a target reference model identifier that matches the object model from the reference model library based on the attribute information of the object model and the attribute information of each reference model in the reference model library; or In practical applications, when establishing each digital human model, corresponding attribute information can be further set to describe or limit the model, for example, the attribute information can be middle-aged woman, child, elderly man, etc. At this time, based on the attribute information of the object model, an identifier of a reference model with the same attribute information can be selected from the model reference library as the identifier of the target reference model.

[0043] B. Displaying attribute information of each reference model in the reference model library to the user, and receiving an identifier of a target reference model that matches the object model selected by the user.

[0044] In this method, the user manually selects the identifier of the target reference model that matches the object model.

[0045] In short, by using any of the above methods, it is possible to efficiently and accurately obtain the identifier of the target reference model that matches the object model.

[0046] S205, obtaining a facial expression library of the target reference model according to the identifier of the target reference model; S206, perform global registration on the target reference model and the object model; In this embodiment, the target reference model is adjusted to perform full-row registration between the object model and the target reference model, aligning their head positions, and using translation, rotation, and / or scaling to match the coordinate system orientation and size of the target reference model and the object model. This step roughly identifies the positions and sizes corresponding to the five sensory organs of the model, facilitating key feature matching in the next step.

[0047] S207, registering the key features corresponding to the facial expressions of the target reference model with the key features of the object model; In this embodiment, an expression involves only one key feature, and the key feature may include, for example, the left eye, the right eye, or the mouth. To achieve accurate facial expression transitions, the key feature registration in this embodiment involves performing operations such as displacement, rotation, and / or scaling on the target reference model to match the key feature corresponding to the expression with the key point coordinates of the key feature of the object model. For example, when registering the left eye, it can be expressed as the coordinates of the second corner of the left eye being the same. When registering the mouth, it can be expressed as the coordinates of the two corners of the mouth being the same.

[0048] By using this step, it is possible to achieve registration of any key feature of any facial expression.

[0049] S208, obtain a first size ratio of facial expression according to the point cloud data of key features corresponding to the final frame of facial expression of the target reference model and the point cloud data of key features in the natural state of the target reference model; S209, based on the first size ratio of the facial expression and the point cloud data of the key features in the natural state of the object model, obtain point cloud data of the key features corresponding to the final frame of the facial expression to which the object model is to transition, and obtain the final frame of the facial expression of the object model; In this embodiment, when transitioning to the final frame of an expression, a direct transition can cause problems such as inability to match. For example, when a normal mouth and lips transition to a small cherry-shaped mouth or European lips, the change in lip width when smiling and the change in the extension of the lower jaw when the mouth is open can cause clear inconsistencies. To solve this problem, this embodiment calculates the length, width, and height of the key feature point cloud corresponding to the final frame of the expression based on the key feature point cloud data corresponding to the final frame of the expression of the target reference model. Next, the length, width, and height of the key feature point cloud in the natural state are calculated based on the key feature point cloud in the natural state of the target reference model. The ratio of the length of the key feature point cloud corresponding to the final frame of the expression to the corresponding length of the key feature point cloud in the natural state, the ratio of the width of the key feature point cloud corresponding to the final frame of the expression to the corresponding width of the key feature point cloud in the natural state, and the ratio of the height of the key feature point cloud corresponding to the final frame of the expression to the corresponding height of the key feature point cloud in the natural state are calculated, respectively, to determine the first size ratio of the expression.

[0050] For example, for an open-mouth expression, the coordinates of the highest point of the upper lip line and the lowest point of the lower lip line of the mouth can be obtained based on the point cloud data corresponding to the final frame of the open-mouth expression of the target reference model, thereby obtaining the width of the point cloud of the final frame of the open-mouth expression of the target reference model. Further, point cloud data of the target reference model in its natural state can be obtained, and the coordinates of the highest point of the lower lip line and the lowest point of the lower lip line in the natural state can be obtained, thereby obtaining the width of the point cloud of the target reference model in its natural state. The width of the point cloud of the final frame of the open-mouth expression of the target reference model can then be divided by the width of the point cloud in the natural state to obtain the proportion of the width of the open-mouth expression. Similarly, the abscissa distance between the two corners of the mouth can be obtained as the length of the point cloud, and the proportion of the length of the corresponding open-mouth expression can be obtained as described above. The distance from the outermost point of the mouth and lips to the height of the innermost corner of the mouth can be used as the height of the point cloud, and the proportion of the height of the corresponding open-mouth expression can be obtained as described above, i.e., the proportions of the three coordinate axes in the first size proportion can be obtained.

[0051] Next, based on the first size ratio of the facial expression and the point cloud data of the key features in the natural state of the object model, when the object model creates the facial expression, the point cloud data of the key features corresponding to the final frame, i.e., the coordinates corresponding to each point in the point cloud of the key features corresponding to the final frame, can be obtained, and different scaling ratios can be performed in the three coordinate axis directions in the updated transition result, so that after transitioning to the object model, the point cloud of the key features corresponding to the final frame of the facial expression and the object model are more matched.

[0052] Experimental verification shows that after processing steps S208 and S209, the negative impact of volumetric deformation during facial expression transitions is reduced, and the range of unreasonable facial expressions such as cherry's small mouth expression, open mouth, smiling, and pouting lips is significantly suppressed.

[0053] Note that, unlike linearly transformed global scaling, this step scales each point's transformation differently along the three axes, ensuring that excessive changes are suppressed and similarities are preserved.

[0054] The above steps S206-S209 are one specific implementation of step S103 in the embodiment shown in FIG.

[0055] S210, transferring point cloud data of key features corresponding to the final frame of the facial expression to which the object model is to transition to the object model; S211, after transitioning to the object model, obtain a third feature curve corresponding to the final frame of the facial expression and a fourth feature curve corresponding to the final frame of the facial expression in the target reference model; S212, adjusting the third characteristic curve with the fourth characteristic curve as a constraint; For example, the third and fourth feature curves in this embodiment may be feature curves that appear in a state where an expression is created for a digital human model. For example, for a closed-eye expression, the overlapping curves of the upper and lower eyelids may be the third and fourth feature curves; for a pouting expression, the overlapping curves of the upper and lower eyelids may be the third and fourth feature curves; TIFF0007749894000001.tif1741For facial expressions, the curves that close the mouth and are convex forward can be the third and fourth feature curves; pursing the lips For example, for the TIFF0007749894000002.tif1740 expression, the curves formed by the lips closing and curling inward can be the third and fourth feature curves. When performing facial expression transitions using different models, the third and fourth feature curves are likely to show significant inconsistencies. Based on this, in this embodiment, the fourth feature curve corresponding to the final frame of the facial expression in the target reference model is used as a constraint. After transition to the object model, the third feature curve corresponding to the final frame of the facial expression is adjusted to more accurately integrate the facial expression into the object model and more completely represent the facial expression in the object model. At the same time, the topology around the features of the transitioned facial expression in the object model changes uniformly according to the facial expression, resulting in a very high overall integration degree.

[0056] Specifically, in this embodiment, the third characteristic curve is adjusted using the fourth characteristic curve as a constraint. The third characteristic curve and the fourth characteristic curve can also be called dynamic curves, and this adjustment method can also be called adjustment constrained by the dynamic curve. The dynamic curve can be a mathematical relationship used in digital human image technology. A dynamic curve is a set of points connected in series to a space curve, where a change at any point on the curve causes a change at all other points on the curve. On the surfaces of a digital human model, such as a reference model and an object model, all points and their surrounding neighboring points are connected by straight lines to form a grid structure. The third characteristic curve and the fourth characteristic curve themselves are also formed by the edges of the grid structure. The grid structure includes horizontal and vertical edges, and the third characteristic curve and the fourth characteristic curve in this embodiment are usually horizontal edges. When adjusting the third characteristic curve using the fourth characteristic curve as a constraint, the positions of each point on the third characteristic curve are adjusted so that the third characteristic curve and the fourth characteristic curve are perfectly aligned, or the distance between the third characteristic curve and the fourth characteristic curve is less than a very small preset threshold.

[0057] FIG. 3 is a schematic diagram of the eye structure of an object model provided by the present disclosure. As shown in FIG. 3, the feature curve indicated by the arrow may be a third feature curve. Due to the edge and face characteristics inherent in the grid structure of the object model itself, the adjusted third feature curve cannot change the topology of the feature curve in the grid structure. Therefore, the change in the third feature curve must be transmitted sequentially to adjacent faces and points. In the case of labeling five sensory organs, this transmission is attenuated outward from the outer feature curves of the five sensory organs. That is, to achieve a smoother adjustment, in actual applications, the change in the third feature curve can be attenuated outward by a predetermined number of curves. The predetermined number can be set according to actual needs, for example, 6, 8, 10, or other values. The adjustment range of the third feature curve is largest to match the corresponding fourth feature curve in the target reference model. With reference to the adjustment range of the third characteristic curve and each characteristic curve in the predetermined number of characteristic curves adjacent to the third characteristic curve, as the distance of the third characteristic curve increases, the adjustment range gradually decreases until the adjustment of the last step of the predetermined number of adjacent characteristic curves is completed.

[0058] Based on the above method, the object model is currently subjected to the transition expression result, the third feature curve is constrained by the dynamic curve constraint, and the feature line itself and the topology line around the feature are successively approached to the fourth feature curve, so that the topology feature under this expression can be completely transferred to the object model.

[0059] S213: Smoothing the seam between the point cloud of the key features corresponding to the final frame of the facial expression after transition to the object model and the original point cloud of the object model; In order to more closely integrate the point cloud of the key feature corresponding to the final frame of the transitioned facial expression with the original point cloud of the object model, this embodiment further needs to detect whether the splice between the two is smooth. For locations that are not smooth, a corresponding smoothing process needs to be performed. For example, the AB edge shown in FIG. 3 may be one of the splices of the original point cloud of the object model among the point cloud of the key feature corresponding to the final frame of the transitioned facial expression, and the AB edge is located in the point cloud of the key feature corresponding to the final frame of the transitioned facial expression. In specific processing, it can be detected whether the angle between the two adjacent edges in the horizontal direction of point B is greater than a predetermined threshold angle. The predetermined threshold angle can be set based on the actual scene, for example, 15 degrees, 20 degrees, or 30 degrees, but is not limited thereto. If so, the splice is deemed to be unsmooth, and the position of point B needs to be adjusted so that the angle between the two adjacent edges in the horizontal direction of point B is equal to or less than the predetermined threshold angle. Similarly, a similar method can be used to detect the angle between two adjacent vertical edges of point B, and if it is not smooth, adjust the position of point B. In short, through the adjustment in the above-mentioned method, all the seams of the key feature point cloud corresponding to the final frame of the facial expression and the original point cloud of the object model are very smooth, so that the key feature point cloud corresponding to the final frame of the facial expression can be perfectly blended with the original point cloud of the object model.

[0060] For example, when the final frame transition of the facial expression is completed, there will be a slight problem in the change of the point cloud of the final frame of the transitioned facial expression and the original point cloud seam, for example, in the places where multiple five sensory organ parts intersect, such as both sides of the nostrils and the middle of the root, the wiring may not be smooth in some cases.The smoothing processing method of this embodiment can fully solve this type of problem and ensure the smoothness of the lines.

[0061] Steps S210-S213 are complementary to the above-mentioned steps S206-S209, which can further improve the transition effect of the final facial expression frame, and can improve the accuracy and efficiency of the facial expression transition.

[0062] S214, according to the point cloud data of the key features corresponding to the intermediate frame of the facial expression in the facial expression library of the target reference model, the point cloud data of the key features in the natural state of the target reference model, and the point cloud data of the key features corresponding to the final frame of the facial expression of the target reference model, obtain a second size ratio of the intermediate frame of the facial expression to the final frame; Specifically, based on the point cloud data of key features corresponding to intermediate frames of facial expressions in the facial expression library of the target reference model, the point cloud data of key features in the natural state of the target reference model, and the point cloud data of key features corresponding to the final frame of facial expressions of the target reference model, a fourth feature curve corresponding to the final frame of facial expressions of the target reference model, a fifth feature curve corresponding to intermediate frames of facial expressions, and a sixth feature curve in the natural state are obtained. The fourth, fifth, and sixth feature curves are all feature curves of key features in the target reference model.

[0063] Similarly, for an open-mouth expression, the fourth feature curve may correspond to the upper lip line corresponding to the maximum mouth opening, the fifth feature curve may be the upper lip line at an intermediate stage during the mouth opening process, and the sixth feature curve may be the closing line when the upper and lower lips are closed when the mouth is not open. Specifically, the highest point on the upper lip line is used as the key point, and the difference in distance between the key point on the fifth feature curve and the key point on the upper lip line on the fourth feature curve is divided by the difference in distance between the key point on the fourth feature curve and the key point on the upper lip line on the sixth feature curve to obtain the ratio of the height of the intermediate frame of the open-mouth expression to the width of the final frame as the width ratio at the second size ratio. Similarly, the length and height ratio at the first size ratio can be similarly combined to obtain the length and height ratio at the second size ratio.

[0064] S215, obtain an intermediate frame of the facial expression of the object model according to a second size ratio of the intermediate frame of the facial expression to the final frame and the final frame of the facial expression of the object model.

[0065] When it is specifically realized, (a3) obtaining a corresponding reference feature curve of the intermediate frame of the facial expression of the object model according to a second size ratio of the intermediate frame of the facial expression to the final frame and a corresponding third feature curve of the final frame of the facial expression of the object model; (b3) transferring point cloud data of key features corresponding to intermediate frames of facial expressions of the target reference model to the object model; (c3) after transitioning to the object model, obtaining a seventh feature curve corresponding to the intermediate frame of the facial expression; (d3) adjusting the seventh characteristic curve using the reference characteristic curve as a constraint.

[0066] In this embodiment, the reference feature curve is the reference feature curve that should theoretically correspond after the facial expression transition of the intermediate frame, but the seventh feature curve is the actual reference feature curve after the facial expression transition of the intermediate frame, so that in order to improve the intermediate frame transition effect, it is necessary to perform dynamic curve adjustment using the method of step (d3).

[0067] Specifically, the implementation of step (d3) is the same as the implementation principle of step S212 in the above-mentioned embodiment, and can realize the adjustment of the outer attenuation characteristic curve, for which details can be referred to the relevant description of the above-mentioned embodiment and will not be described in detail here. In addition, the method of step S213 above can also be used to perform smoothing processing on the transitioned intermediate frame, for which details will not be described here.

[0068] Steps S214 and S215 also need to perform intermediate frame transitions if the expressions in the expression library include intermediate frames.

[0069] In conventional facial expression binding, there are only two data points: the natural state and the expression frame, and the change in the width value of the expression is the interpolated value between the natural state and the target frame. This method cannot express the details of the facial expression change process of a real person, such as the appearance and disappearance of a pear swirl when a person is smiling.

[0070] From the viewpoint of facial expression transition angle, the intermediate frame, the first frame and the final frame of the facial expression have a certain non-linear relationship.

[0071] In this embodiment, by sampling the intermediate and final frames of the feature curve, a change constraint for each intermediate frame with respect to the final frame is established. When transitioning between intermediate frames, not only is the intermediate frame feature curve maintained, but also a change constraint for the final frame needs to be added to this curve, which can effectively improve the transition efficiency of the intermediate frames.

[0072] In this embodiment, the transitioned intermediate frame adds intermediate expressions of some non-linear changes between the neutral state and the final frame. In this way, the change of one expression from 0 to 1 evolves from a linear change to a non-linear change, and detailed changes such as the appearance and disappearance of a pear-shaped curl, or the protruding lip and then straightening out can all be realized, making the transitioned expression more natural, reasonable, and efficient.

[0073] By using the above steps, all expressions in the target reference model can be transferred to the object model one by one by constantly repeating the above transition steps, and the expressions of the transferred object model can also reach a good level based on the above series of constraints.

[0074] The digital human facial expression transition method of this embodiment can quickly and efficiently transition facial expressions in the facial expression library of the target reference model to the object model using the above-mentioned method. Furthermore, the facial expression transition process is fast and requires little computational power. It can be performed in the background of the image production process in real time, completing automatic binding within minutes, and even allowing direct capture and drive preview of the effect. This method does not rely on the support of large-scale computing devices or require long wait times for generation, making the facial expression transition highly efficient and accurate.

[0075] Furthermore, for scenes where it is difficult to fully achieve the effect using other means such as closing the eyes, pouting, or pursing the lips, the digital human facial expression transition method of this embodiment can be used to fully transition the facial expression and effectively achieve the effect of the facial expression transition.

[0076] In addition, the digital human facial expression transition method of this embodiment supports 4D nonlinear transitions. It can also support facial expression transitions using nonlinear interpolation values ​​in intermediate frames. Furthermore, under constraint roles, it can more effectively ensure the 4D nonlinear facial expression effect after transition.

[0077] 4 is a schematic diagram according to a third embodiment of the present disclosure. As shown in FIG. 4, this embodiment provides a digital human facial expression transition device 400, which includes a selection module 401, an acquisition module 402, and a final frame transition module 403; The selection module 401 is used to select an identifier of a target reference model that matches an object model from a pre-defined reference model library, the reference model library including a plurality of reference models; The acquisition module 402 is used to acquire an expression library of the target reference model according to the identifier of the target reference model; The final frame transition module 403 is used to transition the final frame of the expression in the expression library of the target reference model to the object model to obtain the final frame of the expression of the object model.

[0078] The digital human facial expression transition device 400 of this embodiment realizes the principles and technical effects of digital human facial expression transition by using the above-mentioned modules, which are the same as those realized in the above-mentioned related method embodiment. For details, please refer to the description of the above-mentioned related method embodiment, and a detailed description will not be given here.

[0079] 5 is a schematic diagram of a fourth embodiment of the present disclosure. The digital human facial expression transition device 500 of this embodiment is based on the technical solution of the embodiment shown in FIG. 4 described above, and further describes the technical solution of the present disclosure in more detail. As shown in FIG. 5, the digital human facial expression transition device 500 of this embodiment includes modules with the same names and functions as those shown in FIG. 4 above: a selection module 501, an acquisition module 502, and a final frame transition module 503.

[0080] In this embodiment, the sorting module 501 The identifier of the target reference model matching the object model is selected from the reference model library based on the attribute information of the object model and the attribute information of each of the reference models in the reference model library.

[0081] Alternatively, in one embodiment of the present disclosure, the filtering module 501 may: Displaying attribute information of each of the reference models in the reference model library to a user; It is used to receive the target reference model that matches the object model selected by the user.

[0082] Alternatively, in one embodiment of the present disclosure, the filtering module 501 may: obtaining a first characteristic curve of a preset region from the object model; obtaining a second characteristic curve of the preset region from each of the reference models; calculating an offset distance of the preset portion between the object model and each of the reference models based on the first characteristic curve and each of the second characteristic curves; The offset distance of the preset portion between the object model and each of the reference models is used to select an identifier of a target reference model that matches the object model from the reference model library.

[0083] Further, optionally, in one embodiment of the present disclosure, the selection module 501 obtains coordinates of each point on the first characteristic curve and each of the second characteristic curves, respectively; calculating point distances of the same point identifiers on the first feature curve and each of the second feature curves based on coordinates of each point on the first feature curve and each of the second feature curves; adding the point distances of each point identifier on the first feature curve and each of the second feature curves to obtain a sum of point distances; The sum of the point distances and the number of points included in the first feature curve are used to obtain the offset distances of the preset portions between the object model and each of the reference models.

[0084] Further, optionally, in one embodiment of the present disclosure, the filtering module 501 The identifier of the reference model having the smallest offset distance of the preset portion from among a plurality of reference models in the reference model library is used to select the identifier of the target reference model matching the object model.

[0085] Further, optionally, in one embodiment of the present disclosure, when two or more preset portions are included, the selection module 501 calculates a total offset distance between the object model and each of the reference models based on a weight of each of the preset portions and an offset distance corresponding to each of the preset portions; The total offset distance between the object model and each of the reference models is used to select an identifier of a target reference model that matches the object model from the reference model library.

[0086] Further, optionally, in one embodiment of the present disclosure, the final frame transition module 503 performing global registration of the target reference model and the object model; performing registration of key features corresponding to the facial expression of the target reference model with key features of the object model; Obtain a first size ratio of a facial expression based on point cloud data of key features corresponding to the final frame of the facial expression of the target reference model and point cloud data of key features of the target reference model in a natural state; Based on the first size ratio of the facial expression and the point cloud data of the key features in the natural state of the object model, point cloud data of the key features corresponding to the final frame of the facial expression to which the object model is to transition is obtained, and this is used to obtain the final frame of the facial expression of the object model.

[0087] Further, optionally, in one embodiment of the present disclosure, the final frame transition module 503 Transmitting point cloud data of key features corresponding to the final frame of the facial expression to which the object model is to transition to the object model, After transitioning to the object model, obtain a third feature curve corresponding to the final frame of the facial expression and a fourth feature curve corresponding to the final frame of the facial expression in the target reference model; adjusting the third characteristic curve using the fourth characteristic curve as a constraint; It is used to perform smooth processing on the seam between the point cloud of the key features corresponding to the final frame of the facial expression after transition to the object model and the original point cloud of the object model.

[0088] Optionally, as shown in FIG. 5, in one embodiment of the present disclosure, the facial expression transition device 500 of the digital human includes: If the expression library further includes an intermediate frame of the expression, the method further includes an intermediate frame transition module 504 for transitioning the intermediate frame of the expression in the expression library of the target reference model to the object model to obtain an intermediate frame of the expression of the object model.

[0089] Further, optionally, in one embodiment of the present disclosure, the inter-frame transition module 504 Obtain a second size ratio of the intermediate frame of the facial expression to the final frame based on point cloud data of key features corresponding to the intermediate frame of the facial expression in the facial expression library of the target reference model, point cloud data of key features in the natural state of the target reference model, and point cloud data of key features corresponding to the final frame of the facial expression of the target reference model; A second size ratio of the intermediate facial expression frame to the final facial expression frame is used to obtain an intermediate facial expression frame of the object model based on the final facial expression frame of the object model.

[0090] Further, optionally, in one embodiment of the present disclosure, the inter-frame transition module 504 obtain a fourth feature curve corresponding to the final frame of the facial expression of the target reference model, a fifth feature curve corresponding to the intermediate frame of the facial expression, and a sixth feature curve in the natural state based on point cloud data of key features corresponding to the intermediate frame of the facial expression in the facial expression library of the target reference model, point cloud data of key features in the natural state of the target reference model, and point cloud data of key features corresponding to the final frame of the facial expression of the target reference model; A second size ratio of the intermediate frame of the facial expression to the final frame is obtained based on the coordinates of key points on the fourth feature curve, the fifth feature curve, and the sixth feature curve.

[0091] Further, optionally, in one embodiment of the present disclosure, the inter-frame transition module 504 Obtain a corresponding reference feature curve of the intermediate frame of the facial expression of the object model according to a second size ratio of the intermediate frame of the facial expression to the final frame and a corresponding third feature curve of the final frame of the facial expression of the object model; Transferring point cloud data of key features corresponding to the facial expression intermediate frames of the target reference model to the object model; After transitioning to the object model, obtain a seventh feature curve corresponding to the intermediate frame of the facial expression; The reference characteristic curve is used as a constraint to adjust the seventh characteristic curve.

[0092] The digital human facial expression transition device 500 of this embodiment achieves the principles and technical effects of digital human facial expression transition by using the above-mentioned modules, which are the same as those achieved in the above-mentioned related method embodiments. For details, please refer to the description of the above-mentioned related method embodiments, and a detailed description will not be given here.

[0093] In the technical solution disclosed herein, the acquisition, storage, application, etc. of relevant user personal information shall all comply with the provisions of relevant laws and regulations and shall not violate public order and morals.

[0094] According to embodiments of the present disclosure, the present disclosure further provides an electronic device, a readable storage medium, and a computer program product.

[0095] As shown in Figure 6, a block diagram of an electronic device 600 for implementing an embodiment of the present disclosure is shown. The electronic device is intended to represent various types of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframes, and other suitable computers. The electronic device may also represent various types of mobile devices, such as personal digital assistants, mobile phones, smartphones, wearable devices, and other similar computing devices. The components, their connections and relationships, and their functions shown herein are merely examples and are not intended to limit the description herein and / or the practice of the present disclosure as claimed.

[0096] 6, the device 600 includes a computing unit 601, which can perform various appropriate operations and processes based on a computer program stored in a read-only memory (ROM) 602 or loaded from a storage unit 608 into a random access memory (RAM) 603. The RAM 603 can also store various programs and data required for the device 600 to operate. The computing unit 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0097] Multiple components in device 600 are connected to I / O interface 605, including input units 606 such as a keyboard, mouse, etc., output units 607 such as various types of displays, speakers, etc., storage units 608 such as a disk, optical disk, etc., and communication units 609 such as a network card, modem, wireless communication transceiver, etc. The communication unit 609 enables device 600 to exchange information / data with other devices via computer networks such as the Internet and / or various telecommunication networks.

[0098] The computing unit 601 is a general-purpose and / or special-purpose processing component equipped with various processing and computational capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, computing units that execute various machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 601 performs various methods and processes described above, such as the above-described methods of the present disclosure. For example, in some embodiments, the above-described methods of the present disclosure can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 608. In some embodiments, some or all of the computer program is loaded and / or installed into the device 600 via the ROM 602 and / or the communication unit 609. When the computer program is loaded into the RAM 603 and executed by the computing unit 601, it can perform one or more steps of the above-described methods. Alternatively, in other embodiments, the computing unit 601 can be configured to perform the above-described methods of the present disclosure through any other suitable manner (e.g., by firmware).

[0099] Various implementations of the systems and techniques described herein may be realized in digital electronic circuitry systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-a-chip systems (SOCs), chip programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include being implemented in one or more computer programs that can be executed and / or interpreted by a programmable system that includes at least one programmable processor, which may be a special-purpose or general-purpose programmable processor, and that can receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0100] Program codes for implementing the methods of the present disclosure can be written using any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus such that, when executed by the processor or controller, the functions / acts specified in the flowcharts and / or block diagrams are performed. The program code can be executed entirely on the machine, partially on the machine, partially on the machine as a separate software package and partially on a remote machine, or entirely on a remote machine or server.

[0101] In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain or store a program for use with or in connection with an instruction execution system, device, or apparatus. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium includes, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0102] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user, and a keyboard and pointing device (e.g., a mouse or trackball) through which a user can provide input to the computer. Other types of devices can also be used to provide interaction with a user; for example, feedback provided to the user can be any form of sensing feedback (e.g., visual feedback, auditory feedback, or haptic feedback) and can receive input from the user in any form (including acoustic input, voice input, and tactile input).

[0103] The systems and techniques described herein can be implemented in a computing system including a back-end component (e.g., a data server), a computing system including a middleware component (e.g., an application server), a computing system including a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user interacts with an implementation of the systems and techniques described herein), or any combination of such back-end, middleware, and front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), the Internet, and a blockchain network.

[0104] The computer system may include a client and a server. The client and server are generally remote from each other and typically interact through a communication network. The relationship between the client and the server is created by computer programs running on corresponding computers and having a client-server relationship to each other. The server may be a cloud server, a server in a distributed system, or a server that combines blockchains.

[0105] It should be understood that steps can be rearranged, added, or deleted using the various types of flows shown above. For example, the steps described in the present disclosure may be performed in parallel, sequentially, or in a different order, but this specification is not limited thereto as long as the technical solution disclosed in the present disclosure can achieve the desired results.

[0106] The above specific implementation methods do not constitute limitations on the scope of protection of the present disclosure. Those skilled in the art may make various modifications, combinations, subcombinations, and substitutions based on design requirements and other factors. Any modifications, equivalent replacements, and improvements made within the spirit and principle of the present disclosure shall fall within the scope of protection of the present disclosure.

Claims

1. A facial expression transition method for a digital human, comprising: selecting an identifier of a target reference model that matches the object model from a predefined reference model library, the reference model library including a plurality of reference models; obtaining an expression library of the target reference model based on an identifier of the target reference model; and transferring a final frame of an expression in the expression library of the target reference model to the object model to obtain a final frame of an expression of the object model; The step of selecting an identifier of a target reference model that matches the object model from a preset reference model library includes: obtaining a first characteristic curve of a preset region from the object model; obtaining a second characteristic curve of the preset region from each of the reference models; calculating an offset distance of the preset portion between the object model and each of the reference models based on the first characteristic curve and each of the second characteristic curves; selecting an identifier of a target reference model that matches the object model from the reference model library based on an offset distance of the preset portion between the object model and each of the reference models; A method for facial expression transition in digital humans.

2. The step of calculating offset distances of the preset portions between the object model and each of the reference models based on the first feature curve and the second feature curve includes: obtaining coordinates of each point on the first characteristic curve and each of the second characteristic curves; calculating point distances of the same point identifiers on the first feature curve and each of the second feature curves based on coordinates of each point on the first feature curve and each of the second feature curves; adding point distances of each point identifier on the first feature curve and each of the second feature curves to obtain a sum of point distances; and obtaining an offset distance of the preset portion between the object model and each of the reference models based on the sum of the point distances and the number of points included in the first feature curve. The facial expression transition method for a digital human according to claim 1 .

3. selecting an identifier of a target reference model matching the object model from the reference model library based on an offset distance of the preset portion between the object model and each of the reference models, selecting an identifier of a reference model having the smallest offset distance of the preset portion from a plurality of reference models in the reference model library as an identifier of the target reference model matching the object model; The facial expression transition method for a digital human according to claim 2 .

4. selecting an identifier of a target reference model matching the object model from the reference model library based on an offset distance of the preset portion between the object model and each of the reference models, When the preset portions include two or more, calculating a total offset distance between the object model and each of the reference models based on a weight of each of the preset portions and an offset distance corresponding to each of the preset portions; selecting an identifier of a target reference model that matches the object model from the reference model library based on a total offset distance between the object model and each of the reference models; The facial expression transition method for a digital human according to claim 2 .

5. The step of transferring the final frame of the facial expression in the facial expression library of the target reference model to the object model to obtain the final frame of the facial expression of the object model includes: performing global registration of the target reference model and the object model; performing registration of key features corresponding to facial expressions of the target reference model with key features of the object model; obtaining a first size ratio of a facial expression based on point cloud data of key features corresponding to the final frame of the facial expression of the target reference model and point cloud data of key features in a natural state of the target reference model; acquiring point cloud data of key features corresponding to a final frame of the facial expression to which the object model is to transition, based on the first size ratio of the facial expression and point cloud data of key features in the natural state of the object model, and acquiring a final frame of the facial expression of the object model. The facial expression transition method for a digital human according to any one of claims 1 to 4.

6. After obtaining point cloud data of key features corresponding to a final frame of the facial expression to which the object model is to transition based on the size ratio of the facial expression and point cloud data of key features in the natural state of the object model, and before obtaining a final frame of the facial expression of the object model, the method includes: transferring point cloud data of key features corresponding to the final frame of the facial expression to which the object model is to be transferred to the object model; After transitioning to the object model, obtaining a third feature curve corresponding to the final frame of the facial expression and a fourth feature curve corresponding to the final frame of the facial expression in the target reference model; adjusting the third characteristic curve using the fourth characteristic curve as a constraint; and performing a smoothing process on a seam between the point cloud of key features corresponding to the final frame of the facial expression after transition to the object model and the original point cloud of the object model. The facial expression transition method for a digital human according to claim 5 .

7. After the step of transferring the final frame of an expression in the expression library of the target reference model to the object model to obtain the final frame of an expression of the object model, the method further comprises: If the expression library further includes an intermediate frame of the expression, the intermediate frame of the expression in the expression library of the target reference model is transferred to the object model to obtain an intermediate frame of the expression of the object model. The facial expression transition method for a digital human according to any one of claims 1 to 4.

8. The step of transferring the intermediate facial expression frames in the facial expression library of the target reference model to the object model to obtain the intermediate facial expression frames of the object model includes: obtaining a second size ratio of the intermediate frame of the facial expression to the final frame based on point cloud data of key features corresponding to the intermediate frame of the facial expression in the facial expression library of the target reference model, point cloud data of key features in the natural state of the target reference model, and point cloud data of key features corresponding to the final frame of the facial expression of the target reference model; obtaining an intermediate frame of the facial expression of the object model based on a second size ratio of the intermediate frame of the facial expression to the final frame and the final frame of the facial expression of the object model; The facial expression transition method for a digital human according to claim 7 .

9. The step of obtaining a second size ratio of the intermediate frame of the facial expression to the final frame based on point cloud data of key features corresponding to the intermediate frame of the facial expression in the facial expression library of the target reference model, point cloud data of key features in the natural state of the target reference model, and point cloud data of key features corresponding to the final frame of the facial expression of the target reference model, includes: obtaining a fourth feature curve corresponding to the final frame of the facial expression of the target reference model, a fifth feature curve corresponding to the intermediate frame of the facial expression, and a sixth feature curve in the natural state based on point cloud data of key features corresponding to the intermediate frame of the facial expression in the facial expression library of the target reference model, point cloud data of key features in the natural state of the target reference model, and point cloud data of key features corresponding to the final frame of the facial expression of the target reference model; and obtaining a second size ratio of the intermediate frame of the facial expression to a final frame based on coordinates of key points on the fourth feature curve, the fifth feature curve, and the sixth feature curve. The facial expression transition method for a digital human according to claim 8 .

10. The step of obtaining an intermediate frame of the facial expression of the object model based on a second size ratio of the intermediate frame of the facial expression to the final frame and the final frame of the facial expression of the object model includes: obtaining a corresponding reference feature curve of the intermediate frame of the facial expression of the object model according to a second size ratio of the intermediate frame of the facial expression to the final frame and a corresponding third feature curve of the final frame of the facial expression of the object model; transferring point cloud data of key features corresponding to the facial expression intermediate frames of the target reference model to the object model; After transitioning to the object model, obtaining a seventh feature curve corresponding to the intermediate frame of the facial expression; adjusting the seventh characteristic curve using the reference characteristic curve as a constraint. The facial expression transition method for a digital human according to claim 8 .

11. A method for transitioning facial expressions of a digital human, comprising: selecting an identifier of a target reference model that matches the object model from a predefined reference model library, the reference model library including a plurality of reference models; obtaining an expression library of the target reference model based on an identifier of the target reference model; and transferring a final frame of an expression in the expression library of the target reference model to the object model to obtain a final frame of an expression of the object model; The step of transferring the final frame of the facial expression in the facial expression library of the target reference model to the object model to obtain the final frame of the facial expression of the object model includes: performing global registration of the target reference model and the object model; performing registration of key features corresponding to facial expressions of the target reference model with key features of the object model; obtaining a first size ratio of a facial expression based on point cloud data of key features corresponding to the final frame of the facial expression of the target reference model and point cloud data of key features in a natural state of the target reference model; acquiring point cloud data of key features corresponding to a final frame of the facial expression to which the object model is to transition, based on the first size ratio of the facial expression and point cloud data of key features in the natural state of the object model, and acquiring a final frame of the facial expression of the object model. A method for facial expression transition in digital humans.

12. A method for transitioning facial expressions of a digital human, comprising: selecting an identifier of a target reference model that matches the object model from a predefined reference model library, the reference model library including a plurality of reference models; obtaining an expression library of the target reference model based on an identifier of the target reference model; and transferring a final frame of an expression in the expression library of the target reference model to the object model to obtain a final frame of an expression of the object model; After the step of transferring the final frame of an expression in the expression library of the target reference model to the object model to obtain the final frame of an expression of the object model, the method further comprises: If the expression library further includes an intermediate frame of the expression, the intermediate frame of the expression in the expression library of the target reference model is transferred to the object model to obtain an intermediate frame of the expression of the object model; The step of transferring the intermediate facial expression frames in the facial expression library of the target reference model to the object model to obtain the intermediate facial expression frames of the object model includes: obtaining a second size ratio of the intermediate frame of the facial expression to the final frame based on point cloud data of key features corresponding to the intermediate frame of the facial expression in the facial expression library of the target reference model, point cloud data of key features in the natural state of the target reference model, and point cloud data of key features corresponding to the final frame of the facial expression of the target reference model; obtaining an intermediate frame of the facial expression of the object model based on a second size ratio of the intermediate frame of the facial expression to the final frame and the final frame of the facial expression of the object model; A method for facial expression transition in digital humans.

13. A facial expression transition device for a digital human, comprising: a selection module for selecting an identifier of a target reference model matching the object model from a predefined reference model library, the reference model library including a plurality of reference models; an acquisition module for acquiring a facial expression library of the target reference model based on an identifier of the target reference model; a final frame transition module for transitioning a final frame of an expression in the expression library of the target reference model to the object model to obtain a final frame of an expression of the object model; The sorting module comprises: obtaining a first characteristic curve of a preset region from the object model; obtaining a second characteristic curve of the preset region from each of the reference models; calculating an offset distance of the preset portion between the object model and each of the reference models based on the first characteristic curve and each of the second characteristic curves; is used to select an identifier of a target reference model matching the object model from the reference model library based on an offset distance of the preset portion between the object model and each of the reference models. Digital human facial expression transition device.

14. The sorting module comprises: Obtaining coordinates of each point on the first characteristic curve and each of the second characteristic curves; calculating point distances of the same point identifiers on the first feature curve and each of the second feature curves based on coordinates of each point on the first feature curve and each of the second feature curves; adding the point distances of each point identifier on the first feature curve and each of the second feature curves to obtain a sum of point distances; is used to obtain an offset distance of the preset portion between the object model and each of the reference models based on the sum of the point distances and the number of points included in the first feature curve; The facial expression transition device for a digital human according to claim 13.

15. The sorting module comprises: is used to select an identifier of a reference model having the smallest offset distance of the preset portion from a plurality of reference models in the reference model library as an identifier of the target reference model matching the object model. The facial expression transition device for a digital human according to claim 14.

16. The sorting module comprises: When two or more preset portions are included, calculating a total offset distance between the object model and each of the reference models based on a weight of each of the preset portions and an offset distance corresponding to each of the preset portions; used to select an identifier of a target reference model that matches the object model from the reference model library based on a total offset distance between the object model and each of the reference models. The facial expression transition device for a digital human according to claim 14.

17. The final frame transition module: performing global registration of the target reference model and the object model; performing registration of key features corresponding to the facial expression of the target reference model with key features of the object model; Obtain a first size ratio of a facial expression based on point cloud data of key features corresponding to the final frame of the facial expression of the target reference model and point cloud data of key features of the target reference model in a natural state; based on the first size ratio of the facial expression and point cloud data of key features in the natural state of the object model, to obtain point cloud data of key features corresponding to the final frame of the facial expression to which the object model is to transition, and used to obtain the final frame of the facial expression of the object model; The facial expression transition device for a digital human according to any one of claims 13 to 16.

18. The final frame transition module further comprises: Transmitting point cloud data of key features corresponding to the final frame of the facial expression to which the object model is to transition to the object model, After transitioning to the object model, a third feature curve corresponding to the final frame of the facial expression and a fourth feature curve corresponding to the final frame of the facial expression in the target reference model are obtained; adjusting the third characteristic curve using the fourth characteristic curve as a constraint; is used to perform smooth processing on a seam between a point cloud of key features corresponding to the final frame of the facial expression after transition to the object model and the original point cloud of the object model; The facial expression transition device for a digital human according to claim 17.

19. The device comprises: and an intermediate frame transition module for transitioning the intermediate frame of the facial expression in the facial expression library of the target reference model to the object model when the facial expression library further includes an intermediate frame of the facial expression. The facial expression transition device for a digital human according to any one of claims 13 to 16.

20. The mid-frame transition module: Obtain a second size ratio of the intermediate frame of the facial expression to the final frame based on point cloud data of key features corresponding to the intermediate frame of the facial expression in the facial expression library of the target reference model, point cloud data of key features in the natural state of the target reference model, and point cloud data of key features corresponding to the final frame of the facial expression of the target reference model; used to obtain an intermediate facial expression frame of the object model based on a second size ratio of the intermediate facial expression frame to the final facial expression frame and the final facial expression frame of the object model; The facial expression transition device for a digital human according to claim 19.

21. The mid-frame transition module: obtain a fourth feature curve corresponding to the final frame of the facial expression of the target reference model, a fifth feature curve corresponding to the intermediate frame of the facial expression, and a sixth feature curve in the natural state based on point cloud data of key features corresponding to the intermediate frame of the facial expression in the facial expression library of the target reference model, point cloud data of key features in the natural state of the target reference model, and point cloud data of key features corresponding to the final frame of the facial expression of the target reference model; and obtaining a second size ratio of the intermediate frame of the facial expression to the final frame based on coordinates of key points on the fourth feature curve, the fifth feature curve, and the sixth feature curve. The facial expression transition device for a digital human according to claim 20.

22. The mid-frame transition module: Obtaining a corresponding reference feature curve for the intermediate frame of the facial expression of the object model according to a second size ratio of the intermediate frame of the facial expression to the final frame and a corresponding third feature curve for the final frame of the facial expression of the object model; Transferring point cloud data of key features corresponding to the facial expression intermediate frames of the target reference model to the object model; After transitioning to the object model, obtain a seventh feature curve corresponding to the intermediate frame of the facial expression; the reference characteristic curve is used as a constraint to adjust the seventh characteristic curve; The facial expression transition device for a digital human according to claim 20.

23. A facial expression transition device for a digital human, comprising: a selection module for selecting an identifier of a target reference model matching the object model from a predefined reference model library, the reference model library including a plurality of reference models; an acquisition module for acquiring a facial expression library of the target reference model based on an identifier of the target reference model; a final frame transition module for transitioning a final frame of an expression in the expression library of the target reference model to the object model to obtain a final frame of an expression of the object model; The final frame transition module: performing global registration of the target reference model and the object model; performing registration of key features corresponding to the facial expression of the target reference model with key features of the object model; Obtain a first size ratio of a facial expression based on point cloud data of key features corresponding to the final frame of the facial expression of the target reference model and point cloud data of key features of the target reference model in a natural state; based on the first size ratio of the facial expression and point cloud data of key features in the natural state of the object model, to obtain point cloud data of key features corresponding to the final frame of the facial expression to which the object model is to transition, and used to obtain the final frame of the facial expression of the object model; Digital human facial expression transition device.

24. A facial expression transition device for a digital human, comprising: a selection module for selecting an identifier of a target reference model matching the object model from a predefined reference model library, the reference model library including a plurality of reference models; an acquisition module for acquiring a facial expression library of the target reference model based on an identifier of the target reference model; a final frame transition module for transitioning a final frame of an expression in the expression library of the target reference model to the object model to obtain a final frame of an expression of the object model; The device comprises: When the expression library further includes an intermediate frame of the expression, an intermediate frame transition module is further included for transitioning the intermediate frame of the expression in the expression library of the target reference model to the object model to obtain an intermediate frame of the expression of the object model; The mid-frame transition module: Obtain a second size ratio of the intermediate frame of the facial expression to the final frame based on point cloud data of key features corresponding to the intermediate frame of the facial expression in the facial expression library of the target reference model, point cloud data of key features in the natural state of the target reference model, and point cloud data of key features corresponding to the final frame of the facial expression of the target reference model; used to obtain an intermediate facial expression frame of the object model based on a second size ratio of the intermediate facial expression frame to the final facial expression frame and the final facial expression frame of the object model; Digital human facial expression transition device.

25. An electronic device, at least one processor; a memory communicatively coupled to the at least one processor; The memory stores instructions executable by the at least one processor, the instructions, when executed by the at least one processor, causing the at least one processor to perform the method of any one of claims 1 to 4. electronic equipment.

26. A non-transitory computer-readable storage medium having computer instructions stored thereon, comprising: The computer instructions cause a computer to carry out the method according to any one of claims 1 to 4. A non-transitory computer-readable storage medium.

27. A computer program comprising: The computer program, when executed by a processor, implements the method according to any one of claims 1 to 4. Computer program.

Images