Method and device for transferring facial expression of digital human, electronic device, and storage medium

Abstract

To provide a method for transferring facial expression of digital humans applied to scenes such as meta-universe and virtual digital humans, a device for transferring facial expression of digital humans, an electronic device, and a storage medium.SOLUTION: A method for transferring facial expression of digital humans includes a step of screening an identification of a target reference model matched with an object model from a preset reference model library. The reference model library includes a plurality of reference models. The method also includes a step of acquiring an expression library of the target reference model on the basis of the identification of the target reference model, and a step of acquiring a last frame of an expression of the object model by transferring the last frame of an expression in the expression library of the target reference model into the object model.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to computer technology and artificial intelligence technology fields, particularly to technical fields such as augmented reality, virtual reality, computer vision, and deep learning, and can be applied to scenes such as meta-universes and virtual digital humans. More specifically, the present disclosure relates to a method, device, electronic device, and storage medium for facial expression transition of a digital human. [Background technology]

[0002] Binding is a key part of digital human image-driven design, which can realize adding digital human facial expressions to digital human models. In the past, binding was usually completed by professional designers. There are two types of binding: blendshape deformation and bone skin. Some scenes use a combination of the two to obtain optimization.

[0003] For different digital human images, the binding work also requires a certain amount of input. However, it should be noted that the time period is not short, usually more than 1-2 weeks. Under the requirement of high-quality facial expression driving effect of ultra-realistic digital human, this time cost will further increase significantly. Summary of the Invention [Problem to be solved by the invention]

[0004] The present disclosure provides a method, device, electronic device, and storage medium for facial expression transition of a digital human. [Means for solving the problem]

[0005] According to one aspect of the present disclosure, there is provided a method for facial expression transition of a digital human, comprising: selecting an identifier of a target reference model matching the object model from a predefined reference model library, the reference model library including a plurality of reference models; obtaining a facial expression library of the target reference model based on an identifier of the target reference model; and transferring a final frame of an expression in the expression library of the target reference model to the object model to obtain a final frame of an expression of the object model.

[0006] According to another aspect of the present disclosure, there is provided a facial expression transition device for a digital human, comprising: a selection module for selecting an identifier of a target reference model matching an object model from a predefined reference model library, the reference model library including a plurality of reference models; an acquisition module for acquiring a facial expression library of the target reference model based on an identifier of the target reference model; a final frame transition module for transitioning a final frame of an expression in the expression library of the target reference model to the object model to obtain a final frame of an expression of the object model.

[0007] According to another aspect of the present disclosure, there is provided an electronic device, comprising: At least one processor; a memory in communication with the at least one processor; The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enable the at least one processor to perform the methods of the aspects and any possible implementations.

[0008] According to another aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium having computer instructions stored thereon, the computer instructions causing the computer to perform the method of the aspect and any possible implementations thereof.

[0009] According to yet another aspect of the present disclosure, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the method of the aspect and any possible implementation manner.

[0010] According to the technology disclosed herein, facial expression transitions of digital humans can be automatically realized. Compared with conventional technologies, the entire facial expression transition process can be saved in time and effort, and the accuracy of the transitioned facial expressions can be effectively ensured. Furthermore, the efficiency of digital human facial expression transitions can be effectively improved, and the efficiency of generating digital human images can be improved.

[0011] It should be understood that the contents described herein are not intended to identify key or important features of the embodiments of the present disclosure, nor should they be used to limit the scope of the present disclosure. Other features of the present disclosure can be readily understood through the following specification. [Brief description of the drawings]

[0012] The drawings are for a better understanding of the present application and are not intended to limit the present application. [Figure 1] FIG. 1 is a schematic diagram according to a first embodiment of the present disclosure. [Diagram 2] FIG. 2 is a schematic diagram according to a second embodiment of the present disclosure. [Diagram 3] FIG. 2 is a schematic diagram of an eye structure of an object model provided by the present disclosure. [Figure 4] FIG. 11 is a schematic diagram according to a third embodiment of the present disclosure. [Diagram 5] FIG. 11 is a schematic diagram according to a fourth embodiment of the present disclosure. [Figure 6]FIG. 1 is a block diagram of an electronic device for implementing a method according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Hereinafter, exemplary embodiments of the present application will be described based on the drawings. For ease of understanding, various details of the embodiments of the present application are included, and they should be regarded as merely examples. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present application. Similarly, for the sake of simplicity, the following description omits the description of well-known functions and structures.

[0014] Apparently, the described embodiments are some embodiments of the present application, but not all embodiments. According to the embodiments of the present application, all other embodiments that a person skilled in the art can obtain without creative labor all belong to the scope of protection of the present application.

[0015] In addition, the terminal device according to the embodiment of the present disclosure may include, but is not limited to, a mobile phone, a personal digital assistant (PDA (registered trademark)), a wireless handheld device, a tablet computer, and other smart devices. The display device may include, but is not limited to, a personal computer, a television, and other devices having a display function.

[0016] Additionally, the term "and / or" in this specification describes only the relation between related objects and indicates that three types of relations can exist, for example, A and / or B can indicate three cases: only A exists, A and B exist simultaneously, or only B exists. It should be understood that the sign " / " generally indicates that the related objects before and after it are in an "or" relationship.

[0017] In the prior art, binding facial expressions to digital humans is mainly completed manually by professional designers, making the whole process time-consuming and resulting in a very low efficiency of facial expression binding. Based on this, the present disclosure provides a method for digital human facial expression transition, which can automatically transition the facial expressions of a reference model to a digital human object model, and effectively improve the efficiency of binding facial expressions to digital human images.

[0018] 1 is a schematic diagram according to a first embodiment of the present disclosure. As shown in FIG. 1, the present embodiment provides a facial expression transition method for a digital human, which specifically includes the following steps: S101, selecting an identifier of a target reference model that matches an object model from a preset reference model library, the reference model library including a plurality of reference models; The execution subject of the digital human facial expression transition method of this embodiment may be a digital human facial expression transition device, which may be an electronic entity or an application integrated by software, which can transition the facial expression of any reference model in the reference model library to any object model, thereby realizing automatic digital human facial expression transition.

[0019] The object model in this embodiment is a digital human model using facial expressions. Each pre-defined reference model in the reference model library is a pre-defined digital human model of various forms and styles, for example, may include a male model, a female model, or may further include a boy model, a middle-aged male model, an elderly male model, a girl model, a middle-aged female model, an elderly female model, etc. Specifically, various reference models may be pre-defined according to need, and are not limited here.

[0020] The identifier of the target reference model in this embodiment is any identifier that can identify the target reference model, and is not limited here.

[0021] S102, obtaining a facial expression library of the target reference model according to the identifier of the target reference model; In this embodiment, a facial expression library can be preset for various preset reference models, and each reference model's facial expression library can include multiple facial expressions. Specifically, various facial expressions can be stored in the form of point cloud data. For example, the natural state of the reference model is a state in which the digital human has no facial expression, and the natural state can be a standard state. After adding a facial expression to the digital human of the reference model, a position change is necessarily generated in the point cloud that executes the facial expression on the reference model, and no position change is generated in the point cloud that does not execute the facial expression. Therefore, only point cloud data that is changed when the facial expression is executed can be stored for the facial expression of the reference model.

[0022] S103, the final frame of the facial expression in the facial expression library of the target reference model is transferred to the object model to obtain the final frame of the facial expression of the object model.

[0023] In practical application scenarios, the range of many facial expressions is small, and the facial expression library only contains the point cloud data of the final frame of the corresponding facial expression, that is, the facial expression library only records the final state of the facial expression. At this time, when performing facial expression transition, the final frame of the facial expression in the facial expression library of the target reference model can be transitioned to the object model to obtain the final frame of the facial expression of the object model.

[0024] The facial expression transition of this embodiment is to realize automatic, accurate and efficient facial expression transition by transitioning the final frame of the facial expression of the target reference model, which matches the object model in the reference model library, to the object model.

[0025] By using the above technical solution, the digital human facial expression transition method of this embodiment can realize automatic transition of facial expressions in the facial expression library of a target reference model that matches an object model to the object model. Compared with the prior art, this does not require artificial binding by a professional designer, saving time and effort, and not only can effectively ensure the accuracy of the transitioned facial expressions, but also effectively improve the efficiency of digital human facial expression transition, and improve the generation efficiency of digital human images.

[0026] FIG. 2 is a schematic diagram according to a second embodiment of the present disclosure. Based on the technical solution of the embodiment shown in FIG. 1, the present embodiment further describes the technical solution of the present disclosure in more detail. As shown in FIG. 2, the facial expression transition method of a digital human in the present embodiment may specifically include the following steps: S201, obtaining a first feature curve of a preset region from an object model; S202, obtaining a second feature curve of a preset portion from each reference model; S203, calculating offset distances of preset locations between the object model and each of the reference models according to the first feature curve and each of the second feature curves; S204, selecting an identifier of a target reference model matching the object model from the reference model library according to an offset distance of a preset portion between the object model and each reference model; The above steps S201-S204 of this embodiment are a specific implementation of step S101 of the embodiment shown in FIG.

[0027] Since the distinction between different digital human images is mainly expressed in the face, the preset part in this embodiment can refer to one part of the face, such as the eyes, the mouth, etc. For example, the first feature curve may be at least one of six curves, including the upper eyelid, the lower eyelid, the upper eyelid contour, the lower eyelid contour, the inner eye corner divergence, and the outer eye corner divergence. Or the first feature curve may be at least one of multiple curves, such as the upper lip inner line, the upper lip outer line, and the bite line of the mouth. For digital human image models such as the object model and the reference model, the number and distribution of point clouds included in the same preset part are the same. That is, the same preset part has corresponding points between the two models.

[0028] In this embodiment, when performing steps S201-S204, only one preset region can be selected, or two or more preset regions can be selected, and the same preset region can only obtain one first feature curve from the object model and obtain one corresponding second feature curve from each reference model, or can obtain two or more first feature curves from the object model and obtain two or more corresponding second feature curves from each reference model.

[0029] When only one preset region is selected, and one first characteristic curve and one second characteristic curve participate in the above selection, the distribution and number of points included in the first characteristic curve and the second characteristic curve are the same. At this time, when specifically implementing step S203, the following steps can be included: (a1) obtaining coordinates of each point on a first characteristic curve and each second characteristic curve; The coordinates of each point on the first characteristic curve and each of the second characteristic curves can be obtained from the model data of the object model and each of the reference models, respectively.

[0030] (b1) calculating point distances of the same point identifiers on the first feature curve and each of the second feature curves based on coordinates of each point on the first feature curve and each of the second feature curves; The object model and the reference model may use the same point identifiers for points corresponding to the distribution. For example, a first feature curve of a preset region of the object model may include points identified as 1, 2, 3, 4, 5, 6, and correspondingly, a second feature curve corresponding to the first feature curve of the same preset region of the reference model may include points identified as 1, 2, 3, 4, 5, 6. Points with the same identifiers are points with the same distribution and corresponding positions. However, the coordinates of points with the same identifiers are not the same in two different models.

[0031] Specifically, based on the coordinates of two points with the same point identifier on the first characteristic curve and each of the second characteristic curves, the point distances of the same point identifier are calculated.

[0032] (c1) adding the point distances of each point identifier on the first feature curve and each second feature curve to obtain a sum of point distances; (d1) Obtaining offset distances of preset portions between the object model and each reference model based on the sum of the point distances and the number of points included in the first feature curve.

[0033] Specifically, the number of points included in the first characteristic curve is the same as the number of points included in each of the second characteristic curves.

[0034] For example, for a second feature curve of any reference model, add the point distances of each point identifier in the first feature curve and the corresponding second feature curve to obtain a sum of point distances, and then divide the sum of point distances by the number of points included in the first feature curve to obtain an average distance of each point, which is the offset distance of the preset portion between the object model and the reference model. Using this method, the offset distance of the preset portion between the object model and the reference model can be accurately calculated.

[0035] In this case, when step S204 is specifically performed, the identifier of the reference model having the smallest offset distance of the preset portion from among the multiple reference models in the reference model library can be selected as the identifier of the target reference model that matches the object model. By using this method, the target reference model that matches the object model can be obtained accurately and efficiently.

[0036] Optionally, in one embodiment of the present disclosure, for any reference model, in order to accurately calculate the offset distance of the preset portion between the object model and the reference model, it is necessary to perform point cloud registration on the first feature curve and the second feature curve before performing step (a1). If the distance between the first feature curve and the second feature curve is far without registration, even if the shape and distribution of the first feature curve and the second feature curve are completely the same, the calculated offset distance of the preset portion between the object model and the reference model will be large, and there will be a larger error. The point cloud registration of the first feature curve and the second feature curve in this embodiment also operates the second feature curve by at least one operation manner of displacement, rotation, and scaling, so that the first feature curve and the second feature curve have the most common characteristics, such as having the same centroid and having the most intersections. In the registration process, the operation can be tried many times to obtain the optimal registration position until the position at the time of the sum of the point distances between the first feature curve and the second feature curve is found, that is, the registration position of the first feature curve and the second feature curve.

[0037] In one embodiment of the present disclosure, when only one preset region is selected and two or more first feature curves and two or more second feature curves participate in the above-mentioned selection, for the first feature curve of any group and the corresponding second feature curve, the above-mentioned method can be used to calculate the sum of the point distances of each point identifier in the first feature curve and the second feature curve of the group. Then, the integrated value of the sum of the point distances of each point identifier in the first feature curve and the second feature curve of all groups can be obtained as the offset distance of the preset region between the object model and the corresponding reference model.

[0038] In one embodiment of the present disclosure, when two or more preset locations are selected to participate in the above selection, for each preset location, the offset distance corresponding to one of the preset locations can be calculated according to the above method. Then, further perform the following steps: (a2) calculating a total offset distance between the object model and each reference model according to a weight of each preset portion previously set and an offset distance corresponding to each preset portion; The weight of each preset part in this embodiment can be set according to the actual situation. For example, the characteristic expressions of the eyes and mouth on the face are prominent, so the weight of the eyes and mouth can be set high.

[0039] (b2) selecting an identifier of a target reference model that matches the object model from the reference model library based on a total offset distance between the object model and each reference model;

[0040] Specifically, the identifier of a reference model having the smallest total offset distance from among a plurality of reference models in a reference model library can be selected as the identifier of a target reference model matching the object model.

[0041] Using the above method, it is possible to select a target reference model that best matches an object model from a reference model library in a very efficient, accurate and reasonable manner.

[0042] Optionally, in one embodiment of the present disclosure, step S101 of the embodiment shown in FIG. 1 above can be further implemented using any of the following methods: A. selecting an identifier of a target reference model that matches the object model from the reference model library according to the attribute information of the object model and the attribute information of each reference model in the reference model library; or In practical applications, when establishing each digital human model, corresponding attribute information can be further set to describe or limit the model, for example, the attribute information can be middle-aged woman, child, elderly man, etc. At this time, based on the attribute information of the object model, an identifier of a reference model having the same attribute information can be selected from the model reference library as an identifier of a target reference model.

[0043] B. Displaying attribute information of each reference model in the reference model library to a user, and receiving an identifier of a target reference model matching an object model selected by the user.

[0044] In this method, the user manually selects the identifier of the target reference model that matches the object model.

[0045] In summary, using any of the above methods, the identifier of the target reference model that matches the object model can be obtained efficiently and accurately.

[0046] S205, obtaining a facial expression library of the target reference model according to the identifier of the target reference model; S206, perform global registration with respect to the target reference model and the object model; In this embodiment, the target reference model is adjusted to perform full row registration with the object model and the target reference model to align their head positions, and the coordinate system orientations of the target reference model and the object model are matched and their sizes are matched by using translation, rotation and / or scaling. This step is to roughly identify the positions and sizes corresponding to the five sensory organs of the model and facilitate the matching of key features in the next step.

[0047] S207, performing registration between key features corresponding to the facial expression of the target reference model and key features of the object model; In this embodiment, an expression is taken as an example related to only one key feature, and for example, the key feature may include the left eye, the right eye, or the mouth. In order to realize accurate expression transition, in the realization of the registration of the key feature in this embodiment, the key feature corresponding to the expression can be matched with the key point coordinates of the key feature of the object model by performing operations such as displacement, rotation, and / or scaling on the target reference model. For example, when registering the left eye, it can be expressed that the coordinates of the second corner of the left eye are the same. When registering the mouth, it can be expressed that the coordinates of the two corners of the mouth are the same.

[0048] By using this step, it is possible to achieve registration of any key feature of any facial expression.

[0049] S208, obtain a first size ratio of a facial expression according to the point cloud data of the key features corresponding to the final frame of the facial expression of the target reference model and the point cloud data of the key features in the natural state of the target reference model; S209, according to the first size ratio of the facial expression and the point cloud data of the key features in the natural state of the object model, obtain point cloud data of the key features corresponding to the final frame of the facial expression to which the object model is to transition, and obtain the final frame of the facial expression of the object model; In this embodiment, when the final frame of the facial expression is transitioned, there is a problem that the transition cannot be made directly. For example, when a normal mouth and lips are transitioned to a small cherry mouth, European lips, etc., the change in the lip width when smiling and the change in the elongation of the lower jaw when the mouth is opened cause a clear contradiction. In order to solve such a problem, in this embodiment, the length, width, and height of the point cloud of the key feature corresponding to the final frame of the facial expression can be calculated based on the point cloud data of the key feature corresponding to the final frame of the facial expression of the target reference model. Next, based on the point cloud of the key feature in the natural state of the target reference model, the length, width, and height of the point cloud of the key feature in the natural state are calculated, and the ratio of the length of the point cloud of the key feature corresponding to the final frame of the facial expression to the corresponding length of the point cloud of the key feature in the natural state, the ratio of the width of the point cloud of the key feature corresponding to the final frame of the facial expression to the corresponding width of the point cloud of the key feature in the natural state, and the ratio of the height of the point cloud of the key feature corresponding to the final frame of the facial expression to the corresponding height of the point cloud of the key feature in the natural state are calculated, respectively, to obtain the first size ratio of the facial expression.

[0050] For example, for an open-mouth expression, the coordinates of the highest point of the upper lip line and the lowest point of the lower lip line of the mouth can be obtained based on the point cloud data corresponding to the final frame of the open-mouth expression of the target reference model, and the width of the point cloud of the final frame of the open-mouth expression of the target reference model can be obtained; further, the point cloud data of the target reference model in its natural state can be obtained, and the coordinates of the highest point of the lower lip line and the lowest point of the lower lip line in the natural state can be obtained; then, the width of the point cloud of the final frame of the open-mouth expression of the target reference model can be divided by the width of the point cloud in the natural state to obtain the proportion of the width of the open-mouth expression. In a similar manner, the horizontal coordinate distance between the two corners of the mouth can be obtained as the length of the point cloud, and the proportion of the length of the corresponding open-mouth expression can be obtained as in the above-mentioned manner; the distance from the outermost point of the mouth and lips to the height of the innermost corner of the mouth is the height of the point cloud, and the proportion of the high of the corresponding open-mouth expression can be obtained as in the above-mentioned manner, that is, the proportion of the three coordinate axis directions in the first size proportion is obtained.

[0051] Next, based on the first size ratio of the facial expression and the point cloud data of the key features in the natural state of the object model, when the object model creates the facial expression, the point cloud data of the key features corresponding to the final frame, i.e., the coordinates corresponding to each point in the point cloud of the key features corresponding to the final frame, can be obtained, and different ratios of scaling can be performed in the three coordinate axis directions in the updated transition result, so that after transitioning to the object model, the point cloud of the key features corresponding to the final frame of the facial expression and the object model can be better matched.

[0052] Through experimental verification, after processing steps S208 and S209, the negative impact of volumetric deformation during facial expression transition is reduced, and the range of unreasonable facial expressions such as Cherry's small mouth expression, open mouth, smiling, and pouting lips is obviously suppressed.

[0053] Note that, unlike linearly transformed global scaling, this step scales the transformation of each point differently along the three axes, ensuring that excessive changes are suppressed and similarities are preserved.

[0054] The above steps S206-S209 are one specific implementation of step S103 in the embodiment shown in FIG.

[0055] S210, the point cloud data of the key features corresponding to the final frame of the facial expression to which the object model is to transition is transferred to the object model; S211, after transitioning to the object model, obtain a third feature curve corresponding to the final frame of the facial expression and a fourth feature curve corresponding to the final frame of the facial expression in the target reference model; S212, adjusting the third characteristic curve with the fourth characteristic curve as a constraint; For example, the third and fourth feature curves in this embodiment may be feature curves that appear in the state of creating facial expressions for the digital human model. For example, for a closed eye expression, the overlapping curves of the upper eyelid and the lower eyelid can be the third and fourth feature curves; TIFF2025068572000002.tif1741 For facial expressions, the curves that close the mouth and are convex forward can be the third and fourth feature curves; pursing the mouth For the TIFF2025068572000003.tif1740 expression, the third feature curve and the fourth feature curve can be the curve formed by closing the lips and rolling inward. When performing facial expression transition in different models, the third feature curve and the fourth feature curve are likely to show a large discrepancy. Based on this, in this embodiment, the fourth feature curve corresponding to the final frame of the facial expression in the target reference model can be used as a constraint, and after transition to the object model, the third feature curve corresponding to the final frame of the facial expression can be adjusted so that the facial expression can be more accurately integrated into the object model, and the facial expression can be more completely expressed in the object model. At the same time, the topology around the feature of the facial expression transitioned in the object model can be uniformly changed according to the facial expression, and the overall integration degree is very high.

[0056] Specifically, in this embodiment, the third characteristic curve is adjusted by using the fourth characteristic curve as a constraint, and the third characteristic curve and the fourth characteristic curve can also be called dynamic curves, and the adjustment method can also be called adjustment constrained based on the dynamic curve. The dynamic curve can be a mathematical relationship used in the digital human image technology line. When a set of points is connected in series to a space curve, the change of any point on the curve will cause the change of all other points on the curve. In the digital human model, such as the reference model and the object model surface, all points and their surrounding adjacent points are all connected by straight lines to form a grid structure. The third characteristic curve and the fourth characteristic curve themselves are also composed of edges of the grid structure. The grid structure includes horizontal edges and vertical edges, and the third characteristic curve and the fourth characteristic curve in this embodiment are usually horizontal edges. When adjusting the third characteristic curve by using the fourth characteristic curve as a constraint, the position of each point on the third characteristic curve is adjusted so that the third characteristic curve and the fourth characteristic curve are completely consistent, or the distance between the third characteristic curve and the fourth characteristic curve is smaller than a very small preset threshold.

[0057] FIG. 3 is a schematic diagram of the eye structure of an object model provided by the present disclosure. As shown in FIG. 3, the characteristic curve indicated by the arrow may be a third characteristic curve. Due to the edge and surface characteristics existing in the grid structure of the object model itself, the adjusted third characteristic curve cannot change the topological structure of the characteristic curve in the grid structure, so that the change of the third characteristic curve will necessarily be transmitted to adjacent surfaces and adjacent points in sequence. In the labeling of five sense organs, this transmission is attenuated outward from the outer characteristic curves of the five sense organs. That is, in order to realize a smoother adjustment, in actual applications, the change of the third characteristic curve can be attenuated outward by a preset number of curves. The preset number can be set according to actual needs, and may be, for example, 6, 8, 10 or other values. In order to match the corresponding fourth characteristic curve in the target reference model, the adjustment range of the third characteristic curve is the largest. With reference to the adjustment range of the third characteristic curve and each characteristic curve in the predetermined number of characteristic curves adjacent to the third characteristic curve, with the increase in the distance of the third characteristic curve, the adjustment range is gradually decreased until the adjacent predetermined number of characteristic curves complete the adjustment of the last step adjusted to the main wheel.

[0058] Based on the above method, the object model is now subject to the transition expression result, the third feature curve is restricted by the constraint of the dynamic curve, and the feature curve itself, the topology curve around the feature, successively approaches the fourth feature curve, so that the topology feature under this expression can be completely transitioned to the object model.

[0059] S213, smoothing the seam between the point cloud of the key features corresponding to the final frame of the facial expression after transition to the object model and the original point cloud of the object model; In order to make the point cloud of the key feature corresponding to the final frame of the transitioned expression more closely blended with the original point cloud of the object model, this embodiment further needs to detect whether the splicing of the two is smooth. For the position that is not smooth, a corresponding smoothing process needs to be performed. For example, the AB edge shown in FIG. 3 may be one of the seams of the original point cloud of the object model among the point cloud of the key feature corresponding to the final frame of the transitioned expression, and the AB edge is located in the point cloud of the key feature corresponding to the final frame of the transitioned expression. In particular, it can be detected whether the angle between the two adjacent edges in the horizontal direction of point B is greater than a preset threshold angle, and the preset threshold angle can be set according to the actual scene, for example, 15 degrees, 20 degrees, or 30 degrees, etc., and is not limited here. If so, the seam is deemed to be unsmooth, and the position of point B needs to be adjusted so that the angle between the two adjacent edges in the horizontal direction of point B is equal to or less than the preset threshold angle. Similarly, a similar method can also be used to detect the angle between two adjacent vertical edges of point B, and if not smooth, adjust the position of point B. In short, through the adjustment of the above method, all the seams of the key feature point cloud corresponding to the final frame of the facial expression and the original point cloud of the object model are very smooth, so that the key feature point cloud corresponding to the final frame of the facial expression can be perfectly merged with the original point cloud of the object model.

[0060] For example, when the final frame transition of facial expression is completed, there will be a slight problem in the change of the point cloud of the final frame of the transitioned facial expression and the original point cloud seam, for example, in the places where multiple five sensory organ parts intersect, such as both sides of the nose wings and the middle part of the mountain root, the wiring may not be smooth in some cases. Using the smooth processing method of this embodiment, this type of problem can be fully solved and the smoothness of the line can be ensured.

[0061] Steps S210-S213 are the complement of the above-mentioned steps S206-S209, which can further improve the transition effect of the final facial expression frame, and improve the accuracy and efficiency of the facial expression transition.

[0062] S214, according to the point cloud data of the key features corresponding to the intermediate frame of the facial expression in the facial expression library of the target reference model, the point cloud data of the key features in the natural state of the target reference model, and the point cloud data of the key features corresponding to the final frame of the facial expression of the target reference model, obtain a second size ratio of the intermediate frame of the facial expression to the final frame; Specifically, based on the point cloud data of the key features corresponding to the intermediate frames of the facial expressions in the facial expression library of the target reference model, the point cloud data of the key features in the natural state of the target reference model, and the point cloud data of the key features corresponding to the final frame of the facial expression of the target reference model, a fourth feature curve corresponding to the final frame of the facial expression of the target reference model, a fifth feature curve corresponding to the intermediate frames of the facial expression, and a sixth feature curve in the natural state are obtained. The fourth feature curve, the fifth feature curve, and the sixth feature curve are all feature curves of the key features in the target reference model.

[0063] Similarly, for an open-mouth expression, the fourth feature curve can correspond to the upper lip line corresponding to the maximum opening of the mouth, the fifth feature curve can be the upper lip line of an intermediate process in the process of opening the mouth, and the sixth feature curve can be the closing line when the upper and lower lips are closed when the mouth is not opened. In specific calculation, the highest point on the upper lip line is taken as the key point, and the distance difference between the key point on the fifth feature curve and the key point on the upper lip line on the fourth feature curve is divided by the distance difference between the key point on the fourth feature curve and the key point on the upper lip line on the sixth feature curve to obtain the ratio of the height of the intermediate frame of the open-mouth expression to the width of the final frame as the ratio of the width in the second size ratio. Similarly, the length and height determination method in the first size ratio can be combined in a similar manner to obtain the length and height ratio in the second size ratio.

[0064] S215, obtaining an intermediate frame of the facial expression of the object model according to a second size ratio of the intermediate frame of the facial expression to the final frame and the final frame of the facial expression of the object model.

[0065] When it is realized concretely, (a3) obtaining a corresponding reference feature curve of the intermediate frame of the facial expression of the object model according to a second size ratio of the intermediate frame of the facial expression to the final frame and a corresponding third feature curve of the final frame of the facial expression of the object model; (b3) transferring point cloud data of key features corresponding to intermediate frames of facial expressions of the target reference model to the object model; (c3) obtaining a seventh feature curve corresponding to the intermediate frame of the facial expression after transitioning to the object model; (d3) adjusting the seventh feature curve using the reference feature curve as a constraint.

[0066] In this embodiment, the reference feature curve is the reference feature curve that should theoretically correspond after the mid-frame facial expression transition, but the seventh feature curve is the actual reference feature curve after the mid-frame facial expression transition, so that the dynamic curve adjustment needs to be performed using the method of step (d3) in order to improve the mid-frame transition effect.

[0067] Specifically, the implementation of step (d3) is the same as the implementation principle of step S212 in the above embodiment, and can realize the adjustment of the characteristic curve of the outer attenuation type, the details of which can be referred to the relevant description of the above embodiment, and will not be described in detail here. In addition, the method of step S213 described above can also be used to perform smooth processing on the transitioned intermediate frame, and will not be described in detail here.

[0068] Steps S214 and S215 also need to perform intermediate frame transitions if the expressions in the expression library include intermediate frames.

[0069] In conventional facial expression binding, there are only two data, the natural state and the facial expression frame, and the change in the width value of the facial expression is the insertion value between the natural state and the target frame. Such a method cannot express the details of the facial expression change process of a real person, such as the appearance and disappearance of the pear swirl when a person is smiling.

[0070] From the viewpoint of facial expression transition angle, the intermediate frame, the initial frame and the final frame of a facial expression have a certain non-linear relationship.

[0071] In this embodiment, by sampling the intermediate frames and final frames of the feature curve, a change constraint for each intermediate frame with respect to the final frame is established. When transitioning between intermediate frames, not only is it necessary to maintain the intermediate frame feature curve, but it is also necessary to add a change constraint for the final frame to this curve, so as to effectively improve the transition efficiency of the intermediate frames.

[0072] In this embodiment, the transitioned intermediate frame adds intermediate expressions of intermediate frames with some non-linear changes between the natural state and the final frame. In this way, the change from 0 to 1 of one expression evolves from a linear change to a non-linear change, and detailed changes such as the pear swirl appearing first and then disappearing, the lip tragus protruding first and then straightening out are all realized, making the transitioned expression more natural, reasonable and efficient.

[0073] By using the above steps, all the expressions in the target reference model can be transferred to the object model one by one by constantly repeating the above transition steps, and the expressions of the transferred object model can also reach a good level based on the above set of constraints.

[0074] The facial expression transition method of the digital human in this embodiment can quickly and efficiently transition facial expressions in the facial expression library of the target reference model to the object model through the above-mentioned method. In addition, the facial expression transition process speed is fast and the computational power is low, and it can be performed in the background in real time during the image production process, and can complete automatic binding at the minute level, and can also directly perform capture drive to preview the effect. It does not depend on the support of a large-scale computing device, does not need to wait for a long time for generation, and the facial expression transition is very efficient and accurate.

[0075] Furthermore, for scenes where it is difficult to fully achieve the effect using other means such as closing eyes, pouting, or pursing the lips, the digital human facial expression transition method of this embodiment can be used to fully transition facial expressions and effectively achieve the effect of a facial expression transition.

[0076] In addition, the facial expression transition method of the digital human of the present embodiment supports 4D nonlinear transition, and the facial expression transition means of such nonlinear insertion value of the intermediate frame can also be supported, and the effect of 4D nonlinear facial expression after transition can be more effectively ensured under the constraint role.

[0077] 4 is a schematic diagram according to a third embodiment of the present disclosure. As shown in FIG. 4, this embodiment provides a digital human facial expression transition device 400, which includes a selection module 401, an acquisition module 402, and a final frame transition module 403; The selection module 401 is used for selecting an identifier of a target reference model that matches an object model from a preset reference model library, the reference model library including a plurality of reference models; The acquisition module 402 is used to acquire a facial expression library of the target reference model according to the identifier of the target reference model; A final frame transition module 403 is used for transitioning the final frame of an expression in the expression library of the target reference model to the object model to obtain the final frame of an expression of the object model.

[0078] The digital human facial expression transition device 400 of this embodiment realizes the principle and technical effect of transitioning the facial expression of a digital human by using the above-mentioned modules, which is the same as that of the embodiment of the related method described above. For details, please refer to the description of the embodiment of the related method described above, and a detailed description will not be given here.

[0079] 5 is a schematic diagram according to a fourth embodiment of the present disclosure. The digital human facial expression transition device 500 of this embodiment is based on the technical solution of the embodiment shown in the above-mentioned FIG. 4, and the technical solution of the present disclosure will be further described in more detail. As shown in FIG. 5, the digital human facial expression transition device 500 of this embodiment includes modules with the same names and functions as those in the above-mentioned FIG. 4: a selection module 501, an acquisition module 502, and a final frame transition module 503.

[0080] In this embodiment, the selection module 501 includes: It is used to select an identifier of the target reference model matching the object model from the reference model library based on the attribute information of the object model and the attribute information of each of the reference models in the reference model library.

[0081] Alternatively, in one embodiment of the present disclosure, the sorting module 501 may: Displaying attribute information of each of the reference models in the reference model library to a user; It is used to receive the target reference model that matches the object model selected by the user.

[0082] Alternatively, in one embodiment of the present disclosure, the sorting module 501 may: Obtaining a first characteristic curve of a preset portion from the object model; Obtaining a second characteristic curve of the preset region from each of the reference models; calculating an offset distance of the preset portion between the object model and each of the reference models based on the first characteristic curve and each of the second characteristic curves; The offset distance of the preset portion between the object model and each of the reference models is used to select an identifier of a target reference model that matches the object model from the reference model library.

[0083] Further, optionally, in one embodiment of the present disclosure, the selection module 501 obtains coordinates of each point on the first characteristic curve and each of the second characteristic curves, respectively; Calculating point distances of the same point identifiers on the first feature curve and each of the second feature curves based on coordinates of each point on the first feature curve and each of the second feature curves; adding point distances of each point identifier on the first characteristic curve and each of the second characteristic curves to obtain a sum of point distances; is used to obtain an offset distance of the preset portion between the object model and each of the reference models based on the sum of the point distances and the number of points included in the first feature curve.

[0084] Further, optionally, in one embodiment of the present disclosure, the sorting module 501 may further include: The identifier of the reference model having the smallest offset distance of the preset portion from among a plurality of reference models in the reference model library is used to select the identifier of the target reference model matching the object model.

[0085] Further, optionally, in one embodiment of the present disclosure, when the preset portion includes two or more preset portions, the selection module 501 calculates a total offset distance between the object model and each of the reference models based on a weight of each of the preset portions and an offset distance corresponding to each of the preset portions; The total offset distance between the object model and each of the reference models is used to select an identifier of a target reference model that matches the object model from the reference model library.

[0086] Further, optionally, in one embodiment of the present disclosure, the last frame transition module 503 may include: performing global registration of the target reference model and the object model; performing registration of key features corresponding to the facial expression of the target reference model with key features of the object model; Obtain a first size ratio of a facial expression based on point cloud data of key features corresponding to the final frame of the facial expression of the target reference model and point cloud data of key features of the target reference model in a natural state; Based on the first size ratio of the facial expression and the point cloud data of the key features in the natural state of the object model, point cloud data of the key features corresponding to the final frame of the facial expression to which the object model is to transition is obtained, and is used to obtain the final frame of the facial expression of the object model.

[0087] Further, optionally, in one embodiment of the present disclosure, the last frame transition module 503 may include: Transmitting point cloud data of key features corresponding to the final frame of the facial expression to which the object model is to transition to the object model, After transitioning to the object model, obtain a third feature curve corresponding to a final frame of the facial expression and a fourth feature curve corresponding to a final frame of the facial expression in the target reference model; adjusting the third characteristic curve with the fourth characteristic curve as a constraint; It is used to perform smooth processing on the seams between the points of the key features corresponding to the final frame of the facial expression after transition to the object model and the original points of the object model.

[0088] Optionally, as shown in FIG. 5, in one embodiment of the present disclosure, the digital human facial expression transition device 500 includes: If the expression library further includes an intermediate frame of the expression, the method further includes an intermediate frame transition module 504 for transitioning the intermediate frame of the expression in the expression library of the target reference model to the object model to obtain an intermediate frame of the expression of the object model.

[0089] Further, optionally, in one embodiment of the present disclosure, the intermediate frame transition module 504 may further include: Obtain a second size ratio of the intermediate frame of the facial expression to the final frame based on point cloud data of key features corresponding to the intermediate frame of the facial expression in the facial expression library of the target reference model, point cloud data of key features in the natural state of the target reference model, and point cloud data of key features corresponding to the final frame of the facial expression of the target reference model; A facial intermediate frame is used to obtain a facial intermediate frame of the object model based on a second size ratio of the facial intermediate frame to the final frame and the facial final frame of the object model.

[0090] Further, optionally, in one embodiment of the present disclosure, the intermediate frame transition module 504 may further include: obtain a fourth feature curve corresponding to the final frame of the facial expression of the target reference model, a fifth feature curve corresponding to the intermediate frame of the facial expression, and a sixth feature curve in the natural state based on point cloud data of key features corresponding to the intermediate frame of the facial expression in the facial expression library of the target reference model, point cloud data of key features in the natural state of the target reference model, and point cloud data of key features corresponding to the final frame of the facial expression of the target reference model; A second size ratio of the intermediate frame of the facial expression to the final frame is obtained based on coordinates of key points on the fourth feature curve, the fifth feature curve, and the sixth feature curve.

[0091] Further, optionally, in one embodiment of the present disclosure, the intermediate frame transition module 504 may further include: Obtain a corresponding reference feature curve of the intermediate frame of the facial expression of the object model according to a second size ratio of the intermediate frame of the facial expression to the final frame and a corresponding third feature curve of the final frame of the facial expression of the object model; Transferring point cloud data of key features corresponding to the intermediate frames of the facial expression of the target reference model to the object model; After transitioning to the object model, obtain a seventh feature curve corresponding to the intermediate frame of the facial expression; The reference characteristic curve is used as a constraint to adjust the seventh characteristic curve.

[0092] The digital human facial expression transition device 500 of this embodiment realizes the principle and technical effect of transitioning facial expressions of a digital human by using the above-mentioned modules, which is the same as that of the embodiment of the related method described above. For details, please refer to the description of the embodiment of the related method described above, and a detailed description will not be given here.

[0093] In the technical solution disclosed herein, the acquisition, storage, application, etc. of relevant user personal information shall conform to the provisions of relevant laws and regulations and shall not violate public order and morals.

[0094] According to an embodiment of the present disclosure, the present disclosure further provides an electronic device, a readable storage medium, and a computer program product.

[0095] As shown in FIG. 6, a block diagram of an electronic device 600 for implementing an embodiment of the present disclosure is shown. The electronic device is intended to represent various types of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various types of mobile devices, such as personal digital processing, mobile phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure as described herein and / or claimed.

[0096] 6, the device 600 includes a computing unit 601, which can perform various appropriate operations and processes based on a computer program stored in a read-only memory (ROM) 602 or loaded from a storage unit 608 into a random access memory (RAM) 603. The RAM 603 can also store various programs and data required for the device 600 to operate. The computing unit 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0097] Several components in device 600 are connected to an I / O interface 605, including input units 606 such as a keyboard, mouse, etc., output units 607 such as various types of displays, speakers, etc., storage units 608 such as a disk, optical disk, etc., and communication units 609 such as a network card, modem, wireless communication transceiver, etc. The communication units 609 enable device 600 to exchange information / data with other devices via a computer network, such as the Internet, and / or various telecommunication networks.

[0098] The computing unit 601 is a general-purpose and / or dedicated processing component with various processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, computing units that execute various machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 601 executes various methods and processes described above, such as the above-mentioned methods of the present disclosure. For example, in some embodiments, the above-mentioned methods of the present disclosure can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 608. In some embodiments, some or all of the computer program is loaded and / or installed in the device 600 via the ROM 602 and / or the communication unit 609. When the computer program is loaded into the RAM 603 and executed by the computing unit 601, one or more steps of the above-mentioned methods can be executed. Alternatively, in other embodiments, the computing unit 601 can be configured to execute the above-mentioned methods of the present disclosure via any other suitable manner (e.g., by firmware).

[0099] Various implementations of the systems and techniques described herein may be realized in digital electronic circuitry systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system on a chip (SOCs), programming programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include being implemented in one or more computer programs that may be executed and / or interpreted by a programmable system that includes at least one programmable processor, which may be an application specific or general purpose programmable processor, and that may receive data and instructions from and transmit data and instructions to a storage system, at least one input device, and at least one output device.

[0100] The program codes for implementing the methods of the present disclosure can be written using any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general purpose computer, a special purpose computer, or other programmable data processing apparatus such that the functions / operations specified in the flowcharts and / or block diagrams are implemented when the program codes are executed by the processor or controller. The program codes can be executed entirely on the machine, partially on the machine, partially on the machine as a separate software package and partially on a remote machine, or entirely on a remote machine or server.

[0101] In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain or store a program for use with or in combination with an instruction execution system, device, or apparatus. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium includes, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any suitable combination of the above. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above.

[0102] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user, and a keyboard and pointing device (e.g., a mouse or trackball) by which a user can provide input to the computer. Other types of devices can also be used to provide interaction with a user, for example, feedback provided to the user can be any form of sensing feedback (e.g., visual feedback, auditory feedback, or haptic feedback) and can receive input from the user in any form (including acoustic input, speech input, and haptic input).

[0103] The systems and techniques described herein may be implemented in a computing system including a back-end component (e.g., a data server), or a middleware component (e.g., an application server), or a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user interacts with the implementation of the systems and techniques described herein), or any combination of such back-end, middleware, and front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), the Internet, and a blockchain network.

[0104] The computer system may include a client and a server. The client and the server are generally remote from each other and typically interact through a communication network. The relationship between the client and the server is created by computer programs running on corresponding computers and having a client-server relationship to each other. The server may be a cloud server, a server of a distributed system, or a server for combining blockchains.

[0105] It should be understood that steps can be rearranged, added, or deleted using various types of flows shown above. For example, each step described in the present disclosure can be performed in parallel, sequentially, or in a different order, but is not limited herein as long as the technical solution disclosed in the present disclosure can achieve the desired results.

[0106] The above specific implementation methods do not constitute a limitation of the protection scope of the present disclosure. Those skilled in the art can make various modifications, combinations, subcombinations, and substitutions based on design requirements and other factors. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present disclosure should be included in the protection scope of the present disclosure.

Claims

1. A facial expression transition method for a digital human, comprising: selecting an identifier of a target reference model matching the object model from a predefined reference model library, the reference model library including a plurality of reference models; obtaining a facial expression library of the target reference model based on an identifier of the target reference model; and transferring a final frame of an expression in the expression library of the target reference model to the object model to obtain a final frame of an expression of the object model. A method for facial expression transition in digital humans.

2. The step of selecting an identifier of a target reference model matching the object model from a preset reference model library includes: selecting an identifier of the target reference model that matches the object model from the reference model library based on attribute information of the object model and attribute information of each of the reference models in the reference model library; The facial expression transition method for a digital human according to claim 1 .

3. The step of selecting an identifier of a target reference model matching the object model from a preset reference model library includes: displaying attribute information of each of the reference models in the reference model library to a user; receiving the target reference model that matches the object model selected by the user; The facial expression transition method for a digital human according to claim 1 .

4. The step of selecting an identifier of a target reference model matching the object model from a preset reference model library includes: obtaining a first characteristic curve of a preset region from the object model; Obtaining a second characteristic curve of the preset region from each of the reference models; calculating an offset distance of the preset portion between the object model and each of the reference models based on the first characteristic curve and each of the second characteristic curves; and selecting an identifier of a target reference model that matches the object model from the reference model library based on an offset distance of the preset portion between the object model and each of the reference models. The facial expression transition method for a digital human according to claim 1 .

5. The step of calculating an offset distance of the preset portion between the object model and each of the reference models based on the first feature curve and the second feature curve includes: obtaining coordinates of each point on the first characteristic curve and each of the second characteristic curves; calculating a point distance between the same point identifiers on the first feature curve and each of the second feature curves based on coordinates of each point on the first feature curve and each of the second feature curves; adding point distances of each point identifier on the first characteristic curve and each of the second characteristic curves to obtain a sum of point distances; and obtaining an offset distance of the preset portion between the object model and each of the reference models based on the sum of the point distances and the number of points included in the first feature curve. The facial expression transition method for a digital human according to claim 4 .

6. selecting an identifier of a target reference model matching the object model from the reference model library based on an offset distance of the preset portion between the object model and each of the reference models, selecting an identifier of a reference model having the smallest offset distance of the preset portion from a plurality of reference models in the reference model library as an identifier of the target reference model matching the object model; The facial expression transition method for a digital human according to claim 5 .

7. selecting an identifier of a target reference model matching the object model from the reference model library based on an offset distance of the preset portion between the object model and each of the reference models, When the preset portion includes two or more preset portions, calculating a total offset distance between the object model and each of the reference models based on a weight of each of the preset portions and an offset distance corresponding to each of the preset portions; selecting an identifier of a target reference model that matches the object model from the reference model library based on a total offset distance between the object model and each of the reference models; The facial expression transition method for a digital human according to claim 5 .

8. The step of transferring a final frame of an expression in the expression library of the target reference model to the object model to obtain a final frame of an expression of the object model includes: performing global registration of the target reference model and the object model; performing registration of key features corresponding to facial expressions of the target reference model with key features of the object model; obtaining a first size ratio of a facial expression based on point cloud data of key features corresponding to the final frame of the facial expression of the target reference model and point cloud data of key features in a natural state of the target reference model; acquiring point cloud data of key features corresponding to a final frame of the facial expression to which the object model is to transition based on the first size ratio of the facial expression and point cloud data of key features in the natural state of the object model, and acquiring a final frame of the facial expression of the object model. The facial expression transition method for a digital human according to any one of claims 1 to 7.

9. After acquiring point cloud data of key features corresponding to a final frame of the facial expression to which the object model is to transition based on the size ratio of the facial expression and point cloud data of key features in a natural state of the object model, and before acquiring a final frame of the facial expression of the object model, the method includes: transferring point cloud data of key features corresponding to a final frame of the facial expression to which the object model is to be transferred to the object model; After transitioning to the object model, obtaining a third feature curve corresponding to a final frame of the facial expression and a fourth feature curve corresponding to a final frame of the facial expression in the target reference model; adjusting the third characteristic curve using the fourth characteristic curve as a constraint; and performing a smoothing process on a seam between a point cloud of a key feature corresponding to a final frame of the facial expression after transition to the object model and an original point cloud of the object model. The facial expression transition method for a digital human according to claim 8.

10. After the step of transferring a final frame of an expression in the expression library of the target reference model to the object model to obtain a final frame of an expression of the object model, the method further comprises: If the expression library further includes an intermediate frame of the expression, the intermediate frame of the expression in the expression library of the target reference model is transferred to the object model to obtain an intermediate frame of the expression of the object model. The facial expression transition method for a digital human according to any one of claims 1 to 7.

11. The step of transferring the intermediate facial expression frames in the facial expression library of the target reference model to the object model to obtain the intermediate facial expression frames of the object model includes: Obtaining a second size ratio of the intermediate frame of the facial expression to the final frame based on point cloud data of key features corresponding to the intermediate frame of the facial expression in the facial expression library of the target reference model, point cloud data of key features in the natural state of the target reference model, and point cloud data of key features corresponding to the final frame of the facial expression of the target reference model; obtaining an intermediate facial expression frame of the object model based on a second size ratio of the intermediate facial expression frame to the final facial expression frame and the final facial expression frame of the object model; The facial expression transition method for a digital human according to claim 10.

12. The step of obtaining a second size ratio of the intermediate frame of the facial expression to the final frame based on point cloud data of key features corresponding to the intermediate frame of the facial expression in the facial expression library of the target reference model, point cloud data of key features in the natural state of the target reference model, and point cloud data of key features corresponding to the final frame of the facial expression of the target reference model, includes: obtaining a fourth feature curve corresponding to the final frame of the facial expression of the target reference model, a fifth feature curve corresponding to the intermediate frame of the facial expression, and a sixth feature curve in the natural state based on point cloud data of key features corresponding to the intermediate frame of the facial expression in the facial expression library of the target reference model, point cloud data of key features in the natural state of the target reference model, and point cloud data of key features corresponding to the final frame of the facial expression of the target reference model; and obtaining a second size ratio of the intermediate frame of the facial expression to a final frame based on coordinates of key points on the fourth feature curve, the fifth feature curve, and the sixth feature curve. The method for generating facial expression transitions for a digital human according to claim 11.

13. The step of obtaining an intermediate facial expression frame of the object model based on a second size ratio of the intermediate facial expression frame to a final frame and the final facial expression frame of the object model includes: obtaining a corresponding reference feature curve of the intermediate frame of the facial expression of the object model according to a second size ratio of the intermediate frame of the facial expression to the final frame and a corresponding third feature curve of the final frame of the facial expression of the object model; transferring point cloud data of key features corresponding to the intermediate frames of the facial expression of the target reference model to the object model; After transitioning to the object model, obtaining a seventh feature curve corresponding to the intermediate frame of the facial expression; adjusting the seventh characteristic curve using the reference characteristic curve as a constraint. The method for generating facial expression transitions for a digital human according to claim 11.

14. A facial expression transition device for a digital human, comprising: a selection module for selecting an identifier of a target reference model matching an object model from a predefined reference model library, the reference model library including a plurality of reference models; an acquisition module for acquiring a facial expression library of the target reference model based on an identifier of the target reference model; a final frame transition module for transitioning a final frame of an expression in the expression library of the target reference model to the object model to obtain a final frame of the expression of the object model. Digital human facial expression transition device.

15. The sorting module comprises: is used to select an identifier of the target reference model matching the object model from the reference model library based on attribute information of the object model and attribute information of each of the reference models in the reference model library; The facial expression transition device for a digital human according to claim 14.

16. The sorting module comprises: Displaying attribute information of each of the reference models in the reference model library to a user; used to receive the target reference model that matches the object model selected by the user. The facial expression transition device for a digital human according to claim 14.

17. The sorting module comprises: Obtaining a first characteristic curve of a preset portion from the object model; Obtaining a second characteristic curve of the preset region from each of the reference models; calculating an offset distance of the preset portion between the object model and each of the reference models based on the first characteristic curve and each of the second characteristic curves; and selecting an identifier of a target reference model matching the object model from the reference model library based on an offset distance of the preset portion between the object model and each of the reference models. The facial expression transition device for a digital human according to claim 14.

18. The sorting module comprises: Obtaining coordinates of each point on the first characteristic curve and each of the second characteristic curves; Calculating point distances of the same point identifiers on the first feature curve and each of the second feature curves based on coordinates of each point on the first feature curve and each of the second feature curves; adding point distances of each point identifier on the first characteristic curve and each of the second characteristic curves to obtain a sum of point distances; used to obtain an offset distance of the preset portion between the object model and each of the reference models based on the sum of the point distances and the number of points included in the first feature curve. The facial expression transition device for a digital human according to claim 17.

19. The sorting module comprises: is used to select an identifier of a reference model having the smallest offset distance of the preset portion from a plurality of reference models in the reference model library as an identifier of the target reference model matching the object model. The facial expression transition device for a digital human according to claim 18.

20. The sorting module comprises: When the preset portion includes two or more preset portions, calculate a total offset distance between the object model and each of the reference models based on a weight of each of the preset portions and an offset distance corresponding to each of the preset portions, the total offset distance being calculated based on the weight of each of the preset portions and the offset distance corresponding to each of the preset portions; used to select an identifier of a target reference model matching the object model from the reference model library based on a total offset distance between the object model and each of the reference models. The facial expression transition device for a digital human according to claim 18.

21. The final frame transition module: performing global registration of the target reference model and the object model; performing registration of key features corresponding to the facial expression of the target reference model with key features of the object model; Obtain a first size ratio of a facial expression based on point cloud data of key features corresponding to the final frame of the facial expression of the target reference model and point cloud data of key features of the target reference model in a natural state; obtaining point cloud data of key features corresponding to a final frame of the facial expression to which the object model is to transition based on the first size ratio of the facial expression and point cloud data of key features in the natural state of the object model, and used to obtain a final frame of the facial expression of the object model; The facial expression transition device for a digital human according to any one of claims 14 to 20.

22. The final frame transition module further comprises: Transmitting point cloud data of key features corresponding to the final frame of the facial expression to which the object model is to transition to the object model, After transitioning to the object model, obtain a third feature curve corresponding to a final frame of the facial expression and a fourth feature curve corresponding to a final frame of the facial expression in the target reference model; adjusting the third characteristic curve using the fourth characteristic curve as a constraint; used to perform smooth processing on a seam between a point cloud of key features corresponding to the final frame of the facial expression after transition to the object model and an original point cloud of the object model; The facial expression transition device for a digital human according to claim 21.

23. The apparatus comprises: and an intermediate frame transition module for transitioning the intermediate frame of the facial expression in the facial expression library of the target reference model to the object model when the facial expression library further includes an intermediate frame of the facial expression. The facial expression transition device for a digital human according to any one of claims 14 to 20.

24. The mid-frame transition module includes: Obtain a second size ratio of the intermediate frame of the facial expression to the final frame based on point cloud data of key features corresponding to the intermediate frame of the facial expression in the facial expression library of the target reference model, point cloud data of key features in the natural state of the target reference model, and point cloud data of key features corresponding to the final frame of the facial expression of the target reference model; used to obtain an intermediate facial frame of the object model based on a second size ratio of the intermediate facial frame to the final facial frame and the final facial frame of the object model. The facial expression transition device for a digital human according to claim 23.

25. The mid-frame transition module includes: obtain a fourth feature curve corresponding to the final frame of the facial expression of the target reference model, a fifth feature curve corresponding to the intermediate frame of the facial expression, and a sixth feature curve in the natural state based on point cloud data of key features corresponding to the intermediate frame of the facial expression in the facial expression library of the target reference model, point cloud data of key features in the natural state of the target reference model, and point cloud data of key features corresponding to the final frame of the facial expression of the target reference model; and obtaining a second size ratio of the intermediate frame of the facial expression to the final frame according to coordinates of key points on the fourth feature curve, the fifth feature curve, and the sixth feature curve. The facial expression transition device for a digital human according to claim 24.

26. The mid-frame transition module includes: Obtaining a corresponding reference feature curve of the intermediate frame of the facial expression of the object model according to a second size ratio of the intermediate frame of the facial expression to the final frame and a corresponding third feature curve of the final frame of the facial expression of the object model; Transferring point cloud data of key features corresponding to the intermediate frames of the facial expression of the target reference model to the object model; After transitioning to the object model, obtain a seventh feature curve corresponding to the intermediate frame of the facial expression; the reference characteristic curve is used as a constraint to adjust the seventh characteristic curve; The facial expression transition device for a digital human according to claim 24.

27. An electronic device, At least one processor; a memory in communication with the at least one processor; The memory stores instructions executable by the at least one processor, the instructions, when executed by the at least one processor, causing the at least one processor to perform a method according to any one of claims 1 to 7. electronic equipment.

28. A non-transitory computer-readable storage medium having computer instructions stored thereon, comprising: The computer instructions cause a computer to carry out a method according to any one of claims 1 to 7. A non-transitory computer-readable storage medium.

29. A computer program comprising: The computer program, when executed by a processor, implements the method according to any one of claims 1 to 7. Computer program.

Images