Virtual object expression generation method, program product, device and storage medium

By obtaining and binding the connection relationship between the components of the neutral expression base template and the neutral expression base and the facial area, the components are automatically adjusted, and the time-consuming problem of component adjustment in the prior art is solved, and the generation efficiency of the virtual object expression base is improved.

CN113223127BActive Publication Date: 2024-12-03GUANGZHOU HUYA TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202110560384.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-05-21
Publication Date
2024-12-03
Estimated Expiration
2041-05-21

AI Technical Summary

Technical Problem

In the prior art, when generating expressions of virtual objects, facial deformation methods usually only consider facial areas and ignore components such as eyes, teeth, hair, etc., resulting in the manual adjustment of components taking a long time and becoming a bottleneck in expression-based production.

Method used

By obtaining the connection relationship between the facial area and component of the neutral expression base template and the neutral expression base, the component and facial area are automatically bound, and the expression base with components is generated to achieve automatic adjustment of the component following the facial area.

Benefits of technology

The component adjustment process is automated, the expression-based production efficiency is improved, the time for manual adjustment of the artist is reduced, and the defect of component production time accounts for 40% to 80%.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113223127B_ABST
    Figure CN113223127B_ABST
Patent Text Reader

Abstract

The present application provides a method for generating expressions of a virtual object, a computer program product, an electronic device, and a computer-readable storage medium, the method comprising: obtaining a standard template group including a plurality of expression base templates with different expressions and a neutral expression base template, a neutral expression base of a virtual object whose expression is to be generated, and a connection relationship between a facial region and a component of the neutral expression base template, wherein the neutral expression base template and the neutral expression base have components, and the components are non-skin parts of a facial region; based on the connection relationship, binding the neutral expression base template and respective components on the neutral expression base to respective facial regions; generating a first expression with components according to the neutral expression base with components bound to the facial region and the standard template group including the neutral expression base template with components bound to the facial region.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method for generating an expression of a virtual object, a computer program product, an electronic device, and a computer-readable storage medium. Background Art

[0002] Facial deformation (Blendshape) is a commonly used facial expression animation synthesis method. It obtains new expressions by linearly fusing a set of pre-defined expression bases that can cover most expression details. It has been widely used in 3D animation, film production and other fields.

[0003] However, facial deformation usually only considers the facial area, and does not consider components such as eyes, teeth, and hair. In related technologies, after obtaining a new expression base, artists are required to manually place these components into the new expression base, and carefully adjust them to make the fused expression more natural. In the production of the entire human head expression base, the production time of components accounts for 40% to 80%. Therefore, the production of components has become a bottleneck that limits the speed of expression base production. Summary of the invention

[0004] In order to overcome the problem in the related art that the production of components becomes a bottleneck limiting the speed of expression base production, the present application provides a method for generating expressions of virtual objects, a computer program product, an electronic device and a computer-readable storage medium.

[0005] According to a first aspect of an embodiment of the present application, a method for generating expressions of a virtual object is provided, the method comprising: obtaining a standard template group including a plurality of expression base templates with different expressions and a neutral expression base template, a neutral expression base of a virtual object whose expression is to be generated, and a connection relationship between a facial area and components of the neutral expression base template, the neutral expression base template and the neutral expression base having components, and the components being non-skin parts of a facial area; based on the connection relationship, binding the neutral expression base template and respective components on the neutral expression base to respective facial areas; generating a first expression with components according to the neutral expression base in which the components are bound to the facial area and the standard template group including the neutral expression base template in which the components are bound to the facial area.

[0006] According to a second aspect of an embodiment of the present application, a computer program product is provided, including a computer program / instruction, and when the computer program / instruction is executed by a processor, the method described in the first aspect of the embodiment of the present application is implemented.

[0007] According to a third aspect of an embodiment of the present application, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in the first aspect of the embodiment of the present application when executing the program.

[0008] According to a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the method described in the first aspect of the embodiment of the present application is implemented.

[0009] The technical solution provided by the embodiments of the present application may have the following beneficial effects:

[0010] In the expression generation method of the virtual object provided in the embodiment of the present application, since the connection relationship between the facial area and the component of the neutral expression base template in the standard template group is obtained, based on the connection relationship, the component on the neutral expression base template can be bound to the facial area, and the component on the neutral expression base of the virtual object whose expression is to be generated can be bound to the facial area, thereby realizing the association of the component with the facial area on the neutral expression base template and the neutral expression base. In this way, in the process of generating an expression base with different expressions for the virtual object according to the standard template group that binds the component to the facial area and the neutral expression base of the virtual object, the automatic adjustment of the component following the facial area can be realized, and then the expression base with the component of the virtual object, that is, the first expression, can be obtained. Since the expression generation method of the virtual object provided in the embodiment of the present application can automatically generate an expression base with components for the virtual object, it can overcome the defect in the related art that the artist needs to manually adjust the component, which limits the speed of expression base production.

[0011] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present specification. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the specification and, together with the description, serve to explain the principles of the specification.

[0013] Figure 1 It is a schematic diagram of a principle of obtaining expressions for a virtual object according to an exemplary embodiment of the present application.

[0014] Figure 2A It is a schematic diagram of a standard template group shown in the present application according to an exemplary embodiment.

[0015] Figure 2BIt is a schematic diagram of a set of expression bases generated based on a standard template group according to an exemplary embodiment of the present application.

[0016] Figure 3 It is a basic schematic diagram of an expression with eyebrows shown in the present application according to an exemplary embodiment.

[0017] Figure 4 It is a flowchart of a method for generating an expression of a virtual object according to an exemplary embodiment of the present application.

[0018] Figure 5 It is a specific flow chart of binding a neutral expression base template and respective components on the neutral expression base with respective facial regions according to an exemplary embodiment of the present application.

[0019] Figure 6 It is a schematic diagram of a neutral expression base template including components according to an exemplary embodiment of the present application.

[0020] Fig. 7A It is a partially enlarged schematic diagram of a neutral expression base template before a component is bound to a facial area according to an exemplary embodiment of the present application.

[0021] Figure 7B It is a partially enlarged schematic diagram of a neutral expression base template after a component is bound to a facial area according to an exemplary embodiment of the present application.

[0022] Figure 8 It is another specific flow chart of binding a neutral expression base template and respective components on the neutral expression base with respective facial regions according to an exemplary embodiment of the present application.

[0023] Fig. 9 It is a schematic diagram of a triangulation problem of a scattered point set shown in the present application according to an exemplary embodiment.

[0024] Fig.10 It is a schematic diagram of a process of generating a first expression with components according to an exemplary embodiment of the present application.

[0025] Fig.11 It is a flowchart of generating richer expressions for a virtual object according to an exemplary embodiment of the present application.

[0026] Fig.12 It is a structural schematic diagram of an electronic device for generating expressions for a virtual object according to an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0027] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with this specification. Instead, they are merely examples of devices and methods consistent with some aspects of this specification as detailed in the appended claims.

[0028] The terms used in this specification are for the purpose of describing specific embodiments only and are not intended to limit this specification. The singular forms "a", "the" and "the" used in this specification and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0029] It should be understood that although the terms first, second, third, etc. may be used in this specification to describe various information, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this specification, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0030] In the fields of 3D animation, filmmaking, virtual reality, and augmented reality, there is usually a need to generate different expressions for virtual objects. Facial deformation (Blend shape) is a commonly used method for facial expression animation synthesis. The core principle of this method comes from the Facial Action Coding System, which believes that human expressions can be divided into local meta-expressions, that is, each expression can be formed by the linear superposition of several local meta-expressions. Therefore, this method linearly fuses a set of expression bases that can cover most of the expression details, and interpolates to obtain a new expression. It can be expressed in mathematical expressions as:

[0031]

[0032] Among them, f is a new expression; b0 is the neutral expression base, that is, the expression state in the natural state, without any expression such as "happiness", "anger", "sorrow" etc.; b k For predefined expression bases with expressions, different b k Represents different expressions; w k is a weight coefficient used to weight other expression bases b with different expressions based on the neutral expression base.k The difference with the neutral expression base b0 is interpolated and fused as in the above formula (1), thereby obtaining a new expression for the virtual object.

[0033] Based on the above analysis, it can be seen that for a virtual object, a new expression can be obtained based on multiple expression bases including a neutral expression base and a weight coefficient. The corresponding principle diagram is as follows: Figure 1 As shown, where f, b k and w k The meaning is the same as above.

[0034] In the related art, there is a method for automatically generating different expressions for a new character based on a standard template group of an existing character and a neutral expression base of another new character, wherein the standard template group includes a plurality of expression base templates, and the plurality of expression base templates include a neutral expression base template and an expression base template with different expressions. The technical means adopted is: the expression actions of the expression base included in the standard template group of the existing character are transferred to the new character, thereby obtaining a plurality of expression bases of the new character.

[0035] Combination Figure 2A and Figure 2B Provide explanation. Figure 2A It is a standard template group for existing characters, including a neutral expression base template 201 and a plurality of expression base templates with different expressions. Figure 2B It is a set of expression bases generated for the new character based on the neutral expression base 203 of a new character and the standard template group of an existing character. 202 and 203 are a pair of expression base templates and expression bases with corresponding expressions. The expression actions of the expression base templates included in the standard template group of the existing character are transferred to the new character in the following way: first determine the deformation amount of the expression base template with expression in the standard template group, such as 202, relative to the neutral expression base template 201, and then perform corresponding deformation on the neutral expression base of the new character, and then obtain the expression base with corresponding expression such as 204.

[0036] In the related art, this method of directly migrating the facial expressions of the expression bases contained in the standard template group of the existing character to the new character, thereby obtaining several expression bases of the new character, can obtain the expression bases of the new character. However, these methods usually only consider the facial area, and do not consider components such as eyes, teeth, and hair. Most of the virtual objects produced usually need to add the above components in order to present a more realistic effect. In the related art, usually after the facial expression base is produced, the artist manually places these components into each expression base, and carefully adjusts them so that there is no abnormality in the fused expression. Among these components, since the eyeballs, mouths, etc. are approximately rigid body shapes, they are relatively easy to adjust. However, deformable components such as eyelashes and eyebrows are relatively difficult to adjust, and it is more time-consuming to make adjustments. For example Figure 3 As shown, a schematic diagram of an expression base with eyebrows is given, wherein 301 is the eyebrows of the virtual object. In the production process of the entire human head expression base, the production time of these components accounts for 40% to 80%. Therefore, the later addition and adjustment of these components have become a bottleneck limiting the production efficiency of the expression base.

[0037] In order to overcome the above technical problems existing in the related art, the embodiment of the present application provides a method for generating an expression of a virtual object, such as Figure 4 As shown, the method includes:

[0038] Step 401, obtaining an expression template group including a plurality of expression base templates with different expressions and a neutral expression base template, a neutral expression base of a virtual object whose expression is to be generated, and a connection relationship between a facial region and a component of the neutral expression base template, wherein the neutral expression base template and the neutral expression base have components, and the components are non-skin parts of the facial region;

[0039] Step 402, based on the connection relationship, binding the neutral expression base template and respective components on the neutral expression base with respective facial regions;

[0040] Step 403: Generate a first expression with a component according to a neutral expression base bound to the component and the facial region and a standard template group including a neutral expression base template bound to the component and the facial region.

[0041] Among them, the standard template group, similar to Figure 2A The difference between the series of expression base templates shown in FIG. 1 and FIG. 2 is that the neutral expression base template has components. The standard template group containing the neutral expression base template with components can be obtained through art production, or obtained from other open source websites, or can be obtained from Figure 2AThe standard template group without components shown is obtained by manually adding components to the neutral expression base template and adjusting it in the later stage by an artist, and the embodiment of the present application does not impose any limitation on this.

[0042] The neutral expression base of the virtual object to be generated also has components, that is, it can be a component Figure 2B The neutral expression base of the new character shown. The neutral expression base with components of the virtual object whose expression is to be generated can be a neutral expression base of the virtual object obtained from other places, obtained by adding components later by an artist, or a neutral expression base of the virtual object with components directly obtained from other places, or produced by an artist. The embodiment of the present application does not limit the method of obtaining the neutral expression base with components of the virtual object whose expression is to be generated.

[0043] The neutral expression base of the virtual object for which expression is to be generated and the connection relationship between the facial area and components of the neutral expression base template can be obtained from the user, for example, the user specifies the connection between the component and the facial area on the neutral expression base template, or can be obtained from other devices that pre-store the connection relationship. Of course, it can also be a device or apparatus that executes the method for generating expressions of virtual objects described in the embodiment of the present application, which is obtained based on pre-processing. The embodiment of the present application does not limit the method and source of obtaining the connection relationship.

[0044] The connection relationship between the facial area and the components of the neutral expression base template can be expressed in various forms. For example, the connection relationship can be expressed in the form of a mapping table, which contains the mapping relationship between the key points on the facial area of ​​the neutral expression base template and the key points on the components. For another example, the connection relationship can be expressed in topology, and the facial area and the key points on the components can be expressed in the form of elements in the topology, and the connection relationship between the topological elements is the connection relationship between the facial area and the components of the neutral expression base template. In addition, the connection relationship can also be that the user specifies the connection between the component and the facial area with a geometric pattern on the expression base, then the geometric pattern represents the connection relationship between the facial area and the component of the neutral expression base template. Of course, the connection relationship can also be expressed in other forms, and the embodiments of the present application are not limited to this.

[0045] Step 402, binding the neutral expression base template and respective components on the neutral expression base with respective facial regions, and step 403, generating a first expression with components, can be implemented based on a variety of methods, which will be described in detail later.

[0046] Based on the above embodiments, it can be seen that in the expression generation method of the virtual object provided in the embodiment of the present application, since the connection relationship between the facial area and the component of the neutral expression base template in the standard template group is obtained, based on the connection relationship, the component on the neutral expression base template can be bound to the facial area, and the component on the neutral expression base of the virtual object to be generated can be bound to the facial area, thereby realizing the association of the component with the facial area on the neutral expression base template and the neutral expression base. In this way, in the process of generating an expression base with different expressions for the virtual object according to the standard template group that binds the component to the facial area and the neutral expression base of the virtual object, the automatic adjustment of the component following the facial area can be realized, and then the expression base with the component of the virtual object, that is, the first expression, can be obtained. Due to the expression generation method of the virtual object provided in the embodiment of the present application, the expression base with the component can be automatically generated for the virtual object, thereby overcoming the defect in the related art that the artist needs to manually adjust the component, which limits the speed of expression base production.

[0047] The above embodiments have been introduced, and the components are non-skin parts of the facial area. In some embodiments, the components at least include: eyes, teeth, hair, etc. The hair includes eyebrows, hair, eyelashes, sweat hair, etc., and the embodiments of the present application do not limit this.

[0048] Of course, in addition to the organs or tissues of the virtual object itself, the components may also include organs or tissues that do not belong to the virtual object itself but are located on the face of the virtual object, such as stickers, decorations, etc. located on the face of the virtual object. The embodiments of the present application do not limit this.

[0049] For step 402, based on the connection relationship between the facial area and components of the acquired neutral expression base template, the neutral expression base template and the respective components on the neutral expression base are bound to their respective facial areas. This can be achieved based on a variety of methods, which are described in detail below.

[0050] In some embodiments, the binding may be as follows: Figure 5 As shown, this is achieved by:

[0051] Step 501, based on the connection relationship, establishing a first geometric connection between the component and the facial area on the neutral expression base template;

[0052] Step 502: Based on the first geometric connection, a second geometric connection corresponding to the first geometric connection is established on the neutral expression basis.

[0053] As mentioned above, the connection relationship can be expressed in various forms. Here, the connection relationship is that the user specifies the connection between the component and the facial area using a geometric pattern on the expression base, that is, the geometric pattern represents the connection relationship between the facial area of ​​the neutral expression base template and the component. Figure 6 The neutral expression base template containing components as shown and Fig. 7A He Ru Figure 7B The partially enlarged schematic diagram of the binding area is shown to illustrate the principle of the binding.

[0054] Figure 6 The left figure in FIG. 1 is a schematic diagram showing that the component is not bound to the facial region in the neutral expression base template. Figure 6 The right figure in FIG. 1 is a schematic diagram of binding the components to the facial region based on the connection relationship between the facial region and the components of the neutral expression base template obtained. Figure 6 In the figure, 601 is the neutral expression base template, 602 is a component, and eyebrows are used as an example here for the component to be explained, and 603 is a local area of ​​the facial area to which the binding is performed.

[0055] like Figure 6 As shown in the right figure of , the user can specify corresponding line segments on the component 602 and the facial area 603 of the neutral expression base template on the neutral expression base template, and the specified line segments represent that there is a connection relationship between the specified parts. Fig. 7A and Figure 7B As shown, given Figure 6 FIG. 6 is an enlarged schematic diagram of the facial region 603 before and after the component is bound to the facial region. Fig. 7A and Figure 7B In the figure, 701 is the neutral expression base template, 702 is a component, 703 is a line segment corresponding to the component specified by the user, and 704 is a line segment corresponding to the target facial region specified by the user. After obtaining 703 and 704 specified by the user, the connection relationship between the facial region and the component of the neutral expression base template represented by the geometric pattern is actually obtained. Based on the connection relationship, it can be as follows Figure 7B As shown, a first geometric connection between the component and the facial area is established on the neutral expression base template. The first geometric connection can be established by selecting a certain number of key points on line segments 703 and 704, and then establishing a geometric connection for the multiple key points. The key points can be selected randomly or based on preset rules, which is not limited in this embodiment of the present application. The geometric connection can be a polygonal connection such as a triangle, quadrilateral, etc., and of course, it can also be other geometric connections, which is not limited in this embodiment of the present application. Figure 7BIn FIG. 5 , a schematic diagram of the geometric connection being a quadrilateral is given.

[0056] Based on the connection relationship, after the first geometric connection between the component and the facial area is established on the neutral expression base template, since the neutral expression base of the virtual object has a corresponding relationship with the neutral expression base template in the standard template group, the corresponding relationship can be determined by referring to relevant technologies. For example, the corresponding relationship can be determined based on key parts such as the mouth and the corners of the eyes. The embodiments of the present application will not be described in detail here. Therefore, based on the first geometric relationship, the first geometric connection on the neutral expression base template can be copied to the neutral expression base of the virtual object, and then a second geometric connection corresponding to the first geometric connection can be established on the neutral expression base of the virtual object, thereby realizing the binding of its components to the facial area on the neutral expression base of the virtual object.

[0057] It can be seen from the above embodiments that by first establishing a first geometric connection between the component and the facial area on the neutral expression base template based on the acquired connection relationship, and then copying the first geometric connection to the neutral expression base of the virtual object, and then establishing a second geometric connection corresponding to the first geometric connection on the neutral expression base of the virtual object, the neutral expression base template and the components of the neutral expression base can be quickly bound to the corresponding facial area based on the limited information acquired, which is simple, effective, and easy to implement, laying the foundation for the subsequent generation of the first expression for the virtual object.

[0058] In some embodiments, the binding may also be as follows: Figure 8 As shown, this is achieved by:

[0059] Step 801, obtaining the connection relationship between the facial area of ​​the neutral expression base and the components;

[0060] Step 802, based on the connection relationship between the facial region and the component of the neutral expression base template, establish a first geometric connection between the component and the facial region on the neutral expression base template;

[0061] Step 803: Based on the connection relationship between the facial area of ​​the neutral expression base and the component, a second geometric connection is established between the component and the facial area on the neutral expression base.

[0062] In this embodiment, the binding of the facial area and components of the neutral expression base template can be achieved by establishing a first geometric connection between the component and the facial area on the neutral expression base template based on the connection relationship between the facial area and components of the neutral expression base template obtained in step 401, similar to that shown in the previous embodiment.

[0063] As for the binding of the facial area and components of the neutral expression base of the virtual object, the connection relationship between the facial area and the components of the neutral expression base can be obtained, and then similar to the binding of the components and facial areas based on the neutral expression base template, a second geometric connection between the components and the facial area can be established on the neutral expression base according to the connection relationship between the facial area and the components of the neutral expression base.

[0064] The method of obtaining the connection relationship between the facial area and the components of the neutral expression base may be the same as the method of obtaining the connection relationship between the facial area and the components of the neutral expression base template, and the representation of the connection relationship may also be the same as that of the neutral expression base template, which is not limited in this embodiment of the present application.

[0065] The timing for obtaining the connection relationship between the facial area and the components of the neutral expression base may be when obtaining the connection relationship between the facial area and the components of the neutral expression base template and also obtaining the connection relationship of the neutral expression base, or it may be after binding the components and the facial area of ​​the neutral expression base template and before binding the components and the facial area of ​​the neutral expression base. The embodiment of the present application does not impose any restrictions on this.

[0066] It can be seen from the above embodiments that by establishing a second geometric connection between the component and the facial area on the neutral expression base based on the connection relationship of the acquired virtual object's neutral expression base, the characteristics of the neutral expression base of the virtual object itself can be taken into account, and the neutral expression base binds its own components to its own facial area, thereby avoiding a large deviation between the components of the neutral expression base and the facial area due to the large difference between the neutral expression base template and the neutral expression base of the virtual object. For example, the eyebrows should be bound a little bit above the eyelids, but due to the large deviation, the eyebrows are bound to the forehead area in the neutral expression base.

[0067] Of course, those skilled in the art should understand that the above-mentioned embodiments of binding the respective components on the neutral expression base template and the neutral expression base with the respective facial regions are only described by taking the example that the connection relationship obtained is that the user specifies the connection between the component and the facial region with a geometric pattern on the expression base. When the connection relationship is in other forms, it can be similar to the above-mentioned embodiments, and the components can be bound to the facial region according to the connection relationship of other forms obtained accordingly. For example, when the connection relationship is represented in the form of a mapping table, that is, the mapping table contains the mapping relationship between the key points on the facial region of the neutral expression base template and the key points on the components, then, according to the key points on the facial region and the components, the neutral expression base template and the neutral expression base can create a link located at the position of the facial region and the component, and then based on the mapping relationship of the key points, create a corresponding geometric connection for the neutral expression base template and the neutral expression base. This process is similar to the above-mentioned embodiments, and this application will not be repeated here. When the connection relationship is represented by topology, the creation of the geometric connection is also similar to the above-mentioned embodiments, and this application embodiment will not be repeated here.

[0068] As described above, in the process of binding components to facial regions, the geometric connection created may be a polygon connection such as a triangle, a quadrilateral, a pentagon, etc. In some embodiments, the geometric connection created may be a triangular connection, and the creation of the triangular connection is implemented based on Delaunay triangulation.

[0069] Delaunay triangulation is a technique proposed by Delaunay to connect space points into triangles so that the minimum angle of all triangles is the largest. How to divide a scattered point set into an uneven triangle network is the problem of triangulation of scattered point sets. The triangulation of scattered point sets is an extremely important preprocessing technique for numerical analysis and graphics. The problem is shown in the figure below. Fig. 9 As shown, the left figure is a scattered point set, and the right figure is a structural schematic diagram of the scattered point set being divided into an uneven triangular network. Delaunay triangulation establishes the triangular relationship between scattered point sets according to the "minimum angle maximum" and "empty circumscribed circle" criteria. For the relevant contents of these two criteria, reference can be made to the relevant technology, and the embodiments of this application will not be elaborated here.

[0070] In the expression generation method of the virtual object provided in the present application, in the process of binding the neutral expression base template and the components on the neutral expression base with the facial area through geometric connection, based on the obtained connection relationship, multiple key points corresponding to the components and the facial area can be obtained in essence. For example, when the connection relationship is that the user specifies the connection between the component and the facial area with a geometric pattern on the expression base, then some points can be selected on the geometric pattern as key points representing the position of the component and the facial area. For another example, when the connection relationship is represented in the form of a mapping table, the mapping table contains the key points on the component and the facial area. After having these key points, what these key points constitute is actually the "scattered point set" mentioned. Establishing a triangular connection between the component and the facial area actually converts the problem into the problem of "how to divide a scattered point set into an uneven triangular network".

[0071] Therefore, based on the connection relationship between the component and the facial area, a triangular connection is established between the component and the facial area, which can be achieved by using Delaunay triangulation. There are many related algorithms for Delaunay triangulation, such as the flanging algorithm, the point-by-point insertion algorithm including the Lawson algorithm and the Bowyer-Watson algorithm, the segmentation and merging algorithm, and the like. Taking the Lawson algorithm as an example, its basic principle is: first establish a large triangle or polygon to surround all data points, insert a point into it, and the point is connected to the three vertices of the triangle containing it to form three new triangles, and then perform empty circumscribed circle detection on them one by one, and optimize them with a pre-designed local optimization process, that is, by exchanging diagonals to ensure that the formed triangulated network is a Delaunay triangulated network. Other algorithms for implementing triangulation can refer to the relevant technology, and the embodiments of the present application are not limited here. Of course, the creation of the triangular connection can also be implemented based on other algorithms determined based on Delaunay triangulation, and the embodiments of the present application do not limit the specific creation algorithm of the triangular connection.

[0072] It can be seen from the above embodiments that the geometric connection described in the previous embodiments is implemented by creating triangle connections based on Delaunay triangulation. A variety of algorithms based on Delaunay triangulation can be applied, with a wide range of options, stable and reliable performance, and easy implementation.

[0073] Of course, those skilled in the art should understand that the geometric connection may of course also be other polygonal connections, and the specific implementation of the polygonal connection may be achieved by applying existing polygonal connection technology or by polygonal connection technology created by the developer himself, and the embodiments of the present application do not impose any restrictions on this.

[0074] After binding the neutral expression base template and the respective components on the neutral expression base with the respective facial areas based on the connection relationship, step 403 can be performed to generate a first expression with components based on the neutral expression base in which the components are bound to the facial areas and a standard template group including the neutral expression base template in which the components are bound to the facial areas.

[0075] In some embodiments, the generating of the first expression with components, such as Fig.10 As shown, including:

[0076] Step 1001, binding the facial regions of the expression base templates with expressions in the standard template group to the respective components according to the neutral expression base templates to which the components are bound to the facial regions;

[0077] Step 1002, determining the deformation amount of each expression base template with expression relative to the neutral expression base template;

[0078] Step 1003: adjusting the neutral expression base according to the deformation amount to obtain a first expression with components corresponding to the expression base template with expression.

[0079] Since different expression base templates in the standard template group belong to the same object, in step 1001, based on the binding relationship between the component and the facial area in the neutral expression base template, the component and the facial area can be correspondingly bound in other expression base template groups with expressions in the standard template group. The correspondence can be achieved based on multiple corresponding key points, which is not limited in the embodiment of the present application.

[0080] After step 1001, the target standard template group is obtained, in which the facial area of ​​each expression base template is bound to the component. Then, based on the idea of ​​deformation transfer, the deformation amount of each expression base template with expression relative to the neutral expression base template can be determined, and then the deformation amount is transferred to the neutral expression base in which the virtual object component is bound to the facial area, and the neutral expression base in which the virtual object component is bound to the facial area is adjusted to obtain the expression base with expression corresponding to the expression base template with expression in the standard template group, that is, the first expression described in step 403. Since the component is bound to the facial area in all expression base templates and the neutral expression base of the virtual object, when the deformation amount is transferred, not only the deformation amount of the migrated facial area, but also the deformation amount of the component bound to the facial area can be automatically transferred, thereby achieving the simultaneous adjustment of the component and the facial area of ​​the neutral expression base of the virtual object, and obtaining the first expression with the component.

[0081] Of course, those skilled in the art should understand that, in addition to obtaining the first expression with components based on the above-mentioned deformation transfer idea, it is also possible to learn the intrinsic relationship between the neutral expression base template and the expression base template with different expressions from the standard template group based on a trained deep learning model, and input the neutral expression base of the virtual object into the trained deep learning model to obtain the expression base with different expressions of the virtual object, that is, the first expression. Alternatively, the generation of the first expression with components can be achieved based on other algorithms, which is not limited in the embodiments of the present application.

[0082] Through the above-mentioned embodiments, it can be seen that in the expression generation method of the virtual object provided in the embodiment of the present application, since the connection relationship between the facial area and the component of the neutral expression base template in the standard template group is obtained, based on the connection relationship, the component on the neutral expression base template can be bound to the facial area, and the component on the neutral expression base of the virtual object to be generated can be bound to the facial area, thereby realizing the association of the component with the facial area on the neutral expression base template and the neutral expression base. In this way, in the process of generating an expression base with different expressions for the virtual object according to the standard template group that binds the component to the facial area and the neutral expression base of the virtual object, the automatic adjustment of the component following the facial area can be realized, and then the expression base with the component of the virtual object, that is, the first expression, can be obtained. Due to the expression generation method of the virtual object provided in the embodiment of the present application, the expression base with the component can be automatically generated for the virtual object, thereby overcoming the defect in the related art that the artist needs to manually adjust the component, which limits the speed of expression base production.

[0083] In some embodiments, the number of first expressions of the component generated in step 403 is multiple, that is, in step 403, the number of expression bases generated for the virtual object is multiple, and the expression generation method of the virtual object provided in the embodiment of the present application is as follows: Fig.11 As shown, it also includes:

[0084] Step 1101, obtaining a weight sequence, wherein the weight sequence is used to fuse multiple expression bases to generate an expression;

[0085] Step 1102: Based on the weight sequence, a plurality of first expressions with components are merged to obtain a second expression of the virtual object.

[0086] After obtaining an expression base with components and different expressions for the virtual object, the facial deformation method described above can also be applied as described in the above embodiment to interpolate and fuse the expression base based on the obtained weight sequence to generate more types of expressions for the virtual object. The basic principle and related contents of the facial deformation have been introduced in detail in the previous article and will not be repeated here.

[0087] It can be seen from the above embodiments that after multiple first expressions are generated for the virtual object, obtaining a weight sequence to generate more expressions for the virtual object can provide more materials for animation production, virtual reality, augmented reality, etc., enhance the user experience of the audience and users, and can improve the application scenarios and application scope of the first expression.

[0088] Of course, in addition to providing richer expressions for the virtual object based on the expression generation method of the virtual object provided in the embodiment of the present application, the first expression can also be applied to other scenarios, and the embodiment of the present application does not limit this.

[0089] In addition, corresponding to the expression generation method of a virtual object provided in the above embodiment of the present application, the embodiment of the present application also provides a software product, that is, a computer program product, which includes a computer / instruction, and when the computer program / instruction is executed by the processor, the above-mentioned various method embodiments provided in the embodiment of the present application are implemented. For the relevant content of the computer program product, reference can be made to the various method embodiments in the above text, and the present application will not elaborate on it here.

[0090] Corresponding to the embodiment of the aforementioned method, the present application also provides an electronic device, Fig.12 A schematic diagram of the structure of an exemplary electronic device provided in this application.

[0091] like Fig.12 As shown, an electronic device 1200 provided in this embodiment includes: a processor 1201, a memory 1202, and a computer program stored in the memory 1202 and executable on the processor 1201, wherein the processor implements the method of the foregoing application when executing the program. The processor 1201 in the electronic device may be one or more, Fig.12 In the example, a processor 1201 is used, and the processor may include a GPU and a CPU. The processor 1201 and the memory 1202 in the electronic device may be connected via a communication bus or other means. Fig.12 The connection via the communication bus 1203 is taken as an example.

[0092] In this embodiment, the processor 1201 of the electronic device integrates the computer program product provided in the above embodiment. In addition, the memory 1202 in the electronic device, as a computer-readable storage medium, can be used to store one or more programs, which can be software programs, computer executable programs and modules. The processor 1201 executes various functional applications and data processing of the device by running the software programs, instructions and modules stored in the memory 1202, that is, implements the above-mentioned various method embodiments.

[0093] The memory 1202 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, a computer program product required for at least one function; the data storage area may store data created according to the use of the device, etc. In addition, the memory 1202 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 1202 may further include a memory remotely arranged relative to the processor 1201, and these remote memories may be connected to the device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0094] The processor 1201 executes various functional applications and data processing by running the program stored in the memory 1202 to implement the method provided in the embodiment of the present application.

[0095] In addition, the present application also provides a computer-readable storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, any of the methods described above is implemented.

[0096] The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination thereof. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CDROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, device, or device.

[0097] The signal medium of the computer-readable storage medium may include a data signal propagated in baseband or as part of a carrier wave, wherein a computer-readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0098] The program code included in the computer-readable medium can be transmitted with any appropriate medium, including but not limited to wireless, electric wire, optical cable, RF, etc., or any suitable combination of the above. The computer program code for performing the operation of the application can be written in one or more programming languages ​​or their combinations, and the programming language includes object-oriented programming languages-such as Java, Smalltalk, C++, and also includes conventional procedural programming languages-such as "C" language or similar programming languages. The program code can be executed completely on the user's computer, partially on the user's computer, executed as an independent software package, partially on the user's computer and partially on the remote computer, or completely on the remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

[0099] Those skilled in the art will readily appreciate other embodiments of the present application after considering the application and practicing the inventions claimed herein. The present application is intended to cover any variations, uses or adaptations of the present application, which follow the general principles of the present application and include common knowledge or customary techniques in the art that are not claimed in the present application. The description and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0100] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

[0101] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A method for generating an expression of a virtual object, characterized in that: The method comprises: Acquire a standard template group including a plurality of expression base templates with different expressions and a neutral expression base template, a neutral expression base of a virtual object whose expression is to be generated, and a connection relationship between a facial region and a component of the neutral expression base template, wherein the neutral expression base template and the neutral expression base have components, and the components are non-skin parts of the facial region; Based on the connection relationship, the neutral expression base template and respective components on the neutral expression base are bound to respective facial regions; A first expression with a component is generated according to a neutral expression base bound to the component and the facial region and a standard template group including a neutral expression base template bound to the component and the facial region.

2. The method according to claim 1, characterized in that The components include at least: eyes, teeth, and hair.

3. The method according to claim 1, characterized in that The step of binding the neutral expression base template and respective components on the neutral expression base template to respective facial regions based on the connection relationship includes: Based on the connection relationship, establishing a first geometric connection between the component and the facial area on the neutral expression base template; Based on the first geometric connection, a second geometric connection corresponding to the first geometric connection is established on the neutral expression basis.

4. The method according to claim 1, characterized in that: The step of binding the neutral expression base template and respective components on the neutral expression base template to respective facial regions based on the connection relationship includes: Acquire the connection relationship between the facial area of ​​the neutral expression base and the components; Based on the connection relationship between the facial region and the component of the neutral expression base template, establishing a first geometric connection between the component and the facial region on the neutral expression base template; Based on the connection relationship between the facial area of ​​the neutral expression base and the component, a second geometric connection is established between the component and the facial area on the neutral expression base.

5. The method according to any one of claims 3 to 4, characterized in that: The geometric connection is a triangle connection, and the establishment of the geometric connection is achieved based on Delaunay triangulation.

6. The method according to claim 1, characterized in that The generating of the first expression with components includes: Binding the facial regions of the expression base templates with expressions in the standard template group to the respective components according to the neutral expression base templates to which the components are bound to the facial regions; Determine the deformation amount of each expression base template with expression relative to the neutral expression base template; The neutral expression base is adjusted according to the deformation amount to obtain a first expression with components corresponding to the expression base template with expression.

7. The method according to claim 6, characterized in that The generated first expressions with components are multiple, and the method further includes: Obtaining a weight sequence, wherein the weight sequence is used to fuse multiple expression bases to generate an expression; Based on the weight sequence, a plurality of first expressions with components are fused to obtain a second expression.

8. A computer program product, characterized in that The method comprises a computer program / instruction, wherein the computer program / instruction implements the method according to any one of claims 1 to 7 when the computer program / instruction is executed by a processor.

9. An electronic device, characterized in that: The electronic device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the method according to any one of claims 1 to 7 when executing the program.

10. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.