Three-dimensional model expression reorientation method and related products

Abstract

This application discloses a method and related products for retargeting facial expressions in a 3D model. The method includes: obtaining a first facial expression basis and multiple deformation basis bases for a 3D model; outputting a 3D model with a first facial expression using the first facial expression basis and the multiple deformation basis bases; during the process of outputting the 3D model with the first facial expression using the first facial expression basis and the multiple deformation basis bases, the coefficient of the first deformation basis base, which has a risk of conflict with the first facial expression basis, is reduced. During the process of outputting the 3D model with the first facial expression using the first facial expression basis and the multiple deformation basis bases, the coefficient of the first deformation basis base is reduced from an initial parameter to 0. In this application embodiment, reducing the coefficient of the first deformation basis base, which has a risk of conflict with the first facial expression basis, can avoid clipping problems in the output 3D model with the first facial expression, making the output 3D model more realistic and natural.

Description

Technical Field

[0001] This application relates to the field of computer vision, and in particular to a method for retargeting facial expressions in a 3D model and related products. Background Technology

[0002] With the rapid development of the entertainment industry, including animated films and games, people's demands for the expressiveness of virtual avatars are constantly increasing. Realistically creating animated expressions for various virtual avatars can greatly enhance their realism and provide users with a more immersive experience. Virtual avatars can be cartoon characters, virtual humans, or simulated characters based on real-life appearances, or even virtual animal characters.

[0003] Current personalized avatars primarily rely on combining numerous deformable morph targets (such as chin length, eye width, etc.). A morph target for an object represents its deformable features. Current facial expressions, such as augmented reality (AR) expressions, are displayed using morph targets and morph targets, essentially recording the relative vectors of each point. However, 3D models with expressions obtained in this way lack realism and naturalness. Therefore, research is needed to determine how to obtain more realistic and natural 3D models with arbitrary facial expressions. Summary of the Invention

[0004] This application discloses a method and related products for retargeting facial expressions in a 3D model, which can output more realistic and natural 3D models with arbitrary facial expressions.

[0005] In a first aspect, embodiments of this application provide a method for retargeting facial expressions in a three-dimensional model. The method includes: obtaining a first facial expression basis and a plurality of deformation basis bases for a three-dimensional model; outputting the three-dimensional model with a first facial expression using the first facial expression basis and the plurality of deformation basis bases; and reducing the coefficient of the first deformation basis base among the plurality of deformation basis bases that has a risk of conflict with the first facial expression basis during the process of outputting the three-dimensional model with the first facial expression using the first facial expression basis and the plurality of deformation basis bases.

[0006] In this embodiment, during the process of outputting a 3D model with a first expression using a first expression basis and multiple deformation bases, the coefficients of the first deformation base, which has a risk of conflict with the first expression basis, are reduced. Reducing the coefficients of the first deformation base avoids clipping issues in the output 3D model with the first expression, making the output 3D model more realistic and natural. Furthermore, during the process of outputting a 3D model with a first expression using the first expression basis and multiple deformation bases, it is not necessary to regenerate the 3D model, resulting in low computational cost.

[0007] In one possible implementation, during the process of outputting the three-dimensional model with the first expression using the first expression basis and the plurality of deformation basis bases, the coefficients of the first deformation basis base are reduced from an initial parameter to 0, where the initial parameter is a real number greater than 0.

[0008] In this implementation, during the process of outputting a 3D model with the first expression using the first expression basis and multiple deformation basis, the coefficients of the first deformation basis are reduced from the initial parameters to 0; this can gradually reduce the adverse effects of the first deformation basis and make the output first expression more realistic and natural.

[0009] In one possible implementation, during the process of outputting the 3D model with the first expression using the first expression basis and the plurality of deformation basis bases, the coefficients of the first deformation basis base are reduced to 0 from the initial parameters according to a preset transformation curve. The preset transformation curve can be pre-defined. For example, the preset transformation curve is pre-configured according to the type of the first expression.

[0010] In this implementation, the coefficient of the first deformation basis is reduced to 0 from the initial parameter according to the preset change curve, which makes the output first expression more realistic and natural.

[0011] In one possible implementation, during the process of outputting the three-dimensional model with the first expression using the first expression basis and the plurality of deformation bases, the coefficients of the deformation bases among the plurality of deformation bases that do not have a risk of conflict with the first expression basis remain unchanged.

[0012] In this implementation, the coefficients of the deformation bases that do not have a conflict risk with the first expression base remain unchanged, which can avoid distortion of the output 3D model.

[0013] In one possible implementation, among the N three-dimensional vectors corresponding to the first deformation basis, the ratio of the number of first three-dimensional vectors to the number of non-zero three-dimensional vectors is greater than a first threshold; the N three-dimensional vectors corresponding to the first deformation basis correspond one-to-one with the N three-dimensional vectors corresponding to the first expression basis and each corresponds to one of the N vertices of the three-dimensional model; both the first three-dimensional vector and its corresponding three-dimensional vector among the N three-dimensional vectors corresponding to the first expression basis are non-zero vectors; and the angle between the first three-dimensional vector and its corresponding three-dimensional vector among the N three-dimensional vectors corresponding to the first expression basis is less than a second threshold. The first three-dimensional vector can be understood as the three-dimensional vector among the N three-dimensional vectors corresponding to the first deformation basis that satisfies the target condition. Any three-dimensional vector satisfying the target condition has the following characteristics: both the three-dimensional vector and its corresponding three-dimensional vector among the N three-dimensional vectors corresponding to the first expression basis are non-zero vectors; and the angle between the three-dimensional vector and its corresponding three-dimensional vector among the N three-dimensional vectors corresponding to the first expression basis is less than the second threshold.

[0014] In this implementation, the ratio of the number of first three-dimensional vectors to the number of non-zero three-dimensional vectors in the N three-dimensional vectors corresponding to the first deformation basis is greater than a first threshold. It should be understood that if the ratio of the number of first three-dimensional vectors to the number of non-zero three-dimensional vectors in the N three-dimensional vectors corresponding to a deformation basis is greater than the first threshold, then there is a risk of conflict between the deformation basis and the first expression basis. This implementation describes the characteristics of the first deformation basis that has a risk of conflict with the first expression basis. In the process of outputting a three-dimensional model with a first expression using the first expression basis and multiple deformation bases, the coefficient of the first deformation basis that has a risk of conflict with the first expression basis among the multiple deformation bases is reduced. This can avoid clipping problems in the output three-dimensional model with the first expression, making the output three-dimensional model more realistic and natural.

[0015] In one possible implementation, the ratio of the length of the first three-dimensional vector to the length of its corresponding three-dimensional vector among the N three-dimensional vectors corresponding to the first expression base is greater than 1 and less than a third threshold, where the third threshold is a real number greater than 1; or, the ratio of the length of the first three-dimensional vector to the length of its corresponding three-dimensional vector among the N three-dimensional vectors corresponding to the first expression base is less than 1 and greater than a fourth threshold, where the fourth threshold is a real number less than 1; or, the ratio of the length of the first three-dimensional vector to the length of its corresponding three-dimensional vector among the N three-dimensional vectors corresponding to the first expression base is equal to 1.

[0016] In this implementation, the characteristics of the first three-dimensional vector are further defined, which can more accurately identify the deformation basis that has a risk of conflict with the first expression basis.

[0017] In one possible implementation, before outputting the 3D model with the first expression using the first expression basis and the plurality of deformation bases, the method further includes: identifying the first deformation base among the plurality of deformation bases that has a risk of conflict with the first expression basis.

[0018] In this implementation, a first deformation basis that has a conflict risk with the first expression basis is identified among multiple deformation basis bases, so as to reduce the coefficient of the first deformation basis base in the process of outputting a three-dimensional model with the first expression using the first expression basis and multiple deformation basis bases.

[0019] In one possible implementation, determining the first deformation basis among the plurality of deformation bases that has a conflict risk with the first expression basis includes:

[0020] If, among the N three-dimensional vectors corresponding to the first deformation basis, the ratio of the number of first three-dimensional vectors to the number of non-zero three-dimensional vectors is greater than a first threshold, it is determined that there is a conflict risk between the first deformation basis and the first expression basis. The N three-dimensional vectors corresponding to the first deformation basis and the N three-dimensional vectors corresponding to the first expression basis are one-to-one correspondences, and each corresponds to one of the N vertices of the three-dimensional model. Both the first three-dimensional vector and its corresponding three-dimensional vector among the N three-dimensional vectors corresponding to the first expression basis are non-zero vectors, and the angle between the first three-dimensional vector and its corresponding three-dimensional vector among the N three-dimensional vectors corresponding to the first expression basis is less than a second threshold. It should be understood that a similar method can be used to determine whether each of the multiple deformation bases has a conflict risk with the first expression basis.

[0021] In this implementation, if the ratio of the number of the first three-dimensional vectors to the number of non-zero three-dimensional vectors in the N three-dimensional vectors corresponding to the first deformation basis is greater than a first threshold, it is determined that there is a risk of conflict between the first deformation basis and the first expression basis; it can accurately determine whether there is a risk of conflict between the first deformation basis and the first expression basis.

[0022] In one possible implementation, before determining that there is a risk of conflict between the first deformation basis and the first expression basis, the method further includes: determining the number of the first three-dimensional vectors and the number of non-zero three-dimensional vectors among the N three-dimensional vectors corresponding to the first deformation basis.

[0023] In this implementation, the number of the first three-dimensional vectors and the number of non-zero three-dimensional vectors among the N three-dimensional vectors corresponding to the first deformation basis are determined in order to determine whether there is a risk of conflict between the first deformation basis and the first expression basis.

[0024] In one possible implementation, obtaining the first expression basis and multiple deformation basis of the three-dimensional model includes: reading the data information of the deformation basis and expression basis of the three-dimensional model to obtain the first expression basis and the multiple deformation basis.

[0025] In this implementation, by reading the data information of the deformation basis and expression basis of the 3D model, the first expression basis and deformation basis required to generate the 3D model with the first expression can be obtained quickly.

[0026] In one possible implementation, the first expression is a dynamic expression, such as a dynamic closed-eye expression.

[0027] Secondly, embodiments of this application provide an image processing apparatus, comprising: an acquisition unit for acquiring a first expression basis and a plurality of deformation basis for a three-dimensional model; and an output unit for outputting the three-dimensional model with a first expression using the first expression basis and the plurality of deformation basis; wherein, during the process of outputting the three-dimensional model with the first expression using the first expression basis and the plurality of deformation basis, the coefficient of the first deformation basis among the plurality of deformation basis that has a risk of conflict with the first expression basis is reduced.

[0028] In this embodiment, the output unit uses a first expression basis and multiple deformation basis bases to output a 3D model with a first expression. During the process of outputting the 3D model with the first expression using the first expression basis and multiple deformation basis bases, the coefficients of the first deformation basis, which has a risk of conflict with the first expression basis, are reduced. Reducing the coefficients of the first deformation basis, which has a risk of conflict with the first expression basis, can prevent clipping problems in the output 3D model with the first expression, making the output 3D model more realistic and natural.

[0029] In one possible implementation, during the process of outputting the three-dimensional model with the first expression using the first expression basis and the plurality of deformation basis bases, the coefficients of the first deformation basis base are reduced from an initial parameter to 0, where the initial parameter is a real number greater than 0.

[0030] In one possible implementation, during the process of outputting the three-dimensional model with the first expression using the first expression basis and the plurality of deformation basis bases, the coefficients of the first deformation basis base are reduced to 0 from the initial parameters according to a preset change curve.

[0031] In one possible implementation, during the process of outputting the three-dimensional model with the first expression using the first expression basis and the plurality of deformation bases, the coefficients of the deformation bases among the plurality of deformation bases that do not have a risk of conflict with the first expression basis remain unchanged.

[0032] In one possible implementation, among the N three-dimensional vectors corresponding to the first deformation basis, the ratio of the number of first three-dimensional vectors to the number of non-zero three-dimensional vectors is greater than a first threshold; the N three-dimensional vectors corresponding to the first deformation basis correspond one-to-one with the N three-dimensional vectors corresponding to the first expression basis and each corresponds to one of the N vertices of the three-dimensional model; both the first three-dimensional vector and its corresponding three-dimensional vector among the N three-dimensional vectors corresponding to the first expression basis are non-zero vectors; and the angle between the first three-dimensional vector and its corresponding three-dimensional vector among the N three-dimensional vectors corresponding to the first expression basis is less than a second threshold.

[0033] In one possible implementation, the ratio of the length of the first three-dimensional vector to the length of its corresponding three-dimensional vector among the N three-dimensional vectors corresponding to the first expression base is greater than 1 and less than a third threshold, where the third threshold is a real number greater than 1; or, the ratio of the length of the first three-dimensional vector to the length of its corresponding three-dimensional vector among the N three-dimensional vectors corresponding to the first expression base is less than 1 and greater than a fourth threshold, where the fourth threshold is a real number less than 1; or, the ratio of the length of the first three-dimensional vector to the length of its corresponding three-dimensional vector among the N three-dimensional vectors corresponding to the first expression base is equal to 1.

[0034] In one possible implementation, the apparatus further includes: a determining unit, configured to determine the first deformation basis among the plurality of deformation basis bases that has a risk of conflict with the first expression basis.

[0035] In one possible implementation, the determining unit is specifically used to determine that there is a risk of conflict between the first deformation basis and the first expression basis if the ratio of the number of first three-dimensional vectors to the number of non-zero three-dimensional vectors in the N three-dimensional vectors corresponding to the first deformation basis is greater than a first threshold; the N three-dimensional vectors corresponding to the first deformation basis correspond one-to-one with the N three-dimensional vectors corresponding to the first expression basis and all correspond to the N vertices of the three-dimensional model, the first three-dimensional vector and its corresponding three-dimensional vector in the N three-dimensional vectors corresponding to the first expression basis are both non-zero vectors, and the angle between the first three-dimensional vector and its corresponding three-dimensional vector in the N three-dimensional vectors corresponding to the first expression basis is less than a second threshold.

[0036] In one possible implementation, the determining unit is further configured to determine the number of the first three-dimensional vectors and the number of non-zero three-dimensional vectors among the N three-dimensional vectors corresponding to the first deformation basis.

[0037] In one possible implementation, the acquisition unit is specifically used to read the data information of the deformation basis and expression basis of the three-dimensional model to obtain the first expression basis and the plurality of deformation basis.

[0038] For the technical effects of the second aspect or various possible implementation methods, please refer to the introduction of the technical effects of the first aspect or corresponding implementation methods.

[0039] Thirdly, embodiments of this application provide an image processing apparatus, which includes a processor and a memory, wherein the memory is used to store instructions, and the processor is used to execute the instructions stored in the memory, causing the processor to perform a method as described in the first aspect and any possible implementation thereof.

[0040] Fourthly, embodiments of this application provide a chip including a data interface and a processor, wherein the processor is used to execute the methods described in the first aspect and any possible implementation thereof.

[0041] Fifthly, embodiments of this application provide a computer-readable storage medium storing a computer program, the computer program including program instructions that, when executed by a processor, cause the processor to perform the method described in the first aspect and any possible implementation thereof.

[0042] In a sixth aspect, embodiments of this application provide a computer program product including program instructions that, when executed by a processor, cause the processor to perform the method described in the first aspect and any possible implementation thereof. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in the embodiments of this application or the background art, the accompanying drawings used in the embodiments of this application or the background art will be described below.

[0044] Figure 1 A flowchart of a three-dimensional model facial expression retargeting method provided in this application embodiment;

[0045] Figure 2 Flowchart of another three-dimensional model facial expression retargeting method provided in this application embodiment;

[0046] Figure 3 A schematic diagram showing the changes in the coefficients of expression base 1, deformation base 1, and deformation base 2 when expression base 1 is activated;

[0047] Figure 4 An example is shown of outputting a 3D model with a first expression using a first expression basis and multiple deformation basis bases;

[0048] Figure 5 A flowchart illustrating a method for determining whether there is a conflict risk between a first expression base and a first deformation base, provided in an embodiment of this application;

[0049] Figure 6 This is a schematic diagram of the structure of an image processing apparatus provided in an embodiment of this application;

[0050] Figure 7 This is a schematic diagram of the structure of a server provided in an embodiment of this application;

[0051] Figure 8 This is a block diagram of a partial structure of another image processing apparatus provided in the embodiments of this application. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described below in conjunction with the accompanying drawings.

[0053] The terms "first" and "second," etc., used in the specification, claims, and drawings of this application are used only to distinguish different objects and not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0054] The term "embodiment" as used herein means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0055] In this application, "at least one (item)" means one or more, "more than one" means two or more, "at least two (items)" means two or three or more, and "and / or" is used to describe the relationship between related objects, indicating that there can be three relationships. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the related objects before and after are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items. For example, at least one (item) of a, b, or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c".

[0056] First, let's introduce the terms used in the embodiments of this application:

[0057] A 3D morphable model (3DMM) library consists of two parts: an expression basis and a shape basis. 3DMM libraries can be linear or bilinear. A typical bilinear 3DMM library can include multiple expressions for each of k people. For example, a typical bilinear 3DMM library includes k rows and f columns of expressions, where each row represents the same person, and each column in a row corresponds to a different expression. This can be understood as follows: once a 3DMM library is provided, a human face of arbitrary shape with arbitrary expressions can be parameterized using this 3DMM library.

[0058] Generally, when describing a face with no facial expression (i.e., a 3D model) using the 3DMM library, it can be done by mu + ∑(Pface). i -mu)*α i Result. Here, mu is the average face under natural expression, and Pface... i It is the i-th principal component of the face shape, α i It represents the weight of the i-th principal component of the face shape, which is the face-shaping parameter. Pface i It can be called the i-th deformation basis, α i These can be referred to as the coefficients or weights of the i-th deformation basis. In this application, any deformation basis corresponds to N three-dimensional vectors, and the N three-dimensional vectors corresponding to any deformation basis correspond one-to-one with the N vertices of a three-dimensional model (e.g., a three-dimensional face model). Any deformation basis corresponds to a facial image with relatively large variations compared to the average face. The N vertices of the three-dimensional model can be points on the surface of the three-dimensional model, and these N vertices are used to represent the external contour of the three-dimensional model. In this application, any expression basis corresponds to N three-dimensional vectors, and the N three-dimensional vectors corresponding to any expression basis correspond one-to-one with the N vertices of the three-dimensional model. The N three-dimensional vectors corresponding to any expression basis correspond one-to-one with the N three-dimensional vectors corresponding to any deformation basis.

[0059] It should be understood that a face with any expression can be represented by p + ∑(Pface). i -p)*α i We obtain the expression. Here, p is the face shape (i.e., the expression base) under any given expression, Pface. i It is the i-th deformation basis, α i It represents the weight or coefficient of the i-th deformation basis. By combining different expression bases and deformation bases, face shapes with arbitrary expressions can be obtained.

[0060] Current facial expressions, such as those in augmented reality (AR), are typically displayed using deformation and expression bases, which involves recording the relative vectors of each point. However, 3D models with expressions obtained in this way lack realism and naturalness. Therefore, there is a need to investigate how to obtain more realistic and natural 3D models with arbitrary expressions. The 3D model expression retargeting method provided in this application can output more realistic and natural 3D models with arbitrary expressions. In addition, the 3D model expression retargeting method provided in this application does not require regenerating the 3D model, has low computational cost, and can also avoid clipping problems in the output 3D model.

[0061] The following section, with reference to the accompanying drawings, describes the facial expression retargeting method for 3D models provided in this application.

[0062] Figure 1 This is a flowchart illustrating a three-dimensional model facial expression retargeting method provided in an embodiment of this application. Figure 1 As shown, the method includes:

[0063] 101. The image processing device acquires the first expression basis and multiple deformation basis of the three-dimensional model.

[0064] The aforementioned image processing device can be any electronic device with computing and storage capabilities, such as a terminal device or a server. For example, the image processing device can be a mobile phone, tablet computer, desktop computer, laptop computer, multimedia playback device, wearable device, or other terminal device. As another example, the image processing device can be a server. In some embodiments, the user can utilize the terminal device to perform... Figure 1 The method flow in the code is used to display the output 3D model with a first facial expression. In some embodiments, the server executes... Figure 1 The method described herein uses a first expression base and multiple deformation bases to output a 3D model with a first expression to a terminal device. In other words, the server can generate a 3D model with a first expression using the first expression base and multiple deformation bases, and send it to the terminal device. The terminal device can display the 3D model with the first expression from the server. The 3D model can be a 3D human face model, an animal face model, or a virtual avatar face model; this application does not limit the scope of the limitation.

[0065] In some embodiments, the N three-dimensional vectors corresponding to the first expression basis correspond one-to-one with the N vertices of the three-dimensional model, and the N three-dimensional vectors corresponding to any deformation basis among the multiple deformation bases correspond one-to-one with the N vertices of the three-dimensional model. The N three-dimensional vectors corresponding to any deformation basis can be regarded as the relative displacement vectors of the N vertices of the three-dimensional model, that is, each three-dimensional vector is the relative displacement vector of a vertex. The N three-dimensional vectors corresponding to any expression basis can also be regarded as the relative displacement vectors of the N vertices of the three-dimensional model, that is, each three-dimensional vector is the relative displacement vector of a vertex. The first expression basis can be any expression basis of the three-dimensional model. The multiple deformation bases mentioned above can be all or part of the deformation bases of the three-dimensional model.

[0066] One possible implementation of step 101 is as follows: read the data information of the deformation basis and expression basis of the above-mentioned 3D model to obtain the first expression basis and the above-mentioned plurality of deformation basis. The method provided in this application is applicable only if the image processing device can acquire the expression basis and deformation basis of the 3D model, for example, the image processing device stores the expression basis and deformation basis of the 3D model. In some embodiments, the image processing device can receive a 3D model input by a user and read the data information of the 3D model containing the expression basis and deformation basis to obtain the first expression basis and the above-mentioned plurality of deformation basis. For example, the user copies the 3D model to the image processing device via a removable disk (e.g., a USB flash drive). Another example is that the user copies or transmits a 3D model from another device to the image processing device via Bluetooth connection, data cable connection, etc. The image processing device can download the 3D model (with data information of expression basis and deformation basis) from a server via a network. The image processing device can also obtain the 3D model with data information of expression basis and deformation basis through other means, which are not limited in this application.

[0067] 102. The image processing device outputs a three-dimensional model with the first expression using a first expression basis and multiple deformation basis bases; during the process of outputting the three-dimensional model with the first expression using the first expression basis and multiple deformation basis bases, the coefficient of the first deformation basis base, which has a risk of conflict with the first expression basis base, is reduced.

[0068] The aforementioned first expression can be any dynamic expression, such as a dynamic closed-eye expression, a dynamic open-mouth expression, a dynamic open-eye expression, etc. The image processing device outputs a 3D model with the first expression by displaying the 3D model with the first expression through an output device (e.g., a display screen). For example, the image processing device (e.g., a desktop computer) displays (or plays) the 3D model with the first expression through a monitor. The image processing device (e.g., a server) outputs a 3D model with the first expression by sending the 3D model with the first expression to a terminal device through an output interface. For example, the server sends a 3D model with the first expression obtained using the first expression base and multiple deformation bases to a terminal device (e.g., a laptop, mobile phone, etc.) through a communication interface.

[0069] One approach to outputting a 3D model with a first expression using a first expression basis and multiple deformation basis bases involves changing the coefficients of the first expression basis (e.g., changing it from 0 to 1) while keeping the coefficients of each deformation basis constant. In this approach, changing the coefficients of the first expression basis (e.g., changing it from 0 to 1) allows for the output of an animation of that first expression basis, i.e., an output 3D model with the first expression. However, this approach suffers from clipping issues, resulting in an unrealistic and unnatural output 3D model. The solution provided in this application involves changing the coefficients of the first expression basis and reducing the coefficients of the first deformation basis that may conflict with it during the output process. By reducing the coefficients of the first deformation basis that may conflict with it, clipping issues in the output 3D model with the first expression can be avoided, resulting in a more realistic and natural output 3D model.

[0070] In some embodiments, during the process of outputting the aforementioned 3D model with a first expression using a first expression basis and multiple deformation bases, the coefficients of the first deformation bases that have a conflict risk with the first expression basis are reduced from an initial parameter to 0 or a target value. The initial parameter is a real number greater than 0, and the target value is a real number greater than 0 and less than the initial parameter. Reducing the coefficients of the first deformation bases from the initial parameter to 0 or the target value during the process of outputting the 3D model with a first expression using the first expression basis and multiple deformation bases gradually reduces the adverse effects of the first deformation bases and makes the output first expression more realistic and natural. In these embodiments, during the process of outputting the aforementioned 3D model with a first expression using the first expression basis and multiple deformation bases, the coefficients of each deformation base that does not have a conflict risk with the first expression basis remain unchanged. Keeping the coefficients of the deformation bases that do not have a conflict risk with the first expression basis unchanged avoids significant changes to the output 3D model, i.e., avoids 3D model distortion.

[0071] In one possible implementation, during the process of outputting a 3D model with a first expression using a first expression basis and multiple deformation basis bases, the coefficients of the first deformation basis base are reduced from initial parameters to 0 according to a preset change curve. That is, the image processing device can, during the process of outputting a 3D model with a first expression using the first expression basis and multiple deformation basis bases, change the coefficients of the first deformation basis base back to 0 from the current initial parameters according to a preset change curve, such as by accelerating, decelerating, accelerating then decelerating, or decelerating then accelerating. In practical applications, users can configure the required preset change curve according to their actual needs.

[0072] In some embodiments, before performing step 102, the image processing apparatus may perform the following operation: determine a first deformation basis among a plurality of deformation bases that has a risk of conflict with a first expression basis.

[0073] The first deformation basis is merely an example of a deformation basis among a plurality of deformation bases that is at risk of conflicting with the first expression basis. In these embodiments, the image processing apparatus may determine each deformation basis (including the first deformation basis) among a plurality of deformation bases that is at risk of conflicting with the first expression basis, so as to reduce the coefficients of each deformation basis at risk of conflicting with the first expression basis during the process of outputting a three-dimensional model with the first expression using the first expression basis and the plurality of deformation bases.

[0074] In one possible implementation, determining the first deformable basis that poses a conflict risk with the first expression basis among multiple deformable bases can be as follows: If the ratio of the number of first three-dimensional vectors to the number of non-zero three-dimensional vectors in the N three-dimensional vectors corresponding to the first deformable basis is greater than a first threshold, then the first deformable basis is determined to have a conflict risk with the first expression basis. The N three-dimensional vectors corresponding to the first deformable basis and the N three-dimensional vectors corresponding to the first expression basis are one-to-one correspondences, and each corresponds to one of the N vertices of the three-dimensional model. Both the first three-dimensional vector and its corresponding three-dimensional vector in the N three-dimensional vectors corresponding to the first expression basis are non-zero vectors, and the angle between the first three-dimensional vector and its corresponding three-dimensional vector in the N three-dimensional vectors corresponding to the first expression basis is less than a second threshold. It should be understood that a similar method can be used to determine whether each deformable basis among the multiple deformable bases poses a conflict risk with the first expression basis. The first threshold can be 0.85, 0.86, 0.88, 0.90, 0.95, etc., and this application does not limit it. The second threshold can be 5 degrees, 6 degrees, 7 degrees, 8 degrees, 9 degrees, 10 degrees, etc., and this application does not limit it. In this implementation, if the ratio of the number of the first three-dimensional vectors to the number of non-zero three-dimensional vectors in the N three-dimensional vectors corresponding to the first deformation basis is greater than the first threshold, it is determined that there is a risk of conflict between the first deformation basis and the first expression basis; it can accurately determine whether there is a risk of conflict between the first deformation basis and the first expression basis.

[0075] In this embodiment, during the process of outputting a 3D model with a first expression using a first expression basis and multiple deformation bases, the coefficients of the first deformation base, which has a risk of conflict with the first expression basis, are reduced. Reducing the coefficients of the first deformation base avoids clipping issues in the output 3D model with the first expression, making the output 3D model more realistic and natural. Furthermore, during the process of outputting a 3D model with a first expression using the first expression basis and multiple deformation bases, it is not necessary to regenerate the 3D model, resulting in low computational cost.

[0076] Figure 2 A flowchart of another three-dimensional model facial expression retargeting method provided in this application embodiment. Figure 2 The method flow is as follows Figure 1 One possible implementation of the method flow in [the code / process]. For example... Figure 2 As shown, the method includes:

[0077] 201. The image processing device acquires the first expression basis and multiple deformation basis of the three-dimensional model.

[0078] In practical applications, users can first ensure that the facial expression basis and deformation basis are created in the 3D model (i.e., the 3D model contains data information of the facial expression basis and deformation basis), and then use an image processing device to execute... Figure 2The method flow is as follows. For the implementation of step 201, please refer to the implementation of step 101.

[0079] 202. The image processing device determines the number of the first three-dimensional vectors and the number of non-zero three-dimensional vectors among the N three-dimensional vectors corresponding to the first deformation basis.

[0080] The N three-dimensional vectors corresponding to the first deformation basis and the N three-dimensional vectors corresponding to the first expression basis are one-to-one corresponding to the N vertices of the three-dimensional model. In the N three-dimensional vectors corresponding to the first deformation basis, each first three-dimensional vector and its corresponding three-dimensional vector in the N three-dimensional vectors corresponding to the first expression basis are both non-zero vectors, and the angle between each first three-dimensional vector and its corresponding three-dimensional vector in the N three-dimensional vectors corresponding to the first expression basis is less than a second threshold. Any two first three-dimensional vectors correspond to two different vertices of the three-dimensional model. It can be understood that each first three-dimensional vector in the N three-dimensional vectors corresponding to the first deformation basis satisfies the following conditions: each first three-dimensional vector and its corresponding three-dimensional vector in the N three-dimensional vectors corresponding to the first expression basis are both non-zero vectors, and the angle between each first three-dimensional vector and its corresponding three-dimensional vector in the N three-dimensional vectors corresponding to the first expression basis is less than the second threshold. That is, if a certain three-dimensional vector corresponding to the first deformation basis and its corresponding three-dimensional vector in the N three-dimensional vectors corresponding to the first expression basis are both non-zero vectors, and the angle between this three-dimensional vector and its corresponding three-dimensional vector in the N three-dimensional vectors corresponding to the first expression basis is less than the second threshold, then this three-dimensional vector is a first three-dimensional vector. In some embodiments, each of the N three-dimensional vectors corresponding to the first deformation basis further satisfies the following conditions: the ratio of the length of each first three-dimensional vector to the length of its corresponding three-dimensional vector among the N three-dimensional vectors corresponding to the first expression basis is greater than 1 and less than a third threshold, where the third threshold is a real number greater than 1, such as 1.1, 1.2, etc.; or, the ratio of the length of each first three-dimensional vector to the length of its corresponding three-dimensional vector among the N three-dimensional vectors corresponding to the first expression basis is less than 1 and greater than a fourth threshold, where the fourth threshold is a real number less than 1, such as 0.85, 0.9, etc.; or, the ratio of the length of each first three-dimensional vector to the length of its corresponding three-dimensional vector among the N three-dimensional vectors corresponding to the first expression basis is equal to 1.

[0081] 203. If the ratio of the number of the first three-dimensional vectors to the number of non-zero three-dimensional vectors in the N three-dimensional vectors corresponding to the first deformation basis is greater than the first threshold, the image processing device determines that there is a risk of conflict between the first deformation basis and the first expression basis.

[0082] Steps 202 and 203 describe the method by which the image processing apparatus determines whether the first deformation basis has a conflict risk with the first expression basis. It should be understood that the image processing apparatus may use a similar method to determine whether any deformation basis has a conflict risk with the first expression basis. For example, if, among the N three-dimensional vectors corresponding to a third deformation basis (containing multiple deformation bases in a three-dimensional model), the ratio of the number of first three-dimensional vectors to the number of non-zero three-dimensional vectors is greater than a first threshold, the image processing apparatus determines that the third deformation basis has a conflict risk with the first expression basis. In some embodiments, the image processing apparatus may determine each of the multiple deformation bases that has a conflict risk with the first expression basis.

[0083] 204. The image processing device outputs a three-dimensional model with the first expression using a first expression basis and multiple deformation basis bases; during the process of outputting the three-dimensional model with the first expression using the first expression basis and multiple deformation basis bases, the coefficient of the first deformation basis base, which has a risk of conflict with the first expression basis base, is reduced.

[0084] One possible implementation of step 204 is as follows: In the process of outputting a 3D model with the first expression using the first expression base and multiple deformation bases (i.e., the actual animation process), when the first expression base takes effect, simultaneously perform a reverse animation with a certain interpolation on all deformation bases that have a conflict risk with the first expression base, that is, reduce the weights or coefficients of all deformation bases (e.g., the first deformation base) that have a conflict risk with the first expression base; the weights or coefficients of other deformation bases that do not conflict with the first expression base can remain at their original values. When the first expression base takes effect, it can be a process in which the coefficients of the first expression base change, for example, the coefficients of the first expression base change from 0 back to 1. Performing a reverse animation with a certain interpolation on all deformation bases that have a conflict risk with the first expression base can be: changing the weights or coefficients of all deformation bases that have a conflict risk with the first expression base from their original values ​​to 0 using a preset change curve. For example, changing the weight or coefficient of the first deformation basis from its original value to 0 using a preset change curve can make the weight or coefficient of the first deformation basis change from its original value to 0 in any of the following ways: acceleration, deceleration, deceleration followed by deceleration, or deceleration followed by acceleration.

[0085] Figure 3 This diagram illustrates the changes in the coefficients of expression basis 1, deformation basis 1, and deformation basis 2 when expression basis 1 is active. Expression basis 1 can be considered as the first expression basis, deformation basis 1 can be considered as the first deformation basis, and deformation basis 2 is any deformation basis that does not pose a conflict risk with expression basis 1. Figure 3As shown, when expression basis 1 takes effect (i.e., the process of the coefficient of expression basis 1 changing), the coefficient of expression basis 1 changes from 0 back to 1, the coefficient of deformation basis 1 changes from the initial coefficient i1 to 0 according to the mapping relationship of the preset change curve, and the coefficient of deformation basis 2 remains unchanged. i1 is a real number greater than 0. It should be understood that the coefficient change process of deformation basis 1 is an example of the coefficient change process of deformation basis that has a risk of conflict with expression basis 1; the coefficient change process of deformation basis 2 is an example of the coefficient change process of deformation basis that does not have a risk of conflict with expression basis 1. That is to say, when expression basis 1 takes effect, the coefficient change process of any deformation basis that has a risk of conflict with expression basis 1 is similar to the coefficient change process of deformation basis 1, and the coefficient change process of any deformation basis that does not have a risk of conflict with expression basis 1 is similar to the coefficient change process of deformation basis 2.

[0086] Figure 4 An example is shown of outputting a 3D model with the first expression using a first expression basis and multiple deformation basis bases. Figure 4 In the diagram, 401 shows the 3D model corresponding to the expression basis - closed eyes at different coefficients. The 3D model on the left is the 3D model corresponding to the expression basis - closed eyes coefficient of 0, and the 3D model on the right is the 3D model corresponding to the expression basis - closed eyes coefficient of 1. 402 shows the standard 3D model (left) and the 3D model corresponding to the deformation basis - eye curvature (right). 403 shows the standard 3D model (left) and the 3D model corresponding to the deformation basis - eye curvature (right). 404 shows the standard 3D model (left) and the 3D model corresponding to the deformation basis - eye width (right). Figure 4 Step 1 can be to determine whether each deformation basis, such as deformation basis-eye curvature, deformation basis-eye bending, and deformation basis-eye width, conflicts with the expression basis-closed eyes. Figure 4 In the diagram, "judged as conflict" on the left indicates a risk of conflict between the deformation base - eye curvature and the expression base - closed eyes, while "judged as conflict" on the right indicates a risk of conflict between the deformation base - eye curvature and the expression base - closed eyes, and "judged as no conflict" indicates no risk of conflict between the deformation base - eye width and the expression base - closed eyes. Figure 4Step 2 can be to output a 3D model with a closed-eye expression by using the expression base - closed eyes and the deformation base - eye curvature, deformation base - eye slant, and deformation base - eye width respectively. 405 represents a 3D model with a closed-eye expression output by using the expression base - closed eyes and the deformation base - eye curvature with a non-zero coefficient. 406 represents a 3D model with a closed-eye expression output by using the expression base - closed eyes and the deformation base - eye curvature with a zero coefficient. 407 represents a 3D model with a closed-eye expression output by using the expression base - closed eyes and the deformation base - eye slant with a non-zero coefficient. 408 represents a 3D model with a closed-eye expression output by using the expression base - closed eyes and the deformation base - eye slant with a zero coefficient. 409 represents a 3D model with a closed-eye expression output by using the expression base - closed eyes and the deformation base - eye width with a non-zero coefficient. Images 405 and 407 show the pre-redirection effect of the 3D model (coefficients of deformation bases that risk conflict with the expression base - closed eyes remain unchanged); images 406 and 408 show the post-redirection effect of the 3D model (coefficients of deformation bases that risk conflict with the expression base - closed eyes become 0). The pre-redirection effect of the 3D model can refer to the output 3D model using the expression base and deformation bases with unchanged coefficients. The post-redirection effect of the 3D model can refer to the output 3D model using the expression base, deformation bases with coefficients of 0 that risk conflict with that expression base, and deformation bases with unchanged coefficients that do not risk conflict with that expression base. (Comparison) Figure 4 As shown in points 405 and 406 (or 407 and 408), the 3D model before redirection had a clipping problem, while the 3D model after redirection did not. Furthermore, the 3D model after redirection is more realistic and natural than the 3D model before redirection. Therefore, by executing... Figure 4 The methodology can output more realistic and natural 3D models with genuine facial expressions.

[0087] In this embodiment, during the process of outputting a 3D model with a first expression using a first expression basis and multiple deformation bases, the coefficients of the first deformation base, which has a risk of conflict with the first expression basis, are reduced. This avoids clipping problems in the output 3D model with the first expression, making the output 3D model more realistic and natural. Furthermore, during the process of outputting a 3D model with a first expression using the first expression basis and multiple deformation bases, it is not necessary to regenerate the 3D model, resulting in low computational cost.

[0088] The following example, using the determination of whether there is a conflict risk between the first expression base and the first deformation base, will introduce how to determine which deformation bases each expression base has a conflict risk with.

[0089] Figure 5 This is a flowchart illustrating a method for determining whether there is a conflict risk between a first expression base and a first deformation base, as provided in an embodiment of this application. Figure 5 As shown, the method includes:

[0090] 501. The image processing device reads the first expression basis and forms a first array, and reads the first deformation basis and forms a second array.

[0091] The first and second arrays can both be N rows and 3 columns, with each row corresponding to a 3D vector. The first expression basis and the first deformation basis each correspond to N 3D vectors. The N 3D vectors corresponding to the first deformation basis correspond one-to-one with the N 3D vectors corresponding to the first expression basis, and each corresponds to one of the N vertices of the 3D model. Similarly, the N 3D vectors corresponding to the first deformation basis correspond one-to-one with the N vertices of the 3D model, and the N 3D vectors corresponding to the first expression basis correspond one-to-one with the N vertices of the 3D model. Figure 5 The same image processing device is used to execute each step.

[0092] In practical applications, the image processing device can read both the expression basis and the deformation basis, placing the three-dimensional vector corresponding to each expression basis into an array (i.e., one array corresponds to one expression basis), and placing the three-dimensional vector corresponding to each deformation basis into another array (i.e., one array corresponds to one deformation basis). In other words, the image processing device uses the three-dimensional vectors corresponding to each expression basis to form an array, and uses the three-dimensional vectors corresponding to each deformation basis to form another array.

[0093] 502. Calculate the number of non-zero vectors among the N three-dimensional vectors corresponding to the first deformation basis to obtain the first number.

[0094] In this embodiment, the order of step 502 with other steps is not limited. For example, the image processing device may execute step 502 after executing step 504.

[0095] 503. Combine the three-dimensional vectors corresponding to the same vertex in the first and second arrays into one array to obtain N arrays.

[0096] There are N arrays, each with 1 row and 6 columns, where each row corresponds to a vertex. Each of the N arrays contains two 3D vectors: one 3D vector corresponds to the first expression basis, and the other 3D vector corresponds to the first deformation basis.

[0097] In some embodiments, step 501 is optional, and step 503 can be replaced by: placing the three-dimensional vectors corresponding to the same vertex in the three-dimensional vectors corresponding to the first expression basis and the three-dimensional vectors corresponding to the first deformation basis into an array to obtain N arrays.

[0098] 504. Calculate the number of all N arrays that simultaneously satisfy the target condition, and obtain the second number.

[0099] An array that satisfies the target conditions can be: containing two non-zero 3D vectors; having an angle between the two 3D vectors less than a second threshold; and having a ratio of the longer to the shorter vector among the two 3D vectors less than a third threshold. The condition that the angle between the two 3D vectors is less than the second threshold can be replaced with: having an angle greater than (180 - the second threshold). The second threshold can range from 5 to 10 degrees. The condition that the ratio of the longer to the shorter vector among the two 3D vectors is less than the third threshold can be replaced with: having a ratio of the shorter to the longer vector among the two 3D vectors greater than a fourth threshold. If the lengths of the two 3D vectors are equal, the condition that the ratio of the longer to the shorter vector among the two 3D vectors is less than the third threshold is also met.

[0100] 505. Calculate the ratio of the first number to the second number.

[0101] 506. If the ratio of the first number to the second number is greater than the first threshold, it is determined that there is a risk of conflict between the first deformation basis and the first expression basis.

[0102] 507. If the ratio of the first number to the second number is less than or equal to the first threshold, it is determined that there is no risk of conflict between the first deformation basis and the first expression basis.

[0103] It should be understood that the image processing device may employ a similar approach. Figure 5 The method in the process determines which deformation bases are at risk of conflict with each expression base.

[0104] In this embodiment, it is possible to accurately determine whether there is a risk of conflict between the first deformation base and the first expression base.

[0105] Figure 6 This is a schematic diagram of the structure of an image processing apparatus provided in an embodiment of this application. Figure 6 As shown, the image processing apparatus includes:

[0106] Acquisition unit 601 is used to acquire the first expression basis and multiple deformation basis of the three-dimensional model;

[0107] Output unit 602 is used to output the three-dimensional model with the first expression using the first expression basis and the plurality of deformation basis; in the process of outputting the three-dimensional model with the first expression using the first expression basis and the plurality of deformation basis, the coefficient of the first deformation basis that has a conflict risk with the first expression basis among the plurality of deformation basis is reduced.

[0108] In this embodiment, the output unit uses a first expression basis and multiple deformation basis bases to output a 3D model with a first expression. During the process of outputting the 3D model with the first expression using the first expression basis and multiple deformation basis bases, the coefficients of the first deformation basis, which has a risk of conflict with the first expression basis, are reduced. Reducing the coefficients of the first deformation basis, which has a risk of conflict with the first expression basis, can prevent clipping problems in the output 3D model with the first expression, making the output 3D model more realistic and natural.

[0109] In one possible implementation, during the process of outputting the three-dimensional model with the first expression using the first expression basis and the plurality of deformation basis, the coefficients of the first deformation basis are reduced from the initial parameter to 0, where the initial parameter is a real number greater than 0.

[0110] In one possible implementation, during the process of outputting the three-dimensional model with the first expression using the first expression basis and the plurality of deformation basis, the coefficients of the first deformation basis are reduced to 0 from the initial parameters according to a preset change curve.

[0111] In one possible implementation, during the process of outputting the three-dimensional model with the first expression using the first expression basis and the plurality of deformation basis bases, the coefficients of the deformation basis bases among the plurality of deformation basis bases that do not have a conflict risk with the first expression basis remain unchanged.

[0112] In one possible implementation, among the N three-dimensional vectors corresponding to the first deformation basis, the ratio of the number of first three-dimensional vectors to the number of non-zero three-dimensional vectors is greater than a first threshold; the N three-dimensional vectors corresponding to the first deformation basis correspond one-to-one with the N three-dimensional vectors corresponding to the first expression basis and each corresponds to one of the N vertices of the three-dimensional model; both the first three-dimensional vector and its corresponding three-dimensional vector among the N three-dimensional vectors corresponding to the first expression basis are non-zero vectors; and the angle between the first three-dimensional vector and its corresponding three-dimensional vector among the N three-dimensional vectors corresponding to the first expression basis is less than a second threshold.

[0113] In one possible implementation, the ratio of the length of the first three-dimensional vector to the length of its corresponding three-dimensional vector among the N three-dimensional vectors corresponding to the first expression base is greater than 1 and less than a third threshold, where the third threshold is a real number greater than 1; or, the ratio of the length of the first three-dimensional vector to the length of its corresponding three-dimensional vector among the N three-dimensional vectors corresponding to the first expression base is less than 1 and greater than a fourth threshold, where the fourth threshold is a real number less than 1; or, the ratio of the length of the first three-dimensional vector to the length of its corresponding three-dimensional vector among the N three-dimensional vectors corresponding to the first expression base is equal to 1.

[0114] In one possible implementation, the above-mentioned apparatus further includes: a determining unit 603, used to determine the first deformation basis among the plurality of deformation basis bases that has a risk of conflict with the first expression basis.

[0115] In one possible implementation, the determining unit 603 is specifically used to determine that there is a risk of conflict between the first deformation basis and the first expression basis when the ratio of the number of first three-dimensional vectors to the number of non-zero three-dimensional vectors in the N three-dimensional vectors corresponding to the first deformation basis is greater than a first threshold; the N three-dimensional vectors corresponding to the first deformation basis and the N three-dimensional vectors corresponding to the first expression basis are one-to-one corresponding and all correspond to the N vertices of the three-dimensional model; the first three-dimensional vector and its corresponding three-dimensional vector in the N three-dimensional vectors corresponding to the first expression basis are both non-zero vectors; and the angle between the first three-dimensional vector and its corresponding three-dimensional vector in the N three-dimensional vectors corresponding to the first expression basis is less than a second threshold.

[0116] In one possible implementation, the determining unit 603 is further configured to determine the number of the first three-dimensional vectors and the number of non-zero three-dimensional vectors among the N three-dimensional vectors corresponding to the first deformation basis.

[0117] In one possible implementation, the acquisition unit 601 is specifically used to read the data information of the deformation basis and expression basis of the above-mentioned three-dimensional model, and obtain the above-mentioned first expression basis and the above-mentioned multiple deformation basis.

[0118] It should be understood that the division of the various units in the above image processing device is merely a logical functional division. In actual implementation, they can be fully or partially integrated into a single physical entity, or they can be physically separated. For example, each of the above units can be a separate processing element, or they can be integrated into the same chip. Alternatively, they can be stored as program code in the controller's storage element, and called and executed by a processing element of the processor. Furthermore, the units can be integrated together or implemented independently. The processing element here can be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method or each of the above units can be completed through the integrated logic circuits in the hardware of the processor element or through software instructions. This processing element can be a general-purpose processor, such as a central processing unit (CPU), or one or more integrated circuits configured to implement the above method, such as one or more application-specific integrated circuits (ASICs), or one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs), etc.

[0119] Figure 7 This is a schematic diagram of a server structure provided in an embodiment of this application. The server 700 can vary significantly due to different configurations or performance. It may include one or more central processing units (CPUs) 722 (e.g., one or more processors) and a memory 732, and one or more storage media 730 (e.g., one or more mass storage devices) for storing application programs 742 or data 744. The memory 732 and storage media 730 can be temporary or persistent storage. The program stored in the storage media 730 may include one or more modules (not shown in the diagram), each module including a series of instruction operations on the server. Furthermore, the CPU 722 may be configured to communicate with the storage media 730 and execute the series of instruction operations in the storage media 730 on the server 700. The server 700 may be the image processing apparatus provided in this application.

[0120] Server 700 may also include one or more power supplies 726, one or more wired or wireless network interfaces 750, one or more input / output interfaces 758, and / or one or more operating systems 741, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0121] The steps performed by the image processing device in the above embodiments can be based on this Figure 7 The server architecture shown. Specifically, the CPU 722 can implement... Figure 6 The function of unit 603 is determined in the middle. Input / output interface 758 can implement... Figure 6 The functions of the acquisition unit 601 and the output unit 602 are described.

[0122] Figure 8 This is a block diagram of a partial structure of another image processing apparatus provided in an embodiment of this application. For example... Figure 8 As shown, the image processing device 800 may include a processor 801, a memory 802, an input device 803, an output device 804, and a bus 805. The processor 801, memory 802, input device 803, and output device 804 are interconnected via the bus 805.

[0123] The processor 801 can be a general-purpose CPU, microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits to execute relevant programs to implement the technical solutions provided in the embodiments of the present invention. The processor 801 can achieve... Figure 6 The function of the determination unit 603 in the middle.

[0124] The memory 802 may be a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 802 may store the operating system and other application programs. Program code used to implement the functions required by the modules and components included in the image processing apparatus provided in this application embodiment through software or firmware, or to implement the methods provided in the method embodiments of this application, is stored in the memory 802. The processor 801 reads the code from the memory 802 to execute the operations required by the modules and components included in the image processing apparatus, or to execute the methods provided in the embodiments of this application.

[0125] The input device 803, corresponding to the acquisition unit 601, is used to acquire the first expression basis and multiple deformation basis of the three-dimensional model.

[0126] Output device 804, corresponding to output unit 602, is used to output the aforementioned three-dimensional model with the first expression.

[0127] Bus 805 may include a path for transmitting information between various components of the image processing device (e.g., processor 801, memory 802, input device 803, output device 804).

[0128] It should be noted that, although Figure 8 The image processing apparatus 800 shown only illustrates the processor 801, memory 802, input device 803, output device 804, and bus 805. However, in specific implementations, those skilled in the art should understand that the image processing apparatus 800 also includes other devices necessary for normal operation. Furthermore, depending on specific needs, those skilled in the art should understand that the image processing apparatus 800 may also include hardware devices for implementing other additional functions. Moreover, those skilled in the art should understand that the image processing apparatus 800 may only include the devices necessary for implementing the embodiments of this application, and may not necessarily include... Figure 8 All the devices shown.

[0129] In the embodiments of this application, a computer-readable storage medium is provided, which stores a computer program that, when executed by a processor, implements the method provided in the foregoing embodiments.

[0130] This application provides a computer program product containing instructions that, when run on a computer, cause the computer to perform the methods provided in the foregoing embodiments.

[0131] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method of expression re-targeting for a three-dimensional model, the method comprising: include: Obtain a first expression basis and multiple deformation basis of a 3D model. The first expression basis corresponds to a face shape. Among the multiple deformation basis, one deformation basis corresponds to N 3D vectors, and the N 3D vectors corresponding to one deformation basis correspond one-to-one with the N vertices of the 3D model. The three-dimensional model with the first expression is output using the first expression basis and the plurality of deformation basis bases. During the process of outputting the three-dimensional model with the first expression using the first expression basis and the plurality of deformation basis bases, the coefficient of the first deformation basis, which has a risk of conflict with the first expression basis, is reduced. Among the N three-dimensional vectors corresponding to the first deformation basis, the ratio of the number of first three-dimensional vectors to the number of non-zero three-dimensional vectors is greater than a first threshold. The N three-dimensional vectors corresponding to the first deformation basis correspond one-to-one with the N three-dimensional vectors corresponding to the first expression basis and all correspond to the N vertices of the three-dimensional model. The first three-dimensional vector and its corresponding three-dimensional vector among the N three-dimensional vectors corresponding to the first expression basis are both non-zero vectors. The angle between the first three-dimensional vector and its corresponding three-dimensional vector among the N three-dimensional vectors corresponding to the first expression basis is less than a second threshold.

2. The method of claim 1, wherein, In the process of outputting the three-dimensional model with the first expression using the first expression basis and the plurality of deformation basis, the coefficients of the first deformation basis are reduced from the initial parameter to 0, where the initial parameter is a real number greater than 0.

3. The method according to claim 2, characterized in that, In the process of outputting the three-dimensional model with the first expression using the first expression basis and the plurality of deformation basis, the coefficients of the first deformation basis are reduced to 0 from the initial parameters according to a preset change curve.

4. The method according to any one of claims 1 to 3, characterized in that, In the process of outputting the three-dimensional model with the first expression using the first expression basis and the plurality of deformation basis bases, the coefficients of the deformation basis bases that do not have a conflict risk with the first expression basis remain unchanged.

5. The method according to any one of claims 1 to 3, characterized in that, The ratio of the length of the first three-dimensional vector to the length of its corresponding three-dimensional vector among the N three-dimensional vectors corresponding to the first expression basis is greater than 1 and less than a third threshold, wherein the third threshold is a real number greater than 1. Alternatively, the ratio of the length of the first three-dimensional vector to the length of the corresponding three-dimensional vector among the N three-dimensional vectors corresponding to the first expression basis is less than 1 and greater than a fourth threshold, where the fourth threshold is a real number less than 1. Alternatively, the ratio of the length of the first three-dimensional vector to the length of its corresponding three-dimensional vector among the N three-dimensional vectors corresponding to the first expression basis is equal to 1.

6. The method according to any one of claims 1 to 3, characterized in that, Before outputting the 3D model with the first expression using the first expression basis and the plurality of deformation basis bases, the method further includes: Identify the first deformation basis among the plurality of deformation basis bases that has a conflict risk with the first expression basis.

7. An image processing apparatus, characterized in that, include: The acquisition unit is used to acquire a first expression basis and multiple deformation basis of a three-dimensional model. The first expression basis corresponds to a face shape. Among the multiple deformation basis, one deformation basis corresponds to N three-dimensional vectors, and the N three-dimensional vectors corresponding to one deformation basis correspond one-to-one with the N vertices of the three-dimensional model. An output unit is used to output a 3D model with a first expression using the first expression basis and the plurality of deformation basis bases. During the process of outputting the 3D model with the first expression using the first expression basis and the plurality of deformation basis bases, the coefficient of the first deformation basis, which has a risk of conflict with the first expression basis, is reduced. Among the N 3D vectors corresponding to the first deformation basis, the ratio of the number of first 3D vectors to the number of non-zero 3D vectors is greater than a first threshold. The N 3D vectors corresponding to the first deformation basis correspond one-to-one with the N 3D vectors corresponding to the first expression basis and all correspond to the N vertices of the 3D model. Both the first 3D vector and its corresponding 3D vector among the N 3D vectors corresponding to the first expression basis are non-zero vectors. The angle between the first 3D vector and its corresponding 3D vector among the N 3D vectors corresponding to the first expression basis is less than a second threshold.

8. The apparatus according to claim 7, characterized in that, In the process of outputting the three-dimensional model with the first expression using the first expression basis and the plurality of deformation basis, the coefficients of the first deformation basis are reduced from the initial parameter to 0, where the initial parameter is a real number greater than 0.

9. The apparatus according to claim 8, characterized in that, In the process of outputting the three-dimensional model with the first expression using the first expression basis and the plurality of deformation basis, the coefficients of the first deformation basis are reduced to 0 from the initial parameters according to a preset change curve.

10. The apparatus according to any one of claims 7 to 9, characterized in that, In the process of outputting the three-dimensional model with the first expression using the first expression basis and the plurality of deformation basis bases, the coefficients of the deformation basis bases that do not have a conflict risk with the first expression basis remain unchanged.

11. The apparatus according to any one of claims 7 to 9, characterized in that, The ratio of the length of the first three-dimensional vector to the length of its corresponding three-dimensional vector among the N three-dimensional vectors corresponding to the first expression basis is greater than 1 and less than a third threshold, wherein the third threshold is a real number greater than 1. Alternatively, the ratio of the length of the first three-dimensional vector to the length of the corresponding three-dimensional vector among the N three-dimensional vectors corresponding to the first expression basis is less than 1 and greater than a fourth threshold, where the fourth threshold is a real number less than 1. Alternatively, the ratio of the length of the first three-dimensional vector to the length of its corresponding three-dimensional vector among the N three-dimensional vectors corresponding to the first expression basis is equal to 1.

12. The apparatus according to any one of claims 7 to 9, characterized in that, The device further includes: A determining unit is used to determine the first deformation basis among the plurality of deformation basis bases that has a risk of conflict with the first expression basis.

13. An image processing apparatus, characterized in that, The method includes a memory and a processor; the memory is used to store a program; the processor is used to execute the program stored in the memory, and when the program is executed, the processor is used to perform the method as described in any one of claims 1 to 6.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, the computer program including program instructions that, when executed by a processor, cause the processor to perform the method according to any one of claims 1 to 6.

Images