Deformation migration methods, apparatus, equipment and storage media
By using a non-rigid registration algorithm, deformable object models with consistent shape and topology are generated between the target deformable object model, the source deformable object model, and the deformable object model. The deformation variables are determined and transferred, which solves the problem of low accuracy of mixed shapes caused by topological inconsistency and achieves higher shape accuracy.
Patent Information
- Application Number
- CN202211309589.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-10-25
- Publication Date
- 2026-11-10
- Estimated Expiration
- 2042-10-25
AI Technical Summary
The topological inconsistency between the target expressionless face model and the source expressionless face model results in low accuracy of the blending shape of the target expressionless face model obtained by mapping the triangular meshes of the target expressionless face model and the source expressionless face model.
The target indeformable object model is registered to the source indeformable object model and the object model with deformation using a non-rigid registration algorithm. This generates a first deformable object model and a second deformable object model with the same shape and topology, and determines the deformation variables between them. Based on these deformation variables, the deformation of the object model with deformation is transferred to the target indeformable object model.
It improves the accuracy of the hybrid shape corresponding to the target non-deformable object model, ensures that the triangular meshes correspond one-to-one during the deformation transfer process, avoids the problem of some meshes lacking deformation, and improves the accuracy of the shape.
Smart Images

Figure CN117011501B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of artificial intelligence (AI) technology, and in particular to a deformation transfer method, apparatus, device and storage medium. Background Technology
[0002] Blendshape (BS) refers to a composite face model obtained by combining a neutral face model (i.e., an expressionless face model) with an expression base. In applications such as film and games, BS technology is a supporting technology for three-dimensional (3D) face rendering and the creation of 3D virtual humans.
[0003] The process of obtaining a shape blend (BS) using the deformation transfer algorithm is as follows: A 3D model designer or artist designs a source expressionless face model and an expression model (i.e., a blended shape). Given a target expressionless face model, a non-rigid registration algorithm is used to register the target expressionless face model onto the source expressionless face model, generating a target deformable face model. The shape of this model is consistent with the shape of the source expressionless face model, that is, the character image of the target expressionless face model is transformed into the character image of the source expressionless face model, but the topology of the target expressionless face model is preserved. Based on this target deformable face model, the mapping relationship between the triangular mesh of the target expressionless face model and the triangular mesh of the source expressionless face model can be determined. Then, the deformation variables from the source expressionless face model to the expression model are determined. Based on the mapping relationship between the triangular mesh of the target expressionless face model and the triangular mesh of the source expressionless face model, the above deformation variables are transferred to the target expressionless face model, finally obtaining the expression model (i.e., the blended shape) of the target expressionless face model.
[0004] However, due to the potential inconsistency in topology between the target expressionless face model and the source expressionless face model, the accuracy of the blended shape of the target expressionless face model obtained based on the mapping relationship between the triangular meshes of the target expressionless face model and the triangular meshes of the source expressionless face model is not high. Summary of the Invention
[0005] This application provides a deformation transfer method, apparatus, device, and storage medium, which can improve the accuracy of the mixed shape corresponding to the target non-deformable object model, i.e., the target expressionless human face model.
[0006] This application provides a deformation transfer method, comprising: obtaining a source indeformable object model, a deformable object model corresponding to the source indeformable object model, and a target indeformable object model; deforming the target indeformable object model according to the source indeformable object model to obtain a first deformable object model whose shape is consistent with the shape of the source indeformable object model and whose topology is consistent with the topology of the target indeformable object model; deforming the target indeformable object model according to the deformable object model to obtain a second deformable object model whose shape is consistent with the shape of the deformable object model and whose topology is consistent with the topology of the target indeformable object model; determining the deformation amount from the first deformable object model to the second deformable object model; and transferring the deformation of the deformable object model to the target indeformable object model based on the target indeformable object model and the deformation amount.
[0007] Secondly, embodiments of this application provide a deformation transfer device, comprising: an acquisition module, a deformation module, a determination module, and a transfer module. The acquisition module acquires a source indeformable object model, a deformation-bearing object model corresponding to the source indeformable object model, and a target indeformable object model. The deformation module deforms the target indeformable object model based on the source indeformable object model to obtain a first deformable object model whose shape matches the shape of the source indeformable object model and whose topology matches the topology of the target indeformable object model. It also deforms the target indeformable object model based on the deformation-bearing object model to obtain a second deformable object model whose shape matches the shape of the deformation-bearing object model and whose topology matches the topology of the target indeformable object model. The determination module determines the deformation amount from the first deformable object model to the second deformable object model. The transfer module transfers the deformation of the deformation-bearing object model to the target indeformable object model based on the target indeformable object model and the deformation amount.
[0008] Thirdly, embodiments of this application provide an electronic device, including: a processor and a memory, the memory being used to store a computer program, and the processor being used to call and run the computer program stored in the memory to perform the methods as described in the first aspect or its various implementations.
[0009] Fourthly, embodiments of this application provide a computer-readable storage medium for storing a computer program that causes a computer to perform the methods described in the first aspect or its various implementations.
[0010] Fifthly, embodiments of this application provide a computer program product including computer program instructions that cause a computer to perform the methods as described in the first aspect or its various implementations.
[0011] Sixthly, embodiments of this application provide a computer program that causes a computer to perform the methods as described in the first aspect or its various implementations.
[0012] The technical solution provided by the embodiments of this application is based on the topological consistency between the target non-deformable object model and the deformable object model. In other words, the triangular meshes of the target non-deformable object model and the deformable object model correspond one-to-one, and deformation transfer is based on the triangular meshes. If there is a problem of triangular mesh mismatch, some triangular meshes on the target non-deformable object model will inevitably not have corresponding deformation and will remain unchanged. For example, the forehead area of the target non-deformable object model will not undergo deformation transfer. However, since the topological consistency between the target non-deformable object model and the deformable object model is consistent, the accuracy of the mixed shape corresponding to the target non-deformable object model can be improved. Attached Figure Description
[0013] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0014] Figure 1 This is a schematic diagram illustrating the formation principle of the mixed shape provided in the embodiments of this application;
[0015] Figure 2 This is a schematic diagram illustrating the principle of the NICP algorithm provided in the embodiments of this application;
[0016] Figure 3 This is a schematic diagram illustrating an application scenario according to an embodiment of this application;
[0017] Figure 4 A flowchart of a deformation migration method provided in an embodiment of this application;
[0018] Figure 5 This is a schematic diagram of facial expression transfer provided in an embodiment of this application;
[0019] Figure 6 A schematic diagram illustrating a four-corner grid divided into triangular grids, provided for an embodiment of this application;
[0020] Figure 7 A schematic diagram illustrating an octagonal mesh divided into triangular meshes, provided for an embodiment of this application;
[0021] Figure 8 A schematic diagram of key points of the object model provided in the embodiments of this application;
[0022] Figure 9 The deformation migration effect diagram provided in the embodiments of this application;
[0023] Figure 10A schematic diagram of a deformation migration device 1000 provided in an embodiment of this application;
[0024] Figure 11 This is a schematic block diagram of the electronic device provided in the embodiments of this application. Detailed Implementation
[0025] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0026] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or server that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.
[0027] Before introducing the technical solution of this application, the relevant knowledge of this application will be explained below:
[0028] I. Deformation: refers to changes in shape. In the embodiments of this application, deformation can be changes in facial expressions or changes in object movement, etc. In short, the definition of deformation in the embodiments of this application is not limited.
[0029] II. Object: It can be a human face, an animal face, a person's hand, etc. This application embodiment does not limit this. In addition, before implementing this application embodiment, the faces involved in this application embodiment have been authorized by the user.
[0030] 3. Non-deformable object model: refers to an object model that does not undergo any deformation, such as an expressionless face model, which is also called a neutral face model or a natural expression face model.
[0031] IV. Blending Shapes: This refers to a technique where a single mesh is deformed to achieve a combination of many predefined shapes and any number of them, known as a deformable target in Maya / 3ds Max. For example, a single mesh is the default basic shape, such as an expressionless human face model, i.e., the neutral face model involved in this application. Other shapes of the basic shape are used for blending / deformation, such as different expressions like smiling, frowning, closing eyelids, etc., which are collectively referred to as blending shapes or deformable targets.
[0032] For example, Figure 1 This is a schematic diagram illustrating the formation principle of the mixed shape provided in the embodiments of this application. For example... Figure 1 As shown, after obtaining the neutral face model and M expression bases, a blend shape can be constructed based on the neutral face model and the M expression bases. Specifically, the blend shape can be constructed according to the following formula:
[0033]
[0034] Where B0 represents the neutral face model, B i Let α represent the i-th expression base. i B i The corresponding weights, where n represents the number of expression bases.
[0035] V. Deformation Transfer for Triangle Meshes (Dtrans) Algorithm: For example, given a 3D face model A, which is composed of a neutral face model and an expression base, and another 3D neutral face model B, the Dtrans algorithm can transfer the expression base from 3D face model A to 3D neutral face model B, forming the hybrid shape corresponding to neutral face model B, that is, the 3D face model with expression corresponding to neutral face model B.
[0036] VI. Model Topology: In 3D modeling, model topology refers to the layout, structure, and connection of points, lines, and surfaces in a polygonal mesh model.
[0037] 7. Non-rigid registration algorithm: An algorithm that registers one object model to another object model. Registration refers to the process of establishing a correspondence between points in the object models.
[0038] 8. Non-rigid Iterative Closest Point (NICP) Algorithm: This is a non-rigid registration algorithm where, for rigid bodies, deformation occurs through rotation and translation. However, for objects like faces, hands, and bodies, deformation includes both rigid deformation (local rotation and translation caused by different poses) and non-rigid deformation (such as variations in shape resulting in different builds). When seeking a matching relationship between two point sets (source and target sets), the NICP algorithm allows non-rigid deformation within the source point set.
[0039] Figure 2 This is a schematic diagram illustrating the principle of the NICP algorithm provided in the embodiments of this application. For example... Figure 2 As shown, given a template model and a scan model, the NICP algorithm can use the template model to scan the template and obtain an output model. The output model can be a clean 3D model with the same topological relationship as the template model and the same shape as the scan model.
[0040] 9. Deformation Transfer: Its purpose is to transfer the shape changes exhibited by the deformation between the source indeformable object model and the corresponding deformable object model (i.e., the hybrid shape) to the target indeformable object model, so as to obtain the hybrid shape corresponding to the target indeformable object model.
[0041] It should be understood that, in the embodiments of this application, deformation transfer may be facial expression transfer or motion transfer, but is not limited to these.
[0042] 10. Deformation Variables: Includes the radial matrix (i.e., transformation matrix) from the source invariant object model to the corresponding deformable object model, or includes the radial matrix and the offset from the source invariant object model to the corresponding deformable object model.
[0043] 11. A mesh is a point cloud that has connections between points (i.e., topological structure). When the mesh is triangular, it can be called a triangular mesh, triangular patch mesh, triangular patch, etc. When the mesh is quadrilateral, it can be called a quadrilateral mesh, quadrilateral patch mesh, quadrilateral patch, square mesh, square patch mesh, square patch, etc. When the mesh is polygonal, it can be called a polygonal mesh, polygonal patch mesh, polygonal patch, polygonal mesh, polygonal patch, etc.
[0044] 12. Key points: also known as semantic key points, such as the points on the facial features in a face model, such as the points corresponding to the nose, eyes, and ears.
[0045] Figure 3 This is a schematic diagram of an application scenario involved in an embodiment of this application.
[0046] like Figure 3 As shown, it includes a computing device 301. The computing device 301 is used to acquire a source indeformable object model, a deformable object model corresponding to the source indeformable object model, and a target indeformable object model; and to process these three models using the deformation transfer method provided in this application embodiment, transferring the deformation of the deformable object model to the target indeformable object model to obtain a hybrid shape corresponding to the target indeformable object model.
[0047] Optionally, it may also include a display device 302 for displaying the blended shape corresponding to the target non-deformable object model.
[0048] For example, computing device 301 can be a server. There can be one or more servers. When there are multiple servers, at least two servers provide different services, and / or at least two servers provide the same service, such as providing the same service through load balancing. This embodiment of the application does not limit this. The aforementioned server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. A server can also be a node in a blockchain.
[0049] For example, when the computing device 301 has a display function, the display device 302 can be the display in the computing device 301. In other words, the computing device 301 and the display device 302 can be the same device.
[0050] For example, the display device 302 and the computing device 301 are different devices, and the display device 302 is connected to the computing device 301 via a network. The network can be an intranet, the Internet, Global System for Mobile communication (GSM), Wideband Code Division Multiple Access (WCDMA), Generation 4 (G4) network, 5G network, Bluetooth, Wireless Fidelity (Wi-Fi), a voice communication network, or other wireless or wired networks.
[0051] It should be understood that BS (Browser-Based Design) technology is the supporting technology in 3D face-driven rendering and 3D virtual human creation. For example, in 3D face-driven rendering, a live-action video can be acquired, and then BS technology can be used to drive the animation effects of a lifelike animated character through this live-action video. In other words, the deformations between images in the live-action video can be transferred to the animated character, allowing the animated character to display the same animated expressions as in the live-action video. As another example, in pre-screening scenes for movies, it is often necessary to temporarily change different actors to audition for the same set of expressions or movements. Obviously, this method would incur significant manpower costs. Therefore, it is only necessary to acquire a live-action performance video, and then BS technology can be used to transfer deformations from this live-action performance video to any actor.
[0052] As described in the background section, the process of obtaining the BS using the deformation transfer algorithm currently suffers from low accuracy in the blending shape of the target expressionless face model due to the potential inconsistency in topology between the target expressionless face model and the source expressionless face model. This results in a low accuracy in the blending shape of the target expressionless face model obtained based on the mapping relationship between the triangular mesh of the target expressionless face model and the triangular mesh of the source expressionless face model.
[0053] To address this technical problem, this application proposes using a non-rigid registration algorithm to register the target non-deformable object model to the source non-deformable object model, and simultaneously register the target non-deformable object model to the corresponding deformable object model of the source non-deformable object model, resulting in two deformable object models. The deformation between these two deformable object models is known. Since the topology of the target non-deformable object model and the deformable object model is consistent, in other words, the triangular meshes of the target non-deformable object model and the deformable object model correspond one-to-one, the hybrid shape obtained by transferring the above deformation to the target non-deformable object model is more accurate.
[0054] The technical solutions of the embodiments of this application will be described in detail below through some examples. The following embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.
[0055] Figure 4This application provides a flowchart of a deformation migration method, which can be executed by any electronic device with data processing capabilities. For example, the electronic device can be a user terminal, including but not limited to mobile phones, computers, intelligent voice interaction devices, smart home appliances, vehicle terminals, and aircraft. This invention can be applied to various scenarios, including but not limited to cloud technology, artificial intelligence, smart transportation, and assisted driving. The electronic device can also be implemented as a server. The server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, and big data and artificial intelligence platforms. The server can be directly or indirectly connected via wired or wireless communication, and this application does not impose any limitations on this. For ease of description, the deformation migration method provided in this application is illustrated below using an electronic device as an example:
[0056] S410: Obtain the source invariant object model, the invariant object model corresponding to the source invariant object model, and the target invariant object model;
[0057] It should be understood that the difference between the source invariant object model and the target invariant object model is that the invariant object model corresponding to the source invariant object model is known, while the invariant object model corresponding to the target invariant object model is unknown. The purpose of this application embodiment is to determine the invariant object model corresponding to the target invariant object model when the source invariant object model, the invariant object model corresponding to the source invariant object model, and the target invariant object model are known.
[0058] In some possible implementations, the deformable object model corresponding to the source deformable object model can be an object model with at least one expression such as laughing or crying, or with at least one action. The deformable object model can be one or more. This application embodiment uses the case of one deformable object model as an example to illustrate the embodiment of this application.
[0059] In some implementations, the source invariant object model and its corresponding invariant object model can originate from a dataset, but are not limited to this. For example, the source invariant object model and its corresponding invariant object model can originate from a mixed-shape dataset defined in a company's guidelines.
[0060] In some possible implementations, the target invariant object model can be an invariant model of any object. For example, the target invariant object model can be a natural facial expression model of any character designed by an artist, a natural facial expression model of a publicly available 3D facial base such as Flame or Facewarehouse, or a natural facial expression model reconstructed by a 3D reconstruction algorithm, etc., but not limited to once.
[0061] S420: Deform the target indeformable object model according to the source indeformable object model to obtain a first deformable object model whose shape is consistent with the shape of the source indeformable object model and whose topology is consistent with the topology of the target indeformable object model; and deform the target indeformable object model according to the deformable object model to obtain a second deformable object model whose shape is consistent with the shape of the deformable object model and whose topology is consistent with the topology of the target indeformable object model.
[0062] In some feasible implementations, the electronic device can register the target non-deformable object model to the source non-deformable object model based on a non-rigid registration algorithm to obtain a first deformable object model; and register the target non-deformable object model to the deformable object model based on a non-rigid registration algorithm to obtain a second deformable object model.
[0063] In some implementations, the non-rigid registration algorithm can be the NICP algorithm, but it is not limited to this.
[0064] It should be understood that when an electronic device registers a target indeformable object model to a source indeformable object model using a non-rigid registration algorithm, it needs to establish a point correspondence between the target indeformable object model and the source indeformable object model. Based on this point correspondence, a first deformable object model is generated. The shape of this first deformable object model is consistent with the shape of the source indeformable object model, and its topology is consistent with that of the target indeformable object model. Similarly, when an electronic device registers a target indeformable object model to a deformable object model corresponding to a source indeformable object model using a non-rigid registration algorithm, it needs to establish a point correspondence between the target indeformable object model and the deformable object model. Based on this point correspondence, a second deformable object model is generated. The shape of this second deformable object model is consistent with the shape of the deformable object model, and its topology is consistent with that of the target indeformable object model.
[0065] S430: Determine the deformation from the first deformable object model to the second deformable object model;
[0066] It should be understood that since the first and second deformable object models are known, their deformation variables can be calculated. Both the first and second deformable object models can be divided into multiple triangular meshes, and due to their identical topology, they contain the same number of triangular meshes, with a one-to-one correspondence between the number of triangular meshes in the first and second deformable object models. Therefore, the deformation variables of the first and second deformable object models include the deformation variables between corresponding triangular meshes. For example, assuming that triangular mesh A in the first deformable object model corresponds to triangular mesh B in the second deformable object model, then the deformation variables of the first and second deformable object models include the deformation variables from triangular mesh A to triangular mesh B.
[0067] In some implementations, the deformation between any set of corresponding triangular meshes may include the affine matrix between the set of triangular meshes, and may also include the offset between them.
[0068] S440: Based on the target indeformable object model and deformation, transfer the deformation of the deformable object model to the target indeformable object model.
[0069] In this application, electronic devices can employ a deformation transfer algorithm to transfer the aforementioned deformations to a target non-deformable object model, thereby achieving the purpose of transferring deformations from a model with a deformable object model to a target non-deformable object model. This application does not limit the deformation transfer algorithm. The following example illustrates this deformation transfer algorithm:
[0070] For example, since deformation is based on triangular meshes, the electronic device needs to establish a correspondence between the triangular meshes of the target non-deformable object model and the first deformable object model. Let's assume the i-th triangular mesh in the first deformable object model is denoted as s. i Let t be the i-th triangular mesh in the target non-deformable object model. i There is a correspondence between these two triangular meshes, which can be determined by (s) i ,t i Let M represent a set of triangular meshes with a corresponding relationship. Then, the correspondence M between the triangular meshes of the target undeformed object model and the first deformed object model can be expressed as:
[0071] M = {(s1,t1),(s2,t2),…(s M ,t M )}
[0072] Furthermore, electronic devices can calculate the deformation between a target indeformable object model and its corresponding deformable object model using the following formula:
[0073]
[0074] in, This represents the i-th vertex in the deformable object model corresponding to the target non-deformable object model, where j represents the index of the triangular mesh. Let represent the affine matrix between the i-th triangular mesh and the i-th triangular mesh in the first deformable object model. This represents the affine matrix between the i-th triangular mesh in the target non-deformable object model and the i-th triangular mesh in the corresponding deformable object model. F This represents the F-norm. The formula essentially solves for the...
[0075] The following example illustrates the facial expression transfer method:
[0076] For example, Figure 5 This is a schematic diagram of facial expression transfer provided in an embodiment of this application, such as... Figure 5 As shown, this expression transfer process is implemented through a deformation transfer algorithm. The input to this deformation transfer algorithm includes: a source expressionless face model (i.e., Figure 5 (Left-side attached image, first row, last column) Target expressionless human face model ( Figure 5 The models in each row of the first column of the attached diagram on the left are the target expressionless face model and the corresponding expression-bearing face model of the source expressionless face model. Figure 5 The second and third rows of the last column in the attached diagram on the left) show that the source expressionless face model and its corresponding expression-bearing face model can be collectively referred to as the source face model (i.e., Figure 5 (See the last column of the attached diagram on the left). Furthermore, after obtaining the various inputs to the aforementioned deformation transfer algorithm, the non-rigid registration algorithm serves as an intermediate step in the deformation transfer algorithm. The electronic device can then use the non-rigid registration algorithm to transfer the target expressionless face model (…). Figure 5 The models in each row of the first column of the attached image on the left are the target expressionless face models, registered to the source expressionless face model (i.e., Figure 5 The first deformed face model (i.e., ...) is obtained from the last column of the first row of the attached diagram on the left. Figure 5 The second to last column of the first row in the attached diagram on the left); and based on a non-rigid registration algorithm, the target expressionless face model ( Figure 5 The models in each row of the first column of the attached image on the left are the target expressionless face models, registered to the corresponding expression-bearing face models of the source expressionless face models (i.e.,...). Figure 5The second deformed face model (i.e., ...) is obtained from the last column, second and third rows of the attached diagram on the left. Figure 5 The second row and second to last column of the attached diagram on the left (and the third row and second to last column of the attached diagram). Correspondingly, Figure 5 The first, second, and third columns of the first row in the attached diagram on the right represent the first deformed face model and two second deformed face models, respectively. These three models can be collectively referred to as deformed face models (i.e.,...). Figure 5 (See the second-to-last column in the left-hand diagram). Furthermore, the electronic device can determine the deformation of the first deformed face model and the two second deformed face models respectively, and based on these deformations, transfer the expressions of the two second deformed face models to the target expressionless face model, such as... Figure 5 As shown, Figure 5 The first column of the second row in the attached diagram on the right represents the target expressionless face model. The second column of the second row represents a face model with expression obtained by transferring the expression of the face model above it to the target expressionless face model. The third column of the second row represents a face model with expression obtained by transferring the expression of the face model above it to the target expressionless face model.
[0077] This application provides a deformation transfer method, including: obtaining a source indeformable object model, a deformable object model corresponding to the source indeformable object model, and a target indeformable object model; deforming the target indeformable object model according to the source indeformable object model to obtain a first deformable object model whose shape is consistent with the shape of the source indeformable object model and whose topology is consistent with the topology of the target indeformable object model; deforming the target indeformable object model according to the deformable object model to obtain a second deformable object model whose shape is consistent with the shape of the deformable object model and whose topology is consistent with the topology of the target indeformable object model; determining the deformation amount from the first deformable object model to the second deformable object model; and transferring the deformation of the deformable object model to the target indeformable object model based on the target indeformable object model and the deformation amount. Because the topology of the target non-deformable object model and the deformable object model is consistent—in other words, the triangular meshes of the target non-deformable object model and the deformable object model correspond one-to-one—this avoids the situation in the prior art where there is a mismatch between the triangular meshes of the target non-deformable object model and the source non-deformable object model. For example, the triangular mesh corresponding to the forehead region of the target non-deformable object model does not have a corresponding triangular mesh in the source non-deformable object model. Since deformation transfer is based on triangular meshes, if there is a problem of triangular mesh mismatch, it will inevitably lead to some triangular meshes on the target non-deformable object model not having corresponding deformations and remaining unchanged. For example, the forehead region of the target non-deformable object model will not undergo deformation transfer. However, because the topology of the target non-deformable object model and the deformable object model is consistent, the accuracy of the mixed shape corresponding to the target non-deformable object model can be improved.
[0078] It should be understood that since polygonal meshes, such as quadrangular meshes, are more reasonable for wiring, the topology of many object models is composed of polygonal meshes. However, since the deformation transfer algorithm is based on triangular meshes, this application proposes that each polygonal mesh can be divided into multiple triangular meshes.
[0079] In some possible implementations, the first deformable object model, the second deformable object model, and the target non-deformable object model all include multiple polygonal meshes; correspondingly, the electronic device can convert each polygonal mesh in the first deformable object model into multiple triangular meshes, and convert each polygonal mesh in the second deformable object model into multiple triangular meshes; determine the deformation based on the multiple triangular meshes in the first and second deformable object models; convert each polygonal mesh in the target non-deformable object model into multiple triangular meshes; and transfer the deformation of the deformable object model to the target non-deformable object model based on the multiple triangular meshes and the deformation based on the target non-deformable object model.
[0080] In some implementations, the triangular meshing criteria corresponding to the first deformable object model, the second deformable object model, and the target non-deformable object model are the same.
[0081] It should be understood that since the first deformable object model, the second deformable object model, and the target indeformable object model have the same topology, when the triangular meshing criteria corresponding to the first deformable object model, the second deformable object model, and the target indeformable object model are the same, there will be a one-to-one correspondence between the triangular meshes of these three models.
[0082] In some possible implementations, for any one of the first deformable object model, the second deformable object model, and the target non-deformable object model, the electronic device can sort the vertices of the object model to set an index for each vertex. Then, in ascending order of index, every three vertices are connected to form a triangular mesh, with the last vertex potentially connected to the first vertex. After these vertices are connected, if the object model also includes a polygonal mesh, then for that polygonal mesh, in ascending order of index, every three vertices are connected to form a triangular mesh, with the last vertex potentially connected to the first vertex, until the entire object model is divided into triangular mesh positions.
[0083] For example, Figure 6 This is a schematic diagram illustrating a four-corner grid divided into triangular grids, as provided in an embodiment of this application. Figure 6 As shown, the quadrangular grid has four vertices with indices 1, 2, 3, and 4. According to the triangular grid division criteria mentioned above, vertices 1, 2, and 3 can be connected to form a triangular grid, and vertices 3, 4, and 1 can be connected to form a triangular grid.
[0084] For example, Figure 7 A schematic diagram illustrating an octagonal mesh divided into triangular meshes, as provided in an embodiment of this application, is shown below. Figure 7 As shown, the octagonal mesh includes 8 vertices, with vertex indices 1-8. Following the aforementioned triangular mesh division criteria, vertices 1, 2, and 3 can be connected to form one triangular mesh; vertices 3, 4, and 5 can be connected to form another; vertices 5, 6, and 7 can be connected to form another; and vertices 7, 8, and 1 can be connected to form yet another. Since vertices 1, 3, 5, and 7 form a quadrangular mesh, vertices 1, 3, and 5 can be connected to form another triangular mesh, and vertices 5, 7, and 1 can be connected to form yet another. Finally, the octagonal mesh is divided into... Figure 7 The multiple triangular grids shown.
[0085] In the embodiments of this application, the electronic device can convert polygonal meshes into multiple triangular meshes, so that the deformation transfer algorithm can be performed normally based on the triangular meshes, thereby realizing the above-mentioned deformation transfer process. In short, this mesh conversion method can improve the applicability of the deformation transfer method. For example, it is not only applicable to the deformation transfer of object models composed of triangular meshes, but also applicable to the deformation transfer of object models composed of any polygonal meshes.
[0086] It should be understood that when registering the target indeformable object model to the source indeformable object model based on a non-rigid registration algorithm to obtain the first deformable object model, since the non-rigid registration algorithm is actually a non-rigid registration process between the target indeformable object model and the source indeformable object model, this non-rigid registration process needs to be continuously adjusted. Therefore, the electronic device deforms the target indeformable object model according to the source indeformable object model based on the point correspondence between the target indeformable object model and the source indeformable object model to obtain the first deformable object model, which may include:
[0087] S420-1A: Based on the correspondence between points between the target indeformable object model and the source indeformable object model, the target indeformable object model is deformed according to the source indeformable object model to obtain the third deformable object model;
[0088] It should be understood that the explanation of the non-rigid registration algorithm and the object model registration process can be found in S420, and will not be repeated in this embodiment.
[0089] It should be understood that the difference between the third deformation object model and the first deformation object model is that the third deformation object model refers to the deformation object model obtained by the electronic device for the first time by registering the target indeformable deformation object model to the source indeformable deformation object model based on the non-rigid registration algorithm, while the first deformation object model is the deformation object model obtained by registering the target indeformable deformation object model to the source indeformable deformation object model based on the non-rigid registration algorithm.
[0090] S420-2A: Identify multiple key point pairs between the target invariant object model and the source invariant object model;
[0091] In some feasible approaches, key points in the object model can be identified through extensive testing. These key points can be semantically strong, for example, such as... Figure 8 As shown, to better reflect changes in facial expression, these key points can include the starting and ending points of the eyebrows, the corners of the eyes, the corners of the mouth, the tip of the nose, the cupid's bow and valence of the lips, the chin, the root of the ear, and various points on the contour line, totaling 49 key points. Next, manual annotation can be used to annotate the key points on both the target and source invariant object models.
[0092] In some implementations, the electronic device can acquire multiple object models, each carrying keypoints as labels. A neural network model is trained based on these object models and their labels. The input to this neural network model is the object model, and its output is the keypoints of that object model. After training, the electronic device can input a target invariant object model into the neural network model and output its keypoints. Alternatively, the electronic device can input a source invariant object model into the neural network model and output its keypoints.
[0093] It should be understood that the number of key points on the target invariant object model is the same as that on the source invariant object model, and there is a one-to-one correspondence between the key points on the target invariant object model and the source invariant object model. For example, the starting point and ending point of the eyebrows on the target invariant object model correspond to each other on the source invariant object model.
[0094] S420-3A: Calculate the nearest neighbor loss of key points between the target indeformable object model and the source indeformable object model based on multiple key point pairs between them;
[0095] It should be understood that the so-called keypoint nearest neighbor loss refers to the sum of the losses for each keypoint pair between the target invariant object model and the source invariant object model. The electronic device can calculate the loss for each keypoint pair in any of the following ways, but is not limited to:
[0096] In some implementations, the electronic device can calculate the norm of the coordinate difference for each keypoint pair and determine that norm as the loss for that keypoint pair.
[0097] In other possible implementations, the electronic device can calculate the norm square of the coordinate difference for each keypoint pair and determine the result of the norm square as the loss for that keypoint pair.
[0098] In some possible implementations, the norm in the embodiments of this application can be the 1 norm, 2 norm, or F norm, etc., and the embodiments of this application do not limit this.
[0099] Based on this, the keypoint nearest neighbor loss between the target invariant object model and the source invariant object model can be calculated using any of the following formulas, but is not limited to:
[0100]
[0101]
[0102] Among them, v i c represents the i-th vertex in the source invariant object model. iLet represent the i-th vertex in the target invariant object model, i = 1, 2, ..., k, where k represents the number of keypoints in both the source and target invariant object models, i.e., both object models have k keypoints. ||·|| represents the norm, E. lm (·) represents the nearest neighbor loss of key points between the target invariant object model and the source invariant object model.
[0103] S420-4A: Calculate the target loss between the third deformable object model and the source non-deformable object model based on the key point nearest neighbor loss between the target non-deformable object model and the source non-deformable object model;
[0104] In some feasible implementations, the electronic device can determine the keypoint nearest neighbor loss between the target indeformable object model and the source indeformable object model as the target loss between the third indeformable object model and the source indeformable object model.
[0105] For example, an electronic device can calculate the target loss between the third deformable object model and the source non-deformable object model using the following formula, but it is not limited to this:
[0106]
[0107] Among them, v i c represents the i-th vertex in the source invariant object model. i v represents the i-th vertex in the target non-mutable object model. i Let E represent the i-th vertex in the third deformable object model. Let k represent the number of keypoints in both the source and target non-deformable object models (both have k keypoints), and let n represent the number of points in both models (both have n points). Any point here can be a keypoint or a regular point; a regular point refers to any point other than a keypoint. lm (·) represents the keypoint nearest neighbor loss between the target indeformable object model and the source indeformable object model, and E(·) represents the target loss between the third indeformable object model and the source indeformable object model.
[0108] In other possible implementations, the electronic device may also calculate at least one of the following losses: calculate the point nearest neighbor loss between the target indeformable object model and the source indeformable object model based on multiple point pairs between the target indeformable object model and the source indeformable object model; calculate the smoothing loss between the third indeformable object model and the source indeformable object model; correspondingly, the electronic device may calculate the key point nearest neighbor loss between the target indeformable object model and the source indeformable object model, and a weighted sum of at least one of the point nearest neighbor loss and the smoothing loss between the third indeformable object model and the source indeformable object model to obtain the target loss between the third indeformable object model and the source indeformable object model.
[0109] It should be understood that the so-called nearest neighbor loss refers to the sum of the losses for every pair of points between the target invariant object model and the source invariant object model. Any point here can be a key point or an ordinary point. The electronic device can calculate the loss for each pair of points in any of the following ways, but is not limited to:
[0110] In some feasible implementations, the electronic device can calculate the norm of the coordinate difference for each pair of points and determine that norm as the loss for that pair of points.
[0111] In other possible implementations, the electronic device can calculate the norm square of the coordinate difference for each pair of points and determine the result of the norm square as the loss for that pair of points.
[0112] In some possible implementations, the norm in the embodiments of this application can be the 1 norm, 2 norm, or F norm, etc., and the embodiments of this application do not limit this.
[0113] Based on this, the point nearest neighbor loss between the target invariant object model and the source invariant object model can be calculated using any of the following formulas, but is not limited to:
[0114]
[0115]
[0116] Among them, v i c represents the i-th vertex in the source invariant object model. i Let represent the i-th vertex in the target invariant object model, i = 1, 2, ..., n, where n represents the number of points in both the source and target invariant object models (i.e., both object models have n points). Any point here can be a key point or a regular point. ||·|| represents the norm, E. c (·) represents the point nearest neighbor loss between the target invariant object model and the source invariant object model.
[0117] In some implementations, the smoothing loss between the third deformable object model and the source indeformable object model can be the Laplace smoothing loss between the third deformable object model and the source indeformable object model. The electronic device can calculate the Laplace smoothing loss between the third deformable object model and the source indeformable object model using any of the following formulas, but is not limited to:
[0118]
[0119]
[0120] in,
[0121] Among them, v i This represents the i-th vertex in the source invariant object model. Let represent the i-th vertex in the third deformable object model, where i = 1, 2, ..., n, and n represents the number of points in both the source and third deformable object models (i.e., both object models have n points). Any point here can be a keypoint. i Indicates v i d's neighborhood i Indicates v i The number of vertices included in the neighborhood of v j Indicates v i The j-th vertex in the neighborhood, Indicates v i The average coordinates of all vertices in the neighborhood. express The neighborhood, express The number of vertices included in the neighborhood of . express The j-th vertex in the neighborhood, express The average coordinates of all vertices in the neighborhood of E, ||·|| denotes the norm, E lap This represents the Laplace smoothing loss between the third deformed object model and the source non-deformed object model.
[0122] Based on this, the electronic device can calculate the target loss between the third deformable object model and the source non-deformable object model using any of the following formulas, but is not limited to these:
[0123]
[0124]
[0125]
[0126] Among them, vi c represents the i-th vertex in the source invariant object model. i This represents the i-th vertex in the target non-mutable object model. Let represent the i-th vertex in the third deformable object model, where i = 1, 2, ..., n, and n represents the number of points in both the source and target deformable object models. Each of these object models has n points. Any point here can be a key point or a regular point. E lap E represents the Laplace smoothing loss between the third deformable object model and the source non-deformable object model. lm (·) represents the keypoint nearest neighbor loss between the target invariant object model and the source invariant object model, E c (·) represents the point nearest neighbor loss between the target indeformable object model and the source indeformable object model, E(·) represents the target loss between the third indeformable object model and the source indeformable object model, ω lap ω lm ω c These are the weights corresponding to the Laplace smoothing loss, the keypoint nearest neighbor loss, and the point nearest neighbor loss, respectively.
[0127] S420-5A: Determine the first deformable object model based on the target loss between the third deformable object model and the source deformable object model.
[0128] In some implementations, when the target loss between the third deformable object model and the source indeformable object model converges, the electronic device can identify the third deformable object model as the first deformable object model. When the target loss between the third deformable object model and the source indeformable object model diverges, the electronic device can adjust the source indeformable object model and use the adjusted source indeformable object model as the new source indeformable object model, continuing to execute S420-1A to S420-5A until the target loss converges, and finally identifying the obtained deformable object model as the first deformable object model.
[0129] In this embodiment, when an electronic device registers a target non-deformable object model to a source non-deformable object model using a non-rigid registration algorithm, it can determine the keypoint nearest neighbor loss between the target and source non-deformable object models based on multiple keypoint pairs. Further, it determines the target loss between a third deformable object model and the source non-deformable object model based on this keypoint nearest neighbor loss, and determines the first deformable object model based on this target loss. Since keypoints are crucial in non-rigid registration, determining the topological consistency between the first deformable object model and the target non-deformable object model, and the shape consistency between the first deformable object model and the source non-deformable object model, using the keypoint nearest neighbor loss as part of the target loss can yield a more accurate first deformable object model, thereby improving the accuracy of deformation transfer.
[0130] Furthermore, in the embodiments of this application, since the essence of Laplace loss is to make the shape of the source deformable object model the same or similar to that of the first deformable object model, using Laplace loss as part of the target loss can yield a more accurate first deformable object model, thereby improving the accuracy of deformation transfer.
[0131] It should be understood that when registering the target indeformable object model to the deformable object model corresponding to the source indeformable object model using a non-rigid registration algorithm to obtain the second deformable object model, since the non-rigid registration algorithm is actually a non-rigid registration process between the target indeformable object model and the deformable object model, this non-rigid registration process needs to be continuously adjusted. Therefore, the electronic device deforms the target indeformable object model according to the deformable object model based on the point correspondence between the target indeformable object model and the deformable object model to obtain the second deformable object model, which may include:
[0132] S420-1B: Based on the correspondence between points between the target indeformable object model and the deformable object model, the target indeformable object model is deformed according to the deformable object model to obtain the fourth deformable object model;
[0133] It should be understood that the explanation of the non-rigid registration algorithm and the object model registration process can be found in S420, and will not be repeated in this embodiment.
[0134] It should be understood that the difference between the fourth deformation object model and the second deformation object model is that the fourth deformation object model refers to the deformation object model obtained for the first time by registering the target non-deformable object model to the deformable object model based on the non-rigid registration algorithm, while the second deformation object model is the deformation object model obtained by registering the target non-deformable object model to the deformable object model based on the non-rigid registration algorithm.
[0135] S420-2B: Identify multiple key point pairs between the target non-deformable object model and the deformable object model;
[0136] In some feasible approaches, key points in the object model can be identified through extensive testing. These key points can be semantically strong, for example, such as... Figure 8 As shown, to better reflect changes in facial expression, these key points can include the starting and ending points of the eyebrows, the corners of the eyes, the corners of the mouth, the tip of the nose, the cupid's bow and valence of the lips, the chin, the base of the ears, and various points on the contour line, totaling 49 key points. Next, manual annotation can be used to annotate the key points on both the non-deformable and deformable object models.
[0137] In some implementations, the electronic device can acquire multiple object models, each carrying keypoints as labels. A neural network model is trained based on these object models and their labels. The input to this neural network model is the object model, and the output is the keypoints of that object model. After training, the electronic device can input a target, non-deformable object model into the neural network model, and the output will be the keypoints of the target, non-deformable object model. Alternatively, the electronic device can input a deformable object model into the neural network model, and the output will be the keypoints of the deformable object model.
[0138] It should be understood that the number of key points on the target non-variable object model and the deformable object model is the same, and there is a one-to-one correspondence between the key points on the target non-variable object model and the deformable object model. For example, the starting point and ending point of the eyebrows on the target non-variable object model and the deformable object model correspond.
[0139] S420-3B: Calculate the nearest neighbor loss of key points between the target indeformable object model and the deformable object model based on multiple key point pairs between the target indeformable object model and the deformable object model;
[0140] It should be understood that the so-called keypoint nearest neighbor loss refers to the sum of the losses for each keypoint pair between the target undeformed object model and the deformable object model. The electronic device can calculate the loss for each keypoint pair in any of the following ways, but is not limited to:
[0141] In some implementations, the electronic device can calculate the norm of the coordinate difference for each keypoint pair and determine that norm as the loss for that keypoint pair.
[0142] In other possible implementations, the electronic device can calculate the norm square of the coordinate difference for each keypoint pair and determine the result of the norm square as the loss for that keypoint pair.
[0143] In some possible implementations, the norm in the embodiments of this application can be the 1 norm, 2 norm, or F norm, etc., and the embodiments of this application do not limit this.
[0144] Based on this, the keypoint nearest neighbor loss between the target undeformable object model and the deformable object model can be calculated using any of the following formulas, but is not limited to:
[0145]
[0146]
[0147] Among them, v i ' represents the i-th vertex in the deformable object model, c i Let represent the i-th vertex in the target non-mutable object model, i = 1, 2, ..., k, where k represents the number of keypoints in both the deformable object model and the target non-mutable object model, i.e., both object models have k keypoints. ||·|| represents the norm, E. lm (·) represents the nearest neighbor loss of key points between the target undeformed object model and the deformed object model.
[0148] S420-4B: Calculate the target loss between the fourth deformable object model and the deformable object model based on the key point nearest neighbor loss between the target non-deformable object model and the deformable object model;
[0149] In some possible implementations, the electronic device can determine the keypoint nearest neighbor loss between the target non-deformable object model and the deformable object model as the target loss between the fourth deformable object model and the deformable object model.
[0150] For example, an electronic device can calculate the target loss between the fourth deformable object model and the deformable object model using the following formula, but it is not limited to this:
[0151]
[0152] Among them, v i ' represents the i-th vertex in the deformable object model, c i v represents the i-th vertex in the target non-mutable object model. i Let represent the i-th vertex in the fourth deformable object model, k represent the number of keypoints in both the deformable object model and the target non-deformable object model (both have k keypoints), and n represent the number of points in both models (both have n points). Any point here can be a keypoint or a regular point. lm(·) represents the keypoint nearest neighbor loss between the target non-deformable object model and the deformable object model, and E(·) represents the target loss between the fourth deformable object model and the deformable object model.
[0153] In other possible implementations, the electronic device may also calculate at least one of the following losses: calculate the point nearest neighbor loss between the target non-deformable object model and the deformable object model based on multiple point pairs between the target non-deformable object model and the deformable object model; calculate the smoothing loss between the fourth deformable object model and the deformable object model; correspondingly, the electronic device may calculate the key point nearest neighbor loss between the target non-deformable object model and the deformable object model, and a weighted sum of at least one of the point nearest neighbor loss and the smoothing loss between the fourth deformable object model and the deformable object model to obtain the target loss between the fourth deformable object model and the deformable object model.
[0154] It should be understood that the so-called nearest neighbor loss refers to the sum of the losses for each pair of points between the target undeformed object model and the deformable object model. Any point here can be a key point or an ordinary point. The electronic device can calculate the loss for each pair of points in any of the following ways, but is not limited to:
[0155] In some feasible implementations, the electronic device can calculate the norm of the coordinate difference for each pair of points and determine that norm as the loss for that pair of points.
[0156] In other possible implementations, the electronic device can calculate the norm square of the coordinate difference for each pair of points and determine the result of the norm square as the loss for that pair of points.
[0157] In some possible implementations, the norm in the embodiments of this application can be the 1 norm, 2 norm, or F norm, etc., and the embodiments of this application do not limit this.
[0158] Based on this, the point nearest neighbor loss between the target undeformed object model and the deformable object model can be calculated using any of the following formulas, but is not limited to:
[0159]
[0160]
[0161] Among them, v i ' represents the i-th vertex in the deformable object model, c i Let represent the i-th vertex in the target non-mutable object model, i = 1, 2, ..., n, where n represents the number of points in both the deformable object model and the target non-mutable object model. Each of these object models has n points. Any point here can be a key point or a regular point. ||·|| represents the norm, E.c (·) represents the point nearest neighbor loss between the target undeformed object model and the deformed object model.
[0162] In some implementations, the smoothing loss between the fourth deformable object model and the deformable object model can be the Laplace smoothing loss between the fourth deformable object model and the deformable object model. The electronic device can calculate the Laplace smoothing loss between the fourth deformable object model and the deformable object model using any of the following formulas, but is not limited to:
[0163]
[0164]
[0165] in,
[0166] Among them, v i ' represents the i-th vertex in the deformable object model. Let represent the i-th vertex in the fourth deformable object model, where i = 1, 2, ..., n, and n represents the number of points in both the deformable object model and the fourth deformable object model. Any point here can be a keypoint. i ' indicates v i ' ... i ' indicates v i The number of vertices included in the neighborhood of ', v j ' indicates v i The j-th vertex in the neighborhood of ' Indicates v i The average coordinates of all vertices in the neighborhood of '. express The neighborhood, express The number of vertices included in the neighborhood of . express The j-th vertex in the neighborhood, express The average coordinates of all vertices in the neighborhood of E, ||·|| denotes the norm, E lap This represents the Laplace smoothing loss between the fourth deformable object model and the deformable object model.
[0167] Based on this, the electronic device can calculate the target loss between the fourth deformable object model and the deformable object model using any of the following formulas, but is not limited to these:
[0168]
[0169]
[0170]
[0171] Among them, v i c represents the i-th vertex in the deformable object model. i This represents the i-th vertex in the target non-mutable object model. Let represent the i-th vertex in the fourth deformable object model, where i = 1, 2, ..., n, and n represents the number of points in both the deformable object model and the target non-deformable object model. Each of these object models has n points. Any point here can be a key point or a regular point. E lap E represents the Laplace smoothing loss between the fourth deformable object model and the deformable object model. lm (·) represents the keypoint nearest neighbor loss between the standard deformable object model and the deformable object model, E c (·) represents the point nearest neighbor loss between the target undeformed object model and the deformed object model, E(·) represents the target loss between the fourth deformed object model and the deformed object model, ω lap ω lm ω c These are the weights corresponding to the Laplace smoothing loss, the keypoint nearest neighbor loss, and the point nearest neighbor loss, respectively.
[0172] S420-5B: Determine the second deformable object model based on the target loss between the fourth deformable object model and the deformable object model.
[0173] In some implementations, when the target loss between the fourth deformable object model and the deformable object model converges, the electronic device can identify the fourth deformable object model as the second deformable object model. When the target loss between the fourth deformable object model and the deformable object model diverges, the electronic device can adjust the source deformable object model and use the adjusted deformable object model as the new deformable object model, continuing to execute S420-1B to S420-5B until the target loss converges, and finally identifying the obtained deformable object model as the second deformable object model.
[0174] In this embodiment, when an electronic device registers a target non-deformable object model to a deformable object model using a non-rigid registration algorithm, it can determine the keypoint nearest neighbor loss between the target non-deformable object model and the deformable object model based on multiple keypoint pairs. Further, based on this keypoint nearest neighbor loss, it determines the target loss between the fourth deformable object model and the deformable object model, and based on this target loss, it determines the second deformable object model. Since keypoints are crucial in non-rigid registration, determining the topological consistency between the second deformable object model and the target non-deformable object model, and the shape consistency between the second deformable object model and the deformable object model, using the keypoint nearest neighbor loss as part of the target loss can yield a more accurate second deformable object model, thereby improving the accuracy of deformation transfer.
[0175] Furthermore, in the embodiments of this application, since the essence of Laplace loss is to make the shape of the source deformable object model the same or similar to that of the second deformable object model, using Laplace loss as part of the target loss can yield a more accurate second deformable object model, thereby improving the accuracy of deformation transfer.
[0176] Figure 9 The deformation migration effect diagram provided in the embodiments of this application is as follows: Figure 9 As shown, the first row and first column represent the target non-deformable object model designed by the artist, while the second to last columns of the first row represent the corresponding deformable object models obtained through the technical solution of this application. The second row and first column represent the target non-deformable object model from the Flame dataset, while the second to last columns of the first row represent the corresponding deformable object models obtained through the technical solution of this application. Therefore, it can be seen that the deformation transfer method provided in this embodiment has high accuracy.
[0177] The following will provide an exemplary description of the application of the deformation transfer method provided in this application to 3D face-driven rendering and movie pre-visualization scenes:
[0178] Example 1, such as Figure 5 As shown, in 3D face-driven rendering, assuming a real-life video is obtained, the video includes 3 video images, each containing a face model, which are... Figure 5 The last column of the attached image on the left contains three video frames. The first row shows the original expressionless face model, while the second and third rows show the corresponding expression-laden face models. It is also possible to obtain... Figure 5The first column of the attached diagram on the left shows the face model to be driven (i.e., the target expressionless face model). Further, the electronic device can register the face model to be driven onto the source expressionless face model based on a non-rigid registration algorithm, obtaining the first deformed face model (i.e., Figure 5 The second-to-last column of the first row in the attached diagram on the left); and based on a non-rigid registration algorithm, the face model to be driven ( Figure 5 The models in each row of the first column of the attached image on the left are the target expressionless face models, registered to the corresponding expression-bearing face models of the source expressionless face models (i.e.,...). Figure 5 The second deformed face model (i.e., ...) is obtained from the last column, second and third rows of the attached diagram on the left. Figure 5 The second row and second to last column of the attached diagram on the left (and the third row and second to last column of the attached diagram). Correspondingly, Figure 5 The first, second, and third columns of the first row in the attached diagram on the right represent the first deformed face model and two second deformed face models, respectively. These three models can be collectively referred to as deformed face models (i.e.,...). Figure 5 (See the second-to-last column in the left-hand diagram). Furthermore, the electronic device can determine the deformation of the first deformed face model and the two second deformed face models respectively, and based on these deformations, transfer the expressions of the two second deformed face models to the face model to be driven, such as... Figure 5 As shown, Figure 5 The second row and first column of the attached diagram on the right represents the face model to be driven. The second row and second column represent a face model with an expression obtained by transferring the expression of the face model above it to the face model to be driven. The second row and third column represent a face model with an expression obtained by transferring the expression of the face model above it to the face model to be driven. In other words, Figure 5 The three frames shown in the second row of the attached image on the right are image frames obtained by driving the three frames included in the live video.
[0179] Example 2, such as Figure 5 As shown, in the pre-screening scene of the movie, only one performance video of actor A needs to be obtained. This performance video includes three video images, including a face model of actor A, which are respectively... Figure 5 The last column of the attached image on the left contains three video frames. The first row shows the original expressionless face model, while the second and third rows show the corresponding expression-laden face models. It is also possible to obtain... Figure 5 The first column of the attached diagram on the left shows the face model of actor B (i.e., the target expressionless face model). Further, the electronic device can register actor B's face model onto the source expressionless face model based on a non-rigid registration algorithm, obtaining the first deformed face model (i.e.... Figure 5The second to last column of the first row in the attached diagram on the left); and based on a non-rigid registration algorithm, the face model of actor B ( Figure 5 The models in each row of the first column of the attached image on the left are the target expressionless face models, registered to the corresponding expression-bearing face models of the source expressionless face models (i.e.,...). Figure 5 The second deformed face model (i.e., ...) is obtained from the last column, second and third rows of the attached diagram on the left. Figure 5 The second row and second to last column of the attached diagram on the left (and the third row and second to last column of the attached diagram). Correspondingly, Figure 5 The first, second, and third columns of the first row in the attached diagram on the right represent the first deformed face model and two second deformed face models, respectively. These three models can be collectively referred to as deformed face models (i.e.,...). Figure 5 (See the second-to-last column in the left-hand diagram). Furthermore, the electronic device can determine the deformation of the first deformed face model and the two second deformed face models respectively, and based on these deformations, transfer the expressions of the two second deformed face models to actor B's face model, such as... Figure 5 As shown, Figure 5 The second row, first column of the attached diagram on the right represents actor B's facial model. The second row, second column represents an expression-laden facial model of actor B obtained by transferring the expression from the facial model above it to actor B's facial model. The second row, third column represents an expression-laden facial model of actor B obtained by transferring the expression from the facial model above it to actor B's facial model. In other words, Figure 5 The three frames shown in the second row of the attached image on the right are the final video frames of actor B.
[0180] The preferred embodiments of this application have been described in detail above with reference to the accompanying drawings. However, this application is not limited to the specific details of the above embodiments. Within the scope of the technical concept of this application, various simple modifications can be made to the technical solutions of this application, and these simple modifications all fall within the protection scope of this application. For example, the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. To avoid unnecessary repetition, this application will not describe the various possible combinations separately. Furthermore, various different embodiments of this application can also be arbitrarily combined, as long as they do not violate the spirit of this application, they should also be considered as the content disclosed in this application.
[0181] It should also be understood that, in the various method embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.
[0182] The methods provided in the embodiments of this application have been described above. The apparatus provided in the embodiments of this application will be described below.
[0183] Figure 10 This is a schematic diagram of a deformation migration device 1000 provided in an embodiment of this application, as shown below. Figure 10 As shown, the device 1000 includes: an acquisition module 1010, a deformation module 1020, a determination module 1030, and a migration module 1040. The acquisition module 1010 acquires a source indeformable object model, a deformable object model corresponding to the source indeformable object model, and a target indeformable object model. The deformation module 1020 deforms the target indeformable object model according to the source indeformable object model to obtain a first deformable object model whose shape matches the shape of the source indeformable object model and whose topology matches the topology of the target indeformable object model. It also deforms the target indeformable object model according to the deformable object model to obtain a second deformable object model whose shape matches the shape of the deformable object model and whose topology matches the topology of the target indeformable object model. The determination module 1030 determines the deformation amount from the first deformable object model to the second deformable object model. The migration module 1040 migrates the deformation of the deformable object model to the target indeformable object model based on the target indeformable object model and the deformation amount.
[0184] In some possible implementations, the deformation module 1020 is specifically used to: establish a point correspondence between the target indeformable object model and the source indeformable object model; deform the target indeformable object model according to the source indeformable object model based on the point correspondence between the target indeformable object model and the source indeformable object model to obtain a first deformable object model; establish a point correspondence between the target indeformable object model and the deformable object model; deform the target indeformable object model according to the deformable object model based on the point correspondence between the target indeformable object model and the deformable object model to obtain a second deformable object model.
[0185] In some implementations, the deformation module 1020 is specifically used to: deform the target indeformable object model according to the source indeformable object model based on the correspondence between points between the target indeformable object model and the source indeformable object model to obtain a third deformable object model; determine multiple key point pairs between the target indeformable object model and the source indeformable object model; calculate the key point nearest neighbor loss between the target indeformable object model and the source indeformable object model based on the multiple key point pairs between the target indeformable object model and the source indeformable object model; calculate the target loss between the third deformable object model and the source indeformable object model based on the key point nearest neighbor loss between the target indeformable object model and the source indeformable object model; and determine the first deformable object model based on the target loss between the third deformable object model and the source indeformable object model.
[0186] In some implementations, the deformation module 1020 is specifically used to: determine the key point nearest neighbor loss between the target indeformable object model and the source indeformable object model as the target loss between the third deformation object model and the source indeformable object model.
[0187] In some implementations, the deformation module 1020 is further configured to: before calculating the target loss between the third deformable object model and the source non-deformable object model based on the keypoint nearest neighbor loss between the target non-deformable object model and the source non-deformable object model, perform at least one of the following: calculate the point nearest neighbor loss between the target non-deformable object model and the source non-deformable object model based on multiple point pairs between the target non-deformable object model and the source non-deformable object model; calculate the smoothing loss between the third deformable object model and the source non-deformable object model; correspondingly, the deformation module 1020 is specifically configured to: calculate the keypoint nearest neighbor loss between the target non-deformable object model and the source non-deformable object model, and, at least one of the point nearest neighbor loss and the smoothing loss between the third deformable object model and the source non-deformable object model, by weighted summation, to obtain the target loss between the third deformable object model and the source non-deformable object model.
[0188] In some implementations, the deformation module 1020 is specifically used to: calculate the Laplace smoothing loss between the third deformed object model and the source undeformed object model.
[0189] In some implementations, the deformation module 1020 is specifically used to: deform the target indeformable object model according to the deformable object model based on the correspondence between points between the target indeformable object model and the deformable object model to obtain a fourth deformable object model; determine multiple key point pairs between the target indeformable object model and the deformable object model; calculate the key point nearest neighbor loss between the target indeformable object model and the deformable object model based on the multiple key point pairs between the target indeformable object model and the deformable object model; calculate the target loss between the fourth deformable object model and the deformable object model based on the key point nearest neighbor loss between the target indeformable object model and the deformable object model; and determine the second deformable object model based on the target loss between the fourth deformable object model and the deformable object model.
[0190] In some implementations, the deformation module 1020 is specifically used to: determine the keypoint nearest neighbor loss between the target non-deformable object model and the deformable object model as the target loss between the fourth deformable object model and the deformable object model.
[0191] In some implementations, the deformation module 1020 is further configured to: before calculating the target loss between the fourth deformable object model and the deformable object model based on the keypoint nearest neighbor loss between the target non-deformable object model and the deformable object model, perform at least one of the following: calculate the point nearest neighbor loss between the target non-deformable object model and the deformable object model based on multiple point pairs between the target non-deformable object model and the deformable object model; calculate the smoothing loss between the fourth deformable object model and the deformable object model; correspondingly, the deformation module 1020 is specifically configured to: calculate the keypoint nearest neighbor loss between the target non-deformable object model and the deformable object model, and, at least one of the point nearest neighbor loss and the smoothing loss between the fourth deformable object model and the deformable object model, by weighted summation, to obtain the target loss between the fourth deformable object model and the deformable object model.
[0192] In some implementations, the deformation module 1020 is specifically used to: calculate the Laplace smoothing loss between the fourth deformed object model and the deformed object model.
[0193] In some implementations, the first deformable object model, the second deformable object model, and the target non-deformable object model all include multiple polygonal meshes; correspondingly, the determining module 1030 is specifically used to: convert each polygonal mesh in the first deformable object model into multiple triangular meshes, and convert each polygonal mesh in the second deformable object model into multiple triangular meshes; determine the deformation amount based on the multiple triangular meshes in the first deformable object model and the multiple triangular meshes in the second deformable object model; correspondingly, the migration module 1040 is specifically used to: convert each polygonal mesh in the target non-deformable object model into multiple triangular meshes; and migrate the deformation of the deformable object model to the target non-deformable object model based on the multiple triangular meshes and the deformation amount in the target non-deformable object model.
[0194] In some implementations, the triangular meshing criteria corresponding to the first deformable object model, the second deformable object model, and the target non-deformable object model are the same.
[0195] It should be understood that the device embodiments and method embodiments can correspond to each other, and similar descriptions can be found in the method embodiments. To avoid repetition, further details are omitted here. Specifically, Figure 10 The device 1000 shown can perform Figure 4 The corresponding method embodiments, and the foregoing and other operations and / or functions of each module in the device 1000 are respectively for implementing Figure 4 For the sake of brevity, the corresponding processes in each method are not described in detail here.
[0196] The apparatus 1000 of this application embodiment has been described above from the perspective of functional modules in conjunction with the accompanying drawings. It should be understood that this functional module can be implemented in hardware, in software instructions, or in a combination of hardware and software modules. Specifically, the steps of the method embodiments in this application can be completed by integrated logic circuits in the processor's hardware and / or by software instructions. The steps of the method disclosed in this application embodiment can be directly embodied as being executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. Optionally, the software module can be located in a mature storage medium in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps in the above method embodiments.
[0197] Figure 11 This is a schematic block diagram of the electronic device provided in the embodiments of this application.
[0198] like Figure 11 As shown, the electronic device may include:
[0199] The system includes a memory 1110 and a processor 1120. The memory 1110 stores computer programs and transfers the program code to the processor 1120. In other words, the processor 1120 can retrieve and run the computer program from the memory 1110 to implement the methods described in the embodiments of this application.
[0200] For example, the processor 1120 can be used to execute the above-described method embodiments according to instructions in the computer program.
[0201] In some embodiments of this application, the processor 1120 may include, but is not limited to:
[0202] General-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.
[0203] In some embodiments of this application, the memory 1110 includes, but is not limited to:
[0204] Volatile memory and / or non-volatile memory. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), and Direct Rambus RAM (DR RAM).
[0205] In some embodiments of this application, the computer program may be divided into one or more modules, which are stored in the memory 1110 and executed by the processor 1120 to complete the method provided in this application. The one or more modules may be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program in the electronic device.
[0206] like Figure 11 As shown, the electronic device may also include:
[0207] Transceiver 1130, which can be connected to processor 1120 or memory 1110.
[0208] The processor 1120 can control the transceiver 1130 to communicate with other devices; specifically, it can send information or data to other devices or receive information or data sent by other devices. The transceiver 1130 may include a transmitter and a receiver. The transceiver 1130 may further include antennas, and the number of antennas may be one or more.
[0209] It should be understood that the various components in the electronic device are connected through a bus system, which includes a data bus, a power bus, a control bus, and a status signal bus.
[0210] This application also provides a computer storage medium storing a computer program thereon, which, when executed by a computer, enables the computer to perform the methods of the above-described method embodiments. Alternatively, embodiments of this application also provide a computer program product containing instructions that, when executed by a computer, cause the computer to perform the methods of the above-described method embodiments.
[0211] When implemented using software, it can be implemented entirely or partially as a computer program product. This computer program product includes one or more computer instructions. When these computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., digital video disc (DVD)), or a semiconductor medium (e.g., solid-state disk (SSD)).
[0212] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
[0213] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or modules may be electrical, mechanical, or other forms.
[0214] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. For example, the functional modules in the various embodiments of this application may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.
[0215] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
Claims
1. A deformation migration method, characterized in that, include: Acquire a source undeformed face image, a deformed face image corresponding to the source undeformed face image, and a target undeformed face image; The target undeformed face image is deformed according to the source undeformed face image to obtain a first deformed face image whose shape is consistent with the shape of the source undeformed face image and whose topology is consistent with the topology of the target undeformed face image; and the target undeformed face image is deformed according to the strip-deformed face image to obtain a second deformed face image whose shape is consistent with the shape of the strip-deformed face image and whose topology is consistent with the topology of the target undeformed face image. Determine the deformation of the first deformed face image to the second deformed face image; Based on the target undeformed face image and the deformation amount, the deformation of the deformed face image is transferred to the target undeformed face image.
2. The method according to claim 1, characterized in that, The step of deforming the target undeformed face image according to the source undeformed face image to obtain a first deformed face image whose shape is consistent with the shape of the source undeformed face image and whose topology is consistent with the topology of the target undeformed face image includes: Establish the correspondence between points in the target unmodified face image and the source unmodified face image; Based on the correspondence between the target undeformed face image and the source undeformed face image, the target undeformed face image is deformed according to the source undeformed face image to obtain the first deformed face image; The step of deforming the target undeformed face image according to the deformed face image to obtain a second deformed face image whose shape is consistent with the shape of the deformed face image and whose topology is consistent with the topology of the target undeformed face image includes: Establish the correspondence between points in the target undeformed face image and the deformed face image; Based on the correspondence between the target undeformed face image and the deformed face image, the target undeformed face image is deformed according to the deformed face image to obtain the second deformed face image.
3. The method according to claim 2, characterized in that, The step of deforming the target undeformed face image based on the correspondence between points in the target undeformed face image and the source undeformed face image to obtain the first deformed face image includes: Based on the correspondence between the target undeformed face image and the source undeformed face image, the target undeformed face image is deformed according to the source undeformed face image to obtain a third deformed face image; Identify multiple key point pairs between the target unmodified face image and the source unmodified face image; The nearest neighbor loss of key points between the target unmuted face image and the source unmuted face image is calculated based on multiple key point pairs between them. The target loss between the third deformed face image and the source undeformed face image is calculated based on the key point nearest neighbor loss between the target undeformed face image and the source undeformed face image; The first deformed face image is determined based on the target loss between the third deformed face image and the source undeformed face image.
4. The method according to claim 3, characterized in that, The calculation of the target loss between the third deformed face image and the source undeformed face image based on the keypoint nearest neighbor loss between the target undeformed face image and the source undeformed face image includes: The keypoint nearest neighbor loss between the target undeformed face image and the source undeformed face image is determined as the target loss between the third deformed face image and the source undeformed face image.
5. The method according to claim 4, characterized in that, Before calculating the target loss between the third deformed face image and the source undeformed face image based on the keypoint nearest neighbor loss between the target undeformed face image and the source undeformed face image, the method further includes at least one of the following: The point nearest neighbor loss between the target unmuted face image and the source unmuted face image is calculated based on multiple point pairs between them. Calculate the smoothing loss between the third deformed face image and the source undeformed face image; Accordingly, the step of calculating the target loss between the third deformed face image and the source undeformed face image based on the keypoint nearest neighbor loss between the target undeformed face image and the source undeformed face image includes: The target loss is obtained by weighted summation of the nearest neighbor loss of the key point and the nearest neighbor loss of the point. or, The target loss is obtained by weighted summation of the nearest neighbor loss and the smoothing loss for the key points; or, The target loss is obtained by weighted summing of the key point nearest neighbor loss, the point nearest neighbor loss, and the smoothing loss.
6. The method according to claim 5, characterized in that, The calculation of the smoothing loss between the third deformed face image and the source undeformed face image includes: Calculate the Laplacian smoothing loss between the third deformed face image and the source undeformed face image.
7. The method according to claim 2, characterized in that, The step of deforming the target undeformed face image according to the deformed face image based on the correspondence between points between the target undeformed face image and the deformed face image to obtain the second deformed face image includes: Based on the correspondence between the points of the target undeformed face image and the deformed face image, the target undeformed face image is deformed according to the deformed face image to obtain the fourth deformed face image; Identify multiple key point pairs between the target undeformed face image and the deformed face image; The nearest neighbor loss of key points between the target undeformed face image and the deformed face image is calculated based on multiple key point pairs between them. The target loss between the fourth deformed face image and the deformed face image is calculated based on the key point nearest neighbor loss between the target undeformed face image and the deformed face image; The second deformed face image is determined based on the target loss between the fourth deformed face image and the deformed face image.
8. The method according to claim 7, characterized in that, The calculation of the target loss between the fourth deformed face image and the deformed face image based on the keypoint nearest neighbor loss between the target undeformed face image and the deformed face image includes: The keypoint nearest neighbor loss between the target undeformed face image and the deformed face image is determined as the target loss between the fourth deformed face image and the deformed face image.
9. The method according to claim 7, characterized in that, Before calculating the target loss between the fourth deformed face image and the deformed face image based on the keypoint nearest neighbor loss between the target undeformed face image and the deformed face image, the method further includes at least one of the following: The point nearest neighbor loss between the target undeformed face image and the deformed face image is calculated based on multiple point pairs between them. Calculate the smoothing loss between the fourth deformed face image and the deformed face image; Accordingly, the step of calculating the target loss between the fourth deformed face image and the deformed face image based on the keypoint nearest neighbor loss between the target undeformed face image and the deformed face image includes: The target loss is obtained by weighted summation of the nearest neighbor loss of the key point and the nearest neighbor loss of the point. or, The target loss is obtained by weighted summation of the nearest neighbor loss and the smoothing loss for the key points; or, The target loss is obtained by weighted summing of the key point nearest neighbor loss, the point nearest neighbor loss, and the smoothing loss.
10. The method according to claim 9, characterized in that, The calculation of the smoothing loss between the fourth deformed face image and the deformed face image includes: Calculate the Laplacian smoothing loss between the fourth deformed face image and the deformed face image.
11. The method according to any one of claims 1-10, characterized in that, The first deformed face image, the second deformed face image, and the target non-deformed face image all include multiple polygonal grids; Accordingly, determining the deformation amount from the first deformed face image to the second deformed face image includes: Each polygonal mesh in the first deformed face image is converted into multiple triangular meshes, and each polygonal mesh in the second deformed face image is converted into multiple triangular meshes; The deformation amount is determined based on multiple triangular meshes in the first deformed face image and multiple triangular meshes in the second deformed face image; Accordingly, the step of transferring the deformation of the deformed face image to the target undeformed face image based on the target undeformed face image and the deformation amount includes: Each polygonal grid in the target non-transformable human face image is converted into multiple triangular grids; Based on multiple triangular meshes in the target undeformed face image and the deformation amount, the deformation of the deformed face image is transferred to the target undeformed face image.
12. The method according to claim 11, characterized in that, The triangular mesh division criteria corresponding to the first deformed face image, the second deformed face image, and the target undeformed face image are the same.
13. A deformation migration device, characterized in that, include: The acquisition module is used to acquire a source non-deformable face image, a deformed face image corresponding to the source non-deformable face image, and a target non-deformable face image; The deformation module is used to deform the target undeformed face image according to the source undeformed face image to obtain a first deformed face image whose shape is consistent with the shape of the source undeformed face image and whose topology is consistent with the topology of the target undeformed face image; and to deform the target undeformed face image according to the deformed face image to obtain a second deformed face image whose shape is consistent with the shape of the deformed face image and whose topology is consistent with the topology of the target undeformed face image. The determination module is used to determine the deformation amount from the first deformed face image to the second deformed face image; The migration module is used to migrate the deformation of the deformed face image to the target undeformed face image based on the target undeformed face image and the deformation.
14. An electronic device, characterized in that, include: A processor and a memory, the memory being used to store a computer program, the processor being used to invoke and run the computer program stored in the memory to perform the method of any one of claims 1 to 12.
15. A computer-readable storage medium, characterized in that, Used to store a computer program that causes a computer to perform the method as described in any one of claims 1 to 12.
Citation Information
Patent Citations
Substrate generation method and device, electronic equipment and storage medium
CN114972683A
Shape data generation method and device
EP2876607A1
