A three-dimensional face model reconstruction method and electronic device

Abstract

The present application provides a method and electronic device for reconstructing a three-dimensional face model, which is used to improve the realism of the three-dimensional face model. The method includes: matching each two-dimensional facial key point of a target object with each vertex in an initial three-dimensional face model of the target object to obtain each three-dimensional facial key point; detecting the hole area of ​​the initial three-dimensional face model to obtain a hole boundary point set; based on each boundary point and each three-dimensional facial key point in the hole boundary point set, identifying target vertices that match each boundary point from each vertex of a preset standard face model; performing non-rigid registration on the target vertices using the position coordinates of each boundary point and the position coordinates of each target vertex to obtain a non-rigid registration matrix; using the non-rigid registration matrix and each internal point in the target area of ​​the standard face model to obtain a transformation matrix for each internal point; and transforming each internal point into the initial three-dimensional model using the transformation matrix for each internal point to obtain a reconstructed three-dimensional face model.

Description

Technical Field

[0001] The present application relates to the field of three-dimensional reconstruction technology, and in particular to a method and electronic device for reconstructing a three-dimensional face model. Background Art

[0002] The reconstruction of high-precision digital humans has received great attention. As the core of high-precision digital human reconstruction, face reconstruction has become a research hotspot in academia and industry. The main way to reconstruct a three-dimensional face model is to use multi-view images of the face as input to the three-dimensional face reconstruction algorithm to reconstruct the three-dimensional face model. However, the algorithm of the three-dimensional face model needs to take dense multi-view images as input in order to obtain a high-precision three-dimensional face model. When there are fewer multi-view images input, there will be obvious holes in the obtained three-dimensional face model in places where the facial texture features are insufficient. For example, the mandible, the connection between the face and the mandible, and other locations. As Figure 1 As shown in the figure, there is a hole in the mandible of the 3D face model. Therefore, the reconstructed 3D face model needs to be filled with the hole.

[0003] However, the existing hole filling methods for three-dimensional face models generally use nonlinear interpolation of the grid around the hole area without considering the geometric prior of the face. Although the hole filling can be completed, the realism of the three-dimensional face model is poor, which reduces the accuracy of the hole filling of the reconstructed three-dimensional face model. Summary of the Invention

[0004] The present application provides a method for reconstructing a three-dimensional face model and an electronic device for optimizing hole filling in the three-dimensional face model. The hole filling effect retains the facial geometric information to the greatest extent, improves the realism of the three-dimensional face model, and improves the accuracy of hole filling in the three-dimensional face model.

[0005] In a first aspect, an embodiment of the present application provides a method for reconstructing a three-dimensional face model, the method comprising:

[0006] For any target object, matching each two-dimensional facial key point obtained based on the frontal facial image of the target object with each vertex in the initial three-dimensional facial model of the target object to obtain each three-dimensional facial key point in the initial three-dimensional facial model;

[0007] Detecting a hole area of ​​the initial three-dimensional face model based on each triangular face patch in the initial three-dimensional face model to obtain a hole boundary point set;

[0008] For any one boundary point in the hole boundary point set, identifying a target vertex matching the boundary point from among the vertices of a preset standard face model based on the position coordinates of the boundary point and the position coordinates of the three-dimensional facial key points; and

[0009] Using the position coordinates of the boundary points and the position coordinates of the target vertices matching the boundary points, non-rigid registration is performed on the target vertices to obtain a non-rigid registration matrix of the target vertices;

[0010] Obtaining a transformation matrix of each internal point by using the non-rigid registration matrix and the position coordinates of each internal point in a target area of ​​the standard face model, wherein the target area is an area enclosed by each target vertex;

[0011] For any internal point, the internal point is transformed into the initial three-dimensional model using the transformation matrix of the internal point to obtain a reconstructed three-dimensional face model.

[0012] A second aspect of the present application provides an electronic device, comprising a processor and a memory, wherein the processor and the memory are connected via a bus;

[0013] The memory stores a computer program, and the processor is configured to perform the following operations based on the computer program:

[0014] For any target object, matching each two-dimensional facial key point obtained based on the frontal facial image of the target object with each vertex in the initial three-dimensional facial model of the target object to obtain each three-dimensional facial key point in the initial three-dimensional facial model;

[0015] Detecting a hole area of ​​the initial three-dimensional face model based on each triangular face patch in the initial three-dimensional face model to obtain a hole boundary point set;

[0016] For any one boundary point in the hole boundary point set, identifying a target vertex matching the boundary point from among the vertices of a preset standard face model based on the position coordinates of the boundary point and the position coordinates of the three-dimensional facial key points; and

[0017] Using the position coordinates of the boundary points and the position coordinates of the target vertices matching the boundary points, non-rigid registration is performed on the target vertices to obtain a non-rigid registration matrix of the target vertices;

[0018] Obtaining a transformation matrix of each internal point by using the non-rigid registration matrix and the position coordinates of each internal point in a target area of ​​the standard face model, wherein the target area is an area enclosed by each target vertex;

[0019] For any internal point, the internal point is transformed into the initial three-dimensional model using the transformation matrix of the internal point to obtain a reconstructed three-dimensional face model.

[0020] According to a third aspect of an embodiment of the present invention, a computer storage medium is provided, wherein the computer storage medium stores a computer program, and the computer program is used to execute the method according to the first aspect.

[0021] In the above-mentioned embodiment of the present application, by matching each 2D facial key point of the target object with each vertex in the initial 3D facial model of the target object, each 3D facial key point in the initial 3D facial model is obtained, and then each target vertex that matches each hole boundary point of the hole region of the initial 3D facial model is identified from each vertex of the preset standard facial model, and each target vertex is non-rigidly registered to obtain a non-rigid registration matrix for each target vertex. Then, the non-rigid registration matrix of each target vertex is used to determine the transformation matrix of each internal point in the target region of the standard facial model corresponding to the hole region in the initial 3D facial model, and then the transformation matrix of each internal point is used to transform the internal point into the initial 3D model to obtain a reconstructed 3D facial model. Thus, the present application inputs a standard facial model and uses the 2D facial key points extracted from the frontal facial image as a guide to perform non-rigid registration on the target region in the standard facial model corresponding to the hole region in the initial 3D model, thereby geometrically deforming the target region in the template into the hole region in the initial 3D model, thereby achieving hole filling of the 3D facial model. Since the present application introduces a standard face template, the face geometric information is retained to the greatest extent, the realism of the three-dimensional face model is improved, and the accuracy of the hole filling of the three-dimensional face model is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0023] Figure 1 A schematic diagram of a reconstructed three-dimensional face model provided by an embodiment of the present application is exemplarily shown;

[0024] Figure 2 One of the application scenario diagrams provided by the embodiment of the present application is exemplarily shown;

[0025] Figure 3 The second schematic diagram of the application scenario provided by the embodiment of the present application is exemplarily shown;

[0026] Figure 4 One of the flowcharts of the method for reconstructing a three-dimensional face model provided in an embodiment of the present application is exemplarily shown;

[0027] Figure 5 The following is an exemplary diagram of two-dimensional facial key points provided by an embodiment of the present application;

[0028] Figure 6 A schematic diagram illustrating key points of a three-dimensional face provided by an embodiment of the present application is exemplified;

[0029] Figure 7 The following is a schematic diagram illustrating a process for determining a set of hole boundary points provided by an embodiment of the present application;

[0030] Figure 8 A schematic diagram of triangular face patches in a partial area of ​​an initial three-dimensional face model provided by an embodiment of the present application is exemplarily shown;

[0031] Figure 9 A schematic diagram illustrating a preset standard face model provided in an embodiment of the present application is exemplified;

[0032] Figure 10 The following is a schematic diagram illustrating a process for determining a target vertex according to an embodiment of the present application;

[0033] Figure 11 A schematic diagram exemplarily shows the target area of ​​the standard face model provided in an embodiment of the present application;

[0034] Figure 12 The following is a schematic diagram illustrating a process for determining a transformation matrix of an internal point provided by an embodiment of the present application;

[0035] Figure 13 The second flowchart of the method for reconstructing a three-dimensional face model provided in an embodiment of the present application is exemplified;

[0036] Figure 14 The following is a schematic diagram illustrating the structure of a device for reconstructing a three-dimensional face model provided in an embodiment of the present application;

[0037] Figure 15 The hardware structure diagram of the calibration device provided in the embodiment of the present application is exemplified. DETAILED DESCRIPTION

[0038] In order to make the purpose, implementation mode and advantages of the present application clearer, the exemplary implementation mode of the present application will be clearly and completely described below in conjunction with the drawings in the exemplary embodiments of the present application. Obviously, the described exemplary embodiments are only part of the embodiments of the present application, not all of the embodiments.

[0039] All other embodiments derived by persons of ordinary skill in the art based on the exemplary embodiments described herein without inventive effort are within the scope of protection of the claims appended hereto. Furthermore, although the disclosure herein is presented based on one or more exemplary embodiments, it should be understood that each aspect of the disclosure may constitute a complete embodiment on its own.

[0040] It should be noted that the brief descriptions of terms in this application are only for the purpose of facilitating the understanding of the embodiments described below, and are not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their ordinary and usual meanings.

[0041] The terms "first," "second," and the like in the specification and claims of this application and in the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or sequential sequence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. In addition, the terms "including" and "having," and any variations thereof, are intended to cover, but not exclude, inclusion. For example, a product or device comprising a list of components is not necessarily limited to those components expressly listed, but may include other components not expressly listed or inherent to the product or device.

[0042] The term "module" as used in this application refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0043] The following is an overview of the concepts of the embodiments of the present application.

[0044] Current techniques for filling holes in 3D facial models typically use a nonlinear interpolation algorithm based on the mesh surrounding the hole. This approach fails to consider the geometric priors of the face. While successful, this approach results in a less realistic 3D facial model and reduces the accuracy of the hole-filling process.

[0045] Based on the problem of poor realism of three-dimensional face models after hole filling in the prior art, an embodiment of the present application provides a method for reconstructing a three-dimensional face model, by matching each two-dimensional face key point of the target object with each vertex in the initial three-dimensional face model of the target object to obtain each three-dimensional face key point in the initial three-dimensional face model, and then identifying each target vertex that matches each hole boundary point of the hole area of ​​the initial three-dimensional face model from each vertex of the preset standard face model, and performing non-rigid registration on each target vertex to obtain a non-rigid registration matrix of each target vertex, and then using the non-rigid registration matrix of each target vertex to determine the transformation matrix of each internal point in the target area in the standard face model corresponding to the hole area in the initial three-dimensional face model, and then using the transformation matrix of each internal point to transform the internal point to the initial three-dimensional model to obtain a reconstructed three-dimensional face model. Therefore, this application uses a standard face model as input and, guided by the two-dimensional facial key points extracted from the frontal face image, non-rigidly aligns the target area in the standard face model corresponding to the hole area in the initial three-dimensional model. This geometrically deforms the target area in the template into the hole area in the initial three-dimensional model, thereby achieving hole filling in the three-dimensional face model. Because this application introduces a standard face template, it retains facial geometric information to the greatest extent possible, improving the realism of the three-dimensional face model and the accuracy of hole filling in the three-dimensional face model.

[0046] The embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0047] Figure 2 The following is a schematic diagram showing an application scenario of the method for reconstructing a three-dimensional face model provided in an embodiment of the present application; Figure 2 As shown, this application scenario is described using an electronic device as a server. This application scenario includes a multi-view camera 210, a server 220, and a VR device 230. Server 220 can be implemented as a single server or multiple servers. Server 220 can be implemented as a physical server or a virtual server.

[0048] In a possible application scenario, for any target object, a multi-view image of the target object is obtained through a multi-view camera 210, and then the server 220 performs facial key point detection on the frontal face image in the multi-view image of the target object to obtain each two-dimensional facial key point of the target object; and matches each two-dimensional facial key point with each vertex in the initial three-dimensional face model of the target object to obtain each three-dimensional facial key point in the initial three-dimensional face model, wherein the initial three-dimensional face model is obtained based on the multi-view image of the target object. The server 220 then detects the hole area of ​​the initial three-dimensional face model based on the attributes of each triangular face in the initial three-dimensional face model to obtain a set of hole boundary points. For any boundary point in the set of hole boundary points, based on the position coordinates of the boundary point and the position coordinates of each three-dimensional facial key point, the server 220 identifies a target vertex that matches the boundary point from each vertex of the preset standard face model. The server 220 then uses the position coordinates of the boundary point and the position coordinates of the target vertex that matches the boundary point to perform non-rigid registration on the target vertex to obtain a non-rigid registration matrix for the target vertex. The server 220 uses the non-rigid registration matrix and the position coordinates of each internal point in the target area of ​​the standard face model to obtain a transformation matrix for each internal point, wherein the target area is determined based on the hole area in the initial three-dimensional face model. Finally, for any internal point, the server 220 uses the transformation matrix of the internal point to transform the internal point into the initial three-dimensional model to obtain a reconstructed three-dimensional face model, and sends the reconstructed three-dimensional face model to the VR device 230 for display.

[0049] like Figure 3 As shown, it is a schematic diagram of another application scenario of the method for reconstructing a three-dimensional face model provided in an embodiment of the present application, and the application scenario includes a camera 210, a server 220, a VR device 230 and a memory 240.

[0050] In a possible application scenario, for any target object, a multi-view image of the target object is obtained through a multi-view camera 210, and the multi-view image of the target object is stored in a memory 240. Then, the server 220 obtains the frontal face image of the multi-view image of the target object from the memory 240, and performs facial key point detection on the frontal face image to obtain the two-dimensional facial key points of the target object; and matches the two-dimensional facial key points with the vertices in the initial three-dimensional face model of the target object to obtain the three-dimensional facial key points in the initial three-dimensional face model, wherein the initial three-dimensional face model is obtained based on the multi-view image of the target object. The server 220 then detects the hole area of ​​the initial three-dimensional face model based on the attributes of each triangular face in the initial three-dimensional face model to obtain a set of hole boundary points. For any boundary point in the set of hole boundary points, based on the position coordinates of the boundary point and the position coordinates of each three-dimensional facial key point, the server 220 identifies a target vertex that matches the boundary point from each vertex of the preset standard face model. The server 220 then uses the position coordinates of the boundary point and the position coordinates of the target vertex that matches the boundary point to perform non-rigid registration on the target vertex to obtain a non-rigid registration matrix for the target vertex. The server 220 uses the non-rigid registration matrix and the position coordinates of each internal point in the target area of ​​the standard face model to obtain a transformation matrix for each internal point, wherein the target area is determined based on the hole area in the initial three-dimensional face model. Finally, for any internal point, the server 220 uses the transformation matrix of the internal point to transform the internal point into the initial three-dimensional model to obtain a reconstructed three-dimensional face model, and sends the reconstructed three-dimensional face model to the VR device 230 for display.

[0051] in, Figure 2 as well as Figure 3 The server 220 and the VR device 230 can exchange information through a communication network, wherein the communication mode adopted by the communication network can be divided into a wireless communication mode or a wired communication mode.

[0052] For example, the server 220 can access the network through cellular mobile communication technology to communicate with the VR device 230, and the server 220 can access the network through cellular mobile communication technology to communicate with the VR device 230. The cellular mobile communication technology, for example, includes the fifth generation mobile communication (5G) technology.

[0053] Optionally, the server 220 may access the network via short-range wireless communication to communicate with the VR device 230, and the server 220 may access the network via short-range wireless communication to communicate with the VR device 230. The short-range wireless communication method may include, for example, Wireless Fidelity (Wi-Fi) technology.

[0054] Furthermore, the description in this application only details three cameras 210, a single server 220, a single VR device 230, and a single storage 240. However, those skilled in the art should understand that the illustrated cameras 210, server 220, VR device 230, and storage 240 are intended to illustrate the operations of the cameras 210, server 220, VR device 230, and storage 240 involved in the technical solution of this application. This does not imply any limitation on the number, type, or location of the cameras 210, server 220, VR device 230, and storage 240. It should be noted that adding additional modules to the illustrated environment or removing individual modules from it does not change the underlying concepts of the exemplary embodiments of this application.

[0055] It should be noted that the reconstruction method of the three-dimensional face model proposed in this application is not only applicable to Figure 2 and Figure 3 The application scenario shown is also applicable to any device that reconstructs a three-dimensional face model.

[0056] The following describes the method for reconstructing a three-dimensional face model in an exemplary embodiment of the present application in combination with the application scenarios described above and with reference to the accompanying drawings. It should be noted that the above application scenarios are only shown to facilitate understanding of the methods and principles of the present application, and the implementation methods of the present application are not limited in this respect.

[0057] like Figure 4 FIG. 1 is a flow chart of a method for reconstructing a three-dimensional face model, which may include the following steps:

[0058] Step 401: For any target object, matching each 2D facial key point obtained based on the frontal face image of the target object with each vertex in the initial 3D face model of the target object to obtain each 3D facial key point in the initial 3D face model;

[0059] Among them, the two-dimensional face key points include but are not limited to the facial contour, lips, eyes, nose and eyebrows. The specific two-dimensional face key points can be set according to the actual situation, and this embodiment does not limit them here. In addition, each two-dimensional face key point identified in this embodiment contains the semantic information of each two-dimensional face key point. For example, Figure 5As shown, these are the detected two-dimensional facial key points. In this embodiment, each two-dimensional facial key point contains corresponding semantic information, i.e., an identifier. Each identifier has corresponding semantic information, i.e., which identifier is the eye, which identifier is the nose, etc.

[0060] It should be noted that in this embodiment, a preset facial key point detection algorithm is used to detect facial key points in the frontal face image of the target object. However, the facial key point detection algorithm can be set according to actual conditions, and this embodiment does not limit the facial key point detection algorithm.

[0061] In one embodiment, each 3D facial key point in the initial 3D facial model is obtained by:

[0062] For any two-dimensional facial key point in the frontal face image, a ray is emitted from the optical center of the camera and passed through the two-dimensional facial key point. The intersection of the ray and the point closest to the optical center of the camera in the initial three-dimensional face model is then determined as the three-dimensional facial key point in the initial three-dimensional face model that matches the two-dimensional facial key point. The position coordinates of the two-dimensional facial key point are converted using a preset algorithm to obtain the position coordinates of the three-dimensional facial key point in the initial three-dimensional face model that matches the two-dimensional facial key point. Figure 6 As shown in the figure, the white points are the key points of the three-dimensional face in the identified initial three-dimensional face model.

[0063] Step 402: detecting the hole area of ​​the initial 3D face model based on each triangular face patch in the initial 3D face model to obtain a hole boundary point set;

[0064] like Figure 6 As shown in the figure, the hole area of ​​the initial 3D face model can be seen.

[0065] like Figure 7 As shown in FIG, a flow chart of determining a set of void boundary points is shown, which includes the following steps:

[0066] Step 701: For any triangular patch in the initial 3D face model, if there is at least one edge in the triangular patch that belongs only to the triangular patch, and the position coordinates of the two vertices of the edge are within a specified range, then the two vertices of the edge are determined as boundary points of the hole area of ​​the initial 3D face model;

[0067] The requirement that the position coordinates of the two vertices of an edge be within a specified range in this embodiment is to avoid identifying the vertices of the edges of the triangular facets at the boundary of the initial 3D model as the boundary points of the hole region. Therefore, the position coordinates of the vertices are limited in this embodiment to improve the accuracy of the determined boundary points.

[0068] For example, Figure 8 The figure shows a schematic diagram of the triangular patch in some areas of the initial 3D face model. Figure 8 As can be seen from the figure, triangle patch 1 has at least one edge that belongs only to triangle patch 1, but triangle patch 2 does not have an edge that belongs only to triangle patch 2. Furthermore, triangle patch 1 does not belong to any of the triangle patches on the boundary of the initial 3D model. Therefore, vertices A and B of triangle patch 1 can be determined as the boundary points of the hole region.

[0069] Step 702: Based on the determined boundary points of the hole area, obtain the hole boundary point set.

[0070] In one embodiment, step 702 may be implemented as determining a set consisting of boundary points of the determined hole area as the hole boundary point set.

[0071] Step 403: for any boundary point in the hole boundary point set, identify a target vertex matching the boundary point from among the vertices of a preset standard face model based on the position coordinates of the boundary point and the position coordinates of the three-dimensional facial key points;

[0072] For example, Figure 9 Figure 1 shows a schematic diagram of a preset standard face model. The points in the figure are vertices corresponding to key points in the initial 3D face. Since all vertices in the preset standard face model are known, the vertices corresponding to key points in the initial 3D face can be directly determined.

[0073] like Figure 10 FIG. 1 is a flow chart showing a process of determining a target vertex, including the following steps:

[0074] Step 1001: for any boundary point in the hole boundary point set, obtain a feature vector of the boundary point based on the position coordinates of the boundary point and the position coordinates of each 3D facial key point;

[0075] In one embodiment, the feature vector of the boundary point is obtained by:

[0076] For any three-dimensional facial key point, the distance between the three-dimensional facial key point and the boundary point is determined based on the position coordinates of the three-dimensional facial key point and the position coordinates of the boundary point; based on the distances between the boundary point and each three-dimensional facial key point, the feature vector of the boundary point is obtained. The distance between the three-dimensional facial key point and the boundary point can be determined by formula (1):

[0077]

[0078] Among them, d is the distance between the 3D face key point and the boundary point, x1 is the horizontal coordinate of the 3D face key point, x2 is the horizontal coordinate of the boundary point, y1 is the vertical coordinate of the 3D face key point, y2 is the vertical coordinate of the boundary point, z1 is the vertical coordinate of the 3D face key point, and z2 is the vertical coordinate of the boundary point.

[0079] In one embodiment, the distances between the boundary points and the three-dimensional facial key points are used to form the feature vector of the boundary point. For example, the distances determined for boundary point 1 are A, B, C, and D, respectively, and the feature vector of boundary point 1 is determined to be [A, B, C, D].

[0080] Step 1002: For any vertex in the standard face model, obtain a feature vector of the vertex based on the position coordinates of the vertex and the position coordinates of each standard key point, wherein the each standard key point is each vertex in the standard face model having the same identifier as each 3D face key point;

[0081] In one embodiment, the feature vector of the vertex is obtained by:

[0082] For any standard key point, the distance between the vertex and the standard key point is obtained based on the position coordinates of the standard key point and the position coordinates of the vertex; and the feature vector of the vertex is obtained based on the distances between the vertex and each standard key point.

[0083] The distances between the vertex and each standard key point are respectively used to form a feature vector of the vertex.

[0084] It should be noted that the method of determining the distance between the vertex and the standard key point in this embodiment is the same as the method of determining the distance between the three-dimensional face key point and the boundary point as described above. This embodiment no longer limits the method of determining the distance between the vertex and the standard key point.

[0085] Step 1003: for any boundary point, determine the similarity between the boundary point and each vertex according to the feature vector of the boundary point and the feature vectors of each vertex;

[0086] In order to prevent the problem of scale inconsistency between the initial 3D face model and the preset standard face model, before executing step 1003 , the feature vectors of each boundary point and the feature vectors of each vertex need to be normalized.

[0087] The specific method of normalization processing can be set according to actual conditions, and this embodiment does not limit the specific method of normalization processing.

[0088] It should be noted that the method for determining similarity in this embodiment includes but is not limited to the Pearson correlation coefficient method and the Euclidean distance method. The specific method for determining similarity can be set according to actual conditions, and this embodiment does not limit the method for determining similarity in step 1003.

[0089] Step 1004: Determine the vertex with the highest similarity to the boundary point as the target vertex matching the boundary point.

[0090] For example, the similarity between vertex 1 and boundary point A is 0.8, the similarity between vertex 2 and boundary point A is 0.2, and the similarity between vertex 3 and boundary point A is 0.5. Then, the target vertex that matches boundary point A is determined to be vertex 1.

[0091] Step 404: performing non-rigid registration on the target vertices using the position coordinates of the boundary points and the position coordinates of the target vertices that match the boundary points, to obtain a non-rigid registration matrix for the target vertices.

[0092] The non-rigid registration method is a method in the prior art and will not be described in detail in this embodiment.

[0093] Step 405: using the non-rigid registration matrix and the position coordinates of each internal point in the target area of ​​the standard face model, obtaining a transformation matrix of each internal point, wherein the target area is the area enclosed by each target vertex;

[0094] like Figure 11 As shown in the figure, the target area of ​​the standard face model can be seen, where each point in the target area is a target point within the target area. The target area is the area corresponding to the hole area in the initial 3D model. The target area is composed of the target vertices.

[0095] like Figure 12 The figure shows a flow chart of determining the transformation matrix of an internal point, which includes the following steps:

[0096] Step 1201: For any internal point, based on the position coordinates of the internal point and the position coordinates of the target vertices, obtain the distance between the internal point and the target vertices;

[0097] Among them, the method of determining the distance between the internal point and the target vertex is the same as the method of determining the distance between the three-dimensional face key point and the boundary point described above. This embodiment no longer limits the method of determining the distance between the internal point and the target vertex.

[0098] Step 1202: Determine the weight between the internal point and each target vertex based on the distance between the internal point and each target vertex;

[0099] In one embodiment, the weights between the internal points and the target vertices are determined in the following manner:

[0100] The distances between the internal point and each target vertex are added to obtain the total distance; for any target vertex, the square of the total distance is divided by the square of the target distance to obtain the weight between the internal point and the target vertex, where the target distance is the distance between the internal point and the target vertex. The weight can be determined by formula (2):

[0101]

[0102] Among them, ω ij is the weight between internal point j and target vertex i, d ij is the distance between the internal point j and the target vertex i.

[0103] Step 1203: Obtain a transformation matrix of the internal point according to the weights between the internal point and each target vertex and the non-rigid registration matrix of each target vertex.

[0104] In one embodiment, the transformation matrix of the interior point is determined by:

[0105] For any target vertex, the weight between the internal point and the target vertex is multiplied by the non-rigid registration matrix of the target vertex to obtain the intermediate transformation matrix of the internal point; the intermediate transformation matrices of the internal points are added to obtain the transformation matrix of the internal point. The transformation matrix of the internal point can be obtained by formula (3):

[0106]

[0107] Among them, T j is the transformation matrix of the internal point j, is the non-rigid registration matrix of target vertex i.

[0108] Step 406: For any internal point, transform the internal point into the initial three-dimensional model using the transformation matrix of the internal point to obtain a reconstructed three-dimensional face model.

[0109] In one embodiment, the position coordinates of the internal points are transformed using a preset transformation algorithm and the transformation matrix of the internal points to obtain the coordinates of the internal points in the initial three-dimensional model, and a reconstructed three-dimensional face model is obtained based on the coordinates of each internal point in the initial three-dimensional model.

[0110] Among them, the specific transformation method in this embodiment can be set according to actual conditions, and this embodiment does not limit the specific transformation method.

[0111] To further enhance the authenticity of the 3D face model after the hole filling, after executing step 407, the boundary region of the reconstructed 3D face model where the hole is filled may still be uneven. Therefore, a mesh smoothing algorithm is used to locally smooth this region. The mesh smoothing algorithm can be selected from algorithms such as Laplace and can be set based on actual circumstances. This embodiment does not limit the mesh smoothing algorithm.

[0112] In order to further connect the technical solutions in this application, Figure 13 A detailed description may include the following steps:

[0113] Step 1301: for any target object, perform facial key point detection on the frontal face image of the target object to obtain the two-dimensional facial key points of the target object;

[0114] Step 1302: Matching the two-dimensional facial key points with vertices in an initial three-dimensional facial model of the target object to obtain three-dimensional facial key points in the initial three-dimensional facial model, wherein the initial three-dimensional facial model is obtained based on multi-view images of the target object;

[0115] Step 1303: For any triangular patch in the initial 3D face model, if there is at least one edge in the triangular patch that belongs only to the triangular patch, and the position coordinates of two vertices of the edge are within a specified range, then the two vertices of the edge are determined as boundary points of the hole area of ​​the initial 3D face model;

[0116] Step 1304: Obtain the hole boundary point set based on the determined boundary points of the hole area;

[0117] Step 1305: for any boundary point in the hole boundary point set, obtain a feature vector of the boundary point based on the position coordinates of the boundary point and the position coordinates of each 3D facial key point;

[0118] Step 1306: For any vertex in the standard face model, obtain a feature vector of the vertex based on the position coordinates of the vertex and the position coordinates of each standard key point, wherein the each standard key point is each vertex in the standard face model having the same identifier as each 3D face key point;

[0119] Step 1307: For any boundary point, determine the similarity between the boundary point and each vertex according to the feature vector of the boundary point and the feature vectors of each vertex;

[0120] Step 1308: Determine the vertex with the highest similarity to the boundary point as the target vertex matching the boundary point;

[0121] Step 1309: Using the position coordinates of the boundary points and the position coordinates of the target vertices matching the boundary points, perform non-rigid registration on the target vertices to obtain a non-rigid registration matrix for the target vertices.

[0122] Step 1310: For any internal point, based on the position coordinates of the internal point and the position coordinates of the target vertices, obtain the distance between the internal point and the target vertices;

[0123] Step 1311: For any internal point, based on the position coordinates of the internal point and the position coordinates of the target vertices, obtain the distance between the internal point and the target vertices;

[0124] Step 1312: Obtain a transformation matrix of the internal point according to the weights between the internal point and each target vertex and the non-rigid registration matrix of each target vertex;

[0125] Step 1313: For any internal point, use the transformation matrix of the internal point to transform the internal point into the initial three-dimensional model to obtain a reconstructed three-dimensional face model.

[0126] Based on the same inventive concept, the above-mentioned method for reconstructing a 3D face model can also be implemented by a 3D face model reconstruction device. The effect of the 3D face model reconstruction is similar to that of the above-mentioned method, and will not be repeated here.

[0127] Figure 14 The figure is a schematic structural diagram of a device for reconstructing a three-dimensional face model according to an embodiment of the present disclosure.

[0128] like Figure 14 As shown, the three-dimensional face model reconstruction device 1400 of the present disclosure may include a three-dimensional face key point recognition module 1410, a hole area detection module 1420, a target vertex determination module 1430, a non-rigid registration module 1440, a transformation matrix determination module 1450 and an internal point transformation module 1460.

[0129] The 3D facial key point recognition module 1410 is configured to, for any target object, match each 2D facial key point obtained based on the frontal facial image of the target object with each vertex in the initial 3D facial model of the target object to obtain each 3D facial key point in the initial 3D facial model;

[0130] a hole region detection module 1420 configured to detect hole regions of the initial 3D face model based on the triangular face patches in the initial 3D face model to obtain a set of hole boundary points;

[0131] a target vertex determination module 1430 for identifying, for any boundary point in the hole boundary point set, a target vertex matching the boundary point from among the vertices of a preset standard face model based on the position coordinates of the boundary point and the position coordinates of the three-dimensional facial key points;

[0132] A non-rigid registration module 1440 is configured to perform non-rigid registration on the target vertices using the position coordinates of the boundary points and the position coordinates of the target vertices that match the boundary points, to obtain a non-rigid registration matrix for the target vertices.

[0133] a transformation matrix determination module 1450 for obtaining a transformation matrix of each internal point in a target region of the standard face model using the non-rigid registration matrix and the position coordinates of the internal points in the target region, wherein the target region is the region enclosed by the target vertices;

[0134] The internal point transformation module 1460 is configured to transform any internal point into the initial three-dimensional model using the transformation matrix of the internal point to obtain a reconstructed three-dimensional face model.

[0135] In one embodiment, the hole region detection module 1420 is specifically configured to:

[0136] For any triangular face patch in the initial three-dimensional face model, if there is at least one edge in the triangular face patch that belongs only to the triangular face patch, and the position coordinates of two vertices of the edge are within a specified range, then the two vertices of the edge are determined as boundary points of the hole area of ​​the initial three-dimensional face model;

[0137] Based on the determined boundary points of the hole area, the hole boundary point set is obtained.

[0138] In one embodiment, the target vertex determination module 1430 is specifically configured to:

[0139] For any boundary point in the hole boundary point set, obtaining a feature vector of the boundary point based on the position coordinates of the boundary point and the position coordinates of each of the three-dimensional facial key points; and

[0140] For any vertex in the standard face model, obtaining a feature vector of the vertex based on the position coordinates of the vertex and the position coordinates of each standard key point, wherein the each standard key point is each vertex in the standard face model having the same identifier as each three-dimensional face key point;

[0141] For any boundary point, determine the similarity between the boundary point and each vertex according to the feature vector of the boundary point and the feature vector of each vertex; and

[0142] The vertex with the highest similarity to the boundary point is determined as the target vertex matching the boundary point.

[0143] In one embodiment, the target vertex determination module 1430 performs the step of obtaining the feature vector of the boundary point based on the position coordinates of the boundary point and the position coordinates of each 3D facial key point, and is specifically configured as follows:

[0144] For any three-dimensional facial key point, determine the distance between the three-dimensional facial key point and the boundary point according to the position coordinates of the three-dimensional facial key point and the position coordinates of the boundary point;

[0145] Based on the distances between the boundary points and each three-dimensional facial key point, a feature vector of the boundary point is obtained.

[0146] In one embodiment, the target vertex determination module 1430 performs the step of obtaining the feature vector of the vertex based on the position coordinates of the vertex and the position coordinates of each standard key point, and is specifically configured as follows:

[0147] For any standard key point, obtain the distance between the vertex and the standard key point according to the position coordinates of the standard key point and the position coordinates of the vertex;

[0148] Based on the distances between the vertex and each standard key point, a feature vector of the vertex is obtained.

[0149] In one embodiment, the transformation matrix determination module 1450 is specifically configured to:

[0150] For any internal point, based on the position coordinates of the internal point and the position coordinates of the target vertices, obtain the distance between the internal point and the target vertices;

[0151] Determining a weight between the internal point and each target vertex based on a distance between the internal point and each target vertex;

[0152] A transformation matrix of the internal point is obtained according to the weights between the internal point and each target vertex and the non-rigid registration matrix of each target vertex.

[0153] In one embodiment, the transformation matrix determination module 1450 performs the determination of the weights between the interior point and each target vertex based on the distance between the interior point and each target vertex, and is specifically configured as follows:

[0154] Adding the distances between the internal point and each target vertex to obtain a total distance;

[0155] For any target vertex, the square of the total distance is divided by the square of the target distance to obtain the weight between the internal point and the target vertex, wherein the target distance is the distance between the internal point and the target vertex.

[0156] In one embodiment, the transformation matrix determination module 1450 performs the step of obtaining the transformation matrix of the internal point based on the weights between the internal point and each target vertex and the non-rigid registration matrix of each target vertex, and is specifically configured as follows:

[0157] For any target vertex, multiply the weight between the internal point and the target vertex by the non-rigid registration matrix of the target vertex to obtain an intermediate transformation matrix of the internal point;

[0158] The intermediate transformation matrices of the interior points are added together to obtain the transformation matrix of the interior points.

[0159] After introducing a method and apparatus for reconstructing a three-dimensional face model according to an exemplary embodiment of the present invention, an electronic device according to another exemplary embodiment of the present invention is introduced next.

[0160] Those skilled in the art will appreciate that various aspects of the present invention may be implemented as systems, methods, or program products. Therefore, various aspects of the present invention may be implemented in the following forms: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software implementations, which may be collectively referred to herein as "circuits," "modules," or "systems."

[0161] In some possible implementations, the electronic device according to the present invention may include at least one processor and at least one computer storage medium. The computer storage medium stores program code, and when the program code is executed by the processor, the processor executes the steps of the method for reconstructing a three-dimensional face model according to various exemplary embodiments of the present invention described above. For example, the processor may execute the following steps: Figure 4 Steps 401-407 shown in .

[0162] Refer to the following Figure 15 An electronic device 1500 according to this embodiment of the present invention will be described. Figure 15 The electronic device 1500 shown is merely an example and should not limit the functions and scope of use of the embodiments of the present invention.

[0163] like Figure 15 As shown, electronic device 1500 is a general electronic device. Components of electronic device 1500 may include, but are not limited to, at least one processor 1501, at least one computer storage medium 1502, and a bus 1503 connecting different system components (including computer storage medium 1502 and processor 1501).

[0164] Bus 1503 represents one or more of several types of bus structures, including a computer storage media bus or computer storage media controller, a peripheral bus, a processor, or a local bus using any of a variety of bus architectures.

[0165] Computer storage media 1502 may include readable media in the form of volatile computer storage media, such as random access computer storage media (RAM) 1521 and / or cache storage media 1522 , and may further include read-only computer storage media (ROM) 1523 .

[0166] The computer storage medium 1502 may also include a program / utility 1525 having a set (at least one) of program modules 1524, such program modules 1524 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0167] Electronic device 1500 may also communicate with one or more external devices 1504 (e.g., a keyboard, pointing device, etc.), one or more devices that enable a user to interact with electronic device 1500, and / or any device that enables electronic device 1500 to communicate with one or more other electronic devices (e.g., a router, a modem, etc.). Such communication may occur via input / output (I / O) interface 1505. Furthermore, electronic device 1500 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via network adapter 1506. As shown, network adapter 1506 communicates with other modules of electronic device 1500 via bus 1503. It should be understood that, although not shown, other hardware and / or software modules may be used in conjunction with electronic device 1500, including but not limited to microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0168] In some possible embodiments, various aspects of the method for reconstructing a three-dimensional face model provided by the present invention can also be implemented in the form of a program product, which includes program code. When the program product is run on a computer device, the program code is used to enable the computer device to execute the steps of the method for reconstructing a three-dimensional face model according to various exemplary embodiments of the present invention described above in this specification.

[0169] The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: an electrical connection having one or more wires, a portable disk, a hard disk, a random access computer storage medium (RAM), a read-only computer storage medium (ROM), an erasable programmable read-only computer storage medium (EPROM or flash memory), an optical fiber, a portable compact disk read-only computer storage medium (CD-ROM), an optical computer storage medium, a magnetic computer storage medium, or any suitable combination thereof.

[0170] The program product for reconstructing a three-dimensional face model according to an embodiment of the present invention may be a portable compact disc read-only computer storage medium (CD-ROM) and include program code, and may be run on an electronic device. However, the program product of the present invention is not limited thereto. In this document, a readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0171] A readable signal medium may include a data signal transmitted in baseband or as part of a carrier wave, which carries readable program code. Such a transmitted data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium that can transmit, propagate, or transfer a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0172] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0173] The program code for performing the operations of the present invention can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user electronic device, partially on the user device, as a separate software package, partially on the user electronic device and partially on a remote electronic device, or entirely on the remote electronic device or server. In cases involving remote electronic devices, the remote electronic device can be connected to the user electronic device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external electronic device (for example, using an Internet service provider to connect through the Internet).

[0174] It should be noted that although several modules of the device are mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of the present invention, the features and functions of two or more modules described above may be embodied in a single module. Conversely, the features and functions of a single module described above may be further divided and embodied by multiple modules.

[0175] Furthermore, although the operations of the method of the present invention are described in a particular order in the accompanying drawings, this does not require or imply that these operations must be performed in this particular order, or that all illustrated operations must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.

[0176] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk computer storage media, CD-ROMs, optical computer storage media, etc.) containing computer-usable program code.

[0177] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0178] These computer program instructions may also be stored in a computer-readable computer storage medium that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable computer storage medium produce an article of manufacture including an instruction device that implements the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0179] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0180] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if such changes and modifications fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.

Claims

1. A method for reconstructing a three-dimensional face model, characterized in that: The method comprises: For any target object, matching each two-dimensional facial key point obtained based on the frontal facial image of the target object with each vertex in the initial three-dimensional facial model of the target object to obtain each three-dimensional facial key point in the initial three-dimensional facial model; Detecting a hole area of ​​the initial three-dimensional face model based on each triangular face patch in the initial three-dimensional face model to obtain a hole boundary point set; For any one boundary point in the hole boundary point set, identifying a target vertex matching the boundary point from among the vertices of a preset standard face model based on the position coordinates of the boundary point and the position coordinates of the three-dimensional facial key points; and Using the position coordinates of the boundary points and the position coordinates of the target vertices matching the boundary points, non-rigid registration is performed on the target vertices to obtain a non-rigid registration matrix of the target vertices; Obtaining a transformation matrix of each internal point by using the non-rigid registration matrix and the position coordinates of each internal point in a target area of ​​the standard face model, wherein the target area is an area enclosed by each target vertex; For any internal point, the internal point is transformed into the initial three-dimensional model using the transformation matrix of the internal point to obtain a reconstructed three-dimensional face model.

2. The method according to claim 1, characterized in that The detecting of the hole area of ​​the initial three-dimensional face model based on each triangular face patch in the initial three-dimensional face model to obtain a hole boundary point set includes: For any triangular face patch in the initial three-dimensional face model, if there is at least one edge in the triangular face patch that belongs only to the triangular face patch, and the position coordinates of two vertices of the edge are within a specified range, then the two vertices of the edge are determined as boundary points of the hole area of ​​the initial three-dimensional face model; Based on the determined boundary points of the hole area, the hole boundary point set is obtained.

3. The method according to claim 1, characterized in that The step of identifying target vertices matching the boundary points from vertices of a preset standard face model according to the position coordinates of the boundary points and the position coordinates of the three-dimensional facial key points includes: For any boundary point in the hole boundary point set, obtaining a feature vector of the boundary point based on the position coordinates of the boundary point and the position coordinates of each of the three-dimensional facial key points; and For any vertex in the standard face model, obtaining a feature vector of the vertex based on the position coordinates of the vertex and the position coordinates of each standard key point, wherein the each standard key point is each vertex in the standard face model having the same identifier as each three-dimensional face key point; For any boundary point, determine the similarity between the boundary point and each vertex according to the feature vector of the boundary point and the feature vector of each vertex; and The vertex with the highest similarity to the boundary point is determined as the target vertex matching the boundary point.

4. The method according to claim 3, characterized in that The step of obtaining a feature vector of the boundary point based on the position coordinates of the boundary point and the position coordinates of each of the three-dimensional facial key points includes: For any three-dimensional facial key point, determine the distance between the three-dimensional facial key point and the boundary point according to the position coordinates of the three-dimensional facial key point and the position coordinates of the boundary point; Based on the distances between the boundary points and each three-dimensional facial key point, a feature vector of the boundary point is obtained.

5. The method according to claim 3, characterized in that The step of obtaining a feature vector of the vertex based on the position coordinates of the vertex and the position coordinates of each standard key point includes: For any standard key point, obtain the distance between the vertex and the standard key point according to the position coordinates of the standard key point and the position coordinates of the vertex; Based on the distances between the vertex and each standard key point, a feature vector of the vertex is obtained.

6. The method according to claim 1, wherein The method of obtaining a transformation matrix of each internal point by using the non-rigid registration matrix and the position coordinates of each internal point in the target area of ​​the standard face model comprises: For any internal point, based on the position coordinates of the internal point and the position coordinates of the target vertices, obtain the distance between the internal point and the target vertices; Determining a weight between the internal point and each target vertex based on a distance between the internal point and each target vertex; A transformation matrix of the internal point is obtained according to the weights between the internal point and each target vertex and the non-rigid registration matrix of each target vertex.

7. The method according to claim 6, characterized in that The determining, based on the distance between the internal point and each target vertex, the weight between the internal point and each target vertex includes: Adding the distances between the internal point and each target vertex to obtain a total distance; For any target vertex, the square of the total distance is divided by the square of the target distance to obtain the weight between the internal point and the target vertex, wherein the target distance is the distance between the internal point and the target vertex.

8. The method according to claim 6, characterized in that The step of obtaining a transformation matrix of the internal point according to the weights between the internal point and each target vertex and the non-rigid registration matrix of each target vertex includes: For any target vertex, multiply the weight between the internal point and the target vertex by the non-rigid registration matrix of the target vertex to obtain an intermediate transformation matrix of the internal point; The intermediate transformation matrices of the interior points are added together to obtain the transformation matrix of the interior points.

9. An electronic device, characterized in that: comprising a processor and a memory, wherein the processor and the memory are connected via a bus; The memory stores a computer program, and the processor is configured to perform the following operations based on the computer program: For any target object, matching each two-dimensional facial key point obtained based on the frontal facial image of the target object with each vertex in the initial three-dimensional facial model of the target object to obtain each three-dimensional facial key point in the initial three-dimensional facial model; Detecting a hole area of ​​the initial three-dimensional face model based on each triangular face patch in the initial three-dimensional face model to obtain a hole boundary point set; For any boundary point in the hole boundary point set, identifying a target vertex matching the boundary point from among the vertices of a preset standard face model based on the position coordinates of the boundary point and the position coordinates of the three-dimensional facial key points; as well as, Using the position coordinates of the boundary points and the position coordinates of the target vertices matching the boundary points, non-rigid registration is performed on the target vertices to obtain a non-rigid registration matrix of the target vertices; Obtaining a transformation matrix of each internal point by using the non-rigid registration matrix and the position coordinates of each internal point in a target area of ​​the standard face model, wherein the target area is an area enclosed by each target vertex; For any internal point, the internal point is transformed into the initial three-dimensional model using the transformation matrix of the internal point to obtain a reconstructed three-dimensional face model.

10. The electronic device according to claim 9, wherein: The processor performs the detection of the hole area of ​​the initial three-dimensional face model based on each triangular face patch in the initial three-dimensional face model to obtain a hole boundary point set, and is specifically configured as follows: For any triangular face patch in the initial three-dimensional face model, if there is at least one edge in the triangular face patch that belongs only to the triangular face patch, and the position coordinates of two vertices of the edge are within a specified range, then the two vertices of the edge are determined as boundary points of the hole area of ​​the initial three-dimensional face model; Based on the determined boundary points of the hole area, the hole boundary point set is obtained.

Images