Method for calculating volume difference of face model based on three-dimensional grid
Through face key point detection and closest point iteration algorithm, the registration and slow speed problems in the calculation of the volume difference of the face model are solved, and accurate volume difference calculation and fast model comparison are realized, which is suitable for the application of multi-faceted film models. Scene.
Patent Information
- Application Number
- CN202411038971.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-31
- Publication Date
- 2024-11-15
AI Technical Summary
It is difficult to accurately calculate the volume difference between two face models in the prior art, especially the registration and volume detection of non-rigid body models, especially when there are problems such as inaccurate and slow speed, especially when the number of face sheets is large.
The rotation translation matrix is calculated through face key point detection and closest point iterative algorithm, and the volume difference of irregular surfaces is converted into the calculation problem of distance and area. The spatial volume difference is solved by approximating the intersection points and average distances of two-dimensional curves and straight lines. Simulate the dough sheet as a triangular body to calculate the volume difference.
Accurate registration and volume difference calculation of non-rigid body face models are realized, which improves calculation speed, saves auxiliary measurement costs, improves user experience, and is suitable for rapid comparison of multi-faceted film models.
Smart Images

Figure CN118967931A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of 3D face modeling, and in particular to a method for calculating volume differences of face models based on three-dimensional grids. Background Art
[0002] Currently, there are perfect face scanning methods in the field of face 3D modeling, which can obtain accurate face 3D models and textures, but in terms of post-processing and application of the model;
[0003] Currently, it is mainly used as an interface element or for simple 3D dimension measurement. The 3D information of the model itself is rarely used, especially when it is necessary to compare models of the same face taken at different times. Regarding volume detection, the existing patents have not yet published a method for comparing the volume of a face before and after.
[0004] The face is a non-rigid body, and the face model obtained through 3D reconstruction is also a non-rigid body model. The registration between two non-rigid bodies cannot be simply performed through the closest point iterative registration directly through the surface. Registration is a difficult problem.
[0005] Currently, it is difficult to calculate the volume difference between two face models for non-closed models with open boundaries, which leads to inaccurate and inconsistent volume calculations.
[0006] Moreover, the current calculation speed is relatively slow. It is acceptable for models with a small number of faces, but a fast method is needed to calculate the volume difference between two models with hundreds of thousands of faces. Summary of the invention
[0007] The purpose of the present invention is to provide a method for calculating the volume difference of face models based on three-dimensional grids, so as to solve the problem that the face is a non-rigid body proposed in the above background technology, and the face model obtained by three-dimensional reconstruction is also a non-rigid body model. The registration between two non-rigid bodies cannot be simply performed by directly performing closest point iterative registration through the surface. Registration is a difficult problem. At present, calculating the volume difference between two face models is a relatively difficult task for models with open boundaries, resulting in inaccuracy and inconsistency when calculating the volume. In addition, the current calculation speed is slow. The speed is acceptable for models with a small number of patches, but a fast method is needed to calculate the volume difference between two models with hundreds of thousands of patches.
[0008] To achieve the above object, the present invention provides the following technical solution: a method for calculating the volume difference of a face model based on a three-dimensional grid, comprising the following steps:
[0009] Step 1:
[0010] Input two 3D face models A and B. The model files include vertex coordinates v, texture coordinates vt, vertex normal vectors vn, face information index f, and corresponding texture images A_IMG and B_IMG.
[0011] Step 2:
[0012] Through any face key point detection algorithm, the texture image A_IMG (such as Figure 3 As shown) is input into the detection algorithm to obtain a set of two-dimensional key point information:
[0013] 2d_key{[u0, v0], [u1, v1]…, [u n , v n ]};
[0014] Step 3:
[0015] Since the starting points of texture coordinates and pixel coordinates are different, it is necessary to unify the vt coordinate system with the 2d_key coordinate system:
[0016] 2d_key_vt{[U0, V0], [U1, V1]…, [U n , V n ]}=2d_key{[u0, 1-v0], [u1, 1-v1]..., [u n , 1-v n ]};
[0017] Step 4:
[0018] For the 2d_key_vt obtained in step 3, calculate the distance between 2d_key_vt and the texture coordinate vt of each vertex, that is:
[0019]
[0020] dis is the distance from each key point to all uv coordinates. After calculation, we get the distance set:
[0021] 2d_distance{[dis1],[dis2]…[dis m ]};
[0022] Through the key point set 2d_distance, find the minimum distance min_distance, this min_distance is the texture coordinate vt corresponding to the key point, and then find the vertex coordinate v corresponding to the texture coordinate vt;
[0023] Therefore, we can get a set of 3D key points:
[0024] 3d_key{[x0,y0,z0],[x1,y1,z1],…,[x n ,y n , z n ]};
[0025] Step 5:
[0026] Use the nearest point iteration algorithm to calculate the rotation and translation matrices of two sets of 3d_key: R·, R is the rotation matrix, T is the translation matrix;
[0027] Step 6:
[0028] In order to obtain a more accurate fit between the two models A and B, it is necessary to use the nearest point iteration algorithm to calculate the rotation and translation matrix R'T' between all the vertices of A and B. Through steps 5 and 6, the final displacement result B' of the B model can be obtained;
[0029] In order to calculate the volume difference between two open models, the volume difference between the model surfaces of specific irregular surfaces is converted into a distance and area calculation problem. Taking an ideal two-dimensional curve and a straight line as an example, both the straight line and the curve are composed of several points. The normal vector direction of the point on straight line A can be used to obtain the intersection point from the point to B', thereby obtaining the distance in the normal direction of all points from A to B', and calculating the average distance corresponding to each point. In this way, the area within the closed area can be approximated by the distance between the intersection points and the average distance, thereby advancing to the space surface and the plane, and the space distance corresponding to all points in the intersection can be calculated, thus obtaining the average distance corresponding to each space point, thus obtaining the volume of the closed area through the patch area and the average distance between the intersections. The obtained volume is the volume difference between the two open models.
[0030] Further preferably, when actually calculating the volume, the distances corresponding to the three vertices of each facet are required, and each facet is simulated as a triangular prism. The volume of the triangular prism is calculated by the average value of the three heights of the triangular prism and the area of the bottom triangle. The selected area is composed of a known number of triangular facets, so the volume corresponding to each triangular facet in the area is calculated and summed, and the volume difference between the two open models in the entire area can be obtained, which is the visualized volume difference, and it can be seen that the protrusion position is detected.
[0031] Further preferably, the models A and B input in step 1 contain index information that can be used to calculate the vertex coordinates v 匹配 To texture coordinate v t , and the two-dimensional key point set obtained in step 2 2d _ k ey, the corresponding u, v values and texture coordinates vt are from the same picture.
[0032] Further preferably, in step 4, we obtain the corresponding 3D key point sets of the two groups of face models A and B. 3d _ k eyA and 3d _ k eyB, since the key points contain semantic information, such as the corners of the mouth, 眼 The inner corner point, the outer corner point of the eye, etc., so the semantic 3d _ k eyA and 3d _ k eyB are aligned one by one.
[0033] Further preferably, the R · T Displacement Model B , get the model B The initial displacement.
[0034] Compared with the prior art, the present invention has the following beneficial effects:
[0035] This method of calculating the volume difference of face models based on three-dimensional grids can effectively align two face models of the same person, solving the problem of inaccurate registration of non-rigid grid models. The registration effect is stable on the same face because the changes in key point features before and after are smaller than those in facial muscles.
[0036] In actual use, the distance set with vertex index is used as a medium to align the two-dimensional face information and the three-dimensional face model information, so that the recognition and processing methods of the two-dimensional image can be applied to the three-dimensional model;
[0037] It can easily calculate the spatial volume difference without the need for auxiliary fixtures, measuring tools and other auxiliary measurements, saving the cost of measuring tools and improving the user experience of the person being measured. The data collection of the function can be completed by just taking a normal photo.
[0038] When using the present invention for measurement, since only part of the facial features are used, such as feature points, face vertex information, normal vectors, etc., when applied to the comparison of three-dimensional face models with a large number of face patches, less resources are consumed and the time is faster than the global comparison speed;
[0039] Furthermore, the volume difference between the open models of two faces is calculated by abstracting the volume difference as the distance between each face multiplied by the set of triangular face areas for approximate solution. The two non-rigid face models are aligned using facial key points and global registration, which is conducive to the subsequent comparison of models. It can be used in similar application scenarios, that is, comparing two models, or comparing two point clouds, etc., and can effectively distinguish the change trend of the model itself, which has application value in some scenarios where model changes need to be observed.
[0040] Furthermore, in order to calculate the volume difference between two open models, the present invention designs a volume simulation calculation method, which converts the volume difference between the model surfaces of specific irregular curved surfaces into a distance and area calculation problem;
[0041] In the present invention, an ideal two-dimensional curve and a straight line are taken as examples. Both the straight line and the curve are composed of a number of points. The normal vector direction of the point on the straight line A can be used to obtain the intersection point from the point to B', thereby obtaining the distance in the normal direction of all points from A to B', and calculating the average distance corresponding to each point. In this way, the area within the closed region can be approximately obtained through the distance between the intersection points and the average distance. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] Figure 1 It is a schematic diagram of the two-dimensional volume detection method of the present invention;
[0043] Figure 2 Schematic diagram of Model A and Model B of the present invention;
[0044] Figure 3 A schematic diagram of the texture image structure taking A_IMG of the present invention as an example;
[0045] Figure 4 It is a schematic diagram of the 2D key point structure detected by the texture map A_IMG of the present invention;
[0046] Figure 5 This is a schematic diagram of a three-dimensional key point visualization structure using model A as an example of the present invention;
[0047] Figure 6 It is a two-dimensional schematic diagram of volume detection of the present invention;
[0048] Figure 7 It is a three-dimensional schematic diagram of volume detection of the present invention;
[0049] Figure 8 Schematic diagram of the volume difference visualization result (two viewing angles: front and side) of the present invention. DETAILED DESCRIPTION
[0050] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0051] Example 1: Please refer to Figure 1-8The present invention provides the following technical solution: A method for calculating the volume difference of face models based on three-dimensional grids, including a detailed implementation process: inputting two three-dimensional face models A and B, such as Figure 2 As shown, the model file includes vertex coordinates v, texture coordinates vt, vertex normal vector vn, face information index f and corresponding texture images A_IMG, B_IMG;
[0052] Through any facial key point detection algorithm, texture images A_IMG and B_IMG (such as Figure 3 As shown) is input into the detection algorithm to obtain a set of two-dimensional key point information:
[0053] 2d_key{[u0, v0], [u1, v1]…, [u n , v n ]};
[0054] like Figure 4 As shown, it is the result of visualizing the coordinates of two-dimensional key points;
[0055] The models A and B entered in step 1 contain index information that can match the vertex coordinate v to the texture coordinate vt, and the two-dimensional key point set 2d_key obtained in step 2, the corresponding u, v values and texture coordinate vt are the same picture. Since the starting points of texture coordinates and pixel coordinates are different, the coordinate system of vt needs to be unified with the 2d_key coordinate system:
[0056] 2d_key_vt{[U0, V0], [U1, V1]…, [U n , V n ]}=2d_key{[u0, 1-v0], [u1, 1-v1]..., [u n , 1-v n ]};
[0057] For the 2d_key_vt obtained in step 3, we can calculate the distance between 2d_key_vt and the texture coordinate vt of each vertex, that is:
[0058]
[0059] dis is the distance from each key point to all uv coordinates. After calculation, we get the distance set:
[0060] 2d_distance{[dis1],[dis2]…[dis m ]};
[0061] Through the key point set 2d_distance, find the minimum distance min_distance, this min_distance is the texture coordinate vt corresponding to the key point, and then find the vertex coordinate v corresponding to the texture coordinate vt;
[0062] Therefore, we can get a set of 3D key points:
[0063] 3d_key{[x0,y0,z0],[x1,y1,z1],…,[x n ,y n , z n ]};
[0064] like Figure 5 As shown, it is the result after the three-dimensional key point coordinates are visualized;
[0065] In step 4, we obtained the corresponding 3d key point sets 3d_keyA and 3d_keyB of the two sets of face models A and B. Since the key points have semantic information, such as the corners of the mouth, the inner corners of the eyes, and the outer corners of the eyes, we use semantics to align 3d_keyA and 3d_keyB one by one, and use the nearest point iteration algorithm to calculate the rotation and translation matrices of the two sets of 3d_key: R·T, R is the rotation matrix, and T is the translation matrix.
[0066] The R·T displacement model B obtained in step 5 obtains the initial displacement of model B. In order to obtain a more accurate fit between the two models A and B, it is necessary to use the nearest point iteration algorithm to calculate the rotation and translation matrix R'T' between all vertices of A and B. Through steps 5 and 6, the final displacement result B' of model B can be obtained.
[0067] In order to calculate the volume difference between two open models, the present invention designs a volume simulation calculation method to convert the volume difference between the model surfaces of specific irregular surfaces into a distance and area calculation problem. In the present invention, an ideal two-dimensional curve and a straight line are taken as an example. Figure 6 As shown in the figure, both the straight line and the curve are composed of several points. The normal vector direction of the point on the straight line A can be used to find the intersection point from the point to B'. From this, the distance of all the normal directions of the points from A to B' can be obtained, and the average distance corresponding to each point can be calculated. In this way, the area within the closed region can be approximately obtained through the distance between the intersection points and the average distance. Figure 7 As shown, by moving forward to the space surface and the plane, the spatial distance corresponding to all points in the intersection can be calculated, so that the average distance corresponding to each spatial point can be obtained. In this way, the volume of the closed area can be approximated by the patch area and average distance between the intersections. The obtained volume is the volume difference between the two open models.
[0068] When calculating the volume, we need the distances corresponding to the three vertices of each facet. We simulate each facet as a triangular prism and calculate the volume of the triangular prism by taking the average of the three heights and the area of the bottom triangle. The selected area is composed of a known number of triangular facets. Therefore, by calculating and summing the volumes corresponding to each triangular facet in the area, we can get the volume difference between the two open models in the area. Figure 8 As shown, it is the volume difference after visualization, and it can be seen that the bulge position is detected;
[0069] like Figure 1 As shown, firstly, two three-dimensional face models A and B with textures are input, and the AB model needs to have complete face texture information and a complete mesh structure (texture coordinates, vertex coordinates, vertex normal vectors and patch indexes); the face texture information is detected to obtain the pixel coordinates of the face key points on the texture map, and the number of face key points in the present invention is n; the texture coordinates of each vertex in the model are searched using the obtained two-dimensional face key points, and the texture coordinates of the vertices corresponding to the n key points are matched, thereby obtaining the texture coordinates of the vertices corresponding to the n key points, that is, obtaining the 3D coordinates of the corresponding face key points; two sets of 3D key points are used as language The semantics are one-to-one correspondence, that is, the corresponding order of the key points of each face is the same. With the help of the same semantics, the corresponding nearest points are iterated to calculate the total spatial displacement matrix after weighted average of n points, that is, the spatial rotation and translation matrix RT from model B to model A. According to RT, model B is displaced to a new position to obtain two spatially overlapping models A and B'; with model A as the benchmark and model B' as the control group, the normal closest distance of each vertex in model A to model B' is calculated, and the distance is used as a "distance" attribute of each point; the volume difference between A and B' is calculated through the area and distance attribute set of the patch set.
[0070] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "connected" and "connection" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.
[0071] Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments, or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.
Claims
1. A method for calculating the volume difference of face models based on three-dimensional grids, It is characterized in that The following steps are involved: Step 1: Input two 3D face models A and B. The model files include vertex coordinates v, texture coordinates vt, vertex normal vectors vn, face information index f, and corresponding texture images A_IMG and B_IMG. Step 2: Through any facial key point detection algorithm, the texture images A_IMG and B_IMG (as shown in Figure 3) are input into the detection algorithm to obtain a set of several two-dimensional key point information: 2d_key{[u 0 ,v 0 ],[u 1 ,v 1 ]…,[u n ,v n ]}; Step 3: Since the starting points of texture coordinates and pixel coordinates are different, it is necessary to unify the vt coordinate system with the 2d_key coordinate system: 2d_key_vt{[U 0 ,V 0 ],[U 1 ,V 1 ]…,[U n ,V n ]}=2d_key{[u 0 ,1-v 0 ],[u 1 ,1-v 1 ]…,[u n ,1-v n ]}; Step 4: For the 2d_key_vt obtained in step 3, calculate the distance between 2d_key_vt and the texture coordinate vt of each vertex, that is: dis is the distance from each key point to all uv coordinates. After calculation, we get the distance set: 2d_distance{[dis 1 ],[dis 2 ]…[dis m ]}; Through the key point set 2d_dostance, find the minimum distance min_distance, this min_distance is the texture coordinate vt corresponding to the key point, and then find the vertex coordinate v corresponding to the texture coordinate vt; Therefore, we can get a set of 3D key points: 3d_key{[x 0 ,y 0 ,z 0 ],[x 1 ,y 1 ,z 1 ],…,[x n ,y n ,z n ]}; Step 5: Use the nearest point iteration algorithm to calculate the rotation and translation matrices of two sets of 3d_key: R·T, R is the rotation matrix, T is the translation matrix; Step 6: In order to obtain a more accurate fit between the two models A and B, it is necessary to use the nearest point iteration algorithm to calculate the rotation and translation matrix R'T' between all the vertices of A and B. Through steps 5 and 6, the final displacement result B' of the B model can be obtained; In order to calculate the volume difference between two open models, the volume difference between the model surfaces of specific irregular surfaces is converted into a distance and area calculation problem. Taking an ideal two-dimensional curve and a straight line as an example, both the straight line and the curve are composed of several points. The normal vector direction of the point on straight line A can be used to obtain the intersection point from the point to B', thereby obtaining the distance in the normal direction of all points from A to B', and calculating the average distance corresponding to each point. In this way, the area within the closed area can be approximated by the distance between the intersection points and the average distance, thereby advancing to the space surface and the plane, and the space distance corresponding to all points in the intersection can be calculated, thus obtaining the average distance corresponding to each space point, thus obtaining the volume of the closed area through the patch area and the average distance between the intersections. The obtained volume is the volume difference between the two open models.
2. The method for calculating the volume difference of a face model based on a three-dimensional grid according to claim 1, Features: When actually calculating the volume, the distances corresponding to the three vertices of each facet are needed. Each facet is simulated as a triangular prism, and the volume of the triangular prism is calculated by the average value of the three heights of the triangular prism and the area of the bottom triangle. The selected area is composed of a known number of triangular facets. Therefore, the volume corresponding to each triangular facet in the area is calculated and summed up to obtain the volume difference between the two open models in the entire area, which is the visualized volume difference. It can be seen that the protrusion position is detected.
3. The method for calculating the volume difference of a face model based on a three-dimensional grid according to claim 1, Features: The models A and B input in step 1 contain index information that can match the vertex coordinate v to the texture coordinate vt, and the two-dimensional key point set 2d_key obtained in step 2, the corresponding u, v values and texture coordinate vt are the same picture.
4. The method for calculating the volume difference of a face model based on a three-dimensional grid according to claim 1, Features: In step 4, we obtain the corresponding 3d key point sets 3d_key of the two groups of face models A and B A With 3d_key B Since the key points have semantic information, such as the corners of the mouth, the inner corners of the eyes, and the outer corners of the eyes, 3d_key A With 3d_key B Align one by one.
5. The method for calculating the volume difference of a face model based on a three-dimensional grid according to claim 1, Features: The RT displacement model B obtained in step 5 obtains the initial displacement of model B.
Citation Information
Cited By
Face aging detection method and system, electronic equipment and computer program product
CN120976116A
