A low-cost high-efficiency three-dimensional face information acquisition system and method

Abstract

The application discloses a low-cost and high-efficiency three-dimensional face information acquisition system and method, belongs to the technical field of three-dimensional data acquisition, and comprises the steps of determining a shooting position, arranging a left RGB camera, a right RGB camera and a middle RGB camera, arranging a left 3D module, arranging a right 3D module, jointly calibrating, obtaining internal parameters and external parameters, calibrating the left RGB camera, the right RGB camera and the middle RGB camera, collecting images, obtaining a plurality of 2D images and a plurality of 3D point clouds, performing face segmentation on each 2D image by using a semantic segmentation model, filtering background point clouds, performing face key point recognition on each 2D image, fusing point clouds, generating triangular mesh data, selecting an optimal view angle, pasting texture on the triangular mesh according to the selected view angle, and obtaining three-dimensional face information. The application has the effect of more rapidly and accurately collecting three-dimensional face information.

Description

Technical Field

[0001] This invention relates to the field of three-dimensional data acquisition, and in particular to a low-cost and high-efficiency three-dimensional facial information acquisition system and method. Background Technology

[0002] With the continuous upgrading of various advanced 3D scanning instruments and the development of auxiliary software, the application of 3D scanning technology is becoming increasingly widespread and accepted by clinicians and patients. 3D scanning technology is expected to replace 2D photography as a routine assessment method in plastic and cosmetic surgery, providing patients with personalized design plans and offering more accurate guidance for organ reconstruction. Currently, there are two common methods for facial scanning technology. The first method involves using a 3D module to collect information from three angles of the user's face, and then calculating the user's 3D facial information based on the captured point cloud information. The second method involves installing three 3D modules directly in three locations to capture images of the user's face from three angles, and then calculating the user's 3D facial information based on the captured point cloud information. However, to obtain the texture and color of the user's face, a 2D camera is also needed to capture images of the user's face. If only a 2D camera is used to capture images of the user's face, and then an algorithm is used to calculate the 3D facial information, it is necessary to capture images of the face from multiple angles, and the final 3D model construction depends on the algorithm, resulting in low accuracy.

[0003] The existing technical solutions mentioned above have the following drawbacks: when using a single 3D module for shooting, the 3D module needs to be moved once for each shot, which takes a long time. Although using three 3D modules does not require movement, the shooting of the 3D modules needs to be done at intervals to avoid mutual interference, which still takes a long time. Summary of the Invention

[0004] To achieve faster and more accurate 3D facial information acquisition, this application provides a low-cost and high-efficiency 3D facial information acquisition system and method.

[0005] On the one hand, the low-cost and high-efficiency three-dimensional facial information acquisition method provided in this application adopts the following technical solution:

[0006] A low-cost and high-efficiency method for acquiring three-dimensional facial information also includes the following steps:

[0007] Determine the shooting location and set up three RGB cameras and two 3D modules;

[0008] Perform joint calibration and device color correction on the RGB camera and 3D module, and obtain intrinsic and extrinsic parameters;

[0009] 2D images are acquired using an RGB camera, and 3D point clouds are acquired using a 3D module.

[0010] Preprocess 2D images and 3D point clouds to obtain triangular meshes;

[0011] Project each triangular mesh onto each 2D image, and calculate the optimal viewing angle based on the projection of the triangular mesh onto the 2D image:

[0012] A mask is drawn on a 2D image captured by a medium RGB camera, which is a frontal view.

[0013] Set the overall objective function for:

[0014] ;

[0015] In the formula, The data items selected for the perspective. The penalty coefficient is the penalty term. Penalty terms for selecting different viewpoints between adjacent faces; calculation of data items for viewpoint selection. The formula is as follows:

[0016] ;

[0017] In the formula, P is the condition function of the projection surface. It is a triangular grid. for The corresponding view label for the triangular mesh, where m is the number of triangular meshes and n is the number of viewpoints, is calculated. The formula is as follows:

[0018] ;

[0019] In the formula, For another triangular mesh, for The view tag corresponding to the triangular grid. express and Adjacent, S represents the penalty function, Let the cost function be the cost function. This can be expressed as the following formula:

[0020] ;

[0021] In the formula, From a middle perspective, For the mask area, X is a constant value, and X is greater than the values ​​corresponding to the left and right view projection areas. If the projection area of ​​the triangular mesh completely falls into the mask area, then the value corresponding to the middle view projection area is replaced with X, and the middle view is selected according to the formula.

[0022] Textures are applied to the triangular mesh based on the selected viewpoint to obtain 3D facial information.

[0023] By adopting the above scheme, three-dimensional facial information can be quickly and accurately acquired using two sets of 3D modules and three RGB cameras. Since only two sets of 3D modules are used, one shooting interval is reduced compared to using three 3D modules, resulting in higher efficiency. Furthermore, the cost is lower due to the reduction of one 3D module. It is also more efficient than using only one 3D module, and the absence of moving the 3D modules saves on structural movement. Therefore, this application effectively saves the time and cost of acquiring three-dimensional facial information using traditional 3D modules, and achieves higher accuracy than methods that only use 2D cameras to capture faces and calculate three-dimensional facial information using algorithms. Additionally, when texturing the 3D model, this application selects a viewing angle for each triangular mesh, ensuring that the texture of each selected triangular mesh represents the optimal viewing angle. This results in a complete 3D facial model that is less prone to texture misalignment.

[0024] Preferably, the step "setting up three RGB cameras and two 3D modules" further includes:

[0025] Set up a left RGB camera, a right RGB camera, a center RGB camera, a left 3D module, and a right 3D module;

[0026] The step "preprocessing 2D images and 3D point clouds" also includes:

[0027] A semantic segmentation model is used to segment faces in each 2D image, and background point clouds of 3D point clouds are filtered out.

[0028] Facial landmark recognition is performed on each 2D image;

[0029] The two processed 3D point clouds are merged to generate triangular mesh data.

[0030] By adopting the above scheme, image preprocessing includes facial key point recognition of 2D images and background point cloud filtering of 3D point clouds, which can effectively extract facial features in the image and ensure that the generated triangular mesh is on the face.

[0031] Preferably, the step "using a semantic segmentation model to segment faces in each 2D image and filter out background point clouds" further includes:

[0032] A semantic segmentation model is used to segment faces in each 2D image and separate the background portion;

[0033] The 3D point cloud is colored according to the segmentation results. The background part is colored as (0, 0, 0) in RGB. All point clouds with RGB coloring of (0, 0, 0) are deleted.

[0034] By adopting the above approach, since 2D shooting inevitably leaves background parts, the presence of background will introduce a lot of noise, affecting subsequent color transfer and point cloud reconstruction. However, by combining semantic segmentation models, the background parts can be quickly and easily filtered out, and then the point cloud can be deleted, thereby reducing the impact of the background.

[0035] Preferably, the step "acquiring 2D images using an RGB camera and acquiring 3D point clouds using a 3D module" further includes:

[0036] Set the interval time;

[0037] Take intermittent shots when capturing 3D point clouds;

[0038] The step of "merging the two processed 3D point clouds and generating triangular mesh data" further includes:

[0039] Preset the number of attempts and distance thresholds according to the accuracy requirements;

[0040] Step 1: For each point in the target point cloud, find and determine its nearest point in the source point cloud to form multiple pairs of "target point - source point" nearest points;

[0041] Step 2: Based on multiple sets of closest point pairs between target point and source point, construct an optimization objective function. The objective is to minimize the distance between each matching point in the target point cloud and the corresponding point in the source point cloud after transformation. By solving the optimization objective function, obtain the rigid body transformation matrix used for adjusting the position of the target point cloud.

[0042] Step 3: Perform a rigid body transformation on the target point cloud using a rigid body transformation matrix to complete the position update;

[0043] In the updated source and target point clouds, repeat step one and update the set of nearest point pairs.

[0044] Repeat steps two and three until the number of iterations reaches the preset threshold, or the average distance of all nearest point pairs in the updated nearest point pair set is less than the preset distance threshold, then the iteration is complete.

[0045] Point cloud registration is performed on the two 3D point clouds based on the source point cloud and the target point cloud after the iteration is completed.

[0046] By adopting the above scheme, since 3D point clouds need to be captured at intervals to avoid mutual interference of light during shooting, but the face may move slightly in the images at different times, which will produce errors, point cloud registration is required to match the 3D point clouds captured by the two 3D point cloud images, effectively increasing the accuracy of 3D facial information acquisition.

[0047] Preferably, the step "projecting each triangular mesh onto each 2D image and calculating the optimal viewing angle based on the projection of the triangular mesh onto the 2D image" further includes:

[0048] Obtain facial landmark information based on the identified facial landmarks;

[0049] Project the triangular mesh onto each 2D image. If the triangular mesh has a projection that falls inside the mask, then the frontal view is taken as the best view for that triangular mesh.

[0050] For the remaining triangular meshes that do not select the frontal view as the optimal viewpoint, the viewpoint corresponding to the 2D image with the largest projected area is selected as the optimal viewpoint.

[0051] By adopting the above scheme, problems can easily occur during stitching due to the different perspectives of the three 2D images. This application selects the best perspective through specific rules to ensure that the textures after the best perspective corresponding to each triangular mesh can be completely stitched together, reducing the possibility of problems.

[0052] Preferably, after the step of "texturing the triangular mesh according to the selected viewpoint", the method further includes:

[0053] Obtain a texture image, perform cylindrical texture unfolding on the texture image to obtain a cylindrical unfolded 3D model, and establish a coordinate system on the cylindrical unfolded 3D model;

[0054] Set a projection plane, project the unfolded 3D model of the cylinder onto the projection plane, and calculate the projected coordinates of each point of the unfolded 3D model of the cylinder. The calculation formula is as follows:

[0055] ;

[0056] ;

[0057] ;

[0058] In the formula, Let x be the x-axis projection coordinates on the projection plane. Let be the y-axis projection coordinate on the projection plane, and r be the cylinder radius of the unfolded 3D model. θ is the simulated radius under the sigmoid function, x is the x-axis coordinate of the cylindrical unfolded 3D model in the coordinate system, y is the y-axis coordinate of the cylindrical unfolded 3D model in the coordinate system, z is the z-axis coordinate of the cylindrical unfolded 3D model in the coordinate system, and θ is the radian. Based on the calculated projection coordinates, the texture is reselected in the texture image to generate the triangular mesh.

[0059] By adopting the above scheme, a new calculation method is used. By increasing the unfolding radius at the nostrils, sufficient unfolding space is ensured after texture unfolding, while the unfolding radius of other facial points does not increase significantly, ensuring the stability of the overall rendering result. In the 3D facial information acquisition process, cylindrical texture unfolding is typically performed, followed by projection, recalculation, and then texture application. However, due to the unique shape of the face at the nostrils, the angle between the face and the projection plane is nearly perpendicular during projection, resulting in too few pixels after projection. Consequently, the unfolded texture cannot obtain enough pixels, causing texture stretching. Therefore, it is necessary to increase the projection area at the nostrils.

[0060] On the other hand, the low-cost and high-efficiency three-dimensional facial information acquisition system provided in this application adopts the following technical solution:

[0061] A low-cost and high-efficiency three-dimensional facial information acquisition system is used to perform the three-dimensional facial information acquisition method described above, and also includes a shooting position, a left RGB camera, a right RGB camera, a center RGB camera, a left 3D module, a right 3D module, and a computing system;

[0062] The computing system performs joint calibration and device color correction on the RGB camera to obtain intrinsic and extrinsic parameters. It then acquires multiple 2D images and multiple 3D point clouds. The system preprocesses the 2D images and multiple 3D point clouds to obtain triangular mesh data. Each triangular mesh is projected onto each 2D image. The optimal viewing angle is calculated based on the projection of the triangular mesh onto the 2D image. Textures are then applied to the triangular mesh according to the selected viewing angle to obtain 3D facial information.

[0063] By adopting the above scheme, three-dimensional facial information can be quickly and accurately acquired using two sets of 3D modules and three RGB cameras. Since only two sets of 3D modules are used, one shooting interval is reduced compared to using three 3D modules, resulting in higher efficiency. Furthermore, the cost is lower due to the reduction of one 3D module. It is also more efficient than using only one 3D module, and the elimination of the need to move the 3D modules saves on structural movement. Simultaneously, when texturing the 3D model, this application performs viewpoint selection for each triangular mesh, ensuring that the texture of each selected triangular mesh represents the optimal viewpoint. This results in a complete 3D facial model that is less prone to texture misalignment and other issues.

[0064] Preferably, the computing system includes an information storage module, a parameter calculation module, a device color calibration module, an image acquisition module, an image processing module, and an image fusion module;

[0065] The information storage module is pre-set with basic information such as shooting position, left RGB camera, right RGB camera, center RGB camera, left 3D module and right 3D module. The basic information includes camera data and position information.

[0066] The parameter calculation module calls the data stored in the information storage module to perform joint calibration of the left RGB camera, right RGB camera, middle RGB camera, left 3D module and right 3D module, and calculates the intrinsic and extrinsic parameters of the left RGB camera, right RGB camera, middle RGB camera, left 3D module and right 3D module, and transmits the intrinsic and extrinsic parameters to the data storage module for storage;

[0067] The device color calibration module calls the camera data from the information storage module to perform device color calibration on the left RGB camera, right RGB camera, and center RGB camera;

[0068] The image acquisition module controls the left RGB camera, right RGB camera, center RGB camera, left 3D module and right 3D module to acquire images, obtain multiple 2D images and multiple 3D point clouds, and transmit the 2D images and 3D point clouds to the image processing module.

[0069] The image processing module uses a semantic segmentation model to segment faces in each 2D image, filters out background point clouds, performs facial key point recognition on each 2D image, and transmits the processed 2D image and 3D point cloud to the image fusion module.

[0070] The image fusion module merges the two processed 3D point clouds and generates triangular mesh data. For each triangular mesh, the best viewpoint is selected in each 2D image, and textures are applied to the triangular mesh according to the selected viewpoint to obtain three-dimensional facial information.

[0071] Preferably, the image acquisition module has a preset interval time, and the 3D module is controlled to take pictures at intervals according to the interval time when capturing 3D point clouds;

[0072] The image fusion module presets a threshold for the number of iterations and a distance threshold. The module performs point cloud registration on two 3D point clouds. Step 1: For each point in the target point cloud, find and determine its corresponding nearest point in the source point cloud, forming multiple sets of "target point-source point" nearest point pairs. Step 2: Based on these multiple sets of "target point-source point" nearest point pairs, construct an optimization objective function. The objective is to minimize the distance between each matched point in the target point cloud and its corresponding point in the source point cloud after transformation. By solving the optimization objective function, a rigid transformation matrix for adjusting the target point cloud position is obtained. Step 3: Use the rigid transformation matrix to perform a rigid transformation on the target point cloud as a whole to update its position. Repeat Step 1 in both the source and target point clouds after the position update, and update the nearest point pair set. Repeat Steps 2 and 3 until the number of iterations reaches the preset threshold, or the average distance between all nearest point pairs in the updated nearest point pair set is less than the preset distance threshold. The iteration is then completed. Finally, point cloud registration is performed on the two 3D point clouds based on the source and target point clouds after the iteration.

[0073] By adopting the above solution, since the 3D modules need to shoot at intervals to avoid mutual interference of light during shooting, but the face may move slightly in the images at different times, which will cause errors, point cloud registration is required to match the 3D point clouds captured by the two 3D modules, effectively increasing the accuracy of 3D facial information acquisition.

[0074] Preferably, both the left and right 3D modules include two infrared cameras and an infrared grating projection device, and the computing system also includes a model generation module and a light detection module;

[0075] The model generation module receives grating images projected onto a person's face by an infrared grating projection device captured by a left RGB camera, a right RGB camera, and a center RGB camera. It treats the grating images captured at the same time as a group of grating images and adds left, right, or center labels to the grating images in the same group according to the RGB camera used for capturing them. It also adds left or right labels to the grating images according to the 3D module corresponding to the infrared grating projection device in the grating image group. The stored grating images are used to train the convolutional neural network model, and the trained convolutional neural network model is transmitted to the light detection module.

[0076] The light-collecting detection module is connected to the image acquisition module. The light-collecting detection module receives and stores the input brightness standard value. When the image acquisition module controls the left 3D module to perform image acquisition, the light-collecting detection module adds a left label to the currently acquired 2D image and inputs it into the convolutional neural network model. When the image acquisition module controls the right 3D module to perform image acquisition, the light-collecting detection module (18) adds a right label to the currently acquired 2D image and inputs it into the convolutional neural network model. The convolutional neural network model segments the face image in the 2D image, calculates the projection area of ​​the grating in the face image under normal conditions, and judges whether the grating projection in the received 2D image falls completely within the calculated grating projection area based on the calculation result. If it does not fall completely within the grating projection area, the 2D image with the calculated grating projection area is output.

[0077] By adopting the above solution, when acquiring facial images, problems such as lighting and equipment failure may lead to unqualified images. Since the projection shape of the grating varies on different faces, it is impossible to determine whether it is qualified by simple system detection. This application uses an AI large model to simulate and calculate the grating projection area of ​​the face, compares it with the actual acquired image to determine whether there is a problem with the grating, and provides a deviation image to facilitate users to adjust or repair.

[0078] In summary, the present invention has the following beneficial effects:

[0079] 1. Three-dimensional facial information can be quickly and accurately acquired using two sets of 3D modules and three RGB cameras. Since only two sets of 3D modules are used, the shooting interval is reduced by one compared to using three 3D modules, thus increasing efficiency. This is significantly more efficient than using only one 3D module.

[0080] 2. Compared to using three 3D modules, this application has a lower cost because it reduces one 3D module. Attached Figure Description

[0081] Figure 1 This is a schematic diagram of the unfolded three-dimensional model of the prominent cylinder in Embodiment 1 of this application.

[0082] Figure 2 This is an overall schematic diagram of Embodiment 2 of this application.

[0083] Figure 3 This is a schematic diagram of the two prominent angles of implementation of this application.

[0084] Figure 4 This is a system block diagram of the computing system in Embodiment 2 of this application.

[0085] Figure 5 This is a system block diagram of Embodiment 3 of this application.

[0086] Explanation of reference numerals in the attached figures:

[0087] 1. Computing System; 11. Information Storage Module; 12. Parameter Calculation Module; 13. Equipment Color Calibration Module; 14. Image Acquisition Module; 15. Image Processing Module; 16. Image Fusion Module; 17. Model Generation Module; 18. Light Detection Module; 2. Left RGB Camera; 3. Right RGB Camera; 4. Middle RGB Camera; 5. Left 3D Module; 51. Infrared Camera; 52. Infrared Grating Projection Device; 6. Right 3D Module; 7. Left Reflector; 8. Right Reflector; 9. Shooting Position; 10. Upper Fixing Plate; 101. Left Vertical Plate; 102. Middle Vertical Plate; 103. Right Vertical Plate; 104. Lower Fixing Plate; 105. Front Frame Plate; 106. Left Fixing Plate; 107. Right Fixing Plate. Detailed Implementation

[0088] The following is in conjunction with the appendix Figure 1-3 This application will be described in further detail.

[0089] Example 1: This application discloses a low-cost and high-efficiency method for acquiring three-dimensional facial information. The specific steps are as follows:

[0090] Determine shooting position 9, and set up left RGB camera 2, right RGB camera 3 and center RGB camera 4, respectively, to the left, right and directly in front of shooting position 9.

[0091] A left 3D module 5 is positioned at the location of the left RGB camera 2, and a right 3D module 6 is positioned at the location of the right RGB camera 3. Both the left 3D module 5 and the right 3D module 6 include two infrared cameras 51 and an infrared grating projection device 52. The infrared grating projection device 52 projects an infrared grating with coded information onto the shooting position 9, and the infrared cameras 51 obtain the matching point depth information based on the coded information on the infrared grating in the captured image information.

[0092] A left reflector 7 and a right reflector 8 are respectively set on either side of the shooting position 9. The left RGB camera 2 and the left 3D module 5 are positioned facing the left reflector 7, capturing images of shooting position 9 through the left reflector 7. The right RGB camera 3 and the right 3D module 6 are positioned facing the right reflector 8, capturing images of shooting position 9 through the right reflector 8. The left RGB camera 2 and the left 3D module 5 can capture most of the left half of the face through the left reflector 7, while the right RGB camera 3 and the right 3D module 6 can capture most of the right half of the face through the right reflector 8. The use of two reflectors allows for a smaller distance between the cameras, reducing the device size, while also increasing the camera's shooting range, thus increasing the user's shooting space.

[0093] Adjust the left RGB camera 2, right RGB camera 3, center RGB camera 4, left 3D module 5, and right 3D module 6 to capture images at the shooting position 9.

[0094] Set the interval time.

[0095] Joint calibration is performed on the left RGB camera 2, right RGB camera 3, center RGB camera 4, left 3D module 5, and right 3D module 6, and the intrinsic and extrinsic parameters of these cameras are obtained. A camera coordinate system is established, and one of the camera coordinate systems is selected as the world coordinate system.

[0096] The intrinsic and extrinsic parameters of the left RGB camera 2, right RGB camera 3, center RGB camera 4, left 3D module 5, and right 3D module 6 are calculated based on the intrinsic and extrinsic parameter algorithm. The intrinsic and extrinsic parameter algorithm is as follows:

[0097] The intrinsic parameter I is calculated using the following formula:

[0098] ;

[0099] In the formula, Focal length The focal length is the X-axis. The focal length is the Y-axis focal length. The X-axis optical center coordinates are... Using the Y-axis optical center coordinates, the physical pixel size of the image is calculated using the following formula:

[0100] ;

[0101] ;

[0102] In the formula, The pixel size is on the X-axis. Given the pixel size on the Y-axis, the pixel coordinate system is determined based on the pixel size. The camera imaging formula is then generated using the camera intrinsic parameter I, as follows:

[0103] ;

[0104] In the formula, u is the X-axis coordinate of the pixel coordinate system, and v is the Y-axis coordinate of the pixel coordinate system;

[0105] The extrinsic parameter E is calculated using the relevant parameters of the image, as shown in the following formula:

[0106] ;

[0107] In the formula, The X-axis coordinates of the camera coordinate system. The Y-axis coordinate of the camera coordinate system The Z-axis coordinate of the camera coordinate system The x-axis coordinates in the world coordinate system. The Y-axis coordinate in the world coordinate system. The coordinates are the Z-axis coordinates in the world coordinate system.

[0108] Perform device color calibration on the left RGB camera 2, right RGB camera 3 and center RGB camera 4.

[0109] Image acquisition was performed using left RGB camera 2, right RGB camera 3, and center RGB camera 4 to obtain multiple 2D images. Image acquisition was performed using left 3D module 5 and right 3D module 6 to obtain multiple 3D point cloud images. 3D point cloud images were captured at intervals.

[0110] A semantic segmentation model is used to segment faces in each 2D image and separate the background.

[0111] The 3D point cloud is colored according to the segmentation results. The background part is colored as (0, 0, 0) in RGB. All point clouds with RGB coloring of (0, 0, 0) are deleted.

[0112] Facial landmark recognition is performed on each 2D image. Because 2D photography inevitably retains background parts, the presence of the background introduces a lot of noise, affecting subsequent color transfer and point cloud reconstruction. However, by combining a semantic segmentation model, the background part can be quickly and easily filtered out, and then the point cloud can be deleted, thereby reducing the influence of the background.

[0113] The two processed 3D point clouds are merged to generate triangular mesh data.

[0114] To perform point cloud registration on two 3D point clouds, the following steps are taken, based on preset thresholds for the number of attempts and distance thresholds according to accuracy requirements:

[0115] Step 1: For each point in the target point cloud, find and determine its nearest point in the source point cloud to form multiple pairs of "target point - source point" nearest points.

[0116] Step 2: Based on multiple sets of closest point pairs between the target point and the source point, construct an optimization objective function. The objective is to minimize the distance between each matching point in the target point cloud and the corresponding point in the source point cloud after transformation. By solving the optimization objective function, obtain the rigid body transformation matrix used for adjusting the position of the target point cloud.

[0117] Step 3: Use the rigid body transformation matrix to perform a rigid body transformation on the target point cloud to complete the position update.

[0118] In the updated source and target point clouds, repeat step one and update the set of nearest point pairs.

[0119] Repeat steps two and three until the number of iterations reaches the preset threshold, or the average distance of all nearest point pairs in the updated nearest point pair set is less than the preset distance threshold, then the iteration is complete.

[0120] Point cloud registration is performed on the source and target point clouds after iteration. Because the 3D modules need to take photos at intervals to avoid interference from lighting during shooting, but the face may move slightly in images taken at different times, errors can occur. Therefore, point cloud registration is necessary to match the 3D point clouds captured by the two 3D modules, effectively increasing the accuracy of 3D facial information acquisition.

[0121] For each triangular mesh, select the optimal viewing angle in each 2D image. Obtain facial landmark information based on the identified facial landmarks. Draw a mask on the 2D images captured by the RGB camera 4, which represent a frontal view.

[0122] Project the triangular mesh onto each 2D image. If the triangular mesh has a projection that falls inside the mask, then the frontal view is taken as the best view for that triangular mesh.

[0123] For the remaining triangular meshes that do not select the frontal view as the optimal viewpoint, the viewpoint corresponding to the 2D image with the largest projected area is selected as the optimal viewpoint.

[0124] The specific formula calculation method is as follows: Here, the overall objective function... for:

[0125] ;

[0126] In the formula, The data items selected for the perspective. The penalty coefficient is the penalty term. Penalty terms for selecting different viewpoints between adjacent faces; calculation of data items for viewpoint selection. The formula is as follows:

[0127] ;

[0128] In the formula, P is the condition function of the projection surface. It is a triangular grid. for The corresponding view label for the triangular grid, where m is the number of triangular grids and n is the number of viewpoints. Data items The selection of the viewpoint with the largest projected area for each triangular mesh is influenced by the fact that the right side of the formula starts with a negative sign. The optimal viewpoint is the one whose projected area is maximized across all meshes. Minimize this, thus allowing for optimization to obtain the best viewing angle for each scan grid. Calculation The formula is as follows:

[0129] ;

[0130] In the formula, For another triangular mesh, for The view tag corresponding to the triangular grid. express and Adjacent, S represents the penalty function. Let the cost function be the cost function. This can be expressed as the following formula:

[0131] ;

[0132] In the formula, From a middle perspective, The mask region is defined by X, which is a constant value greater than the corresponding values ​​of the projected area from the left and right views. If the projected area of ​​the triangular mesh completely falls within the mask region, the value corresponding to the projected area from the intermediate view is replaced with X. The intermediate view is then selected based on the formula. According to this formula, if the projected area of ​​the triangular mesh completely falls within the mask region, the cost is X. In practical use, a larger projected area is used to replace X, such as a common projected area of ​​100. If the mesh's projection in the frontal view completely falls within the mask, its actual projected area is directly assigned the value 10000, regardless of its actual size. This is much larger than the projected area of ​​this facet in the other two views, thus forcing the viewt of this facet to be designated as the intermediate viewt during optimization.

[0133] Since the three 2D images have different perspectives, problems can easily occur during stitching. This application selects the best perspective through specific rules to ensure that the textures after the best perspective for each triangular mesh can be completely stitched together, reducing the possibility of problems.

[0134] Textures are applied to the triangular mesh based on the selected viewpoint to obtain a texture image.

[0135] like Figure 1 As shown, the texture image is expanded into a cylindrical shape to obtain a cylindrical expanded 3D model, and a coordinate system is established on the cylindrical expanded 3D model;

[0136] Set a projection plane, project the unfolded 3D model of the cylinder onto the projection plane, and calculate the projected coordinates of each point of the unfolded 3D model of the cylinder. The calculation formula is as follows:

[0137] ;

[0138] ;

[0139] ;

[0140] In the formula, Let x be the x-axis projection coordinates on the projection plane. Let be the y-axis projection coordinate on the projection plane, and r be the cylinder radius of the unfolded 3D model. θ is the simulated radius under the sigmoid function, x is the x-axis coordinate of the cylindrical unfolded 3D model in the coordinate system, y is the y-axis coordinate of the cylindrical unfolded 3D model in the coordinate system, z is the z-axis coordinate of the cylindrical unfolded 3D model in the coordinate system, and θ is the radian. Based on the calculated projection coordinates, the texture is reselected in the texture image to generate the triangular mesh and obtain the 3D facial information.

[0141] In the texture mapping step of 3D facial information acquisition, due to the different shooting angles of the three 2D images, some facial features appear in different positions on the three 2D images. If the three 2D images are directly projected and textured, features such as blemishes on the face will appear repeatedly in different positions. Therefore, texture unwrapping is necessary to unify the projection positions of the three 2D images. Texture unwrapping usually uses cylindrical texture unwrapping, treating the face as a cylinder, unwrapping the 2D image on the cylinder, and then projecting it onto a plane to avoid difficulties in subsequent image analysis. However, due to the special shape of the face at the nostrils, the angle between the projection and the projection plane is almost perpendicular, resulting in too few pixels after projection. This leads to insufficient pixels after texture unwrapping, causing texture stretching. Therefore, the projection area at the nostrils needs to be increased. If the unwrapping radius of the entire face is linearly increased, the sides of the face will occupy too many pixels after texture unwrapping, thus squeezing the middle of the face. This application employs a novel calculation method that increases the unfolding radius at the nostrils to ensure sufficient unfolding space for the texture, while the unfolding radius of other facial points does not increase significantly, thus ensuring stable overall rendering results. Using the formula in this application, the cylinder radius increases rapidly with distance within a 5cm range, providing sufficient unfolding radius for the 2D image near the nostrils. Beyond 5cm, the radius remains almost constant, ensuring uniform pixel distribution after texture unfolding in the remaining areas of the face.

[0142] The implementation principle of the low-cost, high-efficiency three-dimensional facial information acquisition method in this application embodiment is as follows: three-dimensional facial information can be acquired quickly and accurately using two sets of 3D modules and three RGB cameras. Since only two sets of 3D modules are used, one shooting interval is reduced compared to using three 3D modules, thus increasing efficiency. Furthermore, the cost is lower due to the reduction of one 3D module. It is also more efficient than using only one 3D module, and saves on moving structures because there is no need to move the 3D module.

[0143] This application discloses a low-cost, high-efficiency three-dimensional facial information acquisition system, using the three-dimensional facial information acquisition method as described in Embodiment 1. Figure 2 and Figure 3 As shown, the device includes a lower fixed plate 104, an upper fixed plate 10, a front frame plate 105 bolted to the upper fixed plate 10 and the lower fixed plate 104, and a computing system 1. A left vertical plate 101, a middle vertical plate 102, and a right vertical plate 103 are bolted between the upper fixed plate 10 and the lower fixed plate 104. A shooting position 9 is set within the device. A left RGB camera 2 and a left 3D module 5 are fixedly connected to the left vertical plate 101; a middle RGB camera 4 is fixedly connected to the middle vertical plate 102; and a right RGB camera 3 and a right 3D module 6 are fixedly connected to the right vertical plate 103. The left RGB camera 2, right RGB camera 3, and middle RGB camera 4 are respectively positioned to the left, right, and front of the shooting position 9. The left 3D module 5 is located at the position of the left RGB camera 2, and the right 3D module 6 is located at the position of the right RGB camera 3. Both the left 3D module 5 and the right 3D module 6 include two infrared cameras 51 and an infrared grating projection device 52. The infrared grating projection device 52 projects an infrared grating with coded information onto the shooting position 9, and the infrared camera 51 obtains the matching point depth information based on the coded information on the infrared grating in the captured image information.

[0144] like Figure 2 and Figure 3As shown, the angle between the line connecting the center of the target surface of the left RGB camera and the shooting position and the central axis of the mirror surface of the middle RGB camera is α; the angle between the center of the target surface of the right RGB camera and the central axis of the mirror surface of the middle RGB camera is β; the angle between the center of the target surface of the camera in the left 3D module and the central axis of the mirror surface of the middle RGB camera is also α; the angle between the center of the target surface of the camera in the right 3D module and the central axis of the mirror surface of the middle RGB camera is also β. A left fixed plate 106 and a right fixed plate 107 are bolted to the upper fixed plate 10 and the lower fixed plate 104 at positions corresponding to the shooting position 9 on both sides, respectively. A left reflector 7 is fixedly connected inside the left fixed plate 106, and a right reflector 8 is fixedly connected inside the right fixed plate 107. A left pressure plate is fixedly installed on the left fixed plate 106 at the position corresponding to the left reflector 7, and the left fixed plate 106 presses the left reflector 7 tightly onto the left pressure plate by bolts. A right pressure plate is fixed on the right fixing plate 107 at the position corresponding to the right reflector 8. The right fixing plate 107 presses the right reflector 8 onto the right pressure plate by bolts. The left RGB camera 2 and the left 3D module 5 are positioned facing the left reflector 7. The point from the center line of the left RGB camera's mirror surface to the surface of the left reflector is taken as the vertex. The angle between the center line of the left RGB camera's mirror surface and the perpendicular line to the surface of the left reflector is γ. The left RGB camera 2 and the left 3D module 5 acquire images from shooting position 9 through the left reflector 7. The left RGB camera 2 and the left 3D module 5 can capture the left half of the face through the left reflector 7. The right RGB camera 3 and the right 3D module 6 are positioned facing the right reflector 8. The point from the center line of the right 3D module camera's mirror surface to the surface of the right reflector is taken as the center. The angle between the center line of the right 3D module camera's mirror surface and the perpendicular line to the surface of the right reflector is δ. The right RGB camera 3 and the right 3D module 6 acquire images from shooting position 9 through the right reflector 8. The right RGB camera 3 and the right 3D module 6 can capture the right half of the face through the right reflector 8.

[0145] like Figure 4 As shown, the computing system 1 includes an information storage module 11, a parameter calculation module 12, a device color calibration module 13, an image acquisition module 14, an image processing module 15, and an image fusion module 16.

[0146] like Figure 2 and Figure 4 As shown, the information storage module 11 has preset basic information on the shooting position 9, the left RGB camera 2, the right RGB camera 3, the center RGB camera 4, the left 3D module 5, and the right 3D module 6. The basic information includes camera data and position information.

[0147] like Figure 2 and Figure 4As shown, the parameter calculation module 12 calls the data stored in the information storage module 11 to perform joint calibration on the left RGB camera 2, right RGB camera 3, center RGB camera 4, left 3D module 5 and right 3D module 6, and calculates the intrinsic and extrinsic parameters of the left RGB camera 2, right RGB camera 3, center RGB camera 4, left 3D module 5 and right 3D module 6, and transmits the intrinsic and extrinsic parameters to the data storage module for storage.

[0148] like Figure 2 and Figure 4 As shown, the device color calibration module 13 calls the camera data from the information storage module 11 to perform device color calibration on the left RGB camera 2, the right RGB camera 3, and the center RGB camera 4.

[0149] like Figure 2 and Figure 4 As shown, the image acquisition module 14 has a preset interval time. The image acquisition module 14 controls the left RGB camera 2, right RGB camera 3, middle RGB camera 4, left 3D module 5 and right 3D module 6 to acquire images. When controlling the 3D point cloud to take pictures, the 3D point cloud is captured at intervals according to the interval time, so as to obtain multiple 2D images and multiple 3D point clouds. The 2D images and 3D point clouds are then transmitted to the image processing module 15.

[0150] like Figure 2 and Figure 4 As shown, the image processing module 15 uses a semantic segmentation model to filter out the background of each 2D image and performs facial landmark recognition on each 2D image. The image processing module 15 uses the semantic segmentation model to segment the face in each 2D image, separating the background portion. Based on the segmentation results, it performs point cloud coloring on the 3D point cloud, where the background portion is RGB-colored as (0, 0, 0), and all point clouds with RGB-colored (0, 0, 0) are deleted. The processed 2D image and 3D point cloud are then transmitted to the image fusion module 16.

[0151] like Figure 2 and Figure 4As shown, the image fusion module 16 presets a number of iterations threshold and a distance threshold. The image fusion module 16 performs point cloud registration on two 3D point clouds. Step 1: For each point in the target point cloud, find and determine its corresponding nearest point in the source point cloud to form multiple sets of "target point-source point" nearest point pairs. Step 2: Based on the multiple sets of "target point-source point" nearest point pairs, construct an optimization objective function. The objective is to minimize the distance between each matching point in the target point cloud and its corresponding point in the source point cloud after transformation. By solving the optimization objective function, obtain the rigid body transformation matrix used for adjusting the position of the target point cloud. Step 3: Use the rigid body transformation matrix to perform a rigid body transformation on the target point cloud as a whole to complete the position update. Repeat Step 1 in the source point cloud and target point cloud after the position update, and update the set of nearest point pairs. Repeat Step 2 and Step 3 until the number of iterations reaches the preset number of iterations threshold, or the average distance of all nearest point pairs in the updated set of nearest point pairs is less than the preset distance threshold, and the iteration is completed. Perform point cloud registration on the two 3D point clouds based on the source point cloud and target point cloud after the iteration.

[0152] like Figure 2 and Figure 4 As shown, the image fusion module 16 selects the optimal viewing angle for each triangular mesh in each 2D image, obtains facial key point information based on the recognized facial key points, and draws a mask on the 2D image captured by the RGB camera 4 (which is a frontal view). The triangular mesh is projected onto each 2D image. If a triangular mesh has a projection that falls inside the mask, the frontal view is taken as the optimal viewing angle for that triangular mesh. For the remaining triangular meshes that do not select the frontal view as the optimal viewing angle, the viewpoint corresponding to the 2D image with the largest projection area is selected as the optimal viewing angle. Textures are applied to the triangular meshes according to the selected viewing angles to obtain 3D facial information. Since the three 2D images have different viewing angles, problems can easily occur during stitching. This application selects the optimal viewing angle through specific rules to ensure that the textures after the optimal viewing angle for each triangular mesh can be completely stitched together, reducing the possibility of problems.

[0153] The specific formula calculation method is as follows: Here, the overall objective function... for:

[0154] ;

[0155] In the formula, The data items selected for the perspective. The penalty coefficient is the penalty term. Penalty terms for selecting different viewpoints between adjacent faces; calculation of data items for viewpoint selection. The formula is as follows:

[0156] ;

[0157] In the formula, P is the condition function of the projection surface. It is a triangular grid. for The corresponding view label for the triangular grid, where m is the number of triangular grids and n is the number of viewpoints. Data items The selection of the viewpoint with the largest projected area for each triangular mesh is influenced by the fact that the right side of the formula starts with a negative sign. The optimal viewpoint is the one whose projected area is maximized across all meshes. Minimize this, thus allowing for optimization to obtain the best viewing angle for each scan grid. Calculation The formula is as follows:

[0158] ;

[0159] In the formula, For another triangular mesh, for The view tag corresponding to the triangular grid. express and Adjacent, S represents the penalty function. Let the cost function be the cost function. This can be expressed as the following formula:

[0160] ;

[0161] In the formula, From a middle perspective, The mask region is defined by X, which is a constant value greater than the corresponding values ​​of the projected area from the left and right views. If the projected area of ​​the triangular mesh completely falls within the mask region, the value corresponding to the projected area from the intermediate view is replaced with X. The intermediate view is then selected based on the formula. According to this formula, if the projected area of ​​the triangular mesh completely falls within the mask region, the cost is X. In practical use, a larger projected area is used to replace X, such as a common projected area of ​​100. If the mesh's projection in the frontal view completely falls within the mask, its actual projected area is directly assigned the value 10000, regardless of its actual size. This is much larger than the projected area of ​​this facet in the other two views, thus forcing the viewt of this facet to be designated as the intermediate viewt during optimization.

[0162] Example 3: This application discloses a low-cost, high-efficiency three-dimensional facial information acquisition system, such as... Figure 5 As shown, the difference from Embodiment 2 is that the computing system 1 also includes a model generation module 17 and a light detection module 18.

[0163] The model generation module 17 receives grating images projected onto a person's face by the infrared grating projection device 52, captured by the left RGB camera 2, the right RGB camera 3, and the center RGB camera 4. It takes the grating images captured at the same time as a group of grating images and adds left, right, or center labels to the grating images in the same group according to the RGB camera that captured them. It adds left or right labels to the grating images according to the 3D module corresponding to the infrared grating projection device 52 in the grating images group. It uses the stored grating images to train the convolutional neural network model and transmits the trained convolutional neural network model to the light detection module 18.

[0164] The light detection module 18 is connected to the image acquisition module 14. The light detection module 18 receives and stores the input brightness standard value. When the image acquisition module 14 controls the left 3D module 5 to acquire images, the light detection module 18 adds a left label to the currently acquired 2D image and inputs it into the convolutional neural network model. When the image acquisition module 14 controls the right 3D module 6 to acquire images, the light detection module 18 adds a right label to the currently acquired 2D image and inputs it into the convolutional neural network model. The convolutional neural network model segments the face image in the 2D image, calculates the projection area of ​​the grating in the face image under normal conditions, and judges whether the grating projection in the received 2D image completely falls within the calculated grating projection area based on the calculation result. If it does not completely fall within the grating projection area, the 2D image with the calculated grating projection area is output.

[0165] When capturing facial images, issues such as lighting and equipment malfunctions may lead to unqualified images. Since the projection shape of the grating varies on different faces, it is impossible to determine whether it is qualified through simple system detection. This application uses a large AI model to simulate and calculate the grating projection area of ​​the face, compares it with the actual captured image to determine whether there is a problem with the grating, and provides a deviation image to facilitate user adjustment or repair.

[0166] The embodiments described herein are preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Therefore, all equivalent changes made in accordance with the structure, shape, and principle of the present invention should be covered within the scope of protection of the present invention.

Claims

1. A low-cost high-efficiency three-dimensional face information acquisition method, characterized by, Further comprising the following steps: Determine the shooting position (9), set three RGB cameras and two 3d modules; Joint calibration and device color correction of RGB cameras and 3d modules, and obtain internal and external parameters; Use RGB cameras to collect 2d images and use 3d modules to collect 3d point clouds; Preprocess 2d images and 3d point clouds to obtain triangular meshes; Project each triangular mesh on each 2d image, and calculate the best viewing angle according to the projection of the triangular mesh to the 2d image: Draw a mask on the 2d image shot by the middle RGB camera (4), which is the front view angle; Setting the overall objective function is: ； wherein is a data item selected for the viewing angle, is a penalty coefficient for the penalty term, is a penalty term for selecting different viewing angles between adjacent faces; Data items for computing view selection The formula is as follows: ; In the formula, P is a conditional function of a projection surface, is a triangular mesh, is a corresponding triangular mesh corresponding to a view label, m is the number of triangular meshes, n is the number of view angles, and the calculation is as follows: ; In the formula, is another triangle mesh, is The corresponding triangle mesh corresponds to the view tag, denotes and Adjacent, S represents the penalty function, is a cost function, and the cost function is expressed as follows: ; In the formula, is the middle view angle, is a mask area, X is a constant value, and X is greater than the values corresponding to the left view angle projection area and the right view angle projection area, if the projection area of the triangular mesh falls completely into the mask area, the value corresponding to the middle view angle projection area is replaced by X, and the middle view angle is calculated according to the formula. According to the selected viewing angle, texture is applied to the triangular mesh to obtain three-dimensional facial information.

2. The low-cost and high-efficiency three-dimensional face information acquisition method according to claim 1, characterized in that, The step of "setting three RGB cameras and two 3d modules" further comprises: Set left RGB camera (2), right RGB camera (3), middle RGB camera (4), left 3d module (5) and right 3d module (6); The step of "preprocessing 2d images and 3d point clouds" further comprises: Use a semantic segmentation model to perform face segmentation on each 2d image to filter out background point clouds of 3d point clouds; Face key point recognition is performed on each 2d image; Point cloud fusion is performed on the processed two 3d point clouds to generate triangular mesh data.

3. The low-cost high-efficiency three-dimensional face information acquisition method according to claim 2, characterized in that, The step of "using a semantic segmentation model to perform face segmentation on each 2d image to filter out background point clouds of 3d point clouds" further comprises: Use a semantic segmentation model to perform face segmentation on each 2d image to segment out the background part; According to the segmentation result, point cloud coloring is performed on the 3d point cloud, wherein the background part is RGB colored as (0, 0, 0), and all point clouds with RGB color (0, 0, 0) are deleted.

4. The low-cost and high-efficiency three-dimensional face information acquisition method according to claim 2, characterized in that, The step of "using RGB cameras to collect 2d images and using 3d modules to collect 3d point clouds" further comprises: Set an interval time; Interval shooting is performed when shooting 3d point clouds; The step of "performing point cloud fusion on the processed two 3d point clouds to generate triangular mesh data" further comprises: According to the accuracy requirement, preset the number of times threshold and distance threshold; Step one: for each point in the target point cloud, find and determine its corresponding nearest point in the source point cloud to form multiple sets of "target point-source point" nearest point pairs; Step two: based on multiple sets of "target point-source point" nearest point pairs, construct an optimization objective function, the goal of which is to minimize the distance between the matching points in the target point cloud and the corresponding points in the source point cloud after transformation, and by solving the optimization objective function, obtain a rigid transformation matrix for position adjustment of the target point cloud; Step three: use the rigid transformation matrix to perform rigid transformation on the entire target point cloud to complete position update; In the source point cloud and the target point cloud after position update, repeat step one and update the nearest point pair set; Repeat steps two and three until the number of iterations reaches the preset number of times threshold, or the average distance of all nearest point pairs in the updated nearest point pair set is less than the preset distance threshold, and the iteration is completed; According to the source point cloud and the target point cloud after completing the iteration, perform point cloud registration on the two 3d point clouds.

5. The low-cost and high-efficiency three-dimensional face information acquisition method according to claim 2, characterized in that, The step of "projecting each triangular mesh on each 2D image, and calculating the optimal view angle according to the projection of the triangular mesh on the 2D image" further comprises: Obtaining face key point information according to the identified face key points; Projecting the triangular mesh on each 2D image, and if the triangular mesh has a projection falling within the mask, taking the front view angle as the optimal view angle of the triangular mesh; The remaining triangular meshes not selecting the front view angle as the optimal view angle select the view angle corresponding to the 2D image with the largest projection area as the optimal view angle.

6. The low-cost and high-efficiency three-dimensional face information acquisition method according to claim 1, characterized in that, The step of "texturing the triangular mesh according to the selected view angle" further comprises: Obtaining a texture image, performing cylindrical texture unwrapping on the texture image to obtain a cylindrical unwrapping three-dimensional model, and establishing a coordinate system on the cylindrical unwrapping three-dimensional model; Setting a projection plane, projecting the cylindrical unwrapping three-dimensional model on the projection plane, and calculating the projection coordinates of each point of the cylindrical unwrapping three-dimensional model after projection, the calculation formula being as follows: ； ； ； In the formula, is the x-axis projection coordinate of the projection surface, is the y-axis projection coordinate of the projection surface, and r is the cylindrical radius of the cylindrical development three-dimensional model, is the simulated radius under the sigmoid function, x is the x-axis coordinate of the cylindrical development three-dimensional model on the coordinate system, y is the y-axis coordinate of the cylindrical development three-dimensional model on the coordinate system, z is the z-axis coordinate of the cylindrical development three-dimensional model on the coordinate system, and θ is the radian, and the texture generation is performed on the triangular mesh by reselecting the texture in the texture image according to the calculated projection coordinate.

7. A low-cost and high-efficiency three-dimensional face information acquisition system for performing the three-dimensional face information acquisition method according to claim 1, characterized by: Further comprising a shooting position (9), a left RGB camera (2), a right RGB camera (3), a middle RGB camera (4), a left 3D module (5), a right 3D module (6), and a computing system (1); The computing system (1) jointly calibrates and device color corrects the RGB cameras to obtain internal parameters and external parameters, performs image acquisition to obtain multiple 2D images and multiple 3D point clouds, pre-processes the 2D images and the multiple 3D point clouds to obtain triangular mesh data, projects each triangular mesh on each 2D image, calculates the optimal view angle according to the projection of the triangular mesh on the 2D image, textures the triangular mesh according to the selected view angle, and obtains three-dimensional face information.

8. The low-cost and high-efficiency three-dimensional face information acquisition system according to claim 7, characterized in that: The computing system (1) comprises an information storage module (11), a parameter calculation module (12), a device color correction module (13), an image acquisition module (14), an image processing module (15), and an image fusion module (16); The information storage module (11) is pre-provided with basic information of the shooting position (9), the left RGB camera (2), the right RGB camera (3), the middle RGB camera (4), the left 3D module (5), and the right 3D module (6), and the basic information comprises camera data and position information; The parameter calculation module (12) calls the data stored in the information storage module (11), jointly calibrates the left RGB camera (2), the right RGB camera (3), the middle RGB camera (4), the left 3D module (5), and the right 3D module (6), and calculates the internal parameters and external parameters of the left RGB camera (2), the right RGB camera (3), the middle RGB camera (4), the left 3D module (5), and the right 3D module (6), and transmits the internal parameters and external parameters to the data storage module for storage; The device color correction module (13) calls the camera data of the information storage module (11) to perform device color correction on the left RGB camera (2), the right RGB camera (3), and the middle RGB camera (4); The image acquisition module (14) controls the left RGB camera (2), the right RGB camera (3), the middle RGB camera (4), the left 3D module (5) and the right 3D module (6) to acquire images, obtains a plurality of 2D images and a plurality of 3D point clouds, and transmits the 2D images and the 3D point clouds to the image processing module (15); The image processing module (15) uses a semantic segmentation model to perform face segmentation on each 2D image, filters out background point clouds, performs face key point recognition on each 2D image, and transmits the processed 2D images and 3D point clouds to the image fusion module (16); The image fusion module (16) performs point cloud fusion on the two processed 3D point clouds, generates triangular mesh data, selects the best viewing angle for each triangular mesh in each 2D image, performs texture mapping on the triangular mesh according to the selected viewing angle, and obtains three-dimensional facial information.

9. The low-cost high-efficiency three-dimensional face information acquisition system according to claim 8, characterized by: The image acquisition module (14) is preconfigured with an interval time, and controls the 3D module to capture 3D point clouds at intervals according to the interval time; The image fusion module (16) is preconfigured with a number threshold and a distance threshold, the image fusion module (16) performs point cloud registration on the two 3D point clouds, step one, for each point in the target point cloud, find and determine its corresponding nearest point in the source point cloud, form a plurality of "target point-source point" nearest point pairs, step two, based on the plurality of "target point-source point" nearest point pairs, construct an optimization objective function, the target is to minimize the distance between the matching points in the target point cloud and the corresponding points in the source point cloud after transformation, solve the optimization objective function to obtain a rigid transformation matrix for position adjustment of the target point cloud, step three, use the rigid transformation matrix to perform rigid transformation on the target point cloud as a whole to complete position updating, in the source point cloud and the target point cloud after position updating, repeat step one and update the nearest point pair set, repeat step two and step three until the number of iterations reaches the preconfigured number threshold, or the average distance of all nearest point pairs in the updated nearest point pair set is less than the preconfigured distance threshold, complete iteration, and perform point cloud registration on the two 3D point clouds according to the source point cloud and the target point cloud after completing the iteration.

10. The low-cost and high-efficiency three-dimensional face information acquisition system according to claim 8, characterized in that: The left 3D module (5) and the right 3D module (6) each include two infrared cameras (51) and an infrared grating projection device (52), and the computing system (1) further includes a model generation module (17) and a lighting detection module (18); The model generation module (17) receives the grating images projected by the infrared grating projection device (52) on the face photographed by the left RGB camera (2), the right RGB camera (3) and the middle RGB camera (4), takes the grating images photographed at the same time as a group of grating image groups, adds a left label, a right label or a middle label to the grating images in the same grating image group according to the RGB camera used for shooting, adds a left label or a right label to the grating image group according to the 3D module corresponding to the infrared grating projection device (52) in the grating image group, trains a convolutional neural network model using the stored grating image groups, and transmits the trained convolutional neural network model to the lighting detection module (18); The light detection module (18) is connected to the image acquisition module (14), the light detection module (18) receives the input brightness standard value and stores, when the image acquisition module (14) controls the left 3d module (5) to carry out image acquisition, the light detection module (18) adds the current 2d image collected to the left label and inputs convolutional neural network model, when the image acquisition module (14) controls the right 3d module (6) to carry out image acquisition, the light detection module (18) adds the current 2d image collected to the right label and inputs convolutional neural network model, the convolutional neural network model divides out the face image in 2d image, calculates the projection area of grating on the face image in normal state, according to the calculation result, whether the grating projection in the received 2d image is completely in the projection area of grating calculated, if not completely in the projection area of grating, then output 2d image with the projection area of grating calculated.

Images