Low-cost and high-efficiency three-dimensional face information acquisition system and method

Abstract

The invention discloses a low-cost and high-efficiency three-dimensional face information acquisition system and method, and belongs to the technical field of three-dimensional data acquisition, and the method comprises the steps: determining a shooting position, setting a left RGB camera, a right RGB camera and a middle RGB camera, setting a left 3d module, setting a right 3d module, carrying out the combined calibration, and obtaining an internal parameter and an external parameter. Equipment color correction and image acquisition are carried out on the left RGB camera, the right RGB camera and the middle RGB camera to obtain a plurality of 2d images and a plurality of 3d point clouds, face segmentation and background point cloud filtering are carried out on each 2d image by using a semantic segmentation model, face key point recognition and point cloud fusion are carried out on each 2d image, triangular mesh data are generated, and an optimal visual angle is selected; the triangular mesh is subjected to texture pasting according to the selected visual angle, three-dimensional face information is obtained, and the three-dimensional face information collection method and device have the effect that three-dimensional face information collection can be carried out more rapidly and accurately.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of three-dimensional data acquisition, in particular to a low-cost and high-efficiency three-dimensional facial information acquisition system and method. BACKGROUND

[0002] At present, with the upgrading of various advanced three-dimensional scanning instruments and the development of auxiliary software, three-dimensional scanning technology is also more and more widely used and accepted by clinical physicians and patients. Three-dimensional scanning technology is expected to replace two-dimensional photography and become a routine evaluation method for plastic and aesthetic surgery, providing personalized design for patients and providing more accurate guidance for human organ reconstruction. In the existing face scanning technology, there are two common methods, the first method is to use a 3D module to collect information of a user's face from three angles, and finally calculate the three-dimensional facial information of the user according to the point cloud information. The second method is to directly install three 3D modules at three positions to shoot the user's face from three angles, and finally calculate the three-dimensional facial information of the user according to the point cloud information. At the same time, in order to obtain the texture and color of the user's face, a two-dimensional camera is also needed to shoot the user's face. If only a two-dimensional camera is used to shoot the user's face and an algorithm is used to calculate the three-dimensional facial information, the face needs to be shot from multiple angles, and the final 3D model construction depends on the algorithm, which is not accurate.

[0003] The prior art in the above has the following defects: when using a single 3D module to shoot, the 3D module needs to be moved every time it is shot, which takes a long time. Although three 3D modules are used, the shooting of the 3D modules needs to be spaced to avoid interference, which still takes a long time. SUMMARY

[0004] In order to more quickly and accurately acquire three-dimensional facial information, the present application provides a low-cost and high-efficiency three-dimensional facial information acquisition system and method.

[0005] In one aspect, the low-cost and high-efficiency three-dimensional facial information acquisition method provided by the present application adopts the following technical solution: A low-cost and high-efficiency three-dimensional facial information acquisition method further comprises the following steps: Determine the shooting position, set three RGB cameras and two 3D modules; Jointly calibrate the RGB cameras and the 3D modules and calibrate the device color, and obtain the internal parameters and external parameters; Use the RGB camera to acquire 2D images and use the 3D module to acquire 3D point clouds; Preprocess the 2D images and 3D point clouds to obtain a triangular mesh; Project each triangle mesh on each 2D image, and calculate the best view angle according to the projection of the triangle mesh on the 2D image: Draw the mask on the 2D image shot by the RGB camera in the middle, and the 2D image shot by the RGB camera in the middle is the front view angle; Set the total target function is: ; In the formula, is the data item of view angle selection, is the penalty coefficient of the penalty item, is the penalty item when different view angles are selected between adjacent surfaces; calculate the data item of view angle selection , the formula is as follows: ; In the formula, P is the condition function of the projection surface, is the triangle mesh, is the view label corresponding to the corresponding triangle mesh, m is the number of triangle meshes, n is the number of view angles, and is calculated, the formula is as follows: ; In the formula, is another triangle mesh, is the view label corresponding to the corresponding triangle mesh, indicates and are adjacent, S represents the penalty function, is the cost function, and the cost function is expressed as the following formula: ; In the formula, is the intermediate view angle, is the mask area, X is a constant value, and X is greater than the corresponding values of the left view angle projection area and the right view angle projection area, if the projection area of the triangle mesh completely falls into the mask area, the corresponding value of the intermediate view angle projection area is replaced by X, and the intermediate view angle is calculated according to the formula; According to the selected view angle, the triangle mesh is textured to obtain three-dimensional facial information.

[0006] By adopting the above scheme, three-dimensional face information can be rapidly and accurately collected through two groups of 3d modules and three RGB cameras. Since only two groups of 3d modules are used, one shooting interval is reduced compared with using three 3d modules, so that the efficiency is higher, and since one 3d module is reduced, the cost is also lower. The efficiency is higher than that of using only one 3d module, and since the 3d module does not need to be moved, the moving structure is also saved. Therefore, the application can effectively save the time and cost of collecting three-dimensional face information by using the traditional 3d module, and the information collection accuracy is higher than that of using only a 2d camera to shoot a face and calculating three-dimensional face information by an algorithm. At the same time, when the texture is pasted on the three-dimensional model, the application selects the view angle for each triangular mesh to ensure that the texture of each triangular mesh selected finally is the best view angle, and the three-dimensional face model generated finally is complete and is not prone to problems such as texture misplacement.

[0007] Preferably, the step of "setting three RGB cameras and two 3d modules" further comprises: setting a left RGB camera, a right RGB camera, a middle RGB camera, a left 3d module and a right 3d module; The step of "preprocessing the 2d image and the 3d point cloud" further comprises: segmenting each 2d image using a semantic segmentation model to filter out background points of the 3d point cloud; performing face key point recognition on each 2d image; fusing the two processed 3d point clouds to generate triangular mesh data.

[0008] By adopting the above scheme, the image preprocessing includes face key point recognition on the 2d image and background point cloud filtering on the 3d point cloud, so that the face features in the image can be effectively extracted, and it is ensured that the generated triangular mesh is on the face.

[0009] Preferably, the step of "segmenting each 2d image using a semantic segmentation model to filter out background points" further comprises: segmenting each 2d image using a semantic segmentation model to filter out background points; performing point cloud coloring on the 3d point cloud according to the segmentation result, wherein the background part is colored as (0, 0, 0), and all point clouds colored as (0, 0, 0) are deleted.

[0010] By adopting the above scheme, since the 2D shooting will inevitably retain the background part, the existence of the background will introduce more noise, which will affect the subsequent color transfer and point cloud reconstruction. By combining the semantic segmentation model, the background part can be quickly and simply screened out, and then the point cloud is deleted, so that the influence of the background is reduced.

[0011] Preferably, the step of "collecting 2d images using an RGB camera and collecting 3d point clouds using a 3d module" further comprises: Setting an interval time; Taking interval shots when taking 3d point clouds; The step of "fusing the two processed 3d point clouds and generating triangular mesh data" further comprises: According to the accuracy requirement, preset the number of times threshold and distance threshold; Step one: for each point in the target point cloud, find and determine its corresponding nearest point in the source point cloud, forming multiple sets of "target point-source point" nearest point pairs; Step two: based on multiple sets of "target point-source point" nearest point pairs, construct an optimization objective function, the goal of which is to minimize the distance between the matching points in the target point cloud after transformation and the corresponding points in the source point cloud, and obtain the rigid transformation matrix for position adjustment of the target point cloud by solving the optimization objective function; Step three: perform rigid transformation on the target point cloud as a whole using the rigid transformation matrix to complete position updating; In the source point cloud and the target point cloud after position updating, repeat step one and update the nearest point pair set; Repeat steps two and three until the number of iterations reaches the preset number of times threshold, or the average distance of all nearest point pairs in the updated nearest point pair set is less than the preset distance threshold, completing iteration; According to the source point cloud and the target point cloud after completing iteration, perform point cloud registration on the two 3d point clouds.

[0012] By adopting the above scheme, since 3d point clouds need to be taken at intervals to avoid the mutual influence of light during shooting, but the face may move slightly in the pictures taken at different times, which will cause errors, so point cloud registration is needed to match the 3d point clouds taken by the two 3d point clouds, effectively increasing the accuracy of three-dimensional facial information collection.

[0013] Preferably, the step of "projecting each triangular mesh on each 2d image and calculating the best viewing angle according to the projection of the triangular mesh on the 2d image" further comprises: Obtaining face key point information according to the identified face key points; Projecting the triangular mesh on each 2d image, if the triangular mesh has a projection falling inside the mask, then the front view angle is selected as the best view angle of the triangular mesh; The remaining triangular meshes that do not select the front view angle as the best view angle select the view angle corresponding to the 2d image with the largest projection area as the best view angle.

[0014] By adopting the above scheme, since the three 2d image perspectives are different, problems are prone to occur during splicing, and the application selects the best perspective according to specific rules to ensure that the texture after the best perspective corresponding to each triangular mesh can be completely spliced, and the possibility of problems is reduced.

[0015] Preferably, the step of "texturing the triangular mesh according to the selected perspective" further comprises: obtaining a texture image, performing cylindrical texture unfolding on the texture image to obtain a cylindrical unfolded three-dimensional model, and establishing a coordinate system on the cylindrical unfolded three-dimensional model; setting a projection plane, projecting the cylindrical unfolded three-dimensional model on the projection plane, calculating the projection coordinates of each point of the cylindrical unfolded three-dimensional model after projection, and the calculation formula is as follows: ; ; ; In the formula, is the x-axis projection coordinate on the projection plane, is the y-axis projection coordinate on the projection plane, r is the cylindrical radius of the cylindrical unfolded three-dimensional model, is the simulated radius under the sigmoid function, x is the x-axis coordinate of the cylindrical unfolded three-dimensional model on the coordinate system, y is the y-axis coordinate of the cylindrical unfolded three-dimensional model on the coordinate system, z is the z-axis coordinate of the cylindrical unfolded three-dimensional model on the coordinate system, and theta is the radian, and the texture of the triangular mesh is generated according to the calculated projection coordinates in the texture image.

[0016] By adopting the above scheme, a new calculation method is used, the unfolding radius at the nasal ala is increased to ensure that there is enough unfolding space after the texture is unfolded, and the unfolding radius of other facial points will not be significantly increased to ensure the stability of the overall rendering result. When the three-dimensional facial information collection is performed in the step of texturing, cylindrical texture unfolding is usually performed, and the texture is pasted after projection and calculation. However, due to the special shape of the nose at the nasal ala, the angle between the projection and the projection plane is close to vertical, resulting in too few pixels after projection, and too few pixels after texture unfolding, which will cause texture stretching. Therefore, the projection area at the nasal ala needs to be increased.

[0017] On the other hand, the low-cost and high-efficiency three-dimensional facial information collection system provided by the application adopts the following technical scheme: A low-cost and high-efficiency three-dimensional facial information collection system, comprising the three-dimensional facial information collection method as described above, further comprising a shooting position, a left RGB camera, a right RGB camera, a middle RGB camera, a left 3d module, a right 3d module, and a computing system. The computing system jointly calibrates and device color corrects the RGB cameras to obtain internal parameters and external parameters, performs image acquisition to obtain a plurality of 2d images and a plurality of 3d point clouds, pre-processes the 2d images and the plurality of 3d point clouds to obtain triangular mesh data, projects each triangular mesh on each 2d image, calculates the best viewing angle according to the projection of the triangular mesh on the 2d image, and performs texture mapping on the triangular mesh according to the selected viewing angle to obtain three-dimensional facial information.

[0018] By adopting the above scheme, three-dimensional facial information can be rapidly and accurately collected by two groups of 3d modules and three RGB cameras. Since only two groups of 3d modules are used, one shooting interval is reduced compared with using three 3d modules, so the efficiency is higher, and since one 3d module is reduced, the cost is also lower. The efficiency is higher than using only one 3d module, and since the 3d module does not need to be moved, the moving structure is also saved. At the same time, when the texture is pasted on the three-dimensional model, the viewing angle of each triangular mesh is selected to ensure that the texture of each triangular mesh selected finally is the best viewing angle, and the three-dimensional facial model generated finally is complete and is not prone to texture misplacement and the like.

[0019] Preferably, the computing system comprises an information storage module, a parameter calculation module, a device color correction module, an image acquisition module, an image processing module and an image fusion module. The information storage module is pre-provided with basic information of a shooting position, a left RGB camera, a right RGB camera, a middle RGB camera, a left 3d module and a right 3d module, and the basic information comprises camera data and position information. The parameter calculation module calls the data stored in the information storage module to jointly calibrate the left RGB camera, the right RGB camera, the middle RGB camera, the left 3d module and the right 3d module, and calculate the internal parameters and external parameters of the left RGB camera, the right RGB camera, the middle RGB camera, the left 3d module and the right 3d module, and transmit the internal parameters and external parameters to the data storage module for storage. The device color correction module calls the camera data of the information storage module to perform device color correction on the left RGB camera, the right RGB camera and the middle RGB camera. The image acquisition module controls the left RGB camera, the right RGB camera, the middle RGB camera, the left 3d module and the right 3d module to perform image acquisition to obtain a plurality of 2d images and a plurality of 3d point clouds, and transmits the 2d images and the 3d point clouds to the image processing module. The image processing module uses a semantic segmentation model to perform face segmentation on each 2d image, filters out background point clouds, performs face key point recognition on each 2d image, and transmits the processed 2d images and 3d point clouds to the image fusion module. The image fusion module fuses the two processed 3D point clouds, generates triangular mesh data, selects the best view angle for each triangular mesh in each 2D image, and performs texture mapping on the triangular mesh according to the selected view angle to obtain three-dimensional facial information.

[0020] Preferably, the image acquisition module is pre-set with an interval time, and the 3D module is controlled to take interval shots according to the interval time when taking 3D point clouds. The image fusion module is pre-set with a number threshold and a distance threshold. The image fusion module performs point cloud registration on the two 3D point clouds. Step one: for each point in the target point cloud, find and determine its corresponding nearest point in the source point cloud to form a plurality of sets of "target point-source point" nearest point pairs. Step two: based on the plurality of sets of "target point-source point" nearest point pairs, construct an optimization objective function, the objective of which is to minimize the distance between the matching points in the target point cloud after transformation and the corresponding points in the source point cloud. By solving the optimization objective function, a rigid transformation matrix for position adjustment of the target point cloud is obtained. Step three: use the rigid transformation matrix to perform rigid transformation on the target point cloud as a whole to complete position updating. In the source point cloud and the target point cloud after position updating, repeat step one and update the nearest point pair set. Repeat steps two and three until the number of iterations reaches the pre-set number threshold, or the average distance of all nearest point pairs in the updated nearest point pair set is less than the pre-set distance threshold, and the iteration is completed. The two 3D point clouds are registered based on the source point cloud and the target point cloud after the iteration is completed.

[0021] By adopting the above scheme, since the 3D module needs to take interval shots to avoid the mutual influence of light during shooting, but the face may move slightly in the pictures taken at different times, which will cause errors. Therefore, point cloud registration is needed to match the 3D point clouds taken by the two 3D modules, effectively increasing the accuracy of three-dimensional facial information acquisition.

[0022] Preferably, the left 3D module and the right 3D module each include two infrared cameras and an infrared grating projection device. The computing system further includes a model generation module and a lighting detection module. The model generation module receives the grating images projected by the infrared grating projection device on the face taken by the left RGB camera, the right RGB camera and the middle RGB camera, takes the grating images taken at the same time as a set of grating image groups, adds a left label, a right label or a middle label to the grating images in the same grating image group according to the shooting RGB camera, adds a left label or a right label to the grating image group according to the 3D module corresponding to the infrared grating projection device in the grating image group, trains the convolutional neural network model using the stored grating image group, and transmits the trained convolutional neural network model to the lighting detection module. The light detection module is connected with the image acquisition module, the light detection module receives the input brightness standard value and stores, when the image acquisition module controls the left 3D module to perform image acquisition, the light detection module adds the current 2D image to the left label and inputs the convolutional neural network model, when the image acquisition module controls the right 3D module to perform image acquisition, the light detection module (18) adds the current 2D image to the right label and inputs the convolutional neural network model, the convolutional neural network model segments the face image in the 2D image, calculates the projection area of the normal state grating on the face image, judges whether the grating projection in the received 2D image is completely in the calculated projection area of the grating, if not completely in the projection area of the grating, the 2D image with the calculated projection area of the grating is output.

[0023] By adopting the above scheme, when the face image is collected, the collected image may be unqualified due to light collection, equipment failure and the like, and the grating cannot be simply detected and judged by the system because the projection shape of the grating changes on different faces, the AI large model is used to simulate and calculate the grating projection area of the face, the grating is compared with the actually collected image to judge whether the grating has a problem, and a deviation image is given, so that the user can adjust or repair.

[0024] In summary, the present application has the following advantages: 1. The three-dimensional face information can be quickly and accurately collected by two groups of 3D modules and three RGB cameras, and the efficiency is higher than that of using three 3D modules because only two groups of 3D modules are used, and the shooting interval is reduced by one time, so the efficiency is higher. The efficiency is obviously higher than that of using only one 3D module.

[0025] 2. Compared with using three 3D modules, the cost is lower because one 3D module is reduced. BRIEF DESCRIPTION OF DRAWINGS

[0026] Figure 1 is a schematic diagram of a protruding cylindrical development three-dimensional model of the embodiment one of the present application.

[0027] Figure 2 is a whole schematic diagram of the embodiment two of the present application.

[0028] Figure 3 is a schematic diagram of a protruding angle of the embodiment two of the present application.

[0029] Figure 4 is a system block diagram of the computing system of the embodiment two of the present application.

[0030] Figure 5 is a system block diagram of the embodiment three of the present application.

[0031] BRIEF DESCRIPTION OF DRAWINGS 1, computing system; 11, information storage module; 12, parameter calculation module; 13, device color correction module; 14, image acquisition module; 15, image processing module; 16, image fusion module; 17, model generation module; 18, lighting detection module; 2, left RGB camera; 3, right RGB camera; 4, middle RGB camera; 5, left 3d module; 51, infrared camera; 52, infrared grating projection device; 6, right 3d module; 7, left mirror; 8, right mirror; 9, shooting position; 10, upper fixed plate; 101, left vertical plate; 102, middle vertical plate; 103, right vertical plate; 104, lower fixed plate; 105, front frame plate; 106, left fixed plate; 107, right fixed plate. DETAILED DESCRIPTION

[0032] The following will be described in detail in combination with the accompanying drawings Figures 1-3 The application is further described in detail.

[0033] Embodiment one, the embodiment of the application discloses a low-cost and high-efficiency three-dimensional face information acquisition method, the specific steps are as follows: Determine the shooting position 9, set the left RGB camera 2, the right RGB camera 3 and the middle RGB camera 4, which are respectively arranged on the left side, the right side and the front of the shooting position 9.

[0034] A left 3d module 5 is arranged at the position of the left RGB camera 2, and a right 3d module 6 is arranged at the position of the right RGB camera 3. The left 3d module 5 and the right 3d module 6 each include two infrared cameras 51 and an infrared grating projection device 52. The infrared grating projection device 52 is used to project an infrared grating with coding information to the shooting position 9, and the infrared cameras 51 acquire matching point depth information according to the coding information on the infrared grating in the photographed image information.

[0035] A left mirror 7 and a right mirror 8 are respectively arranged on the two sides of the shooting position 9, the left RGB camera 2 and the left 3d module 5 are arranged towards the left mirror 7, the left RGB camera 2 and the left 3d module 5 acquire images of the shooting position 9 through the left mirror 7, the right RGB camera 3 and the right 3d module 6 are arranged towards the right mirror 8, and the right RGB camera 3 and the right 3d module 6 acquire images of the shooting position 9 through the right mirror 8. The left RGB camera 2 and the left 3d module 5 can shoot most of the area of the left half of the face through the left mirror 7, and the right RGB camera 3 and the right 3d module 6 can shoot most of the area of the right half of the face through the right mirror 8. The two mirrors can make the distance between the cameras smaller, so as to reduce the size of the device, and the mirrors can increase the shooting range of the cameras, thereby increasing the shooting space of the user.

[0036] Adjust the left RGB camera 2, the right RGB camera 3, the middle RGB camera 4, the left 3d module 5 and the right 3d module 6 to shoot the shooting position 9.

[0037] Set the interval time.

[0038] Joint calibration is performed on the left RGB camera 2, the right RGB camera 3, the middle RGB camera 4, the left 3D module 5 and the right 3D module 6, and the internal parameters and external parameters of the left RGB camera 2, the right RGB camera 3, the middle RGB camera 4, the left 3D module 5 and the right 3D module 6 are obtained. A camera coordinate system is established, and one camera coordinate system is selected as a world coordinate system.

[0039] The internal parameters and external parameters of the left RGB camera 2, the right RGB camera 3, the middle RGB camera 4, the left 3D module 5 and the right 3D module 6 are calculated according to the internal and external parameter algorithm, and the internal and external parameter algorithm is as follows: The internal parameter I is calculated, and the formula is as follows: ; In the formula, f is the focal length, fx is the X-axis focal length, fy is the Y-axis focal length, cx is the X-axis optical center coordinate, cy is the Y-axis optical center coordinate, the pixel physical size of the image is calculated, and the formula is as follows: ; ; In the formula, px is the X-axis pixel size, py is the Y-axis pixel size, the pixel coordinate system is determined according to the pixel size, and the camera imaging formula is generated using the internal parameter I of the camera, as follows: ; In the formula, u is the X-axis coordinate of the pixel coordinate system, and v is the Y-axis coordinate of the pixel coordinate system; The external parameter E is calculated using the related parameters of the image, and the formula is as follows: ; In the formula, x0 is the X-axis coordinate of the camera coordinate system, y0 is the Y-axis coordinate of the camera coordinate system, z0 is the Z-axis coordinate of the camera coordinate system, xw is the X-axis coordinate of the world coordinate system, yw is the Y-axis coordinate of the world coordinate system, zw is the Z-axis coordinate of the world coordinate system.

[0040] The left RGB camera 2, the right RGB camera 3 and the middle RGB camera 4 are calibrated.

[0041] Image acquisition is performed using the left RGB camera 2, the right RGB camera 3, and the middle RGB camera 4 to obtain a plurality of 2D images, and the left 3D module 5 and the right 3D module 6 to obtain a plurality of 3D point clouds. When shooting 3D point clouds, interval shooting is performed according to interval time.

[0042] Each 2D image is subjected to face segmentation using a semantic segmentation model, and the background part is segmented out.

[0043] According to the segmentation result, the 3D point cloud is subjected to point cloud coloring, wherein the background part is colored as (0, 0, 0), and all point clouds with RGB color (0, 0, 0) are deleted.

[0044] Face key point recognition is performed on each 2D image. Because 2D shooting will inevitably leave a background part, the existence of the background will introduce a lot of noise, affecting subsequent color transfer and point cloud reconstruction. In combination with the semantic segmentation model, the background part can be quickly and simply screened out, and then the point cloud is deleted, thereby reducing the influence of the background.

[0045] The two processed 3D point clouds are subjected to point cloud fusion, and triangular mesh data is generated.

[0046] The two 3D point clouds are subjected to point cloud registration, and according to the accuracy requirement, a preset number of threshold values and distance threshold values are used to perform the following steps: Step one, for each point in the target point cloud, find and determine its corresponding nearest point in the source point cloud, forming a plurality of sets of "target point-source point" nearest point pairs.

[0047] Step two, based on the plurality of sets of "target point-source point" nearest point pairs, an optimization objective function is constructed, the objective being to minimize the distance between the matching points in the target point cloud after transformation and the corresponding points in the source point cloud, and by solving the optimization objective function, a rigid transformation matrix for position adjustment of the target point cloud is obtained.

[0048] Step three, the rigid transformation matrix is used to perform rigid transformation on the target point cloud as a whole to complete position updating.

[0049] In the source point cloud and the target point cloud after position updating, step one is repeated, and the nearest point pair set is updated.

[0050] Steps two and three are repeated until the number of iterations reaches the preset number of threshold values, or the average distance of all nearest point pairs in the updated nearest point pair set is less than the preset distance threshold value, and the iteration is completed.

[0051] According to the source point cloud and the target point cloud after completing iteration, the two 3D point clouds are point cloud registered. Since the 3D module needs to be shot at intervals to avoid the mutual influence of light during shooting, but the face may move slightly in the pictures taken at different times, errors will be generated, so point cloud registration is needed to match the 3D point clouds shot by the two 3D modules, effectively increasing the accuracy of three-dimensional facial information acquisition.

[0052] The best view angle of each triangular mesh in each 2D image is selected. The face key point information is obtained according to the identified face key points. The mask is drawn on the 2D image shot by the RGB camera 4, and the 2D image shot by the RGB camera 4 is the front view angle.

[0053] The triangular mesh is projected to each 2D image, and if the triangular mesh has a projection falling inside the mask, the front view angle is selected as the best view angle of the triangular mesh.

[0054] The triangular mesh which is not selected as the best view angle selects the view angle corresponding to the 2D image with the largest projection area as the best view angle.

[0055] The specific formula calculation method is as follows: the total target function is ; ; In the formula, the data item of view angle selection is , the penalty coefficient is , and the penalty item when selecting different view angles between adjacent surfaces is The data item of view angle selection is calculated as , and the formula is as follows: ; In the formula, P is the condition function of the projection surface, is the triangular mesh, is the corresponding triangular mesh corresponding to the view label, m is the number of triangular meshes, and n is the number of view angles. The data item influences the selection of the view angle with the largest projection area for each triangular mesh. Since the right side of the formula starts with a negative sign, when the projection area of the view angle found by all meshes is the largest, is the smallest, so the best view angle of each scanning mesh can be obtained by optimization. The calculation is , and the formula is as follows: ; In the formula, another triangular mesh is , and is the corresponding triangular mesh corresponding to the view label, , and represents ​and S represents a penalty function. is a cost function, the cost function is represented as follows: ; In the formula, is the middle view angle, is the mask area, X is a constant value, and X is greater than the value corresponding to the left view angle projection area and the right view angle projection area, if the projection area of the triangular mesh completely falls into the mask area, the value corresponding to the middle view angle projection area is replaced by X, the middle view angle is calculated according to the formula, and according to the formula, if the projection area of the triangular mesh completely falls into the mask area, the cost is X. In actual use, a larger projection area is used to replace X, for example, the common projection area is 100, if the projection of the grid at the front view angle completely falls within the mask, regardless of the actual projection area, it is directly assigned to 10000, which will far exceed the projection area of this patch at the other two view angles, so as to force the view angle of this patch to be specified as the middle view angle during optimization.

[0056] Due to the difference in the three 2d image view angles, problems are prone to occur during splicing, the application selects the best view angle through specific rules to ensure that the texture corresponding to each triangular mesh after the best view angle can be completely spliced, and the possibility of problems is reduced.

[0057] The triangular mesh is textured according to the selected view angle to obtain a texture image.

[0058] As shown in Figure 1 , the texture image is cylindrically unwrapped to obtain a cylindrical unwrapped three-dimensional model, and a coordinate system is established on the cylindrical unwrapped three-dimensional model; A projection plane is set, the cylindrical unwrapped three-dimensional model is projected on the projection plane, the projection coordinates of each point of the cylindrical unwrapped three-dimensional model after projection are calculated, and the calculation formula is as follows: ; ; ; In the formula, is the x-axis projection coordinate on the projection plane, is the y-axis projection coordinate on the projection plane, and r is the radius of the cylindrical unwrapped three-dimensional model, is the simulation radius under the sigmoid function, x is the x-axis coordinate of the cylindrical three-dimensional model on the coordinate system, y is the y-axis coordinate of the cylindrical three-dimensional model on the coordinate system, z is the z-axis coordinate of the cylindrical three-dimensional model on the coordinate system, theta is the radian, and the three-dimensional face information is obtained by reselecting the texture in the texture image according to the calculated projection coordinates to generate the texture of the triangular mesh.

[0059] In the step of performing texture pasting, due to different shooting angles of the three 2d images, the corresponding positions of part of the facial features on the three 2d images are different. If the three 2d images are directly projected and pasted with textures, the color spots and other features on the face will be repeatedly displayed at different positions. Therefore, the projection positions of the three 2d images need to be processed to be consistent. The texture expansion is usually cylindrical texture expansion, that is, the face is regarded as a cylinder, the 2d image is expanded on the cylinder, and then projected onto a plane to avoid subsequent image analysis difficulties. However, due to the special shape of the nose wing, the angle between the projection and the projection plane is close to vertical, resulting in too few pixels after projection, and the texture expansion cannot obtain enough pixels, which will cause texture stretching. Therefore, it is necessary to increase the projection area of the nose wing. If the expansion radius of the entire face is linearly increased, the pixels on both sides of the face after texture expansion will be too much, which will squeeze the middle of the face. Part of the present application uses a new calculation method, which increases the expansion radius of the nose wing to ensure that there is enough expansion space after texture expansion, and the expansion radius of other facial points will not increase significantly, ensuring the stability of the overall rendering result. Using the formula of the present application, the cylindrical radius will increase rapidly with the distance within the range of 5 cm, so that the 2d image near the nose wing has enough expansion radius, and the expansion radius is almost unchanged after exceeding 5 cm, ensuring that the pixel distribution is uniform after texture expansion of the rest of the face.

[0060] The implementation principle of the low-cost and high-efficiency three-dimensional face information acquisition method of the embodiment of the present application is that: two groups of 3d modules and three RGB cameras can quickly and accurately acquire three-dimensional face information. Since only two groups of 3d modules are used, one shooting interval is reduced compared with using three 3d modules, so the efficiency is higher, and since one 3d module is reduced, the cost is also lower. The efficiency is higher than that of using only one 3d module, and since the 3d module does not need to be moved, the moving structure is also saved.

[0061] The embodiment of the present application discloses a low-cost and high-efficiency three-dimensional face information acquisition system, which uses the three-dimensional face information acquisition method of the embodiment one, such as Figure 2 and Figure 3As shown, it comprises a lower fixed plate 104, an upper fixed plate 10, a front frame plate 105 bolted to the upper fixed plate 10 and the lower fixed plate 104, and a computing system 1, the upper fixed plate 10 and the lower fixed plate 104 are bolted with a left vertical plate 101, a middle vertical plate 102 and a right vertical plate 103, a shooting position 9 is arranged in the device, the left vertical plate 101 is fixedly connected with a left RGB camera 2 and a left 3D module 5, the middle vertical plate 102 is fixedly connected with a middle RGB camera 4, the right vertical plate 103 is fixedly connected with a right RGB camera 3 and a right 3D module 6, the left RGB camera 2, the right RGB camera 3 and the middle RGB camera 4 are arranged at the left side, the right side and the front of the shooting position 9 respectively, the left 3D module 5 is located at the position of the left RGB camera 2, and the right 3D module 6 is located at the position of the right RGB camera 3. The left 3D module 5 and the right 3D module 6 each comprise two infrared cameras 51 and an infrared grating projection device 52. The infrared grating projection device 52 projects an infrared grating with coding information to the shooting position 9, and the infrared cameras 51 obtain matching point depth information according to the coding information on the infrared grating in the photographed image information.

[0062] As Figure 2 and Figure 3As shown, the angle between the line connecting the center of the target surface of the left RGB camera and the shooting position and the central axis of the mirror surface of the middle RGB camera is α, the angle between the center of the target surface of the right RGB camera and the central axis of the mirror surface of the middle RGB camera is β, the angle between the center of the target surface of the camera in the left 3D module and the central axis of the mirror surface of the middle RGB camera is also α, and the angle between the center of the target surface of the camera in the right 3D module and the central axis of the mirror surface of the middle RGB camera is also β. The left fixed plate 106 and the right fixed plate 107 are respectively bolted at positions corresponding to the two sides of the shooting position 9 between the upper fixed plate 10 and the lower fixed plate 104, the left fixed plate 106 is fixedly connected with the left mirror 7 inside, and the right fixed plate 107 is fixedly connected with the right mirror 8 inside. The left fixed plate 106 is provided with a left pressing plate corresponding to the position of the left mirror 7, and the left mirror 7 is pressed tightly on the left pressing plate by the left fixed plate 106 through bolts. The right fixed plate 107 is provided with a right pressing plate corresponding to the position of the right mirror 8, and the right mirror 8 is pressed tightly on the right pressing plate by the right fixed plate 107 through bolts. The left RGB camera 2 and the left 3D module 5 are arranged towards the left mirror 7, and the point on the central axis of the mirror surface of the left RGB camera to the surface of the left mirror is taken as the vertex, the angle between the central axis of the mirror surface of the left RGB camera and the perpendicular of the surface of the left mirror is γ, the left RGB camera 2 and the left 3D module 5 collect the image of the shooting position 9 through the left mirror 7, and the left RGB camera 2 and the left 3D module 5 can shoot the left half of the face through the left mirror 7. The right RGB camera 3 and the right 3D module 6 are arranged towards the right mirror 8, and the point on the central axis of the mirror surface of the right 3D module camera to the surface of the right mirror is taken as the center, the angle between the central axis of the mirror surface of the right 3D module camera and the perpendicular of the surface of the right mirror is δ, the right RGB camera 3 and the right 3D module 6 collect the image of the shooting position 9 through the right mirror 8, and the right RGB camera 3 and the right 3D module 6 can shoot the right half of the face through the right mirror 8.

[0063] As shown in Figure 4 The computing system 1 includes an information storage module 11, a parameter calculation module 12, a device color correction module 13, an image acquisition module 14, an image processing module 15, and an image fusion module 16.

[0064] As shown in Figure 2 and Figure 4 The information storage module 11 is pre-provided with the basic information of the shooting position 9, the left RGB camera 2, the right RGB camera 3, the middle RGB camera 4, the left 3D module 5, and the right 3D module 6, and the basic information includes camera data and position information.

[0065] As shown in Figure 2 and Figure 4As shown in the figure, the parameter calculation module 12 calls the data stored in the information storage module 11 to jointly calibrate the left RGB camera 2, the right RGB camera 3, the middle RGB camera 4, the left 3D module 5 and the right 3D module 6, and calculate the internal and external parameters of the left RGB camera 2, the right RGB camera 3, the middle RGB camera 4, the left 3D module 5 and the right 3D module 6, and transmit the internal and external parameters to the data storage module for storage.

[0066] As shown in the figure, Figure 2 and Figure 4 As shown in the figure, the device color correction module 13 calls the camera data of the information storage module 11 to perform device color correction on the left RGB camera 2, the right RGB camera 3 and the middle RGB camera 4.

[0067] As shown in the figure, Figure 2 and Figure 4 As shown in the figure, the image acquisition module 14 is pre-set with an interval time, and the image acquisition module 14 controls the left RGB camera 2, the right RGB camera 3, the middle RGB camera 4, the left 3D module 5 and the right 3D module 6 to perform image acquisition, controls the 3D point cloud to be photographed according to the interval time, and obtains a plurality of 2D images and a plurality of 3D point clouds, and transmits the 2D images and the 3D point clouds to the image processing module 15.

[0068] As shown in the figure, Figure 2 and Figure 4 As shown in the figure, the image processing module 15 uses a semantic segmentation model to filter out the background of each 2D image, and the image processing module 15 performs face key point recognition on each 2D image. The image processing module 15 uses a semantic segmentation model to perform face segmentation on each 2D image, segments out the background part, performs point cloud coloring on the 3D point cloud according to the segmentation result, wherein the RGB coloring of the background part is (0, 0, 0), and all point clouds with RGB coloring of (0, 0, 0) are deleted. The processed 2D images and 3D point clouds are transmitted to the image fusion module 16.

[0069] As shown in the figure, Figure 2 and Figure 4As shown, the image fusion module 16 presets a number threshold and a distance threshold, the image fusion module 16 performs point cloud registration on the two 3D point clouds, step one, for each point in the target point cloud, find and determine its corresponding nearest point in the source point cloud, form a plurality of sets of “target point-source point” nearest point pairs, step two, based on a plurality of sets of “target point-source point” nearest point pairs, construct an optimization objective function, the goal is to minimize the distance between the matching points in the target point cloud and the corresponding points in the source point cloud after transformation, obtain the rigid transformation matrix for the position adjustment of the target point cloud by solving the optimization objective function, step three, use the rigid transformation matrix to perform rigid transformation on the target point cloud as a whole to complete the position update, in the source point cloud and the target point cloud after the position update, repeat step one and update the nearest point pair set, repeat step two and step three until the number of iterations reaches the preset number threshold, or the average distance of all nearest point pairs in the updated nearest point pair set is less than the preset distance threshold, complete the iteration, and perform point cloud registration on the two 3D point clouds according to the source point cloud and the target point cloud after the completion of the iteration.

[0070] As shown in Figure 2 and Figure 4 , the image fusion module 16 selects the best view angle for each triangular mesh in each 2D image, obtains face key point information according to the identified face key points, draws a mask on the 2D image captured by the RGB camera 4, the 2D image captured by the RGB camera 4 is a front view, projects the triangular mesh to each 2D image, if the triangular mesh has a projection falling inside the mask, the front view is selected as the best view angle for the triangular mesh, and the remaining triangular meshes that do not select the front view as the best view angle select the view angle corresponding to the 2D image with the largest projection area as the best view angle. According to the selected view angle, the triangular mesh is textured to obtain three-dimensional facial information. Since the three 2D images have different view angles, problems are likely to occur during splicing. The present application selects the best view angle through specific rules to ensure that the texture of each triangular mesh after the corresponding best view angle can be completely spliced, reducing the possibility of problems.

[0071] The specific formula calculation method is as follows: the total objective function here is: ; In the formula, is the data item of view angle selection, is the penalty coefficient of the penalty item, is the penalty item when different view angles are selected between adjacent surfaces; the data item of view angle selection is calculated , and the formula is as follows: ; In the formula, P is the condition function of the projection surface, is a triangular mesh, is is a corresponding view tag of the corresponding triangular mesh, m is a number of triangular meshes, and n is a number of view angles. The data item selects a view angle with the largest projection area for each triangular mesh, since the right side of the formula starts with a negative sign, when the projection area of the view angle found by all meshes is the largest, is the smallest, so that the optimal view angle of each scanning mesh can be obtained in an optimized manner. The calculation is as follows: ; In the formula, is another triangular mesh, is is a corresponding view tag of the corresponding triangular mesh, indicates and are adjacent, and S indicates a penalty function. is a cost function, the cost function indicates the following formula: ; In the formula, is an intermediate view angle, is a mask area, X is a constant value, and X is greater than the values corresponding to the projection areas of the left view angle and the right view angle. If the projection area of the triangular mesh completely falls within the mask area, the value corresponding to the projection area of the intermediate view angle is replaced by X. The intermediate view angle is selected according to the formula, and according to the formula, if the projection area of the triangular mesh completely falls within the mask area, the cost is X. In actual use, a larger projection area is used to replace X. For example, a common projection area is 100, and if the projection of the mesh at the front view angle completely falls within the mask, regardless of the actual projection area, it is directly assigned a value of 10000, which is much larger than the projection areas of the mesh at the other two view angles, so that the view angle of the mesh is forced to be designated as the intermediate view angle during optimization.

[0072] Embodiment three, the embodiment of the application discloses a low-cost and high-efficiency three-dimensional face information acquisition system, as Figure 2 Figure 4 Figure 5 shown, which is different from embodiment two. The computing system 1 further includes a model generation module 17 and a lighting detection module 18.

[0073] The model generation module 17 receives the raster images projected by the infrared raster projection device 52 on the human face, photographed by the left RGB camera 2, the right RGB camera 3 and the middle RGB camera 4, takes the raster images photographed at the same time as a group of raster image groups, adds a left label, a right label or a middle label to the raster images in the same group of raster image groups according to the RGB camera used for photographing, adds a left label or a right label to the group of raster images according to the 3d module corresponding to the infrared raster projection device 52 in the group of raster images, trains the convolutional neural network model using the stored group of raster images, and transmits the trained convolutional neural network model to the light detection module 18. The light detection module 18 is connected to the image acquisition module 14. The light detection module 18 receives and stores the input brightness standard value. When the image acquisition module 14 controls the left 3d module 5 to perform image acquisition, the light detection module 18 adds a left label to the currently acquired 2d image and inputs the 2d image into the convolutional neural network model. When the image acquisition module 14 controls the right 3d module 6 to perform image acquisition, the light detection module 18 adds a right label to the currently acquired 2d image and inputs the 2d image into the convolutional neural network model. The convolutional neural network model segments the human face image in the 2d image, calculates the projection area of the raster in the normal state on the human face image, judges whether the received raster projection in the 2d image completely falls within the calculated projection area of the raster, and outputs the 2d image with the calculated projection area of the raster if the raster projection does not completely fall within the projection area of the raster.

[0074] When the human face image is acquired, the acquired image may not be qualified due to light collection, equipment failure and other problems. Since the shape of the raster changes when projected on different human faces, the raster cannot be simply detected and judged by the system. The present application simulates and calculates the raster projection area of the human face by using the AI large model, compares the raster projection area with the actually acquired image to judge whether the raster has a problem, and outputs a deviation image to facilitate user adjustment or repair.

[0075] The embodiments of the specific implementation are the preferred embodiments of the present application, but do not limit the protection scope of the present application, so that: any equivalent changes made according to the structure, shape, principle of the present application should be covered within the protection scope of the present application.

Claims

1. A low-cost, high-efficiency method for acquiring three-dimensional facial information, characterized in that, It also includes the following steps: Determine the shooting location (9), and set up three RGB cameras and two 3D modules; Perform joint calibration and device color correction on the RGB camera and 3D module, and obtain intrinsic and extrinsic parameters; 2D images are acquired using an RGB camera, and 3D point clouds are acquired using a 3D module. Preprocess 2D images and 3D point clouds to obtain triangular meshes; Project each triangular mesh onto each 2D image, and calculate the optimal viewing angle based on the projection of the triangular mesh onto the 2D image: A mask is drawn on the 2D image captured by the RGB camera (4), which is a frontal view. Set the overall objective function for: ； In the formula, The data items selected for the perspective. The penalty coefficient is the penalty term. Penalties for selecting different perspectives between adjacent faces; Calculate the selected data items from the perspective The formula is as follows: ; In the formula, P is the condition function of the projection surface. It is a triangular grid. for The corresponding view label for the triangular mesh, where m is the number of triangular meshes and n is the number of viewpoints, is calculated. The formula is as follows: ; In the formula, For another triangular mesh, for The view tag corresponding to the triangular grid. express and Adjacent, S represents the penalty function, Let the cost function be the cost function. This can be expressed as the following formula: ; In the formula, From a middle perspective, X is a constant value and is greater than the values ​​corresponding to the left and right view projection areas. If the projection area of ​​the triangular mesh falls completely into the mask area, the value corresponding to the middle view projection area is replaced with X, and the middle view is selected according to the formula. Textures are applied to the triangular mesh based on the selected viewpoint to obtain 3D facial information.

2. The low-cost, high-efficiency three-dimensional facial information acquisition system and method according to claim 1, characterized in that, The step "setting up three RGB cameras and two 3D modules" also includes: Set up a left RGB camera (2), a right RGB camera (3), a center RGB camera (4), a left 3D module (5), and a right 3D module (6); The step "preprocessing the 2D image and 3D point cloud" also includes: A semantic segmentation model is used to segment faces in each 2D image, and background point clouds of 3D point clouds are filtered out. Perform facial landmark recognition on each 2D image; The two processed 3D point clouds are merged to generate triangular mesh data.

3. The low-cost and high-efficiency three-dimensional facial information acquisition method according to claim 2, characterized in that, The step "using a semantic segmentation model to segment faces in each 2D image and filter out background point clouds from the 3D point cloud" also includes: A semantic segmentation model is used to segment faces in each 2D image and separate the background portion; The 3D point cloud is colored according to the segmentation results. The background part is colored as (0, 0, 0) in RGB. All point clouds with RGB coloring of (0, 0, 0) are deleted.

4. The low-cost and high-efficiency three-dimensional facial information acquisition method according to claim 2, characterized in that, The step "acquiring 2D images using an RGB camera and acquiring 3D point clouds using a 3D module" also includes: Set the interval time; Take intermittent shots when capturing 3D point clouds; The step "merging the two processed 3D point clouds and generating triangular mesh data" further includes: Preset the number of attempts and distance thresholds according to the accuracy requirements; Step 1: For each point in the target point cloud, find and determine its nearest point in the source point cloud to form multiple pairs of "target point - source point" nearest points; Step 2: Based on multiple sets of "target point - source point" nearest point pairs, construct an optimization objective function. The objective is to minimize the distance between each matching point in the target point cloud and the corresponding point in the source point cloud after transformation. By solving the optimization objective function, obtain the rigid body transformation matrix used for adjusting the position of the target point cloud. Step 3: Perform a rigid body transformation on the target point cloud using a rigid body transformation matrix to complete the position update; In the updated source and target point clouds, repeat step one and update the set of nearest point pairs. Repeat steps two and three until the number of iterations reaches the preset threshold, or the average distance of all nearest point pairs in the updated nearest point pair set is less than the preset distance threshold, then the iteration is complete. Point cloud registration is performed on the two 3D point clouds based on the source point cloud and the target point cloud after the iteration is completed.

5. The low-cost and high-efficiency three-dimensional facial information acquisition method according to claim 2, characterized in that, The step "projecting each triangular mesh onto each 2D image and calculating the optimal viewing angle based on the projection of the triangular mesh onto the 2D image" further includes: Obtain facial landmark information based on the identified facial landmarks; Project the triangular mesh onto each 2D image. If the triangular mesh has a projection that falls inside the mask, then the frontal view is taken as the best view for that triangular mesh. For the remaining triangular meshes that do not select the frontal view as the optimal viewpoint, the viewpoint corresponding to the 2D image with the largest projected area is selected as the optimal viewpoint.

6. The low-cost and high-efficiency three-dimensional facial information acquisition method according to claim 1, characterized in that, The step "texturing the triangular mesh according to the selected viewpoint" is followed by: Obtain a texture image, perform cylindrical texture unfolding on the texture image to obtain a cylindrical unfolded 3D model, and establish a coordinate system on the cylindrical unfolded 3D model; Set a projection plane, project the unfolded 3D model of the cylinder onto the projection plane, and calculate the projected coordinates of each point of the unfolded 3D model of the cylinder. The calculation formula is as follows: ； ； ； In the formula, Let x be the x-axis projection coordinates on the projection plane. Let be the y-axis projection coordinate on the projection plane, and r be the cylinder radius of the unfolded 3D model. θ is the simulated radius under the sigmoid function, x is the x-axis coordinate of the cylindrical unfolded 3D model in the coordinate system, y is the y-axis coordinate of the cylindrical unfolded 3D model in the coordinate system, z is the z-axis coordinate of the cylindrical unfolded 3D model in the coordinate system, and θ is the radian. Based on the calculated projection coordinates, the texture is reselected in the texture image to generate the triangular mesh.

7. A low-cost, high-efficiency three-dimensional facial information acquisition system, comprising the three-dimensional facial information acquisition method as described in claim 1, characterized in that: It also includes the shooting position (9), the left RGB camera (2), the right RGB camera (3), the center RGB camera (4), the left 3D module (5), the right 3D module (6) and the computing system (1); The computing system (1) performs joint calibration and device color correction on the RGB camera to obtain intrinsic and extrinsic parameters, acquires images, obtains multiple 2D images and multiple 3D point clouds, preprocesses the 2D images and multiple 3D point clouds to obtain triangular mesh data, projects each triangular mesh onto each 2D image, calculates the optimal viewing angle based on the projection of the triangular mesh onto the 2D image, applies texture to the triangular mesh based on the selected viewing angle, and obtains three-dimensional facial information.

8. A low-cost, high-efficiency three-dimensional facial information acquisition system according to claim 7, characterized in that: The computing system (1) includes an information storage module (11), a parameter calculation module (12), a device color calibration module (13), an image acquisition module (14), an image processing module (15), and an image fusion module (16); The information storage module (11) is pre-set with basic information of shooting position (9), left RGB camera (2), right RGB camera (3), center RGB camera (4), left 3D module (5) and right 3D module (6). The basic information includes camera data and position information. The parameter calculation module (12) calls the data stored in the information storage module (11) to perform joint calibration on the left RGB camera (2), right RGB camera (3), middle RGB camera (4), left 3D module (5) and right 3D module (6), and calculates the internal and external parameters of the left RGB camera (2), right RGB camera (3), middle RGB camera (4), left 3D module (5) and right 3D module (6), and transmits the internal and external parameters to the data storage module for storage; The device color calibration module (13) calls the camera data of the information storage module (11) to perform device color calibration on the left RGB camera (2), right RGB camera (3) and center RGB camera (4); The image acquisition module (14) controls the left RGB camera (2), right RGB camera (3), middle RGB camera (4), left 3D module (5) and right 3D module (6) to acquire images, obtain multiple 2D images and multiple 3D point clouds, and transmit the 2D images and 3D point clouds to the image processing module (15). The image processing module (15) uses a semantic segmentation model to segment faces in each 2D image, filters out background point clouds, performs facial key point recognition on each 2D image, and transmits the processed 2D image and 3D point cloud to the image fusion module (16). The image fusion module (16) merges the two processed 3D point clouds and generates triangular mesh data. For each triangular mesh, the best viewpoint is selected in each 2D image, and texture is applied to the triangular mesh according to the selected viewpoint to obtain three-dimensional facial information.

9. A low-cost, high-efficiency three-dimensional facial information acquisition system according to claim 8, characterized in that: The image acquisition module (14) is preset with an interval time, and controls the 3D module to take 3D point cloud images at intervals according to the interval time. The image fusion module (16) presets a number threshold and a distance threshold. The image fusion module (16) performs point cloud registration on two 3D point clouds. Step 1: For each point in the target point cloud, find and determine its corresponding nearest point in the source point cloud to form multiple sets of "target point-source point" nearest point pairs. Step 2: Based on multiple sets of "target point-source point" nearest point pairs, construct an optimization objective function. The objective is to minimize the distance between each matching point in the target point cloud and the corresponding point in the source point cloud after transformation. By solving the optimization objective function, obtain the rigid body transformation matrix used for target point cloud position adjustment. Step 3: Use the rigid body transformation matrix to perform rigid body transformation on the target point cloud as a whole to complete the position update. In the source point cloud and target point cloud after the position update, repeat step 1 and update the nearest point pair set. Repeat steps 2 and 3 until the number of iterations reaches the preset number threshold, or the average distance of all nearest point pairs in the updated nearest point pair set is less than the preset distance threshold, and complete the iteration. Perform point cloud registration on the two 3D point clouds based on the source point cloud and target point cloud after the iteration.

10. A low-cost, high-efficiency three-dimensional facial information acquisition system according to claim 8, characterized in that: The left 3D module (5) and the right 3D module (6) each include two infrared cameras (51) and an infrared grating projection device (52). The computing system (1) also includes a model generation module (17) and a light detection module (18). The model generation module (17) receives the grating images projected onto the face by the infrared grating projection device (52) captured by the left RGB camera (2), the right RGB camera (3) and the middle RGB camera (4). The grating images captured at the same time are taken as a group of grating images. The grating images in the same group are labeled with left, right or middle labels according to the RGB camera. The grating images are labeled with left or right labels according to the 3D module corresponding to the infrared grating projection device (52) in the grating image group. The stored grating images are used to train the convolutional neural network model. The trained convolutional neural network model is transmitted to the light detection module (18). The light-collecting detection module (18) is connected to the image acquisition module (14). The light-collecting detection module (18) receives and stores the input brightness standard value. When the image acquisition module (14) controls the left 3D module (5) to perform image acquisition, the light-collecting detection module (18) adds a left label to the currently acquired 2D image and inputs it into the convolutional neural network model. When the image acquisition module (14) controls the right 3D module (6) to perform image acquisition, the light-collecting detection module (18) adds a right label to the currently acquired 2D image and inputs it into the convolutional neural network model. The convolutional neural network model segments the face image in the 2D image, calculates the projection area of ​​the grating in the face image under normal conditions, and judges whether the grating projection in the received 2D image completely falls within the calculated grating projection area based on the calculation result. If it does not completely fall within the grating projection area, the 2D image with the calculated grating projection area is output.

Images