Face scanning model head completion method, three-dimensional scanning device and electronic equipment

Abstract

The invention provides a face scanning model head completion method, a three-dimensional scanning device and electronic equipment. The method comprises the following steps: acquiring a key feature point set of a face scanning model and a basic head model; registering the face scanning model and the basic head model based on the key feature point set to obtain a registered head model; extracting a face region in the face scanning model to obtain a to-be-stitched face model; the registered head model is deformed, so that the face area of the registered head model is aligned with the to-be-stitched face model, and a deformed head model is obtained; and stitching the to-be-stitched face model and the deformed head model to obtain a complemented full-head model. According to the method, the face area of the face scanning model is segmented firstly, then the basic head model is deformed and sewn with the face area of the face scanning model, quasi-seamless connection of the basic head model and the face scanning model is achieved, the fidelity of an original scanning model is improved, and user experience is improved.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and more specifically, to a method for completing the head of a face scanning model, a 3D scanning device, and an electronic device. Background Technology

[0002] With the widespread application of 3D face model construction based on acquired facial images in many fields, such as animation and game development, the requirements for face scan model completion are becoming increasingly stringent.

[0003] However, current methods typically involve non-rigid registration of the base head model and the face scan model, followed by segmentation of the facial features from the face scan model and replacement of those features with the corresponding facial features in the base head model. This registration method only uses local facial feature information from the original scan data, sacrificing model fidelity. This results in significant discrepancies between the completed head model's facial region and the face scan model, negatively impacting the user experience. Summary of the Invention

[0004] This application provides a method for completing the head of a face scanning model, a 3D scanning device, an electronic device, and a storage medium. By first extracting the facial region of the face scanning model and then deforming the basic head model to align with the boundary of the facial region of the face scanning model, the boundary between the basic head model and the face scanning model is fitted, so as to completely preserve the face scanning model data and improve the fidelity of the face scanning model.

[0005] Firstly, a method for completing the head of a face scanning model is provided. The method includes: obtaining a set of key feature points of a face scanning model and a basic head model; registering the face scanning model and the basic head model based on the set of key feature points to obtain a registered head model; extracting the facial region from the face scanning model to obtain a face model to be stitched; deforming the registered head model to align the facial region of the registered head model with the face model to be stitched to obtain a deformed head model; and stitching the face model to be stitched and the deformed head model together to obtain a completed full-head model.

[0006] In this embodiment, key feature point sets of the face scanning model and the basic head model are extracted to provide millimeter-level geometric anchor points for subsequent alignment, deformation, and stitching operations, avoiding errors. Based on the key feature point sets, the face scanning model and the basic head model are coarsely registered, which can eliminate translation, rotation, and scale differences and ensure accurate alignment between the face scanning model and the basic head model. The facial region in the face scanning model is extracted to preserve as much facial information as possible. The registered head model is deformed to align with the facial region in the scanning model, achieving a near-seamless connection between the basic head model and the face scanning model. Stitching is performed using the face model to be stitched and the deformed head model, which can provide a near-seamless completed full-head model while preserving as much scanning model data as possible.

[0007] In conjunction with the first aspect, in some implementations of the first aspect, obtaining the key feature point set of the face scanning model and the basic head model includes: obtaining the face scanning model and the basic head model, both of which include textureless meshed models indexed by point clouds and triangular faces; normalizing the face scanning model and the basic head model to obtain a first face scanning model and a first basic head model; and extracting the facial key feature point positions of the first face scanning model and the first basic head model respectively to obtain a key feature point set, which includes the first facial feature point set of the face scanning model and the second facial feature point set of the basic head model.

[0008] In this embodiment, by employing a textureless mesh model, storage space can be saved, computational costs reduced, and rendering time decreased, thereby improving efficiency. Furthermore, normalizing the data before extracting the key feature point sets of the face scanning model and the basic feature model facilitates subsequent data processing, improving processing efficiency and accuracy.

[0009] In conjunction with the first aspect, in certain implementations of the first aspect, extracting the facial key feature point positions of the first face scanning model and the first basic head model to obtain a key feature point set includes: sampling the first face scanning model and the first basic head model respectively to obtain a two-dimensional image of the first face scanning model and a two-dimensional image of the first basic head model; rendering the two-dimensional images to obtain a face scanning model rendering image and a basic head model rendering image respectively; and obtaining a key feature point set based on the two-dimensional face key point detection module, wherein the key feature point set includes a first facial feature point set of the face scanning model rendering image and a second facial feature point set of the basic head model rendering image.

[0010] In this embodiment, lighting information is added to the model through rendering, highlighting features such as facial skin color, shadows, and transmission in the two-dimensional image, which greatly improves the accuracy of subsequent face detection using deep learning.

[0011] In conjunction with the first aspect, in some implementations of the first aspect, the registration of the face scanning model and the basic head model based on the key feature point set to obtain the registered head model includes: based on the key feature point set, performing rigid registration and / or non-rigid registration between the basic head model and the face scanning model to transform the basic head model to the coordinate system of the face scanning model, thereby obtaining the registered head model.

[0012] In this embodiment, the basic head model and the face scanning model are rigidly registered, transforming the basic head model to the coordinate system of the face scanning model. This eliminates misalignment and ensures anatomical consistency based on a set of key feature points. Therefore, this approach facilitates subsequent operations, saves computational resources, and improves accuracy.

[0013] In conjunction with the first aspect, in some implementations of the first aspect, extracting the facial region from the face scanning model to obtain the face model to be stitched includes: segmenting the face scanning model based on a set of key feature points to extract the facial region from the face scanning model to obtain the face model to be stitched.

[0014] In this embodiment, the face scanning model is segmented using a first set of facial feature points, and the facial region is extracted to obtain the face model to be stitched. The segmented model has no intersecting triangular faces in three-dimensional space, which is beneficial for preserving the data of the face scanning model, improving the model fidelity, and facilitating the seamless stitching of the subsequent basic head model and face scanning model.

[0015] In conjunction with the first aspect, in some implementations of the first aspect, deforming the registered head model to align the facial region of the registered head model with the face model to be stitched, thereby obtaining a deformed head model, includes: deforming the registered head model based on a set of key feature points to align the facial region of the registered head model with the face model to be stitched, thereby obtaining a deformed head model.

[0016] In this embodiment, the registered head model is further deformed to align with the boundary of the face model to be stitched, which helps to achieve seamless connection between the basic head model and the face scanning model.

[0017] In conjunction with the first aspect, in some implementations of the first aspect, the deformation process includes: deforming the facial boundary region of the registered head model so that the boundary of the facial region of the registered head model is aligned with the boundary of the face model to be stitched, thereby obtaining a deformed head model.

[0018] In this embodiment, mesh deformation technology is used to deform the facial boundary region of the basic head model, which can further align the facial region of the basic head model with the facial region boundary of the face model to be stitched. This is beneficial for achieving near-seamless head completion of the face scanning model and improves the user experience.

[0019] In conjunction with the first aspect, in some implementations of the first aspect, the process of stitching together the face model to be stitched and the deformed head model to obtain the completed full-head model includes: segmenting the facial region of the deformed head model to separate the facial region of the deformed head model to obtain the head model to be stitched; stitching the face model to be stitched together with the head model to be stitched to obtain the stitched model; and smoothing the seam area of ​​the stitched model to obtain the completed full-head model.

[0020] In this application, by smoothing the mesh of the stitching model, a smooth fusion between the face model to be stitched and the head model to be stitched is achieved, ensuring the local smoothness and continuity of the seam area, and avoiding affecting the mesh accuracy of other areas besides the seam area, thereby improving the completion effect of the full head model and enhancing the user experience.

[0021] Secondly, a three-dimensional scanning device is provided, comprising: an acquisition unit for acquiring a face scanning model and a basic head model; and a processing unit for: acquiring a set of key feature points based on the face scanning model and the basic head model; registering the face scanning model and the basic head model based on the set of key feature points to obtain a registered head model; extracting the facial region from the face scanning model to obtain a face model to be stitched; deforming the registered head model to align the facial region of the registered head model with the face model to be stitched to obtain a deformed head model; and stitching the face model to be stitched and the deformed head model together to obtain a completed full-head model.

[0022] In conjunction with the second aspect, in some implementations of the second aspect, the three-dimensional scanning device further includes a sampling unit for sampling the face scanning model and the basic head model respectively to obtain their two-dimensional images.

[0023] Thirdly, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method of the first aspect.

[0024] Fourthly, a computer-readable storage medium is provided, which stores program code that, when executed on a computer, causes the computer to perform the face scanning model head completion method as described in the first aspect. Attached Figure Description

[0025] Figure 1This is an application environment diagram of a face scanning model head completion method provided in an embodiment of this application; Figure 2 This is a schematic flowchart illustrating a method for head completion of a face scanning model provided in an embodiment of this application; Figure 3 This is an architecture diagram of a first processing layer provided in an embodiment of this application; Figure 4 This is an architecture diagram of a second processing layer provided in an embodiment of this application; Figure 5 This is a schematic diagram of a pre-suture procedure provided in an embodiment of this application; Figure 6 This is a schematic diagram of a sutured garment provided in an embodiment of this application; Figure 7 This is a schematic diagram of the structure of a device provided in an embodiment of this application; Figure 8 This is a schematic diagram of the structure of another electronic device provided in an embodiment of this application. Detailed Implementation

[0026] The technical solutions in this application will now be described with reference to the accompanying drawings.

[0027] The use of prefixes such as "first" and "second" in this application embodiment is solely for distinguishing different descriptive objects and does not limit the position, order, priority, quantity, or content of the described objects. The use of ordinal numbers and other prefixes to distinguish descriptive objects in this application embodiment does not constitute a limitation on the described objects. The description of the described objects is found in the claims or the context of the embodiments, and the use of such prefixes should not constitute unnecessary restrictions.

[0028] Figure 1 This is an application environment diagram of a face scanning model head completion method provided in an embodiment of this application. It should be understood that... Figure 1 The descriptions and related information are merely examples and do not limit the application environment of the embodiments in this application.

[0029] Reference Figure 1 A face scanning model head completion method is applied to a face scanning model head completion system. This face scanning model head completion system includes an acquisition device 110 and a processing device 120. The acquisition device 110 and the processing device 120 can be integrated into the same device, or they can be set up independently, or the processing device 120 can be integrated into the acquisition device 110. The acquisition device 110 and the processing device 120 can be connected via wired or wireless means. Specifically, the acquisition device 110 can be any device capable of outputting depth maps and / or color maps, such as a 3D camera or 3D sensor, for example, an RGBD device.

[0030] The processing device 120 can be implemented using a single processing unit or a cluster of processing units. The processing unit may include integrated circuit devices with signal processing capabilities.

[0031] For example, the processing device may include an integrated circuit device with instruction read and execute capabilities, such as a central processing unit (CPU), a microprocessor, a graphics processing unit (GPU) (which can be understood as a type of microprocessor), or a digital signal processor (DSP).

[0032] For example, the processing device can also implement certain functions through the logical relationships of hardware circuits. These logical relationships can be fixed or reconfigurable. For instance, the processing device can be a hardware circuit implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD), such as a field-programmable gate array (FPGA). In a reconfigurable hardware circuit, the process of the processor loading a configuration document and configuring the hardware circuit can be understood as the processor loading instructions to implement the functions of some or all of the above units. Furthermore, the processing device 120 can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as a neural network processing unit (NPU), tensor processing unit (TPU), or deep learning processing unit (DPU). In addition, the system can also include a memory for storing instructions. The processing device 120 can call the instructions in the memory to implement the corresponding functions.

[0033] Facial head completion is based on the geometric and topological information of the human body's external structure. It utilizes digital data generated through 3D scanning technology, point cloud processing, and shape modeling. This data primarily includes: geometric shape (the precise 3D shape of the human body surface); topological structure (a meshed representation of the human body surface, such as triangular or quadrilateral meshes); human feature points (markers of key anatomical locations); and surface attributes (potentially surface texture or physical properties). Generally, 3D scanning equipment (such as laser scanning, structured light scanning, or depth cameras) is used to acquire facial data and generate a point cloud model. Then, a template matching method is used, employing a predefined, general-purpose human head model for deformation fitting.

[0034] Current techniques typically involve non-rigid registration between a base head model and a facial scan model, followed by segmentation of the facial features from the facial scan model and replacement of those features with the corresponding facial features in the base head model. While this method is simple to perform, it struggles to accurately capture individual-specific anatomical features, relying solely on localized information from the original scan data and sacrificing some model fidelity. This results in significant discrepancies between the completed head model's facial region and the facial scan model, negatively impacting the user experience.

[0035] Furthermore, embodiments of this application provide a method, apparatus, and storage medium for completing the head of a face scanning model, which can more accurately complete the head of a face scanning model and provide a near-seamless method for completing the head of a face scanning model while preserving as much face scanning model data as possible.

[0036] Figure 2 This is a schematic flowchart of a face scanning model head completion method 200 provided in an embodiment of this application, as shown below. Figure 2 As shown, method 200 includes: S210, Obtain the key feature point set of the face scanning model and the basic head model.

[0037] In some embodiments, step S210 includes: first, inputting textureless mesh models of the face scan model and the basic head model, which have been processed by the reconstruction algorithm respectively. Both the face scan model and the basic head model are meshed models output by a 3D scanner, containing point clouds and triangular face indices.

[0038] Specifically, the raw sensor data of the scanned face and basic head are converted into a meshed model including point cloud and triangular face index by the 3D reconstruction algorithm in the 3D scanner, and output as the face scanning model and the basic head model respectively. Then, the face scanning model and the basic head model are subjected to mesh post-processing, including processes such as denoising, normal reconstruction, surface reconstruction, topology repair, and detexturing, to obtain a textureless mesh model.

[0039] In some embodiments, step S210 includes: normalizing the face scanning model and the basic head model respectively, that is, linearly transforming the data to the interval [0,1] to obtain the first face scanning model and the first basic head model. Normalization can be specifically calculated using the following formula: x'=(x-min) / (max-min) Where x' is the normalized data value; x is the original data value of the model; min is the minimum value of all data in the corresponding model; and max is the maximum value of all data in the corresponding model.

[0040] In the above embodiments, normalizing the face scanning model and the basic head model respectively can eliminate persistent differences, which is beneficial for subsequent coordinate system transformation and completion operations, and can also reduce the amount of computation. Therefore, normalization helps to improve the accuracy and efficiency of calculation.

[0041] In some embodiments, step S210 includes: sampling the textureless mesh model of the first face scanning model and the textureless mesh model of the first basic head model from all directions using a spherical camera to obtain a two-dimensional image of the first face scanning model and a two-dimensional image of the first basic head model. The two-dimensional images of the first face scanning model and the first basic head model are rendered using a rendering system to obtain the face scanning model rendering image and the basic head model rendering image. Based on the two-dimensional face key point detection module, the face scanning model rendering map and the basic head model rendering map are detected respectively to obtain the two-dimensional facial feature points in the face scanning model rendering map and the two-dimensional facial feature points in the basic head model rendering map. Based on the projection of light rays from the two-dimensional facial feature point positions, the two-dimensional facial feature points in the face scan model rendering and the two-dimensional facial feature points in the basic head model rendering are restored to three-dimensional space using the ray tracing back projection method, respectively, to obtain the three-dimensional facial feature point positions, that is, the facial feature point set of the face scan model and the facial feature point set of the basic head model.

[0042] In the embodiments of this application, a textureless mesh model is used, which helps to save storage space, reduce computational costs, reduce rendering time, and thus improve efficiency.

[0043] In the embodiments of this application, "performing omnidirectional sampling of the textureless mesh models of the first face scanning model and the textureless mesh models of the first basic head model respectively" can be a completely virtual simulation process performed in a computer. The "spherical camera" can be a virtual, idealized mathematical model, and its design simulates the imaging principle of a real spherical camera. "Omnidirectional" can be a 360° surround in the horizontal direction (longitude) and / or a 180° span in the vertical direction (latitude).

[0044] For example, the rendering system integrates visual enhancement techniques such as physically based rendering (PBR), screen-space ambient occlusion (SSAO), shadow mapping (SM), and sub-surface scattering (SSS) to support high-quality image generation and subsequent feature analysis. The rendering system adds lighting information to the model, highlighting features such as facial skin color, shadows, and transmission in the 2D image, greatly improving the accuracy of subsequent face detection using deep learning.

[0045] Optionally, the facial landmark detection module can use 128×128 pixel RGB input. The specific processing procedure is shown in Table 1. A 5×5 convolutional kernel is halved in size by sliding with a stride of 2. At the same time, the three color channels are converted into several feature channels required by subsequent layers of the network, providing initial feature maps for the lightweight feature extraction unit (BlazeBlock). The initial feature maps are then input into 5 single BlazeBlocks (i.e., the first processing layer) and 6 double BlazeBlocks (i.e., the second processing layer), where the maximum number of channels in each layer is 96, and the minimum spatial resolution of the feature maps is 8×8.

[0046] Figure 3 This is an architecture diagram of the first processing layer in an embodiment of this application. Figure 3As shown, the initial input feature map is grouped using depthwise separable convolutional layers, and each convolutional kernel is also grouped accordingly. Convolution is performed within each group to extract features, thus reducing the number of parameters and computational cost. The number of groups can be equal to the number of initial input feature maps. For example, if the number of feature maps is N, then the number of groups is N, and the number of parameters after grouping becomes 1 / N of the number of ungrouped parameters. The number of channels in the convolutional layer can be adjusted to prepare for residual connections. Max pooling layers and channel padding layers can be used selectively. Max pooling layers perform sampling operations, typically with a stride of 2, to reduce the spatial size of the feature map, and / or channel padding layers add a certain number of rows and columns to each side of the initial input feature map, making the output and input feature map sizes the same. After each convolution, the size of the original image decreases, making it difficult to set more layers in the neural network, thus failing to achieve optimal computational results. Setting channel padding layers ensures that the size of the feature map remains unchanged after convolution, thus avoiding limitations on the design of the neural network. The number of layers can be set according to requirements to obtain ideal computational results. Finally, the feature maps that have undergone convolution and pooling operations are summed to form the first feature map.

[0047] Figure 4 This is an architecture diagram of the second processing layer in an embodiment of this application. Figure 4 As shown, unlike the first processing layer, two depthwise separation convolution operations are performed to obtain a larger receptive field. After pointwise convolutional layers and pointwise convolutional expansion layers, the number of channels can be adjusted to match the number of channels in subsequent residual connections, resulting in the second feature map, which corresponds to the two-dimensional facial feature points of the model.

[0048] The BlazeFace feature extraction network, composed of a single BlazeBlock and two BlazeBlocks, terminates when the feature map is scaled down to 8×8, which helps to ensure that the feature map reaches a specific resolution while avoiding additional computation.

[0049] This rendering system integrates a deep learning-based facial landmark detection module, which can automatically identify 2D facial feature points in multi-view rendered images. Specifically, it acquires the coordinates and pose transformation information of feature points located at key facial positions in the input 2D image. Facial landmarks can be located at facial boundaries. In addition, they can also be located at key facial positions such as the center of the eyes, earlobes, center of the mouth, and tip of the nose. If no valid facial features are identified in the scanned face model, the deep learning-based neural network will provide a confidence score for the face detection. If this score is lower than a preset threshold, the system will terminate subsequent calculations to ensure the effectiveness and stability of the processing.

[0050] Table 1

[0051] In the above embodiments, the textureless network model is reduced in dimension by a spherical camera to obtain a two-dimensional image, which helps to reduce computational complexity. The key points on the two-dimensional rendering image are projected back into the three-dimensional space, thereby obtaining the facial feature points of the two models in the three-dimensional space, providing high-precision anchor points for subsequent three-dimensional registration, completion and completion.

[0052] In some embodiments, step S210 further includes: unifying the scale of the facial regions of the face scanning model and the basic head model based on the facial feature point set of the face scanning model and the facial feature point set of the basic head model. The unification of the scale of the facial regions of the face scanning model and the basic head model can be achieved through similar transformations such as translation, rotation, and scaling. By unifying the scale of the facial regions of the face scanning model and the basic head model, both models can be fully displayed in the camera viewport for subsequent processing. For ease of description, the facial feature point set of the face scanning model with unified scale is referred to as the first facial feature point set, and the facial feature point set of the basic head model is referred to as the second facial feature point set.

[0053] Specifically, firstly, a similarity transformation matrix is ​​obtained between the facial feature point set of the face scanning model and the facial feature point set of the basic head model. Then, the basic head model is transformed to a similar scale based on the face scanning model using the similarity transformation matrix, thereby achieving scale unification.

[0054] S220, based on the key feature point set, registers the face scanning model and the basic head model to obtain the registered head model.

[0055] In some embodiments, in step S220, based on the first and second facial feature point sets, an Iterative Closest Point (ICP) algorithm can be used for precise matching to solve for the optimal rigid registration parameters for rotation, translation, and scaling. According to the rigid registration parameters, rigid registration is performed on the base head model and the face scan model, transforming the base head model to the coordinate system of the face scan model to obtain the registered head model. This achieves rigid alignment between the base head model and the face scan model while ensuring the consistency of the overall orientation of the models.

[0056] Specifically, if the face scan model and the base head model have matching 3D feature points, and the distance iteration error is less than a preset threshold, then alignment is considered achieved. After registration, the normal directions and topology of the face scan model and the base head model will be updated synchronously to maintain the continuity of the geometric structure and visual consistency in subsequent rendering processes.

[0057] In the above embodiments, rigid registration can reduce translation and rotation errors, and after rough alignment, the processing time for non-rigid registration can be reduced, which is beneficial to improving efficiency.

[0058] In the embodiments of this application, registration can be divided into rigid registration and non-rigid registration depending on whether the object to be registered has undergone deformation.

[0059] S230, extract the facial region from the face scan model to obtain the face model to be stitched.

[0060] In some embodiments, step S230 includes: segmenting the face scan model using a first set of facial feature points to obtain a face model to be stitched. The segmented face model to be stitched has no intersecting triangular faces in three-dimensional space.

[0061] S240, deform the registered head model so that the facial area of ​​the registered head model is aligned with the face model to be stitched, thus obtaining the deformed head model.

[0062] In some embodiments, step S240 includes: registering the registered head model obtained in step S220 onto the face model to be stitched using a non-rigid registration algorithm to obtain a non-rigid registered head model.

[0063] Specifically, non-rigid registration is similar to rigid registration in step S220. While rigid registration emphasizes overall pose transformation, non-rigid registration only performs pose transformation locally on the outer contour of the face region. Non-rigid registration involves performing high-order singular value decomposition on the first and second facial feature point sets to be registered, followed by optimization using alternating least squares to solve for the pose transformation parameters. Then, the head model to be registered is processed based on the pose transformation parameters to obtain the non-rigidly registered head model, thus achieving accurate matching between the point clouds of the face model to be stitched and the non-rigidly registered head model. Simultaneously, the second facial feature point set is processed based on the pose transformation parameters to obtain the facial feature point set of the non-rigidly registered head model, facilitating subsequent deformation processing.

[0064] In some embodiments, step S240 further includes: deforming the facial boundary region of the non-rigidly registered head model using a mesh deformation method, so that the boundary of the facial region of the non-rigidly registered head model is further aligned with the boundary of the face model to be stitched. Then, based on the facial feature points of the mesh-deformed head model, the face region mesh in the mesh-deformed head model is segmented and removed to obtain the head model to be stitched.

[0065] For example, the mesh deformation method specifically involves: selecting a radial basis function and obtaining an interpolation function composed of a linear combination of the radial basis functions based on the first set of facial feature points. The radial basis function can be a Gaussian function. The offset of the facial feature point set of the non-rigidly registered head model is solved using the interpolation function. Based on the offset, the non-rigidly registered head model is processed, further deforming the face region of the non-rigidly registered head model to align with the boundary of the face model to be stitched, facilitating a smooth connection between the non-rigidly registered head model and the face model to be stitched. The mesh deformation method relies on control point design. Simultaneously, the facial feature points of the non-rigidly registered head model are processed based on the offset to obtain the facial feature points of the mesh-deformed head model. In the mesh deformation, the first facial feature points and the facial feature points of the non-rigidly registered head model are used as 3D control point information, and some 3D control points are kept fixed to ensure the head visualization effect.

[0066] Optionally, the registered head model can be deformed using only high-precision non-rigid registration to obtain a deformed head model, or non-rigid registration can be used to first obtain a non-rigid registered head model, and then mesh deformation processing can be performed to obtain a deformed head model. This application does not impose any restrictions on this.

[0067] In the above embodiment, after the processing of the first four steps, the face scanning model and the basic head model have respectively completed the extraction of the face region mesh of the face scanning model and the stripping of the face region mesh of the basic head model. At this time, the face scanning model retains only the facial feature regions to obtain the face model to be stitched, while the basic head model has removed the corresponding facial regions to obtain the head model to be stitched. The two have achieved uniform size and consistent orientation in three-dimensional space and are in a non-overlapping state to be stitched.

[0068] S250, using the face model to be stitched and the deformed head model to stitch together, to obtain the completed full-head model.

[0069] In some embodiments, step S250 includes: First, extracting the boundary loops of the face model and head model to be stitched, and cleaning up the degenerate surfaces and isolated points of the boundary loops to ensure the stability of the topology. Next, performing smoothing, resampling, and topology unification operations on the boundary loops to eliminate the interference of jagged contours on subsequent bridging. Finally, uniformly adding sampling points in the stitching region between the two boundary loops, generating multiple triangular patches based on the sampling point set, and ensuring that the circumcircle of the generated triangular patches (excluding the boundary) does not contain any points from the sampling point set. This setup generates structurally sound and high-quality bridging patches, thereby achieving smooth fusion between the face model and head model to be stitched.

[0070] Figure 5 This is a schematic diagram of the pre-suture stage in an embodiment of this application. Figure 6 This is a schematic diagram showing the stitched result in an embodiment of this application. Figure 5 and Figure 6 As shown, the sutured boundary ring 300 is relatively smooth, indicating that the above treatment is beneficial to improving the fusion effect between the face model and the head model to be sutured.

[0071] In some embodiments, step S250 further includes: applying mask-controlled mesh smoothing to the seam area of ​​the obtained stitched model, combined with normal guidance, to improve the geometric continuity and visual consistency at the fusion point. Each triangular face has a face normal, which can determine the orientation of nearby faces to calculate curvature, thereby ensuring local smoothness and continuity without affecting the mesh accuracy at other locations besides the seam area.

[0072] For example, in mesh smoothing, to avoid affecting the overall geometric accuracy of the model, mask control can be applied only to the seam area near the boundary ring, thus limiting the smoothing operation to the seam transition area.

[0073] Specifically, mesh smoothing improves the surface consistency and visual naturalness of the seam transition area by applying a mask to the seam region and then applying a geometric filter, such as Laplacian smoothing, within the masked area. This ultimately achieves a smooth integration and overall harmony in the model's appearance. Then, a new normal is calculated using a formula, and the vertices of the triangular facets are adjusted based on this new normal to make the normals of the triangular facets as close as possible to the adjusted normals. The formula for calculating the new normal is as follows:

[0074] In the above formula, k represents the number of iterations, which can be adjusted as needed; Let i be the i-th triangular facet; This represents the normal to the i-th face after k iterations; This represents the center of the i-th triangular facet. Represents all adjacent faces of the i-th face; Let represent the area of ​​the j-th face; , where x is a vector Length, It is an adjustable parameter. The value is the average of the center differences of all adjacent faces; , where y is a vector Length, It is an adjustable parameter, with a value of 1.

[0075] Finally, adjust the vertices of the triangular facets. You can use a single-step Gaussian iteration method, adjusting only one vertex at a time, so that the normal is as close as possible to the calculated normal, and you will get the completed full-head model.

[0076] It should be understood that, in the various embodiments of this application, unless otherwise specified or in case of logical conflict, the terms and / or descriptions between the various embodiments are consistent and can be referenced by each other, and the technical features in different embodiments can be combined to form new embodiments according to their inherent logical relationships.

[0077] This application also provides an apparatus for implementing any of the above methods, the apparatus including units for implementing the steps performed in any of the above methods.

[0078] Figure 7 This is a schematic diagram of a three-dimensional scanning device 800 provided in an embodiment of this application. The device 800 may include an acquisition unit 810 and a processing unit 820. The acquisition unit 810 is used to acquire instructions and / or data, and the acquisition unit 810 may also be referred to as a communication interface or communication unit; the processing unit 820 is used to perform data processing so that the device 800 implements the aforementioned head completion method.

[0079] In one embodiment, the apparatus 800 includes: an acquisition unit 810 and a processing unit 820. The acquisition unit 810 is used to acquire a face scanning model and a basic head model. The processing unit 820 is used to: acquire a set of key feature points based on the face scanning model and the basic head model; register the face scanning model and the basic head model based on the set of key feature points to obtain a registered head model; extract the facial region from the face scanning model to obtain a face model to be stitched; deform the registered head model so that the facial region of the registered head model is aligned with the face model to be stitched to obtain a deformed head model; and stitch the face model to be stitched and the deformed head model to obtain a completed full-head model.

[0080] Optionally, the processing unit 820 described above may be Figure 1 The processing device 120 shown, the acquisition unit 810 can be Figure 1 The data acquisition device 110 shown.

[0081] In some embodiments, the apparatus further includes a sampling unit for sampling the face scanning model and the basic head model respectively to obtain their two-dimensional images.

[0082] Figure 8This is a schematic diagram of an electronic device 900 provided in an embodiment of this application. The electronic device 900 includes a memory 910, a processor 920, and a communication interface 930. The memory 910, processor 920, and communication interface 930 are connected via an internal connection path. The memory 910 stores instructions, and the processor 920 executes the instructions stored in the memory 910 to control the communication interface 930 to acquire information, or to enable the electronic device 900 to implement the aforementioned face scanning model head completion method. Optionally, the memory 910 can be coupled to the processor 920 via an interface, or it can be integrated with the processor 920.

[0083] It should be noted that the communication interface 930 described above uses a transceiver device, such as, but not limited to, a transceiver. The communication interface 930 may also include an input / output interface.

[0084] The processor 920 stores one or more computer programs, which include instructions. When the instructions are executed by the processor 920, the electronic device 900 performs the completion methods described in the above embodiments.

[0085] In implementation, each step of the above method can be completed by the integrated logic circuitry of the hardware in the processor 920 or by instructions in software form. The method disclosed in the embodiments of this application can be directly implemented by the hardware processor, or by a combination of hardware and software modules in the processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory 910, and the processor 920 reads the information in memory 910 and, in conjunction with its hardware, completes the steps of the above method. To avoid repetition, detailed descriptions are not provided here.

[0086] Optionally, Figure 8 The communication interface 930 in the middle can achieve Figure 7 The acquisition unit 810 in the middle, Figure 8 The processor 920 in the middle can achieve Figure 7 The processing unit 820 in the middle.

[0087] This application also provides a computer-readable storage medium storing program code that, when executed on a computer, causes the computer to perform the above-described... Figure 2 The method in the middle.

[0088] This application also provides a computer program product, which includes a computer program that, when run, causes the computer to perform the above-described actions. Figure 2 The method in the middle.

[0089] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0090] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0091] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0092] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0093] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0094] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0095] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for head completion in a face scanning model, characterized in that, The method includes: Obtain the key feature point set of the face scanning model and the basic head model; Based on the set of key feature points, the face scanning model and the basic head model are registered to obtain a registered head model. Extract the facial region from the face scanning model to obtain the face model to be stitched; The registered head model is deformed so that the facial region of the registered head model is aligned with the face model to be stitched, thus obtaining a deformed head model; The face model to be stitched and the deformed head model are stitched together to obtain the completed full-head model.

2. The method according to claim 1, characterized in that, The key feature point set for obtaining the face scanning model and the basic head model includes: Obtain the face scanning model and the basic head model, both of which include textureless meshed models indexed by point clouds and triangular faces; The face scanning model and the basic head model are normalized to obtain the first face scanning model and the first basic head model. The locations of key facial feature points are extracted from the first face scanning model and the first basic head model respectively to obtain the key feature point set, which includes the first facial feature point set of the face scanning model and the second facial feature point set of the basic head model.

3. The method according to claim 2, characterized in that, The step of extracting the locations of key facial feature points from the first face scanning model and the first basic head model to obtain the key feature point set includes: The first face scanning model and the first basic head model are sampled respectively to obtain a two-dimensional image of the first face scanning model and a two-dimensional image of the first basic head model. The two-dimensional image is rendered to obtain a face scan model rendering image and a basic head model rendering image, respectively. Based on the two-dimensional face key point detection module, the key feature point set is obtained. The key feature point set includes the first facial feature point set of the face scan model rendering map and the second facial feature point set of the basic head model rendering map.

4. The method according to any one of claims 1 to 3, characterized in that, The process of registering the face scanning model and the basic head model based on the key feature point set to obtain the registered head model includes: Based on the set of key feature points, the basic head model and the face scanning model are rigidly registered and / or non-rigidly registered to transform the basic head model into the coordinate system of the face scanning model, thereby obtaining the registered head model.

5. The method according to any one of claims 1 to 4, characterized in that, The step of extracting the facial region from the face scan model to obtain the face model to be stitched includes: Based on the set of key feature points, the face scanning model is segmented to extract the facial region in the face scanning model, thereby obtaining the face model to be stitched.

6. The method according to any one of claims 1 to 5, characterized in that, The step of deforming the registered head model to align the facial region of the registered head model with the face model to be stitched, resulting in a deformed head model, includes: Based on the set of key feature points, the registered head model is deformed so that the facial region of the registered head model is aligned with the face model to be stitched, thus obtaining the deformed head model.

7. The method according to claim 6, characterized in that, The deformation process includes: The facial boundary region of the registered head model is deformed so that the boundary of the facial region of the registered head model is aligned with the boundary of the face model to be stitched, thus obtaining the deformed head model.

8. The method according to any one of claims 1 to 7, characterized in that, The process of stitching together the face model to be stitched and the deformed head model to obtain the completed full-head model includes: The facial region of the deformed head model is segmented to peel off the facial region of the deformed head model, resulting in a head model to be stitched. The face model to be stitched together with the head model to be stitched together are stitched together to obtain the stitched model. The seam area of ​​the stitched model is smoothed to obtain the completed full-head model.

9. A three-dimensional scanning device, characterized in that, The device includes: The acquisition unit is used to acquire the face scanning model and the basic head model; The processing unit is configured to: obtain a set of key feature points based on the face scanning model and the basic head model; register the face scanning model and the basic head model based on the set of key feature points to obtain a registered head model; extract the facial region from the face scanning model to obtain a face model to be stitched; deform the registered head model to align the facial region of the registered head model with the face model to be stitched to obtain a deformed head model; and stitch the face model to be stitched and the deformed head model together to obtain a completed full-head model.

10. The three-dimensional scanning device according to claim 9, characterized in that, The three-dimensional scanning device also includes a sampling unit for sampling the face scanning model and the basic head model to obtain their two-dimensional images.

11. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the face scanning model head completion method as described in any one of claims 1 to 8.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores program code that, when executed on a computer, causes the computer to perform the face scanning model head completion method as described in any one of claims 1 to 8.

Images