Facial soft tissue defect point cloud completion method based on PF-Net

Abstract

The application discloses a face soft tissue defect point cloud completion method based on a PF-Net, and belongs to the technical field of face soft tissue defect completion. The method comprises the following steps: S1, a data preparation stage: a data enhancement algorithm is designed to obtain a paired data set of incomplete input and defect labels; S2, a model construction stage: a dynamic graph convolution and a boundary attention mechanism are introduced into an encoder to optimize a point cloud completion network model PF-Net; S3, a model training stage: a training set is input into the PF-Net model for training; S4, a model testing stage: a test set is input into a generator for testing; and S5, a model inference stage: for clinical face soft defect point cloud data, multi-scale input data are acquired and completion is performed. The application realizes autonomous simulation design of face soft tissue defects, solves technical barriers in existing digital prosthetic technology, such as dependence on healthy side mirror image data or standard organ models, and solves the problem of unstable repair effect caused by dependence on doctor experience and aesthetic literacy.

Description

Technical Field

[0001] This invention relates to the field of facial soft tissue defect repair technology, and more specifically, to a point cloud repair method for facial soft tissue defects based on PF-Net. Background Technology

[0002] Facial soft tissue defects are a complex type of deformity that severely impacts patients' quality of life, caused by various factors such as tumor resection, trauma, and congenital developmental abnormalities. Due to the unique anatomical morphology and tissue structure of the facial organs, traditional surgical methods are complex and often fail to achieve satisfactory repair results. While allogeneic transplantation can restore a patient's normal facial appearance, its widespread adoption is limited by issues such as immune rejection, medical ethics, and patient psychological factors. Therefore, facial prosthesis techniques are gradually becoming the mainstream choice for facial defect repair, which places higher demands on physicians' professional capabilities in the field of aesthetic restoration.

[0003] Facial prosthesis technology has achieved a complete digital transformation. Existing digital prosthesis technologies replace traditional processes with digital means, including digital impression acquisition, software design, and the determination of rapid production processes. However, the design method using mirrored flipping of healthy side data faces application challenges due to the lack of original morphological data. Some studies have attempted to alleviate this problem by establishing a three-dimensional morphological database of facial organs, but model adjustments still require manual operation. Currently, the digital design and production of facial prostheses mainly employs negative mold design and printing technology, which involves first designing the prosthesis in software, then creating a negative mold and printing it. The key to this technology lies in the software design stage, which relies on initial data for morphological design. This is not only limited by the physician's experience and aesthetic judgment but also makes it difficult to achieve reference-free prosthesis design, thus hindering the guarantee of stable repair results. Although digital technology has improved efficiency to some extent, it still falls short of meeting the rapid repair needs of complex soft tissue defects in the face.

[0004] In view of this, we propose a point cloud completion method for facial soft tissue defects based on PF-Net. Summary of the Invention

[0005] The purpose of this invention is to provide a point cloud completion method for facial soft tissue defects based on PF-Net, so as to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution: A point cloud completion method for facial soft tissue defects based on PF-Net includes the following steps: S1. Data preparation stage: Design data augmentation algorithms to obtain paired datasets with incomplete inputs and missing labels, and randomly divide them into training set, test set and validation set; S2, Model Building Stage: The point cloud completion network model PF-Net is optimized by introducing dynamic graph convolution and boundary attention mechanisms into the encoder; S3. Model Training Phase: Input the training set into the PF-Net model for training. During the model training process, use the validation set to evaluate the model performance and adjust the hyperparameters. After training is completed, extract the generator as the core model for subsequent inference. S4. Model Testing Phase: Input the test set into the generator to obtain the completed 3D point cloud, and calculate the chamfer distance from the label, accuracy, recall, and F1-Score to evaluate the model accuracy. S5. Model Inference Stage: For clinical facial soft defect point cloud data, the farthest point sampling technique is used to obtain multi-scale input data, and the generator completes the point cloud of the defect area.

[0007] Preferably, in step S1, the core of the data augmentation algorithm is to simulate facial soft tissue defects by cropping normal facial scan data, thereby obtaining paired data of incomplete input and defect labels. Specifically, two cropping methods are used: One method is feature sphere clipping, which involves using feature spheres with different center coordinates and radii to perform Boolean operations with a normal surface scan model; Another method is manual selection and cropping, which involves manually selecting areas on a normal facial scan model according to common clinical facial defect types to simulate defects that are closer to actual clinical conditions. Both cropping methods are implemented in Geomagic Wrap 2021 software by executing custom Python scripts. A single raw normal surface scan data can be expanded by 14 times after processing, covering a variety of defect types.

[0008] Preferably, the data partitioning is performed before data augmentation of the original data, randomly dividing it into a training set, a validation set, and a test set in a ratio of 8:1:1. The three datasets are independent of each other and have no overlap.

[0009] Preferably, in the S2 model construction stage, the optimized PF-Net model is based on the PF-Net point fractal network framework. Dynamic graph convolution and boundary attention mechanisms are introduced in the encoder feature extraction module. The PF-Net technique of hierarchically estimating missing point clouds based on feature points multi-scale generative networks is adopted. The incomplete point cloud is used as input, and only the missing point cloud is output.

[0010] Preferably, the technical process of the PF-Net model is as follows: First, the incomplete point cloud input with defects is sampled three times from the farthest point to obtain multi-scale input; Then, a multi-resolution encoder (MRE) is used as the feature extraction network, and a joint multilayer perceptron (CMLP) is used instead of the traditional multilayer perceptron (MLP) as the feature extractor for a single point cloud. The CMLP is a multilayer perceptron that combines features at multiple scales. It extracts features by fusing local features at different scales, thus preserving the local features of the original point cloud to the greatest extent. The feature vectors obtained through the above process are used to generate the skeleton of the missing point cloud by supervising key points through the point cloud pyramid decoder PPD. Then, the number of predicted points is increased in turn to generate the structure of the missing point cloud.

[0011] Preferably, in the S4 model testing phase, when calculating the quantitative index between the completed point cloud and the true value, the distance threshold d is set to 1mm, that is, if the minimum distance between any point in the predicted point cloud and the true point cloud is less than 1mm, the prediction of that point is considered correct. The formulas for calculating chamfer distance (CD), precision (P), recall (R), and F1-Score are as follows: Chamfer distance CD formula: (1); Accuracy P formula: (2); Recall R formula: (3); F1-Score formula: (4); In the above formula, T is the real point cloud, and t represents any point in the real point cloud. R represents the number of points in the true point cloud; R is the predicted point cloud from the model, where r represents any point in the predicted point cloud. This indicates the number of points in the predicted point cloud; This represents the Euclidean distance between two points.

[0012] Preferably, the specific operations in the S5 model inference stage include: Facial scan data from patients with facial soft tissue defects collected clinically are cropped along the defect boundary to obtain mesh data of the facial defects. After converting the mesh data into point cloud data, multi-scale input data is obtained using farthest point sampling technology, and then the generator completes the point cloud of the defect area.

[0013] Preferably, the method is applicable to the design of facial prostheses and can realize the autonomous simulation reconstruction of point clouds in areas of facial soft tissue defects.

[0014] Compared with the prior art, the beneficial effects of the present invention are as follows: (1) To address the key challenge of scarce clinical facial defect data, this invention proposes a data augmentation strategy based on a constructed dataset of normal human facial soft tissue models. By using topological Boolean operations, paired training data with incomplete inputs and defect labels that are amplified 14 times are generated. This not only effectively solves the problem of deep learning's need for massive labeled data, but also endows the model with the ability to adapt to clinical scenarios.

[0015] (2) This invention incorporates dynamic graph convolution and boundary attention mechanisms into the PF-Net architecture, enhancing the model's boundary awareness capabilities and enabling the model to better reconstruct boundaries. It not only achieves defect reconstruction from 0 to 1, alleviating the requirements of existing designs for initial models, but also provides key technical support for "one-click" repair of facial soft tissue defects.

[0016] (3) The present invention can achieve point cloud reconstruction time of a single facial defect area in seconds, which is far more efficient than manual design. Attached Figure Description

[0017] Figure 1 This is an overall flowchart of a point cloud completion method for facial soft tissue defects based on PF-Net according to the present invention. Figure 2 This is a schematic diagram of the data augmentation algorithm of the present invention; Figure 3 This is a schematic diagram of the data augmentation results of the present invention; Figure 4 This is a flowchart of the point cloud completion network model for facial soft tissue defects based on PF-Net, as described in this invention. Figure 5 This is a front view of the results of reconstructing point clouds with different defects according to the present invention. Figure 6 This is a side view of the results of reconstructing point clouds with different defects according to the present invention. Detailed Implementation

[0018] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0019] Example: Please see Figure 1-6 A point cloud completion method for facial soft tissue defects based on PF-Net includes the following steps: S1. Data Preparation Stage: Design data augmentation algorithms to obtain paired datasets with incomplete inputs and missing labels, and randomly divide them into training set, test set, and validation set.

[0020] In a specific embodiment, the data preparation phase is as follows: This embodiment selects 360 cases of normal facial soft tissue scan data, and first randomly divides them into training set, validation set and test set in a ratio of 8:1:1 (the three datasets are independent of each other and have no overlap to avoid data leakage); then, the designed data augmentation algorithm is used to process the data to obtain paired data with incomplete input and missing labels.

[0021] In the data preparation phase, the core of the data augmentation algorithm is to simulate facial soft tissue defects by cropping normal facial scan data, obtaining paired data of incomplete input and defect labels. Specifically, two cropping methods are used (e.g., ...). Figure 2 (as shown) One method is feature sphere clipping, which involves using feature spheres with different center coordinates and radii to perform Boolean operations with a normal surface scan model; Another method is manual selection and cropping, which involves manually selecting areas on a normal facial scan model according to common clinical facial defect types to simulate defects that are closer to actual clinical conditions. Both cropping methods were implemented using custom Python scripts in Geomagic Wrap 2021 software. A single raw normal surface scan data point can be expanded by 14 times after processing, covering various defect types (such as...). Figure 3 (As shown). The above data augmentation algorithm not only increases the amount of data to meet the needs of neural network model training for large amounts of data, but also ensures the diversity of defect types, thereby improving the model's generalization ability and performance.

[0022] Data partitioning involves randomly dividing the original data into training, validation, and test sets in an 8:1:1 ratio before data augmentation. The three datasets are independent and have no overlap to avoid data leakage that could lead to overly optimistic test results.

[0023] S2, Model Building Stage: The point cloud completion network model PF-Net is optimized by introducing dynamic graph convolution and boundary attention mechanisms in the encoder.

[0024] In a specific embodiment, the solution for the model building stage is as follows: based on the PF-Net point fractal network framework, dynamic graph convolution and boundary attention mechanism are introduced into the encoder feature extraction module (to enhance the attention to boundary information); the technical process of "taking incomplete point cloud as input and only outputting missing point cloud" is followed, which preserves the geometric characteristics of the input point cloud while improving the quality of generated details.

[0025] In the model building phase, the optimized PF-Net model is based on the PF-Net point fractal network framework. Dynamic graph convolution and boundary attention mechanisms are introduced in the encoder feature extraction module. The PF-Net technique of hierarchically estimating missing point clouds based on feature points multi-scale generative networks is adopted. Taking incomplete point clouds as input, only missing point clouds are output. This not only preserves the unique geometric characteristics of the input point clouds, but also generates richer, smoother, and higher-quality detailed features of the missing parts.

[0026] like Figure 4 As shown, the technical process of the PF-Net model is as follows: First, the incomplete point cloud input with defects is sampled three times from the farthest point to obtain multi-scale input; Then, a multi-resolution encoder (MRE) is used as the feature extraction network, and a joint multilayer perceptron (CMLP) is used instead of the traditional multilayer perceptron (MLP) as the feature extractor for a single point cloud. The CMLP is a multilayer perceptron that combines features at multiple scales. It extracts features by fusing local features at different scales, thus preserving the local features of the original point cloud to the greatest extent. The feature vectors obtained through the above process are used to generate the skeleton of the missing point cloud by supervising key points through the point cloud pyramid decoder PPD. Then, the number of predicted points is increased in turn to generate the structure of the missing point cloud.

[0027] S3. Model Training Phase: The training set is input into the PF-Net model for training. During the model training process, the validation set is used to evaluate the model performance and adjust the hyperparameters. After training is completed, the generator is extracted as the core model for subsequent inference.

[0028] S4. Model Testing Phase: Input the test set into the generator to obtain the completed 3D point cloud, and calculate the chamfer distance from the label, accuracy, recall, and F1-Score to evaluate the model accuracy.

[0029] In a specific embodiment, the model testing phase scheme is as follows: During the model testing phase, when calculating the quantitative indicators between the completed point cloud and the ground truth, a distance threshold d is set to 1mm. That is, if the minimum distance between any point in the predicted point cloud and the ground truth point cloud is less than 1mm, the prediction of that point is considered correct. The model testing involves inputting the test set into the generator to obtain the completed 3D point cloud, and calculating the quantitative indicators between the completed point cloud and the ground truth: chamfer distance CD, accuracy P, recall R, and F1-Score, to evaluate the model accuracy.

[0030] The formulas for calculating chamfer distance (CD), precision (P), recall (R), and F1-Score are as follows: Chamfer distance CD formula: (1); Accuracy P formula: (2); Recall R formula: (3); F1-Score formula: (4); In the above formula, T is the real point cloud, and t represents any point in the real point cloud. R represents the number of points in the true point cloud; R is the predicted point cloud from the model, where r represents any point in the predicted point cloud. This indicates the number of points in the predicted point cloud; This represents the Euclidean distance between two points. d: Distance threshold (usually 1mm in point cloud completion scenarios for soft tissue defects on the face), used to determine whether the point prediction is correct.

[0031] The test set described in S1 is input into the optimization model, and the test results are obtained using the above indicators. The test results are shown in Table 1. Table 1. Quantitative results of the method described in this invention on the test set. S5. Model Inference Stage: For clinical facial soft defect point cloud data, the farthest point sampling technique is used to obtain multi-scale input data, and the generator completes the point cloud of the defect area.

[0032] In a specific embodiment, during the model inference stage, i.e., for clinical facial soft defect point cloud data, the farthest point sampling technique is used to obtain multi-scale input data, and the generator completes the point cloud of the defect area.

[0033] This solution involves cropping facial scan data from patients with soft facial defects collected clinically, obtaining mesh data of the facial defects along the defect boundaries, converting it into point cloud data, first using farthest-point sampling technology to obtain multi-scale input data, and then using the generator to complete the point cloud of the defect area. The inference result is as follows: Figure 5 , Figure 6 As shown.

[0034] This embodiment is based on the PF-Net point fractal network framework. Dynamic graph convolution and boundary attention mechanisms are introduced into the encoder feature extraction module, enabling the model to focus on more boundary information to reconstruct the defects. The optimized model follows the PF-Net multi-scale generative network hierarchical estimation technique for missing point clouds based on feature points. It takes incomplete point clouds as input and only outputs the missing point clouds, preserving the unique geometric characteristics of the input point clouds while generating richer, smoother, and higher-quality detailed features for the missing parts.

[0035] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely preferred examples and are not intended to limit the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A point cloud completion method for facial soft tissue defects based on PF-Net, characterized in that, Includes the following steps: S1. Data preparation stage: Design data augmentation algorithms to obtain paired datasets with incomplete inputs and missing labels, and randomly divide them into training set, test set and validation set; S2, Model Building Stage: The point cloud completion network model PF-Net is optimized by introducing dynamic graph convolution and boundary attention mechanisms into the encoder; S3. Model training phase: The training set is input into the PF-Net model for training. During the model training process, the validation set is used to evaluate the model performance and adjust the hyperparameters. After training is completed, the generator is extracted as the core model for subsequent inference. S4. Model testing phase: Input the test set into the generator to obtain the completed 3D point cloud, and calculate the chamfer distance with the label, accuracy, recall, and F1-Score to evaluate the model accuracy. S5. Model Inference Stage: For clinical facial soft defect point cloud data, the farthest point sampling technique is used to obtain multi-scale input data, and the generator completes the point cloud of the defect area.

2. The method for point cloud completion of facial soft tissue defects based on PF-Net according to claim 1, characterized in that: In the data preparation stage described in S1, the core of the data augmentation algorithm is to simulate facial soft tissue defects by cropping normal facial scan data, thereby obtaining paired data of incomplete input and defect labels. Specifically, two cropping methods are used: One method is feature sphere clipping, which involves using feature spheres with different center coordinates and radii to perform Boolean operations with a normal surface scan model; Another method is manual selection and cropping, which involves manually selecting areas on a normal facial scan model according to common clinical facial defect types to simulate defects that are closer to actual clinical conditions. Both cropping methods are implemented in Geomagic Wrap 2021 software by executing custom Python scripts. A single raw normal surface scan data can be expanded by 14 times after processing, covering a variety of defect types.

3. The method for point cloud completion of facial soft tissue defects based on PF-Net according to claim 2, characterized in that: The data partitioning involves randomly dividing the original data into a training set, a validation set, and a test set in a ratio of 8:1:1 before data augmentation. The three datasets are independent of each other and have no overlap.

4. The method for point cloud completion of facial soft tissue defects based on PF-Net according to claim 1, characterized in that: In the model construction stage described in S2, the optimized PF-Net model is based on the PF-Net point fractal network framework. Dynamic graph convolution and boundary attention mechanisms are introduced in the encoder feature extraction module. The PF-Net technique of hierarchically estimating missing point clouds based on feature points multi-scale generative networks is adopted. The incomplete point cloud is used as input, and only the missing point cloud is output.

5. The method for point cloud completion of facial soft tissue defects based on PF-Net according to claim 4, characterized in that: The technical process of the PF-Net model is as follows: First, the incomplete point cloud input with defects is sampled three times from the farthest point to obtain multi-scale input; Then, a multi-resolution encoder (MRE) is used as the feature extraction network, and a joint multilayer perceptron (CMLP) is used instead of the traditional multilayer perceptron (MLP) as the feature extractor for a single point cloud. The CMLP is a multilayer perceptron that combines features at multiple scales. By combining local features at different scales, it extracts features and preserves the local features of the original point cloud to the greatest extent. The feature vectors obtained through the above process are used to generate the skeleton of the missing point cloud by supervising key points through the point cloud pyramid decoder PPD. Then, the number of predicted points is increased in turn to generate the structure of the missing point cloud.

6. The method for point cloud completion of facial soft tissue defects based on PF-Net according to claim 1, characterized in that: In the model testing phase described in S4, when calculating the quantitative index between the completed point cloud and the true value, the distance threshold d is set to 1mm. That is, if the minimum distance between any point in the predicted point cloud and the true point cloud is less than 1mm, the prediction of that point is considered to be correct. The calculation formulas for chamfer distance CD, precision P, recall R, and F1-Score are as follows: Chamfer distance CD formula: (1); Accuracy P formula: (2); Recall R formula: (3); F1-Score Official: (4); In the above formula, T is the real point cloud, and t represents any point in the real point cloud. R represents the number of points in the true point cloud; R is the predicted point cloud from the model, where r represents any point in the predicted point cloud. This indicates the number of points in the predicted point cloud; This represents the Euclidean distance between two points.

7. The method for point cloud completion of facial soft tissue defects based on PF-Net according to claim 1, characterized in that: The specific operations of the model inference phase described in S5 include: The facial scan data of patients with facial soft tissue defects collected clinically are cropped along the defect boundary to obtain the mesh data of the facial defect. After the mesh data is converted into point cloud data, the farthest point sampling technology is used to obtain multi-scale input data, and then the generator completes the point cloud of the defect area.

8. A point cloud completion method for facial soft tissue defects based on PF-Net according to any one of claims 1-7, characterized in that: The method is applicable to the design of facial prostheses and can realize the autonomous simulation reconstruction of point clouds in areas of facial soft tissue defects.

Images