Facial reshaping effect design analysis system and analysis method

Abstract

This invention provides an analysis system and method for facial plastic surgery effect design. The analysis system includes a facial 3D model acquisition and generation module, a plastic organ design module, a plastic surgery model generation module, and a pre- and post-operative comparison analysis module. The facial 3D model acquisition and generation module acquires a facial 3D point cloud model. The plastic organ design module designs the original 3D point cloud model of the plastic organ based on the facial 3D point cloud model to obtain an ideal 3D point cloud model of the plastic organ. The plastic surgery model generation module compares the original 3D point cloud model with the ideal 3D point cloud model, extracts the difference point cloud, and generates a difference 3D point cloud model as a plastic surgery reference or for prosthesis fabrication. The pre- and post-operative comparison analysis module quantifies the plastic surgery effect. This invention can provide a high-precision 3D plastic surgery model, providing doctors with accurate measurement methods and a basis for comparing pre- and post-operative effects.

Description

Technical Field

[0001] This invention relates to the field of medical plastic surgery applications, and in particular to an analysis system and method for designing facial plastic surgery results. Background Technology

[0002] Currently, in the cosmetic surgery industry, the methods used to analyze the facial skin of cosmetic surgery patients and before and after surgery are basically intuitive and subjective evaluations or simple contour and texture analysis based on two-dimensional images captured by cameras to give a rough description and evaluation conclusion. These methods cannot provide objective and accurate evaluation and analysis results of the current condition of cosmetic surgery patients, and are also prone to problems of disharmony between the surgical organs and other areas of the face. Summary of the Invention

[0003] The purpose of this invention is to overcome the shortcomings of inaccurate analysis of pre- and post-operative effects for cosmetic surgery patients in the cosmetic surgery industry. It proposes an analysis system and method for facial plastic surgery effect design, which can provide high-precision 3D plastic surgery models, provide doctors with accurate measurement means for diagnosis, and provide strong evidence for plastic surgery patients to understand their own conditions and compare pre- and post-operative effects.

[0004] To achieve the above objectives, the technical solution created by this invention is implemented as follows:

[0005] A facial plastic surgery effect design analysis system includes a facial 3D model acquisition and generation module, a plastic organ design module, a plastic surgery model generation module, and a before-and-after comparison analysis module.

[0006] The 3D face model acquisition and generation module is used to acquire 3D point cloud models of human faces;

[0007] The plastic organ design module is used to design the original 3D point cloud model of the plastic organ based on the 3D point cloud model of the human face, so as to obtain the ideal 3D point cloud model of the plastic organ.

[0008] The model generation module is used to compare the original 3D point cloud model with the ideal 3D point cloud model, extract the difference point cloud, and generate a difference 3D point cloud model for use as a reference for plastic surgery or for making prostheses.

[0009] The pre- and post-operative comparison analysis module uses the ICP fine registration algorithm to register the original 3D point cloud model with the post-operative 3D point cloud model, calculates the quantitative index value of the post-operative 3D point cloud model, and compares it with the quantitative index value of the ideal 3D point cloud model to quantify the effect of plastic surgery.

[0010] Furthermore, the face 3D model acquisition and generation module includes a first high-speed camera, a second high-speed camera, a projector, and a rotating mechanism. The projector is mounted in the middle of the rotating mechanism, and the first high-speed camera and the second high-speed camera are mounted on the left and right sides of the rotating mechanism, respectively.

[0011] A rotating mechanism drives a projector, a first high-speed camera, and a second high-speed camera to rotate rapidly around a person's face. The projector projects a structured light pattern onto the person's face, while the first and second high-speed cameras are used to capture structured light images from multiple perspectives.

[0012] Furthermore, the face 3D model acquisition and generation module also includes a synchronization triggering unit, a face 3D point cloud image generation unit, and a face 3D point cloud model establishment unit;

[0013] The synchronous triggering unit is used to drive the rotating mechanism to rotate and synchronously trigger the first high-speed camera, the second high-speed camera, and the projector.

[0014] The face 3D point cloud image generation unit is used to process the structured light image from the current viewpoint using the structured light 3D reconstruction method to generate a face 3D point cloud image from the current viewpoint.

[0015] The facial 3D point cloud model establishment unit is used to register facial 3D point cloud images from different angles using the ICP 3D point cloud registration method, and establish a facial 3D point cloud model without blind spots.

[0016] Furthermore, the plastic organ design module includes an edge detection submodule, a surface change calculation submodule, a key point determination submodule, and a 3D model creation submodule, or it may include a data preprocessing submodule, a feature extraction and alignment submodule, and a fusion model generation submodule, wherein:

[0017] The edge detection submodule is used to extract the point cloud of plastic organs by traversing all the point cloud data of the facial 3D point cloud model;

[0018] The surface change calculation submodule is used to calculate the main direction of surface change based on the surface changes of the neighboring regions of the point cloud of the orthopedic organ, and to calculate the key point interest value of the orthopedic organ based on the main direction of surface change.

[0019] The key point determination submodule is used to perform smoothing filtering on the key point interest values ​​of the orthopedic organ and determine the specific key point location through the non-maximum suppression method.

[0020] The 3D model creation submodule is used to obtain an ideal 3D point cloud model of the organ based on the key points of the organ and the original 3D point cloud model, either by self-design or by comparing with the target model.

[0021] Alternatively, the data preprocessing submodule is used to unify the point cloud density of the model's 3D point cloud model and the original 3D point cloud model using the Poisson disk sampling method, so that the feature extraction dimensions of the two are consistent; and a 3D coordinate system is established with the center of the human eye as the origin, with the direction perpendicular to the face forward as the positive Z-axis, the horizontal direction of the face as the X-axis, and the vertical direction of the face as the Y-axis.

[0022] The feature extraction and alignment submodule is used to extract organ features from the 3D point cloud model of the model organ and the original 3D point cloud model using an improved PointNet++ network structure, and to generate an alignment matrix based on the extraction results using a singular value decomposition algorithm.

[0023] The fusion model generation submodule is used to input the alignment matrix into the generative adversarial network to perform organ fusion and obtain an ideal 3D point cloud model.

[0024] Furthermore, the 3D model creation submodule includes self-designed units or target model comparison units, wherein:

[0025] The self-designed unit includes a cubic spline fitting curve generation sub-unit, a deformed model fitting sub-unit, and a 3D model generation sub-unit;

[0026] The cubic spline fitting curve generation subunit is used to generate cubic spline interpolation fitting curves for key points of orthopedic organs using the cubic spline curve fitting method.

[0027] The deformed model fitting subunit is used to fit the cubic spline interpolation fitting curve after the parameters are changed, so as to obtain the three-dimensional point cloud model of the deformed orthopedic organ. The parameters of the cubic spline interpolation fitting curve include direction, position, number of interpolation points and slope at both ends of the curve.

[0028] The 3D model generation sub-unit is used to generate an ideal 3D point cloud model by changing the position of key points, based on the cubic spline fitting curve generation sub-unit and the deformed model fitting sub-unit.

[0029] Alternatively, the target model unit can be compared to include a deformation gradient calculation subunit, a correspondence construction subunit, and a model fusion subunit.

[0030] The deformation gradient calculation subunit is used to calculate the deformation gradient between the key points of the orthopedic organ and the key points of the target model.

[0031] The correspondence construction sub-unit is used to construct the correspondence between the original 3D point cloud model and the target model based on the key point positions of the orthopedic organ and the target model;

[0032] The model fusion sub-unit is used to perform linear or curvilinear interpolation on the original 3D point cloud model based on the calculation results of the correspondence and deformation gradient, so that the target model and the original 3D point cloud model coincide, generating an ideal 3D point cloud model.

[0033] Furthermore, the before-and-after plastic surgery comparison and analysis module includes a model registration unit, a pre- and post-operative plastic surgery parameter acquisition unit, and a surgical effect analysis unit;

[0034] The model registration unit is used to perform point-to-point ICP fine registration algorithm and transform the original 3D point cloud model using an initial rotation and translation matrix. The transformed original 3D point cloud model is compared with the postoperative 3D point cloud model. Points with corresponding point cloud distances less than a threshold in the two 3D point cloud models are the corresponding points. The rotation and translation matrix is ​​solved using the least squares method and iteratively solved to complete the preoperative and postoperative registration of the facial 3D point cloud model.

[0035] The preoperative and postoperative plastic surgery parameter acquisition unit is used to perform meshing processing on the differential 3D point cloud model using the greedy projection triangulation method, calculate the area of ​​all meshes, obtain the area of ​​the differential 3D point cloud, process the area of ​​the differential 3D point cloud using the slicing method, and obtain the quantitative index values ​​of the original 3D point cloud model and the postoperative 3D point cloud model.

[0036] The surgical effect analysis unit is used to compare the quantitative index values ​​of the acquired postoperative 3D point cloud model with the quantitative index values ​​of the ideal 3D point cloud model, and evaluate the plastic surgery effect in combination with the preoperative and postoperative registration results.

[0037] Furthermore, the improved PointNet++ network structure replaces the feature extraction modules in each encoder of the PointNet++ network with multi-scale feature extraction modules. The multi-scale feature extraction modules extract features from the input features at different scales and fuse the feature extraction results at each scale before performing a residual connection operation. The feature transformation modules in each decoder of the PointNet++ network are replaced with multi-scale feature fusion modules. The multi-scale feature fusion modules extract features from the input features at different scales and adaptively fuse the feature extraction results at each scale before performing a residual connection operation.

[0038] The training set used to train the improved PointNet++ network structure consists of at least 30 key points annotated for different organs of different ages and ethnicities, and all the annotation results are used as the training set.

[0039] The loss function L used in training the improved PointNet++ network architecture is:

[0040] ;

[0041] ;

[0042] ;

[0043] Where M is the total number of keypoints in the current sample, and C is the number of organ categories. For real labels, For the predicted label, j is the j-th keypoint, c is the c-th category, K is the number of scales, and k is the k-th scale. ( ) represents L2 normalization. For the feature at the k-th scale, This is the average value of all scale features. It is the minimum mean square loss function. The loss is pixel-level cross-entropy.

[0044] An analytical method for facial plastic surgery effect design, using a facial plastic surgery effect design analysis system, specifically includes the following steps:

[0045] S1: Use the face 3D model acquisition and generation module to acquire the 3D point cloud model of the face;

[0046] S2: Using the plastic organ design module, the original three-dimensional point cloud model of the plastic organ is designed based on the three-dimensional point cloud model of the human face to obtain the ideal three-dimensional point cloud model of the plastic organ.

[0047] S3: Use the model generation module to compare the original 3D point cloud model with the ideal 3D point cloud model, extract the difference point cloud, and generate a difference 3D point cloud model as a reference for shaping or for making prostheses.

[0048] S4: The pre- and post-operative comparison analysis module uses the ICP fine registration algorithm to register the original 3D point cloud model with the post-operative 3D point cloud model, calculates the quantitative index value of the post-operative 3D point cloud model, and compares it with the quantitative index value of the ideal 3D point cloud model to quantify the effect of plastic surgery.

[0049] Furthermore, step S1 specifically includes the following steps:

[0050] S11. The rotating mechanism is driven to rotate by the synchronous triggering unit, and the first high-speed camera, the second high-speed camera and the projector are triggered synchronously.

[0051] S12. The structured light image of the face from the current viewpoint is processed by the structured light 3D reconstruction method using the face 3D point cloud image generation unit to generate a face 3D point cloud image from the current viewpoint.

[0052] S13. Using the face 3D point cloud model establishment unit, the ICP 3D point cloud registration method is used to register the face 3D point cloud images from different angles to establish a face 3D point cloud model without blind spots.

[0053] Furthermore, step S2 specifically includes the following steps:

[0054] S2A1: The edge detection submodule extracts the point cloud of plastic organs by traversing all point cloud data of the facial 3D point cloud model.

[0055] S2A2: The surface change calculation submodule calculates the main direction of surface change based on the surface changes of the neighboring regions of the point cloud of the orthopedic organ, and calculates the key point interest value of the orthopedic organ based on the main direction of surface change.

[0056] S2A3: The key point determination submodule performs smoothing filtering on the interest values ​​of key points of the orthopedic organs and determines the specific key point locations through non-maximum suppression.

[0057] S2A4: The 3D model creation submodule uses the key points of the organ and the original 3D point cloud model to obtain an ideal 3D point cloud model of the organ by using an independent design method or by comparing with the target model.

[0058] Furthermore, step S2 specifically includes the following steps:

[0059] S2B1: The data preprocessing submodule uses the Poisson disk sampling method to unify the point cloud density of the model's 3D point cloud model and the original 3D point cloud model, so that the feature extraction dimensions of the two are consistent; and a 3D coordinate system is established with the center of the human eye as the origin, with the direction perpendicular to the face forward as the positive Z-axis, the horizontal direction of the face as the X-axis, and the vertical direction of the face as the Y-axis.

[0060] S2B2: The feature extraction and alignment submodule uses an improved PointNet++ network structure to extract organ features from the 3D point cloud model of the model's organs and the original 3D point cloud model. Based on the extraction results, an alignment matrix is ​​generated using the singular value decomposition algorithm.

[0061] S2B3: The alignment matrix is ​​input into the generative adversarial network using the fusion model generation submodule to perform organ fusion and obtain an ideal 3D point cloud model.

[0062] Furthermore, step S2A4 specifically includes the following steps:

[0063] S2A41: Using cubic spline fitting curve generation subunits, cubic spline interpolation fitting curves are generated for key points of orthopedic organs through cubic spline fitting methods.

[0064] S2A42: The deformed model fitting sub-unit is used to fit the cubic spline interpolation fitting curve with changed parameters to obtain the three-dimensional point cloud model of the deformed orthopedic organ. The parameters of the cubic spline interpolation fitting curve include direction, position, number of interpolation points and slope at both ends of the curve.

[0065] S2A43: Generating sub-units using a 3D model. By changing the positions of key points, generating sub-units based on cubic spline fitting curves and deformed model fitting sub-units, an ideal 3D point cloud model is generated.

[0066] Furthermore, step S2A4 specifically includes the following steps:

[0067] S2A41`: Calculates the deformation gradient between the key points of the orthopedic organ and the key points of the target model using the deformation gradient calculation sub-unit;

[0068] S2A42`: Construct sub-units based on the correspondence relationship. Based on the key point positions of the orthopedic organ, construct the correspondence relationship between the original 3D point cloud model and the target model.

[0069] S2A43`: Based on the calculation results of the correspondence and deformation gradient of the model fusion sub-unit, the original 3D point cloud model is linearly or curvilinearly interpolated using the shape interpolation method, so that the target model and the original 3D point cloud model can generate an ideal 3D point cloud model.

[0070] Furthermore, step S4 specifically includes the following steps:

[0071] S41: The model registration unit uses the point set to point set ICP fine registration algorithm, and the original three-dimensional point cloud model is transformed using the initial rotation and translation matrix. The transformed original three-dimensional point cloud model is compared with the postoperative three-dimensional point cloud model. The corresponding points in the two three-dimensional point cloud models with a distance of less than a threshold are the corresponding points. The rotation and translation matrix is ​​solved by the least squares method and iteratively solved to complete the preoperative and postoperative registration of the facial three-dimensional point cloud model.

[0072] S42: Using the preoperative and postoperative plastic surgery parameter acquisition unit, the greedy projection triangulation method is used to mesh the differential 3D point cloud model, calculate the area of ​​all meshes, obtain the area of ​​the differential 3D point cloud, and process the area of ​​the differential 3D point cloud by slicing method to obtain the quantitative index values ​​of the original 3D point cloud model and the postoperative 3D point cloud model.

[0073] S43: The surgical effect analysis unit is used to compare the quantitative index values ​​of the obtained postoperative 3D point cloud model with the quantitative index values ​​of the ideal 3D point cloud model, and the plastic surgery effect is evaluated in combination with the preoperative and postoperative registration results.

[0074] Compared with the prior art, the present invention can achieve the following beneficial effects:

[0075] (1) The facial plastic surgery effect design analysis system and analysis method described in this invention adopts high frame rate ultra-high resolution image sensor structured light stereo reconstruction technology, establishes facial organ model through multi-view structured light point cloud image stereo matching, mutually compensates for the point cloud of its own shooting blind spot, truly restores the high precision 3D model of facial skin, and uses a high resolution color camera to realize the three-dimensional color restoration of the face.

[0076] (2) The facial plastic surgery effect design analysis system and analysis method created by the present invention can accurately extract the subtle features of the model's organs (such as the 0.1mm level arc change of the tip of the nose and the subtle wrinkles at the corners of the eyes) through the improved PointNet++ network and adaptive fusion. Compared with the original comparison target model unit with a detail error of about 1mm, the detail adjustment accuracy is improved by more than 10 times, which meets the user's demand for refined plastic surgery effect.

[0077] (3) The facial plastic surgery effect design analysis system and analysis method created by the present invention support large-scale dataset training and incremental training of the model training and optimization unit, which can adapt to the facial features of users of different races and age groups. At the same time, it can be quickly extended to new organ types (such as lips and ears) through transfer learning, thereby enhancing the applicability of the system. In addition, it supports users to upload custom model organ point cloud models, further enhancing the personalized service capabilities of the system.

[0078] (4) The facial plastic surgery effect design analysis system and analysis method created by the present invention uses a three-dimensional reconstruction of the human face model to design the model after plastic surgery, and calculates the precise differences before and after the operation to give high-precision analysis and measurement results. Attached Figure Description

[0079] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments and descriptions of the invention are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings:

[0080] Figure 1 This is a schematic diagram of the logical structure of an analysis system for facial plastic surgery effect design provided according to an embodiment of the present invention;

[0081] Figure 2 This is an analysis system and method for facial plastic surgery effect design provided according to embodiments of the present invention. Detailed Implementation

[0082] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not constitute a limitation thereof.

[0083] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other.

[0084] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicating orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation on this invention. Furthermore, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, features defined with "first," "second," etc., may explicitly or implicitly include one or more of that feature. In the description of this invention, unless otherwise stated, "a plurality of" means two or more.

[0085] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art will understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0086] The invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0087] like Figure 1 As shown, this invention provides an analysis system for facial plastic surgery effect design, including a three-dimensional face model acquisition and generation module, a plastic organ design module, a plastic surgery model generation module, and a before-and-after comparison analysis module.

[0088] The 3D face model acquisition and generation module is used to acquire 3D point cloud models of human faces;

[0089] The plastic organ design module is used to design the original three-dimensional point cloud model of the plastic organ based on the three-dimensional point cloud model of the human face, so as to obtain the ideal three-dimensional point cloud model of the plastic organ.

[0090] The model generation module is used to compare the original 3D point cloud model with the ideal 3D point cloud model, extract the difference point cloud, and generate a difference 3D point cloud model for use as a reference for plastic surgery or for making prostheses.

[0091] The pre- and post-operative comparison analysis module is used to register the original three-dimensional point cloud model and the post-operative three-dimensional point cloud model using the ICP fine registration algorithm, calculate the quantitative index value of the post-operative three-dimensional point cloud model, and compare it with the quantitative index value of the ideal three-dimensional point cloud model to quantify the effect of plastic surgery.

[0092] Currently, the design of a 3D reconstruction system for facial cosmetic medical needs requires comprehensive consideration of accuracy, speed, and data integrity. Since the scale of the entire human face varies greatly, the system design must combine a large field of view with high precision. Secondly, during 3D reconstruction, changes in posture caused by breathing and muscle tremors will affect the alignment of the reconstructed data. Finally, facial contours can lead to blind spots in visual measurement, affecting the integrity of the 3D reconstruction data. Therefore, the facial plastic surgery effect design and analysis measurement system provided in this embodiment of the invention employs a structure combining a color camera and binocular structured light. Using two high-resolution measurement cameras improves the accuracy of the reconstructed data. The high-resolution color camera captures color texture images of the face to reconstruct the 3D model texture. Furthermore, the high-speed scanning, rotation, and translation device combined with a high-frame-rate camera improves the accuracy of camera motion direction data, reducing the impact of changes in human posture to some extent. The two measurement cameras capture images of the face from two directions, reducing the impact of blind spots in visual measurement and improving the integrity and density of the data. Therefore, the face structured light image acquisition device built by the face 3D model acquisition generator module in this invention includes a first high-speed camera, a second high-speed camera, a projector, and a rotating mechanism. The projector is mounted in the middle of the rotating mechanism, and the first high-speed camera and the second high-speed camera are mounted on the left and right sides of the rotating mechanism, respectively. Here, both the first high-speed camera and the second high-speed camera are 8K ultra-high resolution cameras of 7680*4320.

[0093] A rotating mechanism drives a projector, a first high-speed camera, and a second high-speed camera to rotate rapidly around a person's face. The projector projects a structured light pattern onto the person's face, while the first and second high-speed cameras are used to capture structured light images from multiple perspectives.

[0094] In some embodiments, the face 3D model acquisition and generation module further includes a synchronization triggering unit, a face 3D point cloud image generation unit, and a face 3D point cloud model establishment unit;

[0095] The synchronous triggering unit is used to drive the rotating mechanism to rotate and synchronously trigger the first high-speed camera, the second high-speed camera, and the projector.

[0096] The face 3D point cloud image generation unit is used to process the structured light image from the current viewpoint using the structured light 3D reconstruction method to generate a face 3D point cloud image from the current viewpoint.

[0097] The facial 3D point cloud model establishment unit is used to register facial 3D point cloud images from different angles using the ICP 3D point cloud registration method, and establish a facial 3D point cloud model without blind spots.

[0098] In some embodiments, the orthopedic organ design module includes an edge detection submodule, a surface change calculation submodule, a key point determination submodule, and a 3D model creation submodule, or includes a data preprocessing submodule, a feature extraction and alignment submodule, and a fusion model generation submodule, wherein:

[0099] The edge detection submodule is used to extract the point cloud of plastic organs by traversing all the point cloud data of the facial 3D point cloud model;

[0100] The surface change calculation submodule is used to calculate the main direction of surface change based on the surface changes of the neighboring regions of the point cloud of the orthopedic organ, and to calculate the key point interest value of the orthopedic organ based on the main direction of surface change.

[0101] The calculation of the principal direction of surface changes is specifically achieved by constructing a covariance matrix Z for the nearest neighbor regions of the orthopedic organ:

[0102] ;

[0103] Where d is the number of nearest neighbors. The average change within the region is represented by the eigenvector corresponding to the largest eigenvalue of the covariance matrix, which represents the main direction of surface change.

[0104] The key point determination submodule is used to perform smoothing filtering on the key point interest values ​​of the orthopedic organ and determine the specific key point location through the non-maximum suppression method.

[0105] The 3D model creation submodule is used to obtain an ideal 3D point cloud model of the organ based on the key points of the organ and the original 3D point cloud model, either by self-design or by comparing with the target model.

[0106] Alternatively, the data preprocessing submodule is used to unify the point cloud density of the model's 3D point cloud model and the original 3D point cloud model using the Poisson disk sampling method, so that the feature extraction dimensions of the two are consistent; and a 3D coordinate system is established with the center of the human eye as the origin, with the direction perpendicular to the face forward as the positive Z-axis, the horizontal direction of the face as the X-axis, and the vertical direction of the face as the Y-axis.

[0107] The feature extraction and alignment submodule is used to extract organ features from the 3D point cloud model of the model organ and the original 3D point cloud model using an improved PointNet++ network structure, and to generate an alignment matrix based on the extraction results using a singular value decomposition algorithm.

[0108] The fusion model generation submodule is used to input the alignment matrix into the generative adversarial network to perform organ fusion and obtain an ideal 3D point cloud model.

[0109] It should be noted that the data preprocessing submodule is a crucial step in ensuring the input quality of deep learning models. Its core function is to eliminate format differences and noise interference between the user's original organ point cloud and the model's organ point cloud. The specific processing flow is as follows:

[0110] Point cloud density uniformity: Since the user's original organ point cloud is acquired by the high-speed camera and projector of the face 3D model acquisition module, its point cloud density depends on the resolution and sampling frequency of the acquisition device; while the model's organ point cloud may come from different acquisition devices or model libraries, resulting in differences in point cloud density. The data preprocessing unit uses a Poisson disk-based sampling method to unify the point cloud density of the user's original organ point cloud and the model's organ point cloud to a preset value (10 points per square millimeter), ensuring that the feature dimensions of the two are consistent during subsequent feature extraction.

[0111] Coordinate System 1: The coordinate system of the user's original organ point cloud is consistent with the coordinate system of the facial 3D point cloud model output by the face 3D model acquisition and generation module (with the center of the face as the origin, the horizontal direction to the right as the X-axis, the vertical direction upward as the Y-axis, and the vertical direction forward as the Z-axis). The model's organ point cloud may use a different coordinate system. The data preprocessing unit uses a coordinate transformation algorithm to convert the coordinates of the model's organ point cloud into a coordinate system consistent with the user's original organ point cloud to avoid subsequent alignment deviations due to coordinate differences.

[0112] Noise Removal: During the acquisition of user's original organ point cloud, noise points may be introduced due to ambient light interference, camera noise, etc. The model's organ point cloud may also contain redundant points from model construction. The data preprocessing unit uses a statistical filtering algorithm to calculate the mean and standard deviation of the distance between each point and its neighboring points. Points with a distance mean exceeding 3 times the standard deviation are identified as noise points and removed, retaining the valid point cloud data.

[0113] In some embodiments, the 3D model creation submodule includes an autonomous design unit or a target model comparison unit, wherein:

[0114] The self-designed unit includes a cubic spline fitting curve generation sub-unit, a deformed model fitting sub-unit, and a 3D model generation sub-unit;

[0115] The cubic spline fitting curve generation subunit is used to generate cubic spline interpolation fitting curves for key points of orthopedic organs using the cubic spline curve fitting method.

[0116] The deformed model fitting sub-unit is used to fit the cubic spline interpolation fitting curve after the parameters are changed to obtain the three-dimensional point cloud model of the deformed orthopedic organ. The parameters of the cubic spline interpolation fitting curve include direction, position, number of interpolation points and slope at both ends of the curve.

[0117] By changing the direction and position of the cubic spline interpolation fitting curve, altering the slope at both ends of the cubic spline interpolation fitting curve, and changing the number of interpolation points on the cubic spline interpolation fitting curve, a three-dimensional point cloud model of the deformed orthopedic organ can be obtained.

[0118] The 3D model generation sub-unit is used to generate an ideal 3D point cloud model by changing the position of key points, based on the cubic spline fitting curve generation sub-unit and the deformed model fitting sub-unit.

[0119] Alternatively, the target model unit can be compared to include a deformation gradient calculation subunit, a correspondence construction subunit, and a model fusion subunit.

[0120] The deformation gradient calculation subunit is used to calculate the deformation gradient between the key points of the orthopedic organ and the key points of the target model.

[0121] The deformation gradient of a keypoint in a plastic organ is the Jacobian matrix of that point's affine transformation. This affine transformation maps the original keypoint to the keypoint of the target model. The deformation gradient reflects the rotation, scaling, translation, and shearing transformations of the keypoint in the plastic organ.

[0122] The correspondence construction sub-unit is used to construct the correspondence between the original 3D point cloud model and the target model based on the key point positions of the plastic organ and the target model (the model organ selected by the user);

[0123] The model fusion sub-unit is used to perform linear or curvilinear interpolation on the original 3D point cloud model based on the calculation results of the correspondence and deformation gradient, so as to generate an ideal 3D point cloud model from the target model and the original 3D point cloud model.

[0124] To deepen the understanding of the workflow of each unit in the nasal organ design module, the following description uses the nose as an example. First, the nasal organ designer automatically detects and selects five points around the nose area as facial key points: the inner corners of the left and right eyes, the tip of the nose, and the base of the left and right noses. Three-dimensional cubic spline interpolation is used to fit these key points and generate control curves. By changing the shape of these spatial curves, the contour of the nasal organ is designed and fitted. Then, the contour of the nasal organ after surgery is accurately located through contour fitting. Finally, control lines are used to realize the overall nasal organ design. Doctors can also use an existing nasal organ 3D model library as the target design after surgery. The 3D point cloud model of the user's nasal organ is deformed into the target design model. Moving least squares is used to obtain a reference model that blends with the user's original nasal organ 3D model. This reference model has the same number of points and consistent correspondence with the user's original nasal organ 3D model, and has a similar geometric shape to the target nasal model.

[0125] In some embodiments, the pre- and post-operative plastic surgery comparison and analysis module includes a model registration unit, a pre- and post-operative plastic surgery parameter acquisition unit, and a surgical effect analysis unit.

[0126] The model registration unit is used to perform point-to-point ICP fine registration algorithm and transform the original 3D point cloud model using an initial rotation and translation matrix. The transformed original 3D point cloud model is then compared with the postoperative 3D point cloud model. Points with corresponding point cloud distances less than a threshold in the two 3D point cloud models are identified as corresponding points. The rotation and translation matrix is ​​solved using least squares and other methods, and the rotation and translation matrix is ​​iteratively solved to complete the preoperative and postoperative registration of the facial 3D point cloud model.

[0127] The principal orientation of the point cloud (e.g., facial orientation) is estimated by utilizing the symmetry of the facial point cloud, and the principal orientations of the preoperative / postoperative point clouds are aligned (the initial rotation matrix is ​​the identity matrix); the centroid of the point cloud (e.g., the center of the face) is calculated, and the difference in centroid coordinates is used as the initial translation vector.

[0128] The preoperative and postoperative plastic surgery parameter acquisition unit is used to perform meshing processing on the differential 3D point cloud model using the greedy projection triangulation method, calculate the area of ​​all meshes, obtain the area of ​​the differential 3D point cloud, process the area of ​​the differential 3D point cloud using the slicing method, and obtain the quantitative index values ​​of the original 3D point cloud model and the postoperative 3D point cloud model.

[0129] Along a preset direction (such as the vertical direction of the face) z Axis, horizontal direction y A series of parallel slice planes are set up (axis); each plane intersects with the difference point cloud to obtain a contour line, and the area enclosed by the contour line (i.e., the slice area) is calculated; statistical analysis is performed on the area of ​​all slices to extract quantitative index values.

[0130] The surgical effect analysis unit is used to compare the quantitative index values ​​of the acquired postoperative 3D point cloud model with the quantitative index values ​​of the ideal 3D point cloud model, and evaluate the plastic surgery effect in combination with the preoperative and postoperative registration results.

[0131] In some embodiments, the improved PointNet++ network structure replaces the feature extraction modules in each encoder of the PointNet++ network with multi-scale feature extraction modules. The multi-scale feature extraction modules extract features from the input features at different scales and fuse the feature extraction results at each scale before performing a residual connection operation. The feature transformation modules in each decoder of the PointNet++ network are replaced with multi-scale feature fusion modules. The multi-scale feature fusion modules extract features from the input features at different scales and adaptively fuse the feature extraction results at each scale before performing a residual connection operation.

[0132] The multi-scale feature extraction module extracts features from the input features at different scales based on parallel 1×1 convolution, 3×3 convolution, and 5×5 convolution.

[0133] Adaptive fusion involves a multi-scale feature extraction module that outputs features at different scales. These features have varying importance across different tasks or regions. For example, subtle features in the tip of the nose are more important, while global features of the overall facial contour are more important.

[0134] Adaptive fusion allows the network to autonomously determine the contribution of features at each scale and ultimately fuse them into a more discriminative feature vector (set according to actual needs). Adaptive fusion can adopt the adaptive fusion method in the paper "Adaptive Fusion 3D Reconstruction Network of Multi-Layer Multi-Function Feature Weights for High-Frequency Information Objects" published in the journal Optics and Precision Engineering in 2025.

[0135] The training set used to train the improved PointNet++ network structure consists of at least 30 key points annotated for different organs of different ages and ethnicities, and all the annotation results are used as the training set.

[0136] The loss function used to train the improved PointNet++ network architecture is:

[0137] ;

[0138] ;

[0139] ;

[0140] Where M is the total number of keypoints in the current sample, and C is the number of organ categories. For real labels, For the predicted label, j is the j-th keypoint, c is the c-th category, K is the number of scales, and k is the k-th scale. ( ) represents L2 normalization. For the feature at the k-th scale, This is the average value of all scale features. It is the minimum mean square loss function. The loss is pixel-level cross-entropy.

[0141] The PointNet++ network is based on the PointNet++ network described in the paper "PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space" published by Charles R et al. in the conference Neural Information Processing System in 2017.

[0142] The improved PointNet++ network structure extracts subtle textures, local contours, and global structural features of organs through multi-scale convolutional kernels (1×1, 3×3, 5×5), while introducing residual connections to avoid gradient vanishing. Combined with a dynamic attention mechanism, key regions with high cosine similarity (such as the tip of the nose and the corner of the eye) are assigned weights of 0.7-0.9, while non-key regions are assigned weights of 0.1-0.3. An alignment matrix is ​​generated through the singular value decomposition (SVD) algorithm.

[0143] The fusion model submodule is based on the Generative Adversarial Network (GAN) framework. The generator constructs the fusion model through 5 layers of deconvolution operations (3×3 kernel, stride 2), and the discriminator optimizes the model's realism through a 3-layer convolutional network. A facial coordination constraint loss function is introduced, which is summed with the mean squared error loss function with a weight of 3:7 as the total loss function. When the total loss function is <0.01, the fusion model is output to ensure that the fused organs match the adjacent facial regions.

[0144] To avoid visual disharmony between the fused organs and other areas of the user's face, a facial harmony constraint loss function is introduced. This function quantifies the harmony between the fused model and the overall face by calculating the smoothness of the surface curvature transition and the consistency of the normal direction of adjacent regions (such as the nose and forehead, eyes and cheeks) in the fused organ point cloud model and the user's facial 3D point cloud model. This loss function is then weighted and summed with the traditional mean squared error loss function (MSE, which measures the similarity between the fused model and the model's organ features) according to a preset weight (3:7) to serve as the generator's total loss function, guiding the generator to produce a fused model that both conforms to the model's organ features and is harmonious with the overall face.

[0145] Facial coordination constraint loss function for:

[0146] ;

[0147] in, For boundary connection loss, For surface smoothing loss, This is due to anatomical prior loss.

[0148] Boundary connection loss is used to ensure seamless connection between the fused organ and the boundaries of adjacent regions, without obvious gaps or overlaps:

[0149] ;

[0150] in, N The number of boundary points between the fused organ and adjacent areas (such as the junction of the root of the nose and the forehead, and the junction of the ala of the nose and the cheek). For the fusion model i The three-dimensional coordinates of the boundary points; For adjacent facial regions, the first i The three-dimensional coordinates of each corresponding boundary point;

[0151] The physical meaning is: the average Euclidean distance between boundary point pairs. The smaller the value, the tighter the connection (the ideal value is close to 0).

[0152] Surface smoothing loss is used to constrain the continuity of surface normal vector changes between the fused region and adjacent regions, avoiding "sharp edges" or "abrupt changes":

[0153] ;

[0154] in, M The number of transition points between the merging area and the adjacent area (areas where the boundary extends inward by 5-10 mm). For the first in the transition zone j The angle between the "fusion side normal vector" and the "original surface normal vector" at each point; (Dot product of normal vectors; the closer the cosine of the angle between the normal vectors is to 1, the better.) The closer the value is to 0, the smaller the loss, and the smoother the surface transition.

[0155] Anatomical prior loss is used based on the anatomical proportions of a normal human face (e.g., nasofrontal angle approximately 120°-130°, nasolabial angle approximately 90°-100°) to constrain the key angle proportions of the fused organs to be within a reasonable range:

[0156] ;

[0157] in, KThe number of key anatomical angles and proportions (such as the nasofrontal angle, nasolabial angle, and the ratio of nasal alar width to interocular distance). For the fusion model k The actual value of the angle ratio; This is a normal reference value for the proportion of this angle (based on a large number of normal human face statistics). The allowable error range (e.g., angle error ≤ 5°);

[0158] Physical meaning: When the anatomical parameters of the fusion model exceed the normal range, the loss value increases, forcing it to conform to the natural growth law of human face.

[0159] Weight parameters:

[0160] Based on the sensitivity settings of the facial area, priority is given to ensuring seamless boundary transitions and smooth surfaces:

[0161] Values: α =0.4, β =0.4, γ =0.2 (total is 1), which can be finely adjusted according to specific organs (such as nose, jaw).

[0162] The training dataset includes: 1. Covering three races (Asian, Caucasian, and African) and three age groups (18-30, 31-45, and 46-60 years old), with ≥10,000 samples for each organ type (nose, eyes, and jaw), and labeled with ≥30 key feature points; 2. Using mini-batch stochastic gradient descent (SGD) to train the model, with a batch size of 32, a training set, validation set, and test set ratio of 7:2:1, an initial learning rate of 0.001, and a halving of the learning rate if there is no improvement in validation set performance for 5 consecutive rounds; 3. Supporting incremental training, collecting user feedback on fusion effect data, fixing the parameters of the bottom 6 layers of the model, fine-tuning the parameters of the top 6 layers, and iterating and optimizing once every 500 accumulated feedback data points to ensure the fusion accuracy of the test set.

[0163] like Figure 2 As shown, the present invention also provides an analysis method for facial plastic surgery effect design, which is completed using a facial plastic surgery effect design analysis system, and specifically includes the following steps:

[0164] S1: Use the face 3D model acquisition and generation module to acquire the 3D point cloud model of the face;

[0165] S2: Using the plastic organ design module, the original three-dimensional point cloud model of the plastic organ is designed based on the three-dimensional point cloud model of the human face to obtain the ideal three-dimensional point cloud model of the plastic organ.

[0166] S3: Use the model generation module to compare the original 3D point cloud model with the ideal 3D point cloud model, extract the difference point cloud, and generate a difference 3D point cloud model as a reference for shaping or for making prostheses.

[0167] S4: The pre- and post-operative comparison analysis module uses the ICP fine registration algorithm to register the original three-dimensional point cloud model with the post-operative three-dimensional point cloud model, calculates the quantitative index value of the post-operative three-dimensional point cloud model, and compares it with the quantitative index value of the ideal three-dimensional point cloud model to quantify the effect of plastic surgery.

[0168] In some embodiments, step S1 specifically includes the following steps:

[0169] S11. The rotating mechanism is driven to rotate by the synchronous triggering unit, and the first high-speed camera, the second high-speed camera and the projector are triggered synchronously.

[0170] S12. The structured light image of the face from the current viewpoint is processed by the structured light 3D reconstruction method using the face 3D point cloud image generation unit to generate a face 3D point cloud image from the current viewpoint.

[0171] S13. Using the face 3D point cloud model establishment unit, the ICP 3D point cloud registration method is used to register the face 3D point cloud images from different angles to establish a face 3D point cloud model without blind spots.

[0172] In some embodiments, step S2 specifically includes the following steps:

[0173] S2A1: The edge detection submodule extracts the point cloud of plastic organs by traversing all point cloud data of the facial 3D point cloud model.

[0174] S2A2: The surface change calculation submodule calculates the main direction of surface change based on the surface changes of the neighboring regions of the point cloud of the orthopedic organ, and calculates the key point interest value of the orthopedic organ based on the main direction of surface change.

[0175] Key point interest values ​​can characterize changes in that direction and location.

[0176] S2A3: The key point determination submodule performs smoothing filtering on the interest values ​​of key points of the orthopedic organs and determines the specific key point locations through non-maximum suppression.

[0177] S2A4: The 3D model creation submodule uses the key points of the organ and the original 3D point cloud model to obtain an ideal 3D point cloud model of the organ by using an independent design method or by comparing with the target model.

[0178] In some embodiments, step S2 specifically includes the following steps:

[0179] S2B1: The data preprocessing submodule uses the Poisson disk sampling method to unify the point cloud density of the model's 3D point cloud model and the original 3D point cloud model, so that the feature extraction dimensions of the two are consistent; and a 3D coordinate system is established with the center of the human eye as the origin, with the direction perpendicular to the face forward as the positive Z-axis, the horizontal direction of the face as the X-axis, and the vertical direction of the face as the Y-axis.

[0180] S2B2: The feature extraction and alignment submodule uses an improved PointNet++ network structure to extract organ features from the 3D point cloud model of the model's organs and the original 3D point cloud model. Based on the extraction results, an alignment matrix is ​​generated using the singular value decomposition algorithm.

[0181] S2B3: The alignment matrix is ​​input into the generative adversarial network using the fusion model generation submodule to perform organ fusion and obtain an ideal 3D point cloud model.

[0182] In some embodiments, step S2A4 specifically includes the following steps:

[0183] S2A41: Using cubic spline fitting curve generation subunits, cubic spline interpolation fitting curves are generated for key points of orthopedic organs through cubic spline fitting methods.

[0184] S2A42: The deformed model fitting sub-unit is used to fit the cubic spline interpolation fitting curve with changed parameters to obtain the three-dimensional point cloud model of the deformed orthopedic organ. The parameters of the cubic spline interpolation fitting curve include direction, position, number of interpolation points and slope at both ends of the curve.

[0185] S2A43: Generating sub-units using a 3D model. By changing the positions of key points, generating sub-units based on cubic spline fitting curves and deformed model fitting sub-units, an ideal 3D point cloud model is generated.

[0186] In some embodiments, step S2A4 specifically includes the following steps:

[0187] S2A41`: Calculates the key points of the orthopedic organ and the key points of the target model using the deformation gradient calculation sub-unit;

[0188] S2A42`: Construct sub-units based on the correspondence relationship. Based on the key point positions of the orthopedic organ, construct the correspondence relationship between the original 3D point cloud model and the target model.

[0189] S2A43`: Based on the calculation results of the correspondence and deformation gradient of the model fusion sub-unit, the original 3D point cloud model is linearly or curvilinearly interpolated using the shape interpolation method, so that the target model and the original 3D point cloud model can generate an ideal 3D point cloud model.

[0190] In some embodiments, step S4 specifically includes the following steps:

[0191] S41: The model registration unit uses the point set to point set ICP fine registration algorithm, and the original 3D point cloud model is transformed using the initial rotation and translation matrix. The transformed original 3D point cloud model is compared with the postoperative 3D point cloud model. Points with corresponding point cloud distances less than a threshold in the two 3D point cloud models are the corresponding points. The rotation and translation matrix is ​​solved using the least squares method, and the rotation and translation matrix is ​​solved iteratively to complete the preoperative and postoperative registration of the facial 3D point cloud model.

[0192] S42: Using the preoperative and postoperative plastic surgery parameter acquisition unit, the greedy projection triangulation method is used to mesh the differential 3D point cloud model, calculate the area of ​​all meshes, obtain the area of ​​the differential 3D point cloud, and process the area of ​​the differential 3D point cloud by slicing method to obtain the quantitative index values ​​of the original 3D point cloud model and the postoperative 3D point cloud model.

[0193] The quantitative indicators specifically refer to the length, width, and height of the orthopedic organ.

[0194] S43: The surgical effect analysis unit is used to compare the quantitative index values ​​of the obtained postoperative 3D point cloud model with the quantitative index values ​​of the ideal 3D point cloud model, and the plastic surgery effect is evaluated in combination with the preoperative and postoperative registration results.

[0195] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this invention disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this invention can be achieved, and this is not limited herein.

[0196] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. An analysis system for facial plastic surgery effect design, characterized in that: It includes a 3D face model acquisition and generation module, a plastic surgery organ design module, a plastic surgery model generation module, and a before-and-after plastic surgery comparison and analysis module. The 3D face model acquisition and generation module is used to acquire 3D point cloud models of human faces; The plastic organ design module is used to design the original three-dimensional point cloud model of the plastic organ based on the three-dimensional point cloud model of the human face, so as to obtain the ideal three-dimensional point cloud model of the plastic organ. The plastic organ design module includes an edge detection submodule, a surface change calculation submodule, a key point determination submodule, and a 3D model creation submodule, or it may include a data preprocessing submodule, a feature extraction and alignment submodule, and a fusion model generation submodule, wherein: The edge detection submodule is used to extract the point cloud of plastic organs by traversing all the point cloud data of the facial 3D point cloud model; The surface change calculation submodule is used to calculate the main direction of surface change based on the surface changes of the neighboring regions of the point cloud of the orthopedic organ, and to calculate the key point interest value of the orthopedic organ based on the main direction of surface change. The key point determination submodule is used to perform smoothing filtering on the key point interest values ​​of the orthopedic organ and determine the specific key point location through the non-maximum suppression method. The 3D model creation submodule is used to obtain an ideal 3D point cloud model of the plastic organ based on the key points of the organ and the original 3D point cloud model, either by self-design or by comparing with the target model. Alternatively, the data preprocessing submodule is used to unify the point cloud density of the model's 3D point cloud model and the original 3D point cloud model using the Poisson disk sampling method, so that the feature extraction dimensions of the two are consistent; and a 3D coordinate system is established with the center of the human eye as the origin, with the direction perpendicular to the face forward as the positive Z-axis, the horizontal direction of the face as the X-axis, and the vertical direction of the face as the Y-axis. The feature extraction and alignment submodule is used to extract organ features from the 3D point cloud model of the model organ and the original 3D point cloud model using an improved PointNet++ network structure, and to generate an alignment matrix based on the extraction results using a singular value decomposition algorithm. The fusion model generation submodule is used to input the alignment matrix into the generative adversarial network to perform organ fusion and obtain an ideal 3D point cloud model. The model generation module is used to compare the original 3D point cloud model with the ideal 3D point cloud model, extract the difference point cloud, and generate a difference 3D point cloud model for use as a reference for plastic surgery or for making prostheses. The pre- and post-operative comparison analysis module is used to register the original three-dimensional point cloud model and the post-operative three-dimensional point cloud model using the ICP fine registration algorithm, calculate the quantitative index value of the post-operative three-dimensional point cloud model, and compare it with the quantitative index value of the ideal three-dimensional point cloud model to quantify the effect of plastic surgery.

2. The facial plastic surgery effect design analysis system according to claim 1, characterized in that: The face 3D model acquisition and generation module includes a first high-speed camera, a second high-speed camera, a projector, and a rotating mechanism. The projector is mounted in the middle of the rotating mechanism, and the first high-speed camera and the second high-speed camera are mounted on the left and right sides of the rotating mechanism, respectively. A rotating mechanism drives a projector, a first high-speed camera, and a second high-speed camera to rotate rapidly around a person's face. The projector projects a structured light pattern onto the person's face, while the first and second high-speed cameras are used to capture structured light images from multiple perspectives.

3. The facial plastic surgery effect design analysis system according to claim 2, characterized in that: The face 3D model acquisition and generation module also includes a synchronization triggering unit, a face 3D point cloud image generation unit, and a face 3D point cloud model establishment unit; The synchronous triggering unit is used to drive the rotating mechanism to rotate and synchronously trigger the first high-speed camera, the second high-speed camera, and the projector. The face 3D point cloud image generation unit is used to process the structured light image from the current viewpoint using the structured light 3D reconstruction method to generate a face 3D point cloud image from the current viewpoint. The facial 3D point cloud model establishment unit is used to register facial 3D point cloud images from different angles using the ICP 3D point cloud registration method, and establish a facial 3D point cloud model without blind spots.

4. The facial plastic surgery effect design analysis system according to claim 1, characterized in that: The 3D model creation submodule includes self-designed units or units that compare with target models, wherein: The self-designed unit includes a cubic spline fitting curve generation sub-unit, a deformed model fitting sub-unit, and a 3D model generation sub-unit; The cubic spline fitting curve generation subunit is used to generate cubic spline interpolation fitting curves for key points of orthopedic organs using the cubic spline curve fitting method. The deformed model fitting subunit is used to fit the cubic spline interpolation fitting curve after the parameters are changed, so as to obtain the three-dimensional point cloud model of the deformed orthopedic organ. The parameters of the cubic spline interpolation fitting curve include direction, position, number of interpolation points and slope at both ends of the curve. The 3D model generation sub-unit is used to generate an ideal 3D point cloud model by changing the position of key points, based on the cubic spline fitting curve generation sub-unit and the deformed model fitting sub-unit. Alternatively, the target model unit can be compared to include a deformation gradient calculation subunit, a correspondence construction subunit, and a model fusion subunit. The deformation gradient calculation subunit is used to calculate the deformation gradient between the key points of the orthopedic organ and the key points of the target model. The correspondence construction sub-unit is used to construct the correspondence between the original 3D point cloud model and the target model based on the key point positions of the orthopedic organ and the target model; The model fusion sub-unit is used to perform linear or curvilinear interpolation on the original 3D point cloud model based on the calculation results of the correspondence and deformation gradient, so that the target model and the original 3D point cloud model coincide, generating an ideal 3D point cloud model.

5. The facial plastic surgery effect design analysis system according to claim 1, characterized in that: The pre- and post-operative comparison and analysis module for plastic surgery includes a model registration unit, a pre- and post-operative plastic surgery parameter acquisition unit, and a surgical effect analysis unit. The model registration unit is used to perform point-to-point ICP fine registration algorithm and transform the original 3D point cloud model using an initial rotation and translation matrix. The transformed original 3D point cloud model is compared with the postoperative 3D point cloud model. Points with corresponding point cloud distances less than a threshold in the two 3D point cloud models are the corresponding points. The rotation and translation matrix is ​​solved using the least squares method and iteratively solved to complete the preoperative and postoperative registration of the facial 3D point cloud model. The preoperative and postoperative plastic surgery parameter acquisition unit is used to perform meshing processing on the differential 3D point cloud model using the greedy projection triangulation method, calculate the area of ​​all meshes, obtain the area of ​​the differential 3D point cloud, process the area of ​​the differential 3D point cloud using the slicing method, and obtain the quantitative index values ​​of the original 3D point cloud model and the postoperative 3D point cloud model. The surgical effect analysis unit is used to compare the quantitative index values ​​of the acquired postoperative 3D point cloud model with the quantitative index values ​​of the ideal 3D point cloud model, and evaluate the plastic surgery effect in combination with the preoperative and postoperative registration results.

6. The facial plastic surgery effect design analysis system according to claim 1, characterized in that: The improved PointNet++ network structure replaces the feature extraction modules in each encoder of the PointNet++ network with multi-scale feature extraction modules. The multi-scale feature extraction modules extract features from the input features at different scales and fuse the feature extraction results at each scale before performing a residual connection operation. The feature transformation modules in each decoder of the PointNet++ network are replaced with multi-scale feature fusion modules. The multi-scale feature fusion modules extract features from the input features at different scales and adaptively fuse the feature extraction results at each scale before performing a residual connection operation. The training set used to train the improved PointNet++ network structure consists of at least 30 key points annotated for different organs of different ages and ethnicities, and all the annotation results are used as the training set. The loss function L used in training the improved PointNet++ network architecture is: ； ； ； Where M is the total number of keypoints in the current sample, and C is the number of organ categories. For real labels, For the predicted label, j is the j-th keypoint, c is the c-th category, K is the number of scales, and k is the k-th scale. ( ) represents L2 normalization. For the feature at the k-th scale, This is the average value of all scale features. It is the minimum mean square loss function. The loss is pixel-level cross-entropy.

7. A method for analyzing facial plastic surgery effect design, performed using the facial plastic surgery effect design analysis system described in claim 2, characterized in that, Specifically, the steps include the following: S1: Use the face 3D model acquisition and generation module to acquire the 3D point cloud model of the face; S2: Using the plastic organ design module, the original three-dimensional point cloud model of the plastic organ is designed based on the three-dimensional point cloud model of the human face to obtain the ideal three-dimensional point cloud model of the plastic organ. Step S2 specifically includes the following steps: S2A1: The edge detection submodule extracts the point cloud of plastic organs by traversing all point cloud data of the facial 3D point cloud model. S2A2: The surface change calculation submodule calculates the main direction of surface change based on the surface changes of the neighboring regions of the point cloud of the orthopedic organ, and calculates the key point interest value of the orthopedic organ based on the main direction of surface change. S2A3: The key point determination submodule performs smoothing filtering on the interest values ​​of key points of the orthopedic organs and determines the specific key point locations through non-maximum suppression. S2A4: The 3D model creation sub-module uses the key points of the organ and the original 3D point cloud model to obtain an ideal 3D point cloud model of the organ by using an independent design method or by comparing with the target model. Alternatively, step S2 may specifically include the following steps: S2B1: The data preprocessing submodule uses the Poisson disk sampling method to unify the point cloud density of the model's 3D point cloud model and the original 3D point cloud model, so that the feature extraction dimensions of the two are consistent; and a 3D coordinate system is established with the center of the human eye as the origin, with the direction perpendicular to the face forward as the positive Z-axis, the horizontal direction of the face as the X-axis, and the vertical direction of the face as the Y-axis. S2B2: The feature extraction and alignment submodule uses an improved PointNet++ network structure to extract organ features from the 3D point cloud model of the model's organs and the original 3D point cloud model. Based on the extraction results, an alignment matrix is ​​generated using the singular value decomposition algorithm. S2B3: The alignment matrix is ​​input into the generative adversarial network through the fusion model generation submodule to perform organ fusion and obtain an ideal 3D point cloud model. S3: Use the model generation module to compare the original 3D point cloud model with the ideal 3D point cloud model, extract the difference point cloud, and generate a difference 3D point cloud model as a reference for shaping or for making prostheses. S4: The pre- and post-operative comparison analysis module uses the ICP fine registration algorithm to register the original three-dimensional point cloud model with the post-operative three-dimensional point cloud model, calculates the quantitative index value of the post-operative three-dimensional point cloud model, and compares it with the quantitative index value of the ideal three-dimensional point cloud model to quantify the effect of plastic surgery.

8. The analytical method for facial plastic surgery effect design according to claim 7, characterized in that... Step S1 specifically includes the following steps: S11. The rotating mechanism is driven to rotate by the synchronous triggering unit, and the first high-speed camera, the second high-speed camera and the projector are triggered synchronously. S12. The structured light image of the face from the current viewpoint is processed by the structured light 3D reconstruction method using the face 3D point cloud image generation unit to generate a face 3D point cloud image from the current viewpoint. S13. Using the face 3D point cloud model establishment unit, the ICP 3D point cloud registration method is used to register the face 3D point cloud images from different angles to establish a face 3D point cloud model without blind spots.

9. The analytical method for facial plastic surgery effect design according to claim 7, characterized in that... Step S2A4 specifically includes the following steps: S2A41: Using cubic spline fitting curve generation subunits, cubic spline interpolation fitting curves are generated for key points of orthopedic organs through cubic spline fitting methods. S2A42: The deformed model fitting sub-unit is used to fit the cubic spline interpolation fitting curve with changed parameters to obtain the three-dimensional point cloud model of the deformed orthopedic organ. The parameters of the cubic spline interpolation fitting curve include direction, position, number of interpolation points and slope at both ends of the curve. S2A43: Generating sub-units using a 3D model. By changing the positions of key points, generating sub-units based on cubic spline fitting curves and deformed model fitting sub-units, an ideal 3D point cloud model is generated.

10. The method for analyzing facial plastic surgery effects according to claim 7, characterized in that... Step S2A4 specifically includes the following steps: S2A41`: Calculates the deformation gradient between the key points of the orthopedic organ and the key points of the target model using the deformation gradient calculation sub-unit; S2A42`: Construct sub-units based on the correspondence relationship. Based on the key point positions of the orthopedic organ, construct the correspondence relationship between the original 3D point cloud model and the target model. S2A43`: Based on the calculation results of the correspondence and deformation gradient of the model fusion sub-unit, the original 3D point cloud model is linearly or curvilinearly interpolated using the shape interpolation method, so that the target model and the original 3D point cloud model can generate an ideal 3D point cloud model.

11. The analytical method for facial plastic surgery effect design according to claim 7, characterized in that... Step S4 specifically includes the following steps: S41: The model registration unit uses the point set to point set ICP fine registration algorithm, and the original three-dimensional point cloud model is transformed using the initial rotation and translation matrix. The transformed original three-dimensional point cloud model is compared with the postoperative three-dimensional point cloud model. The corresponding points in the two three-dimensional point cloud models with a distance of less than a threshold are the corresponding points. The rotation and translation matrix is ​​solved by the least squares method and iteratively solved to complete the preoperative and postoperative registration of the facial three-dimensional point cloud model. S42: Using the preoperative and postoperative plastic surgery parameter acquisition unit, the greedy projection triangulation method is used to mesh the differential 3D point cloud model, calculate the area of ​​all meshes, obtain the area of ​​the differential 3D point cloud, and process the area of ​​the differential 3D point cloud by slicing method to obtain the quantitative index values ​​of the original 3D point cloud model and the postoperative 3D point cloud model. S43: The surgical effect analysis unit is used to compare the quantitative index values ​​of the obtained postoperative 3D point cloud model with the quantitative index values ​​of the ideal 3D point cloud model, and the plastic surgery effect is evaluated in combination with the preoperative and postoperative registration results.

Images