Generative skull face restoration method
By combining generative adversarial network training with a multi-scale discriminator, the problem of insufficient multi-scale feature fusion in cranial facial reconstruction is solved, achieving high-precision facial reconstruction with images highly similar to the real face, thus improving feature capture and recognition capabilities.
Patent Information
- Application Number
- CN202511482489.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-10-16
- Publication Date
- 2026-01-27
AI Technical Summary
Existing cranial facial reconstruction techniques suffer from insufficient multi-scale feature fusion, leading to loss of texture details and global structural distortion. Traditional methods are inefficient and difficult to standardize and process in batches.
Generative adversarial networks (GANs) are used for training, combined with a frequency-domain perceptual generator and a multi-scale discriminator. Through a semi-wavelet attention mechanism and a high-low frequency decomposition module, the feature representation and capture capabilities are improved. The GAN generates restored facial images, and the results are verified by objective evaluation metrics.
It achieves high-precision cranial facial reconstruction, with images retaining texture details and highly similar to the real face, improving multi-scale feature capture and image recognition capabilities, and maintaining the accuracy of the overall facial structure and the clarity of local features.
Smart Images
Figure CN121414904A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of facial reconstruction technology, and in particular to a generative cranial facial reconstruction method. Background Technology
[0002] In the fields of forensic anthropology, archaeology, and clinical medicine, craniofacial reconstruction is a crucial research topic, aiming to reconstruct the facial features of an deceased person based on unknown skull morphological characteristics. Traditional methods rely primarily on empirical techniques, such as manual sculpting, which, while intuitive, is highly dependent on the operator's professional experience and subjective judgment, resulting in low efficiency and difficulty in standardization and batch processing. With the development of computer vision and 2D graphics, statistical model-based methods have gradually become the mainstream.
[0003] In recent years, deep learning methods, especially generative adversarial networks (GANs), have shown great potential in image generation and style transfer tasks, providing new ideas for solving the aforementioned nonlinear mapping problems. Some studies have attempted to apply two-dimensional GAN architectures to craniofacial reconstruction tasks, mining deep feature associations from large amounts of craniofacial data through end-to-end learning. Some technical solutions use basic two-dimensional convolutional neural networks or standard U-Net structures as generators, attempting to learn a direct conversion from cranial volume data to facial skin volume data. However, such deep learning methods based on voxels or point clouds still face challenges when processing high-precision craniofacial data. Summary of the Invention
[0004] In view of the aforementioned existing problems, the present invention is proposed.
[0005] Therefore, this invention provides a generative cranial facial reconstruction method that solves the problems of texture detail loss and global structural distortion caused by insufficient multi-scale feature fusion in existing cranial facial reconstruction techniques.
[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution: In a first aspect, the present invention provides a generative cranial facial reconstruction method, which includes scanning a subject to obtain craniofacial CT data and preprocessing the CT data. By setting a threshold to segment the contours of the skull and facial skin, the three-dimensional data of the contour information is orthogonally projected along the Frankfurt plane to generate a two-dimensional craniofacial contour image. The generative adversarial network is trained using a sample training set. A two-dimensional craniofacial contour image is input into the trained generative adversarial network, and the facial image of the skull to be restored is output. The frequency domain sensing generator is based on the U-Net network. It introduces a semi-wavelet attention mechanism in the encoder and skip connections to enhance the feature representation capability of key regions and establish an accurate craniofacial mapping relationship. It adds a high- and low-frequency decomposition module to the decoder to separate and optimize high-frequency and low-frequency features. The multi-scale discriminator is based on the PathGAN network and adds a multi-scale attention module to improve the ability to capture multi-scale features and recognize the restored image. The reconstructed facial image is generated by a trained generative adversarial network, and the reconstruction effect is verified by combining objective evaluation metrics. The image retains texture details and is highly similar to the real face.
[0007] As a preferred embodiment of the generative craniofacial reconstruction method of the present invention, the following steps are included: scanning the subject to obtain craniofacial CT data, and preprocessing the CT data: The subjects were scanned to obtain craniofacial CT slice data, which were then denoised to obtain denoised craniofacial CT slice data.
[0008] As a preferred embodiment of the generative craniofacial reconstruction method of the present invention, the following steps are included: segmenting the contours of the skull and facial skin by setting a threshold, orthogonally projecting the three-dimensional data of the contour information along the Frankfurt plane to generate a two-dimensional craniofacial contour image: The grayscale threshold was set based on the characteristics that different human tissues exhibit different densities and grayscale values in CT images. The skull and facial skin contours were segmented from the denoised craniofacial CT data by grayscale thresholding, resulting in three-dimensional data with contour information. Three-dimensional data with contour information is orthogonally projected along the Frankfurt plane to generate a two-dimensional craniofacial contour image.
[0009] As a preferred embodiment of the generative cranial facial reconstruction method of the present invention, the method includes the following steps: training a generative adversarial network using a sample training set, inputting a two-dimensional cranial contour image into the trained generative adversarial network, and outputting a facial image of the skull to be reconstructed. The improved generative adversarial network is trained using a sample training set to obtain trained weights; the trained weights are then loaded into the improved generative adversarial network to form an improved generative adversarial network that can be used for inference. A two-dimensional craniofacial contour image is input into an improved generative adversarial network loaded with a pre-trained weight file to generate a preliminary reconstruction result. The improved generative adversarial network processes the two-dimensional craniofacial contour image and outputs a facial image of the skull to be reconstructed.
[0010] As a preferred embodiment of the generative craniofacial reconstruction method of the present invention, the frequency domain perceptual generator is based on a U-Net network, and a semi-wavelet attention mechanism is introduced in the encoder and skip connections to enhance the feature representation ability of key regions and establish an accurate craniofacial mapping relationship, including the following steps: The frequency domain sensing generator processes input data based on the U-Net network. The U-Net network applies a semi-wavelet attention mechanism to process features in the encoder part and applies a semi-wavelet attention mechanism to transfer features in the skip connection part. The semi-wavelet attention mechanism enhances the ability to represent key regional features, which in turn helps to establish accurate craniofacial mapping relationships.
[0011] As a preferred embodiment of the generative cranial facial reconstruction method of the present invention, the method includes the following steps: A high- and low-frequency decomposition module is added to the decoder to separate and optimize high-frequency and low-frequency features: The U-Net network incorporates a high-low frequency decomposition module in the decoder section to process features, separating high-frequency and low-frequency features.
[0012] As a preferred embodiment of the generative cranial facial reconstruction method of the present invention, the multi-scale discriminator is based on the PathGAN network, with the addition of a multi-scale attention module to improve the ability to capture multi-scale features and the recognition ability of the reconstructed image, including the following steps: The multi-scale discriminator is based on the PathGAN network to build a discriminative architecture. The PathGAN network adds a multi-scale attention module to process the input features. The multi-scale attention module enhances the ability to capture multi-scale features, which in turn improves the recognition ability of the restored image. The PathGAN network outputs the image authenticity judgment result through a multi-scale discrimination mechanism.
[0013] As a preferred embodiment of the generative cranial facial reconstruction method of the present invention, the reconstructed facial image is generated through a trained generative adversarial network, and the reconstruction effect is verified by combining objective evaluation indicators. The image retains texture details and is highly similar to the real facial appearance, including the following steps: An improved generative adversarial network, after training, generates facial images of the skull to be restored. The generated facial images of the skull to be restored and the real facial images are then input into the FaceNet model for similarity analysis. The FaceNet model calculates the similarity score between the facial image of the skull to be reconstructed and the real facial image. The quality of the facial image of the skull to be reconstructed is comprehensively evaluated by combining the SSIM, PSNR, RMSE, and LPIPS indicators. The validation results show that the facial image of the skull to be reconstructed retains more texture details and is highly similar to the real facial image.
[0014] The beneficial effects of this invention are as follows: By adding a multi-scale attention module to the PathGAN network based on a multi-scale discriminator, the ability to capture multi-scale features of craniofacial images and the discrimination accuracy of the restored images are improved. By processing image features of different resolutions in parallel, global contour information and local texture details are effectively fused, enabling the discriminator to more accurately identify structurally abnormal or detail-distorted areas in the generated images, thereby guiding the generator to optimize the reconstruction process. Combined with the synergistic effect of the semi-wavelet attention mechanism and the high- and low-frequency decomposition module in the frequency domain perceptual generator, the entire network can maintain the accuracy of the overall facial structure and the clarity of local features during the training process. Attached Figure Description
[0015] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0016] Figure 1 This is a normalized two-dimensional skull model in the embodiment.
[0017] Figure 2 The image shown is a craniofacial reconstruction diagram from Example 2. The skull to be reconstructed is used, and facial skin is randomly selected for reconstruction.
[0018] Figure 3 A diagram of the improved generative adversarial network architecture.
[0019] Figure 4 The images show a comparison between the real and restored face skin in Example 2: (a) real face skin, (b) restored face skin.
[0020] Figure 5 The results are for craniofacial reconstruction based on a statistical deformation model method.
[0021] Figure 6 The results of craniofacial reconstruction based on a knowledge analysis model method are shown.
[0022] Figure 7 The results are for craniofacial reconstruction based on the Pix2Pix network.
[0023] Figure 8 The results of craniofacial reconstruction based on the CycleGAN network.
[0024] Figure 9 Results of craniofacial reconstruction based on InfraGAN network Figure 10 The results of craniofacial reconstruction based on the UvcGAN network.
[0025] Figure 11 The results of craniofacial reconstruction based on the StegoGAN network.
[0026] Figure 12 This is a flowchart of a generative cranial facial reconstruction method. Detailed Implementation
[0027] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.
[0028] Example 1, referring to Figures 1-12 This is the first embodiment of the present invention, which provides a generative cranial facial reconstruction method, comprising the following steps: S1. Scan the subject to obtain craniofacial CT data and preprocess the CT data.
[0029] S1.1. Scan the subject to obtain craniofacial CT slice data. The craniofacial CT slice data is then denoised to obtain denoised craniofacial CT slice data.
[0030] Furthermore, a multi-scan spiral CT scanner was used to scan the head of the subjects to obtain craniofacial CT slice data stored in DICOM format. The craniofacial CT slice data was then denoised to preserve key edge information of the skull and facial skin, resulting in denoised craniofacial CT slice data.
[0031] S2. By setting a threshold, the contours of the skull and facial skin are segmented, and the three-dimensional data of the contour information is orthogonally projected along the Frankfurt plane to generate a two-dimensional craniofacial contour image.
[0032] S2.1 A grayscale threshold is set based on the characteristics that different human tissues exhibit different densities and grayscale values in CT images.
[0033] Furthermore, based on the characteristic that different human tissues exhibit different densities and gray values in CT images, the gray histogram distribution of the denoised craniofacial CT data was analyzed. Due to its high density, bone tissue exhibits a high gray value in CT images, while soft tissue exhibits a relatively low gray value. Based on this characteristic, a gray value threshold that can effectively distinguish between bone and soft tissue was set.
[0034] S2.2. After denoising the craniofacial CT data through grayscale thresholding, the contours of the skull and facial skin are segmented to obtain three-dimensional data with contour information.
[0035] Furthermore, the denoised craniofacial CT data is processed by grayscale thresholding. The grayscale value of each voxel in the denoised craniofacial CT data is compared with the set grayscale threshold. Voxels with grayscale values higher than the threshold are determined to belong to the skull contour, and voxels with grayscale values within a specific range are determined to belong to the facial skin contour. Through this binarization segmentation operation, the boundary voxel set of the skull and facial skin is accurately extracted, thereby obtaining three-dimensional data with contour information.
[0036] S2.3. The three-dimensional data with contour information is orthogonally projected along the Frankfurt plane to generate a two-dimensional craniofacial contour image.
[0037] Furthermore, the three-dimensional data with contour information is orthogonally projected along the Frankfurt plane. First, a projection coordinate system is established based on the Frankfurt plane, which is determined by the upper points of the left and right auricular points and the lower point of the left orbit. Then, all contour voxel points in the three-dimensional data with contour information are projected onto the two-dimensional imaging plane along a direction perpendicular to the plane. The position information of each projection point is recorded to generate a depth map, and finally a two-dimensional craniofacial contour image is obtained.
[0038] S3. Train the generative adversarial network using the sample training set. Input the two-dimensional craniofacial contour image into the trained generative adversarial network and output the facial image of the skull to be restored.
[0039] S3.1. Train the improved generative adversarial network using the sample training set to obtain trained weights; load the trained weights into the improved generative adversarial network to form an improved generative adversarial network that can be used for inference.
[0040] Furthermore, the improved generative adversarial network (GAN) is trained using a sample training set consisting of multiple pairs of 2D craniofacial contour images and real facial images. During training, the frequency-domain perceptual generator in the improved GAN is based on the U-Net network structure. It extracts multi-scale features from the 2D craniofacial contour images through the encoder and applies a semi-wavelet attention mechanism in the skip connections to enhance the feature representation of key regions. At the same time, the high- and low-frequency decomposition module in the decoder separates global contour and local texture information. The multi-scale discriminator is based on the PathGAN network structure. It uses a multi-scale attention module to perform multi-level feature comparison between the generated image and the real image. Through joint optimization of the adversarial loss function, reconstruction loss function, and Fourier loss function, the network parameters are iteratively adjusted until the loss function converges. Finally, the trained weight file is saved. The trained weight file is loaded into the corresponding layer of the improved GAN to form a reasoning network model with craniofacial mapping capabilities.
[0041] S3.2 Input the two-dimensional craniofacial contour image into the improved generative adversarial network loaded with the trained weight file to generate a preliminary restoration result. The improved generative adversarial network processes the two-dimensional craniofacial contour image and outputs the facial image of the skull to be restored.
[0042] Furthermore, the two-dimensional craniofacial contour image to be restored is input into an improved generative adversarial network loaded with a pre-trained weight file. The frequency domain perceptual generator first encodes the features of the input image, focuses on the feature mapping of key anatomical structures of the skull through a semi-wavelet attention mechanism, and optimizes the low-frequency information of the facial contour and the high-frequency details of the skin texture using high- and low-frequency decomposition modules, respectively. The multi-scale discriminator performs multi-scale discrimination on the preliminary restoration result output by the generator. By comparing the feature differences between the generated face and the real face at different resolutions, it feeds back the adversarial loss to optimize the generator output. The improved generative adversarial network finally outputs the facial image of the skull to be restored. This image must meet the structural similarity index and perceptual quality evaluation requirements to ensure that the restoration result is highly consistent with the real face in terms of global structure and local details.
[0043] S4, the frequency domain sensing generator, is based on the U-Net network. It introduces a semi-wavelet attention mechanism in the encoder and skip connections to enhance the feature representation capability of key regions and establish an accurate craniofacial mapping relationship.
[0044] S4.1 The frequency domain sensing generator processes input data based on the U-Net network. The U-Net network applies a half-wavelet attention mechanism to process features in the encoder part and applies a half-wavelet attention mechanism to transfer features in the skip connection part.
[0045] Furthermore, the frequency domain-aware generator processes the input two-dimensional craniofacial contour image data based on the U-Net network. In the encoder part, the U-Net network extracts multi-scale features through progressive downsampling by convolutional layers. At the same time, a semi-wavelet attention mechanism is applied to process features at each downsampling stage. This mechanism first decomposes the feature map into different frequency sub-bands by wavelet transform, and then calculates the attention weights of each sub-band and weighted fusion to enhance the feature response of key anatomical regions such as the orbit and nasal bone. In the skip connection part, the U-Net network also applies a semi-wavelet attention mechanism to pass features. The output features of each layer of the encoder are weighted by wavelet domain attention and then concatenated with the features of the corresponding layer of the decoder to ensure that the integrity of key information is maintained during the transmission of multi-scale features.
[0046] S4.2 The semi-wavelet attention mechanism enhances the ability to represent key regional features, which helps to establish accurate craniofacial mapping relationships.
[0047] Furthermore, the semi-wavelet attention mechanism decomposes the feature map into low-frequency approximation components and high-frequency detail components through wavelet transform, and calculates the spatial attention weights of different frequency components respectively. This enables the network to adaptively focus on the key regional features that play a decisive role in the craniofacial mapping, thereby enhancing the key regional feature representation ability. The enhanced key regional feature representation ability helps to establish a precise nonlinear mapping relationship from the skull contour to the distribution of facial soft tissues by providing accurate contextual information of the skull structure and local detail features, ultimately achieving an anatomically reasonable facial reconstruction effect.
[0048] S5. Add a high-low frequency decomposition module to the decoder to separate and optimize high-frequency and low-frequency features.
[0049] The S5.1 and U-Net networks incorporate a high- and low-frequency decomposition module in the decoder section to process features, separating high-frequency and low-frequency features.
[0050] Furthermore, the -Net network incorporates a high- and low-frequency decomposition module in the decoder section to process features. This module performs frequency domain separation processing on the input feature map through a set of parallel convolutional filter banks. The low-pass filter uses Gaussian convolution kernels to smooth the feature map to capture low-frequency information of facial contours and overall structure, while the high-pass filter uses Laplacian operators or gradient operators to enhance the edge and texture details of the feature map to extract high-frequency information. The high- and low-frequency decomposition module sends the processed feature streams to different branch paths to achieve effective separation of high-frequency and low-frequency features, providing a dedicated frequency domain information foundation for subsequent feature optimization and fusion.
[0051] The high- and low-frequency decomposition module uses an adaptive weighting mechanism to dynamically adjust the contribution of features in different frequency bands during feature processing. The low-frequency feature branch mainly preserves the global shape of the face and the distribution trend of soft tissue, while the high-frequency feature branch focuses on restoring local features such as skin texture and pore details. The outputs of the two branches are recalibrated through a gated attention unit to ensure the balance and coordination of high-frequency and low-frequency information during the reconstruction process.
[0052] The separated high-frequency and low-frequency features are refined through independent convolutional layers. The high-frequency feature channel enhances the representation of local details through residual connections, while the low-frequency feature channel optimizes the continuity of the overall structure through long-range dependency modeling. The optimized high-frequency and low-frequency features are then reintegrated through feature stitching operations to form an enhanced feature representation containing multi-scale information, providing sufficient frequency domain feature support for generating high-quality face reconstruction images.
[0053] S6, the multi-scale discriminator is based on the PathGAN network and adds a multi-scale attention module to improve the ability to capture multi-scale features and recognize restored images.
[0054] S6.1 The multi-scale discriminator is based on the PathGAN network to build a discriminative architecture. The PathGAN network adds a multi-scale attention module to process the input features.
[0055] Furthermore, the multi-scale discriminator is based on the PathGAN network to build a discriminative architecture. The PathGAN network adds a multi-scale attention module when processing input features. This module processes image feature maps at different resolutions in parallel, calculates attention weights for global contour features and local texture features respectively, uses convolutional layers to extract multi-scale features and generate corresponding attention masks, and then fuses the weighted multi-scale features and outputs them to the subsequent discriminative layer.
[0056] S6.2 The multi-scale attention module enhances the ability to capture multi-scale features, and the enhanced multi-scale feature capture ability improves the recognition ability of the restored image. The PathGAN network outputs the image authenticity judgment result through the multi-scale discrimination mechanism.
[0057] Furthermore, the multi-scale attention module processes feature maps of different scales simultaneously through a parallel branch structure, and enhances the feature response of important regions by utilizing a spatial attention mechanism, thereby improving the ability to capture multi-scale features. The enhanced multi-scale feature capture capability enables the PathGAN network to simultaneously evaluate the global structural consistency and local detail authenticity of the restored image, thereby improving the recognition ability of the restored image. Finally, the PathGAN network performs a comprehensive judgment on features at each scale through a multi-scale discrimination mechanism, and outputs the image authenticity judgment result after processing by a fully connected layer.
[0058] S7. Generate restored facial images using a trained generative adversarial network, and verify the restoration effect using objective evaluation metrics. The images retain texture details and are highly similar to the real facial features.
[0059] S7.1. Generate facial images of the skull to be restored using an improved generative adversarial network after training. Input the generated facial images of the skull to be restored and the real facial images into the FaceNet model for similarity analysis.
[0060] Furthermore, an improved generative adversarial network is trained to generate facial images of the skull to be restored. The generated facial images of the skull to be restored and the corresponding real facial images are simultaneously input into the FaceNet model for similarity analysis. The FaceNet model extracts the feature embedding vectors of the two images through a deep convolutional neural network and calculates the distance metric of the feature vectors in Euclidean space, thereby achieving a quantitative assessment of the semantic similarity between the two images.
[0061] S7.2 The FaceNet model calculates the similarity score between the facial image of the skull to be restored and the real facial image. The quality of the facial image of the skull to be restored is comprehensively evaluated by combining the SSIM, PSNR, RMSE and LPIPS indicators. The verification results show that the facial image of the skull to be restored retains more texture details and is highly similar to the real facial image.
[0062] The similarity score expression is: ; in, To score the similarity, The image shows the facial features of the skull to be reconstructed. This is a true-to-life image.
[0063] Furthermore, the FaceNet model calculates the similarity score between the facial image of the skull to be reconstructed and the real facial image. It uses the cosine similarity metric to calculate the cosine of the angle between two feature vectors, combines the structural similarity index SSIM to assess the degree of preservation of image structural information, peak signal-to-noise ratio PSNR to measure the degree of pixel-level distortion, root mean square error RMSE to calculate pixel value differences, and learned perceptual patch similarity LPIPS to evaluate the perceptual differences in the deep learning feature space. The comprehensive evaluation of these metrics verifies the quality of the facial image of the skull to be reconstructed. The final verification results show that the facial image of the skull to be reconstructed maintains the accuracy of the overall facial structure while preserving more high-frequency information such as skin texture and pore details, and exhibits a high degree of similarity to the real facial image in both visual perception and numerical measurement.
[0064] Reference Figures 1-12 Example 2 is the second embodiment of the present invention, which provides a generative cranial facial reconstruction method, including the following steps: In this embodiment, the skull and facial skin data are two-dimensional craniofacial contour images. These two-dimensional craniofacial contour images are derived from DICOM-compliant head cross-sectional image data obtained from head CT scans. After denoising and redundancy removal of these CT data, the Marching Cubes method is used for reconstruction to obtain two-dimensional craniofacial contour images of the skull and facial skin. The data volume consists of 440 pairs of one-to-one skull and facial skin data samples. The specific implementation steps are as follows: Step 1: The two-dimensional craniofacial contour images of the skull and facial skin in the training sample set are first unified to the Frankfurt coordinate system, and then all craniofacial data are normalized in terms of coordinates and scale to ensure consistent pose. Subsequently, two-dimensional craniofacial contour images are generated through front view mapping. All images are aligned and cropped to a size of 256×256.
[0065] Step 2: Train the network model using the sample set. The network model is an improved U-Net network and an improved PathGAN network, such as... Figure 3 As shown. The improved U-Net network in this embodiment includes a feature connection layer based on attention learning and a processing layer based on high and low frequency information of the image, which can accurately extract the corresponding features of the skull and facial skin, thereby making the restored facial image more similar to the real facial image; the improved PathGAN network is a processing layer based on multi-scale feature information, which can enhance the discriminator's ability to recognize global structure and local details, thereby optimizing the generation process to generate more realistic facial images. The total loss function of the generative adversarial network in this embodiment is:
[0066]
[0067]
[0068]
[0069] An improved U-Net network method was used to extract global and local features from skull samples and facial skin samples n (n=1, 2, 3, …, 900). By balancing the constraints of global facial structure and local details, higher-quality facial images were reconstructed. An example of the network's specific training process is shown below: First, initialize the weights and bias parameters for each layer; Secondly, a 3×256×256 skull image is used as input. The frequency domain-aware generator adopts a U-Net structure, consisting of 8 downsampling and 8 upsampling convolutional blocks. Skip connections are used to pass shallow features from the encoder to the decoder to preserve detail information and gradually generate the facial image. During convolution, to better capture key region information of the image, a semi-wavelet attention mechanism is applied to the first three convolutional layers of the encoder and the last three skip connections of the decoder. To accurately and fully extract and represent global and local facial texture details, a high-low frequency decomposition module is introduced in the penultimate skip connection of the decoder. Finally, the input image is processed through each layer to output the final facial reconstruction image. Next, the reconstructed facial image and the corresponding skull image are simultaneously input into an improved PathGAN network. The multi-scale discriminator model uses PathGAN to discriminate 70×70 image patches. To better capture the multi-scale features of the craniofacial image and improve the model's ability to recognize the reconstructed image, thereby optimizing the generation process and producing more realistic facial images, a multi-scale attention mechanism is added to the convolutional layers of the discriminator. Finally, the network's sigmoid function determines whether the reconstructed image is real (1 for real, 0 for false). After training the improved generative adversarial network 300 times, when the loss values of the generator and discriminator stabilize, the model reaches the optimal parameters, which can accurately reflect the skull-facial skin mapping relationship and realize the facial reconstruction of unknown skulls. The relevant parameters during the last three training sessions are as follows: Epoch:
[300] [898 / 900], training time: 21589.0467s, discriminator loss d_loss: 0.40671217, generator loss g_loss: 0.37710034; Epoch count:
[300] [899 / 900], Training time: 21594.1639s, Discriminator loss: d_loss 0.40362192, generator loss g_loss: 0.39425612; Epoch:
[300] [900 / 900], Training time: 21599.5005s, Discriminator loss d_loss: 0.40796842, generator loss g_loss: 0.38536143, loss converged.
[0070] Reference Figures 1-12 Example 3 is the third embodiment of the present invention, which provides a generative cranial facial reconstruction method, including the following steps: This embodiment achieves craniofacial reconstruction based on the model constructed in Embodiment 2: Step one, place the skull to be restored ( Figure 1 The two-dimensional craniofacial contour image used in Example 1 is randomly selected, first unified to the Frankfurt coordinate system, and then all craniofacial data are normalized in terms of coordinates and scale to make their posture consistent. The two-dimensional craniofacial contour image to be restored and random facial skin data are generated by mapping the main view. Step two: Using the model trained in Example 1, the two-dimensional craniofacial contour image to be restored and random facial skin data are simultaneously input into the network model of Example 1, finally obtaining the facial image of the skull to be restored, such as... Figure 2 .
[0071] The FaceNet model (from "FaceNet recognition algorithm subject to multiple constraints: Assessment of the performance" by Mensah JA) was used to perform similarity analysis between the restored face image and the real face image, achieving a similarity of 93%.
[0072] Comparative experiment with traditional craniofacial reconstruction methods: Figure 5 The results are based on the craniofacial reconstruction method using a statistical deformation model. Figure 6 The results are based on a knowledge analysis model method for craniofacial reconstruction.
[0073] This invention employs five objective evaluation metrics: SSIM, PSNR, RMSE, LPIPS, and similarity. SSIM measures the similarity between two images in terms of brightness, contrast, and structure; a value closer to 1 indicates greater similarity. PSNR and RMSE measure pixel differences; a higher PSNR value indicates better image quality, while a lower RMSE value indicates smaller differences. LPIPS captures semantic information from images through deep learning features; a value closer to 0 indicates greater similarity. Similarity is calculated using the FaceNet model; a higher value indicates greater image similarity.
[0074] Table 1 compares the objective indicators of the present invention and traditional craniofacial reconstruction methods. As can be seen from Table 1, the method of the present invention outperforms SDM and KAM in all indicators, with the highest scores in SSIM (0.92), PSNR (36.72), and similarity score (0.93), and the lowest in LPIPS (0.13) and RMSE (5.38). Although KAM is slightly better than SDM in some indicators, the gap between both and the method of the present invention is significant, especially in terms of PSNR and similarity.
[0075] Table 1 Objective indicators of traditional craniofacial reconstruction results Table 1 Objective indicators of traditional craniofacial reconstruction results
[0076] Comparative experiment with current state-of-the-art image generation methods: Figure 7 The results are based on craniofacial reconstruction using the Pix2Pix network. Figure 8 The results are based on the CycleGAN network for craniofacial reconstruction. Figure 9 The results are based on craniofacial reconstruction using the InfraGAN network. Figure 10 The results are based on the UvcGAN network for craniofacial reconstruction. Figure 11 The results are based on the StegoGAN network for craniofacial reconstruction.
[0077] This invention employs five objective evaluation metrics: SSIM, PSNR, RMSE, LPIPS, and similarity.
[0078] Table 2 compares the objective metrics of this invention with current advanced image generation methods. As shown in Table 2, the method of this invention outperforms Pix2Pix, CycleGAN, InfraGAN, UvcGAN, and StegoGAN in all metrics. SSIM (0.94) and PSNR (37.72) are the highest, while LPIPS (0.13) and RMSE (5.48) are the lowest, and the similarity score (0.93) is the best. StegoGAN and UvcGAN perform second best, while CycleGAN is relatively weak.
[0079] Table 2 Objective indicators of craniofacial reconstruction results using image generation methods
[0080] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.
Claims
1. A generative cranial facial reconstruction method, characterized in that: This includes scanning the subjects to obtain craniofacial CT data and preprocessing the CT data; By setting a threshold to segment the contours of the skull and facial skin, the three-dimensional data of the contour information is orthogonally projected along the Frankfurt plane to generate a two-dimensional craniofacial contour image. The generative adversarial network is trained using a sample training set. A two-dimensional craniofacial contour image is input into the trained generative adversarial network, and the facial image of the skull to be restored is output. The frequency domain sensing generator is based on the U-Net network. It introduces a semi-wavelet attention mechanism in the encoder and skip connections to enhance the feature representation capability of key regions and establish an accurate craniofacial mapping relationship. A high- and low-frequency decomposition module is added to the decoder to separate and optimize high-frequency and low-frequency features. The multi-scale discriminator is based on the PathGAN network and adds a multi-scale attention module to improve the ability to capture multi-scale features and recognize the restored image. The reconstructed facial image is generated by a trained generative adversarial network, and the reconstruction effect is verified by combining objective evaluation metrics. The image retains texture details and is highly similar to the real face.
2. The generative cranial facial reconstruction method as described in claim 1, characterized in that: The subjects were scanned to obtain craniofacial CT data, and the CT data were preprocessed, including the following steps: The subjects were scanned to obtain craniofacial CT slice data, which were then denoised to obtain denoised craniofacial CT slice data.
3. The generative cranial facial reconstruction method as described in claim 2, characterized in that: The contours of the skull and facial skin are segmented by setting a threshold. The three-dimensional data of the contour information is orthogonally projected along the Frankfurt plane to generate a two-dimensional craniofacial contour image, including the following steps: The grayscale threshold was set based on the characteristics that different human tissues exhibit different densities and grayscale values in CT images. The skull and facial skin contours were segmented from the denoised craniofacial CT data by grayscale thresholding, resulting in three-dimensional data with contour information. Three-dimensional data with contour information is orthogonally projected along the Frankfurt plane to generate a two-dimensional craniofacial contour image.
4. The generative cranial facial reconstruction method as described in claim 3, characterized in that: The generative adversarial network (GAN) is trained using a sample training set. A two-dimensional craniofacial contour image is input into the trained GAN, which outputs a facial image of the skull to be reconstructed. The process includes the following steps: The improved generative adversarial network is trained using a sample training set to obtain trained weights; the trained weights are then loaded into the improved generative adversarial network to form an improved generative adversarial network that can be used for inference. A two-dimensional craniofacial contour image is input into an improved generative adversarial network loaded with a pre-trained weight file to generate a preliminary reconstruction result. The improved generative adversarial network processes the two-dimensional craniofacial contour image and outputs a facial image of the skull to be reconstructed.
5. The generative cranial facial reconstruction method as described in claim 4, characterized in that: The frequency domain-aware generator, based on the U-Net network, introduces a semi-wavelet attention mechanism in the encoder and skip connections to enhance the representation of key region features and establish accurate craniofacial mapping relationships. This includes the following steps: The frequency domain sensing generator processes input data based on the U-Net network. The U-Net network applies a semi-wavelet attention mechanism to process features in the encoder part and applies a semi-wavelet attention mechanism to transfer features in the skip connection part. The semi-wavelet attention mechanism enhances the ability to represent key regional features, which in turn helps to establish accurate craniofacial mapping relationships.
6. The generative cranial facial reconstruction method as described in claim 5, characterized in that: The multi-scale discriminator is based on the PathGAN network, with the addition of a multi-scale attention module to improve the ability to capture multi-scale features and recognize reconstructed images. It includes the following steps: The multi-scale discriminator is based on the PathGAN network to build a discriminative architecture. The PathGAN network adds a multi-scale attention module to process the input features. The multi-scale attention module enhances the ability to capture multi-scale features, which in turn improves the recognition ability of the restored image. The PathGAN network outputs the image authenticity judgment result through a multi-scale discrimination mechanism.
7. The generative cranial facial reconstruction method as described in claim 6, characterized in that: The multi-scale discriminator is based on the PathGAN network, with the addition of a multi-scale attention module to improve the ability to capture multi-scale features and recognize reconstructed images. It includes the following steps: The multi-scale discriminator is based on the PathGAN network to build a discriminative architecture. The PathGAN network adds a multi-scale attention module to process the input features. The multi-scale attention module enhances the ability to capture multi-scale features, which in turn improves the recognition ability of the restored image. The PathGAN network outputs the image authenticity judgment result through a multi-scale discrimination mechanism.
8. The generative cranial facial reconstruction method as described in claim 7, characterized in that, The reconstructed facial image is generated using a trained generative adversarial network, and the reconstruction effect is verified using objective evaluation metrics. The image retains texture details and is highly similar to the real facial appearance. The process includes the following steps: An improved generative adversarial network, after training, generates facial images of the skull to be restored. The generated facial images of the skull to be restored and the real facial images are then input into the FaceNet model for similarity analysis. The FaceNet model calculates the similarity score between the facial image of the skull to be reconstructed and the real facial image. The quality of the facial image of the skull to be reconstructed is comprehensively evaluated by combining the SSIM, PSNR, RMSE, and LPIPS indicators. The validation results show that the facial image of the skull to be reconstructed retains more texture details and is highly similar to the real facial image.