An intelligent craniofacial translation method

Abstract

The application discloses an intelligent craniofacial translation method, comprising the following steps: 1, collecting cranial and facial CT image data; 2, performing image preprocessing, three-dimensional reconstruction and fairing treatment on the cranial and facial CT image data to obtain a complete three-dimensional model of the cranial and facial bones; 3, placing the three-dimensional model of the cranial and facial bones in a Frankfort horizontal plane, and performing normalization operation; 4, in the Frankfort horizontal plane, performing vertical mapping of the three-dimensional model of the cranial and facial bones on the XOZ plane to obtain a main view image of the cranial and facial bones; 5, introducing a depth separable convolution into a generator U-Net network of a GAN network to construct a Skull2Skin-GAN network model for cranial and facial translation; 6, training parameters of the Skull2Skin-GAN network model; and 7, placing the cranial and facial image into the Skull2Skin-GAN network model to generate a two-dimensional facial image, which is not only high in definition but also real.

Description

Technical Field

[0001] This invention belongs to the field of cranial reconstruction technology, specifically an intelligent craniofacial translation method. Background Technology

[0002] In recent years, the research and application of craniofacial reconstruction have received widespread attention. Craniofacial reconstruction uses the skull of an unknown individual as input to reconstruct the human face based on the skull structure and soft tissue morphology. This technology encompasses multiple interdisciplinary fields, including anatomy, anthropology, and computer science. Anatomy provides the relationship between the skull and facial soft tissues, offering a scientific basis for craniofacial reconstruction; anthropology provides information on the skull's ethnicity, sex, age, and other characteristics, providing individualized parameters for reconstruction; and computer science provides digital tools and methods for craniofacial reconstruction. Currently, craniofacial reconstruction technology is widely used in fields such as forensic medicine, archaeology, and plastic surgery.

[0003] Traditional craniofacial reconstruction techniques involve analyzing the skull structure and then having experienced experts manually draw possible facial shapes, such as using clay to reconstruct the three-dimensional facial features on a skull model. This method requires a lot of time and manpower, and the reconstruction results largely depend on experience, making it less than ideal in practice.

[0004] In recent years, the development of computer graphics and visualization technology has led to the computer-aided stage of craniofacial reconstruction. In addition, the emergence of technologies such as MRI and CT has made it easier to acquire craniofacial data. Compared with traditional manual methods, the advantages of computer-aided craniofacial reconstruction methods are that they overcome the defects of strong subjectivity and high operational requirements. They can restore the appearance based on the skull structure and soft tissue morphology. However, they cannot take into account the influence of individual differences and environmental factors on the appearance. Moreover, computer-aided craniofacial reconstruction generally requires high-quality skull data, such as high-quality CT or MRI images.

[0005] Currently, Generative Adversarial Networks (GANs) have made significant progress in the field of image translation. Image translation refers to transforming an image from one image domain to another. It can be viewed as removing a certain attribute of the original image and reassigning it a new one, such as converting a black-and-white photo to a color photo, restoring a mosaic image to a clear image, or converting a sketch into a realistic photograph. Depending on the specific task, the network structure for image translation varies greatly. However, existing GANs achieve image translation through adversarial learning between two neural networks. If the training process is not balanced, problems may arise, such as overfitting between the generator and discriminator. Alternatively, when the number of output samples from the generator is less than the number of input samples from the discriminator, GANs may experience mode collapse, meaning the generator only generates single or similar samples. Therefore, this invention aims to provide an automated and intelligent craniofacial translation network based on GANs, facilitating the rapid and simple generation of clear, high-fidelity facial images from skull images under limited sample conditions. Summary of the Invention

[0006] In view of the shortcomings of existing technologies, the purpose of this invention is to provide an intelligent craniofacial translation method that can quickly and easily generate clear and realistic facial images from skull and facial CT images.

[0007] To achieve the above objectives, the present invention employs the following technical solution: An intelligent craniofacial translation method includes the following steps: Step 1: Acquire CT images of the skull and face; Step 2: Perform image preprocessing, 3D reconstruction, and smoothing on the CT image data of the skull and face to obtain a complete 3D model of the skull and face; Step 3: Place the 3D model of the skull and face in the Frankfurt coordinate system and perform normalization. Step 4: In the Frankfurt coordinate system, the three-dimensional models of the skull and face are vertically mapped to the XOZ plane to obtain the front view images of the skull and face. Step 5: Introduce depthwise separable convolutions into the generator U-Net network of the GAN network to construct the Skull2Skin-GAN network model for skull and face translation. Step 6: Train the parameters of the Skull2Skin-GAN network model; Step 7: Input the skull and facial images into the Skull2Skin-GAN network model to generate a two-dimensional facial image.

[0008] Furthermore, the specific process for acquiring skull and facial CT data in step 1 is as follows: using computed tomography (CT) technology, using... Fine scanning of thick sections of the skull using beam, gamma rays, or ultrasound yields several consecutive CT slice images of the skull and face.

[0009] Furthermore, the depthwise separable convolution in step 5 includes depthwise convolution and pointwise convolution. The depthwise convolution applies a convolution kernel to each channel of the input feature map to generate a feature map with the same depth as the input feature map. The pointwise convolution is a 1×1 ordinary convolution that maps the feature map generated by the depthwise convolution to a new feature space.

[0010] Furthermore, the discriminator used in the Skull2Skin-GAN network model is the PatchGAN multi-task relative discriminator.

[0011] Furthermore, the process of training the Skull2Skin-GAN network model in step 6 is as follows: Step 6.1: Train the Skull2Skin-GAN network model on the skull and face paired datasets to learn the mapping between the image translation domain X and domain Y; Step 6.2: Use the adversarial loss function Reconstruction losses and loss of perception of facial details The overall loss function of the network structure for optimizing the image translation domain transformation problem is expressed as: In the formula, and The values ​​are 100 and 20 respectively, where G represents the generator and D represents the discriminator. This means fixing the parameters of D and optimizing the parameters of G to make... The value is the smallest. This means fixing the parameters of G and optimizing the parameters of D to make... The value is the largest.

[0012] Furthermore, step 6.1 also includes random non-permeable data augmentation of the paired skull and face datasets.

[0013] Furthermore, the random non-penetrating data augmentation includes pixel transformation, geometric transformation, color transformation, image spatial filtering, additive noise, and cropping; The pixel changes include x-flip, 90... 。 Rotation and integer translation.

[0014] Furthermore, step 6.2 specifically includes the following steps: Step 6.2.1: Use the loss function of WGAN-GP as the generative adversarial loss function. This enables the discriminator of the Skull2Skin-GAN network to have 1-Lipschitz continuity, generating adversarial loss. Represented as: In the formula, λ is the weight of the gradient penalty term, set to 10. Represents the gradient. It is a real image and generating images The sampling results between them, where D represents the discriminator, This represents the maximum likelihood estimate of the parameters, where P and Q represent... and The image domain in which it is located; Step 6.2.2: Constrain the similarity between the input and output images using the L1 norm loss function, so that the generated image retains the identity information of the source image, and reconstruct the loss. Represented as: In the formula, Represents the source domain image. Represents the target domain image. Indicates a generator; Step 6.2.3: Calculate the facial detail perception loss using the multi-paradigm facial aesthetic prediction dataset SCUT-FBP5500. To improve the visual similarity between generated images and real images, and reduce the loss of facial detail perception. Represented as: In the formula, This represents the generated facial image. Represents a real human face image. This represents the feature extraction layer of a fixed network. Indicates the specified first Each network layer, This represents the output features of the layer; Step 6.2.4: Adopt the adversarial loss function Reconstruction losses and loss of perception of facial details Constructing the total loss function of the network structure .

[0015] Furthermore, step 7 specifically includes the following steps: Step 7.1: Input the skull and face images into the generator of the Skull2Skin-GAN network model. The encoder of the generator converts the input skull and face images into feature maps. The feature maps obtained in steps 7.2 and 7.1 are processed by depthwise separable convolution and then converted by the generator's decoder into an output image of the same size as the input skull and face images.

[0016] Furthermore, it also includes using a trained face super-resolution network GFPGAN to perform super-resolution inpainting on the two-dimensional facial image generated in step 7, thereby generating a realistic image.

[0017] Compared with the prior art, the present invention has the following technical effects: To address the issues of high data requirements, cumbersome acquisition steps, high storage costs, demanding network design and training, and low accuracy in 3D craniofacial reconstruction methods, this invention proposes an intelligent craniofacial translation network, Skull2Skin-GAN. Its generator replaces the traditional convolutional layer with depthwise separable convolutions, building upon the U-Net model. Since depthwise separable convolutions have fewer parameters and faster inference speed, they perform better than traditional convolutions under small sample conditions. This reduces the size of the Skull2Skin-GAN network model, lowers the number of parameters and the risk of overfitting, and allows for better network training under small sample conditions, facilitating the rapid and simple generation of clear, high-fidelity facial images.

[0018] This invention employs the PatchGAN multi-task relative discriminator and uses random non-penetrating data augmentation to process paired datasets of skull and face, preventing directional distortion in the images. This not only allows for better training of the network with a small number of samples, but also prevents the discriminator from overfitting, thus better adapting to the network structure of this invention and further ensuring the quality of the generated images.

[0019] The SCUT-FBP5500 multi-paradigm facial aesthetic prediction dataset is introduced to calculate the loss of facial feature detail perception, and the GFPGAN network is introduced to perform super-resolution restoration on the generated facial images. This makes the facial features of the network generated by the present invention clearer and more detailed under small sample conditions, and also has the function of identity recognition. Attached Figure Description

[0020] Figure 1 This is a structural diagram of the generator of the present invention; Figure 2 This is a diagram of the random non-permeable data enhancement structure of the present invention; Figure 3 This is a structural diagram of the Skull2Skin-GAN network model of the present invention; Figure 4 The comparison results of skull and facial images generated by this invention and other network models; Figure 5 This is a structural diagram of the GFPGAN network of the present invention; Figure 6 The image is the result of GFPGAN processing according to the present invention. Detailed Implementation

[0021] The specific content of the present invention will be further explained in detail below with reference to the embodiments.

[0022] The three-dimensional reconstruction process of craniofacial data in this embodiment can be referenced in: Lin Pengyue. Research on cranial and facial reconstruction and realism processing based on generative adversarial networks [D]. Northwest University, 2022. DOI:10.27405 / d.cnki.gxbdu.2022.000244).

[0023] The skull and facial CT image data used in this embodiment are from the Affiliated Hospital of Shaanxi University of Traditional Chinese Medicine. The configuration of this computer is shown in Table 1.

[0024] like Figures 1-6 As shown, an intelligent craniofacial translation method includes the following steps: Step 1: Acquire CT image data of the skull and face: First, using computed tomography (CT) technology, Fine scanning of the thick layers of the skull using beam, gamma rays, and ultrasound is performed to obtain several consecutive CT slice images of the skull and face. The obtained CT slice image data is entered into a computer to obtain cross-sectional graphics of the head. In order to represent the shape of the entire craniofacial region, many consecutive cross-sectional graphics images of the skull and face are required, among which cross-sectional graphics images are the most common. Since CT slices can describe the specific morphological structure of the skull and face through three-dimensional reconstruction technology, this embodiment uses CT scanning to obtain skull and face image data, and the image data size is 256×256, and the format is PNG grayscale image. Step 2: Three-dimensional reconstruction of skull and facial CT image data: In order to obtain a more intuitive and complete skull and facial morphology and structure, the acquired skull and facial CT image data were preprocessed, reconstructed in three dimensions and smoothed to obtain a complete three-dimensional skull and facial model, laying the foundation for subsequent craniofacial reconstruction and realistic processing. Step 3: Place the 3D model of the skull and face in the Frankfurt coordinate system and perform normalization to remove factors that are irrelevant to the restoration task (e.g., pose and scale) to ensure that the experimental results are not interfered with. Step 4: In the Frankfurt coordinate system, the three-dimensional models of the skull and face are vertically mapped to the XOZ plane to obtain the front view images of the skull and face. Step 5: Construct an intelligent craniofacial translation network model, Skull2Skin-GAN, based on a GAN network. The structure is as follows: Figure 3 As shown; In this embodiment, the generator of the GAN network used is U-Net. When the number of training samples is small, if the model size is too large, the network will have difficulty converging; if the model size is too small, the network will not have sufficient representational power to capture the complex distribution of the data, resulting in unrealistic generated images. Therefore, this embodiment introduces depthwise separable convolutions into the U-Net network to reduce the number of model parameters, thereby adapting to small sample tasks. The structure of the generator of the Skull2Skin-GAN network model constructed in this embodiment is as follows: Figure 1 As shown, it includes two parts: an encoder and a decoder; The depthwise separable convolution is also a type of convolution operation, consisting of depthwise convolution and pointwise convolution. The Convolutional Network (DN) consists of two convolutional operations, aiming to reduce the number of parameters and computational cost, thereby improving network efficiency. Specifically, depthwise convolution applies a separate kernel to each channel of the input feature map, generating a feature map with the same depth as the input feature map. Pointwise convolution is a 1×1 ordinary convolution that maps the feature map obtained from depthwise convolution to a new feature space. Compared to ordinary convolution operations, depthwise separable convolution has the advantages of fewer parameters, faster inference speed, and better training of generative adversarial networks under small sample conditions. Specifically: firstly, depthwise separable convolution significantly reduces the number of parameters in the convolutional layer, thus reducing the risk of overfitting; secondly, when performing depthwise convolution, specific features can be extracted from each channel, and then these features are combined using pointwise convolution, allowing the network to learn the feature relationships between different channels more precisely, thereby improving network performance. This embodiment uses the PatchGAN multi-task relative discriminator to perform relative real / fake judgment on a 27×27 feature map. The discriminator aims to distinguish between real and fake input samples, that is, to distinguish whether the sample conforms to the real data distribution, thereby guiding the generator to generate more realistic fake samples and increasing the generator's ability to deceive the discriminator. Assuming the skull image is a skull, the fake face image generated by the generator is called a skin fake, and the real face image is called a skin real. The real face in the preprocessed multi-paradigm facial beauty prediction dataset SCUT-FBP5500 is called a face. In order to retain more original information, skin fake, skin real, and face are added to skull in the channel dimension and then fed into the discriminator. The skin fake generated by the generator is judged as fake, and the real face image skin real and the real face are judged as real. Step 6: Train the parameters of the Skull2Skin-GAN network model (parameters are shown in Table 2), which includes the following steps: Step 6.1: First, perform non-penetrating data augmentation on the paired skull and face datasets. Then, train the Skull2Skin-GAN network model on the augmented paired skull and face datasets to learn the mapping between the image translation domain X and domain Y. The training set includes a total of 270 skull and face images. Traditional data augmentation techniques alter the distribution of training samples, causing the distribution of generated samples to deviate. This means data augmentation has "infiltrated" the generator. For example, adding noise to training samples, even though the dataset itself doesn't contain noise, results in a noisy distribution learned by the generator. Therefore, this embodiment employs methods such as... Figure 2 The non-penetrating data enhancement shown is achieved through pixel changes (x-flip, 90°). 。 Augmentation is performed on paired skull and face datasets using rotation and integer translation, geometric transformation, color transformation, image spatial filtering, additive noise, and shearing. This ensures that no augmentation transformation is infiltrated into the generated images. It can be seen that during discriminator training, all input samples are augmented, and during generator training, all input samples are also augmented. When all real and generated samples seen by the discriminator are augmented distributions, the generator must ensure that the generated sample distribution matches the real sample distribution before augmentation in order to match the augmented real data distribution. Therefore, as long as Augment is performed with a probability of p < 1, it is possible to train a generative adversarial network with a small number of samples and prevent the discriminator from overfitting. Step 6.2: Use the adversarial loss function Reconstruction losses and loss of perception of facial details Optimizing the image translation domain transformation problem includes the following steps: Step 6.2.1: Generate the adversarial loss function : Traditional generative adversarial networks (GANs) use JS divergence or KL divergence as the distance metric between the generator and discriminator, which can lead to vanishing or exploding gradients during gradient computation. WGAN, however, proposes using Wasserstein distance to measure the distance between the generator and discriminator and constrains the discriminator's parameters to ensure 1-Lipschitz continuity. Therefore, this embodiment uses the loss function of WGAN-GP as the generative adversarial loss function. This enables the discriminator to have 1-Lipschitz continuity and generates adversarial loss. Represented as: In the formula, λ is the weight of the gradient penalty term, set to 10. Represents the gradient. It is a real image and generating images The sampling results between them, where D represents the discriminator, This represents the maximum likelihood estimate of the parameters, where P and Q represent... and The image domain in which it is located; Step 6.2.2, Reconstructing Losses : In image translation, there is a relationship between the generator's input and output. If only the generative adversarial loss function is used to constrain the network, the generated result will lose the identity information of the input image. Therefore, this embodiment uses the L1 norm loss function to constrain the similarity between the input and output images, so that the generated image retains the identity information of the source image, improving the performance of the image translation task and reconstructing the loss. Represented as: In the formula, Represents the source domain image. Represents the target domain image. Indicates a generator; Step 6.2.3, Loss of perception of facial details : While reconstruction loss captures pixel-level errors in the image, enabling the generator to produce an output image similar to the target image, it cannot guarantee visual similarity between the generated image and the real image. Especially in small samples, using only reconstruction loss can lead to blurred facial details. Therefore, this embodiment introduces the multi-paradigm facial aesthetic prediction dataset SCUT-FBP5500 to calculate the facial detail perception loss. Represented as: In the formula, This represents the generated facial image. Represents a real human face image. This represents the feature extraction layer of a fixed network. Indicates the specified first Each network layer, This represents the output features of the layer; Step 6.2.4: Adopt the adversarial loss function Reconstruction losses and loss of perception of facial details Constructing the total loss function of the network structure , is represented as: In the formula, and The values ​​are 100 and 20 respectively, where G represents the generator and D represents the discriminator. This means fixing the parameters of D and optimizing the parameters of G to make... The value is the smallest. This means fixing the parameters of G and optimizing the parameters of D to make... The value is the largest; Step 7: Input the skull and facial images into the Skull2Skin-GAN network model to generate a two-dimensional facial image. This includes the following steps: Step 7.1: Input the skull and facial images into the generator of the Skull2Skin-GAN network model. The encoder converts the skull image of size 1×256×256 into the first feature map through a normal convolution with a kernel size of 4, a stride of 2 and padding of 1, and the LeakyReLU activation function. Step 7.2: The first feature map is processed by 6 depthwise separable convolutional modules to generate the second feature map. Each depthwise separable convolutional module consists of a depthwise convolution with a kernel size of 4, a stride of 2, and a padding size of 1, and a pointwise convolution with a kernel size of 1 and a stride of 1. The depthwise separable convolutional module also includes batch normalization and a ReLU6 activation function. After the second feature map is processed by a normal convolution with a kernel size of 4, a stride of 2, and a padding size of 1 and a ReLU6 activation function, it is converted into a third feature map of size 512×1×1. Step 7.3: The decoder converts the third feature map into an output image of the same size as the input skull and face images. Specifically, it first performs skip connections between each layer feature map and the corresponding feature map generated by the encoder to pass information and retain more detailed information. Then, it uses a normal transposed convolution with a kernel size of 4, a stride of 2, and a padding size of 1, and processes it with Tanh, sigmoid, or ReLU activation functions to convert the third feature map into a facial image of size 1×256×256. The decoder includes 7 depthwise separable transposed convolution modules. Each depthwise separable transposed convolution module consists of a depthwise transposed convolution with a kernel size of 4, a stride of 2, and a padding size of 1, and a pointwise convolution with a kernel size of 1 and a stride of 1. Step 8: Perform super-resolution repair on the 2D facial image generated in Step 7 to generate a realistic image; Because high-resolution images contain a great many features and details, more complex networks are needed to capture this information and generate realistic images. However, if trained on a small number of samples, the network struggles to learn enough features from the limited training samples. This results in the blurring of details in low-resolution images while the overall structure and features of the image remain intact. Therefore, this embodiment breaks down the task of directly generating high-resolution facial images into two steps: First, learning the overall features of the image in low-resolution images; second, performing super-resolution restoration on the low-resolution facial images using an image super-resolution network. This embodiment uses a pre-trained GFPGAN as the super-resolution inpainting network, which consists of two modules: a de-degradation module and a pre-trained generative adversarial network module, such as StyleGAN2. GFPGAN utilizes face priors for face image translation. Super-resolution translation enables the restoration of high-quality faces with rich geometry, texture, and color from low-quality faces with unknown degradation. Figure 5 As shown, the GFPGAN network consists of a de-degradation module and a pre-trained generative adversarial network module, which takes a low-quality face image as input. Output a high-quality restored image The image processed by GFPGAN is as follows Figure 6As shown.

[0025] like Figure 4 As shown, this embodiment selects four sets of skull and road surface images (the test set includes 30 images) from the test set, and translates them into facial images using the method of this invention and existing methods, respectively. The images are then compared with their corresponding Ground-Truth images. To ensure a fair comparison, the same dataset segmentation method, number of iterations, and data processing method are used. The results show that, under small sample conditions, although existing methods perform well on the training set, there is a significant gap between the facial images generated on the test set and the Ground-Truth images, indicating overfitting. Existing methods include Pix2Pix, CycleGAN, MUNIT, and CWT-GAN.

[0026] The Skull2Skin-GAN model generator of the craniofacial translation network in this embodiment replaces ordinary convolution with depthwise separable convolution to reduce the model size based on U-Net. It adopts a series of data augmentations in random non-penetrating data augmentation to prevent discriminator overfitting and image orientation disorder. The multi-paradigm facial aesthetic prediction dataset SCUT-FBP5500 is introduced to calculate the loss of facial feature detail perception. Finally, the GFPGAN network is introduced to perform super-resolution repair on the generated face. Experimental comparison shows that this embodiment can quickly and easily generate clear and high-fidelity facial images from skull images, and the images can be used for identity recognition.

[0027] Table 1 Experimental Configuration Table 2 Training Parameters

Claims

1. An intelligent craniofacial translation method, characterized in that, Includes the following steps: Step 1: Acquire CT images of the skull and face; Step 2: Perform image preprocessing, 3D reconstruction, and smoothing on the CT image data of the skull and face to obtain a complete 3D model of the skull and face; Step 3: Place the 3D model of the skull and face in the Frankfurt coordinate system and perform normalization. Step 4: In the Frankfurt coordinate system, the three-dimensional models of the skull and face are vertically mapped to the XOZ plane to obtain the front view images of the skull and face. Step 5: Introduce depthwise separable convolutions into the generator U-Net network of the GAN network to construct the Skull2Skin-GAN network model for skull and face translation. Step 6: Train the parameters of the Skull2Skin-GAN network model. The process is as follows: Step 6.1: Train the Skull2Skin-GAN network model on the skull and face paired datasets to learn the mapping between the image translation domain X and domain Y; Step 6.2: Use the adversarial loss function Reconstruction losses and loss of perception of facial details The overall loss function of the network structure for optimizing the image translation domain transformation problem is expressed as: In the formula, and The values ​​are 100 and 20 respectively, where G represents the generator and D represents the discriminator. This means fixing the parameters of D and optimizing the parameters of G to make... The value is the smallest. This means fixing the parameters of G and optimizing the parameters of D to make... The value is the largest; Step 6.2 specifically includes the following steps: Step 6.2.1: Use the loss function of WGAN-GP as the generative adversarial loss function. This enables the discriminator of the Skull2Skin-GAN network to have 1-Lipschitz continuity, generating adversarial loss. Represented as: In the formula, λ is the weight of the gradient penalty term, set to 10. Represents the gradient. It is a real image and generating images The sampling results between them, where D represents the discriminator, This represents the maximum likelihood estimate of the parameters, where P and Q represent... and The image domain in which it is located; Step 6.2.2: Constrain the similarity between the input and output images using the L1 norm loss function, so that the generated image retains the identity information of the source image, and reconstruct the loss. Represented as: In the formula, Represents the source domain image. Represents the target domain image. Indicates a generator; Step 6.2.3: Calculate the facial detail perception loss using the multi-paradigm facial aesthetic prediction dataset SCUT-FBP5500. To improve the visual similarity between generated images and real images, and reduce the loss of facial detail perception. Represented as: In the formula, This represents the generated facial image. Represents a real human face image. This represents the feature extraction layer of a fixed network. Indicates the specified first Each network layer, This represents the output features of the layer; Step 6.2.4: Adopt the adversarial loss function Reconstruction losses and loss of perception of facial details Constructing the total loss function of the network structure ; Step 7: Input the skull and facial images into the Skull2Skin-GAN network model to generate a two-dimensional facial image.

2. The intelligent craniofacial translation method according to claim 1, characterized in that, The specific process for acquiring skull and facial CT data in step 1 is as follows: using computed tomography (CT) technology, using... Fine scanning of thick sections of the skull using beam, gamma rays, or ultrasound yields several consecutive CT slice images of the skull and face.

3. The intelligent craniofacial translation method according to claim 1, characterized in that, The depthwise separable convolution in step 5 includes depthwise convolution and pointwise convolution. The depthwise convolution applies a convolution kernel to each channel of the input feature map to generate a feature map with the same depth as the input feature map. The pointwise convolution is a 1×1 ordinary convolution that maps the feature map generated by the depthwise convolution to a new feature space.

4. The intelligent craniofacial translation method according to claim 1, characterized in that, The discriminator used in the Skull2Skin-GAN network model is the PatchGAN multi-task relative discriminator.

5. The intelligent craniofacial translation method according to claim 1, characterized in that, Step 6.1 also includes random non-permeable data augmentation of the paired skull and face datasets.

6. The intelligent craniofacial translation method according to claim 5, characterized in that, The random non-penetrating data augmentation includes pixel transformation, geometric transformation, color transformation, image spatial filtering, additive noise, and cropping; The pixel changes include x-flip, 90... 。 Rotation and integer translation.

7. The intelligent craniofacial translation method according to claim 1, characterized in that, Step 7 specifically includes the following steps: Step 7.1: Input the skull and face images into the generator of the Skull2Skin-GAN network model. The encoder of the generator converts the input skull and face images into feature maps. The feature maps obtained in steps 7.2 and 7.1 are processed by depthwise separable convolution and then converted by the generator's decoder into an output image of the same size as the input skull and face images.

8. The intelligent craniofacial translation method according to claim 1, characterized in that, It also includes using a trained face super-resolution network GFPGAN to perform super-resolution inpainting on the two-dimensional facial image generated in step 7, thereby generating a realistic image.

Images