Craniofacial restoration method based on diffusion model

Abstract

The invention discloses a craniofacial restoration method based on a diffusion model, and the method comprises the steps: S1, collecting a three-dimensional medical image of a head, carrying out the standardization processing and biological information labeling of the three-dimensional medical image, and obtaining a skull-craniofacial data set; s2, training a PiPNet network by using the craniofacial image marked with the craniofacial key structure mark points to obtain a craniofacial mark point recognition network; s3, constructing a craniofacial restoration network, and training the craniofacial restoration network by using the skull-craniofacial data set to obtain a craniofacial restoration model; and S4, inputting the skull image and the corresponding biological information into the craniofacial restoration model, and outputting a restored craniofacial image. According to the method, personal identity features are comprehensively captured by integrating multi-modal information; a diffusion model is used as a backbone network, the dependence of a traditional method on training data distribution is broken through, and the quality of a recovery result is remarkably improved. And meanwhile, an editable text prompt mechanism is introduced, so that the uncertainty of skull-face mapping is effectively reduced.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer image processing, and in particular relates to a craniofacial restoration method based on a diffusion model. Background Art

[0002] In the fields of forensic identification of unidentified corpses and restoration of portraits of historical figures in archaeology, craniofacial reconstruction technology is the core means to restore individual facial features.

[0003] In one existing craniofacial restoration method, based on statistical principles, multiple reference craniofacial samples are first selected from a database according to the attributes of the target skull. These samples are then deformed and analyzed using principal component analysis to produce a series of potential reconstruction results. Another craniofacial restoration method is based on GAN. During GAN training, adversarial training is performed using a generator and a discriminator to force the generator to learn the mapping relationship between the skull and the craniofacial face. Skull images and individual attribute values ​​(age, gender) are used as input in the generator, and additional network structures are designed to ensure the authenticity of the reconstructed details. A multi-scale identification method is used in the discriminator to achieve global consistency of identity and retention of identity features. During inference, the skull image and attribute values ​​are input into the generator to directly obtain the craniofacial restoration result.

[0004] However, the above-mentioned existing technologies have significant limitations: First, traditional statistical methods rely on average craniofacial templates, which make it difficult to characterize the complex mapping relationship between individual skull morphology and facial features, resulting in distortion in the restoration of details such as soft tissue thickness and facial contours; Second, although the GAN-based deep learning method improves the restoration accuracy, it is limited by the distribution of training data and is prone to geometric distortion (such as facial disproportion) or texture blurring (such as missing wrinkle details), and it is difficult to overcome the uncertainty in skull estimation, resulting in insufficient matching between the restoration results and the real facial features. Summary of the Invention

[0005] In response to the above-mentioned deficiencies in the prior art, the diffusion model-based craniofacial restoration method provided by the present invention solves the problem that traditional statistical methods are difficult to accurately characterize the complex mapping relationship between individual skull morphology and facial features, and deep learning methods are limited by the distribution of training data and have difficulty in overcoming the uncertainty in skull estimation, which in turn leads to insufficient matching between the restoration results and the real facial features.

[0006] In order to achieve the above-mentioned object of the invention, the technical solution adopted by the present invention is: a craniofacial restoration method based on a diffusion model, comprising the following steps:

[0007] S1. Collect 3D medical images of the head, perform standardization and bio-information annotation on them, and obtain a skull-cranial face dataset;

[0008] S2. Use the craniofacial images marked with key craniofacial structure landmarks to train the PiPNet network to obtain a craniofacial landmark recognition network;

[0009] S3. Construct a craniofacial reconstruction network based on the craniofacial landmark recognition network, and train it using the skull-craniofacial dataset to obtain a craniofacial restoration model.

[0010] S4. Input the skull image and its corresponding biological information into the craniofacial reconstruction model, and output the restored craniofacial image.

[0011] Furthermore, in step S1, the annotated biological information includes gender, population, age group, and obesity level.

[0012] Furthermore, in step S3, the craniofacial reconstruction network includes a skull feature capture module, a biological information processing module, and a craniofacial feature capture module;

[0013] The skull feature capture module includes a diffusion model and a ContorlNet network, which is used to output a noise prediction result containing skull features;

[0014] The biological information processing module includes a text processing unit and a text encoder, which are used to perform standardization processing on the biological information and encode and input it into the skull feature capture module;

[0015] The craniofacial feature capture module includes a craniofacial landmark point recognition network and a discriminator network, which is used to capture craniofacial contour and texture features based on a predicted craniofacial image restored from a noise prediction result containing skull features.

[0016] Furthermore, the process of the skull feature capture module outputting the skull feature prediction result is as follows:

[0017] The craniofacial image and random noise are used as the input of the diffusion model, and the skull image is used as the input of the ContorlNet network. The outputs of the diffusion model and the ContorlNet network are added according to the corresponding feature layers, and the skull feature prediction results are output through the diffusion model.

[0018] Furthermore, the biological information processing module processes the biological information as follows:

[0019] The annotated biological information is processed into a standardized text form by the text processing unit, encoded by the text encoder, and then input into the diffusion model and ContorlNet network of the skull feature capture module through the cross attention layer.

[0020] Furthermore, the process of the craniofacial feature capture module capturing the contour and texture features of the craniofacial is as follows:

[0021] The noise prediction result output by the skull feature capture module is restored to a noise-free skull face prediction image through the forward diffusion formula and decoder of the diffusion model;

[0022] Input the predicted craniofacial image into the craniofacial landmark recognition network to obtain the corresponding predicted craniofacial landmarks, and then capture the craniofacial contour features;

[0023] The predicted craniofacial image is concat- ed with the real skull image and input into the discriminator network. The feature vector of the middle layer of the discriminator network is extracted as the captured craniofacial texture feature.

[0024] Furthermore, in step S3, when training the craniofacial reconstruction network, based on the adversarial training concept, the craniofacial reconstruction network is divided into a generator and a discriminator and trained separately;

[0025] The generator is used to generate predicted craniofacial images, and its loss function for:

[0026]

[0027] Where, represents the noise loss of the skull feature capture module and the biological information processing module, ∈ represents the random noise input to the diffusion model, ∈ θ represents the skull feature prediction result output by the skull feature capture module, z t represents the facial feature representation of the craniofacial image after being encoded by the encoder and denoised to time step t, c p Represents the standardized text form of biological information, c i Represents a skull image;

[0028] Indicates the loss of craniofacial contour features, represents the craniofacial landmark recognition network, x0 represents the craniofacial image, represents the predicted craniofacial image;

[0029] represents the confidence loss, represents the discriminator network;

[0030] represents the craniofacial texture feature loss, N represents the number of intermediate layers of the discriminator network, represents the feature representation of the jth layer of the discriminator network, and Cosine(·) represents the Cosine similarity calculation function;

[0031] λ1, λ2, λ3 represent the weight coefficients of different loss terms;

[0032] The discriminator is used to determine whether the input data is real data or fake data generated by the generator. The discrimination result is used to guide the training process of the generator, and its loss function is

[0033]

[0034] Furthermore, the step S4 includes the following sub-steps:

[0035] S41. Introducing the CFG idea into the denoising step of the diffusion model;

[0036] S42. Use the DDIM sampling method to accelerate the generation process of the diffusion model and set the guidance weight of each network block in the ContorlNet network;

[0037] S43, setting the gradient guidance process of the discriminator network;

[0038] S44. After standardization and encoding, the biological information is injected into the diffusion model and Controlnet network respectively, and the skull image is input into the ContorlNet network. Based on S41 to S43, the DDIM denoising process is gradually performed in the craniofacial restoration model starting from pure noise, and then the restored craniofacial image is output.

[0039] Furthermore, the step S43 is specifically as follows:

[0040] The predicted craniofacial image obtained at each step in the craniofacial reconstruction model is input into the discriminator network to calculate After being encoded by the encoder, it is added to the sampling result of the current sampling step, so that the craniofacial restoration image corresponding to this step is gradually aligned with the real craniofacial image; represents the gradient, represents the discriminator network, c i represents the skull image, Represents the predicted craniofacial image.

[0041] The beneficial effects of the present invention are:

[0042] (1) High quality of restored images: The general image quality evaluation indicators LPIPS, SSIM, and PSNR were used as measurement standards to evaluate the quality of the craniofacial images restored using the method of the present invention. Compared with the traditional method, the restored image quality obtained by using the method of the present invention is the best.

[0043] (2) Strong identity feature preservation: The craniofacial images restored using the method of the present invention achieve the highest identity feature retrieval accuracy compared with traditional methods.

[0044] (3) Simple condition control and diverse results: The present invention uses standardized bioinformation text as the control condition, which is simple to operate and can generate diverse individual craniofacial reconstruction images that conform to biological characteristics based on different texts.

[0045] (4) High credibility: The visualization results of craniofacial reconstruction under different conditions (different random seeds, texts and angles, and DDIM sampling) show that in any case, the craniofacial reconstruction images obtained using the method of the present invention can maintain the identity characteristics of the individual and have high credibility. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 Flowchart of the craniofacial restoration method based on the diffusion model provided by the present invention.

[0047] Figure 2 This is a system architecture diagram for the craniofacial restoration provided by the present invention.

[0048] Figure 3 This is a schematic diagram of the craniofacial image landmark points provided by the present invention.

[0049] Figure 4 This is a schematic diagram of the image quality visualization results provided by the present invention.

[0050] Figure 5 This is a schematic diagram of the restoration results provided by the present invention that vary with age, gender, and obesity.

[0051] Figure 6 This is a schematic diagram of the results of different random seeds provided by the present invention.

[0052] Figure 7 A comparison chart of the results with and without text provided by the present invention.

[0053] Figure 8 The restoration results of the same cranial face at different angles provided by the present invention are as follows:

[0054] Figure 9 This is a diagram of the restoration results of different steps of DDIM provided by the present invention. DETAILED DESCRIPTION

[0055] The specific embodiments of the present invention are described below to facilitate understanding of the present invention by those skilled in the art. However, it should be clear that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the appended claims, these changes are obvious, and all inventions and creations utilizing the concepts of the present invention are protected.

[0056] The embodiment of the present invention provides a method for craniofacial reconstruction based on a diffusion model, such as Figure 1-2 As shown, the following steps are included:

[0057] S1. Collect 3D medical images of the head, perform standardization and bio-information annotation on them, and obtain a skull-cranial face dataset;

[0058] S2. Use the craniofacial images marked with key craniofacial structure landmarks to train the PiPNet network to obtain a craniofacial landmark recognition network;

[0059] S3. Construct a craniofacial reconstruction network based on the craniofacial landmark recognition network, and train it using the skull-craniofacial dataset to obtain a craniofacial restoration model.

[0060] S4. Input the skull image and its corresponding biological information into the craniofacial reconstruction model, and output the restored craniofacial image.

[0061] In step S1 of the embodiment of the present invention, a three-dimensional medical image is obtained by performing a multi-slice computed tomography scan on the head of a real individual; the obtained three-dimensional image is rotated to obtain a two-dimensional skull-to-skull face from multiple angles, and the image is centrally cropped to obtain 256*256 standardized image data.

[0062] In this embodiment, the biological information in the standardized image data is labeled, including gender, population, age group, and fatness level; among them, individual gender ("female" or "male") and population information ("Asian") retain the true value, age is distinguished by age group (0-13 years old is labeled as "1", 14-29 years old is labeled as "2", 30-49 years old is labeled as "3", 50-64 years old is labeled as "4", and 65 years old and above is labeled as "5"), and fatness is obtained by manual labeling (from thin to fat, it is labeled as "verythin", "thin", "slightly thin", "normal", and "slightly fat" in order).

[0063] In this embodiment, a total of 72,016 pairs of skull-to-cranial faces of 4,802 individuals were obtained according to the above method, and a skull-to-cranial face dataset was constructed, of which 90% were used for training and 10% were used for testing.

[0064] In step S2 of the embodiment of the present invention, several craniofacial images from different angles are used, and 106 landmarks are manually annotated on each image to identify key craniofacial structures, including eyebrows, eyes, nose, mouth, and facial contour information. Figure 3 Representative examples of annotation results are shown.

[0065] Furthermore, the labeled facial annotation points are used to train the PiPNet network to obtain a facial landmark point recognition network; among them, the PiPNet (Pixel-in-pixel net: Towards efficient facial landmark detection in the wild) network is trained to obtain a model that can capture the contour landmark points of key facial structures (eyebrows, eyes, nose, mouth and face shape), which can be used for coarse-grained facial similarity comparison.

[0066] In step S3 of the embodiment of the present invention, Figure 2 In the craniofacial reconstruction network, there are skull feature capture modules, biological information processing modules and craniofacial feature capture modules;

[0067] The skull feature capture module includes a diffusion model and a ContorlNet network, which is used to output a noise prediction result containing skull features;

[0068] The biological information processing module includes a text processing unit and a text encoder, which are used to perform standardization processing on the biological information and encode and input it into the skull feature capture module;

[0069] The craniofacial feature capture module includes a craniofacial landmark point recognition network and a discriminator network, which is used to capture craniofacial contour and texture features based on a predicted craniofacial image restored from a noise prediction result containing skull features.

[0070] In this embodiment, the process of the skull feature capture module outputting the noise prediction result is as follows:

[0071] The craniofacial image x0 and random noise As the input of the diffusion model, the skull image c i As the input of the ContorlNet network, the output of the diffusion model and the ContorlNet network are added according to the corresponding feature layer, and the noise prediction result ∈ containing the skull feature is output by the diffusion model θ .

[0072] In this embodiment, the diffusion model (Denoising Diffusion Probabilistic Models) is a generative model based on a Markov chain. It generates new data by gradually adding Gaussian noise to the data (forward diffusion process) and then learning the reverse denoising process. Specifically, this embodiment uses SD 1.5 (Stable Diffusion 1.5) as the pre-trained diffusion model. SD 1.5 is a text-to-image generation model based on the diffusion model. By introducing an encoder and decoder, the diffusion and denoising processes are performed in the latent space, improving computational efficiency. At the same time, a text encoder is introduced to input the encoded text into the diffusion model through a cross-attention layer, thereby greatly improving its controllable generation capability.

[0073] In this embodiment, the ContorlNet (Adding Conditional Control to Text-to-Image Diffusion Models) network is an efficient network structure used to add additional image control conditions to the pre-trained generative model. This allows the model to generate images that are more consistent with expectations while retaining prior knowledge, achieving conversion from the source domain to the target domain and significantly improving the controllability of the generative model.

[0074] In this embodiment, the biological information processing module processes the biological information as follows:

[0075] The annotated biological information is processed into a standardized text form by the text processing unit, encoded by the text encoder, and then input into the diffusion model and ContorlNet network of the skull feature capture module through the cross attention layer.

[0076] In a specific example of this embodiment, the standardized text form of the corresponding biometric information is "3D facialscan, Asian female, age level 3, slightly fat face", where "3D facial scan" is a keyword specially designed to identify the craniofacial image, "Asian" represents population information, "female" represents gender information, "agelevel 3" represents age information, and "slightly fat" represents fatness information.

[0077] In this embodiment, the text encoder converts text data into feature representations that can be processed by a machine learning model by encoding standardized text.

[0078] In this embodiment, the process of the craniofacial feature capture module to capture craniofacial contour and texture features is as follows:

[0079] The noise prediction result ∈ θ , through the forward diffusion formula and decoder of the diffusion model, it is restored to the craniofacial prediction image without noise (time step t = 0)

[0080] The predicted craniofacial image Input into the craniofacial landmark recognition network to obtain the corresponding predicted craniofacial landmark points, and then capture the craniofacial contour features;

[0081] The predicted craniofacial image Perform concat operation with the real skull image x0 and input it into the discriminator network to extract the feature vector of the middle layer of the discriminator network as the captured cranial facial texture feature.

[0082] In step S3 of the embodiment of the present invention, when training the craniofacial reconstruction network, based on the adversarial idea, the network is divided into a generator and a discriminator and trained separately;

[0083] The generator is used to generate predicted craniofacial images, and its loss function is for:

[0084]

[0085] Where, represents the noise loss of the skull feature capture module and the biological information processing module, ∈ represents the random noise input to the diffusion model, ∈ θ represents the skull feature prediction result output by the skull feature capture module, z t represents the facial feature representation of the craniofacial image after being encoded by the encoder and denoised to time step t, c p Represents the standardized text form of biological information, c i Represents a skull image;

[0086] Indicates the loss of craniofacial contour features, represents the craniofacial landmark recognition network, x0 represents the craniofacial image, represents the predicted craniofacial image;

[0087] represents the confidence loss, represents the discriminator network;

[0088] represents the craniofacial texture feature loss, N represents the number of intermediate layers of the discriminator network, represents the feature representation of the jth layer of the discriminator network, and Cosine(·) represents the Cosine similarity calculation function;

[0089] λ1, λ2, λ3 represent the weight coefficients of different loss terms;

[0090] Among them, when calculating the noise loss of the skull feature capture module and the biological information processing module, the skull image x0 and the random noise As the input of the diffusion model, the skull image c i As the input of ContorlNet network, the text c p As the input of the text encoder, it is injected into the diffusion model and ContorlNet network respectively, and the outputs of the diffusion model and ContorlNet network are added according to the corresponding feature layers, and the noise prediction result ∈ is output by the diffusion model. θ , and then calculate the noise loss.

[0091] When calculating the craniofacial contour feature loss, the craniofacial image is predicted The predicted and real craniofacial landmarks are input into the craniofacial landmark recognition network respectively to obtain the predicted and real craniofacial landmarks, and then the craniofacial contour feature loss is calculated based on them.

[0092] When calculating the confidence loss, the craniofacial image is predicted With real skull pictures c i Perform the concat operation and input it into the discriminator network to obtain the confidence result value, and then calculate the confidence loss.

[0093] When calculating the craniofacial texture feature loss, the predicted craniofacial image With the real craniofacial image x0, with the real skull image c i Perform the concat operation and input it into the discriminator network to extract the feature vector of the middle layer of the discriminator network (a total of N layers). The craniofacial texture feature loss is obtained by calculating the Cosine similarity of the feature vectors in the corresponding feature layer.

[0094] In this embodiment, when training the generator, the CFG idea is adopted, and the text is set to empty with a probability of 0.5; based on the above loss function, the weights of different loss items are manually set to ensure that each part is fully trained.

[0095] Among them, CFG (Classifier-Free Guidance) represents a technology used to improve the generation quality of diffusion models. It achieves a better balance between the diversity and authenticity of generated images by integrating conditional information within the model without relying on external classifiers.

[0096] In this embodiment, the discriminator is used to determine whether the input data is real data or fake data generated by the generator. The discrimination result is used to guide the training process of the generator, and its loss function is

[0097]

[0098] In this embodiment, when training the discriminator, the predicted craniofacial image and the real craniofacial image are concat-operated with the real skull image, respectively, and input into the discriminator network to obtain the confidence result value, calculate the above-mentioned loss function value, and then complete the training.

[0099] Step S4 of the embodiment of the present invention includes the following sub-steps:

[0100] S41. Introducing the CFG idea into the denoising step of the diffusion model;

[0101] S42. Use the DDIM sampling method to accelerate the generation process of the diffusion model and set the guidance weight of each network block in the ContorlNet network;

[0102] S43, setting the gradient guidance process of the discriminator network;

[0103] S44. After standardization and encoding, the biological information is injected into the diffusion model and Controlnet network respectively, and the skull image is input into the ContorlNet network. Based on S41 to S43, the DDIM denoising process is gradually performed in the craniofacial restoration model starting from pure noise, and then the restored craniofacial image is output.

[0104] In step S41 of this embodiment, when the CFG concept is introduced, γ is set to 9.0, taking into account the situations with and without text.

[0105] In step S42 of this embodiment, when DDIM sampling is used, the sampling step is set to 50 and η = 1.0. Specifically, DDIM (Denoising Diffusion Implicit Models) is a sampling strategy that accelerates the generation process during the diffusion model inference phase. Through implicit modeling and non-Markov updates, it significantly reduces the number of sampling steps while ensuring generation quality, thereby improving inference efficiency.

[0106] In step S42 of this embodiment, in a specific example, the guidance weights of network blocks of different sizes in the ContorlNet network are set to [1.4, 1.4, 1.4, 1.2, 1.2, 1.2, 0.8, 0.8, 0.8, 0.8, 0.8, 0.8, 0.8] from shallow to deep.

[0107] Step S43 of this embodiment is specifically as follows:

[0108] The predicted craniofacial image obtained at each step in the craniofacial reconstruction model is input into the discriminator network to calculate After being encoded by the encoder, it is added to the sampling result of the current sampling step, so that the craniofacial restoration image corresponding to this step is gradually aligned with the real craniofacial image; represents the gradient, represents the discriminator network, c i represents the skull image, Represents the predicted craniofacial image.

[0109] In an embodiment of the present invention, an experimental example of verifying the effect of the above-mentioned craniofacial restoration method is provided.

[0110] In this embodiment, the general indicators for evaluating image quality, LPIPS, SSIM, and PSNR, are used as measurement standards (wherein, LPIPS (Learned Perceptual Image Patch Similarity) is a perceptual similarity indicator based on deep learning, used to measure the similarity of two images in high-level semantics; SSIM (Structural Similarity Index Measure) is an image quality evaluation indicator used to measure the structural similarity of two images, focusing on the differences in brightness, contrast, and structure; PSNR (Peak Signal-to-Noise Ratio) is an image quality evaluation indicator based on mean square error to measure the degree of signal distortion between two images), where the lower the LPIPS value, the better, and the higher the SSIM and PSNR values, the better. The results are shown in Table 1, which proves that the solution of the present invention achieves the best image quality; Figure 4 The visualization results shown further prove this point.

[0111] Table 1: Image quality results

[0112] method LPIPS SSIM PSNR Pix2Pix 0.270 0.632 <![CDATA[ 18.54 ]]> CycleGAN <![CDATA[ 0.252 ]]> 0.653 18.38 CR-GAN 0.333 <![CDATA[ 0.654 ]]> 17.64 ContorlNet 0.288 0.529 18.49 Ours 0.226 0.701 19.86

[0113] In this example, 401 frontal craniofacial images were used to search 4802 real frontal craniofacial images, and the proportion of correct results appearing in the top one and top five was calculated to verify the accuracy of the generated results. When measuring the similarity between craniofacial images, a vector representation of the craniofacial image was first obtained using feature extraction methods (PiPNet, Discriminator, SSIM, Arcface), and then the Euclidean distance between the vectors was calculated. The results are shown in Table 2, which shows that the method of the present invention achieves the task of restoring individual identity features and obtains the highest retrieval accuracy compared to previous methods.

[0114] Table 2: Identity retrieval results

[0115]

[0116] In this embodiment, the standardized text is used as the control condition, and the generated individual craniofacial restoration image is as follows: Figure 5 As shown, it can be seen that the method of the present invention uses standardized text as a control condition, is simple to operate, and can generate diverse individual result graphs that conform to biological characteristics based on different texts.

[0117] In this embodiment, the visualization results of craniofacial reconstruction using the method of the present invention are given under different conditions, as follows:

[0118] (1) Different random seeds: In order to ensure the reproducibility of the generated results, in addition to consistent input, the same random seed must be set. In order to avoid the uncertainty caused by different random seeds, the present invention verifies the generation results of the same skull with different random seeds (0, 1000, 10000, 6000) as follows: Figure 6 As shown, the generated results show that the restored individuals are highly similar.

[0119] (2) Different texts: The text setting in the method of the present invention gives a stronger constraint to the generation process. However, in practical applications, the individual biological information estimated based on the skull image is uncertain. Therefore, the present invention explores the case without text prompts, and the generated results are as follows: Figure 7 As shown, the results show that the present invention can still restore the identity information of an individual in the absence of text, further proving the reliability of the results.

[0120] (3) Different angles: In the absence of a real craniofacial image, the generated results of the same individual at different angles can indirectly prove the accuracy of the generated results. The present invention tests this situation and generates results that highly preserve individual identity characteristics, such as Figure 8 shown.

[0121] (4) DDIM sampling process: DDIM sampling is deterministic sampling, but setting η = 1.0 introduces randomness. Therefore, the present invention verifies the identity preservation ability during the sampling process. The results are as follows: Figure 9 As shown, from left to right are the noise-free restoration results under different sampling steps (50, 40, 30, 20, 10, 0). The results show that the sampling process also retains a high degree of individual identity characteristics.

[0122] In the above process, the present invention explores the results of craniofacial reconstruction under different circumstances, such as Figures 6 to 9 The visualization results show that under any circumstances, the method of the present invention can highly maintain the identity characteristics of the individual, further proving that the credibility of the craniofacial restoration results of the method of the present invention is very high.

[0123] The present invention comprehensively captures individual identity features by integrating multimodal information such as skull structure, facial contour landmarks, skin texture features, and biometric features (age, gender, and population); using a pre-trained diffusion model as the backbone network, it breaks through the traditional method's dependence on the distribution of training data and significantly improves the quality of the restoration results. At the same time, an editable text prompt mechanism is introduced to achieve cross-modal alignment of skull images and text information, dynamically adjust the generation path to adapt to the differences in individual features, and effectively reduce the uncertainty of skull-face mapping. Ultimately, a multi-feature craniofacial restoration image containing different ages, genders, and fatness conditions is generated, providing richer feature references for forensic identification and providing technical support for the reconstruction of archaeological historical figures with both scientific and artistic expression.

[0124] Specific embodiments are used in the present invention to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core ideas. At the same time, for those skilled in the art, according to the ideas of the present invention, there may be changes in the specific implementation methods and application scopes. In summary, the contents of this specification should not be understood as limiting the present invention.

[0125] Those skilled in the art will appreciate that the embodiments described herein are intended to help readers understand the principles of the present invention, and it should be understood that the scope of protection of the present invention is not limited to such specific descriptions and embodiments. Those skilled in the art can make various other specific variations and combinations based on the technical teachings disclosed in the present invention without departing from the essence of the present invention, and such variations and combinations are still within the scope of protection of the present invention.

Claims

1. A method for craniofacial reconstruction based on a diffusion model, characterized in that: The following steps are involved: S1. Collect 3D medical images of the head, perform standardization and bio-information annotation on them, and obtain a skull-cranial face dataset; S2. Use the craniofacial images marked with key craniofacial structure landmarks to train the PiPNet network to obtain a craniofacial landmark recognition network; S3. Construct a craniofacial reconstruction network based on the craniofacial landmark recognition network, and train it using the skull-craniofacial dataset to obtain a craniofacial restoration model. S4. Input the skull image and its corresponding biological information into the craniofacial reconstruction model, and output the restored craniofacial image.

2. The method for craniofacial reconstruction based on a diffusion model according to claim 1, characterized in that: In step S1, the annotated biological information includes gender, population, age group, and obesity level.

3. The method for craniofacial reconstruction based on diffusion model according to claim 1, characterized in that: In step S3, the craniofacial reconstruction network includes a skull feature capture module, a biological information processing module, and a craniofacial feature capture module; The skull feature capture module includes a diffusion model and a ContorlNet network, which is used to output a noise prediction result containing skull features; The biological information processing module includes a text processing unit and a text encoder, which are used to perform standardization processing on the biological information and encode and input it into the skull feature capture module; The craniofacial feature capture module includes a craniofacial landmark point recognition network and a discriminator network, which is used to capture craniofacial contour and texture features based on a predicted craniofacial image restored from a noise prediction result containing skull features.

4. The method for craniofacial reconstruction based on diffusion model according to claim 3, characterized in that: The process of the skull feature capture module outputting the skull feature prediction result is as follows: The craniofacial image and random noise are used as the input of the diffusion model, and the skull image is used as the input of the ContorlNet network. The outputs of the diffusion model and the ContorlNet network are added according to the corresponding feature layers, and the skull feature prediction results are output through the diffusion model.

5. The method for craniofacial reconstruction based on diffusion model according to claim 3, characterized in that: The process of biological information processing by the biological information processing module is as follows: The annotated biological information is processed into a standardized text form by the text processing unit, encoded by the text encoder, and then input into the diffusion model and ContorlNet network of the skull feature capture module through the cross attention layer.

6. The method for craniofacial reconstruction based on diffusion model according to claim 3, characterized in that: The process of the craniofacial feature capture module capturing the contour and texture features of the craniofacial is as follows: The noise prediction result output by the skull feature capture module is restored to a noise-free skull face prediction image through the forward diffusion formula and decoder of the diffusion model; Input the predicted craniofacial image into the craniofacial landmark recognition network to obtain the corresponding predicted craniofacial landmarks, and then capture the craniofacial contour features; The predicted craniofacial image is concat- ed with the real skull image and input into the discriminator network. The feature vector of the middle layer of the discriminator network is extracted as the captured craniofacial texture feature.

7. The method for craniofacial reconstruction based on diffusion model according to claim 3, characterized in that: In step S3, when training the craniofacial reconstruction network, based on the adversarial training concept, the craniofacial reconstruction network is divided into a generator and a discriminator and trained separately; The generator is used to generate predicted craniofacial images, and its loss function for: Where, represents the noise loss of the skull feature capture module and the biological information processing module, ∈ represents the random noise input to the diffusion model, ∈ θ represents the skull feature prediction result output by the skull feature capture module, z t represents the facial feature representation of the craniofacial image after being encoded by the encoder and denoised to time step t, c p Represents the standardized text form of biological information, c i Represents a skull image; Indicates the loss of craniofacial contour features, represents the craniofacial landmark recognition network, x0 represents the craniofacial image, represents the predicted craniofacial image; represents the confidence loss, represents the discriminator network; represents the craniofacial texture feature loss, N represents the number of intermediate layers of the discriminator network, represents the feature representation of the jth layer of the discriminator network, and Cosine(·) represents the Cosine similarity calculation function; λ1, λ2, λ3 represent the weight coefficients of different loss terms; The discriminator is used to determine whether the input data is real data or fake data generated by the generator. The discrimination result is used to guide the training process of the generator, and its loss function is 8. The method for craniofacial reconstruction based on diffusion model according to claim 3, characterized in that: The step S4 comprises the following sub-steps: S41. Introducing the CFG idea into the denoising step of the diffusion model; S42. Use the DDIM sampling method to accelerate the generation process of the diffusion model and set the guidance weight of each network block in the ContorlNet network; S43, setting the gradient guidance process of the discriminator network; S44. After standardization and encoding, the biological information is injected into the diffusion model and Controlnet network respectively, and the skull image is input into the ContorlNet network. Based on S41 to S43, the DDIM denoising process is gradually performed in the craniofacial restoration model starting from pure noise, and then the restored craniofacial image is output.

9. The method for craniofacial reconstruction based on diffusion model according to claim 8, characterized in that: The step S43 is specifically as follows: The predicted craniofacial image obtained at each step in the craniofacial reconstruction model is input into the discriminator network to calculate After being encoded by the encoder, it is added to the sampling result of the current sampling step, so that the craniofacial restoration image corresponding to this step is gradually aligned with the real craniofacial image; represents the gradient, represents the discriminator network, c i represents the skull image, Represents the predicted craniofacial image.

Images