An interpretable craniomaxillofacial rare disease diagnosis model, a diagnosis method and an electronic device

Abstract

This invention discloses an interpretable diagnostic model, method, and electronic device for rare craniofacial diseases. The model includes a medical vision-language feature encoding network, a supervised hierarchical contrastive learning module, a KL regularized progressive unfreezing module, and an interpretable diagnostic output module. First, image-text pairs are constructed. By introducing disease semantic priors, an adaptive soft-boundary contrastive learning loss function is built, which tolerates visual overlap while preserving the feature manifold topology of phenotypic similar diseases. Second, KL divergence is used to monitor the drift of feature distribution between layers, and progressive unfreezing of the model is achieved through trust domain constraints. This invention effectively avoids catastrophic forgetting during model fine-tuning, improving the diagnostic accuracy of rare craniofacial diseases while outputting interpretable attention heatmaps and facial feature descriptions that are precisely aligned with pathological features, providing an efficient and reliable technical solution for early clinical screening and assisted diagnosis.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, specifically to an interpretable diagnostic model, diagnostic method, and electronic device for rare craniofacial diseases. Background Technology

[0002] The diagnosis of rare craniofacial diseases relies on the accurate identification of subtle facial deformities such as wide-set eyes and abnormal philtrum. In clinical practice, due to the difficulty of manual identification, misdiagnosis and missed diagnosis are common. In recent years, with the rapid development of deep learning technology, computer-aided diagnosis (CAD) has become a key technology for genetic counseling and early screening for rare diseases.

[0003] In early research, methods based on convolutional neural networks (CNNs) dominated. For example, models such as DeepGestalt, pre-trained on large-scale face datasets and transferred to the medical field, achieved significant results in the identification of specific genetic syndromes. However, these CNN-based diagnostic models heavily rely on large-scale labeled data, while rare disease data typically exhibits a long-tailed distribution, leading to a significant decrease in diagnostic accuracy for rare disease categories with few samples, and failing to provide interpretable clinical evidence for the diagnostic results.

[0004] With the rise of Multimodal Large Language Models (MLLMs) or Visual-Language Models (VLMs), researchers have begun to explore the use of these models' powerful generalization capabilities and rich semantic knowledge for medical diagnosis. Existing general-purpose VLMs, by aligning image and text features, can handle more complex reasoning tasks. However, when dealing with the subtle malformation phenotypes of rare craniofacial diseases, general-purpose MLLMs exhibit significant limitations: firstly, they are prone to attention drift, where the model focuses on irrelevant areas such as the image background rather than the pathological features of the craniofacial deformity; secondly, they are prone to generating hallucinations, producing diagnostic descriptions that do not match the actual phenotype in the image. The core reason for these problems is that general-purpose MLLMs do not incorporate specialized medical prior knowledge of craniofacial diagnosis.

[0005] To improve the feature extraction capabilities of models with limited samples, contrastive learning has been widely adopted. Standard contrastive learning methods (such as SimCLR and MoCo) aim to maximize the uniformity between samples of different classes. However, these standard contrastive learning strategies are not applicable in the diagnosis of craniofacial rare diseases. Many genetic syndromes exhibit hierarchical relationships in biology, and their phenotypic features also have inherent visual overlap. Existing contrastive learning methods tend to over-separate these phenotypically similar syndromes, disrupting the inherent association structure of disease features and causing the model to be unable to distinguish between different phenotypic rare disease types.

[0006] Furthermore, to adapt large models to medical diagnostic tasks, low-rank adaptation (LoRA) and other parameter-efficient fine-tuning (PEFT) techniques have become mainstream solutions. However, when fine-tuning on extremely small sample data of rare diseases, existing LoRA methods have a serious risk of catastrophic forgetting: in the process of fitting a small amount of rare disease data, the model will destroy the general visual-language alignment ability obtained in the pre-training stage, and there is a lack of an effective mechanism to restrict parameter updates to a reliable feature distribution range. Summary of the Invention

[0007] To address the aforementioned issues, this invention provides an interpretable diagnostic model, diagnostic method, and electronic device for rare craniofacial diseases. It constructs a soft-boundary contrastive learning module using prior medical knowledge to maintain the manifold topology of similar disease features; it utilizes KL divergence to monitor feature distribution drift and achieves fine-tuning of trust domain constraints, ultimately improving diagnostic accuracy while ensuring model generalization ability and diagnostic interpretability.

[0008] According to a first aspect of the present disclosure, an interpretable diagnostic model for rare craniofacial diseases is provided, comprising a medical visual-language feature encoding network, a supervised hierarchical contrastive learning module, a KL regularized progressive thawing module, and an interpretable diagnostic output module, wherein... The medical vision-language feature encoding network is used to extract high-order multimodal features and achieve cross-modal alignment. It adopts a dual-tower structure with visual encoding branch and text encoding branch, and outputs normalized visual feature vector and text feature vector respectively. The supervised hierarchical contrastive learning module includes a semantic similarity calculation unit and an adaptive soft boundary loss calculation unit. The semantic similarity calculation unit is used to construct a semantic relationship graph between diseases using human phenotypic ontology, generating a multidimensional semantic similarity matrix. The adaptive soft boundary loss calculation unit is used to dynamically adjust the exclusion boundary of contrastive learning so that the model retains the topological proximity of the disease in the feature space. The KL regularized progressive unfreezing module is used for monitoring model parameter updates. It includes a distribution monitor and a dynamic unfreezing controller. The distribution monitor models the hidden state of each network layer as a multivariate Gaussian distribution. The dynamic unfreezing controller sets a dynamic trust domain threshold. When the distribution drift exceeds the threshold, it automatically triggers a regularization penalty mechanism to limit the parameter update amplitude. The interpretable diagnostic output module is used to generate the final diagnostic results and visualization basis, including a classification head, a visualization generator, and a large language model module. The classification head concatenates the aligned visual feature vectors and text feature vectors, and outputs the predicted probability of each disease category through an activation function. The visualization generator uses visualization technology to trace back the gradient information of the classification layer, calculates the spatial weights of the last convolutional layer, and generates a pathological feature attention heatmap that highlights the lesion area. The large language module outputs the text information of the pathological feature attention heatmap features, intuitively describing the diagnostic basis.

[0009] In some embodiments, the visual coding branch utilizes a pre-trained convolutional neural network as the backbone network to process the pre-processed standardized facial image and output a processed image. Normalized visual feature vectors The text encoding branch utilizes a pre-trained medical large language model as the backbone network to process clinical phenotype texts and output the results. Normalized text feature vectors .

[0010] In some embodiments, the interpretable diagnostic output module satisfies at least one of the following conditions: (1) Output the predicted probability of each disease category through the Sigmoid activation function; (2) The visualization generator is built based on the Grad-CAM algorithm; (3) Output the interpretability feature text description of the corresponding image.

[0011] In some embodiments, low-rank adapters are inserted in parallel into the attention layer and fully connected layer of the pre-trained convolutional neural network; low-rank adapters are embedded in the Transformer layer of the pre-trained medical large language model.

[0012] In some embodiments, the specific expression for the low-rank adapter is: ,in Let the rank of the adapter be . Let A be a scaling factor, and B and A be trainable low-rank matrices. For the frozen weight matrix of a pre-trained convolutional neural network, This is the input feature tensor of the network layer.

[0013] In some embodiments, the supervised hierarchical contrastive learning module adjusts the penalty strength for negative sample pairs using a semantic similarity matrix, wherein, The specific expression for similarity score is as follows: ,in For the original similarity score, For visual feature vectors, For text feature vectors, For temperature parameters, For indicator functions, matrix elements , Scaling factor ∈[0,1] represents the semantic similarity of the diseases corresponding to the i-th and j-th samples.

[0014] In some embodiments, the final supervised hierarchical contrastive loss of the supervised hierarchical contrastive learning module is: ,in The image-to-text contrast loss is calculated symmetrically using the transpose similarity matrix, and the specific expression is as follows: , The text-to-image contrast loss is specifically expressed as follows: , i , j This represents different samples.

[0015] According to a second aspect of the present disclosure, a method for diagnosing craniofacial rare diseases using the above-described interpretable craniofacial rare disease diagnostic model is provided, comprising: Preprocessing and enhancement of facial images and clinical phenotype texts are performed to generate a training dataset; Initialize the parameters of the interpretable diagnostic model for rare craniofacial diseases; An interpretable diagnostic model for rare craniofacial diseases was trained using a training dataset. The data to be diagnosed is input into the trained model, and the final interpretable diagnostic report is output.

[0016] According to a third aspect of the present disclosure, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the above-described method steps for diagnosing rare craniofacial diseases.

[0017] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium is provided, the storage medium storing computer instructions that, when executed by a processor, implement the above-described method steps for diagnosing rare craniofacial diseases.

[0018] This disclosure provides an interpretable diagnostic model, method, and electronic device for craniofacial rare diseases. Addressing the limitations of existing diagnostic methods, such as long-tailed data distribution, model attention drift, and hallucinations, it proposes a diagnostic model incorporating medical visual-linguistic feature encoding, supervised hierarchical contrastive learning (SHCL), KL regularized progressive unfreezing (KRU), and interpretable diagnostic output. First, image-text pairs are constructed. An adaptive soft-boundary contrastive learning loss function is built by introducing disease semantic priors, tolerating visual overlap while preserving the feature manifold topology of phenotypic similar diseases. Second, KL divergence is used to monitor inter-layer feature distribution drift, and progressive unfreezing of the model is achieved through trust domain constraints. This invention effectively avoids catastrophic forgetting during model fine-tuning, improving the diagnostic accuracy of craniofacial rare diseases while outputting interpretable attention heatmaps and facial feature descriptions precisely aligned with pathological features. This provides an efficient and reliable technical solution for early clinical screening and assisted diagnosis. Specifically, its beneficial effects include: 1) Significantly improves the fine-grained discrimination ability for diseases with similar clinical phenotypes. Addressing the challenge of highly overlapping visual features among syndrome subtypes in rare diseases, this invention proposes a hierarchical supervised contrastive learning (SHCL) method incorporating clinical priors. Unlike traditional contrastive learning that brute-force distances between all samples, this invention uses an adaptive soft-margin mechanism to allow reasonable feature overlap between clinically similar syndromes while maintaining disease category discriminability. This mechanism effectively preserves the natural topological structure of the disease semantic space, significantly improving the model's robustness in modeling easily confused genetic syndromes.

[0019] 2) Effectively overcomes the "catastrophic forgetting" problem during model fine-tuning, ensuring training convergence. Addressing the drawback of small-sample fine-tuning easily destroying the general knowledge of large models, this invention designs a progressive parameter unfreezing (KRU) mechanism based on the trust region. By introducing a KL divergence regularization term during fine-tuning, the drift of the representation distribution of each network layer is explicitly constrained, forcing model parameter updates to always remain within the "trust region." This method avoids the parameter oscillation and forgetting problems caused by traditional heuristic unfreezing strategies, enabling the model to quickly and stably adapt to rare disease diagnosis tasks while retaining general visual understanding capabilities.

[0020] 3) It achieves state-of-the-art diagnostic accuracy and clinical interpretability (SOTA). Experiments on multiple real-world craniofacial rare disease datasets demonstrate that the framework proposed in this invention significantly outperforms existing general-purpose large models and specialized medical models.

[0021] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0022] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0023] Figure 1 These are the original input image, the generated pathological feature attention heatmap, and the comparison image in this embodiment of the invention; Figure 2 This is a schematic diagram of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0024] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention. Furthermore, it should be noted that, for ease of description, only the parts relevant to the present invention are shown in the drawings, not the entire structure.

[0025] Before discussing the exemplary embodiments in more detail, it should be noted that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe the steps as sequential processes, many of these steps can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the steps can be rearranged. The process can be terminated when its operation is complete, but may also have additional steps not included in the figures. The process can correspond to a method, function, procedure, subroutine, subroutine, etc.

[0026] This invention relates to an interpretable diagnostic model, diagnostic method, and electronic device for craniofacial rare diseases based on semantically perceptive manifold reshaping and trust domain adaptation. It includes methods for achieving fine-grained medical image analysis using multimodal large model (LMM) technology, supervised contrastive learning theory, and low-rank adaptation (LoRA) fine-tuning technology. This method can be applied to smart healthcare scenarios such as hospital genetics-assisted diagnosis, pediatric rare disease screening, and newborn birth defect monitoring. It can also be flexibly applied to areas such as remote medical consultations, intelligent organization of medical image databases, medical research and teaching, and phenotypic analysis before gene testing.

[0027] In terms of application, taking the auxiliary diagnosis of rare diseases based on multimodal information such as facial phenotypic images and clinical symptom text descriptions as an example: by collecting facial frontal / lateral photos of patients diagnosed with rare diseases and corresponding standardized clinical phenotypic text data, as well as corresponding clinical phenotypic description text data such as "wide-set eyes", "collapsed nasal bridge", and "micrognathia" (covering multiple genetic syndromes), the collected image and text data are input into the training scheme of this invention. The feature space is optimized using the semantically aware manifold reshaping (SHCL) module and fine-tuned using the KRU module to train the diagnostic model for this type of disease. Once the model is trained, it can be deployed on hospital workstations, cloud servers, or embedded in portable screening devices to perform real-time inference on newly collected suspected patient data and output an interpretable diagnostic report containing disease prediction results, confidence levels, and visualized lesion heat maps.

[0028] Considering the uneven distribution of high-quality medical resources, this invention is also applicable to telemedicine and primary care screening scenarios. In these scenarios, image data may come from facial photographs taken in non-standardized environments using different devices (such as mobile phones and ordinary cameras), and may also contain non-professional textual information described by family members. Medical image data containing various background interferences can be collected and used as training data for this invention to train a robust model with strong anti-interference capabilities. After the model is trained, primary care physicians or patients' families only need to upload photos, and the system can use the method of this invention to perform feature extraction and comparison, quickly providing high-risk disease alerts and highlighting abnormal facial areas, providing intuitive evidence for subsequent referrals or genetic testing.

[0029] Similarly, the technical concept of this invention has general transferability. In other medical analysis fields that also face difficulties in fine-grained feature recognition and long-tailed data distribution, such as skin lesion identification, fundus lesion screening, and pathological section analysis, those skilled in the art can replace the visual encoder with a dedicated encoder for skin lesions / fundus images, reuse the core strategies of SHCL semantic manifold shaping and KRU trust domain adaptation, and quickly train a multimodal diagnostic model for specific diseases. By collecting relevant lesion images and corresponding medical report texts and performing adaptive annotation, the core strategies of this invention (SHCL strategy and KRU fine-tuning strategy) are used to train a multimodal analysis model for specific diseases. This method can provide an effective technical solution with semantic understanding capabilities and anti-forgetting properties for assisted diagnosis in the above-mentioned fields.

[0030] An interpretable diagnostic model for rare craniofacial diseases includes a medical visual-language feature encoding network, a supervised hierarchical contrastive learning module, a KL regularized progressive thawing module, and an interpretable diagnostic output module. The medical vision-language feature encoding network is used to extract high-order multimodal features and achieve cross-modal alignment. It adopts a dual-tower structure with visual encoding branch and text encoding branch, and outputs normalized visual feature vector and text feature vector respectively. The supervised hierarchical contrastive learning module includes a semantic similarity calculation unit and an adaptive soft-boundary loss calculation unit. The semantic similarity calculation unit is used to construct a semantic relationship graph between diseases using human phenotypic ontology, generating a multidimensional semantic similarity matrix. The adaptive soft boundary loss calculation unit is used to dynamically adjust the exclusion boundary of contrastive learning so that the model retains the topological proximity of the disease in the feature space. The KL regularized progressive unfreezing module is used for monitoring model parameter updates. It includes a distribution monitor and a dynamic unfreezing controller. The distribution monitor models the hidden state of each network layer as a multivariate Gaussian distribution. The dynamic unfreezing controller sets a dynamic trust domain threshold. When the distribution drift exceeds the threshold, it automatically triggers a regularization penalty mechanism to limit the magnitude of parameter updates. The interpretable diagnostic output module generates the final diagnostic results and visualization basis, including a classification head, a visualization generator, and a large language model module. The classification head concatenates the aligned visual feature vectors and text feature vectors, and outputs the predicted probability of each disease category through an activation function. The visualization generator uses visualization technology to trace back the gradient information of the classification layer, calculates the spatial weights of the last convolutional layer, and generates a heatmap of pathological features highlighting the lesion area. The large language module outputs text information corresponding to the image features, intuitively describing the diagnostic basis.

[0031] Specifically, the interpretable craniofacial rare disease diagnostic model consists of four core modules: medical vision-language feature encoding network, supervised hierarchical contrastive learning (SHCL) module, KL regularized progressive thawing (KRU) module, and interpretable diagnostic output module.

[0032] In the medical vision-language feature encoding network, the visual encoding branch uses a pre-trained convolutional neural network as the backbone network to process pre-processed standardized facial images and output the result. Normalized visual feature vectors The text encoding branch utilizes a pre-trained medical large language model as the backbone network to process clinical phenotype texts and output the results. Normalized text feature vectors .

[0033] Low-rank adapters are inserted in parallel into the attention layer and fully connected layer of the pre-trained convolutional neural network; low-rank adapters are embedded in the Transformer layer of the pre-trained medical large language model.

[0034] In specific implementation, the medical vision-language feature encoding network is responsible for extracting high-order multimodal features and achieving cross-modal alignment, employing a dual-tower structure including visual and text branches. The visual encoding branch uses a pre-trained convolutional neural network (or Vision Transformer) as its backbone, with input being a pre-processed, standardized facial image (e.g., 896×896 resolution). To adapt to the characteristics of the medical field, low-rank adaptation (LoRA) units are inserted in parallel into the attention layer and fully connected layer of the backbone network. In a preferred embodiment, the rank of the LoRA unit is set to r=16, and the scaling factor is set to α=16. The final output of this branch is... Normalized visual feature vectors The text encoding branch uses a pre-trained medical large language model (such as MedGemma-27B) as its backbone. The input is structured clinical phenotype text based on medical guidelines (such as "wide-set eyes, flat philtrum"). Similarly, a LoRA adapter is embedded in the Transformer layer, and the original backbone parameters are frozen. The output is processed... Normalized text feature vectors In addition, this module also includes a cross-modal alignment layer, which maps visual features and textual features to the same semantic space through linear projection to eliminate the distributional heterogeneity between modalities.

[0035] The Supervised Hierarchical Contrastive Learning (SHCL) module is a feature regularization component based on medical priors, designed to address the problem of excessive feature separation among similar diseases. This module consists of a semantic similarity calculation unit and an adaptive soft-boundary loss calculation unit. Its working mechanism utilizes the Human Phenotypic Ontology (HPO) to construct a semantic relationship graph between diseases, generating... 3D semantic similarity matrix This module does not change the backbone structure of the network, but rather serves as part of the training objective function. By dynamically adjusting the exclusion boundary of contrastive learning, it forces the model to preserve the topological proximity of "familial" diseases in the feature space.

[0036] In practice, the KL Regularized Progressive Unfreeze (KRU) module is a parameter update monitoring component used to balance the model's generality and specificity. It comprises a distributed monitor and a dynamic unfreeze controller. The distributed monitor models the hidden states of each network layer as a multivariate Gaussian distribution. This is used to calculate the KL divergence between the currently fine-tuned model and the original pre-trained model in real time. The dynamic unfreeze controller sets a dynamic trust domain threshold. When the distribution drift of a certain layer exceeds the threshold, this module will automatically trigger a regularization penalty mechanism to limit the parameter update magnitude of that layer, thereby preventing catastrophic forgetting.

[0037] In practical implementation, the interpretable diagnostic output module is responsible for generating the final diagnostic results and visualization evidence. It mainly consists of a classification head, a visualization generator, and a large language model module. The classification head concatenates aligned visual and textual features and outputs the predicted probability of each disease category through a Sigmoid activation function. The visualization generator, built on the Grad-CAM algorithm, can trace back the gradient information of the classification layer, calculate the spatial weights of the final convolutional layer, and generate a heatmap highlighting pathological features of lesion areas (such as ear, nose, and mouth abnormalities). The large language model module, after LoRA fine-tuning, has the ability to analyze the disease features of input images, outputting feature descriptions of the corresponding images and providing relevant diagnostic evidence.

[0038] In the specific implementation process, multimodal data preprocessing and augmentation are first performed, and the multimodal training dataset is then prepared. middle, For facial images, For phenotypic description text, For disease labels, in terms of data augmentation strategies, brightness / contrast fine-tuning (±0.15) was used for rare disease samples (Williams syndrome, Goldenhar syndrome), and random cropping (scale [0.9,1.0]) and horizontal flipping (probability 0.1) were used for healthy control groups and Down syndrome samples to balance the data distribution.

[0039] Further network initialization and LoRA construction are performed, and the weights of the pre-trained model are loaded. In visual encoders and text encoder Inserting LoRA adapters (rank-order adapters) into the attention layer and fully connected layer scaling factor Freeze backbone network parameters and initialize only LoRA parameters. The expression for calculating LoRA is: ,in For the frozen weight matrix of the pre-trained backbone network, The input feature tensor of the network layer. and It is a trainable low-rank matrix.

[0040] Further feature encoding and cross-modal alignment are performed on the preprocessed image. and text Input to the encoding network. The visual branch outputs normalized features. Text branch output normalized features Then, a linear projection layer is used to map both features to the same dimensional space, completing feature alignment.

[0041] To further compute the supervised hierarchical contrastive learning (SHCL) loss, this step first constructs a... The semantic similarity matrix M is defined as follows: ,in Scaling factor ∈[0,1] represents the semantic similarity between the diseases corresponding to the i-th and j-th samples (the higher the similarity, the closer the value is to 1). Traditional contrastive learning applies the same penalty to all negative samples. This module adjusts the penalty strength for negative sample pairs through the semantic similarity matrix, defining the adjusted similarity score as... ,in For the original similarity score, For temperature parameters, Let be the indicator function. Combining image-text and text-image bidirectional contrast, the final supervised hierarchical contrast loss is: Image-to-text contrast loss The calculation expression is The text-to-image contrast loss is The specific expression is .

[0042] Introducing semantically adaptive damping factor It can "soften" the gradient for clinically similar diseases, preventing the optimizer from over-separating biologically related disease clusters, while maintaining a strong exclusion gradient for diseases with significant clinical differences to ensure clear classification boundaries.

[0043] Further performing KL regularized progressive unfreezing (KRU) involves modeling the hidden states hl(t) of each layer of the visual encoder (where t is the training epoch) as a multivariate Gaussian distribution during training, expressed as: ,in The characteristic mean, The feature variance is supported by the central limit theorem. Using the feature distribution P0 of the pre-trained model as a baseline, this baseline is dynamically updated during training using an exponential moving average (EMA). The momentum factor is set to 0.1, as shown in the following formula: , The corresponding calculation expression is: .

[0044] Setting dynamic KL threshold The update rule for the thawing layer set is as follows: .

[0045] To prevent excessive drift in the feature distribution after thawing, KL regularization loss is introduced, with the following expression: .

[0046] This step is used to constrain feature drift and prevent forgetting.

[0047] Further multi-objective joint optimization is performed to calculate the classification loss. (Cross-entropy) and language modeling loss Combining the above steps, construct the total loss function: .

[0048] The model parameters are updated using the AdamW optimizer, and the baseline distribution is updated using the exponential moving average (EMA).

[0049] Finally, testing and interpretable diagnostic output are performed. During the testing phase, the data to be diagnosed is input into the trained model. First, the disease probability is output through the classification head, and a threshold of 0.5 is used to determine the disease label. Second, based on the Grad-CAM algorithm, the weights of the classification gradient on the visual features are calculated, generating a pathological feature attention heatmap, as shown in the image. Figure 1 As shown, abnormal areas such as the auricle and interocular distance are highlighted, and the final interpretable diagnostic report is output based on the classification results.

[0050] In addition to the modules described above, the diagnostic model for rare craniofacial diseases may also include other components; however, since these components are not relevant to the embodiments of this disclosure, their illustrations and descriptions are omitted herein. Another embodiment provides a method for diagnosing craniofacial rare diseases using the above-described interpretable craniofacial rare disease diagnostic model, comprising: Preprocessing and enhancement of facial images and clinical phenotype texts are performed to generate a training dataset; Initialize the parameters of the interpretable diagnostic model for rare craniofacial diseases; An interpretable diagnostic model for rare craniofacial diseases was trained using a training dataset. The data to be diagnosed is input into the trained model, and the final interpretable diagnostic report is output.

[0051] Other specific procedures for diagnosing craniofacial rare diseases using the above-described interpretable craniofacial rare disease diagnostic model are described in the above-described embodiment of the interpretable craniofacial rare disease diagnostic model and will not be repeated here.

[0052] Another embodiment illustrating that the system of the present invention can also be achieved by means of... Figure 2 The architecture of the computing device shown is used to implement this. Figure 2 The architecture of the computing device is shown. For example... Figure 2As shown, the computer system 210 includes a system bus 230, one or more CPUs 240, input / output 220, and memory 250. Memory 250 can store various data or files used for computer processing and / or communication, as well as program instructions executed by the CPU, including methods for diagnosing rare craniofacial diseases using interpretable craniofacial rare disease diagnostic models. Figure 2 The architecture shown is merely exemplary and should be adjusted according to actual needs when implementing different devices. Figure 2 One or more components are included. The memory 250, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules, such as the program instructions / modules corresponding to the method for diagnosing craniofacial rare diseases using the aforementioned interpretable craniofacial rare disease diagnostic model in this embodiment of the invention. One or more CPUs 240 execute various functional applications and data processing of the system of the present invention by running the software programs, instructions, and modules stored in the memory 250. Of course, the server provided in the embodiments of the present invention is not limited to performing the method operations described above, but can also perform related operations in the method for diagnosing craniofacial rare diseases using the above-described interpretable craniofacial rare disease diagnostic model provided in any embodiment of the present invention.

[0053] The memory 250 may primarily include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on terminal usage. Furthermore, the memory 250 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 250 may further include memory remotely configured relative to one or more CPUs 240, which can be connected to the device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0054] Input / output 220 can be used to receive input digital or character information, and to generate key signal inputs related to user settings and function control of the device. Input / output 220 may also include a display device such as a display screen.

[0055] This invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, this computer program implements the method for diagnosing craniofacial rare diseases using an interpretable craniofacial rare disease diagnostic model as described in the above embodiments. The computer-readable storage medium of this invention can be any combination of one or more computer-readable media. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0056] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0057] The program code contained on the storage medium can be transmitted using any suitable medium, including but not limited to wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0058] Furthermore, other specific operational processes of a non-transitory computer-readable storage medium are described in the above-described embodiments of the method for diagnosing rare craniofacial diseases using an interpretable craniofacial rare disease diagnostic model, and will not be repeated here.

[0059] To verify the effectiveness of the model of this invention, the diagnostic performance of the model and method of this invention is compared with that of the existing state-of-the-art (SOTA) methods, as shown in Table 1. In terms of diagnostic performance, this invention refreshes the current best record with an accuracy of 96.22% and an F1 score of 95.02%, which greatly solves the problem of misdiagnosis under long-tailed data distribution.

[0060] Table 1

[0061] Regarding interpretability, a clinical reasoning quality assessment was conducted, as shown in Table 2. This invention achieved the highest scores (average 4.82 / 5.0) in all three dimensions: diagnostic accuracy, completeness, and logical rationality. This demonstrates that this invention can not only clearly locate lesion areas such as "wide-set eyes" using heatmaps, but also effectively avoid the "illusion" problem of large models, providing highly reliable auxiliary diagnostic evidence for clinical practice.

[0062] Table 2

[0063] In this document, the terms “comprising,” “including,” or any other variations thereof are intended to cover non-exclusive inclusion, such that a step or method that comprises a list of elements includes not only those elements but also other elements not expressly listed or inherent to such a step or method.

[0064] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such modifications and substitutions should be considered within the scope of protection of the present invention.

Claims

1. A diagnostic model for rare craniofacial diseases that can be explained, characterized in that, It includes a medical vision-language feature encoding network, a supervised hierarchical contrastive learning module, a KL regularized progressive unfreezing module, and an interpretable diagnostic output module. The medical vision-language feature encoding network is used to extract high-order multimodal features and achieve cross-modal alignment. It adopts a dual-tower structure with visual encoding branch and text encoding branch, and outputs normalized visual feature vector and text feature vector respectively. The supervised hierarchical contrastive learning module includes a semantic similarity calculation unit and an adaptive soft boundary loss calculation unit. The semantic similarity calculation unit is used to construct a semantic relationship graph between diseases using human phenotypic ontology and generate a multidimensional semantic similarity matrix. The adaptive soft boundary loss calculation unit is used to dynamically adjust the exclusion boundary of contrastive learning so that the model retains the topological proximity of diseases in the feature space. The KL regularized progressive unfreezing module is used for monitoring model parameter updates. It includes a distribution monitor and a dynamic unfreezing controller. The distribution monitor models the hidden state of each network layer as a multivariate Gaussian distribution. The dynamic unfreezing controller sets a dynamic trust domain threshold. When the distribution drift exceeds the threshold, it automatically triggers a regularization penalty mechanism to limit the parameter update amplitude. The interpretable diagnostic output module is used to generate the final diagnostic results and visualization basis, including a classification head, a visualization generator, and a large language model module. The classification head concatenates the aligned visual feature vectors and text feature vectors, and outputs the predicted probability of each disease category through an activation function. The visualization generator uses visualization technology to trace back the gradient information of the classification layer, calculates the spatial weights of the last convolutional layer, and generates a pathological feature attention heatmap that highlights the lesion area. The large language module outputs the text information of the pathological feature attention heatmap features.

2. The interpretable diagnostic model for rare craniofacial diseases according to claim 1, characterized in that, The visual coding branch uses a pre-trained convolutional neural network as its backbone to process the pre-processed standardized facial images and output the result. Normalized visual feature vectors; the text encoding branch utilizes a pre-trained medical large language model as the backbone network to process clinical phenotype texts and output normalized visual feature vectors. Normalized text feature vectors.

3. The interpretable diagnostic model for rare craniofacial diseases according to claim 1, characterized in that, The interpretable diagnostic output module satisfies at least one of the following conditions: (1) Output the predicted probability of each disease category through the Sigmoid activation function; (2) The visualization generator is built based on the Grad-CAM algorithm.

4. The interpretable diagnostic model for rare craniofacial diseases according to claim 2, characterized in that, Low-rank adapters are inserted in parallel into the attention layer and fully connected layer of the pre-trained convolutional neural network; low-rank adapters are embedded in the Transformer layer of the pre-trained medical large language model.

5. The interpretable diagnostic model for rare craniofacial diseases according to claim 2, characterized in that, The specific expression for the low-rank adapter is: ,in Let the rank of the adapter be . Let A be a scaling factor, and B and A be trainable low-rank matrices. For the frozen weight matrix of a pre-trained convolutional neural network, This is the input feature tensor of the network layer.

6. The interpretable diagnostic model for rare craniofacial diseases according to claim 1, characterized in that, The supervised hierarchical contrastive learning module adjusts the penalty intensity for negative sample pairs using a semantic similarity matrix, wherein... The specific expression for similarity score is as follows: ,in For the original similarity score, For visual feature vectors, For text feature vectors, For temperature parameters, For indicator functions, matrix elements , Scaling factor ∈[0,1] represents the semantic similarity of the diseases corresponding to the i-th and j-th samples.

7. The interpretable diagnostic model for rare craniofacial diseases according to claim 6, characterized in that, The final supervised hierarchical contrastive learning module's supervised hierarchical contrastive loss is: ,in The image-to-text contrast loss is calculated symmetrically using the transpose similarity matrix, and the specific expression is as follows: , The text-to-image contrast loss is specifically expressed as follows: , i , j This represents different samples.

8. A method for diagnosing rare craniofacial diseases using the interpretable craniofacial rare disease diagnostic model according to any one of claims 1-7, characterized in that, Includes the following steps: Preprocessing and enhancement of facial images and clinical phenotype texts are performed to generate a training dataset; Initialize the parameters of the interpretable diagnostic model for rare craniofacial diseases; An interpretable diagnostic model for rare craniofacial diseases was trained using a training dataset. The data to be diagnosed is input into the trained model, and the final interpretable diagnostic report is output.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method steps for diagnosing rare craniofacial diseases as described in claim 8.

10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, When the instructions are executed by the processor, they implement the method steps for diagnosing rare craniofacial diseases as described in claim 8.

Images