Intelligent disease diagnosis system

Abstract

The invention relates to the technical field of medical artificial intelligence, in particular to an intelligent disease diagnosis system which comprises a data entry module, a data processing module and a data output module. Wherein the data entry module is used for receiving and primarily processing multi-modal medical data, and the data entry module comprises a data input unit, a data preprocessing unit and a data feature extraction unit; the data processing module comprises a knowledge graph construction unit and a multi-modal feature alignment unit; and the data output module comprises a reasoning diagnosis unit, a dialogue state tracking unit and a diagnosis result output unit. The technical problems that in the prior art, the multi-mode cooperation capability is weak, and the knowledge base is not updated in time are solved.

Description

Technical Field

[0001] This invention relates to the field of medical artificial intelligence technology, and more specifically, to an intelligent disease diagnosis system. Background Technology

[0002] Intelligent disease diagnosis systems combine artificial intelligence technology with medical diagnostics. By integrating technologies such as machine learning, natural language processing, and computer vision, they intelligently analyze patients' medical data to assist doctors in quickly identifying diseases, assessing the severity of conditions, and providing treatment recommendations. They also support patients in conducting preliminary self-diagnosis outside of medical settings. Existing intelligent disease diagnosis systems generally include functions such as multi-module data integration, disease risk prediction, differential diagnosis ranking, and evidence-based treatment recommendations. They are significant in improving diagnostic efficiency, reducing misdiagnosis and missed diagnosis rates, alleviating the uneven distribution of high-quality medical resources, and supporting telemedicine and primary healthcare services. They particularly provide technical support for the diagnosis of complex diseases and the response to public health emergencies.

[0003] Existing intelligent disease diagnosis systems are mostly developed based on single or limited technological frameworks. Early systems focused on structured data processing, using medical knowledge bases and rule engines to match and analyze standardized patient information (such as structured fields in electronic medical records and laboratory test data) and output diagnostic suggestions. With the development of deep learning technology, some intelligent disease diagnosis systems have introduced neural network models to enhance their ability to process unstructured data. For example, convolutional neural networks are used to parse medical images, and recurrent neural networks are used to process medical record text. In addition, some intelligent disease diagnosis systems also combine large language models to improve natural language interaction and knowledge retrieval capabilities.

[0004] While existing systems possess rich functionality and have demonstrated certain value in specific scenarios, they still suffer from significant shortcomings. Current intelligent disease diagnosis systems exhibit weak multimodal collaboration capabilities; text, image, and laboratory data are often processed in isolation, failing to effectively integrate multi-source heterogeneous data such as patient complaints, medical images, and laboratory indicators. Insufficient cross-modal feature correlation analysis further limits the diagnostic accuracy for complex diseases. Therefore, this invention proposes an intelligent disease diagnosis system that addresses the problems of weak multimodal collaboration capabilities and untimely knowledge base updates in existing intelligent disease diagnosis systems. Summary of the Invention

[0005] The purpose of this application is to provide an intelligent disease diagnosis system that solves the technical problems of weak multimodal collaboration capabilities and untimely knowledge base updates in the prior art.

[0006] To solve the above-mentioned technical problems, the solution adopted in this application is as follows:

[0007] The intelligent disease diagnosis system includes a data entry module, a data processing module, and a data output module.

[0008] Preferably, the data entry module is used to receive and preliminarily process multimodal medical data, and the data entry module includes a data input unit, a data preprocessing unit, and a data feature extraction unit;

[0009] Preferably, the data processing module includes a knowledge graph construction unit and a multimodal feature alignment unit;

[0010] Preferably, the data output module includes a reasoning diagnosis unit, a dialogue state tracking unit, and a diagnosis result output unit.

[0011] Preferably, the data input unit receives three types of data: text data, image data, and structured data;

[0012] When inputting text data, both structured and unstructured input are supported. Structured input is form filling; unstructured input includes speech-to-text and document upload.

[0013] The data input unit performs format validation on the input data. When the input data is formatted incorrectly, it rejects the input of the incorrectly formatted data and prompts for correction. When the data is successfully entered, the data input unit adds a type label to the data that passes the format validation.

[0014] Preferably, the data preprocessing unit includes a data cleaning unit and a data standardization unit;

[0015] The data cleaning unit cleans the data received by the data input unit; the text data cleaning process includes removing irrelevant symbols, correcting typos, and filtering duplicate content; the image data cleaning process includes eliminating image noise, repairing missing pixels, and enhancing blurred images; the structured data cleaning process includes identifying and processing outliers and filling in missing values.

[0016] The data standardization unit performs standardization processing on the data; for text data, the standardization process includes unifying terminology; for image data, the standardization process includes scaling images from different imaging devices and at different resolutions to the same size and normalizing pixels to the [0,1] range; for structured data, the standardization process includes unifying physical units and aligning time-series data.

[0017] Preferably, the data feature extraction unit extracts data features from the preprocessed data, transforms the entered data into a quantifiable feature vector, and provides data input to the data processing module.

[0018] Preferably, the knowledge graph construction unit is used to transform medical knowledge into a structured knowledge graph and update it in real time;

[0019] The knowledge graph construction unit extracts core entities and relationships from multi-source medical texts. After obtaining the core entities and their relationships, the knowledge graph construction unit constructs an initial knowledge graph with entities as nodes, forming a knowledge graph with entities as nodes and relationships as edges.

[0020] The knowledge graph construction unit periodically updates the multi-source medical text library to obtain the latest medical text library, performs core entity and relation extraction operations on the newly added medical text content, and adds the newly extracted core entities and their relations to the initial knowledge graph to realize the dynamic updating of the knowledge graph.

[0021] Preferably, the multimodal feature alignment unit is used to eliminate modal differences between text data features, image data features, and structured data features, so that different types of features are comparable in the same semantic space;

[0022] The multimodal feature alignment unit uses a neural network with two fully connected layers. The first fully connected layer has 512 neurons and uses the ReLU function as the activation function. The second fully connected layer maps the 512-dimensional features to 1024-dimensional features without using an activation function, ultimately mapping the text data features to 1024-dimensional features.

[0023] The multimodal feature alignment unit uses a fully connected layer with 256 neurons and the ReLU activation function. Then, it is mapped to 1024 dimensions through a fully connected layer with 1024 neurons without using an activation function, thus finally mapping the image data features to 1024 dimensions.

[0024] The multimodal features are mapped to 1024 dimensions based on the original dimensions of the structured data features through a combination of corresponding fully connected layers.

[0025] Preferably, after the multimodal feature alignment unit maps text data features, image data features, and structured data features to the same dimensional space, it uses a multimodal feature alignment loss function to quantify the correlation between different modal features, making semantically related features closer in space and semantically unrelated features farther apart in space, thereby achieving multimodal feature alignment.

[0026] After multimodal feature alignment, aligned text data features, aligned image data features, and aligned structured data features with strong correlation in a unified semantic space are obtained. The multimodal feature alignment unit uses a distance formula to calculate the semantic relevance between these aligned features and entities in the knowledge graph. The closer the aligned feature is to the entity, the higher the semantic relevance. The feature is then bound to the entity with the highest semantic relevance. The multimodal feature alignment unit uses a distance formula to calculate the correlation strength between pairwise aligned features and adjusts the relationship weights between entities in the knowledge graph based on the correlation strength between aligned features. When the correlation strength between aligned features increases, the relationship weights between entities in the knowledge graph are increased simultaneously.

[0027] Preferably, the dialogue status tracking unit is used to engage in real-time dialogue with the user and receive user input data, including symptoms, examination results, medical history, etc.

[0028] Preferably, the reasoning and diagnosis unit, based on multimodal features and dynamic knowledge graphs, counts the frequency of occurrence of each disease entity to generate a more comprehensive diagnostic conclusion;

[0029] When a user inputs relevant symptom features through the dialogue state tracking unit, the reasoning and diagnosis unit generates an initial hypothesis based on the aligned multimodal feature vector and the entity relationships in the knowledge graph.

[0030] After multiple disease entities are matched based on multiple data input by the user, the reasoning and diagnosis unit calculates the frequency of each disease entity. The higher the frequency, the greater the probability of having the disease; the lower the frequency, the less likely the probability of having the disease. The disease entities are sorted according to their frequency to generate a reasoning and diagnosis report.

[0031] Preferably, the diagnostic result output unit outputs and displays the diagnostic report from the reasoning diagnostic unit to the user, providing the user with a reference for medical treatment.

[0032] The technical solution of this application has at least the following advantages and beneficial effects:

[0033] 1. Traditional systems cannot effectively integrate multi-source heterogeneous data such as patient complaint text (unstructured), medical images (e.g., CT / DICOM: a medical digital imaging and communication standard), and laboratory indicators (structured data), resulting in insufficient cross-modal feature correlation. Therefore, this invention uses a multimodal feature alignment unit to map text (e.g., symptom description), images (e.g., CT / MRI), and structured data (e.g., laboratory indicators) to a unified semantic space. Furthermore, it quantifies feature correlation through a multimodal feature alignment loss function, enabling strong correlations between cross-modal features (e.g., the text "fever" and the image "lung inflammation shadow"), thus overcoming the limitations of isolated data processing in traditional systems.

[0034] 2. Existing knowledge bases rely on regular manual updates (with an average delay of ≥3 months), failing to incorporate the latest clinical research evidence, emerging infectious disease data, and drug safety warnings in real time, resulting in diagnostic recommendations being out of touch with the forefront of medicine. The knowledge graph construction unit in this invention solves the problem of lagging knowledge updates in traditional systems by regularly updating multi-source medical text databases (such as the latest clinical guidelines and research literature), extracting new entities and relationships (such as symptoms of novel diseases and treatment plans for drug-resistant bacteria) in real time, and integrating them into the knowledge graph. This ensures that diagnostic criteria are always synchronized with the forefront of medicine. Attached Figure Description

[0035] Figure 1 This is a system block diagram of the intelligent disease diagnosis system. Detailed Implementation

[0036] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0037] Please refer to Figure 1 The present invention provides an intelligent disease diagnosis system, including a data input module, a data processing module, and a data output module.

[0038] Furthermore, the data entry module receives and performs preliminary processing of multimodal medical data, including text data, image data, and structured data. Text data includes patient complaints and medical records; image data includes CT, MRI, and DICOM images; and structured data includes laboratory indicators and vital signs. The data entry module includes a data input unit, a data preprocessing unit, and a data feature extraction unit.

[0039] Specifically, the data input unit is the interface between the intelligent disease diagnosis system and external data. It is used to receive multi-source, heterogeneous raw medical data and achieve compatible input and preliminary classification of the data, providing data for subsequent data preprocessing.

[0040] More specifically, the data input unit receives three types of data: text data, image data, and structured data. Text data includes patient complaints, such as fever for three days accompanied by cough; it also includes electronic medical records and doctor's orders. Input supports both structured and unstructured methods. Structured input involves filling out forms; unstructured input includes speech-to-text and document upload. Image data includes medical image files, pathological slide images, and ultrasound images, uploaded through the hospital's PACS system or local files. Structured data includes laboratory test indicators, such as complete blood count and biochemical indicators; it also includes vital sign data, such as body temperature, heart rate, and blood pressure; and medication records. Structured data can be imported through the HIS / LIS system or entered manually.

[0041] More specifically, the data input unit performs format validation on the input data. When the input data is formatted incorrectly, it rejects the input of the incorrectly formatted data and prompts for correction. For example, the text encoding format of the input text should be uniformly UTF-8. If the encoding format of the text data in the data input unit is not UTF-8, the data input unit will refuse to accept the text data and prompt the data entry user to re-enter it.

[0042] More specifically, after successful data entry, the data input unit adds type labels to the data that passes format validation. For example, when entering patient complaint data, a text label is added; when entering chest CT images, an image label is added; and when entering structured data such as blood routine tests, a structured data label is added. After adding labels to the data, they are associated with an identifier that distinguishes the patient. In this invention, the associated identifier that distinguishes the patient is the patient's medical record number.

[0043] Specifically, the data preprocessing unit includes a data cleaning unit and a data standardization unit.

[0044] More specifically, the data cleaning unit cleans the data received by the data input unit. Specifically, for text data, the text data cleaning process includes removing irrelevant symbols, such as punctuation and special characters; correcting typos, such as spelling correction based on a medical dictionary; and filtering duplicate content, such as multiple entries of the same chief complaint.

[0045] For image data, the image cleaning process includes noise removal, missing pixel repair, and image enhancement. Noise removal uses a Gaussian filtering algorithm to filter noise in medical images; missing pixel repair uses interpolation to restore missing pixels; and image enhancement uses contrast stretching to process blurred images.

[0046] It should be noted that Gaussian filtering for noise reduction, interpolation, and contrast stretching are all basic image processing methods and are existing technologies. The protection of this invention does not lie in these image processing methods, so their principles and image processing processes will not be elaborated here.

[0047] For structured data, the structured data cleaning process includes identifying and handling outliers, such as using medical threshold filtering to filter out blood pressure values ​​that are outside the physiological range; and filling in missing values, such as filling in missing blood routine indicators with the mean of the same age group, or marking them as "to be supplemented".

[0048] More specifically, the data standardization unit is used to standardize data. For text data, the standardization process includes unifying terminology, such as standardizing "heart attack" as "acute myocardial infarction".

[0049] For image data, the standardization process includes scaling images from different imaging devices and at different resolutions to the same size and normalizing pixels to the [0,1] range.

[0050] For structured data, the standardization process includes unifying physical units, such as unifying body temperature units to degrees Celsius; it also includes time alignment of time-series data, such as arranging consecutive blood pressure test results in chronological order.

[0051] Specifically, the data feature extraction unit is used to extract data features with clinical diagnostic value from the preprocessed data, transform the entered data into quantifiable feature vectors, and provide data input to the data processing module.

[0052] More specifically, for text data, text data feature extraction is performed on the standardized text data obtained after data standardization unit processing. The extraction process follows these three steps:

[0053] S10: The pre-trained medical language model BioBERT performs word segmentation, part-of-speech tagging, and entity recognition on standardized text to extract medical entities such as diseases, symptoms, and signs, such as "fever" and "difficulty breathing".

[0054] S11: Based on the medical entities and original text context extracted in S10, the text data is transformed into a 768-dimensional semantic vector through the pre-trained medical language model BioBERT, encoding the semantic information of the text into a quantifiable vector form.

[0055] S12: Extract statistical structured features such as symptom frequency, disease duration, and number of complications from text data using the pre-trained medical language model BioBERT.

[0056] It should be noted that the statistical structured features extracted in S12, such as the frequency of symptom occurrence, duration of illness, and number of complications, are supplements to steps S10 and S11. The three steps are performed sequentially: S10 extracts medical entities, S11 converts medical entities and their context into semantic vectors, and S12 supplements statistical indicators. The final text features include medical entities, statistical information, and semantic associations.

[0057] It should be noted that the pre-trained medical language model BioBERT is a pre-trained language representation model specifically designed for biomedical text mining. It has been pre-trained on a large-scale biological corpus. On the NCBI-disease dataset, the BioBERT model can accurately identify disease-related entities. This is not existing technology. In this invention, it is used to identify disease-related entities in text data, extract medical entities, and extract structured features. The innovation that this invention aims to protect does not lie in the principle and training process of the model, so its working principle and training process will not be elaborated here.

[0058] More specifically, for image data, image data features are extracted from standardized image data obtained through data standardization unit processing.

[0059] The lightweight impact feature extractor MobileNetV3 is used to extract features layer by layer from standardized image data, generating feature vectors that include visual features such as edges, textures, and lesion areas.

[0060] It should be noted that the lightweight impact feature extractor MobileNetV3 is an existing technology used to process input image data. The processing includes feature extraction and feature enhancement, etc., and its principles and working process will not be elaborated here.

[0061] More specifically, for structured data, structured data features are extracted from the standardized structured data obtained after data standardization unit processing.

[0062] LSTM networks are used to extract trend features, extreme value features, and classification features from standardized structured data. Trend features include the slope of body temperature changes; extreme value features include the highest and lowest blood pressure values; and classification features include whether blood glucose and blood pressure are above the standard.

[0063] It should be noted that LSTM networks are existing technology. LSTM networks can analyze input data using time series data. In this invention, LSTM networks are used to analyze and extract features from structured data such as laboratory indicators and vital signs data. However, the principle behind this is not within the scope of protection of this invention, so its principle and training process will not be described in detail here.

[0064] Furthermore, the data processing module consists of two parts: a knowledge graph construction unit and a multimodal feature alignment unit.

[0065] Specifically, the knowledge graph construction unit is used to transform scattered medical knowledge such as clinical guidelines, literature, medical records, and drug instructions into a structured knowledge graph, which is then updated in real time.

[0066] More specifically, the knowledge graph construction unit extracts core entities and relationships from multi-source medical texts, such as "pneumonia - typical symptoms - fever" and "aspirin - indications - myocardial infarction". The medical texts include WHO treatment guidelines, hospital medical records, and existing medical papers; the core entities include diseases, symptoms, signs, drugs, and examination items.

[0067] After obtaining the core entities and their relationships, the knowledge graph construction unit builds an initial knowledge graph with entities as nodes, forming a knowledge graph with entities as nodes and relationships as edges. For example, the nodes are: pneumonia (disease), fever (symptom), chest CT (examination item), penicillin (drug); and the edges could be: pneumonia-typical symptoms-fever, pneumonia-recommended examination-chest CT, pneumonia-recommended medication-penicillin.

[0068] The knowledge graph construction unit periodically updates the multi-source medical text database, acquiring the latest medical texts. It then extracts core entities and relationships from newly added medical text content and adds these extracted entities and relationships to the initial knowledge graph, achieving dynamic updates. For example, when new research proves that "a certain novel antibiotic is more effective against drug-resistant pneumonia," this novel antibiotic is added to the "Pneumonia-Recommended Drug" node relationship chain in the knowledge graph.

[0069] Specifically, the multimodal feature alignment unit is used to eliminate modal differences between text data features, image data features, and structured data features, making different types of features comparable in the same semantic space and solving the problem of isolated features from each other that are difficult to process collaboratively.

[0070] More specifically, text data features, image data features, and structured data features are mapped to a 1024-dimensional semantic space. Specifically, text data features are mapped from 768 dimensions to 1024 dimensions through a non-linear transformation, image data features are expanded from 512 dimensions to 1024 dimensions, and discretized structured data features are mapped to 1024 dimensions.

[0071] For text data features, mapping from 768 dimensions to 1024 dimensions, a neural network with two fully connected layers is used. The first fully connected layer has 512 neurons, and the ReLU activation function is employed.

[0072] y = max(0,x)

[0073] Where x is the input value and y is the output value. The second fully connected layer maps the 512-dimensional features to 1024 dimensions without using an activation function.

[0074] For image data features, the 512-dimensional data is first processed through a fully connected layer with 256 neurons using the ReLU activation function, and then mapped to 1024 dimensions through a fully connected layer with 1024 neurons without using an activation function. Structured data features, based on their original dimensions, are mapped to 1024 dimensions through appropriate combinations of fully connected layers. For example, when the structured data feature is 10-dimensional, it first passes through a fully connected layer with 32 neurons using the ReLU activation function, and then through a fully connected layer with 1024 neurons to complete the mapping; this mapping does not use an activation function.

[0075] After mapping text data features, image data features, and structured data features to the same dimensional space, a multimodal feature alignment loss function is used to quantify the correlation between different modal features, making semantically related features closer together in space and semantically unrelated features farther apart. The multimodal feature alignment loss function is as follows:

[0076]

[0077] Among them, T i Let V be the text feature vector of the i-th sample. i Let ||T| be the image feature vector of the i-th sample. i ||、||V i || are the magnitudes of the text feature vector and the image feature vector, respectively, used to normalize the features and eliminate distance bias caused by scale differences in different modal features; This is the normalized feature vector with a magnitude of 1, ensuring that features of different modalities are comparable at the same scale.

[0078] The overall loss function is:

[0079]

[0080] in, The alignment loss between text data features and image data features; The loss is used to align text data features with structured data features; The loss is the alignment loss between image data features and structured data features.

[0081] After aligning multimodal features using a multimodal feature alignment loss function, aligned text data features, aligned image data features, and aligned structured data features with strong correlation in a unified semantic space are obtained. Using a distance formula, the semantic relevance between these aligned features and entities in the knowledge graph is calculated. The closer the aligned feature is to the entity, the higher the semantic relevance. Features are then bound to the entity with the highest semantic relevance; for example, the aligned feature "fever + lung inflammation shadow" is bound to the entity "pneumonia".

[0082] After multimodal feature alignment is completed using the multimodal feature alignment loss function, the aligned text data features, aligned image data features, and aligned structured data features are mapped to the same high-dimensional semantic space. The spatial distance between features directly reflects the strength of the association; the smaller the distance between features, the stronger the association. For example, the smaller the distance between aligned text features and aligned image features, the stronger the association.

[0083] Similarly, the correlation strength between pairwise aligned features is calculated using a distance formula, and the relationship weights between entities in the knowledge graph are adjusted based on the correlation strength between aligned features. As the correlation strength between aligned features increases, the relationship weights between entities in the knowledge graph are increased accordingly. For example, increasing the correlation strength between the aligned text feature "cough" and the aligned image feature "bronchitis" can improve the relationship weight of "bronchitis-typical symptoms-cough".

[0084] Therefore, multimodal feature alignment is achieved through the multimodal feature alignment loss function, and the correlation strength between each pair of aligned features is calculated. Based on this correlation strength, the relation weights between entities in the knowledge graph are adjusted, which solves the feature silo problem in traditional intelligent disease diagnosis systems, links aligned features with medical concepts, and provides a knowledge basis for disease diagnosis reasoning.

[0085] Furthermore, the data output module includes a reasoning and diagnosis unit, a dialogue state tracking unit, and a diagnosis result output unit. The data output module serves as the interaction interface between the intelligent disease diagnosis system and the user. Through the collaborative work of these three processing units—the reasoning and diagnosis unit, the dialogue state tracking unit, and the diagnosis result output unit—it utilizes existing multimodal features and other data to transform knowledge into targeted and valuable disease diagnosis and treatment plans.

[0086] Specifically, based on multimodal features and dynamic knowledge graphs, the reasoning and diagnosis unit counts the frequency of occurrence of each disease entity and generates a more comprehensive diagnostic conclusion.

[0087] Specifically, the dialogue status tracking unit is used to engage in real-time dialogue with the user and receive user input data, including symptoms, examination results, medical history, etc.

[0088] More specifically, when a user inputs relevant symptom features through the dialogue state tracking unit, the reasoning and diagnosis unit generates initial hypotheses based on the aligned multimodal feature vectors and entity relationships in the knowledge graph. For example, when a user inputs text such as "fever, cough," the reasoning and diagnosis unit matches entities such as "pneumonia" and "bronchitis"; when a user inputs structured data such as "elevated white blood cell count," the reasoning and diagnosis unit matches entities such as "bacterial infection."

[0089] After matching multiple disease entities based on user-input data, the inference diagnosis unit calculates the frequency of each disease entity. The higher the frequency, the greater the likelihood of having the disease; the lower the frequency, the less likely the likelihood. The disease entities are then sorted according to their frequency to generate an inference diagnosis report.

[0090] The dialogue state tracking unit can receive new data input by the user in real time. When it receives new data input by the user in a new round of dialogue, it transmits the new data to the inference and diagnosis unit. The inference and diagnosis unit then makes disease predictions based on the new data input by the user and updates the inference and diagnosis report.

[0091] Specifically, the diagnostic results output unit will output and display the diagnostic report from the reasoning diagnostic unit to provide users with a reference for medical treatment. At the same time, when users seek medical treatment, doctors can retrieve the diagnostic report to assist doctors in diagnosing and treating patients.

[0092] Furthermore, after a doctor examines and treats a patient, the system can obtain the doctor's diagnostic report.

[0093] Specifically, after the system obtains the doctor's diagnostic report, it parses the report. The data input unit receives text data, image data, and structured data from the report. The data cleaning unit cleans the data, and the data standardization unit standardizes the cleaned data. Then, the data feature extraction unit extracts features from the preprocessed data, thus obtaining the text, image, and structured data features from the report. More specifically, the multimodal feature unit aligns the text, image, and structured data features from the report to obtain aligned text, image, and structured data features.

[0094] Specifically, the system compares the diagnostic report generated by the system with the doctor's diagnostic report. In particular, the system compares the features of the generated diagnostic report with the features extracted from the doctor's diagnostic report to determine the accuracy of the system's diagnostic report.

[0095] More specifically, when the overlap of contrasting features is greater than or equal to 90%, the system's inference and diagnosis based on the patient's input data is considered to be highly accurate, providing good reference for patient self-diagnosis and assisting doctors in disease diagnosis and treatment. When the overlap of contrasting features is less than 90%, the system's diagnostic report is considered to be less accurate, offering no reference for patient self-diagnosis or assisting doctors in disease diagnosis and treatment. When the accuracy of the system's diagnostic report is less than 90%, the knowledge graph construction unit processes the doctor's diagnostic report, extracts core and entity relationships from the report, and uses these relationships to correct the original knowledge graph, improving the accuracy of entities and their relationships within the knowledge graph. For example, when the system initially diagnoses "common cold - typical symptoms - cough, runny nose, sore throat" while the doctor diagnoses "viral pneumonia - typical symptoms - fever, cough, lung CT inflammatory shadows," the system will add this relationship chain to the original knowledge graph, supplementing and correcting it.

[0096] The various embodiments of the present invention have now been described in detail. To avoid obscuring the concept of the invention, some details known in the art have not been described. Those skilled in the art will fully understand how to implement the technical solutions of this invention based on the above description, and the scope of the invention is defined by the appended claims.

Claims

1. An intelligent disease diagnosis system, characterized in that: It includes a data entry module, a data processing module, and a data output module. The data entry module is used to receive and preliminarily process multimodal medical data. The data entry module includes a data input unit, a data preprocessing unit, and a data feature extraction unit. The data processing module includes a knowledge graph construction unit and a multimodal feature alignment unit; The data output module includes a reasoning diagnosis unit, a dialogue state tracking unit, and a diagnosis result output unit.

2. The intelligent disease diagnosis system according to claim 1, characterized in that, The data input unit receives three types of data: text data, image data, and structured data. When inputting text data, both structured and unstructured input are supported. Structured input refers to form filling. Unstructured data entry includes speech-to-text and document upload; The data input unit performs format validation on the input data. When the input data is formatted incorrectly, it rejects the input of the incorrectly formatted data and prompts for correction. When the data is successfully entered, the data input unit adds a type label to the data that passes the format validation.

3. The intelligent disease diagnosis system according to claim 1, characterized in that, The data preprocessing unit includes a data cleaning unit and a data standardization unit; The data cleaning unit cleans the data received by the data input unit; the text data cleaning process includes removing irrelevant symbols, correcting typos, and filtering duplicate content. The image data cleaning process includes removing image noise, repairing missing pixels, and enhancing blurred images. The process of cleaning structured data includes identifying and handling outliers and filling in missing values; The data standardization unit performs standardization processing on the data; for text data, the standardization process includes unifying terminology; for image data, the standardization process includes scaling images from different imaging devices and at different resolutions to the same size and normalizing pixels to the [0,1] range; for structured data, the standardization process includes unifying physical units and aligning time-series data.

4. The intelligent disease diagnosis system according to claim 1, characterized in that, The data feature extraction unit extracts data features from the preprocessed data and transforms the entered data into a quantifiable feature vector, providing data input for the data processing module.

5. The intelligent disease diagnosis system according to claim 1, characterized in that, The knowledge graph construction unit is used to transform medical knowledge into a structured knowledge graph and update it in real time; The knowledge graph construction unit extracts core entities and relationships from multi-source medical texts. After obtaining the core entities and their relationships, the knowledge graph construction unit constructs an initial knowledge graph with entities as nodes, forming a knowledge graph with entities as nodes and relationships as edges. The knowledge graph construction unit periodically updates the multi-source medical text library to obtain the latest medical text library, performs core entity and relation extraction operations on the newly added medical text content, and adds the newly extracted core entities and their relations to the initial knowledge graph to realize the dynamic updating of the knowledge graph.

6. The intelligent disease diagnosis system according to claim 1, characterized in that, The multimodal feature alignment unit is used to eliminate modal differences between text data features, image data features, and structured data features, making different types of features comparable in the same semantic space; The multimodal feature alignment unit uses a neural network with two fully connected layers. The first fully connected layer has 512 neurons and uses the ReLU function as the activation function. The second fully connected layer maps the 512-dimensional features to 1024-dimensional features without using an activation function, ultimately mapping the text data features to 1024-dimensional features. The multimodal feature alignment unit uses a fully connected layer with 256 neurons and the ReLU activation function. Then, it is mapped to 1024 dimensions through a fully connected layer with 1024 neurons without using an activation function, thus finally mapping the image data features to 1024 dimensions. The multimodal features are mapped to 1024 dimensions based on the original dimensions of the structured data features through a combination of corresponding fully connected layers.

7. The intelligent disease diagnosis system according to claim 6, characterized in that, The multimodal feature alignment unit maps text data features, image data features, and structured data features to the same dimensional space, and then uses the multimodal feature alignment loss function to quantify the correlation between different modal features, so that semantically related features are closer in space and semantically unrelated features are farther apart in space, thereby achieving multimodal feature alignment. After multimodal feature alignment, we obtain aligned text data features, aligned image data features, and aligned structured data features that are strongly correlated in a unified semantic space. The multimodal feature alignment unit uses a distance formula to calculate the semantic relevance between these alignment features and entities in the knowledge graph. The closer the alignment feature is to the entity, the higher the semantic relevance. The feature is then bound to the entity with the highest semantic relevance. The multimodal feature alignment unit also uses a distance formula to calculate the association strength between pairwise alignment features and adjusts the relationship weights between entities in the knowledge graph based on the association strength between alignment features. As the correlation strength between alignment features increases, the relation weights between entities in the knowledge graph also increase simultaneously.

8. The intelligent disease diagnosis system according to claim 1, characterized in that, The dialogue status tracking unit is used to engage in real-time dialogue with the user and receive user input data, including symptoms, examination results, medical history, etc.

9. The intelligent disease diagnosis system according to claim 1, characterized in that, Based on multimodal features and dynamic knowledge graphs, the reasoning and diagnosis unit counts the frequency of occurrence of each disease entity and generates a more comprehensive diagnostic conclusion. When a user inputs relevant symptom features through the dialogue state tracking unit, the reasoning and diagnosis unit generates an initial hypothesis based on the aligned multimodal feature vector and the entity relationships in the knowledge graph. After multiple disease entities are matched based on multiple data input by the user, the reasoning and diagnosis unit calculates the frequency of each disease entity. The higher the frequency, the greater the probability of having the disease; the lower the frequency, the less likely the probability of having the disease. The disease entities are sorted according to their frequency to generate a reasoning and diagnosis report.

10. The intelligent disease diagnosis system according to claim 1, characterized in that, The diagnostic result output unit outputs and displays the diagnostic report from the reasoning diagnostic unit to the user, providing the user with a reference for medical treatment.

Images