Medical intelligent dialogue method based on deep learning

Abstract

The invention discloses a medical intelligent dialogue method based on deep learning, relates to the technical field of intelligent medical treatment, constructs a rare disease knowledge graph, and deduces implicit relations between diseases and symptoms through association relations in the graph even under the condition of insufficient training data. The ability of processing complex cases such as rare diseases is greatly enhanced, a more accurate rare disease list can be provided through a map reasoning mechanism, related diagnosis schemes are reasonably and preferentially displayed to a user, the misjudgment risk is reduced, and multi-source data such as medical record texts, medical images and laboratory detection results are effectively integrated through a multi-mode Transform model, so that the accuracy of diagnosis is improved. According to the method, a unified comprehensive feature representation is generated, the ability of comprehensively analyzing the health condition of the patient from different angles is deeply fused, the diagnosis accuracy is improved, the inquiry sequence and the problem depth can be adjusted according to the specific illness conditions of different patients based on the preliminary diagnosis result, and more accurate and personalized inquiry experience is provided.

Description

Technical Field

[0001] The present invention relates to the field of intelligent medical technology, and in particular to a medical intelligent dialogue method based on deep learning. Background Art

[0002] Rare diseases refer to diseases with extremely low incidence rates. There are more than 7,000 known rare diseases in the world, but the number of individual cases is small, and the causes, symptoms and treatment options of each disease are different. Data on these diseases are scarce and the conditions are highly complex.

[0003] Current medical intelligent dialogue methods rely on large-scale labeled data for training, but the number of cases of rare diseases is extremely small, and there is a lack of sufficient training data. It is difficult for the model to capture the characteristics of rare diseases, resulting in insufficient generalization capabilities in the diagnosis of rare diseases. In addition, rare diseases are usually accompanied by multiple causes, such as genetic factors, environmental influences, etc. The diagnostic data of rare diseases is also complex in causes and diverse in manifestations, making it difficult to effectively use data to prescribe the right medicine during inference and processing.

[0004] Traditional medical intelligent dialogue solutions, in response to the scarcity of rare disease data, mostly perform reasoning and diagnosis of rare diseases based on existing medical knowledge. They can provide a certain degree of accuracy in some scenarios, but their scalability is limited. Or they can expand the data set to cover more rare disease scenarios through data enhancement, but cannot make up for the inherent shortage of rare disease samples. The generated data is also likely to differ greatly from actual cases, resulting in low reasoning and diagnostic accuracy. Or they use expert systems. Although expert systems are effective in some rare disease scenarios, their rule bases are slow to update and are difficult to cope with the rapidly changing medical field. Therefore, there is an urgent need for a medical intelligent dialogue method based on deep learning to solve such problems. Summary of the invention

[0005] In view of the above existing problems, the present invention is proposed.

[0006] Therefore, the present invention provides a medical intelligent dialogue method based on deep learning to solve the problems of data scarcity, insufficient knowledge coverage, and limited reasoning ability faced by traditional medical intelligent dialogue solutions when dealing with rare diseases and utilizing knowledge graphs.

[0007] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0008] The present invention provides a medical intelligent dialogue method based on deep learning, which comprises:

[0009] Step S1, constructing a rare disease knowledge graph,

[0010] Based on existing literature, clinical guidelines, expert knowledge and data from medical research, the causes, symptoms, diagnostic methods, treatment options and drug interactions of rare diseases are structured to generate a knowledge graph. In the absence of sufficient data training, the knowledge graph serves as a knowledge base for preliminary diagnosis and provides a list of potential rare diseases related to symptoms.

[0011] Step S2, integrated multimodal data processing,

[0012] Introducing multimodal data, including medical records, medical images, and laboratory test results, using a multimodal Transformer model to process them, using the node information in the knowledge graph as prior knowledge, and jointly encoding and analyzing the input multimodal data to generate a comprehensive understanding of the multimodal information, including feature extraction results based on patient symptoms, images, and genetic data, namely multimodal features;

[0013] Step S3, knowledge graph reasoning and symptom matching,

[0014] Use knowledge graphs to match symptoms and combine multimodal data for preliminary diagnosis;

[0015] Step S4: Dialogue management and patient customization.

[0016] The order and content of the inquiries are dynamically adjusted based on the diagnosis results obtained in step S3, and the depth and direction of the questions are adjusted for patients with different conditions.

[0017] Furthermore, the knowledge graph contains nodes for symptoms, diseases, drugs, and diagnostic methods, and the knowledge graph serves as the core knowledge source of the dialogue system.

[0018] Furthermore, in step S1, the method of constructing a rare disease knowledge graph is as follows:

[0019] Based on literature, clinical guidelines and expert knowledge, collect relevant data on rare diseases, including causes, symptoms, diagnostic methods, treatment options and drug interactions, and define the data matrix D∈R m×n , where: D = {d ij}, d ij Represents the jth category of information about the ith rare disease, including causes, symptoms, and drugs. m is the total number of rare diseases, and n is the number of categories of information related to each disease. The heterogeneous data is normalized, and the normalization function is:

[0020] in, is the normalized data, μ j is the mean value of the j-th type of information, σ j is the standard deviation of the category information, for each rare disease Define the following nodes:

[0021] S i ={s1,s2,…,s p} indicates disease Related p symptoms,

[0022] T i ={t1,t2,…,t q} indicates disease Related q treatment methods,

[0023] M i ={m1,m2,…,m r} indicates disease Related r drugs,

[0024] The relationship between nodes is represented by the relationship matrix R∈R k×k express: Among them, R ij Indicates disease and diseases The strength of the relationship between , f(·) represents the relationship definition function based on similarity, using the Jaccard similarity coefficient;

[0025] Use graph embedding method to embed nodes and relationships in knowledge graph into vector space. Let for each node v i , and its corresponding vector is represented as v i ∈R d , d is the embedding dimension, then the relationship r in the knowledge graph ij The embedding representation is: ij =v i +rv j , where r ij Indicates disease and diseases The relationship between them is represented by the embedding of r, v i and v j Diseases and The node vector representation of i And the relationship r, derive the possibility of potential diseases and define the inference probability function: in, It means that given a set of symptoms S, the disease The diagnosis probability, <v j ,S> represents the disease node v i The inner product between the symptom set S, where S is the embedding representation of the given symptom set, outputs the most likely diagnosis result of the rare disease based on the result of reasoning;

[0026] Diagnosis The score is: in, Indicates disease With input symptoms k The matching degree, ω k Symptoms k The weights are used to sort and output a potential list of rare diseases by combining the matching degree between the comprehensive symptoms and the diseases.

[0027] Furthermore, in step S2, a multimodal Transformer model is used to jointly encode and analyze different data types such as medical record text, medical images, and laboratory test results:

[0028] Process medical record text data and extract symptoms, diagnosis and medical history information. Suppose the medical record text sequence is T = {t1, t2, …, t n}, where t i The word vector embedding representing the i-th word is converted into a high-level semantic representation through the self-attention mechanism: Among them, H T ∈R n×d A high-dimensional semantic representation of medical record text, are the query, key, and value matrices of the text sequence, respectively, and d k For the dimension of each attention head, represents the transposition operation, n is the length of the medical record text, and d is the final embedding dimension;

[0029] For medical image data, convolutional neural network CNN is used for feature extraction. The input medical image is assumed to be a three-dimensional matrix I∈R h×w×c , where h and w are the height and width of the image, and c is the number of channels. Through the CNN network, it is converted into a low-dimensional feature representation: F I =CNN(I), where F I ∈R m×d represents the extracted image features, m is the number of spatial positions in the feature map, and d is the feature dimension of each position;

[0030] The laboratory test results are represented by a vector L = {l1,l2,…,l k}, where l i is the result of the i-th detection, mapped to the feature space through the fully connected layer: F L =W L L+b L , where F L ∈R d Represents the characteristic representation of laboratory test results, W L ∈R d×k is the weight matrix, b L ∈R dis the bias term, k is the number of detection items, and d is the final feature dimension.

[0031] Furthermore, in step S2, a multimodal Transformer model is used, and the characteristics of the three types of data are H T 、F I and F L , fused through a multimodal self-attention mechanism:

[0032] Among them, Z∈R (n+m+1)×d is the fused multimodal feature representation, where n, m, and 1 correspond to the number of features of text, image, and laboratory test data, respectively, and Q M ,K M ,V M ∈R (n+m+1)×d are the fused query, key, and value matrices respectively, and d is the embedding dimension, where:

[0033] Q M =[H T ; F I ; F L ]W Q , K M =[H T ; F I ; F L ]W K , V M =[H T ; F I ; F L ]W V , here

[0034] [H T ; F I ; F L ] represents the concatenation of text, image and laboratory test features, W Q , W K , W V ∈R d×d are the mapping matrices for query, key, and value respectively;

[0035] The fused multimodal feature Z is obtained through the multimodal self-attention mechanism, and global pooling is performed to generate the final comprehensive feature vector F M : F M =GlobalPool(Z), where F M ∈R d is a multimodal feature vector, which is used for subsequent diagnostic reasoning and symptom matching. The multimodal feature vector F M As a result of comprehensive understanding of multimodal information; feature extraction and comprehensive understanding of multimodal data are achieved.

[0036] Furthermore, in step S3, based on the knowledge graph generated in step S1, the symptoms input by the patient and the multimodal features are matched through the knowledge reasoning mechanism, and a preliminary diagnosis result is output, including a list of possible diseases, etiology speculation and preliminary treatment recommendations.

[0037] Furthermore, in step S3, the symptoms and multimodal features input by the patient are matched with the knowledge graph generated in step S1 through the knowledge reasoning mechanism, and a preliminary diagnosis result is output:

[0038] Suppose the rare disease knowledge graph G = (V, E), where V is a node set representing disease, symptom, drug, and diagnostic method entities, and E is an edge set representing the relationship between entities. For each node v i ∈V, with vector v i ∈R d Represents, where d is the embedding dimension, and the node embedding formula is: i =f(v i ), where v i is the node v in the knowledge graph i The embedding vector, f(v i ) is to convert node v i The function converted to an embedding vector is converted by a graph embedding method (such as TransE or GraphSAGE), where d is the dimension of the embedding vector;

[0039] Multimodal feature F M A comprehensive vector representing the patient's input symptoms, medical images, and laboratory test results, with the same dimension as the knowledge graph node embedding v i Same, multimodal feature F M Match with the nodes in the knowledge graph, F M =GlobalPool(Z), where F M is the fused multimodal feature vector, representing the comprehensive information of the patient, and Z is the feature matrix after fusing text, image, and laboratory data.

[0040] Furthermore, in step S3, symptoms and diseases are matched.

[0041] In the knowledge graph, there is an edge between diseases and symptoms. ij ∈E, the multimodal features F of the patient input M and the disease node v in the graph i Match and calculate the similarity between symptoms and diseases. Matching formula:

[0042] Among them, S(F M ,v i ) represents the multimodal features F of the patientM With disease node v i The cosine similarity of |F is used to evaluate the matching degree between the input symptoms and the disease. M | and |v i |F M and v i The vector norm of , is the inner product of two vectors;

[0043] Define a relationship-based reasoning model, deduce the relationships in the knowledge graph, and calculate the matching degree of potential diseases. The reasoning formula is: in, Represents the multimodal features of the patient F M Matching diseases The probability of S(F M ,v j ) is the multimodal feature and disease The similarity, R ij Indicates disease and disease The strength of the association between them comes from the edge weight in the knowledge graph, α j For each disease node The importance weight of a is set according to its centrality or frequency of occurrence in the knowledge graph.

[0044] Furthermore, in step S3, the preliminary diagnosis result is output.

[0045] According to the inferred disease matching probability Sort the diseases and output the most likely diagnosis results for rare diseases. The disease list is based on Sort from high to low, take the first k results, sorting formula:

[0046] in, is the sorted list of the first k diseases, According to Sort the diseases from large to small, and k is the number of candidate diseases output.

[0047] Furthermore, in step S4, the dialogue management and patient personalization are as follows:

[0048] Based on the preliminary diagnosis results of rare diseases in step S3, diseases with higher probabilities in the preliminary diagnosis results are prioritized for inquiry;

[0049] Dynamically adjust the depth of the questions based on the complexity and severity of different conditions.

[0050] The beneficial effects of the present invention are:

[0051] The present invention constructs a rare disease knowledge graph. Even when the training data is insufficient, it can still deduce the implicit connection between the disease and the symptoms through the association in the graph, which greatly enhances the ability to handle complex cases such as rare diseases. Through the graph reasoning mechanism, it can provide a more accurate list of rare diseases and reasonably prioritize the relevant diagnostic schemes to users, reducing the risk of misjudgment.

[0052] This invention uses a multimodal Transformer model to effectively integrate multi-source data such as medical record texts, medical images, and laboratory test results, generate a unified comprehensive feature representation, and deeply integrate the ability to comprehensively analyze the patient's health status from different angles, thereby improving the accuracy of diagnosis.

[0053] Based on the preliminary diagnosis results, the present invention can adjust the order of inquiries and the depth of questions according to the specific conditions of different patients. For situations with a high probability of rare diseases, the system will give priority to exploring the key symptoms related to the disease, providing a more accurate and personalized inquiry experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0055] Figure 1 This is a flowchart of the deep learning-based medical intelligent dialogue method of the present invention. DETAILED DESCRIPTION

[0056] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.

[0057] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0058] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0059] Example 1, reference Figure 1, this embodiment provides a medical intelligent dialogue method based on deep learning, comprising the following steps:

[0060] Step S1, constructing a rare disease knowledge graph,

[0061] Based on existing literature, clinical guidelines, expert knowledge and data from medical research, the causes, symptoms, diagnostic methods, treatment options and drug interactions of rare diseases are structured to generate a knowledge graph. In the absence of sufficient data training, the knowledge graph serves as a knowledge base for preliminary diagnosis and provides a list of potential rare diseases related to symptoms.

[0062] The knowledge graph contains nodes for symptoms, diseases, drugs, and diagnostic methods. The knowledge graph serves as the core knowledge source of the dialogue system.

[0063] In step S1, the method of constructing a rare disease knowledge graph is as follows:

[0064] Based on literature, clinical guidelines and expert knowledge, collect relevant data on rare diseases, including causes, symptoms, diagnostic methods, treatment options and drug interactions, and define the data matrix D∈R m×n , where: D = {d ij}, d ij Represents the jth category of information about the ith rare disease, including causes, symptoms, and drugs. m is the total number of rare diseases, and n is the number of categories of information related to each disease. The heterogeneous data is normalized, and the normalization function is:

[0065] in, is the normalized data, μ j is the mean value of the j-th type of information, σ j is the standard deviation of the category information, for each rare disease Define the following nodes:

[0066] S i ={s1,s2,…,s p} indicates disease Related p symptoms,

[0067] T i ={t1,t2,…,t q} indicates disease Related q treatment methods,

[0068] M i ={m1,m2,…,m r} indicates disease Related r drugs,

[0069] The relationship between nodes is represented by the relationship matrix R∈Rk×k express: Among them, R ij Indicates disease and diseases The strength of the relationship between , f(·) represents the relationship definition function based on similarity, using the Jaccard similarity coefficient;

[0070] Use graph embedding method to embed nodes and relationships in knowledge graph into vector space. Let for each node v i , and its corresponding vector is represented as v i ∈R d , d is the embedding dimension, then the relationship r in the knowledge graph ij The embedding representation is: ij =v i +rv j , where r ij Indicates disease and diseases The relationship between them is represented by the embedding of r, v i and v j Diseases and The node vector representation of i And the relationship r, derive the possibility of potential diseases and define the inference probability function: in, It means that given a set of symptoms S, the disease The diagnosis probability, <v j ,S> represents the disease node v i The inner product between the symptom set S, where S is the embedding representation of the given symptom set, outputs the most likely diagnosis result of the rare disease based on the result of reasoning;

[0071] Diagnosis The score is: in, Indicates disease With input symptoms k The matching degree, ω k Symptoms k The weights are used to sort and output a potential list of rare diseases by the matching degree between comprehensive symptoms and diseases;

[0072] Specifically, the constructed rare disease knowledge graph achieves a highly structured integration of medical knowledge. The knowledge graph covers information such as the causes, symptoms, diagnostic methods, treatment plans, and drug interactions of rare diseases. In the absence of sufficient training data, it can rely on existing medical knowledge for preliminary diagnosis, and based on the embedded similarity of symptoms and disease nodes, it can quickly identify a list of possible rare diseases.

[0073] Step S2, integrated multimodal data processing,

[0074] Introducing multimodal data, including medical records, medical images, and laboratory test results, using a multimodal Transformer model to process them, using the node information in the knowledge graph as prior knowledge, and jointly encoding and analyzing the input multimodal data to generate a comprehensive understanding of the multimodal information, including feature extraction results based on patient symptoms, images, and genetic data, namely multimodal features;

[0075] In step S2, a multimodal Transformer model is used to jointly encode and analyze different data types such as medical record text, medical images, and laboratory test results:

[0076] Process medical record text data and extract symptoms, diagnosis and medical history information. Suppose the medical record text sequence is T = {t1, t2, …, t n}, where t i The word vector embedding representing the i-th word is converted into a high-level semantic representation through the self-attention mechanism: Among them, H T ∈R n×d A high-dimensional semantic representation of medical record text, are the query, key, and value matrices of the text sequence, respectively, and d k For the dimension of each attention head, represents the transposition operation, n is the length of the medical record text, and d is the final embedding dimension;

[0077] For medical image data, convolutional neural network CNN is used for feature extraction. The input medical image is assumed to be a three-dimensional matrix I∈R h×w×c , where h and w are the height and width of the image, and c is the number of channels. Through the CNN network, it is converted into a low-dimensional feature representation: F I =CNN(I), where F I ∈R m×d represents the extracted image features, m is the number of spatial positions in the feature map, and d is the feature dimension of each position;

[0078] The laboratory test results are represented by a vector L = {l1,l2,…,l k}, where l i is the result of the i-th detection, mapped to the feature space through the fully connected layer: F L =W L L+b L , where F L ∈R d Represents the characteristic representation of laboratory test results, W L ∈R d×kis the weight matrix, b L ∈R d is the bias term, k is the number of detection items, and d is the final feature dimension;

[0079] In step S2, a multimodal Transformer model is used, and the characteristics of the three types of data are H T 、F I and F L , fused through a multimodal self-attention mechanism:

[0080] Among them, Z∈R (n+m+1)×d is the fused multimodal feature representation, where n, m, and 1 correspond to the number of features of text, image, and laboratory test data, respectively, and Q M ,K M ,V M ∈R (n+m+1)×d are the fused query, key, and value matrices respectively, and d is the embedding dimension, where:

[0081] Q M =[H T ; F I ; F L ]W Q , K M =[H T ; F I ; F L ]W K , V M =[H T ; F I ; F L ]W V , here

[0082] [H T ; F I ; F L ] represents the concatenation of text, image and laboratory test features, W Q , W K , W V ∈R d×d are the mapping matrices for query, key, and value respectively;

[0083] The fused multimodal feature Z is obtained through the multimodal self-attention mechanism, and global pooling is performed to generate the final comprehensive feature vector F M : F M =GlobalPool(Z), where F M ∈R d is a multimodal feature vector, which is used for subsequent diagnostic reasoning and symptom matching. The multimodal feature vector F MAs a result of comprehensive understanding of multimodal information; to achieve feature extraction and comprehensive understanding of multimodal data;

[0084] Specifically, the introduction of multimodal data processing (including medical record text, medical images and laboratory test results) can effectively enhance diagnostic accuracy. The multimodal Transformer model is used to fuse data from various sources and generate a unified comprehensive feature representation, which can comprehensively analyze the patient's condition from multiple angles and improve the ability to identify complex cases.

[0085] Step S3, knowledge graph reasoning and symptom matching,

[0086] Use knowledge graphs to match symptoms and combine multimodal data for preliminary diagnosis;

[0087] In step S3, based on the knowledge graph generated in step S1, the symptoms input by the patient and the multimodal features are matched through the knowledge reasoning mechanism to output a preliminary diagnosis result, including a list of possible diseases, etiology speculation, and preliminary treatment recommendations;

[0088] In step S3, the symptoms and multimodal features input by the patient are matched with the knowledge graph generated in step S1 through the knowledge reasoning mechanism, and the preliminary diagnosis results are output:

[0089] Suppose the rare disease knowledge graph G = (V, E), where V is a node set representing disease, symptom, drug, and diagnostic method entities, and E is an edge set representing the relationship between entities. For each node v i ∈V, with vector v i ∈R d Represents, where d is the embedding dimension, and the node embedding formula is: i =f(v i ), where v i is the node v in the knowledge graph i The embedding vector, f(v i ) is to convert node v i The function converted to an embedding vector is converted by a graph embedding method (such as TransE or GraphSAGE), where d is the dimension of the embedding vector;

[0090] Multimodal feature F M A comprehensive vector representing the patient's input symptoms, medical images, and laboratory test results, with the same dimension as the knowledge graph node embedding v i Same, multimodal feature F M Match with the nodes in the knowledge graph, F M =GlobalPool(Z), where F Mis the fused multimodal feature vector, representing the comprehensive information of the patient, and Z is the feature matrix after fusing text, image, and laboratory data;

[0091] In step S3, symptoms and diseases are matched.

[0092] In the knowledge graph, there is an edge between diseases and symptoms. ij ∈E, the multimodal features F of the patient input M and the disease node v in the graph i Match and calculate the similarity between symptoms and diseases. Matching formula:

[0093] Among them, S(F M ,v i ) represents the multimodal features F of the patient M With disease node v i The cosine similarity of |F is used to evaluate the matching degree between the input symptoms and the disease. M | and |v i |F M and v i The vector norm of , is the inner product of two vectors;

[0094] Define a relationship-based reasoning model, deduce the relationships in the knowledge graph, and calculate the matching degree of potential diseases. The reasoning formula is: in, Represents the multimodal features of the patient F M Matching diseases The probability of S(F M ,v j ) is the multimodal feature and disease The similarity, R ij Indicates disease and disease The strength of the association between them comes from the edge weight in the knowledge graph, α j For each disease node The importance weight of a word is set according to its centrality or frequency of occurrence in the knowledge graph;

[0095] In step S3, the preliminary diagnosis result is output.

[0096] According to the inferred disease matching probability Sort the diseases and output the most likely diagnosis results for rare diseases. The disease list is based on Sort from high to low, take the first k results, sorting formula:

[0097] in, is the sorted list of the first k diseases, According to Sort the diseases from largest to smallest, k is the number of candidate diseases output;

[0098] Step S4: Dialogue management and patient customization.

[0099] Dynamically adjust the order and content of the inquiries based on the diagnosis results obtained in step S3, and adjust the depth and direction of the questions for patients with different conditions;

[0100] In step S4, the dialogue management and patient personalization methods are as follows:

[0101] Based on the preliminary diagnosis results of rare diseases in step S3, diseases with higher probabilities in the preliminary diagnosis results are prioritized for inquiry;

[0102] Dynamically adjust the depth of questions according to the complexity and severity of different conditions;

[0103] Specifically, by combining knowledge graphs and multimodal features, and using reasoning mechanisms to achieve deeper diagnostic capabilities, it not only relies on a simple match between symptoms and diseases, but also infers through the associations in the graph. It can identify implicit associations between symptoms and potential rare diseases, greatly improving the depth and breadth of diagnosis.

[0104] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A medical intelligent dialogue method based on deep learning, characterized by: include, Step S1, constructing a rare disease knowledge graph, based on existing literature, clinical guidelines, expert knowledge and data from medical research, to structure the information representation of the causes, symptoms, diagnostic methods, treatment options and drug interactions of rare diseases and generate a knowledge graph; Step S2, integrated multimodal data processing, introduces multimodal data, including medical record text, medical images and laboratory test results, and uses a multimodal Transformer model for processing. The node information in the knowledge graph is used as prior knowledge, and is jointly encoded and analyzed with the input multimodal data to generate a comprehensive understanding of the multimodal information, including feature extraction results based on patient symptoms, images and genetic data, namely multimodal features; Step S3: knowledge graph reasoning and symptom matching, using knowledge graph to match symptoms and combining multimodal data for preliminary diagnosis; Step S4, dialogue management and patient customization, dynamically adjusts the order and content of inquiries based on the diagnosis results obtained in step S3, and adjusts the depth and direction of questions for patients with different conditions.

2. According to claim 1, a medical intelligent dialogue method based on deep learning is characterized in that: The knowledge graph contains nodes for symptoms, diseases, drugs, and diagnostic methods.

3. The medical intelligent dialogue method based on deep learning according to claim 2, characterized in that: In step S1, the method of constructing a rare disease knowledge graph is as follows: Based on literature, clinical guidelines and expert knowledge, collect relevant data on rare diseases, including causes, symptoms, diagnostic methods, treatment options and drug interactions, and define the data matrix D∈R m×n , where: D = {d ij }, d ij Represents the jth category of information about the ith rare disease, including causes, symptoms, and drugs. m is the total number of rare diseases, and n is the number of categories of information related to each disease. The heterogeneous data is normalized, and the normalization function is: in, is the normalized data, μ j is the mean value of the j-th type of information, σ j is the standard deviation of the category information, for each rare disease Define the following nodes: S i ={s1,s2,…,s p } indicates disease Related p symptoms, T i ={t1,t2,…,t q } indicates disease Related q treatment methods, M i ={m1,m2,…,m r } indicates disease Related r drugs, The relationship between nodes is represented by the relationship matrix R∈R k×k express: Among them, R ij Indicates disease and diseases The strength of the relationship between , f(·) represents the relationship definition function based on similarity, using the Jaccard similarity coefficient; Use graph embedding method to embed nodes and relationships in knowledge graph into vector space. Let for each node v i , and its corresponding vector is represented as v i ∈R d , d is the embedding dimension, then the relationship r in the knowledge graph ij The embedding representation is: ij =v i +rv j , where r ij Indicates disease and diseases The relationship between them is represented by the embedding of r, v i and v j Diseases and The node vector representation of i And the relationship r, derive the possibility of potential diseases and define the inference probability function: in, It means that given a set of symptoms S, the disease The diagnosis probability, <v j ,S> represents the disease node v i The inner product between the symptom set S, where S is the embedding representation of the given symptom set, outputs the most likely diagnosis result of the rare disease based on the result of reasoning; Diagnosis The score is: in, Indicates disease With input symptoms k The matching degree, ω k Symptoms k The weights are used to sort and output a potential list of rare diseases by combining the matching degree between the comprehensive symptoms and the diseases.

4. The medical intelligent dialogue method based on deep learning according to claim 3, characterized in that: In step S2, a multimodal Transformer model is used for joint encoding and analysis: Process medical record text data and extract symptoms, diagnosis and medical history information. Suppose the medical record text sequence is T = {t1, t2, …, t n }, where t i The word vector embedding representing the i-th word is converted into a high-level semantic representation through the self-attention mechanism: Among them, H T ∈R n×d Represents the high-dimensional semantic representation of medical record text, are the query, key, and value matrices of the text sequence, respectively, and d k is the dimension of each attention head, represents the transposition operation, n is the length of the medical record text, and d is the final embedding dimension; For medical image data, convolutional neural network CNN is used for feature extraction. The input medical image is assumed to be a three-dimensional matrix I∈R h×w×c , where h and w are the height and width of the image, and c is the number of channels. Through the CNN network, it is converted into a low-dimensional feature representation: F I =CNN(I), where F I ∈R m×d represents the extracted image features, m is the number of spatial positions in the feature map, and d is the feature dimension of each position; The laboratory test results are represented by a vector L = {l1,l2,…,l k }, where l i is the result of the i-th detection, mapped to the feature space through the fully connected layer: F L =W L L+b L , where F L ∈R d Represents the characteristic representation of laboratory test results, W L ∈R d×k is the weight matrix, b L ∈R d is the bias term, k is the number of detection items, and d is the final feature dimension.

5. The medical intelligent dialogue method based on deep learning according to claim 4, characterized in that: In step S2, a multimodal Transformer model is used, and the characteristics of the three types of data are H T 、F I and F L , fused through a multimodal self-attention mechanism: Among them, Z∈R (n+m+1)×d is the fused multimodal feature representation, where n, m, and 1 correspond to the number of features of text, image, and laboratory test data, respectively, and Q M ,K M ,V M ∈R (n+m+1)×d are the fused query, key, and value matrices respectively, and d is the embedding dimension, where: Q M = [H T ; F I ; F L W Q , K M = [H T ; F I ; F L W K , V M = [H T ; F I ; F L W V , where [H T ; F I ; F L ] represents the concatenation of text, image and laboratory test features, W Q , W K , W V ∈R d×d are the mapping matrices for query, key, and value respectively; The fused multimodal feature Z is obtained through the multimodal self-attention mechanism, and global pooling is performed to generate the final comprehensive feature vector F M : F M =GlobalPool(Z), where F M ∈R d is a multimodal feature vector, which is used for subsequent diagnostic reasoning and symptom matching. The multimodal feature vector F M As a result of comprehensive understanding of multimodal information.

6. The medical intelligent dialogue method based on deep learning according to claim 5, characterized in that: In step S3, based on the knowledge graph generated in step S1, the symptoms input by the patient and the multimodal features are matched through the knowledge reasoning mechanism to output a preliminary diagnosis result, including a list of possible diseases, etiology speculation and preliminary treatment recommendations.

7. The medical intelligent dialogue method based on deep learning according to claim 6, characterized in that: In step S3, the symptoms and multimodal features input by the patient are matched with the knowledge graph generated in step S1 through the knowledge reasoning mechanism, and the preliminary diagnosis results are output: Suppose the rare disease knowledge graph G = (V, E), where V is a node set representing disease, symptom, drug, and diagnostic method entities, and E is an edge set representing the relationship between entities. For each node v i ∈V, with vector v i ∈R d Represents, where d is the embedding dimension, and the node embedding formula is: i =f(v i ), where v i is the node v in the knowledge graph i The embedding vector, f(v i ) is to convert node v i Function converted to embedding vector, d is the dimension of the embedding vector; Multimodal feature F M A comprehensive vector representing the patient's input symptoms, medical images, and laboratory test results, with the same dimension as the knowledge graph node embedding v i Same, multimodal feature F M Match with the nodes in the knowledge graph, F M =GlobalPool(Z), where F M is the fused multimodal feature vector, representing the comprehensive information of the patient, and Z is the feature matrix after fusing text, image, and laboratory data.

8. The medical intelligent dialogue method based on deep learning according to claim 7, characterized in that: In step S3, symptoms and diseases are matched. In the knowledge graph, there is an edge between diseases and symptoms. ij ∈E, the multimodal features F of the patient input M and the disease node v in the graph i Match and calculate the similarity between symptoms and diseases. Matching formula: Among them, S(F M ,v i ) represents the multimodal features F of the patient M With disease node v i The cosine similarity of |F is used to evaluate the matching degree between the input symptoms and the disease. M | and |v i |F M and v i The vector norm of , is the inner product of two vectors; Define a relationship-based reasoning model, deduce the relationships in the knowledge graph, and calculate the matching degree of potential diseases. The reasoning formula is: in, Represents the multimodal features of the patient F M Matching diseases The probability of S(F M ,v j ) is the multimodal feature and disease The similarity, R ij Indicates disease and disease The strength of the association between them comes from the edge weight in the knowledge graph, α j For each disease node The importance weight of a is set according to its centrality or frequency of occurrence in the knowledge graph.

9. The medical intelligent dialogue method based on deep learning according to claim 8, characterized in that: In step S3, the preliminary diagnosis result is output. According to the inferred disease matching probability Sort the diseases and output the most likely diagnosis results for rare diseases. The disease list is based on Sort from high to low, take the first k results, sorting formula: in, is the sorted list of the first k diseases, According to Sort the diseases from large to small, and k is the number of candidate diseases output.

10. The medical intelligent dialogue method based on deep learning according to claim 9, characterized in that: In step S4, the dialogue management and patient personalization methods are as follows: Based on the preliminary diagnosis results of rare diseases in step S3, diseases with higher probabilities in the preliminary diagnosis results are prioritized for inquiry; Dynamically adjust the depth of the questions based on the complexity and severity of different conditions.

Images