An auxiliary diagnosis method and system based on multi-view knowledge graph
A multi-view knowledge graph system integrates patient data and medical knowledge to enhance diagnostic accuracy by constructing node representations and prediction models, addressing the complexity of medical data integration and association challenges.
Patent Information
- Application Number
- CN202411164369.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-23
- Publication Date
- 2024-12-13
- Estimated Expiration
- 2044-08-23
AI Technical Summary
Existing medical diagnosis systems face challenges in integrating and reasoning with the numerous concepts and complex internal relationships of medical knowledge, particularly in disease diagnosis, due to the complexity of medical data and the lack of comprehensive association between patient information and external medical knowledge.
An auxiliary diagnosis method and system utilizing a multi-view knowledge graph that integrates patient examination data, imaging data, medical records, and historical medication data, leveraging large language models and neural networks to construct node representations and perform contrastive learning, ultimately constructing a diagnosis-assisted prediction model for improved diagnostic accuracy.
Enhances diagnostic prediction by effectively mining comprehensive associations through multi-view patient data, providing quantifiable and professional knowledge-based support for medical diagnosis, improving diagnostic accuracy and providing references for doctors.
Smart Images
Figure CN118692661B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of smart medical technology, and more specifically, to an auxiliary diagnosis method and system based on a multi-view knowledge graph. Background Art
[0002] With the rapid development of medical technology and the advent of the big data era, the acquisition, integration and application of knowledge in the medical and health field are particularly important. As a tool that graphically represents knowledge in the medical and health field, knowledge graphs provide new ideas and methods for solving these problems. As an important achievement in the field of information technology and artificial intelligence, the application of knowledge graphs in the field of disease diagnosis has gradually become a research hotspot. By integrating and associating a large amount of medical information and data, knowledge graphs provide a multi-dimensional and structured knowledge framework for the diagnosis and treatment of diseases. On this basis, combined with machine learning and data mining technology, knowledge graphs can provide potential associations, etiology analysis and treatment plan recommendations for diseases. In the field of medical health, knowledge graphs can cover many entities such as diseases, symptoms, drugs, and treatment methods. Compared with traditional single entity representation and representation training, they provide richer information, including connection information between entities and explicit and implicit relationships of long paths. However, due to the numerous concepts and complex internal relationships of medical knowledge, data-based reasoning has brought great challenges.
[0003] In recent years, with the rapid development and popularization of large language models, their rich background knowledge and massive data training models have provided new technologies and data support for improving medical and health auxiliary diagnosis based on data mining. Therefore, this solution proposes an auxiliary diagnosis method and system based on multi-view knowledge graphs, which provides quantifiable and professional knowledge-based auxiliary support for medical and health auxiliary diagnosis by promoting the fusion analysis of background knowledge and multi-view data of patients. Summary of the invention
[0004] In order to solve the above technical problems, the present invention proposes an auxiliary diagnosis method and system based on a multi-view knowledge graph. With the help of external medical and health knowledge and multi-view patient current status information and historical conditions, more comprehensive association relationships are mined through the knowledge graph, thereby improving the patient's diagnostic prediction effect and providing reference and verification for the doctor's further diagnosis.
[0005] The first aspect of the present invention provides an auxiliary diagnosis method based on a multi-view knowledge graph, comprising:
[0006] Obtaining preliminary conclusions of the patient's examination data from a patient information database, performing prompt query on the preliminary conclusions of the examination data using a large language model, and obtaining knowledge expansion data based on the preliminary conclusions of the examination data;
[0007] Obtain the patient's structured examination data, imaging data, medical records, historical medication data, and knowledge expansion data, and train and construct node representations of patient information according to the above four views;
[0008] The node representations of the four views are constructed into a knowledge graph, and after pre-training on the knowledge graph, the patient fusion information representation is obtained;
[0009] A diagnosis-assisted prediction model is constructed, and the patient fusion information representation is input into the diagnosis-assisted prediction model to obtain the patient's diagnosis prediction result.
[0010] In this solution, the preliminary conclusion of the inspection data is used to perform prompt query using a large language model to obtain knowledge expansion data based on the preliminary conclusion of the inspection data, specifically:
[0011] Generating a query instruction according to the format requirements of the pre-trained large language model interface prompt template using the preliminary examination conclusion data retrieved from the patient information database, and using the query instruction to call at least one pre-trained large language model interface to ask knowledge expansion questions;
[0012] Generate knowledge expansion question and answer pairs for the preliminary conclusion data of the examination, and obtain n knowledge vectors with the highest similarity to the query quality as return results based on the knowledge expansion question and answer pairs; remove duplicates and splice multiple returned results to form knowledge expansion data of the preliminary conclusion of the patient's examination data.
[0013] In this solution, we obtain the patient's structured examination data, imaging data, medical records, historical medication data, and knowledge extension data, and train and construct node representations of patient information according to the above four views, specifically:
[0014] Retrieve the patient's structured examination data, imaging data, medical records and historical medication data based on the patient information database, and build a structured examination data view, imaging data view, medical records and historical medication data view and knowledge expansion data view respectively;
[0015] Given a multi-view dataset , , Shidi Information about the view, Indicates View View information for each patient, , import the multi-view dataset into a multi-layer perceptron, and use the encoder of the multi-layer perceptron and decoder right Reconstruction is performed to obtain the latent space representation of the patient based on the nonlinear mapping ability of the neural network, which is expressed as:
[0016] ,
[0017] ,
[0018] in Indicates View The latent space representation of each patient, represents the corresponding reconstructed attribute representation;
[0019] Contrastive multi-view learning is performed on the latent space features of the patient in the four views. The patient nodes between different views are used as positive samples, and other nodes within and between views are used as negative samples. The node representations of patient information in different views are obtained through contrastive learning.
[0020] In this scheme, for contrastive multi-view learning, the loss function Defined as:
[0021] ,
[0022] in and Respectively represent The reconstruction loss and contrast loss of each view, represents the trade-off parameter, Indicates the number of views, represents the number of patients, and They are defined as:
[0023] ,
[0024] ,
[0025] in, Indicates View The latent space representation of each patient is View The similarity between the latent space representations of each patient, Respectively represent View Each patient and The latent space representation of each patient, Represents a hyperparameter.
[0026] In this solution, the node representations of the four views are constructed into a knowledge graph. After pre-training of the knowledge graph, the patient fusion information representation is obtained, which is as follows:
[0027] The node representations of the patient information in the four views are converted into triples to construct a knowledge graph, where the patient is the head node, the relationship is the patient's attribute, and the tail node is the node representation of different views;
[0028] Initialize the 𝑘 nearest neighbor graph of the patient node according to the patient's structured examination data, imaging data, medical records and historical medication data, and knowledge extension data, and use the 𝑘 nearest neighbor graph to obtain the connection relationship of the patient nodes in the knowledge graph;
[0029] A structure-related convolutional network framework is constructed to train the knowledge graph, including a weighted graph convolutional network encoding module and a convolutional TransE decoding module, to obtain a fusion information representation of patients based on corresponding subgraphs of different views.
[0030] In this scheme, for the weighted graph convolutional network encoding module, by aggregating the adjacent node relationships of the patient nodes, the patient fusion information representation based on the corresponding subgraphs of different views is obtained:
[0031] ,
[0032] in Indicates The patient fusion information representation of the layer, From multiple views Splicing obtained, represents the weighted sum of the subgraph adjacency matrix plus the self-connections, represents the weight parameter, represents the activation function;
[0033] For the convolutional TransE decoding module, the embedding convolution is performed based on C convolution kernels of different sizes, and the three feature maps obtained are concatenated and sent to the fully connected layer to be projected to the embedding dimension;
[0034] Scoring function of the convolutional TransE decoding module after nonlinear convolution Defined as:
[0035] ,
[0036] in represents a nonlinear function, Represents the feature map extracted by the convolution kernel, represents the head node and tail node embedding, represents relation embedding, represents the weight coefficient, Indicates the concatenation of multiple feature maps. During the training process, the logistic sigmoid function is applied to the score to obtain the final score function , expressed as:
[0037]
[0038] in Represents the logistic sigmoid function.
[0039] In this solution, a diagnosis-assisted prediction model is constructed, the diagnosis-assisted prediction model is trained using training data, the patient fusion representation samples in the training data are input into the diagnosis-assisted prediction model for training and verification, and the diagnosis-assisted prediction model that meets the preset standards after training is exported;
[0040] Obtain patient fusion information representation based on corresponding sub-images of different views As the model input, the parameters are mapped in the diagnosis-assisted prediction model and then the predicted probability of illness of different patients is obtained through the activation function, which is expressed as:
[0041] ,
[0042] in Indicates the probability of different patients suffering from the preset disease category, It represents parameter characterization;
[0043] The patient's diagnostic prediction results are sent and visualized according to the preset method, and the doctor's feedback on the accuracy of the diagnostic prediction results is obtained. Based on the accuracy feedback information, the diagnostic prediction results that meet the preset accuracy requirements are screened and returned to the diagnostic auxiliary prediction model for the next optimization training.
[0044] The second aspect of the present invention provides an auxiliary diagnosis system based on a multi-view knowledge graph, comprising: a patient information database, a patient examination result expansion module, a prompt information generation module, a patient structured examination data representation module, a patient imaging data representation module, a patient medical record and historical medication data representation module, a knowledge graph construction module, a patient fusion information representation module, a patient diagnosis prediction module, a prediction training module and a prediction result output and feedback module;
[0045] The patient information database is responsible for storing the patient's historical examination data, medical record text data, and medication information data, and storing them according to different data types for specific view data processing and query;
[0046] The patient examination result expansion module: obtains a query instruction according to the preliminary conclusion of the patient's examination data, calls at least one pre-trained large language model interface to ask questions for knowledge expansion, and when there are multiple returned results, is responsible for de-duplication and splicing the obtained question result data to form the patient examination result expansion data;
[0047] The prompt information generation module is responsible for managing the information template that calls the pre-trained large language model interface, and generates a query instruction according to the format requirements of the prompt template for the preliminary conclusion data of the examination provided by the patient examination result extension module, and returns the query instruction to the patient examination result extension module;
[0048] The patient structured examination data representation module is responsible for formatting and preprocessing the patient structured examination data, generating fusion representation information of the structured data, and returning the representation information;
[0049] The patient image data representation module is responsible for denoising and size standardizing the image data of the patient examination, calling the pre-trained graph representation model to generate the representation of the image data, and returning the representation information;
[0050] The patient medical record and historical medication data representation module is responsible for generating text representations of the patient medical record and historical medication data through a text representation pre-training model, and returning the representation information;
[0051] The patient examination result extended data representation module is responsible for preprocessing the patient's examination result extended data, generating representation information of the examination result extended data, and returning the representation information;
[0052] The knowledge graph construction module is responsible for pre-processing the representation of the patient's structured examination data, imaging data, medical records and historical medication data, and knowledge extension data into a triple format, and adding it to the knowledge graph for extended training;
[0053] The patient fusion information representation module is responsible for generating information representation from the patient fusion information in the knowledge graph and returning the representation information;
[0054] The patient diagnosis prediction module is responsible for inputting the patient fusion information representation into the diagnosis auxiliary prediction model to generate the diagnosis prediction result of the patient's condition;
[0055] The prediction training module is responsible for pre-training the diagnosis-assisted prediction model by inputting the patient's fusion information representation and verification results, and is responsible for regularly re-training new data to optimize the diagnosis-assisted prediction model;
[0056] The prediction result output and feedback module outputs prediction result information based on the pre-trained diagnosis-assisted prediction model, obtains feedback on the accuracy of the diagnosis prediction result, and returns the accurate diagnosis prediction result to the diagnosis-assisted prediction model for use in the next optimization training.
[0057] Compared with the prior art, the present invention has the following beneficial effects:
[0058] The present invention uses the representation method of knowledge graph to effectively integrate and mine the relationship between the patient's examination information, medical record information and medical health knowledge, and obtains the integrated representation of the patient's current condition information. While considering diversity, it better explores the potential connections within different health data;
[0059] By effectively leveraging external medical and health knowledge and multi-view patient status and history information, more comprehensive associations can be mined through knowledge graphs to improve patient diagnosis and prediction results, providing reference and verification for doctors' further diagnosis. BRIEF DESCRIPTION OF THE DRAWINGS
[0060] In order to more clearly illustrate the technical solutions in the embodiments or exemplary embodiments of the present invention, the drawings required for use in the embodiments or exemplary descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained according to the drawings without paying creative work.
[0061] Figure 1 A flowchart of an auxiliary diagnosis method based on a multi-view knowledge graph is shown;
[0062] Figure 2 A flow chart for obtaining a patient fusion information representation is shown;
[0063] Figure 3 A process architecture diagram of an auxiliary diagnosis system based on a multi-view knowledge graph is shown. DETAILED DESCRIPTION
[0064] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.
[0065] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.
[0066] Figure 1 A flowchart of an auxiliary diagnosis method based on a multi-view knowledge graph is shown.
[0067] like Figure 1 As shown, this embodiment provides an auxiliary diagnosis method based on a multi-view knowledge graph, including:
[0068] S102, obtaining a preliminary conclusion of the patient's examination data from a patient information database, performing prompt query on the preliminary conclusion of the examination data using a large language model, and obtaining knowledge expansion data based on the preliminary conclusion of the examination data;
[0069] S104, obtaining the patient's structured examination data, imaging data, medical records and historical medication data, and knowledge extension data, and training and constructing node representations of the patient information according to the above four views;
[0070] S106, constructing the node representations of the four views into a knowledge graph, and obtaining the patient fusion information representation through knowledge graph pre-training;
[0071] S108, constructing a diagnosis-assisted prediction model, inputting the patient fusion information representation into the diagnosis-assisted prediction model, and obtaining a diagnosis prediction result for the patient.
[0072] It should be noted that, the preliminary conclusion of the patient's examination data is obtained based on the patient's identity information, and the preliminary conclusion of the examination data retrieved from the patient information database is used to generate a query instruction according to the format requirements of the pre-trained large language model (such as GPT, etc.) interface prompt template, and the query instruction is used to call no less than one pre-trained large language model interface to ask knowledge expansion questions; knowledge expansion question and answer pairs are generated for the preliminary conclusion of the examination data, and n knowledge vectors with the highest similarity to the query quality are obtained as return results based on the knowledge expansion question and answer pairs; multiple returned results are deduplicated and spliced to form knowledge expansion data of the preliminary conclusion of the patient's examination data.
[0073] By utilizing the language understanding and generation capabilities of the large language model, knowledge extension data based on the preliminary conclusions of patient examination data is generated before building the knowledge graph, which reduces the workload required to build the knowledge graph and greatly improves efficiency compared to traditional retrieval enhancement generation methods.
[0074] It should be noted that based on the patient information database, the patient's structured examination data, imaging data, medical records, historical medication data and other text data are retrieved to construct structured examination data views, imaging data views, medical records and historical medication data views and knowledge expansion data views respectively; given a multi-view dataset , , Shidi Information about the view, Indicates View View information for each patient, , import the multi-view dataset into a multi-layer perceptron, and use the encoder of the multi-layer perceptron and decoder right Reconstruction is performed to obtain the latent space representation of the patient based on the nonlinear mapping ability of the neural network, which is expressed as:
[0075] ,
[0076] ,
[0077] in Indicates View The latent space representation of each patient, Represents the corresponding reconstructed attribute representation; performs contrastive multi-view learning on the latent space features of the patient in the four views, takes the patient nodes between different views as positive samples, and takes other nodes within and between views as negative samples, and obtains the node representation of the patient information in different views through contrastive learning.
[0078] For contrastive multi-view learning, the reconstruction loss and contrastive loss are combined to minimize the objective function to learn the parameters. The loss function Defined as:
[0079] ,
[0080] in and Respectively represent The reconstruction loss and contrast loss of each view, represents the trade-off parameter, Indicates the number of views, represents the number of patients, and They are defined as:
[0081] ,
[0082] ,
[0083] in, Indicates View The latent space representation of each patient is View The similarity between the latent space representations of each patient, Respectively represent View The patient and The latent space representation of each patient, Represents a hyperparameter.
[0084] Figure 2 A flow chart for obtaining a patient fusion information representation is shown.
[0085] According to an embodiment of the present invention, the patient fusion information representation is obtained, specifically:
[0086] S202, converting the node representations of the patient information in the four views into triples to construct a knowledge graph, where the patient is the head node, the relationship is the patient's attributes, and the tail node is the node representation of different views;
[0087] S204, initializing the 𝑘 nearest neighbor graph of the patient node according to the patient's structured examination data, imaging data, medical records and historical medication data, and knowledge extension data, and using the 𝑘 nearest neighbor graph to obtain the connection relationship of the patient node in the knowledge graph;
[0088] S206, constructing a structure-related convolutional network framework to train the knowledge graph, including a weighted graph convolutional network encoding module and a convolutional TransE decoding module, to obtain a patient fusion information representation based on subgraphs corresponding to different views.
[0089] It should be noted that for the weighted graph convolutional network encoding module, by aggregating the adjacent node relationships of the patient nodes, the patient fusion information representation based on the corresponding subgraphs of different views is obtained:
[0090] ,
[0091] in , Indicates Layer, The patient fusion information representation of the layer, From multiple views Splicing obtained, represents the weighted sum of the subgraph adjacency matrix plus the self-connections, , represents an identity matrix, Indicates the total number of subgraph types, Indicates the subgraph type, Indicates The relationship weight of the layer; represents the weight parameter, represents the activation function;
[0092] For the convolutional TransE decoding module, multiple convolution kernels are used to extract features from the stacked two-dimensional vectors, and convolution is performed based on C convolution kernels of different sizes. The three feature maps obtained are concatenated and sent to the fully connected layer to be projected to the embedded dimension.
[0093] Scoring function of the convolutional TransE decoding module after nonlinear convolution Defined as:
[0094] ,
[0095] in represents a nonlinear function, Represents the feature map extracted by the convolution kernel, represents the head node and tail node embedding, represents relation embedding, represents the weight coefficient, Indicates the concatenation of multiple feature maps. During the training process, the logistic sigmoid function is applied to the score to obtain the final score function , expressed as:
[0096] ,
[0097] in Represents the logistic sigmoid function.
[0098] It should be noted that a diagnosis-assisted prediction model is constructed, and the diagnosis-assisted prediction model is trained through training data. The patient fusion representation samples in the training data are input into the diagnosis-assisted prediction model for training and verification. The training data and the verification data set are divided according to 8:2, and the diagnosis-assisted prediction model that meets the preset standards after training is exported; the patient fusion information representation based on the corresponding sub-graphs of different views is obtained. As the model input, the parameters are mapped in the diagnosis-assisted prediction model and then the predicted probability of illness of different patients is obtained through the activation function, which is expressed as:
[0099] ,
[0100] in Indicates the probability of different patients suffering from the preset disease category, It represents parameter characterization;
[0101] The patient's diagnostic prediction results are sent and visualized according to the preset method, and the doctor's feedback on the accuracy of the diagnostic prediction results is obtained. Based on the accuracy feedback information, the diagnostic prediction results that meet the preset accuracy requirements are screened and returned to the diagnostic auxiliary prediction model for the next optimization training.
[0102] Figure 3 A process architecture diagram of an auxiliary diagnosis system based on a multi-view knowledge graph is shown.
[0103] This embodiment provides an auxiliary diagnosis system based on a multi-view knowledge graph, including: a patient information database, a patient examination result expansion module, a prompt information generation module, a patient structured examination data representation module, a patient imaging data representation module, a patient medical record and historical medication data representation module, a knowledge graph construction module, a patient fusion information representation module, a patient diagnosis prediction module, a prediction training module, and a prediction result output and feedback module;
[0104] The patient information database is responsible for storing the patient's historical examination data, medical record text data, and medication information data, and storing them according to different data types for specific view data processing and query;
[0105] The patient examination result expansion module: obtains a query instruction according to the preliminary conclusion of the patient's examination data, calls at least one pre-trained large language model interface to ask questions for knowledge expansion, and when there are multiple returned results, is responsible for de-duplication and splicing the obtained question result data to form the patient examination result expansion data;
[0106] The prompt information generation module is responsible for managing the information template that calls the pre-trained large language model interface, and generates a query instruction according to the format requirements of the prompt template for the preliminary conclusion data of the examination provided by the patient examination result extension module, and returns the query instruction to the patient examination result extension module;
[0107] The patient structured examination data representation module is responsible for formatting and preprocessing the patient structured examination data, generating fusion representation information of the structured data, and returning the representation information;
[0108] The patient image data representation module is responsible for denoising and size standardizing the image data of the patient examination, calling the pre-trained graph representation model to generate the representation of the image data, and returning the representation information;
[0109] The patient medical record and historical medication data representation module is responsible for generating text representations of the patient medical record and historical medication data through a text representation pre-training model, and returning the representation information;
[0110] The patient examination result extended data representation module is responsible for preprocessing the patient's examination result extended data, generating representation information of the examination result extended data, and returning the representation information;
[0111] The knowledge graph construction module is responsible for pre-processing the representation of the patient's structured examination data, imaging data, medical records and historical medication data, and knowledge extension data into a triple format, and adding it to the knowledge graph for extended training;
[0112] The patient fusion information representation module is responsible for generating information representation from the patient fusion information in the knowledge graph and returning the representation information;
[0113] The patient diagnosis prediction module is responsible for inputting the patient fusion information representation into the diagnosis auxiliary prediction model to generate the diagnosis prediction result of the patient's condition;
[0114] The prediction training module is responsible for pre-training the diagnosis-assisted prediction model by inputting the patient's fusion information representation and verification results, and is responsible for regularly re-training new data to optimize the diagnosis-assisted prediction model;
[0115] The prediction result output and feedback module outputs prediction result information based on the pre-trained diagnosis-assisted prediction model, obtains feedback on the accuracy of the diagnosis prediction result, and returns the accurate diagnosis prediction result to the diagnosis-assisted prediction model for use in the next optimization training.
[0116] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.
[0117] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.
[0118] In addition, all functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.
[0119] A person of ordinary skill in the art can understand that: all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: a mobile storage device, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.
[0120] Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention can be essentially or partly reflected in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.
[0121] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.
Claims
1. An auxiliary diagnosis method based on multi-view knowledge graph, characterized in that: The following steps are involved: Obtaining preliminary conclusions of the patient's examination data from a patient information database, performing prompt query on the preliminary conclusions of the examination data using a large language model, and obtaining knowledge expansion data based on the preliminary conclusions of the examination data; Obtain the patient's structured examination data, imaging data, medical records, historical medication data, and knowledge expansion data, and train and construct node representations of patient information according to four views; The node representations of the four views are constructed into a knowledge graph, and after pre-training on the knowledge graph, the patient fusion information representation is obtained; Constructing a diagnosis-assisted prediction model, inputting the patient fusion information representation into the diagnosis-assisted prediction model, and obtaining a diagnosis prediction result for the patient; The preliminary conclusion of the inspection data is used to perform prompt query using a large language model to obtain knowledge expansion data based on the preliminary conclusion of the inspection data, specifically: Generating a query instruction according to the format requirements of the pre-trained large language model interface prompt template using the preliminary examination conclusion data retrieved from the patient information database, and using the query instruction to call at least one pre-trained large language model interface to ask knowledge expansion questions; Generate knowledge expansion question and answer pairs for the preliminary conclusion data of the examination, and obtain n knowledge vectors with the highest similarity to the query quality as return results based on the knowledge expansion question and answer pairs; remove duplicates and splice multiple return results to form knowledge expansion data of the preliminary conclusion of the patient examination data; Obtain the patient's structured examination data, imaging data, medical records, historical medication data, and knowledge extension data, and train and construct node representations of patient information according to the above four views, specifically: Retrieve the patient's structured examination data, imaging data, medical records and historical medication data based on the patient information database, build structured examination data views, imaging data views, medical records and historical medication data views respectively, and build a knowledge extension data view based on the knowledge extension data; Given a multi-view dataset , , Shidi Information about the view, Indicates View View information for each patient, , import the multi-view dataset into a multi-layer perceptron, and use the encoder of the multi-layer perceptron and decoder right Reconstruction is performed to obtain the latent space representation of the patient based on the nonlinear mapping ability of the neural network, which is expressed as: in Indicates View The latent space representation of each patient, Represents the corresponding reconstructed attribute representation; performs contrastive multi-view learning on the latent space representation of the patient in the four views, takes the patient nodes between different views as positive samples, and takes other nodes within and between views as negative samples, and obtains the node representation of the patient information in different views through contrastive learning; For contrastive multi-view learning, the loss function Defined as: in and Respectively represent The reconstruction loss and contrast loss of each view, represents the trade-off parameter, Indicates the number of views, represents the number of patients, and They are defined as: in, Indicates View The latent space representation of each patient is View The similarity between the latent space representations of each patient, Respectively represent View The patient and The latent space representation of each patient, represents the hyperparameter; the node representations of the four views are constructed into a knowledge graph. After pre-training the knowledge graph, the patient fusion information representation is obtained, which is specifically: The node representations of the patient information in the four views are converted into triples to construct a knowledge graph, where the patient is the head node, the relationship is the patient's attribute, and the tail node is the node representation of different views; Initialize the 𝑘 nearest neighbor graph of the patient node according to the patient's structured examination data, imaging data, medical records and historical medication data, and knowledge extension data, and use the 𝑘 nearest neighbor graph to obtain the connection relationship of the patient nodes in the knowledge graph; Construct a structure-related convolutional network framework to train the knowledge graph, including a weighted graph convolutional network encoding module and a convolutional TransE decoding module, to obtain a patient fusion information representation based on subgraphs corresponding to different views; For the weighted graph convolutional network encoding module, by aggregating the adjacent node relationships of the patient nodes, the patient fusion information representation based on the corresponding subgraphs of different views is obtained: in Indicates The patient fusion information representation of the layer, From multiple views Splicing obtained, represents the weighted sum of the subgraph adjacency matrix plus the self-connections, represents the weight parameter, represents the activation function; For the convolutional TransE decoding module, embedded convolution is performed based on C convolution kernels of different sizes, and the C feature maps obtained are concatenated and sent to the fully connected layer to be projected to the embedded dimension; Scoring function of the convolutional TransE decoding module after nonlinear convolution Defined as: in represents a nonlinear function, Represents the feature map extracted by the convolution kernel, represents the head node and tail node embedding, represents relation embedding, represents the weight coefficient, Indicates the concatenation of multiple feature maps. During the training process, the logistic sigmoid function is applied to the score to obtain the final score function , expressed as: in Represents the logistic sigmoid function.
2. The auxiliary diagnosis method based on multi-view knowledge graph according to claim 1, characterized in that: Constructing a diagnosis-assisted prediction model, training the diagnosis-assisted prediction model using training data, inputting patient fusion information representation in the training data into the diagnosis-assisted prediction model for training and verification, and deriving a diagnosis-assisted prediction model that meets preset standards after training; Obtain patient fusion information representation based on corresponding sub-images of different views As the model input, the parameters are mapped in the diagnosis-assisted prediction model and then the predicted probability of illness of different patients is obtained through the activation function, which is expressed as: in represents the probability of different patients suffering from the preset disease category, and W represents the parameter representation; The patient's diagnostic prediction results are sent and visualized according to the preset method, and the doctor's feedback on the accuracy of the diagnostic prediction results is obtained. Based on the accuracy feedback information, the diagnostic prediction results that meet the preset accuracy requirements are screened and returned to the diagnostic auxiliary prediction model for the next optimization training.
3. An auxiliary diagnosis system based on multi-view knowledge graph, characterized in that: Implementing the auxiliary diagnosis method based on multi-view knowledge graph as described in any one of claims 1-2, comprising: a patient information database, a patient examination result expansion module, a prompt information generation module, a patient structured examination data representation module, a patient imaging data representation module, a patient medical record and historical medication data representation module, a knowledge graph construction module, a patient fusion information representation module, a patient diagnosis prediction module, a prediction training module and a prediction result output and feedback module; The patient information database is responsible for storing the patient's historical examination data, medical record text data, and medication information data, and storing them according to different data types for specific view data processing and query; The patient examination result expansion module: obtains a query instruction according to the preliminary conclusion of the patient's examination data, calls at least one pre-trained large language model interface to ask questions for knowledge expansion, and when there are multiple returned results, is responsible for de-duplication and splicing the obtained question result data to form the patient examination result expansion data; The prompt information generation module is responsible for managing the information template that calls the pre-trained large language model interface, and generates a query instruction according to the format requirements of the prompt template for the preliminary conclusion data of the examination provided by the patient examination result extension module, and returns the query instruction to the patient examination result extension module; The patient structured examination data representation module is responsible for formatting and preprocessing the patient structured examination data, generating fusion representation information of the structured examination data, and returning the fusion representation information of the structured examination data; The patient image data representation module is responsible for denoising and size standardizing the image data of the patient examination, calling the pre-trained graph representation model to generate representation information of the image data, and returning the representation information of the image data; The patient medical record and historical medication data representation module is responsible for generating representation information of the patient medical record and historical medication data through the text representation pre-training model, and returning the representation information of the patient medical record and historical medication data; The patient examination result extended data representation module is responsible for preprocessing the patient's examination result extended data, generating representation information of the examination result knowledge extended data, and returning the representation information of the knowledge extended data; The knowledge graph construction module is responsible for preprocessing the representation information of the patient's structured examination data, imaging data, medical records and historical medication data, and knowledge extension data into a triple format, and adding it to the knowledge graph for extended training; The patient fusion information representation module is responsible for obtaining the patient fusion information representation based on the knowledge graph and returning the patient fusion information representation; The patient diagnosis prediction module is responsible for inputting the patient fusion information representation into the diagnosis auxiliary prediction model to generate the diagnosis prediction result of the patient's condition; The prediction training module is responsible for pre-training the diagnosis-assisted prediction model by integrating the patient's fusion information representation and the historical input of the verification results, and is responsible for regularly re-training the new data to optimize the diagnosis-assisted prediction model; The prediction result output and feedback module outputs prediction result information based on the pre-trained diagnosis-assisted prediction model, obtains feedback on the accuracy of the diagnosis prediction result, and returns the accurate diagnosis prediction result to the diagnosis-assisted prediction model for use in the next optimization training.