Knowledge graph construction method, system and terminal based on clinical business data

Abstract

The present invention provides a method, system and terminal for constructing a knowledge graph based on clinical business data, which respectively constructs a medical knowledge-based knowledge graph and a medical event-based knowledge graph through the collected medical field knowledge data and patient clinical and medical record data, and fuses and integrates the constructed medical knowledge-based knowledge graph and medical event-based knowledge graph to obtain a fused knowledge graph, and then accesses multimodal data related to medical information and fuses it with the fused knowledge graph to obtain the final constructed knowledge graph. The present invention realizes the integration and expression of medical knowledge by making full use of structured, unstructured and semi-structured clinical data and multimodal data, and using advanced data processing and analysis technologies. The knowledge graph constructed by this solution can provide comprehensive, accurate and dynamic medical information, provide strong support for medical decision-making, research and management, and promote the progress and development of the medical industry.

Description

Technical Field

[0001] The present invention relates to the field of medical data processing technology, and in particular to a method, system and terminal for constructing a knowledge graph based on clinical business data. Background Art

[0002] In today's medical field, the rapid growth of clinical business data and its dispersion and heterogeneity have brought many challenges. There are a large number of clinical business data such as patient medical records, doctor's orders, test results, imaging data, etc. in medical institutions. These data exist in various systems and databases in different formats and storage methods. In addition, the scale and complexity of medical data make traditional data processing and analysis methods difficult and time-consuming.

[0003] At present, although there are some knowledge graph construction methods, these methods are mainly focused on general fields and are difficult to be directly applied to the medical field. In addition, existing methods usually rely only on structured data, and cannot fully utilize unstructured or semi-structured clinical text data, as well as multimodal data such as medical images, which limits the comprehensive utilization of clinical business data. Summary of the invention

[0004] In view of the shortcomings of the prior art mentioned above, the purpose of the present invention is to provide a method, system and terminal for constructing a knowledge graph based on clinical business data, so as to solve the technical problem that the prior art only relies on structured data and cannot make full use of unstructured or semi-structured clinical text data, as well as multimodal data such as medical images, thus limiting the comprehensive utilization of clinical business data.

[0005] To achieve the above-mentioned purpose and other related purposes, the present invention provides a method for constructing a knowledge graph based on clinical business data, the method comprising: constructing a medical knowledge-based knowledge graph based on collected medical field knowledge data; constructing a medical event-based knowledge graph based on collected patient clinical and medical record data; fusing and integrating the constructed medical knowledge-based knowledge graph and the medical event-based knowledge graph to obtain a fused knowledge graph; accessing multimodal data related to patient medical information, and fusing it with the fused knowledge graph to obtain the final constructed knowledge graph.

[0006] In one embodiment of the present invention, the construction of a medical knowledge-based knowledge graph based on the collected medical field knowledge data includes: based on the collected medical field knowledge data, constructing a medical knowledge-based knowledge graph with a multi-level structure composed of multiple medical knowledge-based entities according to the medical knowledge graph structure design; wherein the medical knowledge graph structure design is divided into main categories based on diseases, clinical findings, drugs, operations, human morphology and structure, treatment plans, genes, biological inheritance, variations, physical entities, organisms, organizations, people, places and literature.

[0007] In one embodiment of the present invention, the construction of a medical event-based knowledge graph based on collected patient clinical and medical record data includes: extracting event information from the collected patient clinical and medical record data; based on the extracted event information, constructing a medical event-based knowledge graph with a multi-level structure composed of multiple medical event-type entities according to the medical event knowledge graph structure design; wherein the medical event knowledge graph structure design is divided into patient process events, clinical events, and prevention and health care events as the main categories.

[0008] In one embodiment of the present invention, the constructed medical knowledge-based knowledge graph and medical event-based knowledge graph are fused and integrated to obtain a fused knowledge graph, including: performing entity alignment between knowledge-based entities and medical event-based entities related to target entity categories in the medical knowledge-based knowledge graph and the medical event-based knowledge graph to obtain a fused knowledge graph; wherein the target entity categories include: drugs, diagnosis, surgery, testing, examination, and microorganisms.

[0009] In one embodiment of the present invention, the entity alignment between the knowledge entities and medical event-type entities related to the target entity category in the medical knowledge-based knowledge graph and the medical event-type knowledge graph includes: standardizing the entity names of the knowledge entities and medical event-type entities related to the target entity category in the medical knowledge-based knowledge graph and the medical event-type knowledge graph to obtain corresponding standard terminology numbers; comparing the obtained standard terminology numbers of each knowledge entity with the standard terminology numbers of each medical event-type entity; establishing associations between knowledge entities and medical event-type entities with consistent numbers, and setting them to the same entity name.

[0010] In one embodiment of the present invention, the accessing of multimodal data related to the patient's medical information and fusing it with the fused knowledge graph to obtain a finally constructed knowledge graph includes: accessing the multimodal data related to the patient's medical information; extracting features of the multimodal data and establishing associations between features of each modality; extracting features from the fused knowledge graph using a graph neural network; establishing an association between features of the multimodal data after establishing an association and features extracted from the fused knowledge graph by calculating feature similarity to obtain a finally constructed knowledge graph.

[0011] In one embodiment of the present invention, the extraction of features of multimodal data and the establishment of an association relationship between each modality include: extracting features of each modality in the multimodal data in a corresponding manner; normalizing the features extracted from the data of each modality, and calculating the cosine similarity between the features of each modality, so as to establish an association relationship between different modal features that meet the similarity conditions.

[0012] In one embodiment of the present invention, the modal types in the multimodal data include: multiple types of text data, image data, sound data, video data, Internet of Things data, physiological signal data and genomics data.

[0013] To achieve the above-mentioned purpose and other related purposes, the present invention provides a knowledge graph construction system based on clinical business data, the system comprising: a medical knowledge graph construction module, used to construct a medical knowledge-based knowledge graph based on collected medical field knowledge data; a medical event graph construction module, used to construct a medical event-based knowledge graph based on collected patient clinical and medical record data; a knowledge graph fusion module, connecting the medical knowledge graph construction module and the medical event graph construction module, used to fuse and integrate the constructed medical knowledge-based knowledge graph and the medical event-based knowledge graph to obtain a fused knowledge graph; a multimodal data fusion module, connected to the knowledge graph fusion module, used to access multimodal data related to patient medical information, and fuse it with the fused knowledge graph to obtain the final constructed knowledge graph.

[0014] To achieve the above-mentioned objectives and other related objectives, the present invention provides a knowledge graph construction terminal based on clinical business data, comprising: one or more memories and one or more processors; the one or more memories are used to store computer programs; the one or more processors are connected to the memories and are used to run the computer program to execute the knowledge graph construction method based on clinical business data.

[0015] As described above, the present invention is a method, system and terminal for constructing a knowledge graph based on clinical business data, which has the following beneficial effects: the present invention constructs a medical knowledge-based knowledge graph and a medical event-based knowledge graph respectively through the collected medical field knowledge data and patient clinical and medical record data, and fuses and integrates the constructed medical knowledge-based knowledge graph and medical event-based knowledge graph to obtain a fused knowledge graph, and then accesses multimodal data related to medical information, and fuses it with the fused knowledge graph to obtain the final constructed knowledge graph. The present invention realizes the integration and expression of medical knowledge by making full use of structured, unstructured and semi-structured clinical data and multimodal data, and using advanced data processing and analysis technologies. The knowledge graph constructed by this solution can provide comprehensive, accurate and dynamic medical information, provide strong support for medical decision-making, research and management, and promote the progress and development of the medical industry. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 Shown is a flowchart of a method for constructing a knowledge graph based on clinical business data in one embodiment of the present invention.

[0017] Figure 2 Shown is a schematic diagram of the hierarchical structure of the medical knowledge graph structure design in one embodiment of the present invention.

[0018] Figure 3 Shown is a schematic diagram of the hierarchical structure of the medical event knowledge graph structure design in one embodiment of the present invention.

[0019] Figure 4 Shown is a schematic diagram of the knowledge graph entity alignment process in one embodiment of the present invention.

[0020] Figure 5 Shown is a schematic diagram of a multimodal data fusion process in an embodiment of the present invention.

[0021] Figure 6 Shown is a structural diagram of a knowledge graph construction system based on clinical business data in one embodiment of the present invention.

[0022] Figure 7 Shown is a schematic diagram of the structure of a knowledge graph construction terminal based on clinical business data in one embodiment of the present invention. DETAILED DESCRIPTION

[0023] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0024] It should be noted that in the following description, reference is made to the accompanying drawings, which describe several embodiments of the present invention. It should be understood that other embodiments may also be used, and that mechanical composition, structural, electrical and operational changes may be made without departing from the spirit and scope of the present invention. The following detailed description should not be considered restrictive, and the scope of the embodiments of the present invention is limited only by the claims of the published patents. The terms used herein are only for describing specific embodiments and are not intended to limit the present invention. Spatially related terms, such as "upper", "lower", "left", "right", "below", "below", "lower", "above", "upper", etc., may be used in the text to facilitate the description of the relationship between an element or feature shown in the figure and another element or feature.

[0025] Throughout the specification, when a part is said to be "connected" to another part, this includes not only the case of "direct connection" but also the case of "indirect connection" by placing other elements therebetween. In addition, when a part is said to "include" a certain constituent element, unless otherwise stated, it does not exclude other constituent elements, but means that other constituent elements may be included.

[0026] The terms first, second and third mentioned herein are used to describe various parts, components, regions, layers and / or segments, but are not limited thereto. These terms are only used to distinguish a certain part, component, region, layer or segment from other parts, components, regions, layers or segments. Therefore, the first part, component, region, layer or segment described below may refer to the second part, component, region, layer or segment within the scope of the present invention.

[0027] Furthermore, as used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless there is an indication to the contrary in the context. It should be further understood that the terms "comprise", "include" indicate the presence of the described features, operations, elements, components, items, kinds, and / or groups, but do not exclude the presence, occurrence or addition of one or more other features, operations, elements, components, items, kinds, and / or groups. The terms "or" and "and / or" used herein are interpreted as inclusive, or mean any one or any combination. Therefore, "A, B or C" or "A, B and / or C" means "any of the following: A; B; C; A and B; A and C; B and C; A, B and C". Exceptions to this definition will only occur when the combination of elements, functions or operations is inherently mutually exclusive in some way.

[0028] The present invention provides a method for constructing a knowledge graph based on clinical business data, which constructs a medical knowledge-based knowledge graph and a medical event-based knowledge graph respectively through the collected medical field knowledge data and patient clinical and medical record data, and fuses and integrates the constructed medical knowledge-based knowledge graph and medical event-based knowledge graph to obtain a fused knowledge graph, and then accesses multimodal data related to medical information and fuses it with the fused knowledge graph to obtain the final constructed knowledge graph. The present invention realizes the integration and expression of medical knowledge by making full use of structured, unstructured and semi-structured clinical data and multimodal data, and using advanced data processing and analysis technologies. The knowledge graph constructed by this solution can provide comprehensive, accurate and dynamic medical information, provide strong support for medical decision-making, research and management, and promote the progress and development of the medical industry.

[0029] The following is a detailed description of the embodiments of the present invention with reference to the accompanying drawings so that those skilled in the art can easily implement the present invention. The present invention can be embodied in many different forms and is not limited to the embodiments described herein.

[0030] like Figure 1 A flowchart showing a method for constructing a knowledge graph based on clinical business data in an embodiment of the present invention is shown.

[0031] The method comprises:

[0032] Step S1: Construct a medical knowledge graph based on the collected medical field knowledge data.

[0033] In one embodiment, step S1 includes:

[0034] Based on the collected medical knowledge data, a medical knowledge graph with a multi-level structure consisting of multiple medical knowledge entities is constructed by designing the medical knowledge graph structure;

[0035] Specifically, medical domain knowledge data may include knowledge in the medical field, such as diseases, drugs, treatment methods, etc. Such knowledge may be collected and organized from sources such as literature, knowledge bases, and expert opinions.

[0036] The medical knowledge graph structure design adopts schema design, which is a multi-level structure. According to the medical knowledge graph structure, the collected medical field knowledge data is classified to obtain a medical knowledge graph. The graph has a multi-level structure, and each level has a corresponding medical knowledge entity. Among them, the construction of the medical knowledge graph can use traditional knowledge graph construction methods, such as ontology modeling and semantic association.

[0037] The structure design of the medical knowledge graph is based on the main categories of diseases, clinical findings, drugs, operations, human morphology and structure, treatment plans, genes, biological inheritance, mutations, physical entities, organisms, organizations, people, places and literature. As the first-level entity, each is further divided into its subcategories. The hierarchical structure is as follows Figure 2 As shown: the second-level entities under clinical findings are symptoms and signs; the second-level entities under physics are medical devices; the second-level entities under drugs are traditional Chinese medicine, biological drugs and chemical drugs; the second-level entities under human morphology and structure are abnormal morphology and structure; the second-level entities under treatment plans are psychological treatment plans, drug treatment plans and surgical treatment plans; the second-level entities under genes are gene mutations; the second-level entities under operations are observation operations, inspections, tests, operations and clinical non-surgical treatments; the second-level entities under diseases are physical diseases and psychological diseases; the second-level entities under biology are microorganisms, plants and animals; the second-level entities under literature are data, clinical guidelines and clinical pathways; the second-level entities under people are people and population. There are also more entities under the levels, which will not be repeated here.

[0038] Step S2: Construct a medical event-based knowledge graph based on the collected patient clinical and medical record data.

[0039] In detail, medical events refer to specific events related to patients, such as surgery, diagnosis, treatment process, etc. Medical event-based knowledge graphs can be constructed based on patients' clinical data and medical records.

[0040] In one embodiment, step S2 includes:

[0041] Event information is extracted from the collected clinical and medical record data of the patient; specifically, NLP technology can be used to extract event information from the clinical and medical record data of the patient.

[0042] Based on the extracted event information and the event association information obtained from multimodal data analysis, a medical event knowledge graph with a multi-level structure consisting of multiple medical event entities is constructed according to the medical event knowledge graph structure design;

[0043] The medical event knowledge graph structure design adopts schema design, and the structure is a multi-level structure. According to the medical event knowledge graph structure design, the event information is classified to obtain a medical event knowledge graph. The graph has a multi-level structure, and each level has a corresponding medical event entity.

[0044] The structure design of the medical event knowledge graph is divided into the main categories of patient process events, clinical events, prevention and health care events, as the first-level entities, each of which is further subdivided into its subcategories. The hierarchical structure is as follows Figure 3 As shown: the second-level entities under patient process events include: outpatient events, hospitalization events, emergency events, and registration events; the second-level entities under clinical events include: first aid events, diagnosis events, and patient care events; the third-level entities under first aid events are emergency rescue events, critical patient events, and cardiac resuscitation events; the third-level entities under diagnosis events are clinical diagnosis events, disease diagnosis events, and laboratory test events; the third-level entities under patient care events are ward care events, rehabilitation care events, condition observation events, and disease assessment events; the third-level entities under prevention and health care events are prevention screening events, vaccination events, health check events, and health education events. There are also more entities at different levels, which will not be described here.

[0045] Step S3: Fuse and integrate the constructed medical knowledge-based knowledge graph and medical event-based knowledge graph to obtain a fused knowledge graph.

[0046] In one embodiment, when constructing two knowledge graphs, it is necessary to consider how to fuse and integrate different types of knowledge and data. The knowledge graph fusion technology can be used to associate the two knowledge graphs to better support clinical decision-making and medical research. Because both are standard knowledge graphs, the association between entities can be performed by adding relationship types.

[0047] Step S3 includes:

[0048] Performing entity alignment between knowledge entities and medical event entities related to the target entity category in the medical knowledge-based knowledge graph and the medical event-based knowledge graph to obtain a fused knowledge graph;

[0049] Among them, the target entity categories include: drugs, diagnosis, surgery, testing, examination and microorganisms.

[0050] In a specific embodiment, if Figure 4 The entity alignment between the knowledge entities and medical event entities related to the target entity category in the medical knowledge graph and the medical event knowledge graph includes:

[0051] Standardizing the entity names of knowledge entities and medical event entities related to the target entity category in the medical knowledge graph and the medical event knowledge graph to obtain corresponding standard terminology numbers;

[0052] Compare the obtained standard terminology numbers of each knowledge-based entity with the standard terminology numbers of each medical event-based entity;

[0053] Associate knowledge entities and medical event entities with the same number and set them to the same entity name.

[0054] The present invention can associate the same medical terms in two atlases respectively through the normalization and standardization operation of medical terms, thereby realizing the overall association of the atlases.

[0055] Step S4: Access multimodal data related to the patient's medical information and fuse it with the fused knowledge graph to obtain the final constructed knowledge graph.

[0056] In one embodiment, step S4 includes:

[0057] Access multimodal data related to patient medical information;

[0058] Extract the features of multimodal data and establish the correlation between the features of each modality;

[0059] A graph neural network is used to extract features from the fused knowledge graph; specifically, each entity of the fused knowledge graph is converted into a unified vectorized representation through the graph neural network.

[0060] An association relationship is established between the features of the multimodal data after the association relationship is established and the features extracted from the fused knowledge graph by calculating the feature similarity to obtain the final constructed knowledge graph.

[0061] Specifically, the cosine similarity calculated between the features of the multimodal data after the association relationship is established and the features extracted from the fused knowledge graph is used to establish an association relationship between the features that meet the similarity conditions, and after the association relationship is established, the final constructed knowledge graph is generated.

[0062] In a specific embodiment, the modality types in the multimodal data include: multiple types of text data, image data, sound data, video data, Internet of Things data, physiological signal data and genomics data.

[0063] Specifically, text data refers to medical texts, medical records, and test reports; graphic data refers to image files generated by CT and X-ray examinations; sound data refers to voice data sets of medical symptoms, such as patient breathing sounds, and other voice data collected by specialists and diseases. Video data refers to video data such as surgical videos and teaching videos. Physiological signal data refers to biological signal data such as electromyogram, electrocardiogram, and electroencephalogram of instruments and equipment. Genomic data refers to data related to the structure and function of the genome of an organism measured by instruments. IoT data refers to motion data monitored by IoT technologies such as smart bracelets.

[0064] In one embodiment, extracting features of multimodal data and establishing associations between modalities includes:

[0065] For each modality of the multimodal data, feature extraction is performed in a corresponding manner;

[0066] Specifically, for image data, models such as ResNet and VIT can be used as backbones to extract features; these models are pre-trained and can capture key information in images and convert them into vector form. For text, the BERT model is generally used to extract entity features. BERT encodes text through the Transformer structure and generates fixed-dimensional vector representations. These vectors can capture the semantic information in the text. For data in other modalities, transformers are used to extract entity features. The self-attention mechanism of the Transformer model enables it to process data of different sequence lengths and extract useful features.

[0067] The features extracted from the data of each modality are normalized, and the cosine similarity between the features of each modality is calculated to establish an association relationship between the features of different modalities that meet the similarity conditions.

[0068] Specifically, the features of each modality will be normalized after extraction to ensure that the features of different modalities are compared on the same scale. The normalized features can be directly multiplied to calculate the cosine similarity of the features of each modality, which is a method to measure the cosine value of the angle between two vectors. It can be used to evaluate the correlation or matching degree between any two modalities to establish an association relationship between different modal features that meet the similarity conditions.

[0069] like Figure 5, the present invention can use the method of constructing a multimodal data fusion model to perform efficient feature association. Since multiple patients are involved, each patient has multimodal data, and the amount of data is large. In order to improve processing efficiency, we select the features of the multimodal data of a small number of patients for normalization, and directly multiply and calculate the cosine similarity of each modal feature, and establish an association relationship between different modal features that meet the similarity conditions. A multimodal data fusion model is constructed using the features of the multimodal data of a small number of patients, normalized features, calculated cosine similarities of each modal feature, and the corresponding constructed association relationship; then the features of the multimodal data of the remaining patients are input into the multimodal data fusion model, so that the association relationship can be directly predicted and constructed.

[0070] Preferably, a contrastive loss function is used to optimize the multimodal data fusion model. The basic idea of ​​the contrastive loss function is to make the similarity between positive samples (i.e., modal combinations that meet the similarity conditions) as large as possible, and to make the similarity between negative samples (i.e., modal combinations that do not meet the similarity conditions) as small as possible.

[0071] Similar to the principles of the above-mentioned embodiments, the present invention provides a knowledge graph construction system based on clinical business data.

[0072] The following provides specific embodiments in conjunction with the accompanying drawings:

[0073] like Figure 6 A structural schematic diagram of a knowledge graph construction system based on clinical business data in an embodiment of the present invention is shown.

[0074] The system comprises:

[0075] Medical knowledge graph construction module 1, used to construct a medical knowledge graph based on the collected medical field knowledge data;

[0076] Medical event graph construction module 2, used to construct a medical event knowledge graph based on the collected patient clinical and medical record data;

[0077] The knowledge graph fusion module 3 is connected to the medical knowledge graph construction module 1 and the medical event graph construction module 2, and is used to fuse and integrate the constructed medical knowledge graph and the medical event knowledge graph to obtain a fused knowledge graph;

[0078] The multimodal data fusion module 4 is connected to the knowledge graph fusion module 3, and is used to access the multimodal data related to the patient's medical information and fuse it with the fused knowledge graph to obtain the final constructed knowledge graph.

[0079] It should be understood that Figure 6The division of each module in the system embodiment is only a division of logical functions. In actual implementation, they can be fully or partially integrated into one physical entity, or physically separated. And these units can all be implemented in the form of software calling through processing elements; they can also be all implemented in the form of hardware; some units can also be implemented in the form of processing elements calling software, and some units can be implemented in the form of hardware.

[0080] Since the implementation principle of the knowledge graph construction system based on clinical business data has been described in the above embodiments, it will not be repeated here.

[0081] In one embodiment, the construction of a medical knowledge-based knowledge graph based on the collected medical field knowledge data includes: based on the collected medical field knowledge data, constructing a medical knowledge-based knowledge graph with a multi-level structure composed of multiple medical knowledge-based entities according to the medical knowledge graph structure design; wherein the medical knowledge graph structure design is divided into main categories based on diseases, clinical findings, drugs, operations, human morphology and structure, treatment plans, genes, biological inheritance, variations, physical entities, organisms, organizations, people, places and literature.

[0082] In one embodiment, the construction of a medical event-based knowledge graph based on collected patient clinical and medical record data includes: extracting event information from the collected patient clinical and medical record data; based on the extracted event information, constructing a medical event-based knowledge graph with a multi-level structure composed of multiple medical event-type entities according to the medical event knowledge graph structure design; wherein the medical event knowledge graph structure design is divided into patient process events, clinical events, and prevention and health care events as the main categories.

[0083] In one embodiment, the constructed medical knowledge-based knowledge graph and medical event-based knowledge graph are fused and integrated to obtain a fused knowledge graph, including: performing entity alignment between knowledge-based entities and medical event-based entities related to target entity categories in the medical knowledge-based knowledge graph and the medical event-based knowledge graph to obtain a fused knowledge graph; wherein the target entity categories include: drugs, diagnosis, surgery, testing, examination, and microorganisms.

[0084] In one embodiment, the entity alignment between the knowledge entities and medical event entities related to the target entity category in the medical knowledge-based knowledge graph and the medical event-based knowledge graph includes: standardizing the entity names of the knowledge entities and medical event-type entities related to the target entity category in the medical knowledge-based knowledge graph and the medical event-type knowledge graph to obtain corresponding standard terminology numbers; comparing the obtained standard terminology numbers of each knowledge entity with the standard terminology numbers of each medical event-type entity; associating the knowledge entities and medical event-type entities with the same numbers and setting them to the same entity name.

[0085] In one embodiment, the accessing of multimodal data related to the patient's medical information and fusing it with the fused knowledge graph to obtain a finally constructed knowledge graph includes: accessing the multimodal data related to the patient's medical information; extracting features of the multimodal data and establishing associations between features of each modality; extracting features from the fused knowledge graph using a graph neural network; establishing an association between features of the multimodal data after establishing an association and features extracted from the fused knowledge graph by calculating feature similarity to obtain a finally constructed knowledge graph.

[0086] In one embodiment, the extracting of features of multimodal data and establishing an association relationship between each modality includes: extracting features of each modality in the multimodal data in a corresponding manner; normalizing the features extracted from the data of each modality, and calculating the cosine similarity between the features of each modality, so as to establish an association relationship between different modal features that meet the similarity conditions.

[0087] In one embodiment, the modality types in the multimodal data include: multiple types of text data, image data, sound data, video data, Internet of Things data, physiological signal data and genomics data.

[0088] The method for constructing a knowledge graph based on clinical business data provided in the embodiment of the present invention can be implemented on the terminal side or the server side. As for the hardware structure of the terminal for constructing a knowledge graph based on clinical business data, please refer to Figure 7, which is an optional hardware structure diagram of a terminal 1000 for constructing a knowledge graph based on clinical business data provided in an embodiment of the present invention. The terminal 1000 may be a mobile phone, a computer device, a tablet device, a personal digital processing device, a factory background processing device, etc. The terminal 1000 includes: at least one processor 1001, a memory 1002, at least one network interface 10010 and a user interface 1009. The various components in the device are coupled together through a bus system 1005. It can be understood that the bus system 1005 is used to realize connection and communication between these components. In addition to the data bus, the bus system 1005 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, in Figure 7 In the specification, various buses are labeled as bus systems.

[0089] The user interface 1009 may include a display, a keyboard, a mouse, a trackball, a click gun, keys, buttons, a touch pad or a touch screen.

[0090] It is understood that the memory 1002 can be a volatile memory or a non-volatile memory, and can also include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), which is used as an external cache. By way of exemplary but not limiting explanation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM). The memory described in the embodiments of the present invention is intended to include but is not limited to these and any other suitable categories of memory.

[0091] The memory 1002 in the embodiment of the present invention is used to store various categories of data to support the operation of the terminal 1000. Examples of these data include: any executable program for operating on the terminal 1000, such as an operating system 10021 and an application 10022; the operating system 10021 includes various system programs, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks. The application 10022 may include various applications, such as a media player (MediaPlayer), a browser (Browser), etc., for implementing various application services. The knowledge graph construction method based on clinical business data provided in the embodiment of the present invention may be included in the application 10022.

[0092] The method disclosed in the above embodiment of the present invention can be applied to the processor 1001, or implemented by the processor 1001. The processor 1001 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit in the processor 1001 or the instruction in the form of software. The above processor 1001 may be a general processor, a digital signal processor (DSP, Digital Signal Processor), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The processor 1001 can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiment of the present invention. The general processor 1001 can be a microprocessor or any conventional processor, etc. In combination with the steps of the accessory optimization method provided in the embodiment of the present invention, it can be directly embodied as a hardware decoding processor to execute, or it can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium, which is located in a memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0093] In an exemplary embodiment, the terminal 1000 may be implemented by one or more application specific integrated circuits (ASIC), DSP, programmable logic device (PLD), complex programmable logic device (CPLD) to execute the aforementioned method.

[0094] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to the computer program. The aforementioned computer program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk and other media that can store program codes.

[0095] In the embodiments provided in the present application, the computer readable and writable storage medium may include a read-only memory, a random access memory, an EEPROM, a CD-ROM or other optical disk storage device, a disk storage device or other magnetic storage device, a flash memory, a USB flash drive, a mobile hard disk, or any other medium that can be used to store a desired program code in the form of an instruction or data structure and can be accessed by a computer. In addition, any connection can be appropriately referred to as a computer-readable medium. For example, if the instruction is sent from a website, a server or other remote source using a coaxial cable, an optical fiber cable, a twisted pair, a digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwaves, the coaxial cable, optical fiber cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwaves are included in the definition of the medium. However, it should be understood that computer readable and writable storage media and data storage media do not include connections, carriers, signals, or other temporary media, but are intended to be non-temporary, tangible storage media. Disk and disc, as used in this application, includes compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk and Blu-ray disc where disks usually reproduce data magnetically, while discs reproduce data optically with lasers.

[0096] The present invention has the following advantages:

[0097] 1. The present invention uses clinical business data to construct a multi-level, multi-dimensional knowledge graph, covering key information such as diagnosis, treatment, medication, medical records, etc. in clinical practice. In this way, medical professionals can have a more comprehensive understanding of the patient's condition and treatment history, assisting them in making more accurate and personalized treatment decisions and improving medical results.

[0098] 2. This invention can help medical researchers discover association patterns and potential disease patterns hidden in massive data. By analyzing the node and edge relationships in the knowledge graph, researchers can discover important information such as interactions between diseases, risk factors, and treatment effects. This not only helps to deepen the understanding of disease mechanisms, but also provides valuable clues for the development of new drugs and the exploration of treatment methods.

[0099] 3. The present invention helps improve the efficiency and quality of the medical system. By integrating scattered clinical data into a unified knowledge graph, medical institutions can better manage and utilize these data, optimize resource allocation, and improve workflow efficiency. This also helps reduce data redundancy and inconsistency caused by information islands and improve the accuracy and reliability of medical data.

[0100] In summary, the knowledge graph construction method, system and terminal based on clinical business data of the present invention respectively construct medical knowledge-based knowledge graphs and medical event-based knowledge graphs through the collected medical field knowledge data and patient clinical and medical record data, and fuse and integrate the constructed medical knowledge-based knowledge graphs and medical event-based knowledge graphs to obtain fused knowledge graphs, and then access multimodal data related to medical information, and fuse it with the fused knowledge graph to obtain the final constructed knowledge graph. The present invention realizes the integration and expression of medical knowledge by making full use of structured, unstructured and semi-structured clinical data and multimodal data, and using advanced data processing and analysis technologies. The knowledge graph constructed by this solution can provide comprehensive, accurate and dynamic medical information, provide strong support for medical decision-making, research and management, and promote the progress and development of the medical industry. Therefore, the present invention effectively overcomes the various shortcomings in the prior art and has a high industrial utilization value.

[0101] The above embodiments are merely illustrative of the principles and effects of the present invention, and are not intended to limit the present invention. Anyone familiar with the technology may modify or change the above embodiments without violating the spirit and scope of the present invention. Therefore, all equivalent modifications or changes made by a person of ordinary skill in the art without departing from the spirit and technical ideas disclosed by the present invention shall still be covered by the claims of the present invention.

Claims

1. A method for constructing a knowledge graph based on clinical business data, characterized in that: The method comprises: Based on the collected medical knowledge data, build a medical knowledge graph; Construct a medical event-based knowledge graph based on the collected patient clinical and medical record data; The constructed medical knowledge-based knowledge graph and medical event-based knowledge graph are merged and integrated to obtain a fused knowledge graph, which includes: entity alignment between knowledge-based entities and medical event-based entities related to the target entity category in the medical knowledge-based knowledge graph and the medical event-based knowledge graph to obtain a fused knowledge graph; wherein the target entity category includes: drugs, diagnosis, surgery, testing, examination and microorganisms; the entity alignment between knowledge-based entities and medical event-based entities related to the target entity category in the medical knowledge-based knowledge graph and the medical event-based knowledge graph includes: standardizing the entity names of knowledge-based entities and medical event-based entities related to the target entity category in the medical knowledge-based knowledge graph and the medical event-based knowledge graph to obtain corresponding standard term numbers; comparing the obtained standard term numbers of each knowledge-based entity with the standard term numbers of each medical event-based entity; establishing associations between knowledge-based entities and medical event-based entities with consistent numbers and setting them to the same entity name; Access multimodal data related to the patient's medical information and fuse it with the fused knowledge graph to obtain a finally constructed knowledge graph; the modal types in the multimodal data include: text data, image data, sound data, video data, Internet of Things data, physiological signal data and multiple genomics data.

2. The method for constructing a knowledge graph based on clinical business data according to claim 1, characterized in that: The construction of a medical knowledge graph based on the collected medical field knowledge data includes: Based on the collected medical knowledge data, a medical knowledge graph with a multi-level structure consisting of multiple medical knowledge entities is constructed according to the medical knowledge graph structure design; Among them, the medical knowledge graph structure design is divided into main categories based on diseases, clinical observations, drugs, operations, human morphology and structure, treatment plans, genes, biological inheritance, mutations, physical entities, organisms, organizations, people, places and literature.

3. The method for constructing a knowledge graph based on clinical business data according to claim 2, characterized in that: The construction of a medical event-based knowledge graph based on the collected patient clinical and medical record data includes: Extract event information from collected patient clinical and medical record data; Based on the extracted event information, a medical event knowledge graph with a multi-level structure consisting of multiple medical event entities is constructed according to the medical event knowledge graph structure design; Among them, the medical event knowledge graph structure design is divided into patient process events, clinical events, prevention and health care events as the main categories.

4. The method for constructing a knowledge graph based on clinical business data according to claim 1, characterized in that: The accessing of multimodal data related to the patient's medical information and fusing it with the fused knowledge graph to obtain a finally constructed knowledge graph includes: Access multimodal data related to patient medical information; Extract the features of multimodal data and establish the correlation between the features of each modality; Using a graph neural network to extract features from the fused knowledge graph; An association relationship is established between the features of the multimodal data after the association relationship is established and the features extracted from the fused knowledge graph by calculating the feature similarity to obtain the final constructed knowledge graph.

5. The method for constructing a knowledge graph based on clinical business data according to claim 4, characterized in that: The extracting of features of multimodal data and establishing associations between the modalities includes: For each modality of the multimodal data, feature extraction is performed in a corresponding manner; The features extracted from the data of each modality are normalized, and the cosine similarity between the features of each modality is calculated to establish an association relationship between the features of different modalities that meet the similarity conditions.

6. A knowledge graph construction system based on clinical business data, characterized in that: The system comprises: Medical knowledge graph construction module, used to construct medical knowledge graph based on collected medical knowledge data; Medical event graph construction module, which is used to construct a medical event knowledge graph based on the collected patient clinical and medical record data; A knowledge graph fusion module is connected to the medical knowledge graph construction module and the medical event graph construction module, and is used to fuse and integrate the constructed medical knowledge graph and medical event knowledge graph to obtain a fused knowledge graph, which includes: entity alignment between knowledge entities and medical event entities related to the target entity category in the medical knowledge graph and the medical event knowledge graph to obtain a fused knowledge graph; wherein the target entity category includes: drugs, diagnosis, surgery, inspection, examination and microorganisms; the entity alignment between knowledge entities and medical event entities related to the target entity category in the medical knowledge graph and the medical event knowledge graph includes: standardizing the entity names of knowledge entities and medical event entities related to the target entity category in the medical knowledge graph and the medical event knowledge graph to obtain corresponding standard term numbers; comparing the obtained standard term numbers of each knowledge entity with the standard term numbers of each medical event entity; establishing associations between knowledge entities and medical event entities with consistent numbers, and setting them to the same entity name; A multimodal data fusion module is connected to the knowledge graph fusion module, and is used to access the multimodal data related to the patient's medical information and fuse it with the fused knowledge graph to obtain a finally constructed knowledge graph; the modal types in the multimodal data include: text data, image data, sound data, video data, Internet of Things data, physiological signal data and multiple genomics data.

7. A knowledge graph construction terminal based on clinical business data, characterized in that: include: one or more memories and one or more processors; The one or more memories are used to store computer programs; The one or more processors, connected to the memory, are configured to run the computer program to perform the method according to any one of claims 1 to 5.

Images