Ultra-long-range medical record diagnosis and treatment question-answering system and method based on dynamic time sequence semantic map

Abstract

The invention relates to an ultra-long-range medical record diagnosis and treatment question answering system and method based on a dynamic time sequence semantic map, belongs to the technical field of medical artificial intelligence and information processing, and solves the problems that in the prior art, the inference ability is insufficient when ultra-long-range medical records are processed. The system comprises a pre-processing module for pre-processing overlong-range medical record information of a patient to generate standardized data; the medical entity recognition module is used for recognizing event entities and descriptive entities in the standardized data; associating a time attribute for the event type entity to generate a time sequence entity tuple; outputting a medical entity recognition result; the dynamic semantic map construction module is used for constructing a specific dynamic semantic map of the corresponding patient; the text-map dual-mode fusion query engine is used for analyzing the medical problem proposed by the user and carrying out text-map dual-mode fusion query based on the specific dynamic semantic map; and the diagnosis and treatment reply module is used for generating a diagnosis and treatment reply result according to the text-map dual-mode fusion query result.

Description

Technical Field

[0001] This invention relates to the fields of medical artificial intelligence and information processing technology, and in particular to an ultra-long-range medical record diagnosis and treatment question-and-answer system and method based on dynamic temporal semantic graph. Background Technology

[0002] With the development of modern medicine and the deepening of information technology infrastructure, the accumulation of clinical data has exploded. Especially for patients with chronic diseases such as cancer, cardiovascular disease, and diabetes, their complete electronic medical records (EMRs) often span several years or even decades, containing hundreds or even thousands of independent medical documents, such as admission records, progress notes, examination and test reports, imaging reports, surgical records, nursing records, discharge summaries, etc., easily exceeding hundreds of thousands of words in total. This medical data, spanning the patient's entire life cycle, is the cornerstone of precision medicine, clinical research, and personalized health management, containing immense value.

[0003] However, efficiently and accurately extracting and understanding information from these vast amounts of diverse, highly unstructured texts remains a core bottleneck in current medical informatics. Traditional keyword retrieval or simple text mining techniques are far from meeting clinical needs.

[0004] In recent years, large-scale language model (LLM) applications, represented by Retrieval-Augmented Generation (RAG), have been introduced into the medical field. The core idea is to segment long texts into semantically relevant chunks, use vector retrieval techniques to identify the most relevant text chunks to the user's question, and then provide these chunks as context to the LLM to generate the answer. While RAG has improved the performance of question-answering systems to some extent, its inherent technical limitations are clearly exposed when dealing with extreme scenarios involving extremely long medical records. (1) The fundamental problem of long-distance dependency breakdown: The core value of ultra-long-term medical records lies in their internal, time-spanning causal, logical, and evolutionary relationships. For example, a patient used oxaliplatin chemotherapy after gastric cancer surgery in 2015, and began to experience persistent peripheral neuritis symptoms in the 2018 medical record. The segmentation mechanism of RAG makes it almost impossible to recall and place these two events, recorded in documents three years ago and three years later respectively, within the limited context window of LLM, and therefore cannot answer key clinical reasoning questions such as "Which previous treatment might be related to the patient's peripheral neuritis symptoms?"

[0005] (2) "Seeing only a part of the whole" incomplete context: The isolated text blocks retrieved are like pieces of a puzzle, lacking a description of the patient's overall health status and the full picture of the disease's evolution. LLM answers based on this fragmented information are prone to bias and may even contradict the patient's overall medical history, resulting in factual errors.

[0006] (3) Incomplete recall in information summarization and statistical problems: For problems that require a global perspective, such as "summarizing the lowest white blood cell count and corresponding dates after each chemotherapy session for this patient", RAG theoretically needs to recall every chemotherapy record and all subsequent blood routine reports. Semantic similarity-based retrieval mechanisms perform poorly in such tasks, often missing key information points, resulting in inaccurate statistical results.

[0007] Knowledge graphs, as a technology capable of effectively organizing and representing complex relationships between entities, are considered a potential solution to the aforementioned problems. However, existing general-purpose medical knowledge graphs (such as UMLS and SNOMED-CT) are static, universal knowledge bases. They can tell us that "oxaliplatin" may cause "peripheral neuritis," but they cannot answer whether and when this event occurred in a specific patient. Therefore, there is a lack of innovative methods that can dynamically construct a personal "medical life trajectory" graph for each patient and seamlessly combine the structured reasoning capabilities of the graph with the rich details of the original text. Summary of the Invention

[0008] In view of the above analysis, the embodiments of the present invention aim to provide an ultra-long-range medical record diagnosis and treatment question answering system and method based on dynamic temporal semantic graph, so as to solve the core technical problems faced by the prior art in processing ultra-long-range and massive medical records, such as the breakage of long-distance dependency relationships, incomplete contextual information, difficulty in summarizing global information, and insufficient reasoning ability.

[0009] On one hand, the present invention provides an ultra-long-range medical record diagnosis and treatment question-answering system based on dynamic temporal semantic graphs, the system comprising: The preprocessing module is used to preprocess patients' extremely long medical records to generate standardized data; The medical entity recognition module is used to identify event-type entities and descriptive entities in standardized data; associate time attributes with event-type entities to generate time-series entity tuples; summarize the time-series entity tuples and descriptive entities, and output the medical entity recognition results. The dynamic semantic graph construction module is used to construct a specific dynamic semantic graph for the corresponding patient by taking each temporal entity tuple as a graph node based on the medical entity recognition results and extracting the information of the descriptive entities related to the temporal entity tuple as the attribute information of the temporal entity tuple. The text-graph dual-mode fusion query engine is used to parse the medical questions raised by users and perform text-graph dual-mode fusion queries based on a specific dynamic semantic graph, using a collaborative approach of graph path retrieval mode and context-aware text retrieval mode. The diagnosis and treatment response module is used to generate diagnosis and treatment response results based on the text-graph dual-mode fusion query results.

[0010] Based on the above solution, the present invention also makes the following improvements: Furthermore, the medical entity recognition module performs the following: A sequence labeling model based on BERT-CRF was used to identify medical entities from standardized data, resulting in various types of medical entities. Determine whether each identified medical entity is an event-type entity; Associate event-type entities with time attributes to generate time-series entity tuples; Extract textual and medical type information from descriptive entities.

[0011] Furthermore, the dynamic semantic graph construction module includes a cross-text complex relationship inference and linking unit and a graph dynamic aggregation and storage unit; wherein, A cross-text complex relationship inference and linking unit is used to construct a relationship classification model based on graph neural networks based on medical entity recognition results; The graph dynamic aggregation and storage unit is used to construct a specific dynamic semantic graph for the corresponding patient based on a graph neural network-based relational classification model.

[0012] Furthermore, the cross-text complex relationship inference and linking unit performs the following: The information extraction results of descriptive entities related to time-series entity tuples are used as attribute information of time-series entity tuples. The relationship between pairs of time-series entity tuples is analyzed to determine candidate relationship pairs. Based on all candidate relation pairs, construct a relation classification model based on a graph neural network.

[0013] Furthermore, the construction of the relation classification model based on graph neural networks is performed as follows: Each temporal entity tuple is treated as a graph node, and the corresponding candidate relation pairs are treated as candidate edges between the corresponding graph nodes. Each graph node’s feature vector consists of entity text embedding, type embedding, and temporal embedding. The feature vector is propagated and aggregated through graph convolution or graph attention network. For each candidate edge, an edge classifier is used to predict the probability of its existence and the type of relationship.

[0014] Furthermore, the dynamic aggregation and storage unit of the map performs the following: By using graph nodes in the graph neural network-based relation classification model as graph nodes, candidate edges with a probability greater than a probability threshold as directed edges between corresponding graph nodes, and relation types of candidate edges as relation types of corresponding directed edges, a specific dynamic semantic graph for each patient is constructed.

[0015] Furthermore, the analysis of the relationships between pairwise temporal entity tuples, the determination of candidate relation pairs, and the execution of: For any two time-series entity tuples A and B, if they are mentioned in the same medical record, or their timestamps are within a preset threshold, or the combination of medical entity types has a known association in the medical knowledge base, then they are included as candidate relation pairs.

[0016] Furthermore, the text-graph dual-mode fusion query engine executes: The system identifies and converts user-generated medical questions into corresponding graph query languages. A coarse-grained text retrieval method based on question vector representation is used to retrieve patients' ultra-long-term medical records and determine the preliminary set of text fragments in the patients' ultra-long-term medical records. Input the generated graph query language into the specific dynamic semantic graph to obtain the graph path retrieval results; For each path in the map path retrieval results, a corresponding text retrieval query statement is generated. The text retrieval query statement is then used to filter and reorder the preliminary text fragment set to obtain accurate medical record fragments from the patient's ultra-long-term medical record information.

[0017] Furthermore, the diagnosis and treatment response module includes a two-way alignment and cross-validation unit and a traceable answer generation unit; wherein, The bidirectional alignment and cross-validation unit is used to perform bidirectional alignment and cross-validation between the paths returned by the graph path retrieval and the accurate medical record fragments returned by the context-aware text retrieval, so as to obtain a fused evidence set that has been validated and corrected. The traceable answer generation unit is used to organize medical questions and integrate evidence sets based on structured prompts to generate diagnostic and treatment responses.

[0018] On the other hand, the present invention also provides a long-range medical record diagnosis and treatment question-and-answer method based on dynamic temporal semantic graphs, the method comprising: Preprocess patients' extremely long medical records to generate standardized data; Identify event-type entities and descriptive entities in standardized data; associate time attributes with event-type entities to generate time-series entity tuples; summarize the time-series entity tuples and descriptive entities, and output the medical entity identification results; Based on the medical entity recognition results, each temporal entity tuple is used as a graph node, and the information extraction results of descriptive entities related to the temporal entity tuple are used as the attribute information of the temporal entity tuple to construct a specific dynamic semantic graph for the corresponding patient. It analyzes the medical questions raised by users and, based on a specific dynamic semantic graph, performs text-graph dual-mode fusion query by adopting a collaborative working method of graph path retrieval mode and context-aware text retrieval mode. Based on the text-graph dual-mode fusion query results, a diagnosis and treatment response is generated.

[0019] Compared with the prior art, the present invention can achieve at least one of the following beneficial effects: This invention provides a long-term medical record-based diagnostic question-answering system and method based on dynamic temporal semantic graphs. It relates to an intelligent information acquisition approach for massive, long-term, unstructured electronic medical records. By dynamically constructing a personal temporal semantic graph for each patient and combining it with graph-text dual-modal fusion retrieval technology, it can achieve a deep and global understanding and reasoning of a patient's decades-long medical history, answering complex clinical questions requiring long-term tracing with extremely high accuracy. This invention completely solves the core technical challenges faced by existing technologies in processing long-term, massive medical records, such as broken long-distance dependencies, incomplete contextual information, difficulties in summarizing global information, and insufficient reasoning ability, achieving a high-precision, high-timeliness, and strong logical reasoning diagnostic question-answering system.

[0020] In this invention, the above-described technical solutions can be combined with each other to achieve more preferred combinations. Other features and advantages of this invention will be set forth in the following description, and some advantages may become apparent from the description or be learned by practicing the invention. The objects and other advantages of this invention can be realized and obtained from what is particularly pointed out in the description and drawings. Attached Figure Description

[0021] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts. Figure 1 This is a schematic diagram of the structure of the ultra-long-range medical record diagnosis and treatment question-and-answer system based on dynamic temporal semantic graph provided in Embodiment 1 of the present invention; Figure 2 This is a flowchart of the ultra-long-range medical record diagnosis and treatment question-and-answer method based on dynamic temporal semantic graph provided in Embodiment 2 of the present invention. Detailed Implementation

[0022] Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, which form part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not intended to limit the scope of the present invention.

[0023] Specific embodiment 1 of the present invention discloses an ultra-long-range medical record diagnosis and treatment question-answering system based on dynamic temporal semantic graphs. A schematic diagram of the structure of this question-answering system is shown below. Figure 1 As shown below, the various modules of the question-and-answer system are explained in detail below.

[0024] (1) Preprocessing module The preprocessing module is used to preprocess patients' extremely long medical records to generate standardized data.

[0025] It should be noted that, in this embodiment, the patient's extended medical record covers all medical information up to the present. Therefore, during the data source access process, it is necessary to receive and integrate the patient's extended medical record to ensure comprehensive coverage of the patient's medical history.

[0026] The types of medical records for very long durations include structured data, unstructured data, and semi-structured data.

[0027] Structured data includes test data presented in tabular form, such as complete blood count, biochemical tests, and tumor markers. This data has clear headers and corresponding test result values, and is typically organized in rows and columns, facilitating data retrieval and analysis.

[0028] Unstructured data includes various text records from the medical process, such as admission records, progress notes, surgical records, discharge summaries, and nursing records.

[0029] Semi-structured data includes medical examination reports and imaging data, such as electrocardiogram (ECG) reports, pathology reports, and medical imaging reports. Specifically, the textual descriptions in ECG, pathology, and medical imaging reports are usually unstructured text information because they are recorded in natural language and do not have fixed fields or formats. However, these reports and imaging files may also contain some structured information, such as examination dates, basic patient information, and examination equipment parameters; these parts are considered structured data.

[0030] For medical data of different types in patients' ultra-long medical records, the following methods are used for preprocessing.

[0031] For structured data, preprocessing is performed as follows: each table header in the structured data is mapped to a standardized test item name, and each test result is precisely associated with its corresponding test date and time. This is then converted into a standardized text format, forming formatted data. Specifically, the structured data undergoes data cleaning, mapping different table headers (such as "white blood cell count" and "WBC") to standardized test item names, and precisely associating each test result with its corresponding test date and time. The processed structured data is converted into a standardized text or JSON format, containing the following fields: {test item standard name, test value, unit, reference range, test timestamp, source document ID}. This standardized, formatted text representation preserves the accuracy of the structured data while facilitating subsequent unified processing and graph construction. For example, a record in the original table, "WBC: 4.5 (10^9 / L), 2020-06-20 08:30", would be standardized to {"Test Item":"White Blood Cell Count", "Test Value":4.5, "Unit":"10^9 / L", "Reference Range":"3.5-9.5", "Test Timestamp":"2020-06-20 08:30:00", "Source Document ID":"Test Report_20200620"}.

[0032] For unstructured data, preprocessing is performed as follows: text cleaning, document deduplication, and content segmentation are carried out to obtain standardized text. For example, text cleaning involves removing irrelevant characters and formatting tags. During content segmentation, initial division can be based on natural paragraphs, headings, or record dates.

[0033] It should be clarified that, in the specific implementation process, if the structured or unstructured data is presented in the form of scanned documents, optical character recognition (OCR) technology will be used first to convert the scanned paper medical records or image-format reports into editable text, and then the corresponding preprocessing operations will be carried out.

[0034] For semi-structured data, the following preprocessing method is adopted: extract the structured and unstructured data from the semi-structured data respectively, process the extracted structured and unstructured data according to the corresponding preprocessing methods, and associate the processed formatted data and standardized text with the reports / images in the semi-structured data.

[0035] Furthermore, a unified method is used to normalize time information for all types of medical data, forming a standardized time format. Specifically, vague time expressions such as "3 days post-surgery," "this morning," and "Spring 2018" are analyzed in conjunction with the context and converted into the standard YYYY-MM-DD HH:MM:SS format.

[0036] (2) Medical Entity Recognition Module The medical entity recognition module is used to identify event-type entities and descriptive entities in standardized data; associate time attributes with event-type entities to generate time-series entity tuples; summarize the time-series entity tuples and descriptive entities, and output the medical entity recognition results.

[0037] First, it's important to clarify that current medical entity recognition technologies (such as BERT-based named entity recognition models) primarily focus on identifying and classifying text fragments containing medical concepts (e.g., identifying "aspirin" as a drug entity). However, these methods have the following limitations: the identified entities are static and isolated, lacking a temporal dimension; they cannot distinguish between multiple occurrences of the same entity at different times (e.g., a patient using the same drug multiple times at different periods); and they struggle to directly support temporal reasoning and causal analysis. While adding temporal attributes to entities is theoretically feasible, in existing technologies in this field, temporal information extraction is typically treated as a separate task of temporal expression recognition, distinct from entity recognition; even when research attempts to associate entities with time, it is limited to simple co-occurrence associations, lacking a rigorous event-time binding mechanism; and there is a lack of specific solutions tailored to the characteristics of the medical field (e.g., complex temporal representations and multiple temporal associations).

[0038] To address the aforementioned issues, this embodiment innovatively proposes the concept of "temporal entity tuples," extending traditional entity recognition to "event-based entity recognition." Specifically, this embodiment not only identifies the medical entities themselves, but more importantly, it identifies the occurrence time of the medical events corresponding to those entities, and forcibly binds one or more time attributes to each identified event-type entity, forming a "temporal entity" tuple. This technological breakthrough enables subsequent graph construction to naturally support complex queries such as temporal reasoning and causal tracing. This embodiment, through a specially designed joint training model and a strict temporal constraint mechanism, ensures that each event-type entity can be accurately associated with its occurrence time.

[0039] In this embodiment, event-type entities refer to medical events that occur or are recorded at a specific point in time and have a clear temporal attribute. These are observable clinical activities or changes in condition that actually occur during the patient's diagnosis and treatment, and have a start and end point in time. Event-type entities specifically include, but are not limited to, the following events: Diagnostic events: [Diagnosis] (refers to a confirmed diagnosis, such as "diagnosed with lung cancer on May 10, 2020"), [Suspected diagnosis] (refers to a preliminary diagnosis result), [Diagnosis correction] (refers to a change in the original diagnosis); Treatment-related events: [Medication] (a single medication event, including starting medication, stopping medication, and dosage adjustment), [Surgery] (a single surgical procedure), [Radiotherapy] (a single radiotherapy course), [Chemotherapy] (a single chemotherapy cycle); Examination and testing events: [Examination items] (such as an imaging examination, including CT, MRI), [Testing items] (such as a laboratory test, such as a complete blood count), [Pathological examination] (such as a histopathological examination); Symptom and sign events: [Symptoms] (the onset, worsening, and relief of symptoms, such as "coughing occurred on June 1, 2020"), [Signs] (abnormal signs found during physical examination); Clinical decision-making events: [treatment plan adjustment], [consultation], [departmental transfer]; Nursing record events: [vital signs measurement], [nursing procedures]; Patient-reported events: [Adverse reaction reports], [Efficacy evaluations].

[0040] In contrast to event-type entities are descriptive entities (i.e., non-event-type entities), including: anatomical locations (such as "left upper lobe of the lung"), disease names (when used as general concepts rather than specific diagnostic events), medical institution names, and doctor names, etc. These entities themselves do not have the attribute of time occurrence.

[0041] Specifically, the medical entity recognition module in this embodiment identifies event-type entities and descriptive entities in standardized data through the following steps: 1) A sequence labeling model based on BERT-CRF is used to identify medical entities from standardized data, resulting in various types of medical entities.

[0042] Sequence labeling models can be used to identify all types of medical entities in standardized data, including diagnoses, drugs, surgeries, symptoms, anatomical sites, and laboratory tests.

[0043] 2) Determine whether each identified medical entity is an event-type entity.

[0044] For example, based on the identified medical entities and their contextual semantics in standardized data, it is determined whether the corresponding medical entity belongs to the "event-type entity". For instance, in the text "a mass is visible in the left upper lobe of the patient's lung" in standardized data, "left upper lobe" is identified as an anatomical location (descriptive entity), while "mass" is identified as an imaging finding (event-type entity, corresponding to the result of a certain examination).

[0045] 3) Associate event-type entities with time attributes to generate time-series entity tuples.

[0046] The medical entity recognition module in this embodiment can identify all medical entities in standardized data, but only performs mandatory time association on event-type entities. This is an extension and deepening of traditional entity recognition. Specifically, in this embodiment, the association of time attributes is achieved in the following way: the associated time attribute is determined based on the time description of the identified event-type entity in the context of standardized data. Specifically, an event-type entity can be associated with a single timestamp (e.g., an examination on a specific date) or a time period (e.g., a chemotherapy cycle lasting several weeks, including start and end times). In the specific implementation process, a joint learning approach can be used to train the entity recognition and time relationship extraction models simultaneously. For each identified event-type entity, the model searches for a time expression in the surrounding text window and uses a relationship classifier to determine the relationship type between the time expression and the entity (e.g., "occurrence time", "start time", "end time"). If the time is not explicitly given in the text, the system will backtrack to the metadata timestamp of the paragraph or document (e.g., the record date) as the default time.

[0047] By forcibly associating one or more time attributes with an event-type entity, a corresponding time-series entity tuple can be constructed. That is, in this embodiment, the final structured information generated by the time-series entity tuple is represented as a single tuple, defined in the format: (entity content, entity type, start timestamp, end timestamp, source document and location). For example, ("Osimertinib", "Drugs", "2020-03-05 09:00:00", "2021-01-10 17:00:00", "Long-term Medical Orders_ID123").

[0048] The medical entity recognition module also extracts textual and medical type information from descriptive entities without binding them to temporal attributes. The extracted information from descriptive entities serves as attributes or descriptive information for event-type entities in the subsequent graph construction stage, enriching the semantic descriptions of graph nodes. For example, anatomical locations can serve as spatial positioning attributes for diagnostic or surgical events, and disease names can be used for type labeling of diagnostic events, thereby enhancing the semantic completeness of the graph and the accuracy of queries.

[0049] (3) Dynamic semantic graph construction module The dynamic semantic graph construction module is used to construct a patient-specific dynamic semantic graph by taking each temporal entity tuple as a graph node based on the medical entity recognition results and extracting the information of descriptive entities related to the temporal entity tuple as the attribute information of the temporal entity tuple.

[0050] The specific dynamic semantic graph constructed in this embodiment functions to transform isolated and disordered medical records into a structured and computable personal health knowledge network centered on the patient and with time as the core dimension.

[0051] In this embodiment, the dynamic semantic graph construction module includes a cross-text complex relationship inference and linking unit and a graph dynamic aggregation and storage unit, as detailed below.

[0052] 1) Cross-text complex relationship inference and linking unit A cross-text complex relationship inference and linking unit is used to construct a relationship classification model based on graph neural networks based on medical entity recognition results.

[0053] Preferably, this embodiment employs a global relation candidate generation mechanism to process the medical entity recognition results and construct a global candidate relation graph. It should be noted that in this embodiment, a temporal entity tuple is a structured data unit formed by binding event-type entities with time attributes. Its format is: entity content, entity type, start timestamp, end timestamp, source document, and location. It is the basic constituent element of the graph nodes in this embodiment, while a temporal entity refers to the core medical event entity contained in the temporal entity tuple. This embodiment processes complete temporal entity tuples, rather than individual event-type entities, because it is necessary to use the time attributes and source information contained in the temporal entity tuple for relation inference. The specific implementation method is described below.

[0054] First, the relationships between pairs of temporal entity tuples are analyzed to determine candidate relation pairs. Specifically, for any two temporal entity tuples A and B, if they are mentioned in the same case, or their timestamps are within a preset threshold, or the combination of medical entity types has a known association in the medical knowledge base, they are included as candidate relation pairs. In the specific implementation process, the information extraction results of descriptive entities related to the temporal entity tuples can also be used as the attribute information of the temporal entity tuples. Based on this, the relationships between pairs of temporal entity tuples are analyzed to determine candidate relation pairs. In addition, this embodiment also introduces temporal consistency constraints and medical knowledge enhancement mechanisms. During the relation inference process, temporal logic constraints and external medical knowledge graphs are introduced to filter candidate relations that do not meet temporal logic or obviously conflict with medical knowledge, thereby determining the final candidate relation pairs. It should be noted that the relation inference basis here includes information such as entity type combination, temporal logic, semantic context, and global co-occurrence frequency. These bases are used for preliminary screening and filtering in the candidate relation pair determination stage to ensure that only meaningful entity pairs are subsequently classified into deep relations.

[0055] Based on all candidate relation pairs, a relation classification model based on a graph neural network is constructed. Specifically, each temporal entity tuple is treated as a graph node, and the corresponding candidate relation pair is treated as a candidate edge between the corresponding graph nodes. The feature vector of each graph node consists of entity text embedding, type embedding, and temporal embedding, and the feature vector is propagated and aggregated through graph convolution or graph attention network. For each candidate edge, an edge classifier is used to predict the existence probability and relation type of the corresponding candidate edge.

[0056] In this embodiment, the edge classifier can be trained as follows: the training data includes medical expert-annotated relationship pairs, remote supervision data based on an external knowledge base, and negative samples. By introducing features such as semantic context and global co-occurrence frequency, the accuracy of relationship classification is improved. The relationship types predicted by the edge classifier include, but are not limited to: [treatment], [induction], [improvement], [deterioration], [accompanying], [diagnostic basis], [prior to], etc., which are consistent with the relationship types defined below.

[0057] It should be noted that traditional methods mainly extract relationships within a single sentence or document, failing to handle long-distance relationships across documents; existing knowledge graph construction methods largely rely on predefined templates or rules, lacking the ability to mine complex and implicit relationships; and most methods do not fully utilize temporal information as a constraint for relationship inference. In contrast, the cross-text complex relationship inference and linking unit provided in this embodiment represents a core technological breakthrough in knowledge graph construction, manifested in the following three key innovations: Innovation Point 1: Long-distance relationship inference capability across documents and over long distances This unit can infer long-distance relationships across hundreds of documents spanning several years on a complete set of "temporally sequenced entity tuples." Traditional relationship extraction methods can only handle local, short-distance entity relationships and cannot establish causal or temporal relationships between events separated by several years and distributed across different documents. This invention, through global relationship inference, can discover potential causal associations such as "chemotherapy medication in 2015" and "neurological damage symptoms appearing in 2018."

[0058] Innovation Point Two: Intelligent Relationship Judgment Integrating Multi-Dimensional Information Relationship inference takes into account information from multiple dimensions, including entity type, temporal logic, semantic context, prior medical knowledge, and global statistical features, to make intelligent judgments.

[0059] Innovation Point 3: Construction of Dynamic and Adaptive Relationship Networks The relationship network constructed in this unit is dynamically generated based on the individual medical records of a specific patient. The system can update the relationship network in real time according to newly added medical record data, ensuring that the graph always reflects the patient's latest health status.

[0060] 2) Dynamic map aggregation and storage unit The graph dynamic aggregation and storage unit is used to construct a specific dynamic semantic graph for the corresponding patient based on a graph neural network-based relational classification model.

[0061] In the specific implementation process, the graph dynamic aggregation and storage unit uses graph nodes in the graph neural network-based relation classification model as graph nodes, candidate edges with a probability greater than a probability threshold as directed edges between corresponding graph nodes, and relation types of candidate edges as relation types of corresponding directed edges, thereby constructing a specific dynamic semantic graph for the corresponding patient.

[0062] That is, in this embodiment, the graph dynamic aggregation and storage unit treats the generated "temporal entity tuples" as graph nodes and "inferred relations" as directed edges. Here, the "temporal entity tuples" correspond to the temporal entity tuples in the medical entity recognition results output by the medical entity recognition module, while the "inferred relations" correspond to the relation types and their corresponding entity pairs predicted by the cross-text complex relation inference and linking unit through the edge classifier, which meet the confidence threshold requirements. This constructs a specific dynamic semantic graph applicable only to this patient. This graph is stored in a high-performance graph database, supporting subsequent complex path queries and dynamic incremental updates.

[0063] In addition, this embodiment also sets up a dynamic incremental update mechanism: when a patient generates new medical record data, the system only performs preprocessing and entity recognition on the newly added documents to generate new temporal entity tuples; after disambiguation and deduplication, the new entities are inserted into the specific dynamic semantic graph, and relationship inference is performed with existing nodes within a limited time window. Finally, the incremental writing of graph nodes and directed edges is completed through the graph database transaction mechanism to realize the dynamic incremental update of the specific dynamic semantic graph.

[0064] (4) Text-Graph Dual-Mode Fusion Query Engine The text-graph dual-mode fusion query engine is used to parse medical questions raised by users and perform text-graph dual-mode fusion queries based on a specific dynamic semantic graph, using a collaborative approach of graph path retrieval mode and context-aware text retrieval mode.

[0065] In this embodiment, the text-graph dual-mode fusion query engine is key to achieving high-precision question answering. It receives medical questions in natural language form from users and achieves text-graph dual-mode fusion query through the collaborative operation of graph path retrieval mode and context-aware text retrieval mode.

[0066] The system identifies the intent of users' medical questions and transforms the queries to generate corresponding graph query languages.

[0067] In practice, the intent recognition and slot filling model transforms user-generated medical questions in natural language into graph query languages ​​(such as Cypher or GQL). The training data for the intent recognition and slot filling model consists of manually labeled data, automatically generated templates, and real user query annotations. This model can translate questions such as "What chemotherapy drugs did the patient use after lung cancer surgery?" into corresponding Cypher queries.

[0068] For example, regarding the question "Did the medication B used by the patient after surgery A cause abnormal liver function?", Intent recognition and slot filling model generation: MATCH p = (s:surgery{name:'A'})-[:after]->(m:drug{name:'B'})-[:inducing]->(l:test items{name:'liver function', result:'abnormal'}) WHERE s.starttimestamp < m.starttimestamp AND m.starttimestamp < l.starttimestamp RETURN p 1) Graph Path Retrieval Mode The generated graph query language is input into the specific dynamic semantic graph to obtain the graph path retrieval results. The graph path retrieval results are presented in the form of a set of paths, each path consisting of several temporal entity tuple nodes connected by directed relation edges, reflecting the causal, temporal, or accompanying relationships between the medical events involved in the user's question.

[0069] Specifically, the generated graph query language is input into the specific dynamic semantic graph. The following path search and verification operations are used to obtain graph path retrieval results: First, the generated graph query language is input into the specific dynamic semantic graph. During graph traversal, time-constrained pruning, bidirectional breadth-first search, and heuristic search strategies based on graph statistics are introduced to find and verify paths that meet the query conditions. Next, the found paths are scored for confidence, and finally, the paths with the highest confidence are returned. These paths constitute the output of the graph path retrieval mode, i.e., the "graph path retrieval return results".

[0070] 2) Context-aware text retrieval mode In practice, when a user submits a question, the system initiates graph path retrieval and coarse-grained text retrieval based on question vector representation in parallel.

[0071] A coarse-grained text retrieval method based on question vector representation is used to retrieve patients' ultra-long-term medical records and determine the preliminary set of text fragments in the patients' ultra-long-term medical records.

[0072] For each path in the map path retrieval results, a corresponding text retrieval query statement is generated. The text retrieval query statement is then used to filter and reorder the preliminary text fragment set to obtain accurate medical record fragments from the patient's ultra-long-term medical record information.

[0073] In other words, after the graph path retrieval returns results, the system extracts key entities, time windows, and source document IDs from the paths to construct enhanced text retrieval queries. It employs entity keyword filtering, time range filtering, and document source priority strategies to finely filter and reorder the coarse recall results. The coarse recall results referred to here are the initial set of text fragments returned by coarse-grained text retrieval based on question vector representation, rather than the multiple paths returned by the graph path retrieval.

[0074] In practice, text retrieval can be achieved through a search engine (such as Elasticsearch). First, a coarse-grained search is performed based on the vector representation of the original question to obtain preliminary recall results. Then, based on the key entities, time windows, and source document IDs extracted from the graph path, an enhanced text retrieval query is constructed. On the basis of vector similarity retrieval, Boolean queries and time interval filtering are superimposed to finely filter and reorder the coarse recall results, and finally return highly relevant text fragments.

[0075] Therefore, the text-graph dual-mode fusion query results in this embodiment include graph path retrieval results and precise medical record fragments in ultra-long-range medical record information.

[0076] (5) Diagnosis and treatment response module The diagnosis and treatment response module, also known as the multi-source evidence fusion and traceable answer generation module, is used to generate diagnosis and treatment response results based on the text-graph dual-mode fusion query results.

[0077] The diagnosis and treatment response module includes a two-way alignment and cross-validation unit and a traceable answer generation unit, the functions of which are described in detail below.

[0078] 1) Two-way alignment and cross-validation unit The bidirectional alignment and cross-validation unit is used to perform bidirectional alignment and cross-validation on the (structured) paths returned by the graph path retrieval and the accurate medical record fragments (i.e. original narrative fragments) returned by the context-aware text retrieval, so as to obtain a validated and corrected fused evidence set.

[0079] Graph-to-text verification: Using the source document and location information of nodes, verify whether the corresponding events or relationships are explicitly or implicitly described in the original text; Text-to-graph verification: Perform fast entity and relation recognition on text fragments and check whether they have corresponding nodes or edges in the graph path.

[0080] By scoring and ranking the confidence of "path text representation" and "original text fragment" pairs through cross-encoder models, the most reliable evidence combination is selected.

[0081] 2) Traceable Answer Generation Unit The traceable answer generation unit is used to organize medical questions and integrate evidence sets based on structured prompts to generate diagnostic and treatment responses.

[0082] Specifically, in this embodiment, structured prompts can be constructed as follows: the question, evidence list, and citation rules are organized into structured prompts, listing the ID, type, content, and source of each piece of evidence. The question in the structured prompts is determined based on medical questions, and the evidence list and citation rules are determined based on a fused evidence set.

[0083] The process of generating traceable answers can be based on a large language model that is instructed to generate answers strictly according to evidence, attaching an evidence ID after each key argument. For example: The patient started taking osimertinib on March 5, 2020 [G1]. On June 18, 2020, the medical record recorded the onset of a dry cough [T1]. A subsequent CT scan (June 20, 2020) showed "multiple patchy ground-glass opacities in both lungs" [T2]. Based on this, on June 21, 2020, the clinic suspected drug-induced interstitial pneumonia [T3] and took measures such as discontinuing osimertinib and hormone therapy [T3]. The initial imaging finding when this side effect was suspected was multiple patchy ground-glass opacities in both lungs [T2].

[0084] In practice, the system backend parses the evidence ID, which enables visualization of the subgraphs and highlighting of the original medical records on the frontend interface, thus achieving a fully traceable answer.

[0085] Specific embodiment 2 of the present invention discloses a long-range medical record diagnosis and treatment question-answering method based on dynamic temporal semantic graphs. The flowchart of the method is as follows: Figure 2 As shown, the method includes: Step S1: Preprocess the patient's extremely long medical record information to generate standardized data; Step S2: Identify event-type entities and descriptive entities in the standardized data; associate time attributes with event-type entities to generate time-series entity tuples; summarize the time-series entity tuples and descriptive entities, and output the medical entity identification results; Step S3: Based on the medical entity recognition results, each temporal entity tuple is used as a graph node, and the information extraction results of descriptive entities related to the temporal entity tuple are used as the attribute information of the temporal entity tuple to construct a specific dynamic semantic graph for the corresponding patient. Step S4: Analyze the medical questions raised by the user, and based on the specific dynamic semantic graph, use a collaborative approach of graph path retrieval mode and context-aware text retrieval mode to perform text-graph dual-mode fusion query; Step S5: Generate a diagnosis and treatment response based on the text-graph dual-mode fusion query results.

[0086] The above-described method and device embodiments are based on the same principle, and their related aspects can be referenced from each other to achieve the same technical effect.

[0087] Specific embodiment 3 of the present invention illustrates the implementation process of the method of the present invention in detail through a specific clinical scenario.

[0088] Scenario: A 65-year-old lung cancer patient has an electronic medical record spanning eight years, totaling approximately 350,000 words. The clinician poses a complex question: "After this patient began receiving osimertinib targeted therapy in 2020, how was the risk of interstitial pneumonia monitored and managed? What were the initial imaging findings when this side effect was suspected?" Step S1: Load / build patient-specific atlas.

[0089] The system first checks whether the patient's dynamic temporal semantic graph already exists. If it does not exist, the graph construction module is activated to process the entire 8-year medical record, generate the graph, and store it.

[0090] Step S2: Start the dual-mode fusion query engine.

[0091] After receiving the question, the engine initiates the collaborative search process.

[0092] Step S3: Perform a map path retrieval.

[0093] The problem was resolved as a graph query: find a path starting from the node [Drug: Osimertinib, Dosage Time > 2020-01-01] that can connect to related nodes such as [Diagnosis: Interstitial Pneumonia] or [Symptoms: Dyspnea / Dry Cough].

[0094] The image search results might follow a path: (Osimertinib, 2020-03-05) - [Medication] -> (Patient) <- [Symptoms] - (Symptoms: Dry cough, 2020-06-18) <- [Imaging Findings] - (CT scan: Ground-glass opacities in both lungs, 2020-06-20) <- [Diagnostic Criteria] - (Diagnosis: Suspected drug-induced interstitial pneumonia, 2020-06-21) This path clearly reveals the logic and chronological order of the events.

[0095] Step S4: Perform context-aware text retrieval.

[0096] Extract key entities and time from the map path in step S3: osimertinib, interstitial pneumonia, dry cough, CT, ground-glass opacity, 2020-06-18 to 2020-06-21.

[0097] This information was used to construct a highly focused query that guided the vector retrieval model to precisely find text paragraphs containing these keywords and with timestamps around June 2020 in 350,000 words of original medical records.

[0098] Search results may include: outpatient medical records from June 18, 2020, the original CT report from June 20, and progress notes from June 21 (which recorded treatment measures such as discontinuation of osimertinib and administration of hormone pulse therapy).

[0099] Step S5: Perform multi-source evidence fusion and answer generation.

[0100] Upon receiving the graph path and related original text paragraphs, the system verifies that the logical chain of the graph is detailed in the original text and assigns a high confidence score to the evidence combination. Finally, this evidence is integrated and input into a large language model to generate an answer with evidence labels.

[0101] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware, and the program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0102] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A dynamic timing semantic graph-based ultra-long-range medical record diagnosis and treatment question and answer system, characterized in that, The system comprises: a preprocessing module configured to preprocess the ultra-long medical record information of a patient to generate standardized data; a medical entity recognition module configured to recognize event-type entities and descriptive entities in the standardized data, associate time attributes with the event-type entities to generate time-series entity tuples, and aggregate the time-series entity tuples and the descriptive entities to output medical entity recognition results; a dynamic semantic graph construction module configured to construct a specific dynamic semantic graph of a patient based on the medical entity recognition results, by taking each time-series entity tuple as a graph node and taking information extraction results of the descriptive entities related to the time-series entity tuple as attribute information of the time-series entity tuple; a text-graph dual-mode fusion query engine configured to analyze a medical question raised by a user and perform text-graph dual-mode fusion query based on the specific dynamic semantic graph by using a collaborative working mode based on a graph path retrieval mode and a context-aware text retrieval mode; a diagnosis and treatment reply module configured to generate diagnosis and treatment reply results based on the text-graph dual-mode fusion query results.

2. The dynamic timing semantic graph based ultra-long-range medical record diagnosis and treatment question-answering system according to claim 1, characterized in that, The medical entity recognition module is configured to: perform medical entity recognition on the standardized data by using a BERT-CRF-based sequence labeling model to obtain various types of medical entities; determine whether each recognized medical entity is an event-type entity; associate time attributes with the event-type entities to generate time-series entity tuples; and perform text and medical type information extraction on the descriptive entities.

3. The dynamic timing semantic graph based ultra-long-range medical record diagnosis and treatment question-answering system according to claim 2, characterized in that, The dynamic semantic graph construction module comprises a cross-text complex relationship inference and linking unit and a graph dynamic aggregation and storage unit; wherein: the cross-text complex relationship inference and linking unit is configured to construct a graph neural network-based relationship classification model based on the medical entity recognition results; the graph dynamic aggregation and storage unit is configured to construct a specific dynamic semantic graph of a patient based on the graph neural network-based relationship classification model.

4. The dynamic timing semantic graph-based ultra-long-range medical record diagnosis and treatment question-answering system according to claim 3, characterized in that, The cross-text complex relationship inference and linking unit is configured to: take information extraction results of the descriptive entities related to the time-series entity tuples as attribute information of the time-series entity tuples, analyze relationships between two time-series entity tuples, and determine candidate relationship pairs; construct a graph neural network-based relationship classification model based on all candidate relationship pairs.

5. The dynamic timing semantic graph based ultra-long-range medical record diagnosis and treatment question-answering system according to claim 4, characterized in that, The graph neural network-based relationship classification model is configured to: take each time-series entity tuple as a graph node and take a corresponding candidate relationship pair as a candidate edge between the corresponding graph nodes; a feature vector of each graph node is composed of entity text embedding, type embedding, and time embedding, and the feature vector is propagated and aggregated by graph convolution or graph attention network; for each candidate edge, an edge classifier is used to predict the existence probability and relationship type of the corresponding candidate edge.

6. The dynamic timing semantic graph-based ultra-long-range medical record diagnosis and treatment question-answering system according to claim 5, characterized in that, The graph dynamic aggregation and storage unit is configured to: take graph nodes in the graph neural network-based relationship classification model as graph nodes, take candidate edges with an existence probability greater than a probability threshold as directed edges between the corresponding graph nodes, take the relationship type of the candidate edge as the relationship type of the corresponding directed edge, and construct a specific dynamic semantic graph of a patient.

7. The dynamic timing semantic graph-based ultra-long-range medical record diagnosis and treatment question-answering system according to claim 6, characterized in that, The analysis of relationships between two time-series entity tuples and the determination of candidate relationship pairs are performed by: For any two time series entity tuples A and B, if they are mentioned in the same case, or the time stamps are within a preset threshold, or the combination of medical entity types has a known association in the medical knowledge base, the candidate relationship pair is included.

8. The dynamic timing semantic graph-based ultra-long-range medical record diagnosis and treatment question-answering system according to any one of claims 1-7, characterized in that, The text-graph dual-mode fusion query engine performs: Intention recognition and query conversion on the medical problem proposed by the user to generate the corresponding graph query language; Adopting a coarse-grained text retrieval method based on question vector representation, the patient's super-long medical record information is retrieved to determine the preliminary text segment set in the patient's super-long medical record information; The generated graph query language is input into the specific dynamic semantic graph to obtain the graph path retrieval result; According to each path in the graph path retrieval result, a corresponding text retrieval query statement is generated; Using the text retrieval query statement to filter and reorder in the preliminary text segment set, the accurate medical record segment in the patient's super-long medical record information is obtained.

9. The dynamic timing semantic graph-based ultra-long-range medical record diagnosis and treatment question-answering system according to claim 8, wherein, The diagnosis and treatment reply module includes a bidirectional alignment and cross-validation unit and a traceable answer generation unit; wherein, The bidirectional alignment and cross-validation unit is used to bidirectionally align and cross-validate the path returned by the graph path retrieval and the accurate medical record segment returned by the context-aware text retrieval to obtain a verified and corrected fusion evidence set; The traceable answer generation unit is used to organize the medical problem and the fusion evidence set according to the structured prompt words to generate the diagnosis and treatment reply result.

10. A dynamic timing semantic graph-based ultra-long-range medical record diagnosis and treatment question and answer method, characterized in that, The method comprises: Pretreating the patient's super-long medical record information to generate standardized data; Identifying event class entities and descriptive entities in the standardized data; associating time attributes with event class entities to generate time series entity tuples; aggregating time series entity tuples and descriptive entities to output medical entity recognition results; According to the medical entity recognition results, each time series entity tuple is taken as a graph node, and the information extraction results of the descriptive entities related to the time series entity tuple are taken as attribute information of the time series entity tuple to construct a specific dynamic semantic graph of the corresponding patient; Analyzing the medical problem proposed by the user and based on the specific dynamic semantic graph, adopting a collaborative working mode based on graph path retrieval mode and context-aware text retrieval mode, text-graph dual-mode fusion query is performed; According to the text-graph dual-mode fusion query result, a diagnosis and treatment reply result is generated.

Images