Method and system for quality detection of medical documents based on large model and knowledge graph

Abstract

This invention relates to the field of data processing technology, and in particular to a method and system for quality inspection of medical documents based on large models and knowledge graphs. By matching the target attribute tags of the target medical document with the preset attribute tags of preset quality inspection standards in a standard database, the corresponding target quality inspection standards are obtained. Combined with the deep understanding and analysis of the content of the medical document and the target quality inspection standards by the preset large language model, it can comprehensively cover various medical concepts, relationships and logics involved in the medical document. It can accurately identify various abnormal triple information such as logical contradictions, data errors, semantic errors, omissions, and format errors in the medical document, which greatly improves the reliability of quality inspection.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a method and system for quality inspection of medical documents based on large models and knowledge graphs. Background Technology

[0002] Medical records, as crucial documentation of medical activities, directly impact the reliability of clinical diagnoses and treatment decisions, as well as the handling of medical disputes. With the rapid development of medical informatization, the number of medical records has exploded. Traditional manual review methods, due to their low efficiency, high subjectivity, and susceptibility to variations in the professional competence of reviewers, are insufficient to meet the demands of large-scale medical record quality inspection.

[0003] Existing technologies utilize rule engines and simple machine learning algorithms for automated quality detection of medical documents. However, rule engine-based methods rely on manually written rule bases, which struggle to cover complex and ever-changing medical knowledge and semantic scenarios, and cannot effectively detect situations not covered by the rules. Simple machine learning algorithms exhibit low accuracy and poor generalization ability when dealing with semantic understanding and logical reasoning in medical documents. Furthermore, medical knowledge is characterized by its high specialization, rapid updates, and complex relationships, making it difficult for existing methods to fully utilize the relationships between medical knowledge and to deeply uncover potential quality issues in medical documents, resulting in insufficient reliability of the detection results.

[0004] Therefore, improving the reliability of medical record quality inspection has become an urgent problem to be solved. Summary of the Invention

[0005] To address the aforementioned technical problems, the present invention employs a quality inspection method for medical documents based on large models and knowledge graphs. This quality inspection method includes:

[0006] S10, obtain the target medical document to be detected and the target attribute label corresponding to the target medical document.

[0007] S20. Based on the target attribute tag, obtain the target quality detection standard corresponding to the target medical document from the standard database. The standard database stores the preset quality detection standard corresponding to each preset attribute tag. Each preset quality detection standard includes preset quality evaluation content and preset quality scoring standard. All preset quality detection standards are stored in the standard database in the form of a knowledge graph.

[0008] S30: Input the target medical document and the target quality inspection standard into the preset large language model to obtain the abnormal triplet information corresponding to the target medical document.

[0009] S40: Based on the target medical document and the abnormal triplet information, obtain the quality inspection report corresponding to the target medical document.

[0010] This invention also provides a quality inspection system for medical documents based on a large model and knowledge graph. The system includes a processor and a memory. The memory stores a standard database, which includes T standard medical texts and attribute tags and knowledge graph information corresponding to each standard medical document, where T is a positive integer. The processor includes:

[0011] The data acquisition module is used to acquire the target medical document to be detected and the target attribute tags corresponding to the target medical document.

[0012] The quality inspection standard acquisition module is used to obtain the target quality inspection standard corresponding to the target medical document from the standard database based on the target attribute tag. The standard database stores the preset quality inspection standard corresponding to each preset attribute tag. Each preset quality inspection standard includes preset quality evaluation content and preset quality scoring standard. All preset quality inspection standards are stored in the standard database in the form of a knowledge graph.

[0013] The anomaly information acquisition module is used to input the target medical document and the target quality testing standard into a preset large language model to obtain the anomaly triple information corresponding to the target medical document.

[0014] The test report acquisition module is used to obtain the quality test report corresponding to the target medical document based on the target medical document and abnormal triplet information.

[0015] The present invention has at least the following beneficial effects: by matching the target attribute tags of the target medical document with the preset attribute tags of the preset quality inspection standards in the standard database, the corresponding target quality inspection standards are obtained. Combined with the deep understanding and analysis of the content of the medical document and the target quality inspection standards by the preset large language model, it can comprehensively cover various medical concepts, relationships and logics involved in the medical document. It can accurately identify various abnormal triple information such as logical contradictions, data errors, semantic errors, omissions, and format errors in the medical document, which greatly improves the reliability of quality inspection. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1A flowchart illustrating a quality detection method for medical documents based on a large model and knowledge graph, provided in Embodiment 1 of the present invention;

[0018] Figure 2 This is a schematic diagram of the target processor module in a medical document quality inspection system based on a large model and knowledge graph, provided in Embodiment 2 of the present invention.

[0019] Figure 3 This is a flowchart of the computer program executing a medical document quality inspection system provided in Embodiment 3 of the present invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It is understood that, where appropriate, the terms used to distinguish similar objects can be interchanged so that the invention can also be implemented in other embodiments besides the illustrated or described embodiments. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products, or devices.

[0022] Example 1

[0023] This first embodiment provides a quality inspection method for medical documents based on large models and knowledge graphs. See [link to relevant documentation]. Figure 1 The quality inspection method includes:

[0024] S10, obtain the target medical document to be detected and the target attribute label corresponding to the target medical document.

[0025] Among them, target medical documents refer to specific medical documents that need to be quality tested, such as medical records, diagnostic reports, surgical records, and other documents that record important medical information such as patient information, condition information, diagnostic process, and treatment plan.

[0026] Target attribute tags can be abstract descriptions of the key features of a target medical document, used to identify attributes such as the document type, department, type of disease involved, and stage of diagnosis and treatment. For example, the attribute tag could be "cardiology-coronary heart disease-inpatient medical record".

[0027] The system retrieves target medical documents from the medical information system through methods such as API calls and file reading. At the same time, it automatically or manually generates target attribute tags based on the content and metadata of the documents, and establishes a correspondence between medical documents and attribute tags.

[0028] The aforementioned target attribute tags provide a basis for accurately retrieving standard databases and obtaining corresponding quality testing standards in subsequent steps, making the testing work more targeted, improving testing efficiency and reliability, avoiding blind searching and analysis, and reducing the waste of computing resources.

[0029] S20. Based on the target attribute tag, obtain the target quality detection standard corresponding to the target medical document from the standard database. The standard database stores the preset quality detection standard corresponding to each preset attribute tag. Each preset quality detection standard includes preset quality evaluation content and preset quality scoring standard. All preset quality detection standards are stored in the standard database in the form of a knowledge graph.

[0030] In one specific embodiment, S20 includes the following steps:

[0031] The preset quality inspection standard that matches the target attribute label is determined as the target quality inspection standard.

[0032] The standard database is a database specifically designed to store various preset quality inspection standards. It contains a large amount of quality inspection standard information for different types of medical documents, providing a unified standard and basis for the quality inspection of medical documents.

[0033] Preset attribute tags are a series of attribute tags predefined in a standard database, covering the attribute characteristics of various possible medical documents. They are used to identify attributes such as document type, department, disease type, and treatment stage. By matching them with the target attribute tags of the target medical document, the corresponding quality inspection standards can be determined.

[0034] Preset quality inspection standards refer to detailed quality inspection specifications formulated for each preset attribute label, including preset quality evaluation content and preset quality scoring standards, which are used to clarify which aspects should be evaluated for quality and how to score medical documents with specific attribute labels.

[0035] The pre-defined quality evaluation content is part of the pre-defined quality inspection standards. It specifically stipulates various aspects of the quality evaluation of medical documents, such as the completeness, accuracy, standardization, and logic of the document content, and clarifies the specific items and indicators that need to be checked and evaluated.

[0036] The preset quality scoring criteria are part of the preset quality inspection criteria. Based on the preset quality evaluation content, corresponding scoring rules and methods are formulated to quantify the performance of medical documents on various evaluation items, so as to finally obtain a comprehensive quality score, thereby intuitively reflecting the quality level of medical documents.

[0037] A knowledge graph is a graph-based data structure composed of nodes and edges, which can clearly represent various nodes and the relationships between them. In this embodiment, each preset attribute label, preset quality evaluation content, and preset quality scoring standard can be regarded as a node in the knowledge graph, and the relationships between preset attribute labels, preset quality evaluation content, and preset quality scoring standards are represented as edges. Through the structure of the knowledge graph, complex quality inspection standard information can be efficiently organized and managed, facilitating quick querying and retrieval of the target quality inspection standard corresponding to the target attribute label.

[0038] By traversing the standard database, the target attribute labels are matched with the predicted attribute labels. If they match, the corresponding preset quality inspection standard is determined as the target quality inspection standard.

[0039] As described above, by leveraging the pre-defined quality inspection standards corresponding to the standard database and knowledge graph, a rich and accurate medical knowledge reference system is provided for the target medical documents. This enables the subsequent large language model to analyze the medical documents based on the complete knowledge background and quality inspection standards, thereby discovering potential quality problems more comprehensively and in-depth, and improving the comprehensiveness and reliability of the detection.

[0040] S30: Input the target medical document and the target quality inspection standard into the preset large language model to obtain the abnormal triplet information corresponding to the target medical document.

[0041] In one specific embodiment, S30 includes the following steps:

[0042] S301, input the target medical document and the target quality inspection standard into the preset large language model to obtain several abnormal triples corresponding to the target medical document and the abnormal type corresponding to each abnormal triple.

[0043] S302, based on the exception type corresponding to each exception triple, obtain the quantity percentage corresponding to each exception type.

[0044] S303, the abnormal triplet, the proportion of the number of abnormal triplets, and the preset severity weight corresponding to each abnormality type are used as the abnormal triplet information corresponding to the target medical document.

[0045] Among them, the pre-set large language model refers to a language model trained on a large amount of medical text data. It has the ability to understand and generate natural language, analyze the semantics and logic in medical documents, compare and analyze the entities and relationships involved with the knowledge graph, and judge whether each triple is reasonable based on the pre-set rules and the knowledge learned during training, and identify abnormal triple information with problems.

[0046] Abnormal triples refer to "entity-relationship-entity" combinations in the target medical document that do not conform to the logic and norms of medical knowledge, as identified by the large language model. For example, "penicillin, treatment, patients allergic to penicillin" violates common medical knowledge and is an abnormal triple.

[0047] Anomaly types refer to the classification of the nature of errors in anomaly triples, including logical contradictions, emphasizing logical conflicts between different contents in medical documents, such as a diagnosis of "pneumonia" but a treatment plan for "gastritis"; data errors, focusing on numerical and data-related errors, such as incorrect recording of patient age or errors in entering test indicator data; semantic errors, focusing on semantic issues such as medical terminology and sentence expression, such as mistakenly writing "coronary atherosclerotic heart disease" as "coronary atherosclerotic heart disease," affecting the correct semantic communication; omissions, emphasizing situations where medical information that should be recorded is missing, such as a patient's allergy history; and format errors, emphasizing abnormalities in the recording format.

[0048] The percentage of abnormal triples corresponding to each abnormal type is the proportion of the total number of abnormal triples, which is used to measure the distribution of different abnormal types in the target medical document.

[0049] Preset severity weights refer to numerical values ​​pre-set to quantify the severity of anomalies by considering the differences in their impact on the quality of medical documents and medical decisions. For example, logical contradictions involving patient safety are assigned higher weights, while formatting errors are assigned lower weights.

[0050] The specific abnormal triplets corresponding to each abnormality type, the calculated proportion of the number, and the preset severity weights are integrated to form an abnormal triplet information set containing multi-dimensional information. This comprehensively describes the specific content, distribution, and severity of abnormal issues in the target medical document, providing rich and structured data support for the subsequent generation of quality inspection reports.

[0051] The above-mentioned combination of large language model and target quality detection standard realizes intelligent semantic understanding and logical reasoning of medical document content. It can quickly and accurately locate errors and contradictions in medical documents and obtain anomaly triple information set containing multi-dimensional information. It comprehensively describes the specific content, distribution and severity of abnormal problems in the target medical document, and improves the reliability of subsequent anomaly detection of medical documents.

[0052] S40: Based on the target medical document and the abnormal triplet information, obtain the quality inspection report corresponding to the target medical document.

[0053] Among them, the quality inspection report refers to a document that records the quality status of the target medical document, comprehensively summarizes the problems, analyzes the causes, and provides improvement suggestions. It is a direct presentation of the inspection results.

[0054] In one specific embodiment, S40 includes the following steps:

[0055] S401: Input the target medical document and abnormal triple information into the preset large language model to obtain the quality inspection report corresponding to the target medical document.

[0056] The pre-set large language model, based on previously trained medical knowledge, quality inspection logic, and report generation patterns, receives the target medical document text and abnormal triple information, analyzes and integrates the abnormal information, and combines the overall content and characteristics of the medical document with natural language generation technology to automatically generate a quality inspection report covering problem description, quality score, cause analysis, improvement suggestions, and other content.

[0057] In one specific embodiment, S40 further includes the following steps:

[0058] S402, obtain the preset quality inspection report template.

[0059] S403: Input the target medical document, abnormal triplet information, and preset quality inspection report template into the preset large language model to obtain the quality inspection report corresponding to the target medical document.

[0060] The preset quality inspection report template refers to a pre-designed document template with a fixed format and content framework, which includes sections such as report title, basic information of medical documents, summary of abnormalities, quality score, and improvement suggestions, providing a unified and standardized format for quality inspection reports.

[0061] Specifically, a suitable quality inspection report template can be obtained from a database or file system storing report templates via program commands or interface calls. After receiving the target medical document, abnormal triplet information, and the quality inspection report template, a pre-defined large language model uses the template as a framework to populate the corresponding sections of the template with various abnormal issues and their analysis results from the abnormal triplet information. For example, the abnormality type, quantity percentage, and quality score are entered into the abnormality summary section. Based on the specific content and characteristics of the target medical document, content is generated in the quality analysis conclusion and improvement suggestion sections, ensuring that the generated report content perfectly integrates with the template format, is logically coherent, and accurately expressed.

[0062] The above-mentioned standardized report format based on the quality inspection report template ensures consistency and standardization in quality inspection reports of different medical documents, facilitating reading, comparison, and management. Furthermore, the standardized template can guide the large language model to generate more comprehensive and organized report content, avoiding the omission of important information and improving the professionalism and readability of the quality inspection report.

[0063] As described above, by matching the target attribute tags of the target medical document with the preset attribute tags of the preset quality inspection standards in the standard database, the corresponding target quality inspection standards are obtained. Combined with the deep understanding and analysis of the content of the medical document and the target quality inspection standards by the preset large language model, it can comprehensively cover various medical concepts, relationships and logics involved in the medical document. It can accurately identify various abnormal triple information such as logical contradictions, data errors, semantic errors, omissions, and format errors in the medical document, which greatly improves the reliability of quality inspection. It provides medical staff and managers with clear and intuitive medical document quality assessment results, which helps to quickly understand the problems and severity of the document, so as to take timely and targeted improvement measures to improve the quality of medical documents, thereby ensuring the reliability of medical information and providing strong support for medical decision-making. At the same time, it also helps to improve the level of medical quality management.

[0064] Example 2

[0065] This second embodiment provides a quality inspection system for medical documents based on a large model and knowledge graph. The system includes a target processor and a target memory. The target memory stores a standard database, which includes T standard medical texts and attribute tags and knowledge graph information corresponding to each standard medical document, where T is a positive integer. Figure 2 As shown, the target processor includes:

[0066] The data acquisition module 21 is used to acquire the target medical document to be detected and the target attribute tags corresponding to the target medical document.

[0067] The quality inspection standard acquisition module 22 is used to obtain the target quality inspection standard corresponding to the target medical document from the standard database according to the target attribute tag. The standard database stores the preset quality inspection standard corresponding to each preset attribute tag. Each preset quality inspection standard includes preset quality evaluation content and preset quality scoring standard. All preset quality inspection standards are stored in the standard database in the form of a knowledge graph.

[0068] The abnormal information acquisition module 23 is used to input the target medical document and the target quality inspection standard into the preset large language model to obtain the abnormal triplet information corresponding to the target medical document.

[0069] The test report acquisition module 24 is used to obtain the quality test report corresponding to the target medical document based on the target medical document and the abnormal triplet information.

[0070] In one specific embodiment, the quality inspection standard acquisition module 22 includes:

[0071] The target quality inspection standard acquisition submodule is used to determine the preset quality inspection standard whose corresponding preset attribute label is consistent with the target attribute label as the target quality inspection standard.

[0072] In one specific embodiment, the anomaly information acquisition module 23 includes:

[0073] The anomaly type acquisition submodule is used to input the target medical document and the target quality inspection standard into the preset large language model to obtain several anomaly triples corresponding to the target medical document and the anomaly type corresponding to each anomaly triple.

[0074] The quantity percentage acquisition submodule is used to obtain the quantity percentage corresponding to each exception type based on the exception type corresponding to each exception triplet.

[0075] The abnormal information acquisition submodule is used to take the abnormal triplet, the proportion of the number of each abnormal type and the preset severity weight as the abnormal triplet information of the target medical document.

[0076] In one specific embodiment, the test report acquisition module 24 includes:

[0077] The first test report acquisition submodule is used to input the target medical document and abnormal triple information into a preset large language model to obtain the quality test report corresponding to the target medical document.

[0078] In one specific embodiment, the test report acquisition module 24 includes:

[0079] The report template retrieval submodule is used to retrieve preset quality inspection report templates.

[0080] The first detection report acquisition submodule is used to input the target medical document, abnormal triplet information and preset quality detection report template into the preset large language model to obtain the quality detection report corresponding to the target medical document.

[0081] Example 3

[0082] This embodiment provides a quality inspection system for medical documents. The system includes a processor and a memory storing a computer program. The memory also stores a reference database, which includes M reference triples and corresponding reference evaluation values ​​for each triple. Each reference triple includes a first reference entity, a second reference entity, and a reference relation. The reference relations include N levels of association relationships, where M and N are both positive integers. Figure 3 As shown, when a computer program is executed by a processor, it performs the following steps:

[0083] S1. Input the acquired target medical document and several preset attribute information into the preset large language model to obtain several target entities corresponding to the target medical document. The preset attribute information includes at least symptom information, drug information and examination information.

[0084] S2, based on the association information of each preset attribute information, form K target triples from all target entities, where each target triple includes a first target entity, a second target entity, and a target relation. The target relation includes N levels of association relations, and K is a positive integer.

[0085] S3. For any target triplet, match the current target triplet with each reference triplet to obtain the reference triplet corresponding to the current target triplet, and use the reference evaluation value of the reference triplet corresponding to the current target triplet as the target evaluation value of the current target triplet.

[0086] S4. Determine the first comprehensive evaluation value based on each target triplet and the target evaluation value corresponding to each target triplet.

[0087] S5. If the first comprehensive evaluation value does not meet the first preset condition, then input all target triples and the target evaluation value corresponding to each target triple into the preset large language model to obtain the quality inspection report corresponding to the target medical document.

[0088] The reference database can be constructed based on several reference texts, which can refer to historical medical documents. For any given reference text, several reference entities are extracted from it. Based on the preset attribute information of any two reference entities, the level of the association between the two reference entities is determined as the reference relationship, and one of the two reference entities is designated as the first reference entity and the other as the second reference entity, forming a reference triplet.

[0089] It should be noted that when there are identical reference entities in a single reference text, only one reference entity is retained for constructing the reference triple. When the constructed reference triples are identical, only one reference triple is retained.

[0090] In this embodiment, the preset attribute information may also include diagnostic information, treatment plan information, etc. The preset attribute information has a sequential order, and the result of arranging the preset attribute information in sequence may be symptom information, diagnostic information, treatment plan information, drug information, and examination information.

[0091] In one specific implementation, the preset attribute information of the reference entity may also be name, feature, feature category, scheme, project, object, etc., which can be determined by the implementer according to the actual application scenario.

[0092] The level of association can be determined based on the number of preset attribute information intervals between two preset attribute information in a sequence of preset attribute information arranged in chronological order. When two preset attribute information are directly adjacent, the level of association is determined to be 1. When two preset attribute information are separated by one preset attribute information, the level of association is determined to be 2, and so on. Using the previous example, the first target entity can correspond to symptom information, the second target entity can correspond to examination information, and there are three preset attribute information intervals between them: diagnosis information, treatment plan information, and drug information. Therefore, the level of association can be determined to be 4.

[0093] The reference evaluation value can indicate the importance of the corresponding reference triple in the reference text.

[0094] A pre-defined large language model is used to extract entities from medical documents, and the same pre-defined large language model can also be used to extract reference entities.

[0095] The target evaluation value can represent the importance of the corresponding target triple in the target medical document, and the first comprehensive evaluation value can be used to assess the quality of the target medical document.

[0096] When the first comprehensive evaluation value does not meet the first preset condition, the preset large language model, based on the medical knowledge, quality detection logic and report generation mode learned in previous training, receives all target triples and the target evaluation value corresponding to each target triple, analyzes and integrates the abnormal information, and combines the overall content and characteristics of the medical document to automatically generate a quality detection report covering problem description, cause analysis and improvement suggestions using natural language generation technology.

[0097] In one specific implementation, when the computer program is executed by the processor, the following steps are also performed:

[0098] For any reference triplet, determine the reference evaluation value corresponding to the current reference triplet based on the reference relationships and expert evaluation values ​​in the current reference triplet.

[0099] Among them, the expert evaluation value can be evaluated based on the importance of the current reference triple in the reference text by experts in the field of application applied by the implementer. The expert evaluation value can be a normalized value.

[0100] Specifically, the reference relation can be normalized. That is, the maximum value N-1 of the reference relation is subtracted from the current reference relation, and then compared with the difference N-2 between the maximum value N-1 and the minimum value 1 of the reference relation to obtain the normalized result of the current reference relation.

[0101] Multiply the normalized result of the current reference relationship with the expert evaluation value, and use the result of the multiplication as the reference evaluation value corresponding to the current reference triplet.

[0102] In one specific embodiment, S2 includes the following steps:

[0103] S21, for any two target entities, take one target entity as the current first target entity and the other target entity as the current second target entity.

[0104] S22. Based on the association information between the preset attribute information corresponding to the current first target entity and the preset attribute information corresponding to the current second target entity, determine the level n of the association relationship between the current first target entity and the current second target entity. A target triplet is formed by the current first target entity, the current second target entity and the level n of the association relationship, where n is an integer in the range [1, N].

[0105] In this embodiment, the preset attribute information is arranged in a specific order to facilitate the identification of the first target entity and the second target entity. The level of association can be determined based on the number of preset attribute information segments between two preset attribute information segments in the sequentially arranged preset attribute information sequence. For example, when two preset attribute information segments are directly adjacent, the level of association is determined to be 1; when two preset attribute information segments are separated by one preset attribute information segment, the level of association is determined to be 2, and so on.

[0106] In one specific embodiment, S3 includes the following steps:

[0107] S31. For any target triplet, match the current target triplet with each reference triplet. If there is a reference triplet that is consistent with the current target triplet, use the reference triplet as the reference triplet corresponding to the current target triplet, and use the reference evaluation value of the reference triplet corresponding to the current target triplet as the target evaluation value of the current target triplet.

[0108] S32, if there is no reference triplet that matches the current target triplet, the first preset value is used as the target evaluation value of the current target triplet.

[0109] The matching can be achieved by similarity calculation. In this embodiment, the similarity calculation can be achieved by cosine similarity. The value range of cosine similarity is [-1, 1]. It should be noted that the matching in this embodiment is a complete match. That is, when using cosine similarity, if the cosine similarity between the current target triplet and a reference triplet is 1, then the current target triplet is considered to be consistent with the reference triplet. Otherwise, the current target triplet is considered to be inconsistent with the reference triplet.

[0110] Specifically, if there is a text error that may result in no reference triplet that matches the current target triplet, then the first preset value is used as the target evaluation value of the current target triplet. The first preset value can be set to 0.

[0111] In one specific embodiment, S4 includes the following steps:

[0112] S41, a set of triplets is formed by the target triplets that have the same first target entity and target relationship, resulting in Q sets of triplets, where Q is a positive integer.

[0113] S42, for any set of triples, calculate the mean of the target evaluation values ​​corresponding to each target triple in the current set of triples, and use the mean calculation result as the comprehensive sub-evaluation value.

[0114] S43, sum the comprehensive sub-evaluation values ​​corresponding to all triplet sets respectively, and use the sum as the first comprehensive evaluation value.

[0115] In this context, a target medical document may contain multiple target entities with the same target relationship as a target entity. For example, a target entity may correspond to symptom information, which may correspond to other target entities with various drug information. To avoid the simultaneous existence of multiple target triples formed by a target entity and multiple target entities with the same target relationship, thus affecting the normal detection of the target entity and other target entities with the same target relationship, this embodiment forms a set of triples with the first target entity and the same target relationship. The target evaluation values ​​corresponding to each target triple in a single triple set are averaged, and the average calculation result is used as the comprehensive sub-evaluation value of the triple set. Then, the comprehensive sub-evaluation values ​​corresponding to all triple sets are added together, and the sum is used as the first comprehensive evaluation value. This ensures that multiple target triples with the same first target entity and target relationship can affect the first comprehensive evaluation value through the average, avoiding excessive influence of multiple target triples with the same first target entity and target relationship on the comprehensive evaluation.

[0116] In one specific implementation, the first preset condition is that the first comprehensive evaluation value is greater than the first evaluation value threshold.

[0117] The first evaluation value threshold can be used to assess whether the first comprehensive evaluation value meets the first preset condition. When the first comprehensive evaluation value is greater than the first evaluation value threshold, it can be considered that the number of multiple target triples with different target entities and target relationships in the target medical document meets the expectation, and thus it is determined that the first comprehensive evaluation value meets the first preset condition.

[0118] In one specific embodiment, step S4 further includes the following steps:

[0119] S44. Add up the target evaluation values ​​corresponding to all target triples respectively, and use the sum as the second comprehensive evaluation value.

[0120] Accordingly, step S5 includes:

[0121] If the first comprehensive evaluation value does not meet the first preset condition, or the second comprehensive evaluation value meets the second preset condition, then all target triples and the target evaluation value corresponding to each target triple are input into the preset large language model to obtain the quality inspection report corresponding to the target medical document.

[0122] While the first comprehensive evaluation value can avoid the excessive influence of multiple target triples with the same first target entity and target relationship on the comprehensive evaluation, it is difficult to make an effective judgment when there is a need to ensure the existence of multiple target triples with the same first target entity and target relationship, especially when the target evaluation values ​​of multiple target triples with the same first target entity and target relationship are all high. Therefore, in this embodiment, the target evaluation values ​​corresponding to all target triples are added together and the result of the addition is used as the second comprehensive evaluation value.

[0123] The second evaluation threshold can be used to assess whether the second comprehensive evaluation value meets the second preset condition. When the second comprehensive evaluation value is greater than the second evaluation threshold, it can be considered that the number of multiple target triples with the same target entity and target relationship and higher target evaluation value in the target medical document meets the expectation, and thus it is determined that the second comprehensive evaluation value meets the second preset condition.

[0124] Specifically, this embodiment uses a first preset condition and a second preset condition to make a joint judgment. Only when the first comprehensive evaluation value meets the first preset condition and the second comprehensive evaluation value meets the second preset condition is the quality inspection of the target medical document considered to have passed. This avoids the completeness judgment of the target triples in the target medical document being affected by multiple target triples with high target evaluation values ​​and identical target entities and target relationships. It also ensures the judgment requirement that multiple target triples with high target evaluation values ​​and identical target entities and target relationships exist simultaneously. When the preset conditions are not met, the abnormal information is analyzed and integrated through a preset large language model. Combined with the overall content and characteristics of the medical document, natural language generation technology is used to automatically generate a quality inspection report covering problem description, cause analysis, improvement suggestions, and other content.

[0125] In one specific implementation, when the computer program is executed by the processor, the following steps are also performed:

[0126] Obtain P sample texts that belong to the same text type as the target medical document.

[0127] Given P sample texts, determine several sample triples, where P is a positive integer.

[0128] For any reference triplet, count the number of sample triplets that are identical to the current reference triplet, and use the normalized value of the statistical result as the statistical value corresponding to the current reference triplet.

[0129] For any sample text, a first intermediate evaluation value and a second intermediate evaluation value are determined based on several sample triples corresponding to the current sample, several reference triples whose corresponding statistical values ​​are greater than a preset statistical threshold, and reference evaluation values ​​corresponding to each reference triple.

[0130] The threshold value of the first evaluation value is determined based on each first intermediate evaluation value.

[0131] The threshold value for the second evaluation value is determined based on each of the second intermediate evaluation values.

[0132] The sample triple can include a first sample entity, a second sample entity, and a sample relation. The sample text can refer to historical text belonging to the same text type as the target medical document. Here, the text type can be a disease type, that is, the sample text can be historically stored text that records the same disease as the target medical document. Correspondingly, the sample text and the target medical document have a high degree of similarity. Thus, the first evaluation value threshold and the second evaluation value threshold can be obtained by statistically analyzing the evaluation results of P sample data, which can serve as the data basis for subsequent evaluation of the target medical document.

[0133] Statistical values ​​can characterize the frequency of occurrence of corresponding reference triples in each sample text. When determining the first and second intermediate evaluation values, they are determined by several reference triples whose statistical values ​​are greater than the preset statistical threshold. This can effectively avoid the impact of reference triples with fewer occurrences on quality detection, improve the reliability of the first and second intermediate evaluation values, and thus improve the reliability of quality detection.

[0134] Specifically, for any sample triplet, the current sample triplet is matched with several reference triplets whose statistical values ​​are greater than a preset statistical threshold to obtain the reference triplet corresponding to the current sample triplet. The reference evaluation value of the reference triplet corresponding to the current sample triplet is used as the sample evaluation value of the current sample triplet. Similarly, if the current sample triplet does not have a corresponding reference triplet, the sample evaluation value of the current sample triplet is determined to be 0.

[0135] A set of sample triples is formed by sample triples that have the same first sample entity and sample relationship. For any set of sample triples, the mean of the sample evaluation values ​​corresponding to each sample triple in the current set of sample triples is calculated, and the mean calculation result is used as the sample sub-evaluation value. The sample sub-evaluation values ​​corresponding to all sample triples are added together, and the sum is used as the first intermediate evaluation value. The sample evaluation values ​​corresponding to all sample triples are added together, and the sum is used as the second intermediate evaluation value.

[0136] The mean of the first intermediate evaluation values ​​of each sample text is used as the first evaluation value threshold, and the mean of the second intermediate evaluation values ​​of each sample text is used as the second evaluation value threshold.

[0137] In one specific implementation, since the methods for extracting abnormal information from the target medical document are different in Embodiment 1 and Embodiment 3, the extracted differential information is different. Therefore, the target medical document, abnormal triplet information, all target triplets, and the target evaluation value corresponding to each target triplet can be input into a preset large language model to obtain a quality inspection report corresponding to the target medical document. This allows for a more comprehensive extraction and analysis of abnormal information in the target medical document, thereby improving the comprehensiveness and reliability of the quality inspection report.

[0138] The above-mentioned method involves forming multiple target triples from the target entities extracted from the target medical document, determining the target evaluation value of the target triples based on the matching results between the target triples and the reference triples in the reference database, and then determining the first evaluation value threshold of the target medical document based on the target evaluation value of each target triple. The first evaluation value threshold is then used to determine whether the target medical document passes the quality inspection. This method can more comprehensively evaluate the target entities and the relationships between them, thereby improving the reliability of the quality inspection of the target medical document.

[0139] While specific embodiments of the invention have been described in detail by way of example, those skilled in the art should understand that the examples are for illustrative purposes only and not intended to limit the scope of the invention. Those skilled in the art should also understand that various modifications can be made to the embodiments without departing from the scope and spirit of the invention. The scope of this invention is defined by the appended claims.

Claims

1. A method for quality inspection of medical documents based on large models and knowledge graphs, characterized in that, The quality testing method includes: S10, Obtain the target medical document to be detected and the target attribute label corresponding to the target medical document; S20, based on the target attribute tag, obtain the target quality detection standard corresponding to the target medical document from the standard database. The standard database stores the preset quality detection standard corresponding to each preset attribute tag. Each preset quality detection standard includes preset quality evaluation content and preset quality scoring standard. All preset quality detection standards are stored in the standard database in the form of a knowledge graph. S30, input the target medical document and the target quality inspection standard into the preset large language model to obtain the abnormal triplet information corresponding to the target medical document; S40, input the target medical document and several preset attribute information into the preset large language model to obtain several target entities corresponding to the target medical document, wherein the preset attribute information includes at least symptom information, drug information and examination information; Based on the association information of each preset attribute information, K target triples are formed from all target entities. Each target triple includes a first target entity, a second target entity, and a target relation. The target relation includes N levels of association relations, and K is a positive integer. For any target triplet, match the current target triplet with each reference triplet to obtain the reference triplet corresponding to the current target triplet, and use the reference evaluation value of the reference triplet corresponding to the current target triplet as the target evaluation value of the current target triplet. The first comprehensive evaluation value is determined based on each target triplet and the target evaluation value corresponding to each target triplet. The target evaluation values ​​corresponding to all target triples are added together, and the sum is used as the second comprehensive evaluation value. If the first comprehensive evaluation value is less than or equal to the first evaluation value threshold, or the second comprehensive evaluation value is less than or equal to the second evaluation value threshold, then the target medical document, the abnormal triplet information, all target triplets, and the target evaluation value corresponding to each target triplet are input into the preset large language model to obtain the quality inspection report corresponding to the target medical document. The first evaluation value threshold and the second evaluation value threshold are obtained through the following steps: Obtain P sample texts that belong to the same text type as the target medical document, where P is a positive integer; Based on P sample texts, determine several sample triples; For any reference triplet, count the number of sample triplets that are the same as the current reference triplet, and use the normalized value of the statistical result as the statistical value corresponding to the current reference triplet. For any sample text, the first intermediate evaluation value and the second intermediate evaluation value are determined based on several sample triples corresponding to the current sample, several reference triples whose corresponding statistical values ​​are greater than the preset statistical threshold, and the reference evaluation values ​​corresponding to each reference triple. Based on each of the first intermediate evaluation values, determine the threshold value for the first evaluation value; The threshold value for the second evaluation value is determined based on each of the second intermediate evaluation values.

2. The method for quality inspection of medical documents based on large models and knowledge graphs according to claim 1, characterized in that, S20 includes the following steps: The preset quality inspection standard that matches the target attribute label is determined as the target quality inspection standard.

3. The method for quality inspection of medical documents based on large models and knowledge graphs according to claim 1, characterized in that, S30 includes the following steps: S301, Input the target medical document and the target quality detection standard into the preset large language model to obtain a number of abnormal triples corresponding to the target medical document and the abnormal type corresponding to each abnormal triple; S302, based on the exception type corresponding to each exception triple, obtain the quantity percentage corresponding to each exception type; S303, the abnormal triplet, the proportion of the number of each abnormal type, and the preset severity weight are used as the abnormal triplet information corresponding to the target medical document.

4. The method for quality inspection of medical documents based on large models and knowledge graphs according to claim 3, characterized in that, S40 includes the following steps: S401, input the target medical document and the abnormal triple information into the preset large language model to obtain the quality inspection report corresponding to the target medical document.

5. The method for quality inspection of medical documents based on large models and knowledge graphs according to claim 3, characterized in that, S40 also includes the following steps: S402, Obtain the preset quality inspection report template; S403, input the target medical document, the abnormal triplet information, and the preset quality inspection report template into the preset large language model to obtain the quality inspection report corresponding to the target medical document.

6. A quality inspection system for medical documents based on large models and knowledge graphs, characterized in that, The quality inspection system includes a processor and a memory. The memory stores a standard database, which includes T standard medical texts and attribute tags and knowledge graph information corresponding to each standard medical text, where T is a positive integer. The processor includes: The data acquisition module is used to acquire the target medical document to be detected and the target attribute tags corresponding to the target medical document; The quality inspection standard acquisition module is used to obtain the target quality inspection standard corresponding to the target medical document from the standard database according to the target attribute tag. The standard database stores the preset quality inspection standard corresponding to each preset attribute tag. Each preset quality inspection standard includes preset quality evaluation content and preset quality scoring standard. All preset quality inspection standards are stored in the standard database in the form of a knowledge graph. An anomaly information acquisition module is used to input the target medical document and the target quality inspection standard into a preset large language model to obtain the anomaly triplet information corresponding to the target medical document; The test report acquisition module is used to input the target medical document and several preset attribute information into the preset large language model to obtain several target entities corresponding to the target medical document. The preset attribute information includes at least symptom information, drug information and examination information. Based on the association information of each preset attribute information, K target triples are formed from all target entities. Each target triple includes a first target entity, a second target entity, and a target relation. The target relation includes N levels of association relations, and K is a positive integer. For any target triplet, match the current target triplet with each reference triplet to obtain the reference triplet corresponding to the current target triplet, and use the reference evaluation value of the reference triplet corresponding to the current target triplet as the target evaluation value of the current target triplet. The first comprehensive evaluation value is determined based on each target triplet and the target evaluation value corresponding to each target triplet. The target evaluation values ​​corresponding to all target triples are added together, and the sum is used as the second comprehensive evaluation value. If the first comprehensive evaluation value is less than or equal to the first evaluation value threshold, or the second comprehensive evaluation value is less than or equal to the second evaluation value threshold, then the target medical document, the abnormal triplet information, all target triplets, and the target evaluation value corresponding to each target triplet are input into the preset large language model to obtain a quality inspection report corresponding to the target medical document. The inspection report acquisition module is further configured to: Obtain P sample texts that belong to the same text type as the target medical document, where P is a positive integer; Based on P sample texts, determine several sample triples; For any reference triplet, count the number of sample triplets that are the same as the current reference triplet, and use the normalized value of the statistical result as the statistical value corresponding to the current reference triplet. For any sample text, the first intermediate evaluation value and the second intermediate evaluation value are determined based on several sample triples corresponding to the current sample, several reference triples whose corresponding statistical values ​​are greater than the preset statistical threshold, and the reference evaluation values ​​corresponding to each reference triple. Based on each of the first intermediate evaluation values, determine the threshold value for the first evaluation value; The threshold value for the second evaluation value is determined based on each of the second intermediate evaluation values.

7. The medical document quality inspection system based on large models and knowledge graphs according to claim 6, characterized in that, The quality inspection standard acquisition module includes: The target quality inspection standard acquisition submodule is used to determine the preset quality inspection standard whose corresponding preset attribute label is consistent with the target attribute label as the target quality inspection standard.

8. The medical document quality inspection system based on large models and knowledge graphs according to claim 6, characterized in that, The anomaly information acquisition module includes: The anomaly type acquisition submodule is used to input the target medical document and the target quality detection standard into a preset large language model to obtain a number of anomaly triples corresponding to the target medical document and the anomaly type corresponding to each anomaly triple. The quantity percentage acquisition submodule is used to obtain the quantity percentage corresponding to each exception type based on the exception type corresponding to each exception triplet. The abnormal information acquisition submodule is used to take the abnormal triplet, the proportion of the number of each abnormal type and the preset severity weight as the abnormal triplet information corresponding to the target medical document.

9. The medical document quality inspection system based on large models and knowledge graphs according to claim 6, characterized in that, The test report acquisition module includes: The first detection report acquisition submodule is used to input the target medical document and the abnormal triple information into the preset large language model to obtain the quality detection report corresponding to the target medical document.

10. The medical document quality inspection system based on large models and knowledge graphs according to claim 6, characterized in that, The test report acquisition module includes: The report template retrieval submodule is used to retrieve preset quality inspection report templates; The first detection report acquisition submodule is used to input the target medical document, the abnormal triplet information and the preset quality detection report template into the preset large language model to obtain the quality detection report corresponding to the target medical document.

Images