Large model-based medical quality control index automatic calculation method

Abstract

The invention discloses a medical quality control index calculation automation method based on a large model. The method comprises the following steps: firstly, preprocessing electronic medical record data, and storing the electronic medical record data in json according to id, document and field forms; then, the index definition is expanded based on the large model, and an index standardization language I is obtained; secondly, on the basis of the index standardization language I, the electronic medical record is judged in combination with a large model and a rule engine. And finally, evaluating from two perspectives of fact judgment and final answer. According to the method, the semi-automatic expansion index standardization language can be realized on the basis of the large model, the automatic calculation of the medical quality control indexes is realized on the basis of the large model by utilizing the index standardization language, and an effective method is provided for the high efficiency and the accuracy of index calculation.

Description

Technical Field

[0001] The present invention belongs to the field of quality control index calculation in the medical field, and specifically relates to an automatic calculation method for medical quality control indexes based on a large model. Background Art

[0002] Quality Control Indicator is a criterion for evaluating and measuring the quality of medical services provided by medical institutions and a tool for promoting continuous improvement of medical quality. my country has been expanding and promoting single disease quality control indicators since 2009. In 2020, the state requires all medical institutions to strengthen the quality management and control of single diseases and has formed a dynamic detection system for 51 single disease indicators.

[0003] At present, the calculation of quality indicators usually relies on rules and NLP technology to pre-process the text, and then combines it with manual judgment. For some structured texts, rule scripts can be used directly for judgment and statistics. However, for a large amount of unstructured text, it is necessary to formulate rules first, and then process it through NLP technology, such as information extraction, text summarization and text classification, and finally judge by experts. In addition, the BERT-based text classification method lacks a certain degree of interpretability and cannot effectively process long text medical records.

[0004] In summary, the current methods require a lot of manual construction of rules, which is cumbersome and time-consuming. There are problems with the timeliness, accuracy and precision of the data, which cannot meet the management agency's needs for the control and improvement of medical quality. Therefore, it is urgent to develop a method for the automatic calculation of medical quality control indicators to solve the above problems. Summary of the invention

[0005] In view of this, the present invention proposes an automated calculation method for medical quality control indicators based on a large model. The specific steps include:

[0006] S1. First, obtain the original electronic medical record data, perform desensitization and deduplication, save it in JSON format according to the medical record id, document name, and field name, and obtain the electronic medical record collection;

[0007] S2. Then, based on the big model, the original indicator definition is expanded into a standardized language I through fact logic splitting, field positioning, and knowledge enhancement methods;

[0008] S3. Secondly, based on the standardized language I, use the big model and rule engine to make judgments on each electronic medical record;

[0009] S4. Finally, construct an evaluation dataset based on expert-annotated data and use a large model to achieve automated evaluation.

[0010] Further, step S2 includes:

[0011] S21, where the original definition of an indicator is I = <definition, calculation formula>, and after expansion, the standardized language is I = <definition, calculation formula, facts, logical rules, electronic medical record fields, related knowledge>. Wherein, facts represent the judgment items involved in the original definition; logical rules are used to integrate and reason various judgment items; electronic medical record fields represent fields associated with the original definition; related knowledge represents the related supplementary knowledge of indicator I;

[0012] S22, wherein, a relevant example is constructed, and the fact logic splitting of the original definition is realized based on ICL to obtain a fact set and a logic rule set;

[0013] S23, using the sentence-transformer algorithm to calculate the matching score between the indicator definition I and the content of each field in the electronic medical record, and selecting the one with the highest score as the electronic medical record field, wherein the sentence-transformer algorithm embeds sentences, paragraphs or documents into a high-dimensional space to calculate the semantic similarity between the indicator definition and the content of the electronic medical record;

[0014] S24. Use vector matching technology to retrieve the top 5 fragments most relevant to the indicator definition from quality control documents and medical books as relevant knowledge.

[0015] Further, step S3 includes:

[0016] S31. First, based on the electronic medical record field of the standardized language I, the input medical record can be confirmed. If the medical record field does not exist, the method in S23 needs to be used again to match the field content, so as to confirm the medical record content input this time;

[0017] S32. Next, based on the fact set of the normalized language I, the large model is used to perform fact judgment on the medical record content confirmed in S31 to obtain a fact judgment set;

[0018] S33. Finally, the logical rules in the normalized language I are used to infer and judge the fact judgment set in S32 to obtain the final result.

[0019] Further, step S4 includes:

[0020] S41. First, construct an evaluation test set based on the expert-annotated answers. The results of a single test data include two dimensions: fact judgment and final answer judgment.

[0021] S42, then, similar to step S32, use the large model to perform fact judgment on the medical record content, and match the result with the fact judgment annotated answer;

[0022] S43. Finally, the rule engine is used to perform logical reasoning on the fact judgment results in S42 to obtain the final answer, and compared with the standard answer, and the indicator calculation results are optimized by iterating S2 and S3;

[0023] After adopting the above strategy, the positive effects of the present invention are:

[0024] (1) The indicator normalization language can be automatically extended through large models, thereby reducing the process of manually constructing detailed fact and logic sets.

[0025] (2) The above-mentioned indicator normalization language can be used to realize the automatic calculation of medical quality control indicators based on a large model. This method makes the indicator calculation more interpretable, more stable, and more accurate by separating factual judgment from logical rules, thus providing an effective method for the calculation of indicators on a large scale, for different diseases, and for different specialties.

[0026] (3) An automated evaluation method is proposed. Combined with a small amount of expert-annotated data, it can continuously iterate the facts and logic sets in the standardized language, making the final calculation results more reliable and accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to make the purpose, technical solution and advantages of the invention more clear, the present invention will be further described in detail below with reference to the accompanying drawings, in which:

[0028] Figure 1 It is a schematic flow chart of a method for automatically calculating medical quality control indicators based on a large model of the present invention;

[0029] Figure 2 It is an example diagram of an indicator definition of the present invention; DETAILED DESCRIPTION

[0030] In order to enable those skilled in the art to better understand the solutions of the present invention and to make the above-mentioned objectives, technical solutions and advantages of the present invention more obvious and understandable, the present invention is further described in detail below in conjunction with embodiments and accompanying drawings of the embodiments.

[0031] like Figure 1 As shown in the figure, the main process of this case is divided into four steps: 1. Preprocess the electronic medical record data and save it in json format according to id, document, and field. 2. Expand the indicator definition based on the big model to obtain the indicator normalization language I. 3. Based on the indicator normalization language I, combine the big model and the rule engine to judge the electronic medical record. 4. Evaluate from two perspectives: fact judgment and final answer.

[0032] S1. First, obtain the original electronic medical record data, perform desensitization and deduplication, save it in JSON format according to the medical record id, document name, and field name, and obtain the electronic medical record collection;

[0033] S2. Then, based on the big model, the original indicator definition is expanded into a standardized language I through fact logic splitting, field positioning, and knowledge enhancement methods;

[0034] S3. Secondly, based on the standardized language I, use the big model and rule engine to make judgments on each electronic medical record;

[0035] S4. Finally, construct an evaluation dataset based on expert-annotated data and use a large model to achieve automated evaluation.

[0036] Further, step S2 includes:

[0037] S21. Let an indicator be originally defined as I = <definition, calculation formula>. After expansion, the standardized language is I = <definition, calculation formula, facts, logical rules, electronic medical record fields, related knowledge>. Facts represent the judgment items involved in the original definition; logical rules are used to integrate and reason various judgment items; electronic medical record fields represent fields associated with the original definition; and related knowledge represents the related supplementary knowledge of indicator I. Taking "improvement / stability rate at discharge" as an indicator of a single disease type of cerebral hemorrhage as an example, the definition and calculation formula of this indicator are as follows: Figure 2 shown.

[0038] S22, wherein, relevant examples are constructed, and the fact logic splitting of the original definition is realized based on ICL to obtain a fact set and a logic rule set. Specifically, we select relevant examples:

[0039] Sample input definition: "The ratio of successful coronary angiography immediately after coronary intervention to the total number of coronary interventions during the same period. Successful coronary angiography means that the residual stenosis of the lesion after stenting is less than 20% or the residual stenosis of the lesion after simple percutaneous transluminal coronary angioplasty (PTCA) is less than 50%, and the coronary blood flow is thrombolysis in myocardial infarction (TIMI) grade 3."

[0040] Example input calculation formula:

[0041]

[0042] Sample output: "Fact 1: surgical procedure, Fact 2: residual stenosis, Fact 3, TIMI grade; Logic rule 1: if the result of Fact 1 is PCI, then the result of Fact 2 is <20%; if the result of Fact 1 is PTCA, then the result of Fact 2 is <50%; Logic rule 2: the result of Fact 3 is grade 3; finally, the result of Logic rules 1 and 2 is AND"

[0043] Based on the above example, ICL is used to split the original fact logic.

[0044] Input definition: "The ratio of the number of patients whose GCS score or NIHSS score at discharge is equal to or better than that at admission to the total number of patients with cerebral hemorrhage in the same period of time."

[0045] Enter the calculation formula:

[0046]

[0047] Output: "Fact 1: admission GCS / NIHSS score; Fact 2: discharge GCS / NIHSS score; Logical rule: if GCS score, then the discharge score is greater than the admission score; if NIHSS score, then the discharge score is less than the admission score"

[0048] S23. Use the sentence-transformer algorithm to calculate the matching score between each fact in the indicator definition I and each field content in the electronic medical record, and select the one with the highest score as the electronic medical record field. The sentence-transformer algorithm embeds sentences, paragraphs or documents into a high-dimensional space to calculate the semantic similarity between the indicator definition and the electronic medical record content. Taking the improvement / stability rate at discharge as an example, the relevant fields are as follows:

[0049] Field 1: "Admission Record-Undergraduate Examination"

[0050] Field 2: "Discharge Summary-Discharge Condition"

[0051] S24. Use vector matching technology to retrieve the top 5 segments most relevant to the indicator definition from quality control documents and medical books as relevant knowledge. Taking the improvement / stability rate at discharge as an example, the relevant segments are as follows:

[0052] Clip 1: "The Glasgow Coma Scale (GCS) is used to assess the level of consciousness, with scores ranging from 3 to 15, with lower scores indicating more severe damage; the National Institutes of Health Stroke Scale (NIHSS) is used to assess the neurological function of stroke patients, with scores ranging from 0 to 42, with higher scores indicating more severe neurological damage."

[0053] Further, step S3 includes:

[0054] S31. First, based on the electronic medical record field of the standardized language I, the input medical record can be confirmed. If the medical record field does not exist, the method in S23 needs to be used again to match the field content to confirm the medical record content input this time.

[0055] "Admission record - undergraduate examination": "Somnolence, autonomous posture, motor and sensory aphasia, poor cooperation in physical examination, normal head shape, both pupils of equal size and roundness, 3mm in diameter, sensitive to light reflex, no normal eyeballs, no same-direction gaze, normal facial features, soft neck, limb muscle strength grade V, low muscle tone, negative bilateral Babinski sign, GCS 13 points."

[0056] "Discharge Summary - Discharge Status": "The patient is now in a stable general condition. Physical examination: clear consciousness, answers basically relevant questions, pupils of equal size and roundness, 3mm in diameter, sensitive to light reflection, limb muscle strength grade V, low muscle tension, bilateral Babinski sign negative, GCS 15 points."

[0057] S32. Next, based on the fact set of the normalized language I, the large model is used to perform fact judgment on the medical record content confirmed in S31 to obtain a fact judgment set;

[0058] {Input 1: You are a professional doctor. Please tell me whether there is a GCS score or NIHSS score in {admission record-undergraduate examination}. If so, please output the specific score.

[0059] Fact judgment 1: GCS score is 13 points

[0060] Input 2: You are a professional doctor. Please tell me whether there is a GCS score or NIHSS score in {discharge record-discharge status}. If so, please output the specific score.

[0061] Fact judgment 2: GCS score is 15 points}

[0062] S33. Finally, the logical rules in the normalized language I are used to infer the fact judgment set in S32 to obtain the final result. {Judgment result: the patient's GCS score at discharge is greater than the GCS score at admission}

[0063] Further, step S4 includes:

[0064] S41. First, construct an evaluation test set based on the expert-annotated answers. The results of a single test data include two dimensions: fact judgment and final answer judgment.

[0065] Expert answer: Based on the electronic medical records provided, the following information can be observed:

[0066] GCS score on admission: 13

[0067] GCS score at discharge: 15

[0068] Analysis of the reasons:

[0069] The GCS score was 13 on admission and 15 on discharge. According to the scoring criteria, an increase in the GCS score indicates an improvement in the patient's level of consciousness.

[0070] In summary, the patient's GCS score improved.

[0071] Final judgment result: The medical record meets the requirements

[0072] Fact judgment:

[0073] [Fact 1: GCS score was 13 at admission, Fact 2: GCS score was 15 at discharge]

[0074] Final answer: This medical record meets the requirements;

[0075] S42, then, similar to step S32, use the large model to perform fact judgment on the medical record content, and match the result with the fact judgment annotated answer;

[0076] S43. Finally, the rule engine is used to perform logical reasoning on the fact judgment results in S42 to obtain the final answer, and compared with the standard answer, and the indicator calculation results are optimized by iterating S2 and S3;

[0077] In the above, the specific embodiments of the present invention are described with reference to the accompanying drawings. However, those skilled in the art will appreciate that various changes and substitutions may be made to the specific embodiments of the present invention without departing from the spirit and scope of the present invention. These changes and substitutions are all within the scope defined by the claims of the present invention.

Claims

1. A method for automatically calculating medical quality control indicators based on a large model, characterized in that: The specific steps are: S1. Obtain the original electronic medical record data, perform desensitization and deduplication, save it in JSON format according to the medical record id, document name, and field name, and obtain the electronic medical record set to be calculated; S2. Obtain the indicators to be calculated, and based on the big model, expand the original indicator definition into a standardized language I through fact logic splitting, field positioning, and knowledge enhancement methods; S3. Based on the standardized language I, use the big model and rule engine to calculate the results of the corresponding indicators for each electronic medical record.

2. The method for automatically calculating medical quality control indicators based on a large model according to claim 1, characterized in that: The step S2 specifically includes: S21. Set the original definition of the indicator as I = <definition, calculation formula>, expand the original definition of the indicator to obtain the standardized language I = <definition, calculation formula, facts, logical rules, electronic medical record fields, related knowledge>, Among them, facts are used to represent the judgment items involved in the original definition, logical rules are used to integrate and reason various judgment items, electronic medical record fields represent fields associated with the original indicator definition, and related knowledge is used to represent the related supplementary knowledge of indicator I; S22, construct relevant examples, and implement logical splitting of the originally defined facts based on ICL to obtain a fact set and a logic rule set; S23, using the sentence-transformer algorithm to calculate the matching score between the indicator definition I and the content of each field in the electronic medical record, and selecting the one with the highest score as the electronic medical record field, wherein the sentence-transformer algorithm embeds sentences, paragraphs or documents into a high-dimensional space to calculate the semantic similarity between the indicator definition and the content of the electronic medical record; S24. Use vector matching technology to retrieve the top 5 fragments most relevant to the indicator definition from quality control documents and medical books as relevant knowledge.

3. The method for automatically calculating medical quality control indicators based on a large model according to claim 1, characterized in that: The step S3 specifically includes: S31, based on the electronic medical record field of the standardized language I, obtain the input medical record text content. If the medical record field does not exist, it is necessary to re-use the method in S23 to match the field content, so as to confirm the medical record content input this time; S32, based on the fact set of the normalized language I, using the big model to make fact judgments on the medical record content confirmed in S31, and obtaining a fact judgment set; S33. Use the logical rules in the normalized language I to infer and judge the fact judgment set in S32 to obtain the final result.

4. The method for automatically calculating medical quality control indicators based on a large model according to claim 1, characterized in that: S3 also includes step S4, S4. Construct an evaluation dataset based on expert-annotated data and use a large model to achieve automated evaluation.

5. The method for automatically calculating medical quality control indicators based on a large model according to claim 4, characterized in that: The step S4 specifically includes: S41. Construct an evaluation test set based on the expert-annotated answers. The results of a single test data include two dimensions: fact judgment and final answer judgment. S42. Use the big model to make factual judgments on the medical record content, and match the results with the annotated answers to the factual judgments; S43. Use the rule engine to perform logical reasoning on the fact judgment results in S42 to obtain the final answer, and compare it with the standard answer. Optimize the indicator calculation results by iterating S2 and S3.

Images