Medical document quality evaluation method, device and system based on large language model

Abstract

The invention discloses a medical document quality evaluation method based on a large language model, which comprises the following steps: acquiring medical document data, and generating medical document classification structured data; configuring a medical document quality evaluation rule according to the medical document quality evaluation method; and constructing a large language model input item by using the medical document classification structured data and the medical document quality evaluation rule, inputting the large language model input item into a large language model for processing, and outputting a medical document quality evaluation result. The invention further discloses a medical document quality evaluation system and device based on the large language model. The method has the following prominent effects: full-amount quality evaluation becomes possible, the method can adapt to expansion and development of the medical document, and quality monitoring of the medical document is changed into prevention in advance from afterward responsibility investigation; the large language model is applied to medical document quality control, the efficiency is greatly improved, and even quality evaluation can be carried out on past historical archives.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of large language models and medical record sorting, in particular to a medical document quality evaluation method, device and system based on a large language model. BACKGROUND

[0002] Recently, natural language (NLP) technology has developed rapidly, from the earliest BERT model to various Transformer models, which has promoted the development of language models, especially in 2022, with the release of ChatGPT (GPT3.5) by the American OpenAI company, large language models have entered the practical use stage.

[0003] The breakthrough of large language models in natural language processing lies in semantic understanding. Existing large language models can accurately understand the key content in a sentence or a paragraph, especially under precise prompts, and have language content understanding capabilities beyond humans. Large language models have the ability to think in chains (Chain of Thinking), which can continuously interact with inputs and adjust outputs based on inputs to master and judge complex language passages. Large language models have the ability to learn in context (in-Context Learning), which can learn and understand based on the content of the context to adjust the output and follow the input to adjust the content of the output. Large language models have the ability to fine-tune (Fine-Tuning), which can input professional domain data to the large language model to improve the professional knowledge and expression ability of the large language model in the medical professional field.

[0004] Medical document quality evaluation is currently a focus of hospitals. The quality of medical document filling reflects the patient's condition and directly reflects the hospital's medical quality, technical level and management level. High-quality medical documents provide valuable basic data for medical, scientific research and teaching, and are also an important basis for determining legal responsibility when medical disputes arise.

[0005] However, there are still many challenges in current medical document quality evaluation, which mainly include the following. First, in order to record information completely, although the information recording structure, process and filling requirements have become more and more standardized in recent years, in order to improve the efficiency of doctors, the content of the specific filling is still not completely structured and there are still many Chinese colloquial sentences and descriptions. Second, the existing medical document quality evaluation system follows the comparison search of structured data to make judgments, which means that the unstructured part of the content needs to be structured through traditional technical means, which puts high requirements on the flexibility and scalability of the system, and with the increase of data, the accuracy of quality control will be reduced.

[0006] A Chinese invention patent application with publication number CN11663162A discloses a method, device and electronic equipment for generating an electronic medical record discharge record. The method includes constructing an electronic medical record discharge record rule template, the electronic medical record discharge record rule template including admission and discharge information and treatment history; obtaining electronic medical record data; generating admission and discharge information based on the electronic medical record data and the discharge record rule template; and outputting treatment history based on the electronic medical record data and a pre-trained medical large language model. By constructing an efficient and accurate rule template, patient information is effectively extracted. At the same time, the technology based on the large language model generation is used to summarize and process the treatment history of multiple days. This technical solution can take into account rule templates and model generation to provide higher quality text, improve readability, and better meet the needs of the medical field. Although the above technical solution uses a large language model to assist in generating an electronic medical record discharge record, it cannot solve the current urgent medical document quality evaluation technical problem. SUMMARY

[0007] The purpose of the present application is to address the deficiencies of the prior art described above, and to provide a medical document quality evaluation method based on a large language model. Based on the assistance of a large language model, it is used for hospital medical document record quality inspection and control, which can improve the efficiency and accuracy of medical document quality evaluation, so that medical document quality monitoring can be fully implemented in hospitals, improve the filling level of hospital medical documents, and reduce medical disputes and legal risks caused by medical document quality. In addition, the natural language ability of the large language model itself makes it possible to improve the level of medical document quality evaluation through artificial intelligence.

[0008] The other two purposes of the present application are to provide a medical document quality evaluation device and system based on a large language model.

[0009] To achieve the above purpose, the present application is implemented as follows: a medical document quality evaluation method based on a large language model, which includes the following steps:

[0010] (1) Obtain medical document data and generate medical document classification structured data;

[0011] (2) Configure medical document quality evaluation rules according to the medical document quality evaluation method;

[0012] (3) Construct large language model input items (Prompts set) from medical document classification structured data and medical document quality evaluation rules;

[0013] (4) Input the large language model input items (Prompts set) into the large language model for processing, and the large language model outputs the medical document quality evaluation result.

[0014] When the medical document quality evaluation result does not meet the medical document quality evaluation method standard, the relevant person in charge is informed to modify the medical document in the hospital HIS system (hospital management information system); when the medical document quality evaluation result meets the medical document quality evaluation method standard, the evaluated medical record data is archived according to the medical document quality classification.

[0015] The medical document data is desensitized medical document data in the hospital HIS system. Data desensitization includes hiding patient personal information, hiding doctor information, etc., to avoid data leakage leading to adverse consequences or affecting national security.

[0016] Among them, the method for generating medical document classification structured data in the (1) step includes the following steps:

[0017] (1) obtaining desensitized medical document data;

[0018] (2) segmenting the medical document data;

[0019] (3) obtaining medical document content classification data;

[0020] (4) extracting medical document categories in the medical document content classification data;

[0021] (5) constructing large language model input items (Prompts set) with segmented medical document data and medical document categories

[0022] (6) inputting the large language model input items (Prompts set) into the large language model for processing and outputting the generated medical document classification structured data.

[0023] The large language model is a large language model with more than 100B parameters, including but not limited to GPT4, GPT3.5, Baidu Wenxin, ChatGLM Pro, MiniMax, ChatGLM2-6B and Qwen7B, etc.

[0024] The medical document quality evaluation method uses the existing medical document scoring standard of the hospital.

[0025] The medical document quality evaluation system based on the large language model includes a medical document data input module, a medical document quality evaluation rule configuration module and a medical document quality evaluation module,

[0026] The medical document data input module is used to obtain medical document data and generate medical document classification structured data;

[0027] The medical document quality evaluation rule configuration module is configured to generate medical document quality evaluation rules according to a medical document quality evaluation method, and the medical document quality evaluation rules form large language model input structured data with medical document classification structured data;

[0028] The medical document quality evaluation module is configured to input the large language model input items (Prompt collection) constructed by the large language model input structured data formed by the medical document quality evaluation rule configuration module into a large language model processing, and output a medical document quality evaluation result.

[0029] When the medical document quality evaluation result does not meet the medical document quality evaluation method standard, the relevant person in charge is notified to modify the medical document in the hospital HIS system (hospital management information system); when the medical document quality evaluation result meets the medical document quality evaluation method standard, the evaluated medical record data is archived according to the medical document quality classification.

[0030] The medical document data input module includes a medical document database and a medical document content classification library, and the medical document database is segmented and constructed with the medical content categories in the medical document content classification library to form large language model input items (Prompt collection) input to a large language model processing, and output medical document classification structured data.

[0031] A medical document quality evaluation device based on a large language model, comprising a processor, a memory, a computer program stored on the memory and executable on the processor, and medical document classification structured data, wherein the computer program is executed by the processor to implement the steps of the medical document quality evaluation method based on the large language model.

[0032] The application of a large language model in the field of medical document quality evaluation is to expect that the large language model can understand the unstructured content, i.e., the part of the content written by doctors in the medical document, and then rate and score the medical document record according to the quality control requirements, thereby solving the most difficult part of the traditional quality control system. Through the understanding of the input content by the large language model, the requirements of the input content are understood, and all the content points that need to be understood are extracted. Here, it mainly refers to the understanding of quality evaluation requirements.

[0033] According to the understanding result of the input content by the large language model, the output content is cross-checked, and the evaluation is made according to the requirements. Here, it mainly refers to the understanding and evaluation of the content that needs to be checked according to the quality evaluation requirements, and the output of the content according to the output requirements. Through the overall statistics and evaluation of all output results (evaluation results) by the large language model, the final quality control check result is finally output.

[0034] The large language model has good scalability for the length of the input rule. Different sizes of the large language model have corresponding accuracy for understanding content of different input lengths. The size of the model and the input length need to be balanced to consider performance and accuracy. The large model has good scalability for the length of the input content required for quality control inspection. The more granular the input content, the higher the accuracy of understanding.

[0035] It is a feasible and efficient method to check the quality of medical document records by using the large language model to check the quality of medical document records according to the quality control rules. On the one hand, it can solve the evaluation and processing problem of unstructured text data. On the other hand, it can use the ability of the large language model to solve the problem of evaluation content flexibility. In addition, the emergence of the emergence ability of the large language model brings a new idea to the medical document quality evaluation, which can greatly improve the operation efficiency and accuracy of the whole medical document quality evaluation system, and truly realize the leap of medical document quality evaluation.

[0036] Compared with the prior art, the present application has the following outstanding effects: it makes full quality evaluation possible, and can adapt to the expansion and development of medical documents, and changes the quality monitoring of medical documents from ex post facto accountability to ex ante prevention; application of the large language model for medical document quality control greatly improves the efficiency, and even can evaluate the quality of past historical archives. BRIEF DESCRIPTION OF DRAWINGS

[0037] Figure 1 A flowchart of the medical document quality evaluation method based on the large language model of the present application;

[0038] Figure 2 A system block diagram of the medical document quality evaluation system based on the large language model of the present application;

[0039] Figure 3 A content screenshot of A;

[0040] Figure 4 A content screenshot of B;

[0041] Figure 5 A content screenshot of C;

[0042] Figure 6 A flowchart of the large language model for hospital medical document data structuring and traversal. DETAILED DESCRIPTION

[0043] The present application will be further described below in conjunction with specific embodiments. It should be noted that the embodiments do not constitute a limitation on the scope of protection required by the present application.

[0044] The application mainly utilizes the characteristics of large language models in natural language understanding to realize quality inspection and evaluation of medical document records, making the medical document quality evaluation system a reality, and improving the accuracy and efficiency of medical document quality evaluation.

[0045] The medical document quality evaluation system based on a large language model, as shown in Figures 2-5 includes a medical document data input module, a medical document quality evaluation rule configuration module, and a medical document quality evaluation module,

[0046] The medical document data input module is used to obtain desensitized medical document data and generate medical document classification structured data;

[0047] The medical document quality evaluation rule configuration module is used to generate medical document quality evaluation rules according to a medical document quality evaluation method through initialization input, and the medical document quality evaluation rules form large language model input structured data with the medical document classification structured data; including medical document classification structured data {1}, such as past history: surgical history; medical document classification structured data {2}, such as physical examination: abdominal examination; evaluation rules ({1} and {2} do not match), ({1}xx description is unclear) ({1} is missing), etc.

[0048] The medical document quality evaluation module is used to input the large language model input item (Prompts set) constructed by the large language model input structured data formed by the medical document quality evaluation rule configuration module into a large language model processing, and output a medical document quality evaluation result; wherein the Prompts set can be set as: please compare {medical document data {1}} and {medical document data {2}} according to {evaluation rule}, and the output format is {score output rule}.

[0049] When the medical document quality evaluation result does not meet the scoring standard of the medical document quality evaluation method as shown in Figure 3 , the relevant person in charge is notified to modify the medical document in the hospital HIS system (hospital management information system); when the medical document quality evaluation result meets the scoring standard of the medical document quality evaluation method, the evaluated medical record data is filed according to the medical document quality classification. The scoring standard of the medical document quality evaluation method uses the current medical document scoring standard of the hospital.

[0050] The medical document data input module includes a medical document database and a medical document content classification library. The medical document database stores medical document data after segmentation (for example, segmented into past medical history) and medical content category data in the medical document content classification library to construct large language model input items (Prompts set) input to the large language model processing. After large language model processing, the medical document classification structured data is output. The medical document database, as shown in Figure 4 , is used to store medical document data such as medical history records. The medical document content classification library is used to store medical document categories such as specific medical classifications such as surgical history.

[0051] The medical document classification structured data, as shown in Figure 5 , can be structured data information classified by categories such as surgical history, allergy history, mathematical history, infectious disease history, vaccination history, and trauma history. The medical document quality evaluation method based on the large language model includes the following steps: Figure 1

[0052] (1) Obtain medical document data and generate medical document classification structured data;

[0053] (2) Configure medical document quality evaluation rules according to the medical document quality evaluation method;

[0054] (3) Construct large language model input items (Prompts set) from medical document classification structured data and medical document quality evaluation rules;

[0055] (3) Input the large language model input items (Prompts set) into the large language model processing, and the large language model outputs the medical document quality evaluation results.

[0056] When the medical document quality evaluation results do not meet the medical document quality evaluation method standards, notify the responsible person to modify the medical documents in the hospital HIS system (hospital management information system); when the medical document quality evaluation results meet the medical document quality evaluation method standards, the evaluated medical record data is archived according to the medical document quality classification.

[0057] The medical document data is desensitized medical document data in the hospital HIS system. Data desensitization includes hiding patient personal information, hiding doctor information, etc., to avoid data leakage leading to adverse consequences or affecting national security.

[0058] The method for generating medical document classification data matrix in the (1) step includes the following steps:

[0059] (1) Obtain medical document data;

[0060] ​(2) Segmentation of medical document data;

[0061] (3) Obtain medical document content classification data;

[0062] (4) Extract medical document categories in medical document content classification data;

[0063] (5) Construct large language model input items (Prompts set) with segmented medical document data and medical document categories

[0064] (6) Input large language model input items (Prompts set) into large language model processing, and output generated medical document classification structured data.

[0065] Regarding the segmentation of medical document data, taking the segmentation of medical record data as an example. According to the desensitization medical document data retrieved from the hospital HIS system, the medical document data is segmented according to the input content length that can be processed by the large model. The segmentation principle is according to the first-level classification of medical record data. If the length of the first-level classification data exceeds the required length of the large model, the classification is divided into the second-level classification.

[0066] According to the medical record data category library, combine the medical document data (medical record data) to construct the input Prompt set of the large language model. The medical record data category library outputs the second (or third) category data for the medical record data block.

[0067] Among them, the medical record data categories include past history. The past history includes: general disease history, operation history, allergy history, blood transfusion history, infectious disease history, preventive vaccination history, etc.

[0068] The medical document database outputs the "past history" data block.

[0069] Among them, the past history: there is a "hypertension" history, regular use of bisoprolol + metoprolol to control, no history of diabetes, no history of heart disease, no history of kidney disease, no history of infectious diseases such as "hepatitis, tuberculosis", no history of disease trauma, history of blood transfusion, no history of surgery, no history of allergy, no history of infectious disease contact, and preventive vaccination history is unknown. The Prompt set of the large language model can be as follows: Please segment the following past history content according to the general disease history, operation history, allergy history, blood transfusion history, infectious disease history, and preventive vaccination history. If the classification is not mentioned, it can be added: Past history: there is a "hypertension" history, regular use of bisoprolol + metoprolol to control, no history of diabetes, no history of heart disease, no history of kidney disease, no history of infectious diseases such as "hepatitis, tuberculosis", no history of disease trauma, history of blood transfusion, no history of surgery, no history of allergy, no history of infectious disease contact, and preventive vaccination history is unknown.

[0070] The output obtained in this way is as follows:

[0071] The patient's history is as follows:

[0072] General disease history: There is a history of "hypertension", regular use of bisoprolol + metoprolol for control, no history of diabetes, no history of heart disease, no history of kidney disease.

[0073] Operation history: No operation history.

[0074] Allergy history: No allergy history.

[0075] Blood transfusion history: There is a history of blood transfusion.

[0076] Infectious disease history: No history of "hepatitis, tuberculosis" and other infectious diseases, no history of infectious disease contact.

[0077] Preventive vaccination history: The preventive vaccination history is unknown.

[0078] Trauma history: No disease trauma history.

[0079] Output format of Prompt set

[0080] In order to better use the system, the output format of the Prompt set can also be defined, so the output format is added to the above Prompt set: Please divide the following history content into general disease history, operation history, allergy history, blood transfusion history, infectious disease history, preventive vaccination history, etc. If the classification is not mentioned, add it, and output in the form of jason: History: There is a history of "hypertension", regular use of bisoprolol + metoprolol for control, no history of diabetes, no history of heart disease, no history of kidney disease, no history of "hepatitis, tuberculosis" and other infectious diseases, no disease trauma history, history of blood transfusion, no operation history, no allergy history, no history of infectious disease contact, preventive vaccination history unknown.

[0081] The output is as follows:

[0082] {

[0083] "General disease history": "There is a history of "hypertension", regular use of bisoprolol + metoprolol for control, no history of diabetes, no history of heart disease, no history of kidney disease",

[0084] "Operation history": "No operation history",

[0085] "Allergy history": "No allergy history",

[0086] "Blood transfusion history": "There is a history of blood transfusion",

[0087] "Infectious disease history": "No history of "hepatitis, tuberculosis" and other infectious diseases, no history of infectious disease contact",

[0088] "Preventive vaccination history": "Preventive vaccination history is unknown",

[0089] "History of trauma": "Deny history of trauma"

[0090] }。

[0091] The above medical document classification structured data can be stored in a database and provided to the subsequent medical document quality evaluation call.

[0092] Regarding the medical document quality evaluation rules, there are mainly two aspects of requirements:

[0093] (1) The quality requirements for the content of a single medical document category.

[0094] For example: "Admission record lacks physician signature", where the single medical document category content is "admission record", and the quality requirement is "physician signature".

[0095] (2) The quality requirements for different medical document subcategory content.

[0096] For example: "Present history and chief complaint do not match", which involves two medical document subcategories "present history" and "chief complaint", and the quality requirement is "content consistency".

[0097] According to the two quality requirements, an extensible quality evaluation method based on large language model can be established:

[0098] Content integrity evaluation: physician signature, postoperative ward round, postoperative record, etc.

[0099] Content consistency evaluation: content consistency, drug conflict, etc.

[0100] According to the quality evaluation method and medical document record classification data, the medical document evaluation rules can be configured, mainly according to the medical document quality evaluation standard, to define the data of the input Prompt set of the large language model and the output rules.

[0101] The medical document classification data includes the data block of the most detailed category, for example, in the admission record, it will include the present history, past history, etc. According to the actual situation, it can be further subdivided, for example, the past history includes the history of important organ disease, surgery, infectious disease, blood transfusion, drug allergy, etc. Each of them can become a medical document classification data block.

[0102] The evaluation rule configuration is to configure the input and output Prompt set rules of the large language model according to the evaluation model:

[0103] For example, configure the classification data block "chief complaint" and the evaluation rule "physician signature" as a rule; configure the classification data block "surgery history" and "abdominal examination" and the evaluation rule "content consistency" as a rule.

[0104] The input data consists of medical document record classification structured data and medical document evaluation rules, and the composed data forms an input prompt (Prompt set) for a large language model.

[0105] This is a Prompt set sequence, which is based on each configured medical document quality evaluation rule, traverses the corresponding medical document record related classification data, and forms a Prompt set. The composition includes the following two:

[0106] (1) Classification structured data {1} + evaluation rule + output rule

[0107] (2) Classification structured data {1} + classification structured data {2} + output rule

[0108] According to the traversal rule + traversal medical document record classification matrix, all Prompt sets are obtained, as shown in the following table. Figure 6

[0109] The specific implementation is as follows:

[0110] Classification structured data {1}:

[0111] Past history: There is a history of "hypertension", regular use of Baisu Tong + Beteleke to control, no history of diabetes, no history of heart disease, no history of kidney disease, no history of infectious diseases such as "hepatitis, tuberculosis", no history of disease trauma, history of blood transfusion, no history of surgery, no history of allergy, no history of infectious disease contact, and vaccination history is unknown.

[0112] Classification structured data {2}:

[0113] Auxiliary examination: On May 16, 2023, the chest and upper abdomen CT scan + enhancement + three-dimensional reconstruction of the South Medical University Zhujiang Hospital showed: 1. Pulmonary emphysema, right lower lobe posterior basal segment chronic inflammation decreased, coronary artery, aortic sclerosis, scan thyroid density uneven, recommended specialist examination, 2. Liver right lobe - hepatic portal area mass, multiple small nodules with abnormal enhancement. The disease invades the common hepatic duct, cystic duct and right branch of the portal vein, the above changes change little, the intrahepatic bile duct is not uniform, the left hepatic duct - common bile duct stent retention: the hepatic portal area and retroperitoneal lymph nodes are slightly enlarged, cholecystitis, multiple gallstones in the gallbladder, cholecystic chole, please combine with the end and other examinations, 3. The scan shows a slightly low enhancement area in the left kidney, the right kidney is slightly smaller, the nature is pending, still recommended for review, a little exudation around the kidneys, local colon hepatic area wall thickening.

[0114] Evaluation rule: Whether the operation history in the past history is consistent with the content in the auxiliary examination, deduct 1 point if there is inconsistency, and deduct 0 point if there is no inconsistency. ​

[0115] Derive Prompt set:

[0116] Patient's history: The patient has a history of "hypertension", regularly takes bisoprolol + metoprolol to control, denies diabetes history, denies heart disease history, denies kidney disease history, denies "hepatitis, tuberculosis" and other infectious disease history, denies disease trauma history, has a history of blood transfusion, denies surgery history, has no allergy history, denies infectious disease contact history, and vaccination history is unknown.

[0117] Patient's auxiliary examination: On May 16, 2023, the chest and upper abdomen of the patient were scanned by CT, and the results showed that 1. Pulmonary emphysema, chronic inflammation in the posterior basal segment of the right lower lobe decreased, coronary artery and aorta were sclerotic, thyroid density was uneven, and specialist examination was recommended, 2. Right lobe of liver - hepatic portal area mass with multiple small nodules and abnormal enhancement. The changes in the above changes were not much different, the intrahepatic bile duct was not uniform, the left hepatic duct - common bile duct stent was retained: the hepatic portal area and retroperitoneal lymph nodes were slightly enlarged, cholecystitis, multiple gallstones in the gallbladder, and cholecystic bile stasis were found, and the final examination and other examinations were recommended, 3. The slightly low enhancement area in the left kidney was not found, the slightly low enhancement area in the right kidney was slightly smaller, the nature was pending, and reexamination was still recommended, and there was a little exudation around the kidneys, and the colon hepatic area was locally thickened.

[0118] Please find the inconsistencies in the above two segments of the patient's surgery history and auxiliary examination, and list them out. If there is no inconsistency, it does not need to be listed, and no other content needs to be mentioned. The final output score is 1 point if there is inconsistency, and 0 points if there is no inconsistency.

[0119] Regarding the quality assessment of medical documents, the prompts set list obtained in the previous step is traversed, and the large language model is input. Each Prompt set input produces a result. When all prompt sets have been output, the results are integrated and input into the large language model.

[0120] The specific implementation is as follows:

[0121] According to the example Prompt set, the language model output result is as follows:

[0122] In the history, the patient denies surgery history, but in the auxiliary examination, it mentions "left hepatic duct - common bile duct stent retention", which is a surgical operation, so there is inconsistency here.

[0123] The final scoring result is 1 point.

[0124] The large language model calculates the quality assessment score of the current case based on the integrated input results, and judges the level of the current medical document record.

[0125] At the same time, if there is a problem, feedback data to the relevant personnel of the hospital HIS system, and archive at the same time.

[0126] The large language model in the present application can effectively perform the above evaluation process with a mainstream large language model with more than 100B parameters, and the evaluation result is reliable. Including GPT4, GPT3.5, Baidu Wenxin, ChatGLM Pro, MiniMax, all can effectively perform the above evaluation process. In addition, ChatGLM2-6B and Qwen7B need further verification, and there is still a certain illusion condition.

[0127] Although the model with more than 100B parameters can meet the demand of semantic understanding. For large language models with less than 100B parameters, feasibility can be achieved through fine-tuning. Fine-tuning is to input a large amount of medical document data (cleaned into question and answer form) into the large language model for operation and fine-tuning. After fine-tuning, it can be tested whether the large language model meets the requirements of medical document semantic understanding. This is only a suggestion, and it is still preferred to use a large language model with more than 1000B parameters.

[0128] The large language model can also be externally connected to the medical industry vector database to further meet the data processing needs of the medical industry.

[0129] Before applying the large language model to medical document quality evaluation, the traditional quality evaluation system is difficult to conduct full-quality quality inspection on medical documents, and more is sampling inspection, and the adaptability to the expansion of medical documents is poor, and often only after a medical accident. After the application of the large language model for medical document quality evaluation, full-quality quality evaluation becomes possible, and can adapt to the expansion and development of medical documents, and change the quality monitoring of medical documents from post-factum accountability to prior prevention. In addition, the application of the large language model for medical document quality control greatly improves the efficiency, and even the quality of the past historical archives can be evaluated.

[0130] The above is only a preferred embodiment of the present application, which cannot limit the scope of the present application, that is, any simple equivalent change and modification made according to the scope of the present application and the content of the present application is still within the scope of the present application.

Claims

1. A method for medical document quality assessment based on a large language model, characterized in that, It includes the following steps: (1) obtaining medical document data, generating medical document classification structured data; (2) according to the medical document quality evaluation method, configure medical document quality evaluation rules; (3) constructing large language model input items from medical document classification structured data and medical document quality evaluation rules; (4) input the large language model input item into the large language model processing, and the large language model outputs the medical document quality evaluation result. 2.The medical document quality evaluation method based on a large language model according to claim 1, wherein, When the medical document quality evaluation result does not meet the standard of the medical document quality evaluation method, inform the relevant person in charge to modify the medical document in the hospital HIS system; when the medical document quality evaluation result meets the standard of the medical document quality evaluation method, the evaluated medical record data is filed according to the medical document quality classification. 3.The medical document quality evaluation method based on a large language model according to claim 1, wherein, The medical document data is desensitized medical document data desensitized in the hospital HIS system. 4.The medical document quality evaluation method based on a large language model according to claim 1, wherein, In the step (1), the method for generating medical document classification structured data includes the following steps: (1) obtaining desensitized medical document data; (2) segmenting the medical document data; (3) obtaining medical document content classification data; (4) extracting medical document categories in medical document content classification data; (5) constructing large language model input items from segmented medical document data and medical document categories; (6) input the large language model input item into the large language model processing, and output the generated medical document classification structured data. 5.The medical document quality evaluation method based on a large language model according to claim 1, wherein, The large language model is a large language model with more than 100B parameters. 6.The medical document quality evaluation method based on a large language model according to claim 5, wherein, The large language model includes but is not limited to GPT4, GPT3.5, Baidu Wenxin, ChatGLM Pro, MiniMax, ChatGLM2-6B and Qwen7B.

7. A medical document quality evaluation system based on a large language model, characterized by, It includes a medical document data input module, a medical document quality evaluation rule configuration module and a medical document quality evaluation module, The medical document data input module is used to obtain medical document data and generate medical document classification structured data; The medical document quality evaluation rule configuration module is used to generate medical document quality evaluation rules according to the medical document quality evaluation method, and the medical document quality evaluation rules form large language model input structured data with medical document classification structured data; The medical document quality evaluation module is used to input the large language model input item constructed by the large language model input structured data formed by the medical document quality evaluation rule configuration module into the large language model processing, and output the medical document quality evaluation result. 8.The medical document quality evaluation system based on a large language model according to claim 7, wherein, The medical document data input module includes a medical document database and a medical document content classification library, and the segmented medical document database and the medical document content classification library in the medical document content classification library construct large language model input items input into the large language model processing, and output medical document classification structured data.

9. A medical document quality evaluation device based on a large language model, characterized by A computer program product comprising a processor, a memory, and a computer program stored on the memory and executable on the processor, and comprising medical document classification structured data, the computer program, when executed by the processor, implements the steps of the method for medical document quality assessment based on a large language model according to any one of claims 1 to 4.

Images