A medical record connotation quality control method and device based on a large language model

Abstract

The present application relates to the field of medical informatization and artificial intelligence technology, and specifically provides a medical record connotation quality control method and device based on a large language model, which has the following steps: S1, medical record document data access, document content preprocessing; S2, clear quality control content, construct quality control rule library; S3, construct medical knowledge base: based on clinical guidelines and evidence-based medicine evidence base, establish diagnosis standard, drug, treatment project, inspection project knowledge base; S4, entity extraction: use a large language model to extract diagnosis, drug, treatment project and inspection project name; S5, model reasoning: query the relevant information of the extracted entity in the knowledge base, according to the quality control rules, combine the queried relevant information with the medical document and submit them to the large model for reasoning to give the quality control result; S6, problem collection, model optimization. Compared with the prior art, the present application can significantly improve the intelligent level of medical record connotation quality control, effectively assist clinical doctors and quality control personnel to improve the quality of medical record writing, ensure medical safety, and improve hospital management efficiency.

Description

Technical Field

[0001] This invention relates to the fields of medical informatics and artificial intelligence, specifically providing a method and apparatus for quality control of medical record content based on a large language model. Background Technology

[0002] Electronic medical records (EMRs) have been widely adopted in medical institutions at all levels, becoming an important data foundation for clinical diagnosis and treatment, scientific research analysis, and hospital management. However, current EMRs generally suffer from problems such as non-standard writing, illogical reasoning, missing key information, and inconsistent terminology, which seriously affect the clinical value and research usability of medical records and pose significant challenges to medical quality control and medical insurance review. Traditional medical record quality control methods mainly rely on manual sampling or rule-based automated verification. The former is inefficient and has limited coverage, while the latter struggles to cope with the complexity and diversity of clinical language, especially in terms of "intrinsic quality"—such as the logical consistency between diagnosis and medical history, the conformity of treatment plans with guidelines, and the completeness and accuracy of disease descriptions—lacking deep semantic understanding capabilities.

[0003] In recent years, large-scale pre-trained language models (LLMs), represented by the Transformer architecture, have demonstrated outstanding capabilities in natural language understanding, reasoning, and generation tasks. These models, pre-trained on massive amounts of medical text, can deeply understand complex semantic structures such as clinical terminology, disease evolution logic, and treatment pathways, providing a novel technical approach for the intelligent assessment of the quality of medical records. Compared to traditional methods, quality control systems based on large models can achieve contextual awareness, semantic consistency verification, potential contradiction identification, and clinical rationality inference of medical record content, thereby enhancing the depth and breadth of quality control at the level of "understanding medicine, logic, and standards."

[0004] Therefore, exploring and constructing a quality control method for medical records based on large models has become a key research direction for promoting smart healthcare, improving the quality of medical documents, and ensuring patient safety and medical insurance compliance. Summary of the Invention

[0005] This invention addresses the shortcomings of the prior art by providing a highly practical method for quality control of medical record content based on a large language model.

[0006] A further technical objective of this invention is to provide a reasonably designed, safe, and applicable medical record content quality control device based on a large language model.

[0007] The technical solution adopted by this invention to solve its technical problem is: A method for quality control of medical record content based on a large language model has the following steps: S1. Access to medical record documents and preprocessing of document content; S2. Define quality control content and build a quality control rule base: Based on treatment guidelines and hospital management systems, establish a quality control rule base for various types of medical documents, and refine the exclusive rules for each type of document according to the core quality control needs of different document types; S3. Construct a medical knowledge base: Based on clinical guidelines and evidence-based medicine, establish a knowledge base for diagnostic criteria, drugs, treatment items, and examination items; S4. Entity Extraction: Use a large language model to extract the names of diagnoses, drugs, treatments, and examinations. S5. Model Reasoning: Query and extract relevant information about entities from the knowledge base. According to the quality control rules, combine the queried relevant information with medical documents and submit them to the large model for reasoning to give the quality control results. S6. Problem Collection and Model Optimization: Design a quality control error collection table, analyze the causes of errors, and adjust the quality control rule base, knowledge base, and large model prompt words.

[0008] Furthermore, step S1 includes: S1-1, Access to medical record documents; After receiving structured medical record data, the medical record quality control system directly reads the "type" field value to identify the document type. S1-2, Document content preprocessing; Preprocessing of medical records involves desensitization and structuring; desensitization involves anonymizing sensitive fields in medical records that involve patient privacy. Document content structuring involves structuring unstructured text content into pre-defined standardized fields.

[0009] Furthermore, step S2 includes: S2-1. Define Quality Control Content: Before constructing quality control rules for different types of medical records, it is necessary to first define the quality control content for each type of document. The quality control content for medical records is as follows: Admission record: Reasonableness of diagnosis; Initial medical record: rationality of diagnosis, rationality of medication, rationality of treatment, rationality of examination; Routine course of illness: rationality of medication use, rationality of treatment, rationality of examinations; Phase Summary: Consistency of content, rationality of medication, rationality of treatment, rationality of examination; Discharge summary: consistency of content, rationality of medication, rationality of treatment, and rationality of examinations; S2-2, Constructing a quality control rule base: Based on clarifying the quality control content of various medical record documents, further construct a structured, configurable, and scalable quality control rule base to support intelligent and automated medical record quality control. The construction of the rule base follows the principles of "classification management and semantic drive". Categorization management refers to establishing a multi-dimensional rule-based classification system based on the type of medical records and quality control dimensions. Semantic-driven management refers to transforming quality control content into logical relationships by combining medical ontology with clinical guidelines.

[0010] Furthermore, in step S3, the knowledge base includes: domestic and international clinical diagnosis and treatment guidelines, drug databases, disease classification systems, laboratory test reference values, standards, hospital internal regulations, authoritative guidelines and standards.

[0011] Furthermore, in step S4, the open-source Qwen3-32B is used as the large language model for extracting medical entities. The model is guided by structured prompts to accurately identify and extract four types of core clinical entities from unstructured medical record text: diagnosis name, drug, treatment and examination item names. The response_format={“type”: “json_object”} parameter is forcibly enabled to ensure that the output is in a standardized JSON format. Drug entities are structured and split into name, dosage, frequency and route of administration, while standardized item names are extracted for treatment and examination items.

[0012] Furthermore, in step S5, the open-source Qwen3-32B is used as a large language model for quality control document problems, serving as the execution step for medical record quality control: Based on the structured extraction of four key entities—diagnosis name, drug, treatment item, and examination item name—the extracted entities are semantically matched and logically reasoned with a pre-set quality control rule base. The reasoning process is completed by the Qwen3-32B model in a single call. The model is guided by the Prompt function to combine the medical knowledge base context, trigger the corresponding quality control rules for each entity, and finally force the output of structured quality control results with response_format={“type”:“json_object”}.

[0013] Furthermore, in step S6, after each quality control inference is completed, the system automatically records the difference data between the model output results and the clinical physician's manual review feedback, and stores it in a structured manner in the "quality control problem sample library", annotating the error type, the document category, the entity type involved, and the rule ID; Based on the sample library, the Qwen3-32B model is regularly optimized using LoRA lightweight fine-tuning technology. The optimization process supports canary releases and A / B testing. Before the new model version is launched, the improvement in recall, precision, and physician adoption rate is automatically compared with the old version.

[0014] A medical record content quality control device based on a large language model includes: at least one memory and at least one processor; The at least one memory is used to store a machine-readable program; The at least one processor is used to call the machine-readable program to execute a medical record content quality control method based on a large language model.

[0015] Compared with existing technologies, the medical record content quality control method and apparatus based on a large language model of the present invention have the following outstanding advantages: This invention leverages the powerful semantic understanding capabilities of the Qwen3-32B large language model. The system can automatically and accurately extract four core entities—diagnosis, medication, treatment, and examination—from unstructured medical record texts. Based on a preset rule base, it completes intelligent reasoning in seconds, enabling automated review of massive medical records, multiple document types, and multiple quality control dimensions, thus overcoming the inefficiency and blind spots of the traditional manual sampling mode.

[0016] The model's reasoning process deeply integrates medical knowledge base and clinical guidelines semantics, which can not only identify "whether there is a record" but also judge "whether the record is reasonable", such as whether the diagnosis is based on evidence, whether there are contraindications to medication, and whether the examination is repetitive or redundant, effectively reducing false positives and false negatives caused by mechanical keyword matching.

[0017] Built upon the zero-sample / few-sample capabilities of Qwen3-32B, it eliminates the need for large-scale annotation and retraining, and can be deployed simply through Prompt projects and lightweight fine-tuning, significantly lowering the threshold for intelligent applications in hospitals. At the same time, the JSON structured output and modular design facilitate secondary development, adapting to different levels of informatization and helping to widely implement strategies for improving medical quality. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating a medical record content quality control method based on a large language model. Figure 2This is a schematic diagram of entity extraction in a medical record content quality control method based on a large language model; Figure 3 This is a schematic diagram of model reasoning in a medical record content quality control method based on a large language model. Detailed Implementation

[0020] To enable those skilled in the art to better understand the present invention, the present invention will be further described in detail below with reference to specific embodiments. Obviously, the described embodiments are merely some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] The following is a preferred embodiment: like Figure 1 As shown in this embodiment, a method for quality control of medical record content based on a large language model has the following steps: S1. Access to medical record documents and preprocessing of document content; include: S1-1, Medical Record Document Data Access: After receiving structured medical record data, the medical record quality control system directly reads the "type" field value to identify the document type. The quality control documents involved in this application include: admission records, initial progress notes, daily progress notes, interim summaries, and discharge records.

[0022] S1-2. Document Content Preprocessing: Preprocessing the document content of medical records, including anonymization and structuring. Document anonymization involves anonymizing sensitive fields in the medical records that involve patient privacy, including name and ID number. Document structuring follows the "Provincial Medical Record Writing and Management Standards," aligning unstructured text content into pre-defined standardized fields.

[0023] (1) The standardized fields of the admission record document include: Personal information: gender, age, place of birth, etc.

[0024] Present illness, past medical history, and family history.

[0025] Examination results: physical examination and auxiliary examinations.

[0026] Diagnosis: preliminary diagnosis, revised diagnosis, supplementary diagnosis.

[0027] (2) The standardized fields of the initial medical record include: Case characteristics, preliminary diagnosis discussion, and treatment plan; (3) The standardized fields for the phase summary include: Admission details, admission diagnosis, treatment process, current condition, current diagnosis and treatment plan; (4) The standardized fields in the discharge record include: Admission diagnosis, discharge diagnosis, admission status, treatment process, discharge status, and discharge instructions; S2. Define quality control content and build a quality control rule base: Based on treatment guidelines and hospital management systems, establish a quality control rule base for various types of medical documents, and refine the exclusive rules for each type of document according to the core quality control needs of different document types; include: S2-1. Define Quality Control Content: Before constructing quality control rules for different types of medical records, it is necessary to first define the quality control content for each type of document. The quality control content for medical records is as follows: (1) Admission record: Reasonableness of diagnosis.

[0028] (2) Initial medical record: rationality of diagnosis, rationality of medication, rationality of treatment, and rationality of examination.

[0029] (3) Routine course of illness: rationality of medication, rationality of treatment, rationality of examination.

[0030] (4) Phase summary: consistency of content, rationality of medication, rationality of treatment, rationality of examination.

[0031] (5) Discharge record: consistency of content, rationality of medication, rationality of treatment, and rationality of examination.

[0032] S2-2. Constructing a Quality Control Rule Base: Based on clarifying the quality control content of various types of medical records, this application further constructs a structured, configurable, and scalable quality control rule base to support intelligent and automated medical record quality control. The construction of the rule base follows the principles of "classification management and semantic-driven approach."

[0033] Categorization management refers to establishing a multi-dimensional rule-based classification system based on the type of medical record document (such as admission records, initial course of illness, etc.) and quality control dimensions (such as the rationality of diagnosis, the rationality of medication, etc.). Semantic-driven management refers to using a combination of medical ontologies (such as ICD-10, ATC coding systems) and clinical guidelines to transform quality control content into logical relationships. For example: Diagnostic rationality: Is the primary diagnosis logically consistent with the present medical history, physical signs, and auxiliary examination results? Rationality of drug use: Whether the drug selection conforms to the guidelines, whether there are contraindications, and whether the dosage exceeds the limit; Reasonableness of examination: Whether the examination items are related to the current condition and whether there are any contraindications; Treatment rationality: Is the treatment program relevant to the current condition, and are there any contraindications? Content consistency: Does the treatment process in the stage summary and discharge record correspond to each other, and are the descriptions consistent?

[0034] S3. Construct a medical knowledge base: Based on clinical guidelines and evidence-based medicine, establish a knowledge base for diagnostic criteria, drugs, treatment items, and examination items; After completing the structured design of the quality control rule base, a comprehensive and authoritative medical knowledge base will be built to achieve accurate rule matching, semantic understanding, and clinical rationality judgment. This knowledge base will not only serve as the "brain" for the execution of quality control rules, but also as a bridge connecting clinical guidelines, medical terminology, drug information, testing standards, and real-world diagnostic and treatment practices.

[0035] This knowledge base integrates authoritative medical resources, including: domestic and international clinical practice guidelines (such as NCCN, Chinese Medical Association series guidelines, UpToDate, BMJ Best Practice), drug databases (such as pharmacopoeia, FDA / EMA / NMPA approval instructions, Micromedex, ATC classification), disease classification systems (ICD-10 / ICD-11, SNOMED CT, MeSH), laboratory test reference values ​​(CLIA, hospital LIS system standard values, industry consensus), and hospital internal regulations (such as the list of antibiotics used, the list of drugs restricted by medical insurance, and the list of surgical classifications), and other authoritative guidelines and standards.

[0036] S4. Entity Extraction: Use a large language model to extract the names of diagnoses, drugs, treatments, and examinations. The open-source Qwen3-32B was used as the large language model for extracting medical entities. The model was guided by structured prompting engineering to accurately identify and extract four core clinical entities from unstructured medical record text: diagnosis name, medication, treatment, and examination item names. The `response_format={"type": "json_object"}` parameter was enforced to ensure the output was in a strictly standardized JSON format. Medication entities were structured as much as possible into name, dosage, frequency, and route of administration, while standardized item names were extracted for treatments and examinations.

[0037] This method achieves high recall and strong semantic understanding without additional training, effectively supporting the automated matching and rationality judgment of subsequent quality control rules, significantly improving the efficiency of medical record content structuring and the level of intelligent quality control, while ensuring the stability of project deployment and the uniformity of output format, providing core data support for building a "semantic-aware" intelligent medical record quality control system.

[0038] S5. Model Reasoning: Query and extract relevant information about entities from the knowledge base. According to the quality control rules, combine the queried relevant information with medical documents and submit them to the large model for reasoning to give the quality control results. like Figure 2 As shown, the open-source Qwen3-32B is used as the large language model for quality control document problems. The core execution steps for medical record quality control are as follows: Based on the structured extraction of four key entities—diagnosis name, drug, treatment item, and examination item name—the system performs semantic-level matching and logical reasoning with a pre-set quality control rule base. This includes determining the rationality dimensions such as "whether the diagnosis is supported by symptoms or examinations," "whether the drug selection conforms to guideline recommendations or has contraindications," and "whether the treatment / examination items are consistent with the current stage of the condition." The reasoning process is completed in a single call by the Qwen3-32B model. Through a carefully designed Prompt-guided model combined with the medical knowledge base context, the system triggers corresponding quality control rules for each entity and ultimately forces a structured quality control result output with response_format={"type": "json_object"}, containing fields such as {"name":"Occupational and Recreational Therapy","is_right": False, "comment": "The patient currently exhibits impulsive behavior (such as shouting, swearing, or smashing objects), and the attack risk assessment table is Level III. According to the contraindications description for occupational and recreational therapy, this patient does not meet the conditions for receiving occupational and recreational therapy."}

[0039] Therefore, this treatment is not suitable for this patient. This method achieves an end-to-end automated quality control loop from raw text → entity extraction → rule matching → problem localization → suggestion generation, combining clinical professionalism, standardized output, and engineering integrability, significantly improving the efficiency and accuracy of medical record quality control.

[0040] S6. Problem Collection and Model Optimization: Design a quality control error collection table, analyze the causes of errors, and adjust the quality control rule base, knowledge base, and large model prompt words.

[0041] like Figure 3As shown, after each quality control inference, the system automatically records the differences between the model output and the feedback from clinicians' manual review, including typical issues such as false positives, false negatives, semantic understanding biases, and entity boundary errors. These issues are then structured and stored in a "quality control issue sample library," labeled with the error type, document category, entity type involved, and rule ID. Based on this sample library, the Qwen3-32B model is periodically optimized using LoRA (Low-Rank Adaptation) lightweight fine-tuning technology. Only a few hundred high-quality labeled samples are needed to significantly improve the inference accuracy in specific scenarios (such as "judging the rationality of tumor chemotherapy regimens" and "reviewing perioperative antibiotic use"), while maintaining the original JSON output structure. The optimization process supports canary releases and A / B testing. Before a new model version is launched, the system automatically compares the improvement in key indicators such as recall, precision, and physician adoption rate with the old version, ensuring safe and controllable iteration.

[0042] In addition, the system supports a four-step closed loop of "expert feedback - sample entry - model retraining - effect verification", enabling the model's capabilities to continuously evolve with clinical practice and quality control standards. This effectively addresses real-world challenges such as the dynamic nature of medical language, diverse hospital writing styles, and frequent updates of new guidelines, ensuring the long-term stable and efficient operation of the quality control system.

[0043] Based on the above method, a medical record content quality control device based on a large language model in this embodiment includes: at least one memory and at least one processor; The at least one memory is used to store a machine-readable program; The at least one processor is used to call the machine-readable program to execute a medical record content quality control method based on a large language model.

[0044] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for quality control of medical record content based on a large language model, characterized in that, It has the following steps: S1. Access to medical record documents and preprocessing of document content; S2. Define quality control content and build a quality control rule base: Based on treatment guidelines and hospital management systems, establish a quality control rule base for various types of medical documents, and refine the exclusive rules for each type of document according to the core quality control needs of different document types; S3. Construct a medical knowledge base: Based on clinical guidelines and evidence-based medicine, establish a knowledge base for diagnostic criteria, drugs, treatment items, and examination items; S4. Entity Extraction: Use a large language model to extract the names of diagnoses, drugs, treatments, and examinations. S5. Model Reasoning: Query and extract relevant information about entities from the knowledge base. According to the quality control rules, combine the queried relevant information with medical documents and submit them to the large model for reasoning to give the quality control results. S6. Problem Collection and Model Optimization: Design a quality control error collection table, analyze the causes of errors, and adjust the quality control rule base, knowledge base, and large model prompt words.

2. The method for quality control of medical record content based on a large language model according to claim 1, characterized in that, Step S1 includes: S1-1, Access to medical record documents; After receiving structured medical record data, the medical record quality control system directly reads the "type" field value to identify the document type. S1-2, Document content preprocessing; Preprocessing of medical records involves desensitization and structuring; desensitization involves anonymizing sensitive fields in medical records that involve patient privacy. Document content structuring involves structuring unstructured text content into pre-defined standardized fields.

3. The method for quality control of medical record content based on a large language model according to claim 2, characterized in that, Step S2 includes: S2-1. Define Quality Control Content: Before constructing quality control rules for different types of medical records, it is necessary to first define the quality control content for each type of document. The quality control content for medical records is as follows: Admission record: Reasonableness of diagnosis; Initial medical record: rationality of diagnosis, rationality of medication, rationality of treatment, rationality of examination; Routine course of illness: rationality of medication use, rationality of treatment, rationality of examinations; Phase Summary: Consistency of content, rationality of medication, rationality of treatment, rationality of examination; Discharge summary: consistency of content, rationality of medication, rationality of treatment, and rationality of examinations; S2-2, Constructing a quality control rule base: Based on clarifying the quality control content of various medical record documents, further construct a structured, configurable, and scalable quality control rule base to support intelligent and automated medical record quality control. The construction of the rule base follows the principles of "classification management and semantic drive". Classification management refers to establishing a multi-dimensional rule classification system based on the type of medical records and quality control dimensions. Semantic-driven means using a combination of medical ontology and clinical guidelines to transform quality control content into logical relationships.

4. The method for quality control of medical record content based on a large language model according to claim 3, characterized in that, In step S3, the knowledge base includes: domestic and international clinical diagnosis and treatment guidelines, drug databases, disease classification systems, laboratory test reference values, standards, hospital internal regulations, authoritative guidelines and standards.

5. The method for quality control of medical record content based on a large language model according to claim 4, characterized in that, In step S4, the open-source Qwen3-32B is used as the large language model for extracting medical entities. The model is guided by structured prompts to accurately identify and extract four types of core clinical entities from unstructured medical record text: diagnosis name, drug, treatment and examination item name. The response_format={"type": "json_object"} parameter is forced to ensure that the output is in a standardized JSON format. The drug entity is structured and split into name, dosage, frequency and route of administration, and the treatment and examination items are extracted as standardized item names.

6. The method for quality control of medical record content based on a large language model according to claim 5, characterized in that, In step S5, the open-source Qwen3-32B is used as a large language model for quality control document problems, serving as the execution step for medical record quality control: Based on the structured extraction of four key entities—diagnosis name, drug, treatment item, and examination item name—the extracted entities are semantically matched and logically reasoned with a pre-set quality control rule base. The reasoning process is completed by the Qwen3-32B model in a single call. The model is guided by the Prompt to combine the medical knowledge base context, trigger the corresponding quality control rules for each entity, and finally force the output of structured quality control results with response_format={"type":"json_object"}.

7. The method for quality control of medical record content based on a large language model according to claim 6, characterized in that, In step S6, after each quality control inference is completed, the system automatically records the difference data between the model output results and the clinical physician's manual review feedback, and stores it in a structured manner in the "quality control problem sample library", annotating the error type, the document category, the entity type involved and the rule ID; Based on the sample library, the Qwen3-32B model is regularly optimized using LoRA lightweight fine-tuning technology. The optimization process supports canary releases and A / B testing. Before the new model version is launched, the improvement in recall, precision, and physician adoption rate is automatically compared with the old version.

8. A medical record content quality control device based on a large language model, characterized in that, include: At least one memory and at least one processor; The at least one memory is used to store a machine-readable program; The at least one processor is configured to invoke the machine-readable program to perform the method according to any one of claims 1 to 7.

Images