Traditional Chinese medicine clinical data automatic processing system based on medical big language model multi-agent

Abstract

The invention discloses a traditional Chinese medicine clinical data automatic processing system based on a medical big language model multi-agent, and relates to the technical field of medical data treatment. The system comprises a task understanding agent, a clinical data set treatment tool library and a data treatment agent which are in communication connection and cooperate to realize clinical data automatic processing in a traditional Chinese medicine clinical curative effect evaluation scene; the task understanding agent accurately recognizes traditional Chinese medicine clinical curative effect evaluation tasks and data requirements through training data set extension, knowledge distillation weighted fusion and expert preference alignment training; the clinical data set governance tool library comprises five Chinese and western medicine exclusive modules such as text structuring; and the data governance agent automatically generates a governance flow and schedules a tool library to execute through the exclusive fine tuning data set and Lora fine tuning. Full-process automatic processing of traditional Chinese medicine clinical data is realized, the task understanding accuracy and the data governance suitability are improved, and the scientific research efficiency of clinicians is improved.

Description

Technical Field

[0001] This invention relates to the field of medical data governance technology, particularly to the field of medical data governance technology based on artificial intelligence, and more specifically to an automated processing system for traditional Chinese medicine clinical data based on a multi-agent medical language model. Background Technology

[0002] With the deep integration of medical informatics and artificial intelligence technologies, the automated processing of medical data (i.e., real-world data) generated in clinical practice has become a core direction for improving the efficiency of medical research. On the one hand, real-world data involves a large amount of multi-source heterogeneous data (such as electronic medical records, laboratory indicators, medication records, follow-up data, etc.). Traditional manual processing methods suffer from low efficiency, large errors, and reliance on specialized data processing capabilities, making it difficult to meet the demands of clinical research for data quality and processing timeliness. On the other hand, large language models, with their powerful natural language understanding and generation capabilities, are gradually deepening their application in the field of medical data processing. From the cleaning and standardization of general data to the interpretation and report generation of clinical trial results, they have shown the potential to replace some manual work, driving the transformation of medical data processing towards intelligence and automation.

[0003] In the field of TCM clinical efficacy evaluation, its unique characteristics further highlight the necessity of technological upgrades. TCM clinical efficacy evaluation is a comprehensive task integrating multiple disciplines such as clinical epidemiology, statistics, clinical medicine, TCM, and Chinese materia medica. It needs to be conducted around specific dimensions such as "syndrome differentiation," "TCM dosage and course of treatment," and "TCM-specific efficacy indicators (such as symptom improvement and constitution conditioning effects)." Data processing not only needs to address issues such as missing, abnormal, and standardized general medical data, but also needs to adapt to the specificities of TCM terminology (such as "four diagnostic methods of TCM," "syndrome differentiation and treatment," and "one complaint and five histories") and TCM efficacy evaluation indicators (such as "changes in tongue coating and pulse" and "types of adverse reactions to TCM"). However, current industry solutions mostly focus on general medical scenarios and have not yet developed targeted solutions for the unique needs of TCM efficacy evaluation, leading to multiple pain points for clinicians conducting TCM efficacy evaluation research.

[0004] Currently, existing technologies related to this invention's technical field are mainly divided into two categories, focusing on "clinical trial data analysis and interpretation" and "medical data governance," respectively. However, their technical solutions have significant limitations when adapted to the clinical efficacy evaluation scenario of traditional Chinese medicine, as follows: (I) General Clinical Trial Data Analysis and Interpretation Protocol Represented by the published patent CN118116536A (“A Large Language Model System for Clinical Trial Data Analysis and Interpretation”), the core architecture of this type of solution includes a user interface, a large language model, a natural language processing module, a data cleaning and transformation module, a data analysis and modeling module, and a reporting and inference module. It processes complex textual data (such as medical records and research papers) from clinical trials using a single large language model, combined with multimodal data processing capabilities (images, radiological data) to achieve data integration, analysis and modeling, result visualization, and automatic report generation. Its main drawback is: 1. Insufficient scenario adaptability: The model is not customized for the clinical efficacy evaluation scenario of traditional Chinese medicine. The training data of the large language model is mainly general clinical trial data, which cannot accurately identify data requirements such as "dosage of Chinese medicine", "syndrome classification" and "specific indicators for the efficacy evaluation of Chinese medicine". As a result, when clinicians put forward traditional Chinese medicine-related tasks such as "what data is needed to evaluate the effectiveness of Chaiqin Chengqi Decoction", the list of requirements output by the model is out of touch with the actual research. 2. Incomplete functional chain: It focuses on the "data processing-analysis-reporting" link and does not have a dedicated "task understanding" module. It relies on users to actively clarify their data needs (such as manually specifying the indicators to be analyzed and the data format). However, most clinicians lack research experience in evaluating the efficacy of traditional Chinese medicine and find it difficult to accurately express their data needs, which limits the practicality of the solution. 3. Poor targeting of data governance: The data cleaning and transformation modules adopt general medical data processing rules (such as routine missing value filling and numerical normalization), and do not design specific functions for the large number of problems in TCM data such as "non-standardized TCM terminology" (such as different expressions of syndrome descriptions such as "Qi deficiency" and "Yin deficiency") and "structuring of texts on adverse reactions of Chinese medicine". The data after governance is difficult to be directly used for TCM efficacy evaluation and analysis.

[0005] (II) General Medical Data Governance Solution Represented by the published patent CN118155780A (“A Medical Data Governance Method and System Based on Knowledge Graph and Large Language Model”), the core idea of ​​this type of solution is to construct a data governance engine of “knowledge graph + large language model”. This involves using an intelligent agent to break down governance tasks and construct prompt words, leveraging a medical domain knowledge graph to enhance the semantic understanding ability of the large language model on medical data, thereby achieving data quality diagnosis and automated governance (such as missing value handling, redundant data cleaning, and data standardization). Its main drawback is: 1. Failure to connect with the needs of efficacy evaluation tasks: The focus is only on "improving the quality of the data itself" (such as solving problems such as missing data and inconsistent formats), without linking data governance with the "efficacy evaluation task objectives of traditional Chinese medicine" - for example, it is not possible to automatically prioritize the governance of core data such as "liver function indicators" and "traditional Chinese medicine syndrome scores" based on the task of "evaluating the efficacy of a certain traditional Chinese medicine on chronic hepatitis". The governance process is disconnected from the clinical research objectives, requiring manual secondary screening of the governed data, which increases the workload of doctors. 2. Limited functionality of the intelligent agent: The "data governance intelligent agent" is only set up, which lacks the ability to understand the user's task intent. Users need to specify in advance "which data needs to be governed" and "what is the governance goal" (such as manually inputting "to govern the missing values ​​in the patient's age and gender data"). However, in the task of evaluating the efficacy of traditional Chinese medicine, doctors are often unclear about "which data are key to the efficacy evaluation", which means that the solution cannot solve the core pain point of "vague data requirements" from the source. 3. Insufficient security and ease of use: Relying on a cloud-based data governance engine (which requires calling external knowledge graph interfaces and tool libraries), there is a risk of data leakage if clinicians upload TCM clinical data containing patient privacy information; at the same time, the solution still requires users to have basic data governance knowledge (such as understanding terms such as "data standardization" and "feature extraction"), and it has not designed a low-threshold operation process for "clinical doctors without technical backgrounds", making it difficult to popularize and apply.

[0006] Considering the shortcomings of the existing technical solutions, there is currently no technology in the industry that can simultaneously solve the three core problems in the clinical efficacy evaluation of traditional Chinese medicine: 1. Problem of matching task understanding with data requirements: Clinicians lack research experience in evaluating the efficacy of traditional Chinese medicine and cannot clearly define the data dimensions required for the research (such as "what TCM syndrome indicators should be included to evaluate the hypoglycemic effect of traditional Chinese medicine"). Existing technology cannot automatically identify and prompt the core data requirements corresponding to the task. 2. Data governance scenario adaptation and task linkage issues: Existing data governance solutions are either designed for general medical data or are detached from the task objectives of efficacy evaluation. They cannot design governance rules based on the special characteristics of TCM efficacy evaluation (such as TCM terminology and exclusive indicators), nor can they automatically link the governance process with the task objectives. 3. The balance between security and ease of use: External cloud governance services pose data security risks, while localized governance solutions require doctors to have coding skills or data governance experience. Existing technologies have not achieved a "localized, zero-code, task-driven" TCM clinical data processing model, which is difficult to meet the actual needs of clinicians.

[0007] Therefore, there is an urgent need for a dedicated technical solution for the clinical efficacy evaluation of traditional Chinese medicine. This solution should address the shortcomings of existing technologies by automating and adapting the entire process of "task understanding-data governance" to reduce the research burden on clinicians while ensuring data security and processing efficiency. Summary of the Invention

[0008] To overcome the defects and shortcomings of the existing technologies, this invention provides an automated processing system for TCM clinical data based on a multi-agent medical language model. The purpose of this invention is to address the problems in TCM clinical efficacy evaluation scenarios, such as unclear data requirements and lack of professional data processing capabilities among clinicians, privacy and security risks associated with external data governance services, and poor adaptability of general-purpose language models to TCM efficacy evaluation task understanding and data governance. This invention automates, contextualizes, and securely processes the entire TCM clinical efficacy evaluation process from "task understanding to data governance," significantly reducing the research burden on clinicians and improving the efficiency and quality of TCM efficacy evaluation data processing.

[0009] The system of this invention includes a task understanding agent, a clinical dataset governance toolkit, and a data governance agent for clinical TCM efficacy. The task understanding agent is constructed through a three-stage process: "training dataset expansion (dual-model validation and deduplication) - knowledge distillation weighted fusion training - medical expert preference alignment training." This allows it to accurately identify the user's TCM efficacy evaluation task and output the core data dimensions and format requirements matching the task. The clinical dataset governance toolkit, tailored to the characteristics of TCM data, integrates five modules: missing value handling, outlier handling, text structuring (adapting to entity extraction such as "one complaint and five histories" and "adverse reactions of TCM"), text standardization (adapting to TCM diagnostic / surgical terminology), and normalization. It provides dedicated TCM data governance tools. The data governance agent, based on the characteristics of TCM efficacy evaluation data, fine-tunes a large language model. It can receive analysis reports of the raw data and generate appropriate governance process instructions, calling the toolkit modules to complete automated governance.

[0010] The system of this invention achieves a localized closed-loop processing of the entire process of TCM efficacy evaluation tasks, from "clear requirements" to "data governance," through the collaboration of three major components. It eliminates the need for clinicians to have coding or data governance expertise, avoids data security risks associated with external services, and significantly improves the task understanding accuracy and data governance adaptability of large language models in TCM efficacy evaluation scenarios. This provides efficient, safe, and easy-to-use technical support for TCM clinical efficacy evaluation research.

[0011] To address the problems existing in the prior art, the present invention is achieved through the following technical solution.

[0012] This invention provides an automated processing system for TCM clinical data based on a medical big language model multi-agent system. The system includes a task understanding agent, a clinical dataset governance toolkit, and a data governance agent. The three are connected and work together to achieve automated processing of clinical data in the context of TCM clinical efficacy evaluation. The task understanding agent is configured to be formed through training dataset expansion, knowledge distillation weighted fusion training, and direct preference alignment training based on medical expert experience; the task understanding agent is used to receive user input of tasks related to the evaluation of clinical efficacy of traditional Chinese medicine, identify the clinical data requirements corresponding to the task and provide feedback to the user; and analyze the clinical data uploaded by the user in combination with the clinical data requirements to generate a raw data analysis report. The clinical dataset governance toolkit includes a missing value processing module, an outlier processing module, a text structuring module, a text standardization module, and a normalization processing module. Each module is configured to perform missing value imputation, outlier detection and processing, medical entity structure extraction, TCM terminology text standardization, and data normalization processing for TCM clinical data. The data governance agent is communicatively connected to both the task understanding agent and the clinical dataset governance toolkit. The data governance agent is configured to form a large language model based on a dedicated fine-tuning dataset for TCM efficacy evaluation and LoRa high-efficiency parameter fine-tuning. The data governance agent receives the raw data analysis report and parsed data output by the task understanding agent, generates TCM clinical data governance process instructions adapted to the data, and controls the clinical dataset governance toolkit to perform governance operations. After the governance operations are completed, the governed data is output, and the governed data undergoes quality control. If the quality control is satisfactory, the qualified data is analyzed, and a data governance report is generated. If the quality control fails, the data governance process instructions are regenerated based on the current situation, and the clinical dataset governance toolkit is controlled to perform governance operations until the governed data passes quality control. The system parses user tasks and data through the task understanding agent, generates a governance process through the data governance agent, and executes the collaborative logic of the clinical dataset governance tool library to complete the automated processing of TCM clinical data, and outputs the treated data and data governance report that meet the requirements of TCM clinical efficacy evaluation.

[0013] More preferably, the training dataset expansion adopts the method of expanding the question using a large language model A and filtering the effective input using a large language model B to form an input-output pair dataset containing the original input and the expanded input; The original input is the task requirements of researchers in the actual clinical evaluation of the efficacy of traditional Chinese medicine. The output is the typical data variable dimension information used by researchers when processing their task requirements, including categorical variables, numerical variables, text, the type of each variable, and its necessity. The aforementioned Large Language Model A extended question format refers to using Large Language Model A to expand the task requirements in the original input from the question format level, resulting in n task descriptions with the same semantics but different question formats; The large language model B filtering of valid inputs refers to using the large language model B to score the extended n task descriptions, filtering out low-scoring task descriptions, and obtaining k high-scoring task descriptions.

[0014] More preferably, the large language model A is the deepseek-r1 large language model, and the large language model B is the qwen-max large language model.

[0015] More preferably, the specific configuration for the knowledge distillation weighted fusion training of the task understanding agent is as follows: a teacher model with the ability to transfer clinical knowledge of traditional Chinese medicine and a student model with lightweight reasoning and task response capabilities are selected, and training is completed by combining the loss function that integrates the knowledge weights of the teacher model and the weights of the real data distribution based on LoRa efficient parameter fine-tuning technology. The loss function is:

[0016] In the formula, N represents the total number of training samples; V represents the total number of output categories of the model. This represents the probability value predicted by the student model for the j-th class of the i-th sample. This represents the probability value predicted by the teacher model for the j-th class of the i-th sample. It is the true label value of the j-th category, which can be 0 or 1; This represents the knowledge weights of the teacher model. This represents the weights of the actual data distribution.

[0017] Further optimization involves selecting deepdeek as the teacher model and qwen2.5-14b-instruct as the student model.

[0018] More preferably, the direct preference alignment training configuration for the medical expert experience of the task understanding agent is as follows: The evaluation needs related to the clinical efficacy evaluation of traditional Chinese medicine were selected as the basic data. The corresponding responses were generated based on the model trained by knowledge distillation and weighted fusion. Combined with the evaluation results of the superiority and inferiority of the responses by professionals with clinical efficacy evaluation experience, a preference dataset of "evaluation needs - better responses - second-best responses" was constructed. Based on the model trained by knowledge distillation and weighted fusion, the direct preference optimization loss function was adopted. Combined with the reference model with fixed parameters, the preference learning training was completed by comparing the predicted probabilities of the better and second-best responses. The direct preference optimization loss function is:

[0019] In the formula, This represents the sigmoid function. Indicates hyperparameters, This represents the reference model, whose parameters are related to... The parameters are the same, and the parameters are not updated during preference alignment; This represents the model currently being trained, i.e., the model obtained by the above-mentioned knowledge distillation weighted fusion training. This indicates that the response is better if it is judged by professionals with clinical efficacy evaluation experience for the task. This indicates a response that is considered suboptimal by professionals with experience in evaluating the clinical efficacy of the task.

[0020] The task understanding agent parses the clinical data uploaded by users in accordance with clinical data requirements. Specifically, upon receiving the uploaded clinical data, it first classifies the variables into numerical, categorical, and text variables. For numerical variables, it standardizes the units of laboratory variables and converts them to floating-point numbers. For categorical variables, it first desensitizes ID information and then maps and encodes string variables. For text variables, it desensitizes hospital information and personal privacy information. Subsequently, it performs data overview statistics for different variable types, including total patient count, calculation of missing value proportions for different variables, outlier detection for different variables, length distribution statistics for text information, distribution statistics for numerical variables, and distribution statistics for categorical variables. Finally, it generates the corresponding raw data analysis report.

[0021] More preferably, the missing value processing module in the clinical dataset governance tool library is specifically configured to use a logical imputation strategy for missing values ​​in traditional Chinese medicine clinical data. That is, firstly, different data tables uploaded by different users are combined, and then the imputation result of the missing value is determined by the value of the same variable in different data tables.

[0022] More preferably, the outlier processing module in the clinical dataset governance toolkit is specifically configured to use a quantile-based outlier detection method and a medical common sense-based outlier detection method for numerical variables in TCM clinical data; and a medical common sense-based outlier detection method for categorical variables in TCM clinical data.

[0023] More preferably, the text structuring module in the clinical dataset governance toolkit is specifically configured to extract medical entities from long texts of traditional Chinese medicine clinical data through the logic of defining domain entity structures and examples, segmenting and cleaning the long text to be processed, constructing entity recognition prompt templates, generating dynamic prompt words, and performing structured processing. Specifically, this includes: Define the domain entity structure and examples, clarify the types of medical entities to be extracted, define the attributes and structure of each entity type, and collect real TCM clinical text data of the corresponding types as examples; The long text to be processed is segmented and cleaned. Based on the semantic structure and format characteristics of the text, the long text of TCM clinical data is divided into multiple independent logical paragraphs. Irrelevant format symbols, duplicate information and noisy text in each paragraph are removed to extract the effective text content. Construct entity recognition prompt templates. Design exclusive prompt templates for different types of medical entities. The templates contain clear entity extraction instructions and corresponding entity extraction examples. Generate dynamic prompts by generating personalized prompts based on the context information of the text to be processed, the target entity type, and the text length. The system performs structured processing, combining dynamic prompts with cleaned long text paragraphs and inputting them into a large language model for inference. The large language model then extracts target medical entity information based on the prompts and outputs structured entity data.

[0024] Furthermore, the large language model used in the structured processing is the deepseek large language model.

[0025] The text standardization module in the clinical dataset governance toolkit is specifically configured to standardize medical terminology in TCM clinical data through the logic of constructing a standard classification dataset, segmenting and cleaning the text to be processed, constructing a text classification Prompt template, generating dynamic prompt words, and performing standardization processing. Specifically, this includes: A standard classification dataset was constructed by collecting common TCM clinical short texts, diagnostic names, surgical names, and TCM syndrome descriptions in the medical field, and classifying and labeling them in detail according to industry standards to form a standard classification dataset. The text to be processed is segmented and cleaned by splitting the clinical text of traditional Chinese medicine into several independent short text units, removing irrelevant characters, punctuation marks, extra spaces and error messages from the text, and unifying the data format in the text and converting it to lowercase. Build a text classification Prompt template and design a custom Prompt template that includes clear instructions and standardized examples of TCM terminology; Generate dynamic prompts by adjusting and generating targeted prompts in real time based on the characteristics and context of the text to be processed. Standardization processing is performed, combining the segmented and cleaned text with dynamic prompts, and inputting it into a large language model for reasoning. The large language model then outputs standardized medical terminology results that conform to industry standards.

[0026] Furthermore, the large language model used in the standardization process is the deepseek large language model.

[0027] Further preferably, the normalization module in the clinical dataset management toolkit is specifically configured to: normalize the units of variables, converting them into the same indicator; then provide maximum-minimum value normalization for numerical variables in the data, normalizing the variables to the range of 0-1; and simultaneously provide mean-standard deviation normalization for numerical variables, normalizing the variables to a distribution with a mean of 0 and a standard deviation of 1.

[0028] In a further preferred embodiment, the data governance intelligent agent first collects an overview of the datasets used in the TCM clinical efficacy evaluation task as input, and then outputs the governance methods and processes adopted for the datasets. The input and output are combined according to a set template, and the combined input-output pair is the TCM efficacy evaluation dedicated fine-tuning dataset.

[0029] More preferably, the setting template is <|im start |>system prompt<|im end |> <|im start |>user input<|im end |> <|im start |>assistant output<|im end |>.

[0030] More preferably, the data governance agent is configured to use LoRa efficient parameter fine-tuning of the basic large language model, specifically by using LoRa's efficient parameter fine-tuning method to fine-tune and update the basic model parameters; during the model parameter fine-tuning stage, the model's loss function is calculated as follows:

[0031] In the formula, N represents the sample size. This represents the predicted probability that the i-th sample belongs to the true category; finally, the process is iterated until the loss value converges.

[0032] Compared with the prior art, the beneficial technical effects of the present invention are as follows: This invention, through a collaborative architecture of "task understanding agent - clinical dataset governance toolkit - data governance agent," addresses the unique characteristics of TCM clinical efficacy evaluation scenarios (such as non-standardized TCM terminology, specific TCM efficacy indicators, and limitations in clinicians' research capabilities). It achieves automation, scenario-based approach, and security across the entire clinical data processing process. Compared to existing general medical data processing solutions and clinical trial analysis solutions, it possesses the following significant technical advantages: 1. Accurately match the needs of TCM clinical efficacy evaluation, and solve the problem of the disconnect between task understanding and data requirements: —Significantly improving task understanding accuracy, the task understanding agent accurately identifies tasks specific to TCM clinical efficacy evaluation through a three-stage process: "training dataset expansion (dual-model validation and deduplication) - knowledge distillation weighted fusion training - medical expert preference alignment training." Specifically, the input-output pair dataset constructed by the dual models (Deepseek-r1 extended questioning + Qwen-max filtering of effective inputs) covers specific scenario tasks such as "TCM efficacy evaluation" and "TCM syndrome differentiation association analysis." Knowledge distillation training (Deepseek teacher model + Qwen2.5-14b-instruct student model) combined with LoRa efficient fine-tuning enables efficient transfer of medical knowledge. Expert preference alignment training (DPO algorithm) ensures that the model output aligns with the professional judgment logic of clinical efficacy evaluators. Tests showed that the model achieved an accuracy of over 92% in understanding specific tasks such as "what data is needed to evaluate the effectiveness of Chaiqin Chengqi Decoction" and "data dimensions for the correlation analysis between adverse reactions of traditional Chinese medicine and syndrome differentiation," which is 40% higher than general-purpose large language models (such as the untuned GPT-4). This solves the problem that existing technologies "cannot identify the specific data requirements for evaluating the efficacy of traditional Chinese medicine."

[0033] —This system enables automatic generation and adaptation of data requirements. The intelligent agent can automatically output a list of core data dimensions for user-inputted TCM efficacy evaluation tasks. This list includes categorical variables (such as syndrome differentiation and TCM dosage forms), numerical variables (such as drug dosage and laboratory indicators), and textual variables (such as symptom descriptions in medical records). It clarifies the necessity of each variable (e.g., "liver and kidney function indicators are mandatory variables," "tongue and pulse descriptions are optional variables") and format requirements (e.g., "TCM treatment courses must be standardized in 'days / dosages'"). Clinicians do not need prior experience in research data design; they can quickly identify the data required for the task through the intelligent agent, significantly lowering the entry barrier for TCM efficacy evaluation research and addressing the shortcomings of existing technologies that require users to manually specify data requirements, which is incompatible with the research capabilities of clinicians.

[0034] 2. This invention enables customized management of TCM clinical data, solving the problem of poor adaptability of general management solutions: —Multi-module coordinated governance, covering all scenarios of TCM data issues. The five modules of the clinical dataset governance toolkit are customized for the characteristics of TCM clinical data: Missing value handling module: For missing values ​​of variables such as TCM dosage and syndrome scores, it adopts a hierarchical strategy of "cluster mean imputation (missing ratio <10%) - symmetric distribution mean imputation - extreme value median imputation", which improves the imputation accuracy by 25% compared with the general mean imputation method. It is especially suitable for the scenario of "uneven missing data ratio in some centers" in multi-center TCM clinical trials; Text structuring module: It accurately extracts entities such as "one complaint and five histories (TCM-specific medical history records), TCM compatibility combinations (principal, assistant, adjuvant, and guide relationships)" and adverse reactions (such as "rash with itching, relieved after discontinuation of medication") through dynamic Prompt templates. The entity extraction recall rate reaches 89%, which is higher than that of general medical entity extraction models (such as BioBERT). The improvement is 32%, solving the problem of "fragmented information and difficulty in structuring" in long TCM clinical texts; the text standardization module: based on the "TCM Clinical Diagnosis and Treatment Terminology", a standard dataset is constructed, which uniformly maps non-standardized syndrome descriptions such as "liver qi stagnation" and "liver qi stagnation" to "liver qi stagnation syndrome", with a terminology standardization coverage rate of 95%, solving the defect of existing technology "unable to handle the problem of multiple expressions of TCM terminology" that leads to the inability to integrate data across centers; the normalization module: first, the unit of laboratory indicators (such as blood glucose "mg / dL" to "mmol / L") is unified, and then "0-1 interval maximum-minimum value normalization (adapted to traditional statistical analysis)" and "standard normal distribution normalization (adapted to machine learning model)" are provided to meet the diversified needs of "combining traditional statistical methods with AI analysis" in the evaluation of TCM efficacy.

[0035] —Dual improvement in governance efficiency and data quality. Through the automated process scheduling of the data governance intelligent agent (receiving raw data analysis reports → generating governance instructions → calling tool library modules → quality inspection and iteration), the governance time for a single batch of 1,000 cases of TCM clinical data (including 500 text medical records and 50 variables) has been shortened from 3-5 days in traditional manual processing to 1-2 hours, improving processing efficiency by more than 90%. The "completeness (missing values ​​≤2%), consistency (terminology standardization rate ≥95%), and accuracy (outliers ≤1%)" of the governed data all meet the requirements of the Good Clinical Practice (GCP) for TCM clinical trials. The data can be directly used for subsequent efficacy evaluation analysis (such as logistic regression analysis of the correlation between TCM efficacy and syndrome differentiation, and random forest model prediction of TCM adverse reaction risk), avoiding the additional cost of "secondary manual screening of governed data" in existing technologies.

[0036] 3. Promote the standardization and large-scale development of clinical efficacy evaluation research in traditional Chinese medicine: —It can facilitate the integration of multi-center research data. The system standardizes text (unifying TCM terminology), normalizes units (unifying laboratory indicators / drug dosages), and desensitizes privacy (removing center identifiers / patient identity information), ensuring the "homogeneity" of TCM clinical data from different centers. For example, in a multi-center study on "TCM treatment of chronic atrophic gastritis" conducted jointly by 10 hospitals in a certain province, after processing data through this system, the uniformity rate of the "syndrome differentiation" variable among the centers increased from 45% to 95%, and the uniformity rate of "TCM drug dosage" units reached 100%. This enabled direct merging and analysis of multi-center data, solving the industry pain points of "inconsistent data formats and high integration costs" in traditional multi-center studies.

[0037] —Supporting the construction of an evidence system for evaluating the efficacy of traditional Chinese medicine (TCM). The system's output, including "standardized datasets after treatment" and "data treatment reports (including treatment methods, quality indicators, and outlier explanations)," can be directly used for TCM clinical trial applications (such as submitting efficacy evaluation data to the National Medical Products Administration). The reports meet the requirements for "data traceability and completeness" in the "Classification and Application Materials Requirements for Registration of Traditional Chinese Medicine." Simultaneously, the standardized treatment process and data format provide technical support for the "reproducibility" of TCM efficacy evaluation research, avoiding the problem of "opaque treatment methods and difficulty in reproducing research results" in existing technologies, thus contributing to the improvement of the evidence level in TCM clinical research. Attached Figure Description

[0038] Figure 1 This is a framework diagram of the automated clinical data processing system of the present invention; Figure 2 This is a flowchart illustrating the workflow of the intelligent agent task in this invention. Figure 3This is a flowchart illustrating the data parsing process performed by the intelligent agent in the task understanding section of this invention. Figure 4 This is a flowchart of the data governance intelligent agent of the present invention. Detailed Implementation

[0039] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort should fall within the scope of protection of the present invention.

[0040] Example 1 As a preferred embodiment of the present invention, please refer to the appendix to the specification. Figure 1 As shown, this embodiment provides an automated processing system for TCM clinical data based on a multi-agent medical big language model. The system includes a task understanding agent, a clinical dataset governance toolkit, and a data governance agent. The three are connected and work together to achieve automated processing of clinical data in the context of TCM clinical efficacy evaluation.

[0041] The task understanding agent is configured to be formed through training dataset expansion, knowledge distillation weighted fusion training, and medical expert preference alignment training; the task understanding agent is used to receive user input of TCM clinical efficacy evaluation related tasks, identify the corresponding clinical data needs of the task and provide feedback to the user; and analyze the clinical data uploaded by the user in combination with the clinical data needs to generate a raw data analysis report. The clinical dataset governance toolkit includes a missing value processing module, an outlier processing module, a text structuring module, a text standardization module, and a normalization processing module. Each module is configured to perform missing value imputation, outlier detection and processing, medical entity structure extraction, TCM terminology text standardization, and data normalization processing for TCM clinical data. The data governance agent is communicatively connected to both the task understanding agent and the clinical dataset governance toolkit. The data governance agent is configured to form a large language model based on a dedicated fine-tuning dataset for TCM efficacy evaluation and LoRa high-efficiency parameter fine-tuning. The data governance agent receives the raw data analysis report and parsed data output by the task understanding agent, generates TCM clinical data governance process instructions adapted to the data, and controls the clinical dataset governance toolkit to perform governance operations. After the governance operations are completed, the governed data is output, and the governed data undergoes quality control. If the quality control is satisfactory, the qualified data is analyzed, and a data governance report is generated. If the quality control fails, the data governance process instructions are regenerated based on the current situation, and the clinical dataset governance toolkit is controlled to perform governance operations until the governed data passes quality control. The system parses user tasks and data through the task understanding agent, generates a governance process through the data governance agent, and executes the collaborative logic of the clinical dataset governance tool library to complete the automated processing of TCM clinical data, and outputs the treated data and data governance report that meet the requirements of TCM clinical efficacy evaluation.

[0042] Example 2 As another preferred embodiment of the present invention, this embodiment is a further detailed supplement and explanation of the technical solution of the present invention based on the above embodiment 1.

[0043] In the evaluation of clinical efficacy of traditional Chinese medicine, the required data dimensions often differ depending on the specific clinical task. Therefore, we first propose an intelligent agent for understanding clinical efficacy tasks. Based on the user's different clinical task needs, the agent prompts the user with the required data and data requirements; however, if the user's question does not contain specific efficacy evaluation-related requirements, it will provide a normal dialogue.

[0044] Refer to the instruction manual appendix Figure 2 As shown, after receiving user input, the task understanding agent first determines whether the user is uploading a data file or entering text. If it is text, it will understand the user's intent and needs based on prompt word engineering combined with a large language model. If the user's input does not contain any intent related to efficacy evaluation, it will use a general chat model and prompt the user whether they need to perform data processing tasks related to efficacy evaluation. If the user's needs are related to efficacy evaluation, such as "What are the efficacy and safety of Chaiqin Chengqi Decoction? What data is needed for clinical efficacy evaluation?", then the user's input will enter the task understanding model formed through training dataset expansion, knowledge distillation weighted fusion training, and medical expert preference alignment training. The system will prompt the user for the data required to complete the task and the data requirements, and ask if the user wants to process the data. If the user has uploaded a data file, it will determine whether the user has entered their task requirements. If the user has not entered task requirements, it will prompt the user to enter the efficacy evaluation task they need to perform. If so, it will combine the task understanding model. The output data requires parsing the data uploaded by the user.

[0045] After obtaining the user-uploaded data, refer to the attached instruction manual. Figure 3 As shown, the data variables are first categorized into numerical, categorical, and text variables. For numerical variables, the laboratory variables are standardized in units, and numerical variables are converted to floating-point numbers. For categorical variables, ID information is anonymized, and string variables are mapped and encoded; for example, "male" in the gender variable is encoded as 0, and "female" as 1. For text information, privacy information, including hospital information and personal privacy information, is anonymized.

[0046] Then, data overview statistics are performed for different types of variables, including total number of patients, calculation of missing value ratios for different variables, outlier detection for different variables, length distribution of text information, distribution of numerical variables, distribution of categorical variables, and finally, the corresponding raw data analysis report is generated.

[0047] Example 3 As another preferred embodiment of the present invention, this embodiment further elaborates and supplements the technical solution of the present invention based on the above-described Embodiment 1 or Embodiment 2. In this embodiment, the specific configuration of the task understanding agent is described in detail.

[0048] As one implementation method of this embodiment, the training dataset expansion in the task understanding agent configuration process adopts the method of expanding the question by large language model A and filtering the effective input by large language model B to form an input-output pair dataset containing the original input and the expanded input. The original input is the task requirements of researchers in the actual clinical evaluation of the efficacy of traditional Chinese medicine. The output is the typical data variable dimension information used by researchers when processing their task requirements, including categorical variables, numerical variables, text, the type of each variable, and its necessity. The aforementioned Large Language Model A extended question format refers to using Large Language Model A to expand the task requirements in the original input from the question format level, resulting in n task descriptions with the same semantics but different question formats; The large language model B filtering of valid inputs refers to using the large language model B to score the extended n task descriptions, filtering out low-scoring task descriptions, and obtaining k high-scoring task descriptions.

[0049] Specifically, the process collects the task requirements of researchers in evaluating the clinical efficacy of traditional Chinese medicine as input, such as "What are the efficacy and safety of Chaiqin Chengqi Decoction? What data is needed for clinical efficacy evaluation?". Then, it collects typical data variable dimensional information used by researchers in handling this task, including categorical variables, numerical variables, text information, etc., as well as the type and necessity of each variable, and uses this data information as output. Next, the large-scale model Deepseek-r1 is used to expand the collected task requirement input from the question phrasing level, resulting in n task descriptions with the same semantics but different wording. However, the task requirements directly generated by the large model may suffer from model illusion, so we introduce another large-scale model (qwen-max) to score the previously expanded n task requirements, then filter out low-scoring questions to obtain k high-scoring task requirements input1, input2, ..., inputk. Finally, we combine this with the output to construct k+1 input-output pairs.<input, ouptut> ,<input1, output> ,…, <input k, output> .

[0050] As another implementation of this embodiment, the knowledge distillation weighted fusion training of the task understanding agent is specifically configured as follows: a teacher model with the ability to transfer clinical knowledge of traditional Chinese medicine and a student model with lightweight reasoning and task response capabilities are selected, and training is completed by combining the loss function that integrates the knowledge weights of the teacher model and the weights of the real data distribution based on LoRa efficient parameter fine-tuning technology.

[0051] Specifically, the qwen2.5-14b-instruct large language model was selected as the student model for task understanding, and Deepseek was used as the teacher model for task understanding. Then, based on LoRa's efficient parameter fine-tuning method, qwen2.5-14b-instruct was trained using the constructed training dataset. The training process includes two parts: first, the input-output data is organized according to the following template to obtain the input for the model training phase: <|im start |>system prompt<|im end |> <|im start |>user input<|im end |> <|im start|>assistant output<|im end |> Where <|im start |>,<|im end |>, system, user, and assistant all represent special characters in the model, while input and output represent data pairs constructed in the training dataset. We then use a word segmenter to divide the model into N words, inputting each word into the model to obtain the probability p of each word's output. Finally, we use knowledge distillation loss and cross-entropy loss functions to calculate the model's loss value, as follows:

[0052] In the formula, N represents the total number of training samples; V represents the total number of output categories of the model. This represents the probability value predicted by the student model for the j-th class of the i-th sample. This represents the probability value predicted by the teacher model for the j-th class of the i-th sample. It is the true label value of the j-th category, which can be 0 or 1; This represents the knowledge weights of the teacher model. The weights are represented by the actual data distribution. Then, the gradient value of each parameter in the model is calculated using the gradient descent algorithm, and the parameters are updated. Finally, this process is iterated until the loss value converges. The trained model is then obtained. .

[0053] As another implementation of this embodiment, the direct preference alignment training of the medical expert experience of the task understanding agent is specifically configured as follows: Evaluation needs related to the clinical efficacy evaluation of traditional Chinese medicine were selected as the basic data. The corresponding responses were generated based on the model trained by knowledge distillation and weighted fusion. Combined with the evaluation results of the superiority and inferiority of the responses by professionals with clinical efficacy evaluation experience, a preference dataset of "evaluation needs - better responses - second-best responses" was constructed. Based on the model trained by knowledge distillation and weighted fusion, the direct preference optimization loss function was adopted. Combined with the reference model with fixed parameters, the preference learning training was completed by comparing the predicted probabilities of the better and second-best responses.

[0054] Specifically, regarding the construction of the preference dataset, in this part we will further collect several evaluation requirements (task1, task2, ..., task N) from the clinical efficacy evaluation of traditional Chinese medicine, and use the trained model. Two responses are generated for each evaluation requirement. Experienced efficacy evaluators then rank the two responses for each task based on their quality. If neither response is deemed satisfactory, it is discarded, and the model is re-generated. Ultimately, for each task, two responses with different priorities are obtained.<task1,response1,response 2> ,…,<task N,response1,response 2> Here, response1 represents the response that the expert considers better, and response2 represents the response that the expert considers second-best. Model preference alignment training: based on the trained model... The direct preference loss is calculated, and the direct preference optimization loss function is:

[0055] In the formula, This represents the sigmoid function. Indicates hyperparameters, This represents the reference model, whose parameters are related to... The parameters are the same, and the parameters are not updated during preference alignment; This represents the model currently being trained, i.e., the model obtained by the above-mentioned knowledge distillation weighted fusion training. This indicates that the response is better if it is judged by professionals with clinical efficacy evaluation experience for the task. This represents the response deemed suboptimal by professionals with clinical efficacy evaluation experience for the task. The process is iterated until the loss value converges. The resulting trained clinical efficacy task understanding model is then calculated. .

[0056] Example 4 As another preferred embodiment of the present invention, this embodiment further elaborates and supplements the technical solution of the present invention based on the above-described embodiments 1, 2, or 3. In this embodiment, the specific configuration of the clinical dataset governance toolkit is described in detail.

[0057] The missing value handling module in the Clinical Dataset Governance Toolkit is specifically configured to perform missing value imputation on numerical variables in clinical data according to the following rules: When the proportion of missing variables is less than the preset threshold, unsupervised clustering is performed using other variables as features, and the mean of each variable is used to fill the missing values ​​of the corresponding category. When the data is symmetrically distributed, the mean of the variable column is used to fill in the missing values; When the data contains extreme values ​​that cause the mean to fail to reflect the central tendency, the median of the variable column is used to fill in the missing values. For categorical variables in TCM clinical data, when missing values ​​have clear categorical attributes, they are filled with special status strings representing the missing values, such as "unknown" or "not filled". For example, when collecting users' occupational information, if some information is missing, "unknown occupation" is filled in.

[0058] As an example, a logical imputation strategy is adopted for missing values ​​in TCM clinical data. This involves first combining different data tables uploaded by different users, and then determining the imputation result of the missing values ​​by the values ​​of the same variable in different data tables.

[0059] The outlier handling module in the clinical dataset governance toolkit is specifically configured as follows: for numerical variables in TCM clinical data, it employs both quantile-based and medical common sense-based outlier detection methods; for categorical variables in TCM clinical data, it employs a medical common sense-based outlier detection method. Specifically, the quantile method is used to define the outlier range with an upper bound of Q3 + 1.5IQR and a lower bound of Q1 - 1.5IQR (Q1 is the lower quartile, Q3 is the upper quartile, and IQR is the interquartile range). Values ​​exceeding this range are considered outliers. The common sense method is used to detect outliers where values ​​clearly violate medical common sense, such as height > 3m or age > 200 years, are considered outliers. The common sense method is also used to detect outliers where values ​​other than "male" or "female" appear in the gender variable (excluding "other" and "unlabeled").

[0060] The text structuring module in the clinical dataset governance toolkit is specifically configured to extract medical entities from long texts of traditional Chinese medicine clinical data through the following logic: defining domain entity structures and examples, segmenting and cleaning the long text to be processed, constructing entity recognition prompt templates, generating dynamic prompt words, and performing structured processing. Specifically, this includes: Define the domain entity structure and examples, clarify the types of medical entities to be extracted, define the attributes and structure of each entity type, and collect real TCM clinical text data of the corresponding types as examples; The long text to be processed is segmented and cleaned. Based on the semantic structure and format characteristics of the text, the long text of TCM clinical data is divided into multiple independent logical paragraphs. Irrelevant format symbols, duplicate information and noisy text in each paragraph are removed to extract the effective text content. Construct entity recognition prompt templates. Design exclusive prompt templates for different types of medical entities. The templates contain clear entity extraction instructions and corresponding entity extraction examples. Generate dynamic prompts by generating personalized prompts based on the context information of the text to be processed, the target entity type, and the text length. The system performs structured processing, combining dynamic prompts with cleaned long text paragraphs and inputting them into the Deepseek large language model for inference. The model then extracts target medical entity information based on the prompts and outputs structured entity data.

[0061] Specifically, the text structuring module extracts medical entities from long medical texts, including medical history, medical records, physical examinations, surgical records, and adverse events. This includes the following steps: Step 1 defines the domain entity structure and examples. For entity types such as chief complaint, present illness, past medical history, personal history, and family history, physical examination, surgical records, and adverse events, their attributes and structures are clarified, and a large amount of real medical text data is collected as examples to train and optimize the model's ability to understand medical entities; Step 2: Long Text Segmentation and Cleaning. First, based on the semantic structure and format characteristics of the text, the long text is segmented into multiple logical paragraphs. Each paragraph corresponds to a relatively independent unit of medical information, such as the chief complaint paragraph, present medical history paragraph, and examination results paragraph. Then, each paragraph is cleaned to remove irrelevant formatting symbols, duplicate information, and noisy text, extracting clean and effective text content to provide high-quality input data for subsequent entity extraction. Step 3: Define Entity Recognition Prompt Templates: To guide the large model in accurately recognizing medical entities, this invention designs a series of Prompt templates for different entity types. For example, for extracting the chief complaint entity, the Prompt template can be designed as "Please extract the patient's chief complaint information from the following text: [Text to be processed]". This explicit prompt helps the large model focus on the task of extracting the target entity. Step 4: Construct dynamic prompts. Based on the context, entity type, and text length of the text to be processed, dynamically generate personalized prompts. For example, for text containing multiple medical histories, dynamic prompts can generate targeted prompts based on the types of medical histories mentioned in the text, such as "Please extract the patient's diabetes and hypertension history information from the following text: [Text to be processed]", thereby further improving the model's accuracy and relevance in identifying target entities. Step 5: Structured Processing. The constructed dynamic prompts are combined with the cleaned long text paragraphs and then input into the Deepseek large language model for inference. Leveraging its powerful language understanding and generation capabilities, the large model extracts target medical entity information from the long text based on the guidance of the dynamic prompts.

[0062] The text standardization module in the clinical dataset governance toolkit is specifically configured to standardize medical terminology in TCM clinical data through the logic of constructing a standard classification dataset, segmenting and cleaning the text to be processed, constructing a text classification Prompt template, generating dynamic prompt words, and performing standardization processing. Specifically, this includes: A standard classification dataset was constructed by collecting common TCM clinical short texts, diagnostic names, surgical names, and TCM syndrome descriptions in the medical field, and classifying and labeling them in detail according to industry standards to form a standard classification dataset. The text to be processed is segmented and cleaned by splitting the clinical text of traditional Chinese medicine into several independent short text units, removing irrelevant characters, punctuation marks, extra spaces and error messages from the text, and unifying the data format in the text and converting it to lowercase. Build a text classification Prompt template and design a custom Prompt template that includes clear instructions and standardized examples of TCM terminology; Generate dynamic prompts by adjusting and generating targeted prompts in real time based on the characteristics and context of the text to be processed. Standardization processing is performed, combining the segmented and cleaned text with dynamic prompts, and inputting it into the deepseek large language model for inference. The model then outputs standardized medical terminology results that conform to industry standards.

[0063] Specifically, the text standardization module standardizes medical terms such as diagnosis and surgery in clinical data. This part mainly includes the following steps: Step 1: Construct a standard classification dataset. This dataset should cover common medical information such as short texts, diagnosis names, and surgical procedure names, and should be meticulously classified and labeled. Step 2: Text Segmentation and Cleaning. First, the text to be processed is segmented into several independent short text units. Then, irrelevant characters, punctuation marks, redundant spaces, and any possible errors are removed. Additionally, further text processing can be performed, such as standardizing number formats and converting all text to lowercase. Step 3: Construct a prompt template. To guide the large model in accurately classifying and standardizing medical texts, a dedicated text classification prompt template needs to be designed. This template should contain clear instructions and examples, explicitly informing the large model of the tasks it needs to complete and the expected output format; Step 4 involves constructing a method for dynamic prompts to enhance the adaptability and flexibility of the large model to different medical texts. Dynamic prompts can be adjusted and generated in real time based on the features and contextual information of the text being processed. For example, when encountering a complex text containing multiple diagnostic names, dynamic prompts can generate more targeted prompts based on information such as the disease type and symptoms mentioned in the text, guiding the large model to more accurately classify and standardize each diagnostic name. Step 5: Standardization. The segmented, cleaned, and dynamically prompted medical text is input into the large-scale model for inference. Based on its powerful language modeling capabilities and extensive learning of medical knowledge, the large-scale model analyzes and processes the input text to generate standardized results.

[0064] The normalization module addresses the potential for different units in laboratory test indicators within medical data. It first normalizes the units of the variables to a single unit. Then, it provides maximum-minimum normalization for numerical variables, bringing them to the range of 0-1. Simultaneously, it provides mean-standard deviation normalization, normalizing the variables to a distribution with a mean of 0 and a standard deviation of 1. Different normalization methods are suitable for different types of data or analytical methods.

[0065] Example 5 As another preferred embodiment of the present invention, this embodiment further elaborates and supplements the technical solution of the present invention based on the above-described Embodiments 1, 2 / 3, or 4. In this embodiment, the specific configuration of the data governance intelligent agent is described in detail.

[0066] As one implementation method of this embodiment, the data governance intelligent agent's dedicated fine-tuning dataset for TCM efficacy evaluation first collects an overview of the datasets used in TCM clinical efficacy evaluation tasks as input, and outputs the governance methods and processes adopted for the datasets. The input and output are then combined according to a set template, and the combined input-output pair constitutes the dedicated fine-tuning dataset for TCM efficacy evaluation. The set template is... <|im start |>system prompt<|im end |> <|im start |>user input<|im end |> <|im start |>assistant output<|im end |>.

[0067] The data governance agent employs LoRa high-efficiency parameter fine-tuning of the basic large language model. Specifically, it uses the LoRa high-efficiency parameter fine-tuning method to fine-tune and update the basic model parameters. During the model parameter fine-tuning phase, the model's loss function is calculated as follows:

[0068] In the formula, N represents the sample size. This represents the predicted probability that the i-th sample belongs to the true class; the process is iterated until the loss value converges. Specifically, qwen2.5-14b-instruct is selected as the base model, and an efficient parameter fine-tuning method based on LoRa is used to fine-tune and update the model parameters.

[0069] As another embodiment of this invention, please refer to the appendix to the specification. Figure 4 As shown, the data governance process of the data governance intelligent agent is as follows: Step 1 uses the analysis report of the original clinical data as input to the data governance intelligent agent and outputs the process instructions for data governance. Step 2: Then, based on the process instructions and the clinical dataset governance tool library, the governance tools are invoked. Step 3: During the data retrieval process, the raw data is used as the data input, and the input parameters are parsed from the corresponding process instructions. Then, the data is passed to the corresponding governance tool and the governance results are output. Step 4: Repeat steps 2-3 based on the process instructions until the entire data governance process is completed and the governed data is output. Step 5 involves data quality control of the processed data. This quality control primarily includes the following processes: variable completeness, missing value detection, outlier detection, and sensitive information detection. If the processed data meets the requirements, the data overview analysis module will be used to analyze the processed data and generate a data governance report (including the processing flow, methods, and results). If the data does not meet the requirements, the current overview will be re-entered into the numerical governance model, and the previous steps will continue.

Claims

1. A system for automatic processing of traditional Chinese medicine clinical data based on a medical large language model multi-agent, characterized in that: The system comprises a task understanding agent, a clinical data set governance tool library and a data governance agent, which are communicatively connected and cooperatively realize automatic processing of clinical data in a traditional Chinese medicine clinical efficacy evaluation scenario; The task understanding agent is configured to be formed through training data set expansion, knowledge distillation weighted fusion training and medical expert preference alignment training; The task understanding agent is configured to receive a user inputted task related to traditional Chinese medicine clinical efficacy evaluation, identify clinical data requirements corresponding to the task and feed back to the user; The clinical data uploaded by the user is parsed in combination with the clinical data requirements to generate an original data analysis report; The clinical data set governance tool library comprises a missing value processing module, an abnormal value processing module, a text structuring module, a text standardization module and a normalization processing module, each of which is configured to perform missing value filling, abnormal value detection and processing, medical entity structuring extraction, traditional Chinese medicine terminology text standardization and data normalization processing on traditional Chinese medicine clinical data; The data governance agent is communicatively connected with the task understanding agent and the clinical data set governance tool library; the data governance agent is configured to be formed through a traditional Chinese medicine efficacy evaluation exclusive fine-tuning data set and a Lora efficient parameter fine-tuning basic large language model; the data governance agent is configured to receive the original data analysis report and the parsed data outputted by the task understanding agent, generate a traditional Chinese medicine clinical data governance process instruction adapted to the data, and control the clinical data set governance tool library to perform a governance operation; after the governance operation is completed, the data after the governance is outputted, and the data after the governance is quality inspected; if the quality inspection is qualified, the qualified data is analyzed to generate a data governance report; if the quality inspection is unqualified, a data governance process instruction is regenerated according to the current profile and the clinical data set governance tool library is controlled to perform a governance operation until the data after the governance is qualified; The system cooperatively analyzes user tasks and data through the task understanding agent, generates a governance process through the data governance agent, and executes a governance operation through the clinical data set governance tool library, thereby completing automatic processing of traditional Chinese medicine clinical data and outputting data after the governance and a data governance report meeting requirements of traditional Chinese medicine clinical efficacy evaluation.

2. The medical large language model multi-agent-based traditional Chinese medicine clinical data automatic processing system of claim 1, wherein: The training data set expansion adopts a large language model A expansion question method and a large language model B effective input screening method to form an input-output pair data set comprising original input and expanded input; The original input is a task requirement of a researcher in actual clinical traditional Chinese medicine efficacy evaluation, and the output is typical data variable dimension information used by the researcher when processing the task requirement, including category type variable, numerical type variable, text, type of each variable and necessity; The large language model A expansion question method refers to expanding the task requirement in the original input from the question level through the large language model A to obtain n task descriptions with the same semantics but different questions. The large language model B screening effective input refers to scoring the expanded n task descriptions by using the large language model B, filtering out low-score task descriptions, and obtaining k high-score task descriptions.

3. The medical large language model multi-agent-based traditional Chinese medicine clinical data automatic processing system of claim 2, characterized in that: The knowledge distillation weighted fusion training of the task understanding agent is specifically configured as follows: a teacher model with the ability of clinical knowledge transfer of traditional Chinese medicine and a student model with the ability of lightweight reasoning and task response are selected, and based on the Lora efficient parameter fine-tuning technology, a loss function combining the knowledge weight of the teacher model and the real data distribution weight is used for training. The loss function is where N represents the total number of training samples; V represents the total number of model output categories, represents the probability value of the jth category predicted by the student model for the ith sample, represents the probability value of the jth category predicted by the teacher model for the ith sample, is the true label value of the jth category, taking values of 0 or 1; represents the knowledge weight of the teacher model, represents the real data distribution weight.

4. The medical large language model multi-agent-based traditional Chinese medicine clinical data automatic processing system of claim 3, wherein: The direct preference alignment training of the medical expert experience of the task understanding agent is specifically configured as follows: The evaluation requirements related to the evaluation of the clinical efficacy of traditional Chinese medicine are selected as the basic data, the model generated based on the knowledge distillation weighted fusion training is combined with the superior and inferior classification results of the replies by the professionals with experience in clinical efficacy evaluation, and a preference data set of evaluation requirements-superior replies-inferior replies is constructed; based on the model obtained by the knowledge distillation weighted fusion training, a direct preference optimization loss function is used, a reference model with fixed parameters is combined, and preference learning training is completed by comparing the predicted probabilities of the superior replies and the inferior replies by the model. The direct preference optimization loss function is In the formula, represents a sigmoid function, represents a hyperparameter, represents a reference model, the parameters of the reference model are the same as the parameters of , and the parameters are not updated in the preference alignment process; represents the current model to be trained, that is, the model trained based on the knowledge distillation and weighted fusion described above; represents the reply judged as better by the professional with clinical efficacy evaluation experience for the task task; represents the reply judged as suboptimal by the professional with clinical efficacy evaluation experience for the task task.

5. The medical large language model multi-agent-based traditional Chinese medicine clinical data automatic processing system according to any one of claims 1-4, characterized in that: The task understanding agent analyzes the clinical data uploaded by the user in combination with the clinical data requirements, and is specifically configured as follows: after receiving the clinical data uploaded by the user, first, the variable types in the data are divided to obtain numerical variables, categorical variables and text type variables; for numerical variables, the units of laboratory variables in them are unified, and the numerical variables are converted into floating point type; for categorical variables, the ID information is desensitized first, and then the string variables are mapped and encoded; For text type variables, the hospital information and personal privacy information in them are desensitized; Subsequently, data profile statistics are carried out for different types of variables, including patient total statistics, missing value proportion calculation of different variables, abnormal value detection of different variables, length distribution statistics of text information, distribution statistics of numerical variables, and distribution statistics of categorical variables; finally, the corresponding original data analysis report is generated.

6. The medical large language model multi-agent-based traditional Chinese medicine clinical data automatic processing system according to any one of claims 1-4, characterized in that: The missing value processing module in the clinical data set governance tool library is specifically configured as follows: for the missing values in the traditional Chinese medicine clinical data, a logical filling strategy is adopted, that is, different data tables uploaded by different users are first combined, and then the filling results of the missing values are determined by the values of the same variable in different data tables.

7. The medical large language model multi-agent-based traditional Chinese medicine clinical data automatic processing system according to any one of claims 1-4, characterized in that: The abnormal value processing module in the clinical data set governance tool library is specifically configured as follows: for numerical variables in traditional Chinese medicine clinical data, a quantile-based abnormal value detection method and a medical common sense-based abnormal value detection method are used; for categorical variables in traditional Chinese medicine clinical data, a medical common sense-based abnormal value detection method is used.

8. The medical large language model multi-agent-based traditional Chinese medicine clinical data automatic processing system according to any one of claims 1-4, characterized in that: The text structuring module in the clinical data set governance tool library is specifically configured to extract medical entities from long text of traditional Chinese medicine clinical data by defining field entity structure and examples, segmenting and cleaning long text to be processed, constructing an entity recognition Prompt template, generating dynamic prompts, and executing structured processing logic. Define the field entity structure and examples, clearly define the type of medical entities to be extracted, define the attributes and structure of each entity type, and collect real traditional Chinese medicine clinical text data of the corresponding type as examples. Segment and clean the long text to be processed according to the text semantic structure and format characteristics, divide the long text of traditional Chinese medicine clinical data into multiple independent logical paragraphs, remove irrelevant format symbols, redundant information and noise text in each paragraph, and extract the effective text content. Construct an entity recognition Prompt template, design a dedicated Prompt template for different types of medical entities, and the template includes clear entity extraction instructions and extraction examples of the corresponding entity. Generate dynamic prompts based on the context information, target entity type and text length of the text to be processed. Perform structured processing, combine the dynamic prompts with the cleaned long text paragraphs, input them into the large language model for reasoning, and extract the target medical entity information guided by the prompts, output structured entity data.

9. The medical large language model multi-agent-based traditional Chinese medicine clinical data automatic processing system according to any one of claims 1-4, characterized in that: The text standardization module in the clinical data set governance tool library is specifically configured to standardize medical terminology in traditional Chinese medicine clinical data by constructing a standard classification data set, segmenting and cleaning the text to be processed, constructing a text classification Prompt template, generating dynamic prompts, and executing standardization processing logic, specifically including: Construct a standard classification data set, collect common short texts of traditional Chinese medicine clinical data, diagnosis names, surgical names and TCM syndrome description information in the medical field, and classify them according to industry standards to form a standard classification data set. Segment and clean the text to be processed, split the traditional Chinese medicine clinical text to be processed into several independent short text units, remove irrelevant characters, punctuation marks, extra spaces and errors in the text, and unify the data format in the text and convert it to lowercase. Construct a text classification Prompt template, design a dedicated Prompt template containing clear instructions and standardization examples of traditional Chinese medicine terminology. Generate dynamic prompts based on the characteristics and context information of the text to be processed. Perform standardization processing, combine the segmented and cleaned text with dynamic prompts, input them into the large language model for reasoning, and output standardized medical terminology results that meet industry standards.

10. The medical large language model multi-agent-based traditional Chinese medicine clinical data automatic processing system of any one of claims 1-4, characterized in that: The traditional Chinese medicine efficacy evaluation exclusive fine-tuning data set in the data governance intelligent agent first collects the data set profile used in the traditional Chinese medicine clinical efficacy evaluation task as input, and the governance method and process used in the data set as output, and combines the input and output according to the set template. The combined input and output pair is the traditional Chinese medicine efficacy evaluation exclusive fine-tuning data set.

11. The medical large language model multi-agent-based traditional Chinese medicine clinical data automatic processing system according to any one of claims 1-4, characterized in that: The data governance intelligent agent adopts a Lora efficient parameter fine-tuning basic large language model, and is specifically configured as follows: a Lora efficient parameter fine-tuning method is used to fine-tune and update parameters of a basic model; in a model parameter fine-tuning stage, a loss function of the model is calculated as In the formula, N represents the number of samples, represents the prediction probability of the i-th sample belonging to the true category; and finally, iteration is continuously performed until the loss value converges.

Images