Automatic Meta analysis method and device based on large language model information extraction

Abstract

The invention discloses an automatic Meta analysis method and device based on large language model information extraction, and relates to the field of Meta analysis, the device comprises an analysis tool module, a memory module, a planning module and a Meta-Agent module; the Meta-Agent module is used for determining a PICO retrieval formula according to a target research subject, and calling a thinking chain tool to perform analysis task decomposition; for any analysis task, calling a corresponding tool from the analysis tool module to execute the corresponding analysis task; in the process of executing the analysis task, calling the long-term memory tool and the short-term memory tool to store and call task analysis data; after the analysis task is executed, a self-evaluation tool and a self-correction tool are called for decision optimization; and after the analysis task chain is executed, performing Meta analysis calculation according to the extracted control group data and intervention group data. According to the method and the device, the complex Meta analysis task can be automatically and intelligently processed.

Description

Technical Field

[0001] The present application relates to the field of Meta analysis, and in particular to an automated Meta analysis method and device based on large language model information extraction. Background Art

[0002] Randomized Controlled Trial (RCT) is a research design method aimed at evaluating the effect of a certain therapy or drug in medical and health services, and is generally considered the "gold standard" for evaluating the effectiveness of interventions. Due to the limitations of sample size, research conditions, and research design, a single RCT study is often difficult to independently draw sufficiently robust conclusions. At this time, systematic review and / or meta-analysis, as a comprehensive statistical method, can provide more efficient and more powerful conclusions by aggregating data from multiple independent studies.

[0003] Although systematic review / meta-analysis is a powerful research method and statistical tool, the credibility and reliability of the integrated results depend on whether high-quality RCT evidence is included. These RCT evidences are the cornerstone of systematic review / meta-analysis and determine the reliability of the final analysis results. However, the current routine process of systematic review / meta-analysis still mainly relies on researchers to manually screen literature, extract information, evaluate the quality of evidence, and analyze and integrate data. The whole process is not only complicated and time-consuming, but also susceptible to human errors. Researchers need to manually screen studies that meet the inclusion and exclusion criteria from a large number of literature, read and analyze the literature one by one, and manually extract the data required for systematic review / meta-analysis. This process is not only time-consuming and labor-intensive, but also has high requirements for the evidence-based methodology, professional background, and research experience of scientific researchers. Once there is an omission or misjudgment, important data may be ignored or misinterpreted, which in turn affects the accuracy of the results of the systematic review / meta-analysis. Summary of the invention

[0004] The purpose of this application is to provide an automated Meta-analysis method and device based on large language model information extraction, which can automatically and intelligently process complex Meta-analysis tasks.

[0005] To achieve the above objectives, this application provides the following solutions:

[0006] In a first aspect, the present application provides an automated Meta-analysis method based on large language model information extraction, comprising:

[0007] Obtain target research topics;

[0008] Determine the PICO search formula according to the target research topic;

[0009] Calling the thinking chain tool to decompose the PICO search formula into analysis tasks to obtain an analysis task chain; the analysis tasks in the analysis task chain include a literature search task, a PICO condition verification task, a PICO element extraction task and an RCT literature quality evaluation task;

[0010] For any analysis task in the analysis task chain, the corresponding tool is called from the analysis tool module to execute the corresponding analysis task; the analysis tool module includes a storage document retrieval tool, a PICO condition verification tool, a PICO element extraction tool and an RCT document quality evaluation tool;

[0011] In the process of executing the analysis task, calling the long-term memory tool and the short-term memory tool to store and call the task analysis data to link the task context information and optimize the execution process of the analysis task;

[0012] After executing the analysis task, calling the self-evaluation tool and the self-correction tool to optimize the decision and store it in the thinking chain tool;

[0013] After executing the analysis task chain, Meta-analysis calculations are performed based on the extracted control group data and intervention group data to obtain the Meta-analysis results.

[0014] In a second aspect, the present application provides an automated Meta-analysis device based on large language model information extraction, including an analysis tool module, a memory module, a planning module, and a Meta-Agent module;

[0015] The analysis tool module is used to store literature retrieval tools, PICO condition verification tools, PICO element extraction tools and RCT literature quality evaluation tools;

[0016] The memory module is used to store long-term memory tools and short-term memory tools;

[0017] The planning module is used to: store thinking chain tools, self-evaluation tools and self-correction tools;

[0018] The Meta-Agent module is used to: obtain the target research topic; determine the PICO search formula according to the target research topic; call the thinking chain tool to perform analysis task decomposition on the PICO search formula to obtain an analysis task chain; for any analysis task in the analysis task chain, call the corresponding tool from the analysis tool module to execute the corresponding analysis task; in the process of executing the analysis task, call the long-term memory tool and the short-term memory tool to store and call the task analysis data to link the task context information and optimize the execution process of the analysis task; after executing the analysis task, call the self-evaluation tool and the self-correction tool to optimize the decision and store them in the thinking chain tool; after executing the analysis task chain, perform Meta analysis calculation based on the extracted control group data and intervention group data to obtain Meta analysis results; wherein the analysis tasks in the analysis task chain include literature retrieval tasks, PICO condition verification tasks, PICO element extraction tasks and RCT literature quality evaluation tasks.

[0019] According to the specific embodiments provided by the present application, the present application has the following technical effects: the present application provides an automated Meta-analysis method and device based on information extraction of a large language model, and the Meta-Agent module realizes highly automated operations for document retrieval tasks, PICO condition verification tasks, PICO element extraction tasks, RCT document quality evaluation tasks and Meta-analysis, reducing the steps of manual intervention and avoiding the bias caused by human errors. The Meta-Agent module can call long-term memory tools and short-term memory tools, and can optimize the execution of the current system evaluation / Meta-analysis task based on past document records and contextual information of the current analysis task, ensuring that each step is efficiently operated under the support of complete background information, thereby effectively managing information flow. The Meta-Agent module can also call self-evaluation tools and self-correction tools, and has self-reflection and self-correction functions. Whenever an error occurs in data extraction or analysis, it can timely adjust its operation strategy by comparing the differences between known results and the currently executed tasks, ensuring the efficiency and accuracy of the entire system evaluation / Meta-analysis process. Finally, the thinking chain tool is used to record each step of the analysis ideas to ensure that the analysis process is traceable, transparent and efficient.

[0020] In summary, this application combines the agent's autonomous decision-making ability, the information extraction technology of the large language model, and the flexible task decomposition and optimization mechanism, which can not only improve the accuracy and efficiency of system evaluation / Meta-analysis, but also reduce manual intervention, save labor costs, and improve scientific research efficiency, so that researchers can conduct complex medical research more efficiently. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0022] Figure 1 It is a structural schematic diagram of an automated Meta analysis device based on large language model information extraction in one embodiment of the present application. DETAILED DESCRIPTION

[0023] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0024] With the rapid development of artificial intelligence (AI) and natural language processing (NLP) technologies, automated tools for literature screening and data extraction have gradually emerged. The development of large language models (LLMs) has enabled machines to automatically extract meaningful information from massive amounts of text, laying the foundation for the automation of systematic reviews / meta-analyses.

[0025] Based on this, the present application provides an automated Meta analysis method and device based on large language model information extraction driven by a large language model using Agent technology, wherein Agent technology refers to an intelligent entity with autonomous perception, task parsing and execution capabilities, capable of self-decision-making and self-optimization in complex task environments. Unlike traditional artificial automation systems, Agent has higher flexibility and adaptability, can make dynamic adjustments according to different task environments, and execute more complex operation processes.

[0026] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0027] In an exemplary embodiment, Figure 1As shown, an automated Meta-analysis device based on large language model information extraction is provided, including an analysis tool module, a memory module, a planning module, a Meta-Agent module and an action module. Specifically, the analysis tool module is used to store literature retrieval tools, PICO condition verification tools, PICO element extraction tools, RCT literature quality evaluation tools and Meta-analysis visualization tools; the memory module is used to store long-term memory tools and short-term memory tools; the planning module is used to store thinking chain tools, self-evaluation tools and self-correction tools; the action module is used to drive the execution of analysis tasks.

[0028] The Meta-Agent module is used to: obtain the target research topic; determine the PICO search formula according to the target research topic; call the thinking chain tool to perform analysis task decomposition on the PICO search formula to obtain an analysis task chain; for any analysis task in the analysis task chain, call the corresponding tool from the analysis tool module to execute the corresponding analysis task; in the process of executing the analysis task, call the long-term memory tool and the short-term memory tool to store and call the task analysis data to link the task context information and optimize the execution process of the analysis task; after executing the analysis task, call the self-evaluation tool and the self-correction tool to optimize the decision and store them in the thinking chain tool; after any of the analysis tasks is executed, call the Meta analysis visualization tool to perform statistical calculations on the analysis result data of the analysis task and generate charts; after executing the analysis task chain, perform Meta analysis calculations based on the extracted control group data and intervention group data to obtain Meta analysis results; wherein the analysis tasks in the analysis task chain include literature retrieval tasks, PICO condition verification tasks, PICO element extraction tasks and RCT literature quality evaluation tasks.

[0029] In a specific application example, the specific functions of the various tools in the analysis tool module are as follows:

[0030] The document search tool is used to: use information retrieval technology to generate the optimal search strategy based on the PICO search formula, and connect to the preset database for search to obtain search results; specifically, use information retrieval technology to generate the optimal search strategy (such as Boolean logic, keyword expansion, etc.) based on the needs input by the user (such as PICO conditions), and connect to external Chinese and English document databases (such as PubMed, Cochrane, China National Knowledge Infrastructure, etc.) for search. The search algorithms that can be used in the search process include document matching models such as TF-IDF and BM25.

[0031] The PICO condition verification tool is used to: verify the search results based on the preset rule matching algorithm and the preset machine learning classifier to determine whether the search results meet the preset PICO standards; mark all search results that meet the preset PICO standards as included documents and store them in the memory module; specifically, based on the preset rule matching algorithm and the preset machine learning classifier, the documents (i.e., search results) extracted according to the PICO framework (patients, interventions, controls, results) need to be matched and verified with the preset PICO standards, and screened at the same time. The screening process is divided into primary screening and secondary screening. The primary screening is carried out according to the abstract of the document; the secondary screening requires further parsing of the document in PDF format, and after reading the full text, screening is based on the inclusion and exclusion criteria (i.e., the preset PICO criteria). In the screening process, the rule engine and classification model will be used to evaluate whether the document meets the inclusion and exclusion criteria.

[0032] The PICO element extraction tool is used to extract PICO elements from the included literature using information extraction technology; the PICO elements include patient characteristics, intervention measures, intervention time, control methods and outcome indicators. Specifically, information extraction technology (including named entity recognition and relationship extraction) is used to automatically extract PICO elements from the included literature and information extraction tables based on user needs. In the process of extracting PICO elements, deep learning models (such as BERT, GPT) can be used to enhance the accuracy of such information extraction tasks.

[0033] The RCT literature quality evaluation tool is used to: identify and find out the literature content corresponding to each evaluation dimension from the included literature according to the preset RCT literature quality evaluation standards, and determine the corresponding quality evaluation results; specifically, based on the current tools for RCT literature quality evaluation, such as the Cochrane risk bias assessment tool, automatically identify and find out the specific descriptive content corresponding to the 7 evaluation dimensions in the evaluation tool from the included literature, and give the evaluation results according to the evaluation standards.

[0034] The Meta-analysis visualization tool is used to: display the extracted Meta-analysis data (i.e., the analysis result data after the analysis task is completed) through statistical calculation and chart generation tools based on data visualization technology; commonly used tools include: ForestPlot (forest map) to display the results of different studies, customized visualization charts based on Python libraries (such as Matplotlib, Seaborn), and interactive design of charts using front-end visualization tools such as D3.js or Plotly.

[0035] In a specific application example, the memory module is responsible for storing and retrieving the information required by the device. Among them, the long-term memory tool allows the agent to store the literature and research data included in the previous analysis, so that it can better utilize the existing resources in future systematic evaluation / meta-analysis tasks and avoid duplication of work. The short-term memory tool can help the agent maintain contextual coherence in the current dialogue or task execution, so that it can respond to task requirements more accurately and make effective self-adjustments. This combination of long-term memory and short-term memory enables the agent to make better autonomous decisions in complex task environments and optimize its operating procedures according to the changing environment and task requirements. The specific functions of each tool in the memory module are as follows:

[0036] The long-term memory tool is used to store all the included documents in long-term memory based on knowledge graph and database management technology (such as NoSQL, GraphDB) during the execution of the PICO condition verification task. Specifically, the complete document information retrieved from the document library is stored in the long-term memory, and the knowledge graph technology helps organize the key content of these documents for subsequent query and analysis. Long-term memory is managed in the form of an external knowledge base.

[0037] The short-term memory tool is used to: in the process of executing the PICO condition verification task, based on the dynamic cache management technology, perform short-term memory storage on the currently processed included documents; in the process of executing the PICO element extraction task, based on the dynamic cache management technology, provide the Meta-Agent module to call and extract task context information, and perform short-term memory storage on the PICO elements; in the process of executing the RCT document quality evaluation task, based on the dynamic cache management technology, provide the Meta-Agent module to call and extract task context information, and perform short-term memory storage on the included documents and corresponding quality evaluation results.

[0038] The short-term memory involved in the short-term memory tool is the information retrieved and processed in real time during the execution of various analytical tasks, involving the memory of the current conversation or analysis context. Short-term memory is used to process temporary context information required by the current task and is only effective in the current task context, similar to the RAM of a computer.

[0039] In a specific application example, the planning module is the decision-making center of the device's intelligent behavior, and it implements the optimal decision for the task process based on the AI ​​planning algorithm. The planning module has a built-in self-reflection function that can automatically detect and correct errors or anomalies that may occur during the analysis process. During data extraction and analysis, real-time feedback is provided to identify potential problems and adjust the execution process to ensure that the results of the system evaluation / meta-analysis are accurate. The specific functions of the various tools in the planning module are as follows:

[0040] The thinking chain tool is used to: based on the Bayesian network or Markov decision process, use the task decomposition algorithm to decompose the PICO search formula into analysis tasks to obtain an analysis task chain; coordinate the order and dependencies of each analysis task in the analysis task chain. In addition, in the process of decomposing the analysis tasks, it is necessary to coordinate the order and dependencies of each analysis task. That is, by decomposing the complex system evaluation / Meta-analysis task into multiple manageable subtasks through the thinking chain tool, and gradually executing and tracking the progress of each subtask, the orderly advancement of the task and the overall synergy can be ensured. This decomposition and execution strategy makes the entire device have strong flexibility and scalability when handling complex tasks.

[0041] The self-evaluation tool works in conjunction with the self-correction tool to: after each analysis task in the analysis task chain is completed, based on the generative adversarial network and reinforcement learning technology, self-reflect on the analysis result data of the analysis task, identify potential errors or optimization space, and adjust the task execution strategy in real time based on the feedback mechanism. This process is similar to the "exploration-exploitation" strategy in reinforcement learning.

[0042] In a specific application example, the Meta-Agent module is the core brain of the entire device. By scheduling various modules, the full-process automated Meta analysis based on large language model information extraction is realized. The specific functions of each unit in the Meta-Agent module are as follows:

[0043] The natural language processing unit is used to: obtain the target research topic; use the natural language processing algorithm to determine the PICO search formula according to the target research topic; call the thinking chain tool to perform analysis task decomposition on the PICO search formula to obtain an analysis task chain. Specifically, the natural language processing algorithm is used to understand the target research topic and convert it into a machine-executable task. The ability of the natural language processing algorithm enables it to extract complex medical terms, recognize semantics, and construct appropriate query statements.

[0044] The multi-task learning unit is used to: execute the document retrieval task, the PICO condition verification task, the PICO element extraction task and the RCT document quality evaluation task in parallel; that is, the multi-task learning unit has parallel processing capabilities and multi-task learning coordination capabilities.

[0045] The reinforcement learning unit is used to: after executing the analysis task, call the self-evaluation tool and the self-correction tool, optimize the decision based on the reinforcement learning algorithm, and store it in the thought chain tool. In this unit, the task execution strategy is continuously optimized based on the reinforcement learning algorithm to achieve an intelligent feedback loop, and the optimal task path is selected through the reward and punishment mechanism.

[0046] In a specific application example, the action module is the technical driver for the device to perform specific tasks, and is responsible for putting the strategies generated by the planning module into practice, including retrieving literature from the database, extracting key information from the literature, and inputting the data into the Meta-analysis module. The technologies involved in the action module include: batch processing and parallel computing: the processing of literature data is often large-scale, so a parallel computing framework (such as Dask, Ray, etc.) is required to increase the processing speed, especially when processing a large number of literature and complex analysis; automated data analysis: use automated statistical analysis tools (such as meta-analytic models) to complete statistical modeling work, and output visualization results through automated report generation tools.

[0047] In summary, the automated Meta-analysis device based on large language model information extraction provided by the present application has the following advantages: (1) Full automation: the entire process from literature retrieval, literature screening, quality evaluation, data extraction and Meta-analysis is automated, reducing manual intervention. (2) Self-reflection and self-correction: the self-execution process can be continuously optimized through self-criticism and self-correction mechanisms to improve the accuracy and reliability of analysis. (3) Visual analysis results: not only complete Meta-analysis, but also generate visual results to help users intuitively understand the analysis data. (4) Autonomous planning and execution of the intelligent agent system: the Meta-Agent module is used as the brain of the device, which can independently decide the task execution steps and implement specific operations through the tool set. Finally, the combination of the above four settings enables the device of the present application to not only automatically extract data, but also directly input the extracted data into the Meta-analysis tool for calculation, and generate high-quality visual results. The entire process does not require manual intervention, and automation and integration are achieved from data extraction to the final system evaluation / Meta-analysis result display. Obviously, the present application is highly intelligent and innovative, and can automatically and intelligently handle complex Meta-analysis tasks.

[0048] Based on the same inventive concept, the embodiment of the present application also provides an automated Meta-analysis method based on large language model information extraction. The implementation solution provided by the method is similar to the implementation solution recorded in the above-mentioned device, so the specific limitations in one or more method embodiments provided below can refer to the limitations on the device above, and will not be repeated here. In an application example, the automated Meta-analysis method based on large language model information extraction includes:

[0049] Step 100, obtaining a target research topic, such as <non-drug intervention for cardiovascular disease>.

[0050] Step 200, determine the PICO search formula according to the target research topic; in an application example, step 200 includes: based on the natural language processing algorithm, write the PICO search formula according to <population>, <intervention>, <control>, and <result>. In this step, it is necessary to ensure that the search formula accurately covers all important variables; call the self-evaluation tool and the self-correction tool to self-reflect on the PICO search formula to check the completeness and accuracy of the PICO search formula to ensure that key intervention or result variables are not omitted, and obtain the corresponding search formula generation adjustment information; store the search formula generation adjustment information in the thinking chain tool. In actual applications, the thinking chain tool can also record each step of the thinking process of generating the PICO search formula (such as the reason for selecting a specific population or intervention) to help backtrack and adjust in subsequent steps. This step is fully automated, getting rid of the tedious and error-prone limitations of traditional manual screening.

[0051] Step 300, calling the thinking chain tool to decompose the PICO search formula into analysis tasks to obtain an analysis task chain; the analysis tasks in the analysis task chain include a literature retrieval task, a PICO condition verification task, a PICO element extraction task and an RCT literature quality evaluation task.

[0052] Step 400, for any analysis task in the analysis task chain, call the corresponding tool from the analysis tool module to execute the corresponding analysis task; the analysis tool module includes a storage document retrieval tool, a PICO condition verification tool, a PICO element extraction tool and an RCT document quality evaluation tool.

[0053] Step 500, during the execution of the analysis task, long-term memory tools and short-term memory tools are called to store and call task analysis data to link task context information and optimize the execution process of the analysis task.

[0054] Step 600, after executing the analysis task, calling the self-evaluation tool and the self-correction tool to optimize the decision and store it in the thinking chain tool.

[0055] In actual applications, step 500 and step 600 can work together to complete a task. When the analysis task is a literature retrieval task, the literature retrieval tool is called, and based on the information retrieval technology, the PICO retrieval formula is used to connect to an external database (such as PubMed, Cochrane, etc.) to perform literature retrieval; the self-evaluation tool and the self-correction tool are called, and advanced retrieval algorithms, including TF-IDF, BM25 and other literature matching models, are used to evaluate the retrieval results, and the PICO retrieval formula is adjusted and optimized based on the situation that the literature retrieval results are too few or irrelevant, and the reasons for the adjustment are recorded and then stored in the thinking chain tool for subsequent optimization of the retrieval process.

[0056] When the analysis task is a PICO condition verification task, the PICO condition verification tool is called to verify the search results based on the preset rule matching algorithm and the preset machine learning classifier to determine whether the search results meet the preset PICO standards; the long-term memory tool and the short-term memory tool of the memory module are called to mark all the search results that meet the preset PICO standards as included documents, and store them in the long-term memory tool as a reliable theoretical basis for future reference; at the same time, the currently processed included documents are stored in the short-term memory tool for subsequent data extraction and analysis; after the verification is completed, the self-evaluation tool and the self-correction tool are called to self-reflect on the included documents. If the analysis result has a large deviation, the data processing and model selection process is reviewed, and the adjustment ideas are recorded after confirming that the steps are correct, so as to obtain the corresponding verification strategy adjustment information; the verification strategy adjustment information is stored in the thinking chain tool, that is, the selection criteria and logic when screening documents are stored in the thinking chain tool to ensure that the screening decision can be explained based on the chain in the future. It should be noted that in the process of self-reflection, there is no need to deliberately correct the results. As long as the data entry and method are correct, negative or positive results should be accepted to maintain the authenticity and validity of the data.

[0057] When the analysis task is an RCT document quality evaluation task, the RCT document quality evaluation tool and the short-term memory tool are called to identify and find the document content corresponding to each evaluation dimension from the included documents according to the preset RCT document quality evaluation standard, and determine the corresponding quality evaluation results; during the evaluation process, the short-term memory tool of the memory module is called to store the included documents and the corresponding quality evaluation results in the short-term memory tool for subsequent extraction and proofreading of the evaluation results; after the evaluation is completed, the self-evaluation tool and the self-correction tool are called to self-reflect on the quality evaluation results and obtain the corresponding document evaluation strategy adjustment information; the document evaluation strategy adjustment information is stored in the thinking chain tool, that is, the evaluation standards and logic during the document evaluation are stored in the thinking chain tool to ensure that the screening decision can be explained based on the chain in the future. In the steps of this analysis task, the document content is automatically parsed through the large language model, the specific content in the RCT corresponding to the document quality evaluation item is accurately matched, and the evaluation results are given according to the evaluation standards, ensuring the efficiency and convenience of the evaluation process and reducing the subjective bias and errors in manual operation.

[0058] When the analysis task is a PICO element extraction task, the PICO element extraction tool and the short-term memory tool are called, and information extraction technology is used to extract PICO elements from the included literature, specifically extracting the intervention group and control group data in the qualified literature, including sample size, intervention measures, intervention time, frequency and main outcome indicators; the PICO elements are stored in the short-term memory tool for easy modification at any time; the self-evaluation tool and the self-correction tool are called to self-reflect on the PICO elements, reflect on whether the extracted data is complete and accurate, ensure that no important information is omitted, and obtain corresponding extraction strategy adjustment information; the extraction strategy adjustment information is stored in the thinking chain tool, that is, the logic and reasons for data extraction are stored in the thinking chain tool to help maintain consistency in subsequent Meta-analysis; the PICO elements include patient characteristics, intervention measures, intervention time, control method and outcome indicators. In this step, the large language model is used to automatically parse the literature content, accurately extract key information such as the intervention group, control group, sample size, effect index, etc. in the randomized controlled trial (RCT), ensure the efficiency and accuracy of the extraction process, and reduce subjective bias and errors in manual operation.

[0059] Step 700, after executing the analysis task chain, perform Meta analysis calculation based on the extracted control group data and intervention group data to obtain Meta analysis results.

[0060] In a specific application, the Meta-analysis visualization tool is called to analyze the results of the extracted control group data and intervention group data, and to visualize and display them, and to generate charts (such as forest charts) to better interpret the conclusions. In the Meta-analysis process, the self-evaluation tool and the self-correction tool can also be called to reflect on whether the extracted data meet the assumptions of the Meta-analysis, and when visualizing the results, reflect on whether the charts clearly and accurately convey information. If there are any unclear points, make adjustments in time; if problems are found, adjust the analysis methods or data in time; if the analysis results are highly biased, re-examine the data processing and model selection process, and record the adjustment ideas. Finally, the thinking chain tool records the logical deduction process when generating the visualization results to ensure that the analysis ideas behind the visualization can be clearly explained when explaining.

[0061] In another specific application, the self-evaluation tool and the self-correction tool regularly summarize and analyze the effective strategies and problems encountered in the historical analysis process and the corresponding adjustment processes and methods, and continuously accumulate experience for subsequent analysis, select the optimal task path through the reward and punishment mechanism to achieve reinforcement learning, and continuously optimize the Meta-Agent's task execution strategy throughout the process to achieve an intelligent feedback loop.

[0062] In summary, the present application is based on the automated Meta-analysis method and device for information extraction of a large language model. First, the systematic review / Meta-analysis task is reasonably decomposed to determine its core steps and execution plan. For example, for the literature retrieval task, a retrieval strategy is generated according to the PICO framework (population, intervention, control, outcome), and a literature search is automatically performed in the mainstream Chinese and English literature databases to obtain literature on research issues related to the systematic review / Meta-analysis. Secondly, the retrieved literature is screened according to the preset inclusion and exclusion criteria (i.e., the preset PICO criteria), and studies that do not meet the conditions are automatically excluded. Then, evidence is extracted from the included literature, mainly extracting the relevant PICO framework and data required for the study; in this process, the data of the intervention group and the control group are accurately extracted from the literature using a large language model, including key information such as sample size, research subjects, intervention measures, intervention time, intervention frequency, outcome indicators, etc., which no longer relies on manual extraction, greatly improving the efficiency and accuracy of data extraction. After that, the inclusion threshold is set for the methodological quality of the selected RCTs to ensure the quality of the research; the most commonly used and authoritative tool for RCT quality evaluation is the Cochrane risk bias assessment tool, which automatically identifies and finds the specific descriptions corresponding to the 7 evaluation dimensions in the Cochrane risk bias assessment tool from the literature, and gives the evaluation results according to the evaluation criteria. Finally, after the literature screening, data extraction and quality evaluation are completed, the extracted control group and intervention group data are automatically input into the systematic evaluation / meta-analysis tool (such as PyMeta) for meta-analysis calculation.

[0063] This application combines the agent's autonomous decision-making ability, the information extraction technology of the large language model, and the flexible task decomposition and optimization mechanism, which can not only improve the accuracy and efficiency of system evaluation / Meta-analysis, but also reduce manual intervention, save labor costs, improve scientific research efficiency, and enable researchers to conduct complex medical research more efficiently.

[0064] Thanks to the powerful information extraction capabilities of the large language model, this application can significantly improve the accuracy of key data extraction when extracting RCT evidence. This not only improves the reliability of the results of the systematic review / meta-analysis, but also reduces the interference of human factors. Researchers can focus more on analyzing the results of the systematic review / meta-analysis and formulating research conclusions, and no longer need to spend a lot of time on complex data extraction and literature screening tasks.

[0065] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0066] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. An automated Meta-analysis method based on large language model information extraction, characterized in that: The automated Meta-analysis method based on large language model information extraction includes: Obtain target research topics; Determine the PICO search formula according to the target research topic; Calling the thinking chain tool to decompose the PICO search formula into analysis tasks to obtain an analysis task chain; the analysis tasks in the analysis task chain include a literature search task, a PICO condition verification task, a PICO element extraction task and an RCT literature quality evaluation task; For any analysis task in the analysis task chain, the corresponding tool is called from the analysis tool module to execute the corresponding analysis task; the analysis tool module includes a storage document retrieval tool, a PICO condition verification tool, a PICO element extraction tool and an RCT document quality evaluation tool; In the process of executing the analysis task, calling the long-term memory tool and the short-term memory tool to store and call the task analysis data to link the task context information and optimize the execution process of the analysis task; After executing the analysis task, calling the self-evaluation tool and the self-correction tool to optimize the decision and store it in the thinking chain tool; After executing the analysis task chain, Meta-analysis calculations are performed based on the extracted control group data and intervention group data to obtain the Meta-analysis results.

2. The automated Meta-analysis method based on large language model information extraction according to claim 1, characterized in that: In the process of executing the analysis task, the long-term memory tool and the short-term memory tool are called to store and call the task analysis data to link the task context information and optimize the execution process of the analysis task. After the analysis task is completed, the self-evaluation tool and the self-correction tool are called to optimize the decision and store it in the thinking chain tool, including: When the analysis task is a PICO condition verification task, the PICO condition verification tool is called to verify the search results based on the preset rule matching algorithm and the preset machine learning classifier to determine whether the search results meet the preset PICO standard; all search results that meet the preset PICO standard are marked as included documents and stored in the long-term memory tool, and the currently processed included documents are stored in the short-term memory tool; the self-evaluation tool and the self-correction tool are called to reflect on the included documents and obtain corresponding verification strategy adjustment information; the verification strategy adjustment information is stored in the thinking chain tool; When the analysis task is a PICO element extraction task, the PICO element extraction tool and the short-term memory tool are called, and the information extraction technology is used to extract the PICO elements from the included literature; the PICO elements are stored in the short-term memory tool; the self-evaluation tool and the self-correction tool are called to reflect on the PICO elements and obtain corresponding extraction strategy adjustment information; the extraction strategy adjustment information is stored in the thinking chain tool; the PICO elements include patient characteristics, intervention measures, intervention time, control methods and outcome indicators; When the analysis task is an RCT literature quality evaluation task, the RCT literature quality evaluation tool and the short-term memory tool are called to identify and find the literature content corresponding to each evaluation dimension from the included literature according to the preset RCT literature quality evaluation standard, and determine the corresponding quality evaluation results; the included literature and the corresponding quality evaluation results are stored in the short-term memory tool; the self-evaluation tool and the self-correction tool are called to reflect on the quality evaluation results and obtain corresponding literature evaluation strategy adjustment information; the literature evaluation strategy adjustment information is stored in the thinking chain tool.

3. The automated Meta-analysis method based on large language model information extraction according to claim 1, characterized in that: Determine the PICO search formula based on the target research topic, including: Based on the natural language processing algorithm, a PICO search formula is compiled according to <population>, <intervention>, <control>, and <result>; the self-evaluation tool and the self-correction tool are called to reflect on the PICO search formula and obtain corresponding search formula generation adjustment information; the search formula generation adjustment information is stored in the thinking chain tool.

4. An automated Meta-analysis device based on large language model information extraction, characterized in that: The automated Meta-analysis device based on large language model information extraction includes an analysis tool module, a memory module, a planning module and a Meta-Agent module; The analysis tool module is used to store literature retrieval tools, PICO condition verification tools, PICO element extraction tools and RCT literature quality evaluation tools; The memory module is used to store long-term memory tools and short-term memory tools; The planning module is used to: store thinking chain tools, self-evaluation tools and self-correction tools; The Meta-Agent module is used to: obtain the target research topic; determine the PICO search formula according to the target research topic; call the thinking chain tool to perform analysis task decomposition on the PICO search formula to obtain an analysis task chain; for any analysis task in the analysis task chain, call the corresponding tool from the analysis tool module to execute the corresponding analysis task; in the process of executing the analysis task, call the long-term memory tool and the short-term memory tool to store and call the task analysis data to link the task context information and optimize the execution process of the analysis task; after executing the analysis task, call the self-evaluation tool and the self-correction tool to optimize the decision and store them in the thinking chain tool; after executing the analysis task chain, perform Meta analysis calculation based on the extracted control group data and intervention group data to obtain Meta analysis results; wherein the analysis tasks in the analysis task chain include literature retrieval tasks, PICO condition verification tasks, PICO element extraction tasks and RCT literature quality evaluation tasks.

5. The automatic Meta analysis device based on large language model information extraction according to claim 4 is characterized in that: The analysis tool module is also used to store Meta analysis visualization tools; The Meta-Agent module is also used to: after any of the analysis tasks is executed, call the Meta analysis visualization tool to perform statistical calculations on the analysis result data of the analysis task and generate charts.

6. The automatic Meta analysis device based on large language model information extraction according to claim 4 is characterized in that: The document search tool is used to: use information retrieval technology to generate an optimal search strategy based on the PICO search formula, and connect to a preset database to search to obtain search results; The PICO condition verification tool is used to: verify the search results based on a preset rule matching algorithm and a preset machine learning classifier to determine whether the search results meet the preset PICO standards; Mark all search results that meet the preset PICO criteria as included documents and store them in the memory module; The PICO element extraction tool is used to: extract PICO elements from the included literature using information extraction technology; the PICO elements include patient characteristics, intervention measures, intervention time, control methods and outcome indicators; The RCT literature quality evaluation tool is used to: identify and find out the literature content corresponding to each evaluation dimension from the included literature according to the preset RCT literature quality evaluation standard, and determine the corresponding quality evaluation result.

7. The automatic Meta analysis device based on large language model information extraction according to claim 4, characterized in that: The Meta-Agent module includes: A natural language processing unit is used to: obtain a target research topic; use a natural language processing algorithm to determine a PICO search formula according to the target research topic; call the thinking chain tool to perform analysis task decomposition on the PICO search formula to obtain an analysis task chain; A multi-task learning unit, used for: executing the literature retrieval task, the PICO condition verification task, the PICO element extraction task and the RCT literature quality evaluation task in parallel; The reinforcement learning unit is used to: after executing the analysis task, call the self-evaluation tool and the self-correction tool, optimize the decision based on the reinforcement learning algorithm, and store it in the thinking chain tool.

8. The automatic Meta analysis device based on large language model information extraction according to claim 6, characterized in that: The long-term memory tool is used to: during the execution of the PICO condition verification task, based on the knowledge graph and database management technology, perform long-term memory storage on all the included documents; The short-term memory tool is used to: in the process of executing the PICO condition verification task, based on the dynamic cache management technology, perform short-term memory storage on the currently processed included documents; in the process of executing the PICO element extraction task, based on the dynamic cache management technology, provide the Meta-Agent module to call and extract task context information, and perform short-term memory storage on the PICO elements; in the process of executing the RCT document quality evaluation task, based on the dynamic cache management technology, provide the Meta-Agent module to call and extract task context information, and perform short-term memory storage on the included documents and corresponding quality evaluation results.

9. The automatic Meta analysis device based on large language model information extraction according to claim 4, characterized in that: The thought chain tool is used to: based on the Bayesian network or Markov decision process, use the task decomposition algorithm to decompose the PICO search formula into analysis tasks to obtain an analysis task chain; coordinate the sequence and dependency of each analysis task in the analysis task chain; The self-evaluation tool works in conjunction with the self-correction tool to: after each analysis task in the analysis task chain is executed, based on generative adversarial networks and reinforcement learning technology, conduct self-reflection on the analysis result data of the analysis task, identify potential errors or optimization space, and make real-time adjustments to the task execution strategy based on a feedback mechanism.

Images