An automatic meta-analysis method and device based on large language model information extraction

Abstract

The application discloses an automatic Meta analysis method and device based on large language model information extraction, relates to the field of Meta analysis, and comprises an analysis tool module, a memory module, a planning module and a Meta-Agent module; the Meta-Agent module is used for determining a PICO retrieval formula according to a target research theme, calling a thinking chain tool to perform analysis task decomposition; for any analysis task, corresponding tools in the analysis tool module are called to execute corresponding analysis tasks; in the process of executing the analysis tasks, long-term memory tools and short-term memory tools are called to store and call task analysis data; after the analysis tasks are executed, self-evaluation tools and self-correction tools are called to optimize decision-making; after the analysis task chain is executed, Meta analysis calculation is performed according to extracted control group data and intervention group data. The application can automatically and intelligently process complex Meta analysis tasks.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of Meta analysis, in particular to an automatic Meta analysis method and device based on large language model information extraction. BACKGROUND

[0002] Randomized Controlled Trial (RCT) is a research design method aiming to evaluate the effect of a certain therapy or drug in medical and health services, and is generally considered as the "gold standard" for evaluating the effectiveness of interventions. Due to the limitations of sample size, research conditions and research design, a single RCT study is often difficult to independently draw a sufficiently robust conclusion. At this time, Systematic review and / or Meta analysis as a comprehensive statistical method can provide more efficient and powerful conclusions by aggregating data from multiple independent studies.

[0003] Although systematic review / Meta analysis is a powerful research method and statistical tool, the credibility and reliability of the integrated results depend on whether high-quality RCT evidence is included. These RCT evidences are the cornerstone of systematic review / Meta analysis, and determine the reliability of the final analysis results. However, the current routine process of systematic review / Meta analysis still mainly relies on researchers to manually screen literature, extract information, evaluate evidence quality and analyze and integrate data. The whole process is not only complex and time-consuming, but also easily affected by human errors. Researchers need to manually screen studies that meet the inclusion and exclusion criteria from a large number of literatures, read and analyze literatures one by one, and manually extract data required by systematic review / Meta analysis. This process not only consumes time and effort, but also requires high requirements for the evidence-based methodology, professional background and research experience of researchers. Once there is omission or misjudgment, important data may be ignored or misread, which may affect the accuracy of the results of systematic review / Meta analysis. SUMMARY

[0004] The purpose of the present application is to provide an automatic Meta analysis method and device based on large language model information extraction, which can automatically and intelligently process complex Meta analysis tasks.

[0005] To achieve the above-mentioned purpose, the present application provides the following solutions:

[0006] In a first aspect, the present application provides an automatic Meta analysis method based on large language model information extraction, comprising:

[0007] obtaining a target research topic;

[0008] determining a PICO retrieval formula according to the target research topic;

[0009] The PICO search query is decomposed into analytical tasks using the MindChain tool to obtain an analytical task chain. The analytical tasks in the analytical task chain include a literature retrieval task, a PICO condition validation task, a PICO element extraction task, and an RCT literature quality assessment task.

[0010] For any analysis task in the analysis task chain, the corresponding tool is called from the analysis tool module to execute the corresponding analysis task; the analysis tool module includes a document retrieval tool, a PICO condition verification tool, a PICO element extraction tool, and an RCT document quality assessment tool;

[0011] During the execution of the analysis task, long-term memory tools and short-term memory tools are invoked to store and retrieve task analysis data, thereby linking task context information and optimizing the execution process of the analysis task.

[0012] After completing the analysis task, the self-evaluation tool and self-correction tool are invoked to optimize the decision and store it in the mind chain tool;

[0013] After completing the aforementioned analysis task chain, a meta-analysis was performed based on the extracted control group data and intervention group data to obtain the meta-analysis results.

[0014] Secondly, this application provides an automated meta-analysis device based on large language model information extraction, including an analysis tool module, a memory module, a planning module and a Meta-Agent module;

[0015] The analysis tool module is used to store: a literature retrieval tool, a PICO condition verification tool, a PICO element extraction tool, and an RCT literature quality assessment tool;

[0016] The memory module is used to store long-term memory tools and short-term memory tools;

[0017] The planning module is used to store thought chain tools, self-evaluation tools, and self-correction tools.

[0018] The Meta-Agent module is used for: acquiring the target research topic; determining the PICO search query based on the target research topic; calling the thinking chain tool to decompose the PICO search query into an analysis task chain; for any analysis task in the analysis task chain, calling the corresponding tool from the analysis tool module to execute the corresponding analysis task; during the execution of the analysis task, calling the long-term memory tool and the short-term memory tool to store and retrieve task analysis data, linking task context information, and optimizing the execution process of the analysis task; after completing the analysis task, calling the self-evaluation tool and the self-correction tool to optimize decision-making and storing the results in the thinking chain tool; after completing the analysis task chain, performing Meta-analysis calculations based on the extracted control group data and intervention group data to obtain the Meta-analysis results; wherein, the analysis tasks in the analysis task chain include a literature retrieval task, a PICO condition validation task, a PICO element extraction task, and an RCT literature quality evaluation task.

[0019] According to the specific embodiments provided in this application, this application has the following technical effects: This application provides an automated meta-analysis method and apparatus based on large language model information extraction. The Meta-Agent module realizes a high degree of automation in literature retrieval tasks, PICO condition verification tasks, PICO element extraction tasks, RCT literature quality assessment tasks, and meta-analysis, reducing the steps of manual intervention and avoiding bias caused by human error. The Meta-Agent module can call long-term memory and short-term memory tools, and can optimize the execution of the current system evaluation / meta-analysis task based on past literature records and contextual information of the current analysis task, ensuring that each step runs efficiently with complete background information support, thereby effectively managing information flow. The Meta-Agent module can also call self-evaluation and self-correction tools, and has self-reflection and self-correction functions. Whenever an error occurs in data extraction or analysis, it can adjust its operation strategy in a timely manner by comparing the differences between known results and the current task, ensuring the efficiency and accuracy of the entire system evaluation / meta-analysis process. Finally, the mind chain tool records each step of the analysis process, ensuring that the analysis process is traceable, transparent, and efficient.

[0020] In summary, this application, by combining the autonomous decision-making capabilities of agents, information extraction technology from large language models, and flexible task decomposition and optimization mechanisms, can not only improve the accuracy and efficiency of system evaluation / meta-analysis, but also reduce human intervention, save labor costs, and improve research efficiency, enabling researchers to conduct complex medical research more efficiently. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a schematic diagram of the structure of an automated meta-analysis device based on large language model information extraction in one embodiment of this application. Detailed Implementation

[0023] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0024] With the rapid development of artificial intelligence (AI) and natural language processing (NLP) technologies, automated tools for literature screening and data extraction are gradually emerging. The development of large language models (LLMs) has enabled machines to automatically extract meaningful information from massive amounts of text, laying the foundation for the automation of system reviews / meta-analysis.

[0025] Based on this, this application provides an automated meta-analysis method and apparatus driven by a large language model and based on information extraction from a large language model, utilizing Agent technology. Agent technology refers to an intelligent entity with autonomous perception, task parsing, and execution capabilities, capable of self-decision-making and self-optimization in complex task environments. Unlike traditional manual automation systems, Agents possess greater flexibility and adaptability, dynamically adjusting to different task environments and executing more complex operational processes.

[0026] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0027] In one exemplary embodiment, such as Figure 1As shown, an automated meta-analysis device based on large language model information extraction is provided, including an analysis tool module, a memory module, a planning module, a Meta-Agent module, and an action module. Specifically, the analysis tool module is used to store: a literature retrieval tool, a PICO condition validation tool, a PICO element extraction tool, an RCT literature quality assessment tool, and a meta-analysis visualization tool; the memory module is used to store: long-term memory tools and short-term memory tools; the planning module is used to store: a thought chain tool, a self-evaluation tool, and a self-correction tool; the action module is used to drive the execution of analysis tasks.

[0028] The Meta-Agent module is used for: acquiring the target research topic; determining the PICO search query based on the target research topic; calling the thinking chain tool to decompose the PICO search query into an analysis task chain; for any analysis task in the analysis task chain, calling the corresponding tool from the analysis tool module to execute the corresponding analysis task; during the execution of the analysis task, calling the long-term memory tool and the short-term memory tool to store and retrieve task analysis data, linking task context information, and optimizing the execution process of the analysis task; after the analysis task is completed, calling the self-evaluation tool and the self-correction tool to optimize decision-making and storing the results in the thinking chain tool; after any analysis task is completed, calling the Meta-analysis visualization tool to perform statistical calculations on the analysis results data of the analysis task and generate charts; after the analysis task chain is completed, performing Meta-analysis calculations based on the extracted control group data and intervention group data to obtain the Meta-analysis results; wherein, the analysis tasks in the analysis task chain include literature retrieval tasks, PICO condition validation tasks, PICO element extraction tasks, and RCT literature quality evaluation tasks.

[0029] In a specific application example, the specific functions of each tool in the analysis tool module are as follows:

[0030] The document retrieval tool is used to: employ information retrieval technology to generate an optimal retrieval strategy based on the PICO search query, and connect to a preset database for retrieval to obtain search results; specifically, using information retrieval technology, based on user input requirements (such as PICO conditions), it generates an optimal retrieval strategy (such as Boolean logic, keyword expansion, etc.), and connects to external Chinese-English document databases (such as PubMed, Cochrane, CNKI, etc.) for searching. Retrieval algorithms that can be used during the search process include document matching models such as TF-IDF and BM25.

[0031] The PICO criteria validation tool is used to: validate the search results based on a preset rule matching algorithm and a preset machine learning classifier to determine whether the search results meet the preset PICO criteria; mark all search results that meet the preset PICO criteria as included documents and store them in the memory module; specifically, based on the preset rule matching algorithm and preset machine learning classifier, the documents (i.e., search results) extracted according to the PICO framework (patient, intervention, control, outcome) need to be matched and validated against the preset PICO criteria, and simultaneously screened. The screening process is divided into initial screening and secondary screening. Initial screening is based on the abstract of the document; secondary screening requires further parsing of PDF format documents, reading the full text, and then screening according to the inclusion and exclusion criteria (i.e., the preset PICO criteria). During the screening process, a rule engine and classification model are used to evaluate whether the documents meet the inclusion and exclusion criteria.

[0032] The PICO element extraction tool is used to extract PICO elements from the included literature using information extraction techniques. These PICO elements include patient characteristics, intervention measures, intervention duration, control methods, and outcome indicators. Specifically, information extraction techniques (including named entity recognition and relation extraction) are used to automatically extract PICO elements and an information extraction table based on user needs from the included literature. Deep learning models (such as BERT and GPT) can be used to enhance the accuracy of this type of information extraction task during the PICO element extraction process.

[0033] The RCT literature quality assessment tool is used to: identify and find the literature content corresponding to each evaluation dimension from the included literature according to the preset RCT literature quality assessment criteria, and determine the corresponding quality assessment results; specifically, based on current tools for RCT literature quality assessment, such as the Cochrane risk bias assessment tool, it automatically identifies and finds the specific description content corresponding to the 7 evaluation dimensions in the assessment tool from the included literature, and gives the evaluation results according to the evaluation criteria.

[0034] The meta-analysis visualization tool is used to: display the extracted meta-analysis data (i.e., the analysis results after the analysis task is completed) through statistical calculation and chart generation tools based on data visualization technology; commonly used tools include: ForestPlot to display the comparison of results from different studies, customized visualization charts based on Python libraries (such as Matplotlib and Seaborn), and interactive design of charts using front-end visualization tools such as D3.js or Plotly.

[0035] In a specific application example, the memory module is responsible for storing and retrieving the information required by the device. Long-term memory tools allow the agent to store literature and research data incorporated in previous analyses, thereby better utilizing existing resources and avoiding duplication of effort in future systemic reviews / meta-analysis tasks. Short-term memory tools help the agent maintain contextual coherence in the current dialogue or task execution, enabling it to respond more accurately to task requirements and make effective self-adjustments. This combination of long-term and short-term memory allows the agent to make better autonomous decisions in complex task environments and optimize its operational processes according to constantly changing environmental and task requirements. The specific functions of the various tools in the memory module are as follows:

[0036] The long-term memory tool is used to store all included documents in long-term memory during the execution of the PICO condition verification task, based on knowledge graph and database management technologies (such as NoSQL, GraphDB). Specifically, complete document information retrieved from the document database is stored in long-term memory, and knowledge graph technology helps organize the key content of these documents for easy subsequent querying and analysis. Long-term memory is managed using an external knowledge base.

[0037] The short-term memory tool is used for: storing the currently pending inclusion documents in short-term memory based on dynamic cache management technology during the execution of the PICO condition verification task; storing the PICO elements in short-term memory based on dynamic cache management technology for the Meta-Agent module to call and extract task context information during the execution of the PICO element extraction task; and storing the PICO elements in short-term memory based on dynamic cache management technology for the Meta-Agent module to call and extract task context information during the execution of the RCT document quality assessment task.

[0038] Short-term memory tools involve information that is retrieved and processed in real time during the execution of various analysis tasks, including the memory of the current dialogue or analysis context. Short-term memory is used to process temporary contextual information needed for the current task and is only effective within the context of the current task, similar to RAM in a computer.

[0039] In a specific application example, the planning module serves as the decision-making center for the device's intelligent behavior, achieving optimal decision-making for task flows based on AI planning algorithms. The planning module incorporates a self-reflection function, automatically detecting and correcting errors or anomalies that may occur during the analysis process. Real-time feedback is provided during data extraction and analysis to identify potential problems and adjust the execution flow, ensuring the accuracy of system evaluation / meta-analysis results. The specific functions of the various tools within the planning module are as follows:

[0040] The aforementioned thought chain tool is used to: decompose the PICO retrieval query into analytical task chains based on Bayesian networks or Markov decision processes using task decomposition algorithms; and coordinate the sequence and dependencies of each analytical task within the analytical task chain. Furthermore, during the analytical task decomposition process, the order and dependencies of each analytical task must be coordinated. That is, by using the thought chain tool to decompose complex system evaluation / meta-analysis tasks into multiple manageable sub-tasks, and executing and tracking the progress of each sub-task step by step, the orderly advancement of the task and the overall synergistic effect can be guaranteed. This decomposition and execution strategy gives the entire device strong flexibility and scalability when handling complex tasks.

[0041] The self-evaluation tool works in conjunction with the self-correction tool to: after each analysis task in the analysis task chain is completed, reflect on the analysis results data based on generative adversarial networks and reinforcement learning techniques, identify potential errors or optimization space, and adjust the task execution strategy in real time based on a feedback mechanism. This process is similar to the "exploitation" strategy in reinforcement learning.

[0042] In a specific application example, the Meta-Agent module is the core brain of the entire device. It realizes automated meta-analysis based on large language model information extraction by scheduling various modules. The specific functions of each unit in the Meta-Agent module are as follows:

[0043] The natural language processing unit is used to: acquire a target research topic; determine a PICO query based on the target research topic using a natural language processing algorithm; and invoke the thought chain tool to decompose the PICO query into an analysis task chain. Specifically, it uses a natural language processing algorithm to understand the target research topic and transform it into a machine-executable task. The capabilities of the natural language processing algorithm enable it to extract complex medical terms, recognize semantics, and construct appropriate query statements.

[0044] The multi-task learning unit is used to: execute the document retrieval task, the PICO condition verification task, the PICO element extraction task, and the RCT document quality assessment task in parallel; that is, the multi-task learning unit has parallel processing capability and multi-task learning coordination capability.

[0045] The reinforcement learning unit is used to: after completing the analysis task, invoke the self-evaluation tool and the self-correction tool, perform decision optimization based on the reinforcement learning algorithm, and store the results in the thought chain tool. In this unit, the task execution strategy is continuously optimized based on the reinforcement learning algorithm to achieve an intelligent feedback loop, and the optimal task path is selected through reward and punishment mechanisms.

[0046] In a specific application example, the action module is the technology driver for the device to perform specific tasks. It is responsible for putting the strategies generated by the planning module into practice, including retrieving literature from the database, extracting key information from the literature, and inputting the data into the meta-analysis module. The technologies involved in the action module include: batch processing and parallel computing: the processing of literature data is often large-scale, so parallel computing frameworks (such as Dask, Ray, etc.) are needed to improve processing speed, especially when processing a large number of documents and complex analyses; automated data analysis: using automated statistical analysis tools (such as meta-analytic models) to complete statistical modeling work, and outputting visualization results through automated report generation tools.

[0047] In summary, the automated meta-analysis device based on large language model information extraction provided in this application has the following advantages: (1) Full automation: The entire process from literature retrieval, literature screening, quality evaluation, data extraction and meta-analysis is automated, reducing manual intervention. (2) Self-reflection and self-correction: It can continuously optimize its execution process through self-criticism and self-correction mechanisms, improving the accuracy and reliability of analysis. (3) Visualized analysis results: It not only completes meta-analysis but also generates visualized results to help users intuitively understand the analyzed data. (4) Autonomous planning and execution of the intelligent agent system: The Meta-Agent module serves as the brain of the device, enabling it to autonomously decide on task execution steps and implement specific operations through a toolset. Finally, by combining the above four settings, the device in this application not only automatically extracts data but also directly inputs the extracted data into the meta-analysis tool for calculation and generates high-quality visualized results. The entire process requires no manual intervention, and automation and integration are achieved from data extraction to the final system evaluation / meta-analysis result display. Obviously, this application has high intelligence and innovation, and can automatically and intelligently handle complex meta-analysis tasks.

[0048] Based on the same inventive concept, this application also provides an automated meta-analysis method based on large language model information extraction. The solution provided by this method is similar to the solution described in the above-described apparatus; therefore, the specific limitations in one or more method embodiments provided below can be found in the above-described limitations of the apparatus, and will not be repeated here. In one application example, the automated meta-analysis method based on large language model information extraction includes:

[0049] Step 100: Obtain the target research topic, such as <Non-pharmacological interventions for cardiovascular diseases>.

[0050] Step 200 involves determining the PICO search query based on the target research topic. In one application example, step 200 includes: using a natural language processing algorithm to write the PICO search query based on <population>, <intervention>, <control>, and <outcome>. In this step, it is necessary to ensure that the search query accurately covers all important variables. The self-assessment tool and the self-correction tool are then used to reflect on the PICO search query, checking its completeness and accuracy to ensure that no key intervention or outcome variables are missed, and obtaining corresponding search query generation and adjustment information. This search query generation and adjustment information is then stored in the thought process tool. In practical applications, the thought process tool can also record each step of the PICO search query generation process (such as the reasons for selecting a specific population or intervention) to facilitate retrospection and adjustment in subsequent steps. This step is fully automated, eliminating the limitations of traditional manual screening, which is tedious and prone to errors.

[0051] Step 300: Use the MindChain tool to decompose the PICO search query into analysis tasks to obtain an analysis task chain; the analysis tasks in the analysis task chain include a literature retrieval task, a PICO condition verification task, a PICO element extraction task, and an RCT literature quality assessment task.

[0052] Step 400: For any analysis task in the analysis task chain, call the corresponding tool from the analysis tool module to execute the corresponding analysis task; the analysis tool module includes a document retrieval tool, a PICO condition verification tool, a PICO element extraction tool, and an RCT document quality assessment tool.

[0053] Step 500: During the execution of the analysis task, long-term memory tools and short-term memory tools are invoked to store and retrieve task analysis data, so as to link task context information and optimize the execution process of the analysis task.

[0054] Step 600: After completing the analysis task, call the self-evaluation tool and self-correction tool to optimize the decision and store it in the mind chain tool.

[0055] In practical applications, steps 500 and 600 can work together to complete a task. When the analysis task is a literature retrieval task, the literature retrieval tool is invoked, and based on information retrieval technology, the PICO search query is used to connect to external databases (such as PubMed, Cochrane, etc.) to perform literature retrieval. The self-evaluation tool and self-correction tool are invoked, and advanced retrieval algorithms, including TF-IDF, BM25 and other literature matching models, are used to evaluate the retrieval results. If the literature retrieval results are too few or irrelevant, the PICO search query is adjusted and optimized, the reasons for the adjustment are recorded, and then stored in the mind chain tool for subsequent optimization of the retrieval process.

[0056] When the analysis task is a PICO condition verification task, the PICO condition verification tool is invoked to verify the search results based on a preset rule matching algorithm and a preset machine learning classifier to determine whether the search results meet the preset PICO criteria. The long-term memory and short-term memory tools of the memory module are invoked to mark all search results that meet the preset PICO criteria as included literature and store them in the long-term memory tool as reliable theoretical basis for future reference. Simultaneously, the currently pending included literature is stored in the short-term memory tool for subsequent data extraction and analysis. After verification, the self-evaluation and self-correction tools are invoked to reflect on the included literature. If the analysis results have significant deviations, the data processing and model selection processes are reviewed. After confirming that the steps are correct, the adjustment approach is recorded, thus obtaining the corresponding verification strategy adjustment information. The verification strategy adjustment information is stored in the thinking chain tool, that is, the selection criteria and logic when screening literature are stored in the thinking chain tool to ensure that the screening decisions can be explained based on this chain in the future. It should be noted that during the self-reflection process, there is no need to deliberately correct the results. As long as the data entry and methods are correct, negative or positive results should be accepted to maintain the authenticity and validity of the data.

[0057] When the analysis task is an RCT literature quality assessment task, the RCT literature quality assessment tool and the short-term memory tool are invoked. Based on the preset RCT literature quality assessment standards, the literature content corresponding to each assessment dimension is identified from the included literature, and the corresponding quality assessment results are determined. During the assessment process, the short-term memory tool of the memory module is invoked to store the included literature and the corresponding quality assessment results for subsequent retrieval and verification. After the assessment, the self-assessment tool and the self-correction tool are invoked to reflect on the quality assessment results and obtain corresponding literature assessment strategy adjustment information. The literature assessment strategy adjustment information is stored in the thinking chain tool, that is, the assessment standards and logic during literature assessment are stored in the thinking chain tool to ensure that future screening decisions can be explained based on this chain. In the steps of this analysis task, the literature content is automatically parsed through a large language model, accurately matching the specific content in the RCT corresponding to the literature quality assessment items, and providing assessment results based on the assessment standards, ensuring the efficiency and convenience of the assessment process and reducing subjective bias and error in manual operation.

[0058] When the analysis task is a PICO element extraction task, the PICO element extraction tool and the short-term memory tool are invoked, and information extraction technology is used to extract PICO elements from the included literature. Specifically, this involves extracting intervention and control group data from eligible literature, including sample size, intervention measures, intervention time, frequency, and main outcome indicators. The PICO elements are stored in the short-term memory tool for easy correction. The self-evaluation tool and the self-correction tool are invoked to reflect on the PICO elements, checking the completeness and accuracy of the extracted data to ensure no important information is missed, and obtaining corresponding extraction strategy adjustment information. The extraction strategy adjustment information is stored in the mind chain tool, that is, the logic and reasons for data extraction are stored in the mind chain tool to help maintain consistency in subsequent meta-analyses. The PICO elements include patient characteristics, intervention measures, intervention time, control methods, and outcome indicators. In this step, the literature content is automatically parsed through a large language model to accurately extract key information such as intervention group, control group, sample size, and effect indicators from randomized controlled trials (RCTs), ensuring the efficiency and accuracy of the extraction process and reducing subjective bias and error in manual operation.

[0059] Step 700: After completing the analysis task chain, perform meta-analysis calculations based on the extracted control group data and intervention group data to obtain the meta-analysis results.

[0060] In a specific application, a meta-analysis visualization tool is used to analyze and visualize the extracted control group and intervention group data, generating charts (such as forest plots) to better interpret the conclusions. During the meta-analysis process, the self-assessment and self-correction tools can also be used to reflect on whether the extracted data meets the assumptions of the meta-analysis. When visualizing the results, the clarity and accuracy of the charts in conveying information should be considered, and any unclear points should be adjusted promptly. If problems are found, the analysis methods or data should be adjusted accordingly. If the analysis results show significant deviations, the data processing and model selection process should be re-examined, and the adjustment process recorded. Finally, a thought process tool is used to record the logical derivation process when generating the visualization results, ensuring that the analytical thinking behind the visualization can be clearly explained during interpretation.

[0061] In another specific application, the self-evaluation tool and the self-correction tool regularly summarize and analyze the effective strategies, problems, and corresponding adjustment processes and methods in the historical analysis process, and continuously accumulate experience for subsequent analysis. The optimal task path is selected through reward and punishment mechanisms to achieve reinforcement learning. Throughout the process, the task execution strategy of the Meta-Agent is continuously optimized to achieve intelligent feedback loop.

[0062] In summary, this application presents an automated meta-analysis method and apparatus based on large language model information extraction. First, the systematic review / meta-analysis task is rationally decomposed, determining its core steps and execution plan. For example, for the literature retrieval task, a retrieval strategy is generated based on the PICO framework (population, intervention, control, outcome), and literature is automatically retrieved from mainstream Chinese and English literature databases to obtain literature related to the research questions of the systematic review / meta-analysis. Second, the retrieved literature is screened according to preset inclusion and exclusion criteria (i.e., preset PICO criteria), automatically excluding studies that do not meet the criteria. Then, evidence is extracted from the included literature, mainly extracting data related to the PICO framework and the data required for the study. In this process, the large language model accurately extracts data from the intervention and control groups from the literature, including key information such as sample size, research subjects, intervention measures, intervention time, intervention frequency, and outcome indicators, eliminating reliance on manual extraction and greatly improving the efficiency and accuracy of data extraction. Subsequently, an inclusion threshold was established for the methodological quality of selected RCTs to ensure the quality of the studies. Currently, the most commonly used and authoritative tool for assessing the quality of RCTs is the Cochrane Risk of Bias Assessment Tool, which automatically identifies and retrieves the specific descriptions corresponding to the seven evaluation dimensions of the Cochrane Risk of Bias Assessment Tool from the literature and provides evaluation results based on the evaluation criteria. Finally, after literature screening, data extraction, and quality assessment, the extracted control and intervention group data are automatically input into a systematic review / meta-analysis tool (such as PyMeta) for meta-analysis calculations.

[0063] This application combines the autonomous decision-making capabilities of agents, information extraction technology from large language models, and flexible task decomposition and optimization mechanisms to not only improve the accuracy and efficiency of system evaluation / meta-analysis, but also reduce human intervention, save human resources costs, and improve research efficiency, enabling researchers to conduct complex medical research more efficiently.

[0064] Thanks to the powerful information extraction capabilities of large language models, this application significantly improves the accuracy of extracting key data when extracting RCT evidence. This not only enhances the reliability of systematic reviews / meta-analyses but also reduces the interference of human factors. Researchers can focus more on analyzing the results of systematic reviews / meta-analyses and formulating research conclusions, rather than spending a lot of time on complex data extraction and literature screening tasks.

[0065] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0066] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. An automated meta-analysis method based on information extraction from a large language model, characterized in that, The automated meta-analysis method based on large language model information extraction includes: Identify the target research topic; the target research topic corresponds to a systematic review / meta-analysis task, which is a non-pharmacological intervention for cardiovascular disease. Determine the PICO search query based on the stated target research topic; The PICO search query is decomposed into analytical tasks using a thought chain tool to obtain an analytical task chain. The analytical tasks in the analytical task chain include a literature retrieval task, a PICO condition validation task, a PICO element extraction task, and an RCT literature quality assessment task. The thought chain tool is used to: decompose the PICO search query into analytical tasks based on Bayesian networks or Markov decision processes using a task decomposition algorithm to obtain an analytical task chain; and coordinate the sequence and dependencies of the various analytical tasks in the analytical task chain. For any analysis task in the analysis task chain, the corresponding tool is called from the analysis tool module to execute the corresponding analysis task; the analysis tool module includes a document retrieval tool, a PICO condition verification tool, a PICO element extraction tool, and an RCT document quality assessment tool; During the execution of the analysis task, long-term memory tools and short-term memory tools are invoked to store and retrieve task analysis data, thereby linking task context information and optimizing the execution process of the analysis task. After completing the analysis task, the self-evaluation tool and self-correction tool are invoked to optimize the decision and store it in the mind chain tool; After completing the aforementioned analysis task chain, a meta-analysis is performed based on the extracted control group and intervention group data to obtain the meta-analysis results. A meta-analysis visualization tool is then used to analyze the extracted control group and intervention group data and visualize the results, generating charts. During the meta-analysis process, the self-evaluation and self-correction tools are used to reflect on whether the extracted data meets the assumptions of the meta-analysis, and to reflect on whether the charts clearly and accurately convey information when visualizing the results. Finally, the logical derivation process during the generation of the visualization results is recorded using a thought chain tool.

2. The automated meta-analysis method based on large language model information extraction according to claim 1, characterized in that, During the execution of the analysis task, long-term memory and short-term memory tools are invoked to store and retrieve task analysis data, linking task context information and optimizing the execution process. After the analysis task is completed, self-evaluation and self-correction tools are invoked for decision optimization, and the data is stored in the thought chain tool, including: When the analysis task is a PICO condition validation task, the PICO condition validation tool is invoked to validate the search results based on a preset rule matching algorithm and a preset machine learning classifier to determine whether the search results meet the preset PICO criteria. All search results that meet the preset PICO criteria are marked as included documents and stored in the long-term memory tool, while the currently pending included documents are stored in the short-term memory tool. The self-evaluation tool and self-correction tool are invoked to reflect on the included documents and obtain corresponding validation strategy adjustment information. The validation strategy adjustment information is stored in the mind chain tool. When the analysis task is a PICO element extraction task, the PICO element extraction tool and the short-term memory tool are invoked, and information extraction technology is used to extract PICO elements from the included literature; the PICO elements are stored in the short-term memory tool; the self-evaluation tool and the self-correction tool are invoked to reflect on the PICO elements and obtain corresponding extraction strategy adjustment information; the extraction strategy adjustment information is stored in the mind chain tool; the PICO elements include patient characteristics, intervention measures, intervention time, control methods, and outcome indicators; When the analysis task is an RCT literature quality assessment task, the RCT literature quality assessment tool and the short-term memory tool are invoked. Based on the preset RCT literature quality assessment criteria, the literature content corresponding to each assessment dimension is identified from the included literature, and the corresponding quality assessment results are determined. The included literature and the corresponding quality assessment results are stored in the short-term memory tool. The self-assessment tool and the self-correction tool are invoked to reflect on the quality assessment results and obtain the corresponding literature assessment strategy adjustment information. The literature assessment strategy adjustment information is stored in the mind chain tool.

3. The automated meta-analysis method based on large language model information extraction according to claim 1, characterized in that, Based on the stated target research topic, determine the PICO search terms, including: Based on natural language processing algorithms, PICO search queries are compiled according to <population>, <intervention>, <control>, and <outcome>; the self-evaluation tool and the self-correction tool are invoked to reflect on the PICO search queries and obtain corresponding search query generation and adjustment information; the search query generation and adjustment information is stored in the mind chain tool.

4. An automated meta-analysis device based on large language model information extraction, characterized in that, The automated meta-analysis device based on large language model information extraction includes an analysis tool module, a memory module, a planning module, and a Meta-Agent module; The analysis tool module is used to store: a literature retrieval tool, a PICO condition verification tool, a PICO element extraction tool, and an RCT literature quality assessment tool; The memory module is used to store long-term memory tools and short-term memory tools; The planning module is used to store thought chain tools, self-evaluation tools, and self-correction tools. The Meta-Agent module is used for: acquiring the target research topic; determining the PICO search query based on the target research topic; invoking the thinking chain tool to decompose the PICO search query into an analysis task chain; for any analysis task in the analysis task chain, invoking the corresponding tool from the analysis tool module to execute the corresponding analysis task; during the execution of the analysis task, invoking the long-term memory tool and the short-term memory tool to store and retrieve task analysis data, linking task context information, and optimizing the execution process of the analysis task; after completing the analysis task, invoking the self-evaluation tool and the self-correction tool to optimize decision-making and storing the results. The aforementioned thought chain tool, after executing the aforementioned analysis task chain, performs meta-analysis calculations based on the extracted control group data and intervention group data to obtain the meta-analysis results. The analysis tasks in the analysis task chain include literature retrieval, PICO condition validation, PICO element extraction, and RCT literature quality assessment. The target research topic corresponds to a systematic review / meta-analysis task, which is a non-pharmacological intervention for cardiovascular disease. The thought chain tool is used to: decompose the PICO search query into analysis tasks based on Bayesian networks or Markov decision processes using a task decomposition algorithm to obtain the analysis task chain; and coordinate the sequence and dependencies of each analysis task in the analysis task chain. The meta-analysis visualization tool is used to analyze and visualize the extracted control group and intervention group data, generating charts. During the meta-analysis, the self-evaluation and self-correction tools are used to reflect on whether the extracted data meets the assumptions of the meta-analysis, and to reflect on whether the charts clearly and accurately convey information when visualizing the results. Finally, the logical derivation process in generating the visualization results is recorded by the mind chain tool.

5. The automated meta-analysis device based on large language model information extraction according to claim 4, characterized in that, The analysis tools module is also used to store meta analysis visualization tools; The Meta-Agent module is also used to: after any of the analysis tasks is completed, call the Meta analysis visualization tool to perform statistical calculations on the analysis results data of the analysis task and generate charts.

6. The automated meta-analysis device based on large language model information extraction according to claim 4, characterized in that, The document retrieval tool is used to: generate an optimal retrieval strategy based on the PICO search query using information retrieval technology, and connect to a preset database to perform a retrieval to obtain retrieval results; The PICO condition verification tool is used to: verify the search results based on a preset rule matching algorithm and a preset machine learning classifier, so as to determine whether the search results meet the preset PICO standard; All search results that meet the preset PICO criteria are marked as included documents and stored in the memory module; The PICO element extraction tool is used to: extract PICO elements from the included literature using information extraction technology; the PICO elements include patient characteristics, intervention measures, intervention time, control methods, and outcome indicators; The RCT literature quality assessment tool is used to: identify and find the literature content corresponding to each evaluation dimension from the included literature according to the preset RCT literature quality assessment standards, and determine the corresponding quality assessment results.

7. The automated meta-analysis device based on large language model information extraction according to claim 4, characterized in that, The Meta-Agent module includes: The natural language processing unit is used to: acquire a target research topic; use a natural language processing algorithm to determine a PICO search query based on the target research topic; and call the thought chain tool to decompose the PICO search query into an analysis task chain. The multi-task learning unit is used to: execute the document retrieval task, the PICO condition verification task, the PICO element extraction task, and the RCT document quality assessment task in parallel. The reinforcement learning unit is used to: after completing the analysis task, invoke the self-evaluation tool and the self-correction tool, perform decision optimization based on the reinforcement learning algorithm, and store the results in the thought chain tool.

8. The automated meta-analysis device based on large language model information extraction according to claim 6, characterized in that, The long-term memory tool is used to: store all the included documents in long-term memory based on knowledge graph and database management technology during the execution of the PICO condition verification task; The short-term memory tool is used for: storing the currently pending inclusion documents in short-term memory based on dynamic cache management technology during the execution of the PICO condition verification task; storing the PICO elements in short-term memory based on dynamic cache management technology for the Meta-Agent module to call and extract task context information during the execution of the PICO element extraction task; and storing the PICO elements in short-term memory based on dynamic cache management technology for the Meta-Agent module to call and extract task context information during the execution of the RCT document quality assessment task.

9. The automated meta-analysis device based on large language model information extraction according to claim 4, characterized in that, The self-evaluation tool works in conjunction with the self-correction tool to: after each analysis task in the analysis task chain is completed, reflect on the analysis results data of the analysis task based on generative adversarial networks and reinforcement learning techniques, identify potential errors or optimization space, and adjust the task execution strategy in real time based on the feedback mechanism.

Images