A method and device for reviewing a clinical study protocol based on multi-agent

Abstract

The application provides a kind of based on multi-agent clinical research scheme review method and device, in response to the review request of user, the corresponding clinical research scheme to be reviewed is acquired, and according to the corresponding relationship, the review workflow corresponding to the clinical research scheme to be reviewed is determined;Wherein, the review workflow is the directed acyclic graph comprising a plurality of subtask nodes with dependency relationship;The subtask node that all upstream nodes in the directed acyclic graph have been executed is set as a to-be-executed node;Determine target review agent according to the task type of the to-be-executed node;Receive the review result returned by the target review agent, and integrate the review results of each target review agent to generate an interactive review report. The system generalization ability and scene adaptability are improved, the cost of manual planning is reduced, the execution efficiency is improved, and the structured presentation of review opinions and accurate positioning of original text are realized, supporting efficient human-computer collaborative revision.

Description

Technical Field

[0001] This invention relates to the field of intelligent review, and more specifically to a method and apparatus for reviewing clinical research protocols based on multi-agent systems. Background Technology

[0002] Investigator-initiated trials (IITs) are a crucial driver of medical innovation. To ensure the scientific rigor and compliance of these studies, all clinical trials undergo rigorous scientific review before initiation, which heavily relies on a deep understanding and precise analysis of massive amounts of medical literature. However, with the explosive growth in the number of medical documents worldwide, traditional review models face the following challenges:

[0003] Current review methods heavily rely on manual page-by-page review, which is time-consuming. Furthermore, due to limitations in the reviewers' professional backgrounds, they cannot simultaneously cover all review dimensions, easily leading to blind spots in the review process.

[0004] The design and execution of manual review workflows rely entirely on the reviewers' personal experience and subjective judgment. Review quality and efficiency vary from person to person, making it difficult to guarantee consistency and reproducibility. Summary of the Invention

[0005] In view of the aforementioned problems, this application is proposed to provide a review method and apparatus for a multi-agent-based clinical research protocol that overcomes or at least partially solves the aforementioned problems, comprising: A method for reviewing clinical research protocols based on multi-agent intelligence, the method involving an artificial intelligence model, the artificial intelligence model having a pre-built correspondence between reviewing clinical research protocols and review workflow; The method includes: In response to a user's review request, the corresponding clinical research protocol to be reviewed is obtained, and the review workflow corresponding to the clinical research protocol to be reviewed is determined based on the correspondence; wherein, the review workflow is a directed acyclic graph including several sub-task nodes with dependencies. Set the subtask nodes that have been completed by all upstream nodes in the directed acyclic graph as nodes to be executed; The target review agent is determined based on the task type of the node to be executed; The system receives the review results returned by the target review agents and integrates the review results of each target review agent to generate an interactive review report.

[0006] Furthermore, based on the aforementioned correspondence, the review workflow steps corresponding to the clinical study protocol to be reviewed are determined, including: The corresponding task metadata is determined based on the clinical study protocol to be reviewed, and the current task status is initialized based on the task metadata; The current task status is input into the artificial intelligence model to obtain the next planned action; wherein, the next planned action includes creating a sub-task node of a specified type, creating multiple sub-task nodes in parallel, or ending the planning; Based on the next planned action, generate corresponding sub-task nodes and the dependencies between the sub-task nodes and existing sub-task nodes, and update the current task status; The above steps are executed iteratively until the artificial intelligence model outputs the action to end the planning, thus obtaining the review workflow.

[0007] Further, determining the target review agent based on the task type of the node to be executed includes: At least one available review agent instance is determined based on the task type of the node to be executed; When multiple review agent instances are available, the required target review agent is determined from the available review agent instances using a load balancing strategy.

[0008] Furthermore, the review results of each target review agent are integrated to generate an interactive review report, including: The review results returned by each of the target review agents are integrated and deduplicated. Determine whether there are any conflicts in the integrated review results; If so, a ruling is made on conflicting review results, and the interactive review report is generated based on the ruling.

[0009] Furthermore, it also includes: Determine the feedback information on questionable facts and operation log data during the interaction with the interactive review report; The knowledge base is updated according to the data type of the feedback information on the questionable facts, and / or the parameters of the large language model of the reviewing agent are updated, and the artificial intelligence model is trained according to the operation log data.

[0010] Further, the steps of updating the corresponding knowledge base and / or updating the parameters of the large language model of the reviewing agent based on the data type of the questionable fact feedback information include: Identify the knowledge-based data and reasoning-based data in the feedback information regarding the questionable facts; The preset knowledge graph and preset vector database are updated based on the knowledge-based data. The parameters of the large language model of the review agent are updated based on the inference data.

[0011] Furthermore, the step of training the artificial intelligence model based on the operation log data includes: The operation log data is converted into a state-action sequence; The state-action sequence is used as a training sample, and the artificial intelligence model is trained through an adversarial imitation learning algorithm, so that the artificial intelligence model learns the mapping relationship between the review clinical research protocol and the review workflow.

[0012] A review device for clinical research protocols based on multi-agent intelligence, the device involving an artificial intelligence model, the artificial intelligence model having a pre-built correspondence between reviewing clinical research protocols and review workflow; The device includes: The workflow generation module is used to respond to the user's review request, obtain the corresponding clinical research protocol to be reviewed, and determine the review workflow corresponding to the clinical research protocol to be reviewed based on the correspondence; wherein, the review workflow is a directed acyclic graph including several sub-task nodes with dependencies. The node to be executed module is used to set the subtask nodes that have been executed by all upstream nodes in the directed acyclic graph as nodes to be executed. The review agent determination module is used to determine the target review agent based on the task type of the node to be executed. The report generation module is used to receive the review results returned by the target review agents, integrate the review results of each target review agent, and generate an interactive review report.

[0013] A computer electronic device includes a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program, when executed by the processor, implements the steps of the review method for a multi-agent-based clinical research protocol as described above.

[0014] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the review method for a multi-agent-based clinical research protocol as described above.

[0015] This application has the following advantages: In the embodiments of this application, addressing the problems of low review efficiency, limited dimensions, and unreliable review quality in the prior art, this application provides a solution for a human-machine collaborative intelligent review method for clinical research protocols. Specifically, in response to a user's review request, the corresponding clinical research protocol to be reviewed is obtained, and a review workflow corresponding to the clinical research protocol to be reviewed is determined based on the correspondence. The review workflow is a directed acyclic graph (DAG) including several sub-task nodes with dependencies. Sub-task nodes whose upstream nodes in the DAG have been completed are set as nodes to be executed. A target review agent is determined based on the task type of the node to be executed. The review results returned by the target review agent are received, and the review results of each target review agent are integrated to generate an interactive review report. By pre-constructing the correspondence between clinical research protocols and review workflows using an artificial intelligence model, and adaptively generating a directed acyclic graph (DAG) review workflow containing dependent sub-task nodes, this approach addresses the shortcomings of existing technologies that rely on static rules to schedule review processes and cannot adapt to the differentiated needs of different types of clinical research protocols. It achieves the effect of dynamically matching the optimal review path based on protocol characteristics, improving the system's generalization ability and scenario adaptability. By automatically identifying pending nodes based on the completion of all upstream nodes in the DAG as a dependency condition, it overcomes the inefficiencies in workflow coordination and the reliance on manual intervention in task scheduling during traditional multi-stage review processes. This achieves automated and orderly flow of review tasks, reduces manual coordination costs, and improves parallel execution efficiency. Furthermore, by matching the target review agent with the corresponding function based on the task type of the pending node, it addresses the shortcomings of traditional manual review, such as incomplete dimensional coverage and inconsistent evaluation standards, as well as the limitations of existing AI technologies. Overcoming the limitations of single-dimensional review tools, this tool achieves multi-dimensional simultaneous review of clinical research protocols, ensuring consistency and comprehensive coverage of review standards. By integrating the review results of various target review agents to generate interactive review reports, it solves the shortcomings of traditional review models, such as scattered opinions, difficulty in tracing the source of problems, and low efficiency of human-machine collaboration. This achieves the effect of structured presentation of review opinions and accurate positioning of the original text, supporting efficient human-machine collaborative revision. Attached Figure Description

[0016] To more clearly illustrate the technical solution of this application, the drawings used in the description of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart illustrating the steps of a multi-agent-based clinical research protocol review method provided in one embodiment of this application; Figure 2 This is a schematic diagram of the structure of a coordinator intelligent agent provided in an embodiment of this application; Figure 3 This is a structural block diagram of a review device for a clinical research protocol based on multiple agents, provided in one embodiment of this application. Figure 4 This is a schematic diagram of the structure of a computer electronic device provided in an embodiment of the present invention; 1. Computer electronic device; 2. External device; 3. Processing unit; 4. Bus; 5. Network adapter; 6. I / O interface; 7. Display; 8. Memory; 9. Random access memory; 10. Cache memory; 11. Storage system; 12. Program / utility; 13. Program module. Detailed Implementation

[0018] To make the objectives, features, and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0019] The inventors discovered through analysis of existing technology that: Clinical research protocols are lengthy and have many chapters, making manual page-by-page review a labor-intensive task with a limited amount of information processed per unit of time. Furthermore, individual reviewers have limited professional knowledge and cannot be proficient in multiple independent disciplines such as research design, biostatistics, ethics and regulations, and academic writing. Therefore, it is difficult for a single individual to independently complete a comprehensive review.

[0020] Different types of clinical research protocols (such as interventional studies, observational studies, and diagnostic studies) have significantly different focuses and review processes during the review process. When reviewing manually, the workflow design relies entirely on personal experience and lacks objective basis. The review workflow exists in the minds of reviewers as tacit knowledge and is not explicitly modeled or digitized, making it difficult to reuse and standardize experience.

[0021] The reviews are independent of each other, lacking a unified information aggregation node to collect and compare the review findings of all parties; the logical dependencies between the dimensions are not explicitly expressed, and different review dimensions may have different or even contradictory judgments on the same issue, but there is a lack of judgment criteria to handle these conflicts.

[0022] Based on this, the inventors conceived of a human-machine collaborative review process. Clinical researchers complete part or all of their clinical research protocols in a writing environment and proactively upload protocol files (such as PDF or DOCX formats) to trigger an AI-powered intelligent review process. Upon receiving the review instruction, the system's coordinator agent, according to the workflow generated by the task planning engine, schedules multiple specialized review agents in parallel to conduct multi-dimensional pre-reviews of the manuscript.

[0023] It should be noted that, in any embodiment of the present invention, the coordinator intelligent agent structure is as follows: Figure 2 As shown, the coordinator agent comprises: a task receiver and parser, a task planning engine, an agent scheduling engine, and a result synthesis engine. This invention uses a centralized coordinator agent responsible for global task planning and resource scheduling, while the specific review work is performed in parallel by distributed expert agents. This design ensures both the rationality and global optimality of task decomposition, while fully utilizing the advantages of parallel computing to improve the overall efficiency of the system. The coordinator agent is not a simple rule engine, but an intelligent decision-making system based on machine learning, capable of dynamically generating optimal collaboration strategies based on task characteristics and historical experience.

[0024] Reference Figure 1 This application illustrates a method for reviewing a multi-agent-based clinical research protocol according to an embodiment of the present application: S110. In response to the user's review request, obtain the corresponding clinical research protocol to be reviewed, and determine the review workflow corresponding to the clinical research protocol to be reviewed based on the correspondence; wherein, the review workflow is a directed acyclic graph including several sub-task nodes with dependencies. S120. Set the subtask nodes that have been executed by all upstream nodes in the directed acyclic graph as nodes to be executed. S130. Determine the target review agent based on the task type of the node to be executed; S140. Receive the review results returned by the target review agents, and integrate the review results of each target review agent to generate an interactive review report; In the embodiments of this application, an artificial intelligence model is used to pre-construct the correspondence between clinical research protocols and review workflows, and adaptively generate a directed acyclic graph review workflow containing dependent sub-task nodes. This solves the shortcomings of existing technologies that rely on static rules to schedule review processes and cannot adapt to the differentiated needs of different types of clinical research protocols. It achieves the effect of dynamically matching the optimal review path based on protocol characteristics and improving the system's generalization ability and scenario adaptability. By automatically identifying nodes to be executed based on the condition that all upstream nodes in the directed acyclic graph have been completed, the shortcomings of inefficient process connection and reliance on manual intervention in task scheduling during traditional manual multi-stage review are solved, achieving the automation of review tasks. The system achieves several advantages: streamlined workflow, reduced manual coordination costs, and improved parallel execution efficiency. By matching the target review agent with the corresponding function based on the task type of the node to be executed, it solves the shortcomings of traditional manual review, such as incomplete coverage and inconsistent evaluation standards, as well as the single review dimension of existing AI tools. This achieves the effect of completing the review of clinical research protocols simultaneously from multiple dimensions, ensuring consistency and comprehensive coverage of review standards. By integrating the review results of various target review agents to generate interactive review reports, it solves the shortcomings of traditional review models, such as scattered opinions, difficulty in tracing the source of problems, and low efficiency of human-machine collaboration. This achieves the effect of presenting review opinions in a structured manner and accurately locating the original text, supporting efficient human-machine collaborative revision.

[0025] The following will further describe a method for reviewing a multi-agent-based clinical research protocol in this exemplary embodiment.

[0026] In one embodiment of the present invention, the specific process of step S110, which involves "responding to a user's review request, obtaining the corresponding clinical research protocol to be reviewed, and determining the review workflow corresponding to the clinical research protocol to be reviewed based on the correspondence; wherein the review workflow is a directed acyclic graph including several sub-task nodes with dependencies", can be further described in conjunction with the following description.

[0027] Users upload clinical research protocols to be reviewed via the client and click the review button, thus triggering a review request. Clinical research protocols to be reviewed refer to documents requiring review based on scientific validity, statistical accuracy, ethical compliance, and other dimensions. Formats include, but are not limited to, PDF, DOCX, and TXT files. The current task state is input into a pre-trained model, which outputs the next planned action. This process is iteratively executed until the final action is output, resulting in a complete review workflow.

[0028] In one embodiment of the present invention, the specific process of "determining the review workflow corresponding to the clinical study protocol to be reviewed based on the correspondence" can be further described in conjunction with the following description.

[0029] The corresponding task metadata is determined based on the clinical study protocol to be reviewed, and the current task status is initialized based on the task metadata; The current task status is input into the artificial intelligence model to obtain the next planned action; wherein, the next planned action includes creating a sub-task node of a specified type, creating multiple sub-task nodes in parallel, or ending the planning; Based on the next planned action, generate corresponding sub-task nodes and the dependencies between the sub-task nodes and existing sub-task nodes, and update the current task status; The above steps are executed iteratively until the artificial intelligence model outputs the action to end the planning, thus obtaining the review workflow.

[0030] It should be noted that the above steps involve the coordinator agent's task receiver and parser, as well as the task planning engine.

[0031] The task receiver and parser is used to receive file upload and review requests from the user interaction layer, perform preliminary parsing of the solution document, extract task metadata, and initialize a global task context.

[0032] Interface: POST / api / v1 / review-tasks (This interface uses the Hypertext Transfer Protocol for data submission and follows the path specification of the Representational State Transition architecture style; the request uses a multipart form data format to transmit the scheme file and related parameters, and the response uses a lightweight data exchange format to return the task identifier, processing status and review result.) Processing flow: 1. Receive HTTP requests containing schema files and user IDs.

[0033] 2. Call the document parsing service to convert the document into a uniform text format and divide it into chunks.

[0034] 3. Create a task instance, set its status to "queuing", and store it in the database.

[0035] 4. Pass the task ID and initial context to the task planning engine.

[0036] The task planning engine generates an optimal, structured review workflow based on task type and initial context. This workflow is a directed acyclic graph, where nodes represent subtasks and edges represent dependencies.

[0037] Core algorithm: Collaborative strategy model based on imitation learning.

[0038] Model input: Current task status, including completed subtasks, extracted knowledge, user-supplemented information, etc.

[0039] Model output: Next optimal action, such as "call the research agent to perform fact-checking", "call the statistical agent and compliance agent in parallel", or "end and enter the results synthesis stage".

[0040] Training: The model learns the decision-making logic of experts by performing inverse reinforcement learning on a large number of state-action sequences in the "expert solution library".

[0041] Specifically, the system receives clinical research protocol documents and review instructions uploaded by users, performs preliminary parsing of the protocol documents, extracts task metadata, and initializes the global task context. The task metadata is extracted by calling a document parsing service to convert the protocol documents into text in a unified format, and extracting task metadata describing the basic attributes of the clinical research protocol to be reviewed. Task metadata includes document type, length, title, etc. The coordinator agent generates a review workflow based on the task context using the built-in task planning engine. Specifically, the extracted task metadata is written into the task context in a structured form as the first input to the artificial intelligence model. The system creates a task instance corresponding to this review task, initializes the task instance's status to "queuing" and stores it in the database, and then passes the task ID and initial context to the task planning engine. The current task status is the context information of the artificial intelligence model, obtained from the task metadata initialization, and dynamically updated during workflow generation. The initial task status at least includes task metadata, and subsequent iterations gradually add information about generated sub-task nodes, extracted knowledge information, etc. An AI model is used to establish a mapping between clinical research protocols and review workflows. Specifically, it can be a strategy model, whose input is the current task state and output is the next planned action. The planned action output by the strategy model is the type of operation that should be performed in the current state. The review workflow is represented as a directed acyclic graph, where nodes represent subtasks and edges represent dependencies. One or more subtask nodes are created according to the instructions in the planned action. Each node contains a task type identifier and task parameters. Dependencies refer to the execution order between newly generated nodes and existing nodes. When the planned action is "create a subtask node of a specified type," the new node can establish dependencies based on currently generated nodes; when the planned action is "create multiple subtask nodes in parallel," the relationships between the newly generated nodes are parallel, but they may have dependencies on the same existing node. The information of the newly generated nodes (node ​​ID, task type, state, dependencies, etc.) is appended to the current task state, and the updated task state will serve as the input to the strategy model in the next iteration. The strategy model determines that the currently generated workflow has completely covered all review dimensions required for this type of protocol, and no further nodes are needed. At this point, it outputs the "end planning" action, terminating the loop. When the loop terminates, all the generated subtask nodes and their dependencies together form a complete directed acyclic graph. This directed acyclic graph is then passed to the agent scheduling engine, which executes the review tasks of each node according to the dependencies.

[0042] In one specific implementation, the review workflow is generated: Input: Task context; Output: Review workflow; 1. Initialize an empty review workflow.

[0043] 2. Generate the initial task state based on the task context.

[0044] 3. If the current task status does not meet the termination conditions, repeat the following steps: 3.1 Based on the current task status, use the strategy model to determine the next planned action.

[0045] 3.2 Generate one or more subtask nodes based on the planned actions and the current task status.

[0046] 3.3 Execute on each subtask node: (1) Determine the dependencies of the subtask node based on the currently generated subtask nodes; (2) Add the subtask node and its dependencies to the review workflow; (3) Register the subtask node as a currently generated subtask node.

[0047] 3.3 Update the current task status based on the updated review workflow.

[0048] 4. Output the review workflow.

[0049] In one embodiment of the present invention, the specific process of step S120, "setting the subtask node that has been executed by all upstream nodes in the directed acyclic graph as a node to be executed", can be further explained in conjunction with the following description.

[0050] Specifically, this step involves the agent scheduling engine of the coordinator agent, which executes the workflow generated by the task planning engine. It monitors the status of each node in the workflow, and once the dependency conditions of a node are met, it immediately selects the most suitable expert agent from the agent matrix and distributes the subtasks through the Model Context Protocol (MCP).

[0051] Processing flow: 1. Receive workflow (directed acyclic graph).

[0052] 2. Continuously scan the directed acyclic graph to find nodes to be executed with an in-degree of 0 (i.e., dependencies are satisfied).

[0053] 3. For each node to be executed, query the service discovery center for available expert agent instances based on its task type.

[0054] 4. Select an instance using a load balancing strategy.

[0055] 5. Construct an MCP message containing task instructions and required context data (such as relevant text blocks), and send it to the target agent via a message queue.

[0056] Listen for the returned result, update the task node status to "completed", and store the result in the task context.

[0057] In step S120, steps 1 and 2 of the above process are mainly executed. A subtask node can be scheduled if all upstream nodes (i.e., predecessor nodes that directly or indirectly point to the node) in the directed acyclic graph have been completed. If any upstream node has not yet completed, the node cannot be scheduled and must wait. Upstream nodes refer to all predecessor nodes in the directed acyclic graph that point to the current node via directed edges. An edge represents a "must precede" relationship—if there is an edge from node A to node B, then A must be executed before B, and A is an upstream node of B. When the review task corresponding to the subtask node is completed, the corresponding status is marked as "completed".

[0058] In one embodiment of the present invention, the specific process of "determining the target review agent based on the task type of the node to be executed" in step S130 can be further explained in conjunction with the following description.

[0059] At least one available review agent instance is determined based on the task type of the node to be executed; When multiple review agent instances are available, the required target review agent is determined from the available review agent instances using a load balancing strategy.

[0060] In step S130, steps 3-5 of the agent scheduling engine process involving the coordinator agent are described. Based on the review workflow, the coordinator agent schedules multiple specialized review agents with different functions in parallel to conduct multi-dimensional pre-reviews of the proposal. These review agents include, but are not limited to: ① Research intelligent agents, responsible for extracting key fact statements and conducting literature tracing and fact verification; Core capabilities: fact verification, literature retrieval, and knowledge tracing.

[0061] Internal Implementation: Deeply integrated with the RAG (Retrieval-Augmented Generation) framework. Upon receiving a verification instruction, the declaration is first rewritten using queries, then the vector database and knowledge graph are queried in parallel to obtain relevant knowledge fragments. Finally, the declaration and the retrieved knowledge are input into the domain LLM (Large Language Model) to generate judgments, confidence scores, and evidence.

[0062] Questioning mechanism: When the search is unsuccessful, or multiple knowledge sources conflict, or the confidence level generated by LLM is lower than the threshold (e.g., 0.6), the agent will set the status to waiting for expert feedback in the response and return the "questionable fact" along with its verification path to the coordinator, who will then trigger the expert feedback process.

[0063] ② The statistical agent is responsible for reviewing the rationality of the sample size calculation and the appropriateness of the statistical methods; Core competencies: Reviewing the reasonableness of sample size calculations and the appropriateness of statistical analysis methods, etc.

[0064] Internal implementation: In addition to calling LLM for text understanding, it also integrates some calculation tools that can actually verify the parameters described in the solution.

[0065] ③ The compliance and ethics intelligent agent is responsible for conducting compliance and ethical risk reviews in accordance with regulations; Core competency: Conducting compliance reviews based on laws and ethical guidelines.

[0066] Internal implementation: Its knowledge base mainly consists of structured legal provisions, ethical guidelines (such as the Declaration of Helsinki), and historical cases. The review process focuses more on rule matching and pattern recognition than on open-ended generation.

[0067] ④ An academic writing AI agent is responsible for reviewing the logical structure and language standards of the manuscript; ⑤ Feasibility agent, responsible for assessing research resources and risks.

[0068] All of the aforementioned expert agents must implement a standard interface for unified scheduling by the coordinator. This interface is implemented based on HTTP or gRPC.

[0069] Agents share information and constraints through a standardized Model Context Protocol (MGP), avoiding duplication of effort. The MGP message structure includes a message identifier, sender, receiver, message type (task distribution / result return / query / response), timestamp, and payload (including task ID, instructions, context information, and constraints). This protocol not only transmits task instructions but also shares context information among agents, enabling subsequent agents to leverage the discoveries of preceding agents for efficient knowledge transfer and reuse.

[0070] The message structure of the Model Context Protocol contains the following fields: Message Identifier: String type, used to uniquely identify each message; Sender Identifier: String type, filled with the name of the agent sending the message; Receiver Identifier: String type, filled with the name of the target agent; Message Type: Enumerated type, with values ​​of four categories: task dispatch, result return, query, or response; Timestamp: String type, recording the standard time of message sending; Payload object contains the following subfields: Task Identifier, identifying the current subtask; Execution Instruction, describing the specific operation that the target agent needs to perform; Context Information, including document identifier, text fragment, list of declarations to be verified, and prior knowledge from other agents; Constraints, including maximum execution time (unit: seconds) and confidence threshold.

[0071] Protocol features: This protocol not only transmits task instructions, but more importantly, it enables the sharing of contextual information among agents. For example, after a research agent completes a literature search, the key information it discovers can be transmitted to the statistical agent via MCP. The statistical agent can then utilize this information during statistical method review, avoiding duplication of work. Furthermore, the protocol supports the transmission of constraints, allowing the coordinator to exercise fine-grained control over the execution of subtasks.

[0072] In one embodiment of the present invention, the specific process of step S140, "receiving the review results returned by the target review agents and integrating the review results of each target review agent to generate an interactive review report," can be further explained in conjunction with the following description.

[0073] The review results returned by each of the target review agents are integrated and deduplicated. Determine whether there are any conflicts in the integrated review results; If so, a ruling is made on conflicting review results, and the interactive review report is generated based on the ruling.

[0074] Specifically, the system queries the Service Discovery Center for all available agent instances that support the task type. The Service Discovery Center maintains a list of all registered review agent instances in the current system. Each instance declares the supported task types and its current load status upon registration. During the query, the Service Discovery Center returns a list of all instances that meet the task type requirements and are in an "available" status. The review task type required by the subtask node includes tasks such as research, statistical method evaluation, ethical compliance review, and academic writing style review. Different types of subtasks require review agents with different expertise to execute them. Multiple agent instances of the same type can run simultaneously in the system to achieve load sharing. When multiple available instances exist, one instance is selected to execute the current subtask according to the allocation rules. Load balancing strategies ensure that the workload is evenly distributed among multiple agent instances, preventing some instances from being overloaded while others are idle. Load balancing strategies can include round-robin, random, least-connections, and strategies based on the current load status of agent instances.

[0075] Specifically, step S140 involves the coordinator agent's result synthesis engine. This engine is used to generate interactive review reports. Once all nodes in the workflow have been executed, this engine is responsible for collecting the outputs of all expert agents, integrating, deduplicating, sorting, and resolving any potential conflicts.

[0076] Conflict resolution mechanisms: Based on confidence level: Prioritize the adoption of agent outputs with higher confidence levels.

[0077] Based on preset rules: for example, the opinions of the "compliance and ethics agent" have higher priority than those of the "academic writing agent".

[0078] Submit for arbitration: For serious conflicts that cannot be resolved automatically, mark them as "pending arbitration" and notify the coordinator agent (in a more advanced design, the coordinator can initiate a dedicated "debate" subtask).

[0079] Output format: Generate a structured interactive review report (represented as a structured JSON object). This object fully describes the entire content of the interactive report, including highlighted areas, annotations, summary cards, risk levels, etc., for the front-end module to render. The report is presented in three views: ① Macro-level profile (proposal identification): Provides overall proposal score, risk level distribution radar chart and problem type pie chart: Automatically extracts and presents the core elements of the proposal in a structured manner (such as PICOS elements: population, intervention, control, outcome, study design), displays the consistency analysis of study types, sample size calculation results and risk assessment radar chart, providing researchers with a macro-level profile of the proposal.

[0080] ② Problem Summary (View Problems): All problems are categorized and displayed according to severity. This section summarizes all problems found during the review, provides an overall star rating and multi-dimensional assessments (scientific validity, compliance, feasibility), and displays a list of major problems and core improvement suggestions in order of severity (high / medium / low).

[0081] ③ Detailed Annotations (Check Details): Marks the specific locations of problems in the original solution text and displays the original solution text side-by-side in the interface for tracing the source. Each problem is presented as an independent problem card, including a problem description, severity, root cause analysis, improvement suggestions, and relevant evidence chain. As an example, review comments are displayed item by item in the form of a chapter index on the left and detail cards on the right. Each problem card includes: review result status, severity, detailed problem description, root cause analysis, optimization suggestions, and relevant evidence chain (literature citations). In particular, this view supports side-by-side comparison with the original solution text; clicking on a problem highlights it and jumps to the corresponding location in the original text, providing clear and intuitive feedback on the problem and achieving a seamless AI human-computer collaboration experience.

[0082] In one embodiment of the present invention, it further includes: Determine the feedback information on questionable facts and operation log data during the interaction with the interactive review report; The knowledge base is updated according to the data type of the feedback information on the questionable facts, and / or the parameters of the large language model of the reviewing agent are updated, and the artificial intelligence model is trained according to the operation log data.

[0083] It's important to note that researchers can interact with each AI suggestion within the interactive review report. Simultaneously, it supports deep natural language exploration via the "Follow-up Assistant," where the system uses RAG technology combined with a knowledge base and the original text context to provide streaming answers, maintaining a direct comparison with the original text during follow-up questions. Each user interaction triggers a corresponding AI agent for re-analysis, achieving a closed loop of "AI suggestion - human decision - AI execution." Specifically, it receives user instructions in the interactive report, including: ① Acceptance: Agreeing with the suggestion and marking it as confirmed; ② Rejection: Disagreeing with the suggestion, and the system records this feedback as a negative sample; ③ Requesting alternative solutions: Triggering the AI ​​to regenerate other solutions; ④ Natural language follow-up questions: Engaging in multi-round dialogue through the Follow-up Assistant, with the system providing answers based on RAG technology combined with a knowledge base and the original text context. During the review or follow-up process, if a professional intelligent agent (especially the research intelligent agent responsible for fact-checking) encounters any of the following situations, the fact will be marked as a questionable fact: ① No results were found in the knowledge graph and vector database; ② There are contradictions among the multiple knowledge sources retrieved; ③ The confidence level of the large language model's judgment on the fact is lower than a preset threshold (e.g., 0.6). The system pushes the questionable fact, its context, and the preliminary verification path to the expert feedback interface. Domain experts (such as senior clinicians and statisticians) can see the original text of the questionable fact, its context, and the verification paths attempted by the system in this interface. Experts need to provide a clear judgment: Confirmation: The fact is correct, and authoritative sources can be provided selectively (e.g., the latest guidelines, conference reports, etc.). Refutation: The fact is incorrect, and the reason for the error is explained. Correction: The fact is partially correct but needs correction, and the expert provides the accurate statement after correction. The system receives the judgment and detailed evidence from the domain experts and forms the questionable fact feedback information. Simultaneously, during the interaction with the interactive review report, the system also records all key operational information of the human reviewers, including but not limited to: which chapters the reviewers viewed, which specific agents they invoked, the duration of their stay in each chapter or question, and their decision-making actions regarding the review comments (accept / reject / alternative solutions / follow-up questions). These operations are stored in the form of time-series logs, forming operation log data that reflects the reviewers' review behavior and decision-making logic. The system categorizes expert feedback information, using different types of feedback data for different update paths.

[0084] In one embodiment of the present invention, the steps of updating the corresponding knowledge base and / or updating the parameters of the large language model of the reviewing agent according to the data type of the questionable fact feedback information include: Identify the knowledge-based data and reasoning-based data in the feedback information regarding the questionable facts; The preset knowledge graph and preset vector database are updated based on the knowledge-based data. The parameters of the large language model of the review agent are updated based on the inference data.

[0085] It's important to note that traditional AI systems, after deployment, often have static or slowly updated knowledge bases and models. When faced with the latest, not widely collected, knowledge in the domain, or problems requiring complex logical reasoning, the system often falls short. This embodiment proposes an innovative "dual-track reinforcement learning" mechanism that allows the system to continuously learn from feedback from domain experts, simultaneously improving the coverage of the knowledge base and the reasoning ability of the model. Specifically, the system uses an intent recognition module to semantically analyze expert feedback, determining whether the feedback is factual data that can be directly added to the knowledge base, or a question-and-answer sample pair containing reasoning logic. Knowledge-based data consists of direct facts or data that can be added to the knowledge base (such as the latest half-life of a new drug). Reasoning-based data consists of sample pairs containing expert reasoning logic (such as pointing out a logical error that confuses "total survival" and "progression-free survival"). The agent's dual-track reinforcement includes two tracks: Track 1: Knowledge Base Reinforcement: Knowledge-based data is transformed into triples or text blocks, tagged with expert feedback, and the knowledge graph and vector database are updated in real time to achieve incremental knowledge updates. Track Two: Model Capability Enhancement: Inference data is constructed into high-quality fine-tuning samples. Once a certain number of samples have been accumulated, an online fine-tuning task is triggered, such as using LoRA (Low-Rank Adaptation) technology to fine-tune the language model behind the agent, improving its logical reasoning ability in a specific domain.

[0086] Track 1 involves a knowledge base enhancer: Triggering method: Listening to the "knowledge-based data" queue. Algorithm: Upon receiving a new knowledge triple or text block, it calls the API of the knowledge graph or vector database to perform real-time data writing or updating. To avoid contamination, all data from expert feedback is tagged, and the expert's ID and timestamp are recorded for traceability and auditing.

[0087] Track 2 involves a model capability enhancer: Triggering method: It monitors the "inference data" queue and triggers when a certain amount (e.g., 100 records) or a certain time interval (e.g., 24 hours) is accumulated. Algorithm: It initiates an online fine-tuning or parameter-efficient fine-tuning training task. Using the accumulated "inference data" as the training set, it fine-tunes the domain LLM. After training, the newly generated model adapter is loaded into the AI ​​inference server, achieving seamless upgrades to the model's capabilities.

[0088] The enhanced system can make more accurate judgments in subsequent tasks, reduce the number of "questionable facts", and handle more complex verification tasks, forming a capability growth flywheel of "verification-questionable-feedback-enhancement".

[0089] As an example, in interactive review reports, domain experts do not need to perform structured annotation; they only need to output professional review opinions in natural language based on their own professional judgment regarding the AI's review results or the original proposal. After receiving the expert's natural language review opinions, the system's large language model analysis module automatically parses and constructs a "temporary expert rule" using information extraction technology. This rule is structured and assigned multiple attribute dimensions, including at least: review dimensions (such as scientific validity, ethical compliance, statistics); cited regulations or standards (such as the latest treatment guidelines for a specific disease); applicable proposal type (such as observational studies, interventional studies, specific disease areas); and applicable chapter / section (such as inclusion / exclusion criteria, endpoint indicators, statistical analysis plans). The extracted temporary expert rule is dynamically loaded into the rule base of the corresponding professional agent. In subsequent review tasks, when the attributes (type, chapter, etc.) of a new proposal match the applicable conditions of this temporary rule, the agent will apply this rule to generate review suggestions. The system evaluates the effectiveness of the temporary rule by recording expert feedback on the suggestions (such as adoption, modification, rejection). The system continuously collects evaluation feedback on the temporary rule in multiple applications. If the adoption rate or score of a rule reaches a preset expected threshold, the rule is considered to have been validated and effective. The system will then perform semantic refinement and generalization processing on it. Validated rules are formally converted into "fixed rules," which are not only persisted to the system's global rule base and take effect in real time, but also automatically recorded and transformed into high-quality fine-tuning datasets along with their successful application contexts. These datasets are used for subsequent parameter fine-tuning of large language models (such as LoRA), ultimately internalizing the expert's explicit rules into the model's implicit algorithmic capabilities.

[0090] Experts can simply output review comments as they would in their daily work, without needing to learn complex annotation rules, greatly lowering the barrier to digitizing expert experience. The automatically extracted rules possess highly structured multi-dimensional attributes (applicable type, chapter, regulations, etc.), ensuring the accuracy of rule triggering and preventing rule abuse. The effectiveness of the rules is verified through practical application in subsequent reviews, forming a closed loop of "extraction-application-verification-consolidation," ensuring that the knowledge internalized into the system is reliable and highly generalizable.

[0091] In one specific implementation, if the feedback provided by experts consists of "direct facts or data that can be added to the knowledge base" (e.g., "The latest half-life of drug XX is Y hours, based on the updated instructions from a certain food and drug administration bureau in a certain year"), the system will identify it as knowledge-based data. This type of data will be immediately converted into a structured format: For the knowledge graph, generate a triple: (XX drug, half-life, Y hours), and attach metadata {source: 'expert_feedback', expert_id: 123, timestamp: '2025-11-29', reference: 'Instructions updated by a certain food and drug administration bureau in a certain year'}; where source indicates the data source, expert_feedback indicates expert feedback, expert_id indicates expert identifier, timestamp indicates the recording time, and reference indicates the reference basis.

[0092] For a vector database, a text block is generated, and its embedding vector is calculated and stored in the database.

[0093] In this way, when the system encounters the same or similar problem again, it can directly retrieve the correct answer from the updated knowledge base, realizing real-time incremental updates of knowledge.

[0094] However, not all expert feedback can be directly converted into knowledge base entries. More valuable are those feedbacks that contain the expert's reasoning logic. For example, an expert might point out: "This conclusion is incorrect because it confuses the concepts of 'overall survival' and 'progression-free survival,' and in this research design, the latter should be used as the primary endpoint." This type of feedback reveals not an isolated knowledge point, but a reasoning rule or judgment criterion. The system will recognize it as reasoning data and construct a high-quality fine-tuning sample: Once a certain number of such samples are accumulated (e.g., 100), the system will trigger an online fine-tuning task. Fine-tuning employs parameter-efficient methods, such as LoRA, which only update a small subset of the model's parameters, thereby improving its inference capabilities in a specific domain while maintaining the general capabilities of the base model.

[0095] After fine-tuning, the new model adapter will be seamlessly deployed to the AI ​​inference server, replacing the old version. In this way, the system not only "knows" the new knowledge points, but also "learns" the expert's thinking methods, enabling it to make judgments that are more in line with expert logic when encountering similar problems in the future.

[0096] The greatest value of the dual-track reinforcement learning mechanism lies in building a positive-cycle "capability growth flywheel": In the initial stage: due to the limited system capabilities, a large number of "questionable facts" will be generated, requiring frequent requests for expert feedback.

[0097] Learning phase: Expert feedback is transformed into knowledge-based data and reasoning-based data, expanding the system's knowledge base and enhancing the model's reasoning ability.

[0098] Capability Enhancement Phase: The enhanced system is able to make more accurate judgments when handling new tasks, the number of "questionable facts" is reduced, and the reliance on experts is decreased.

[0099] Advanced learning stage: The system is able to handle more complex tasks. Occasionally, it encounters challenging "questionable facts" that are often cutting-edge issues or marginal cases in the field. Experts provide more valuable feedback on these issues, further driving the system to evolve to a higher level.

[0100] Furthermore, to ensure the quality and reliability of expert feedback, this invention also incorporates a multi-layered quality control mechanism: 1. Expert Qualification Certification: Only domain experts who have been certified by the system (such as clinicians with senior professional titles or senior statisticians) can access the expert feedback interface.

[0101] 2. Multi-expert cross-validation: For key "questionable facts", the system can push them to multiple experts. Only when the majority of experts agree will the data be used as high-confidence learning data.

[0102] 3. Feedback Traceability and Auditing: All expert feedback will be fully recorded, including expert ID, timestamp, and judgment criteria, forming a traceable audit chain. If an error is found in a piece of feedback in the future, it can be traced back and corrected.

[0103] 4. Negative feedback mechanism: If the system learns from a piece of expert feedback but finds it ineffective in subsequent applications (e.g., users frequently reject suggestions generated based on that knowledge), the system will reduce the weight of that feedback or even mark it as "to be reviewed".

[0104] As an example, frequent online fine-tuning can lead to high model training overhead and service instability (the model may be temporarily unavailable during updates). A possible solution is: Efficient parameter fine-tuning: Using techniques such as LoRA, only a small portion of the model's parameters (usually <1%) are fine-tuned, significantly reducing computational costs and training time.

[0105] Blue-green deployment: Maintain two sets of model services, one (blue) is providing services, and the other (green) is being fine-tuned. After the fine-tuning is completed, a seamless switchover is performed to ensure uninterrupted service.

[0106] Batch update strategy: Instead of immediately fine-tuning every piece of feedback received, batch fine-tuning is performed after accumulating a certain number of responses or reaching a certain time interval, balancing update frequency and system stability.

[0107] In one embodiment of the present invention, the step of training the artificial intelligence model based on the operation log data includes: The operation log data is converted into a state-action sequence; The state-action sequence is used as a training sample, and the artificial intelligence model is trained through an adversarial imitation learning algorithm, so that the artificial intelligence model learns the mapping relationship between the review clinical research protocol and the review workflow.

[0108] Specifically, the system provides a solution recording mode, allowing domain experts to record all key operations (including viewing chapters, invoking specific agents, decision inflection points, dwell time, etc.) in the background while performing a complete review task, forming a time-series operation log. The solution parsing module analyzes the operation log and abstracts it into a state-action sequence. A state is defined as a snapshot of the task context at a certain decision point (e.g., the type of problem identified, the semantic features of the currently focused text block), and an action is defined as the next macro-level operation chosen by the expert (e.g., invoking a statistical agent, focusing on reviewing the ethics section). A complete task is transformed into a sequence: Structured state-action sequences are used as high-quality teaching data to train the collaborative policy model of the coordinator agent using the Generative Adversarial Imitation Learning (GAIL) algorithm. Through adversarial training, the algorithm enables the policy model (generator) to learn to generate action sequences indistinguishable from expert behavior, thereby learning the expert's decision-making logic and generalizing to new, unseen task scenarios.

[0109] As an example, Expert Solution Recorder: Implementation: A "logger" component is embedded in the browser plugin or front-end application. When the expert activates "solution recording mode," the component listens for and records all of the expert's key operation events (such as clicking buttons, calling APIs, and entering text) along with their timestamps and context, forming a detailed operation log.

[0110] Solution structuring: Algorithm: An offline processing script that periodically analyzes recorded operation logs. It transforms the raw operation log stream into a structured "state-action" sequence through heuristic rules and pattern matching.

[0111] Status: Defined as a snapshot of the task's context at a certain decision point, including reviewed chapters, identified issues, and the current user's focus.

[0112] Action: Defined as the next macroscopic operation chosen by the expert in this state, such as invoking a statistical agent or focusing on an ethical chapter.

[0113] Collaborative strategy trainer: Algorithm: Use inverse reinforcement learning algorithms, such as GAIL.

[0114] Training process: Structured state-action sequences are used as expert teaching data. A generator is trained to mimic the expert's behavior, while a discriminator is trained to distinguish between the expert's behavior and the generator's behavior. Through this adversarial training, the policy model can learn the expert's underlying decision-making logic, rather than just superficial behavior.

[0115] Output: The weights of the trained policy model, deployed in the "task planning engine" of the coordinator agent.

[0116] In one specific implementation, the GAIL algorithm from inverse reinforcement learning is used as the core technology.

[0117] Step 1: Recording and structuring expert solutions The system provides an "Expert Protocol Recording Mode." When a domain expert completes a full review task in the system (e.g., reviewing a clinical research protocol), the system records all key operations performed by the expert in the background, including which chapters were viewed, which tools were used, where the expert spent more time (indicating focused attention), and what review comments were ultimately provided. These operations are recorded as a time-series operation log.

[0118] Subsequently, an offline "solution structuring" module analyzes these logs, abstracting them into a "state-action" sequence. Specifically: A state is defined as a snapshot of the task's context at a given decision point. For example, a state might include: a list of reviewed chapters, the types and number of issues identified, the semantic features of the currently focused text block, and user requests for further information. A state can be represented as a high-dimensional vector.

[0119] An action is defined as the next macroscopic operation chosen by the expert in that state. For example, an action could be: `invoke_agent('statistics')` (invoking a statistical agent), `focus_on_section('ethics')` (focusing on reviewing the ethics section), `request_external_reference('PMID:12345')` (reviewing external literature), etc. The action space is discrete and finite.

[0120] In this way, ,in It is the first The state of the step, It is the action chosen by the expert in this state.

[0121] Step 2: Training the policy model based on GAIL The core idea of ​​the GAIL algorithm is to enable the policy model to learn to generate action sequences that are indistinguishable from expert behavior through adversarial training.

[0122] Generator: This refers to the policy model of the coordinator agent, denoted as... , This represents the model parameters of the generator. Indicates the state Select action The probability distribution is the input state vector and the output probability distribution of the actions. This model is typically a deep neural network, taking a state vector as input and outputting the probability distribution of the actions.

[0123] Discriminator: denoted as ,in, This represents the model parameters of the discriminator. Indicates the task status. This represents the action taken in the stated task state. The discriminator is used to determine the likelihood of a "state-action" pair from the expert teaching data, and its output is a probability value between 0 and 1.

[0124] The training process is an adversarial game: 1. The generator is based on the current strategy. Interact with the environment (perform tasks in a simulated environment) to generate a series of state-action pairs.

[0125] 2. The discriminator receives real "state-action" pairs from experts and "state-action" pairs from the generator, and learns to distinguish between the two.

[0126] 3. The generator's goal is to "deceive" the discriminator, meaning the generated "state-action" pairs are judged by the discriminator as originating from an expert as much as possible. Therefore, the generator's loss function is: ,in, Indicates generator loss; This represents the expectation operation; Indicates the task status; Indicates an action; The parameter is Generation strategy; The parameter is The discriminator.

[0127] 4. The goal of the discriminator is to accurately distinguish between the behavior of experts and generators, and its loss function is the standard binary classification cross-entropy: ,in, Indicates discriminator loss; This represents the state-action pair distribution of the expert teaching data; This represents the expectation operation; Indicates the task status; Indicates an action; The parameter is Generation strategy; The parameter is The discriminator.

[0128] The two optimizations are performed alternately until convergence.

[0129] Step 3: Deployment and Generalization of the Strategy Model After training, the policy model It is deployed into the "task planning engine" of the coordinator agent. When the system receives a new review task, the coordinator will determine the task's initial state. The first action is predicted using a policy model. Then, the action is performed (such as scheduling an expert agent), and the system state transitions to... Then predict the next action, and repeat this process until the task is completed.

[0130] Because the strategy model is trained by learning from a large number of expert solutions, it can not only mimic the behavior of experts in specific situations, but more importantly, it can generalize to new and unseen task scenarios. For example, even if an expert has never reviewed a certain type of solution, as long as the state characteristics of the solution are similar to some cases in the training set, the strategy model can learn by analogy and generate reasonable workflows.

[0131] In a specific implementation, there are still some challenges in imitation learning, such as how to represent complex task states (including text content, discovered problems, user intent, etc.) as a vector that can be processed by the neural network.

[0132] Solution: Employ multimodal embedding technology. Specifically: For text content, pre-trained domain language models (such as BioBERT) are used to generate text embedding vectors.

[0133] For structured information (such as a list of completed subtasks or statistics on problem types), use one-hot encoding or numerical normalization before concatenation.

[0134] For additional user requests (natural language), embeddings are also generated using a language model.

[0135] Finally, these vectors from different modalities are concatenated or fused through an attention mechanism to form a unified state vector.

[0136] For example, experts have limited time and can only record a limited number of "expert solutions." How can an effective policy model be trained under conditions of small sample size?

[0137] Solution: Data augmentation: Generate more training samples by slightly perturbing existing expert solutions (such as changing the initial conditions of the task or simulating different intermediate states).

[0138] Transfer learning: First, the policy model is pre-trained on a large-scale general task planning dataset, and then fine-tuned on expert protocols for clinical research review.

[0139] Active learning: The system proactively identifies the task scenarios that are most valuable for model training and prioritizes requesting experts to provide instruction in these scenarios.

[0140] Example 1 Review of investigator-initiated trial (IIT) protocols for novel cancer drug development from a multidisciplinary perspective Scenario: A doctor at an oncology center drafted an IIT (Intra-Intracytoplasmic Treatment) protocol for "novel targeted therapy combined with immunotherapy for advanced lung cancer". Before submission, he uploaded the protocol to this system for preliminary review.

[0141] Application process: Parallel pre-screening trigger: After the doctor uploads the Word document, the system's coordinator agent instantly completes the solution analysis and schedules four expert agents (research, statistics, compliance and ethics, and academic writing) to start working in parallel.

[0142] Interactive multi-view report presentation: A few minutes later, the system generates a structured report containing three main views: The “Procedure Identification” view automatically extracts the PICOS elements of the treatment plan; where P: advanced non-small cell lung cancer; I: targeted drug A + immunotherapy B; C: targeted drug A alone; O: outcome measure, specifically progression-free survival (PFS); S: randomized controlled trial; and displays an overall risk radar chart of the treatment plan, allowing doctors to have a clear understanding of their treatment plan structure.

[0143] The "Issues View" view shows an overall rating of "Scientific merit: 4 stars, Compliance: 3 stars, Feasibility: 4 stars". In the issue list, an issue marked "High Severity" is pinned to the top: "Insufficient basis for sample size calculation".

[0144] The "Details" view is compared with the original text: When a doctor clicks on a high-risk question, a details card pops up on the right side of the screen, while the original text on the left scrolls synchronously and highlights the corresponding paragraph.

[0145] Problem description: The statistical agent points out that the expected PFS value of the control group is based on a 5-year-old paper, which is inconsistent with the current clinical reality, resulting in an underestimation of the sample size.

[0146] Optimization recommendations: The study cross-referenced the latest guidelines for intelligent agents and suggested using real-world data published in the last two years as a control group reference value for PFS, and provided links to recommended literature.

[0147] Natural Language Follow-up Questions and Collaborative Revision: If a doctor has doubts about a recommended article, they can activate the "Follow-up Question Assistant" below the question card and type: "Are the demographic characteristics of this recommended article completely consistent with my study?" The system, combining the original text and the full-text of the article, provides a streaming answer: "Not completely consistent. The median age of the population in this article is slightly higher, but its epidermal growth factor receptor mutation ratio is highly consistent with your design, making it the closest reference data currently available." The doctor accepts the suggestion and clicks "Accept." The system automatically records and prepares to generate a revised version during export.

[0148] Ultimate value: What used to require weeks of meetings and discussions among statisticians, methodologists, and ethics experts can now be completed in just a few minutes with professional feedback from a multidisciplinary perspective, with the original text reviewed throughout and revisions made based on sound reasoning.

[0149] Example 2 Dynamic Fact Check on Off-Label Drug Use in Rare Diseases Scenario: When reviewing a proposal on "Drug X for the treatment of a rare autoimmune disease", the system's research agent encountered a knowledge blind spot.

[0150] Application process: Questionable Fact Trigger: The research agent searches its internal knowledge graph and vector database for evidence of efficacy of "Drug X for treating this rare disease" and finds a conflict of information: the old version of the instruction manual does not mention it, but a newly published case report claims it is effective. The agent determines that the confidence level of this fact is only 0.4, which is below the threshold, and therefore marks it as a "questionable fact".

[0151] Expert Judgment and In-Depth Structure: The questionable fact was pushed to the interface of a senior immunology expert. After review, the expert gave the following judgment: "The use of case reports as the main basis for the proposed plan is not rigorous enough. Although relevant positive Phase II clinical data were announced at the XX International Conference last month, it is still not recommended as a routine IIT intervention at present." The system's backend feedback processing module deeply parses this natural language into two parts: Knowledge-based data: "The XX conference announced positive Phase II data on drug X for treating this rare disease." Inferential data: "When evaluating IIT regimens for off-label use of drugs in rare diseases, the level of evidence in case reports is insufficient to support the rationality of the intervention. The latest clinical trial data should be supplemented, and a cautious approach should be maintained overall." Dual-track enhanced execution: Track 1 (Knowledge Base Update): The above knowledge data is tagged with "Expert ID: Wang_001" and "Timestamp: 2026-05-27" and written into the knowledge graph in real time.

[0152] Track Two (Online Model Fine-Tuning): The aforementioned inference data is converted into JSON fine-tuning samples. Once the system collects 100 similar "expert inferences about the level of evidence" samples, LoRA-based online fine-tuning is automatically triggered.

[0153] Evolution of capabilities: A week later, when another doctor submitted a similar plan, the updated agent could not only directly cite the data from that international conference (knowledge enhancement), but also proactively point out that "the current level of evidence is insufficient, and it is recommended to adjust the intervention strategy" (reasoning ability enhancement).

[0154] Ultimate value: The system is no longer a dead database that gets worse with use, but a "living brain" that can continuously absorb cutting-edge knowledge and learn rigorous reasoning logic from experts as they use it. Moreover, every knowledge update is traceable and meets medical compliance requirements.

[0155] Example 3 From a single review comment to the adaptive internalization of global rules Scenario: An increasing number of real-world study (RWS) proposals are being submitted. However, the AI ​​system was not initially configured with sophisticated rules specifically for RWS data quality control.

[0156] Application process: Expert Natural Language Feedback: When reviewing an RWS (Real-World Study) proposal, data expert Mr. Zhao noticed that the proposal lacked a data traceability mechanism. He directly wrote a comment in the report: "This is a real-world study based on electronic medical records. The proposal must clearly explain how to conduct source data verification (SDV); otherwise, data quality cannot be guaranteed." Automatic extraction of provisional rules: The system's large language model automatically captured this opinion and extracted and constructed a "provisional expert rule" with four-dimensional attributes: Review dimensions: Scientific rigor / Data quality Citation Standard: Guide to Real-World Research Adaptation Program Type: Observational Study / Real-World Study Adapted Chapter / Section: Data Management Plan Rule logic: If the scheme type is RWS, check whether the "Data Management" section contains a specific description of "Source Data Verification (SDV)".

[0157] Rule Application and Validation: This temporary rule was silently loaded into the research agent. The following week, the system received five more RWS proposals. During the preliminary review of these five proposals, the agent automatically applied this temporary rule and suggested to other review experts that "SDV descriptions are missing." The system recorded that of these five suggestions, four were directly "adopted" by the experts, and one required additional modifications from the expert.

[0158] Rule refinement and internalization: The system determines that the adoption rate (80%) of the temporary rule exceeds the expected threshold, confirming its validity. Subsequently: This rule has been officially converted into a fixed rule, written into the global rule library, and will be permanently effective.

[0159] These six complete contextual cases (one extraction + five validations) were automatically packaged by the system into a fine-tuning dataset. In the next model iteration, this explicit rule of "checking SDV when RWS is seen" is completely internalized into the model's implicit judgment ability.

[0160] Ultimate Value: Experts don't need to fill out complex rule configuration forms; they can simply provide their opinions according to their habits, and the system can automatically complete the closed loop of "extraction-validation-consolidation." This allows the system to adapt to new types of clinical research and regulatory changes at extremely low cost and extremely fast speed, achieving true adaptive evolution.

[0161] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0162] Reference Figure 3 This application illustrates a review device for a multi-agent-based clinical research protocol according to an embodiment of the present application, which specifically includes the following modules: Specifically, it includes: Workflow generation module 310 is used to respond to a user's review request, obtain the corresponding clinical research protocol to be reviewed, and determine the review workflow corresponding to the clinical research protocol to be reviewed based on the correspondence; wherein, the review workflow is a directed acyclic graph including several sub-task nodes with dependencies. The node to be executed module 320 is used to set the subtask nodes that have been executed by all upstream nodes in the directed acyclic graph as nodes to be executed. The review agent determination module 330 is used to determine the target review agent based on the task type of the node to be executed. The report generation module 340 is used to receive the review results returned by the target review agents, integrate the review results of each target review agent, and generate an interactive review report.

[0163] In one embodiment of the present invention, the workflow generation module 310 includes: The initialization submodule is used to determine the corresponding task metadata based on the clinical research protocol to be reviewed, and to initialize the current task status based on the task metadata. The next planning submodule is used to input the current task status into the artificial intelligence model to obtain the next planning action; wherein, the next planning action includes creating a sub-task node of a specified type, creating multiple sub-task nodes in parallel, or ending the planning; The task status update submodule is used to generate corresponding subtask nodes and the dependencies between the subtask nodes and existing subtask nodes based on the next planned action, and to update the current task status. The iterative execution submodule is used to iteratively execute the above steps until the artificial intelligence model outputs the action to end the planning, thus obtaining the review workflow.

[0164] In one embodiment of the present invention, the review agent determination module 330 includes: The instance determination submodule is used to determine at least one available review agent instance based on the task type of the node to be executed. The load balancing selection submodule is used to determine the desired target review agent from the available review agent instances using a load balancing strategy when multiple review agent instances are available.

[0165] In one embodiment of the present invention, the report generation module 340 includes: The deduplication submodule is used to integrate and deduplicate the review results returned by each of the target review agents. The conflict determination submodule is used to determine whether there are any conflicts in the integrated review results; The adjudication submodule is used to adjudicate conflicting review results if the conditions are met, and to generate the interactive review report based on the adjudicated review results.

[0166] In one embodiment of the present invention, it further includes: Interactive data processing module 350 is used to determine the feedback information of questionable facts and operation log data during the interaction with the interactive review report; The reinforcement learning module 360 ​​is used to update the corresponding knowledge base and / or update the parameters of the large language model of the review agent based on the data type of the feedback information of the questionable facts, and to train the artificial intelligence model based on the operation log data.

[0167] In one embodiment of the present invention, the reinforcement learning module 360 ​​includes: The data classification submodule is used to determine the knowledge-based data and reasoning-based data in the feedback information of questionable facts; The first track enhancement submodule is used to update the preset knowledge graph and preset vector database based on the knowledge data; The second track enhancement submodule is used to update the parameters of the large language model of the review agent based on the inference data.

[0168] In one embodiment of the present invention, the reinforcement learning module 360 ​​includes: The data transformation submodule is used to transform the operation log data into a state-action sequence; The imitation learning training submodule is used to train the artificial intelligence model by using the state-action sequence as training samples and an adversarial imitation learning algorithm, so that the artificial intelligence model learns the mapping relationship between the review clinical research protocol and the review workflow.

[0169] Reference Figure 4 The diagram illustrates a computer electronic device for implementing a method for reviewing a multi-agent-based clinical research protocol according to the present invention, which may specifically include the following: The aforementioned computer electronic device 1 is manifested in the form of a general-purpose computing device. The components of the computer electronic device 1 may include, but are not limited to: one or more processors or processing units 3, memory 8, and a bus 4 connecting different system components (including memory 8 and processing unit 3).

[0170] Bus 4 represents one or more of several bus architectures, including memory buses or memory controllers, peripheral buses, graphics acceleration ports, processors, or local buses using any of the various bus architectures. For example, these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Audio / Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0171] Computer electronic device 1 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by computer electronic device 1, including volatile and non-volatile media, removable and non-removable media.

[0172] Memory 8 may include computer system readable media in the form of volatile memory, such as random access memory 9 and / or cache memory 10. Computer electronic device 1 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 11 may be used to read and write non-removable, non-volatile magnetic media (commonly referred to as a "hard disk drive"). Although Figure 4 As not shown, a disk drive for reading and writing to a removable non-volatile disk (such as a "floppy disk") and an optical disk drive for reading and writing to a removable non-volatile optical disk (such as a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 4 via one or more data media interfaces. The memory may include at least one program product having a set (e.g., at least one) of program modules 13 configured to perform the functions of the embodiments of this application.

[0173] A program / utility 12 having a set (at least one) of program modules 13 may be stored, for example, in memory. Such program modules 13 include—but are not limited to—an operating system, one or more application programs, other program modules 13, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 13 typically perform the functions and / or methods described in the embodiments of this application.

[0174] The computer electronic device 1 can also communicate with one or more external devices 2 (e.g., keyboard, pointing device, display 7, camera, etc.), and with one or more devices that enable an operator to interact with the computer electronic device 1, and / or with any device that enables the computer electronic device 1 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed through the I / O interface 6. Furthermore, the computer electronic device 1 can also communicate with one or more networks (e.g., local area network (LAN)), wide area network (WAN), and / or public networks (e.g., the Internet) through the network adapter 5. Figure 4 As shown, network adapter 5 communicates with other modules of computer electronic device 1 via bus 4. It should be understood that, although... Figure 4 Not shown, it may be combined with other hardware and / or software modules, including but not limited to: microcode, device drivers, redundant processing unit 3, external disk drive array, RAID system, tape drive and data backup storage system 11, etc.

[0175] The processing unit 3 executes various functional applications and data processing by running programs stored in memory 8, such as implementing a method for reviewing a multi-agent-based clinical research protocol provided in the embodiments of this application.

[0176] That is, when the processing unit 3 executes the above procedure, it performs the following: in response to the user's review request, it obtains the corresponding clinical research protocol to be reviewed, and determines the review workflow corresponding to the clinical research protocol to be reviewed based on the correspondence; wherein, the review workflow is a directed acyclic graph including several sub-task nodes with dependencies; the sub-task nodes in the directed acyclic graph that have been completed by all upstream nodes are set as nodes to be executed; the target review agent is determined based on the task type of the node to be executed; the review results returned by the target review agent are received, and the review results of each target review agent are integrated to generate an interactive review report.

[0177] In this application embodiment, the present application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a method for reviewing a multi-agent-based clinical research protocol as provided in all embodiments of the present application.

[0178] That is, when the program is executed by the processor, it implements the following: in response to the user's review request, it obtains the corresponding clinical research protocol to be reviewed, and determines the review workflow corresponding to the clinical research protocol to be reviewed based on the correspondence; wherein, the review workflow is a directed acyclic graph including several sub-task nodes with dependencies; the sub-task nodes in the directed acyclic graph that have been completed by all upstream nodes are set as nodes to be executed; the target review agent is determined based on the task type of the node to be executed; the review results returned by the target review agent are received, and the review results of each target review agent are integrated to generate an interactive review report.

[0179] Any combination of one or more computer-readable media may be used. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus, or device.

[0180] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including—but not limited to—electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of transmitting, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0181] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the operator's computer, partially on the operator's computer, as a standalone software package, partially on the operator's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the operator's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider). The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably.

[0182] Although preferred embodiments of the present application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present application.

[0183] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0184] The above provides a detailed description of the review method and apparatus for a multi-agent-based clinical research protocol provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for reviewing clinical research protocols based on multi-agent systems, characterized in that, The method involves an artificial intelligence model, which is pre-built with a correspondence between reviewing clinical research protocols and review workflows; The method includes: In response to a user's review request, the system obtains the corresponding clinical research protocol to be reviewed and determines the review workflow corresponding to the protocol based on the correspondence. The review workflow is a directed acyclic graph (DAG) comprising several sub-task nodes with dependencies. Specifically, the steps for determining the review workflow corresponding to the clinical research protocol to be reviewed based on the correspondence include: determining the corresponding task metadata based on the clinical research protocol to be reviewed, and initializing the current task state based on the task metadata; inputting the current task state into the AI ​​model to obtain the next planning action; wherein the next planning action includes creating a sub-task node of a specified type, creating multiple sub-task nodes in parallel, or ending the planning; generating the corresponding sub-task nodes and the dependencies between the sub-task nodes and existing sub-task nodes based on the next planning action, and updating the current task state; iteratively executing the above steps until the AI ​​model outputs the action to end the planning, thus obtaining the review workflow. Set the subtask nodes that have been completed by all upstream nodes in the directed acyclic graph as nodes to be executed; The target review agent is determined based on the task type of the node to be executed; The system receives the review results returned by the target review agents and integrates the review results of each target review agent to generate an interactive review report.

2. The method according to claim 1, characterized in that, Determining the target review agent based on the task type of the node to be executed includes: At least one available review agent instance is determined based on the task type of the node to be executed; When multiple review agent instances are available, the required target review agent is determined from the available review agent instances using a load balancing strategy.

3. The method according to claim 1, characterized in that, The review results of each target review agent are integrated to generate an interactive review report, including: The review results returned by each of the target review agents are integrated and deduplicated. Determine whether there are any conflicts in the integrated review results; If so, a ruling is made on conflicting review results, and the interactive review report is generated based on the ruling.

4. The method according to claim 1, characterized in that, Also includes: Determine the feedback information on questionable facts and operation log data during the interaction with the interactive review report; The knowledge base is updated according to the data type of the feedback information on the questionable facts, and / or the parameters of the large language model of the reviewing agent are updated, and the artificial intelligence model is trained according to the operation log data.

5. The method according to claim 4, characterized in that, The steps of updating the corresponding knowledge base and / or updating the parameters of the large language model of the reviewing agent based on the data type of the feedback information of the questionable facts include: Identify the knowledge-based data and reasoning-based data in the feedback information regarding the questionable facts; The preset knowledge graph and preset vector database are updated based on the knowledge-based data. The parameters of the large language model of the review agent are updated based on the inference data.

6. The method according to claim 4, characterized in that, The step of training the artificial intelligence model based on the operation log data includes: The operation log data is converted into a state-action sequence; The state-action sequence is used as a training sample, and the artificial intelligence model is trained through an adversarial imitation learning algorithm, so that the artificial intelligence model learns the mapping relationship between the review clinical research protocol and the review workflow.

7. A review device for clinical research protocols based on multi-agent systems, characterized in that, The device involves an artificial intelligence model, which has a pre-built correspondence between reviewing clinical research protocols and review workflows; The device includes: The workflow generation module is used to respond to the user's review request, obtain the corresponding clinical research protocol to be reviewed, and determine the review workflow corresponding to the clinical research protocol to be reviewed based on the correspondence; wherein, the review workflow is a directed acyclic graph including several sub-task nodes with dependencies. The node to be executed module is used to set the subtask nodes that have been executed by all upstream nodes in the directed acyclic graph as nodes to be executed. The review agent determination module is used to determine the target review agent based on the task type of the node to be executed. The report generation module is used to receive the review results returned by the target review agents, integrate the review results of each target review agent, and generate an interactive review report.

8. A computer electronic device, characterized in that, The method includes a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program, when executed by the processor, implements the steps of the review method for a multi-agent-based clinical research protocol as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium, which, when executed by a processor, implements the steps of the review method for a multi-agent-based clinical research protocol as described in any one of claims 1 to 6.

Images