Method and system for scientific review implementation based on agent cooperation and enhanced retrieval

Abstract

The application discloses a scientific review implementation method and system based on agent cooperation and enhanced retrieval, relates to the cross technical field of artificial intelligence and scientific literature review, and comprises the following steps: obtaining scientific literature to be reviewed and performing structural analysis to obtain an analysis result; constructing a knowledge graph and creating a double-channel retrieval mechanism for retrieval in the knowledge graph, wherein the double-channel retrieval mechanism comprises a vector retrieval channel and a graph retrieval channel; based on the analysis result and the knowledge graph, performing review tasks on review agents corresponding to each review dimension, combining a self-correction circulation mechanism, and outputting review conclusions and scores of each review dimension; obtaining the output of each review agent, identifying and processing logical contradictions, calculating a comprehensive score based on a weighted score tree, and generating a review report. The application can effectively implement automatic scientific review of scientific literature.

Description

Technical Field

[0001] This application relates to the interdisciplinary field of artificial intelligence and scientific literature review, specifically to a scientific review method and system based on intelligent agent collaboration and enhanced retrieval. Background Technology

[0002] With the exponential growth in scientific literature output—statistically, the total number of scientific publications roughly doubles every 15 years—the traditional peer review system, which relies on human experts, is facing unprecedented pressure. Problems such as long review cycles, insufficient review resources, and inconsistent review quality are becoming increasingly prominent, severely restricting the efficiency and quality of scientific research.

[0003] Currently, methods for using artificial intelligence to assist in the review of scientific literature have emerged, mainly including the following categories: (1) Text matching-based plagiarism detection and format checking tools: Tools such as Turnitin and CNKI mainly detect plagiarism by comparing text similarity and verifying format specifications based on rules. These tools can only identify surface-level problems and cannot deeply examine the scientific nature of the literature, including core dimensions such as logical consistency, methodological rationality, and data reliability. (2) Review assistance systems based on a single large language model: Systems such as Stanford Agentic Reviewer and PeerReviewerAI use large language models to conduct end-to-end review and feedback generation of papers. However, such systems have three significant drawbacks: First, the single model lacks the ability to specialize for different review dimensions, resulting in insufficient review depth; second, the inherent "illusion" problem of large language models makes the review opinions may lack factual basis; third, the lack of a self-correction mechanism for review conclusions makes it difficult to guarantee the reliability of the review. (3) Literature analysis systems based on traditional RAG (Retrieval-augmented Generation): Systems represented by Elicit introduce external literature knowledge into the model context through retrieval-augmented generation technology. However, traditional RAG methods are based on sparse or dense vector retrieval and can only obtain isolated text fragments. They cannot capture the semantic relationships and structured knowledge between documents (such as method evolution relationships, theoretical dependencies, experimental control relationships, etc.). They are seriously inadequate in complex scientific review scenarios that require multi-hop reasoning. (4) AI-generated content detection tools: Tools such as VIP AIGC detection and Wanfang AIGC detection focus on identifying whether the text is generated by AI. They fall under the category of content tracing and do not involve substantive review of the scientific nature of the documents themselves.

[0004] Therefore, existing methods cannot meet the needs for comprehensive, multi-level, and highly reliable scientific review of scientific literature. Specifically: 1. Scientific review encompasses multiple closely related but relatively independent dimensions, such as logical consistency, methodological rationality, data reliability, authenticity of literature citations, and innovativeness. Existing single-model architectures cannot adequately address the depth and professionalism of review across these dimensions. 2. Review conclusions must be based on verifiable scientific facts, not on model fabrications. The unstructured text fragments retrieved by traditional RAG retrieval are insufficient to provide reliable factual support for complex multi-hop scientific reasoning. 3. In high-risk scientific review scenarios, single-inference results are often unreliable, requiring an iterative optimization mechanism to continuously verify and improve review conclusions. Summary of the Invention

[0005] This application provides a scientific rigor review method and system based on agent collaboration and enhanced retrieval, which can effectively achieve automated scientific rigor review of scientific literature.

[0006] In a first aspect, embodiments of this application provide a method for scientific rigor review based on agent collaboration and enhanced retrieval, the method comprising: The scientific literature to be reviewed is obtained and structured parsed to obtain the parsing results, which include chapter segmentation results, argument-evidence relationship diagrams, and reference metadata lists. Construct a knowledge graph and create a dual-channel retrieval mechanism for searching within the knowledge graph, the dual-channel retrieval mechanism including a vector retrieval channel and a graph retrieval channel; Based on the parsing results and knowledge graph, the review agents corresponding to each review dimension execute review tasks, and combined with the self-correction loop mechanism, output the review conclusions and scores for each review dimension. The system acquires the outputs of each review agent, identifies and handles logical inconsistencies, calculates a comprehensive score based on a weighted scoring tree, and generates a review report.

[0007] In conjunction with the first aspect, in one implementation, the step of acquiring the scientific literature to be examined and performing structured analysis to obtain the analysis results specifically includes: The scientific literature to be reviewed is obtained, and the layout analysis model is used to detect and identify the chapter regions, which include the title, abstract, introduction, methods, results, discussion, and references. Semantic understanding of charts in scientific literature is performed using a multimodal large language model to extract data trends, statistical indicators, and conclusions claimed in the chart titles. Scientific claims in scientific literature are identified by natural language reasoning models, and the arguments on which these claims are based are traced. The arguments are then organized into a directed acyclic graph to obtain an argument-argument relationship graph. The arguments include experimental data, theoretical derivations, and literature citations. Extract each citation from the reference list and parse the DOI, author, journal, and year of each citation.

[0008] In conjunction with the first aspect, in one implementation method, The knowledge graph is constructed by continuously crawling literature metadata and abstracts from open academic databases, using information extraction models to extract entities and relationships, and constructing a continuously updated knowledge graph. The vector retrieval channel is used to perform semantic matching of unstructured text through dense paragraph retrieval; The graph retrieval channel is used to perform structured semantic path retrieval in the knowledge graph through a graph neural network; The retrieval results from the vector retrieval channel and the graph retrieval channel are fused through a path attention mechanism to generate factual evidence context that includes local semantic matching and global structural association for the current review dimension.

[0009] In conjunction with the first aspect, in one implementation method, The entities mentioned include research methods, theoretical concepts, experimental materials, and statistical indicators; The relationships include improvement upon, comparison with, dependence on, and contradiction with; The nodes of the knowledge graph include entity type, domain, and source literature; The edges of the knowledge graph include relation type, confidence level, and timestamp.

[0010] In conjunction with the first aspect, in one implementation, the review agent includes a logical consistency review agent, a methodological rationality review agent, a data reliability review agent, a document authenticity verification agent, and an innovation evaluation agent.

[0011] In conjunction with the first aspect, in one implementation method, The logical consistency review agent is used to check whether there are contradictions between the arguments, whether the claims in the abstract are adequately supported in the main text, and whether the experimental results support the conclusions drawn. The methodology rationality review agent is used to evaluate the scientific validity of experimental design, including whether the sample size is sufficient, whether the control group is set up reasonably, whether the statistical method is selected appropriately, and whether the experimental procedure is reproducible. The data reliability review agent is used to verify the internal consistency of data reported in scientific literature, including whether the text description and the chart data match, whether the statistical values ​​are calculated correctly, and whether the data change trend is consistent with the claimed conclusions. The document authenticity verification agent is used to verify the authenticity of references and the accuracy of citations. The innovative evaluation agent is used to evaluate the novelty of the contributions claimed in the paper.

[0012] In conjunction with the first aspect, in one implementation, the self-correcting loop mechanism specifically includes: The review agent generates a retrieval strategy corresponding to the current review content based on the current review task, and retrieves relevant factual evidence from the knowledge graph; A sufficiency assessment is performed on the context of the retrieved factual evidence to determine whether the current factual evidence can support the review conclusion, and the assessment result is obtained: If the evaluation result is partially sufficient or insufficient, the search strategy is adjusted and the search is performed again to obtain the evaluation result. This process is repeated until the evaluation result is sufficient, or the maximum number of iterations is exceeded, at which point the review conclusion is output and active processing and annotation are performed. If the assessment results are sufficient, the review conclusion will be output directly.

[0013] In conjunction with the first aspect, in one implementation, the step of acquiring the outputs of each review agent, identifying and processing logical contradictions, and calculating a comprehensive score based on a weighted scoring tree specifically includes: Obtain and summarize the outputs of the logical consistency review agent, the methodological rationality review agent, the data reliability review agent, the document authenticity verification agent, and the innovation evaluation agent; Identify and resolve logical contradictions in the various review conclusions; The overall score for the review conclusion is calculated based on a weighted scoring tree.

[0014] In conjunction with the first aspect, in one implementation, generating the review report specifically includes: Based on the output of the review agent, the results of handling logical contradictions, and the comprehensive score, a structured review report is generated; The scientific review report includes a comprehensive score, scores for each review dimension and detailed review comments, a list of key issues, an evidence traceability chain for each review comment, and an interactive, visual review conclusion dashboard.

[0015] Secondly, embodiments of this application provide a scientific rigor review system based on agent collaboration and enhanced retrieval, the system comprising: The document parsing engine is used to perform structured parsing of scientific documents to be reviewed, and to obtain the parsing results, which include chapter segmentation results, argument-evidence relationship diagrams, and reference metadata lists. A scientific knowledge graph module is used to construct a knowledge graph and create a dual-channel retrieval mechanism for searching within the knowledge graph, the dual-channel retrieval mechanism including a vector retrieval channel and a graph retrieval channel; A multi-dimensional review agent cluster includes an arbitration agent and multiple review agents corresponding to review dimensions. The review agents are used to perform review tasks based on parsing results and knowledge graphs, and output the review conclusion and score of the current review dimension in combination with a self-correction loop mechanism. The arbitration agent is used to obtain the output of each review agent, identify and handle logical contradictions, and calculate a comprehensive score based on a weighted scoring tree. A self-correcting loop engine is used to drive the review agent to perform review tasks based on a self-correcting loop mechanism. A review report generator, which is used to generate review reports based on the output of the arbitration agent.

[0016] The beneficial effects of the technical solutions provided in this application include: (1) Multi-dimensional in-depth review capability: Through the parallel collaboration of dedicated review agents, a comprehensive in-depth review of scientific literature is achieved, covering logical consistency, methodological rationality, data reliability, authenticity of literature citations and innovation. The coverage of review dimensions far exceeds that of existing single model solutions. (2) Effectively suppressing hallucinations and making review conclusions traceable: Through dual-channel RAG retrieval enhanced by knowledge graph, each review opinion is anchored to a specific fact node in the knowledge graph, achieving zero-hallucination citation of review conclusions; (3) Self-correction mechanism ensures the reliability of review: The self-correction loop mechanism enables each review agent to automatically iterate and optimize the retrieval strategy when there is insufficient evidence, effectively avoiding review omissions caused by a single retrieval failure; (4) Significantly improved review efficiency: Effectively reduces the time required to complete a full-dimensional review of scientific literature. Compared with human expert review, the review efficiency is effectively improved. At the same time, the review results are highly consistent with the opinions of human experts. (5) Domain Adaptability: The continuous updating mechanism of the knowledge graph enables this application to automatically adapt to the review requirements of different disciplines without the need for model retraining for each domain. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating the scientific rigor review method based on agent collaboration and enhanced retrieval proposed in this application. Figure 2 The flowchart for the implementation of the self-correcting loop mechanism; Figure 3 This is the overall architecture diagram of the scientific review system based on agent collaboration and enhanced retrieval for this application. Detailed Implementation

[0018] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0019] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0020] In the first aspect, embodiments of this application provide a scientific rigor review method based on agent collaboration and enhanced retrieval. Specifically, it adopts a multi-agent collaborative architecture driven by a large language model (LLM) and integrates knowledge graph-enhanced retrieval-enhanced generation (KG-RAG) technology to perform automated scientific rigor review of scientific literature.

[0021] In one embodiment, reference is made to Figure 1 , Figure 1 This is a flowchart illustrating the scientific rigor review method based on agent collaboration and enhanced retrieval proposed in this application. Figure 1 As shown, the scientific rigor review methods based on agent collaboration and enhanced retrieval include: S1: Obtain the scientific literature to be reviewed (such as papers, articles, etc.) and perform structured parsing to obtain the parsing results, which include chapter segmentation results, argument-evidence relationship diagrams, and reference metadata lists; S2: Construct a knowledge graph and create a dual-channel retrieval mechanism for searching within the knowledge graph, the dual-channel retrieval mechanism including a vector retrieval channel and a graph retrieval channel; S3: Based on the parsing results and knowledge graph, the review agents corresponding to each review dimension perform review tasks, and combined with the self-correction loop mechanism, output the review conclusions and scores for each review dimension; S4: Obtain the output of each review agent, identify and handle logical contradictions, calculate the comprehensive score based on the weighted scoring tree, and generate a review report.

[0022] Furthermore, in one embodiment, the scientific literature to be examined is obtained and structured parsing is performed to obtain the parsing results, specifically including: S101: Obtain the scientific literature to be reviewed, perform region detection through the layout analysis model, and identify the chapter regions, which include the title, abstract, introduction, methods, results, discussion, and references; Specifically, based on the layout analysis model, region detection is performed on scientific literature documents to identify chapter regions such as title, abstract, introduction, methods, results, discussion, and references; S102: Semantic understanding of charts and graphs in scientific literature is performed using a multimodal large language model to extract data trends, statistical indicators, and conclusions claimed in the chart titles; the multimodal large language model can be GPT-5.4, ClaudeOpus 4.7, or Qwen3.6-Max-Preview; S103: Identify scientific claims in scientific literature through a natural language reasoning model, trace the arguments on which the scientific claims depend, organize the arguments into a directed acyclic graph, and obtain an argument-argument relationship graph. The arguments include experimental data, theoretical derivations, and literature citations. Specifically, scientific claims in scientific literature are identified through natural language reasoning (NLI) models, and the evidence on which they are based, including experimental data, theoretical derivations, and literature citations, is traced and organized into a directed acyclic graph. S104: Extract each citation from the reference list and parse the DOI, author, journal, and year for each citation. Specifically, extract each citation from the reference list and parse its metadata fields, including DOI (Digital Object Identifier), author, journal, and year.

[0023] Furthermore, in one embodiment, the knowledge graph is constructed by continuously crawling document metadata and abstracts from open academic databases, using an information extraction model to extract entities and relationships, and constructing a continuously updated knowledge graph.

[0024] Specifically, a knowledge graph construction subsystem is employed to automatically and continuously crawl literature metadata and abstracts from open academic databases such as PubMed, arXiv, and Semantic Scholar. An information extraction model is used to extract entities (research methods, theoretical concepts, experimental materials, statistical indicators, etc.) and relationships (improved upon, compared to, dependent on, contradictory to, etc.) from these sources, constructing and continuously updating the knowledge graph. Nodes in the knowledge graph contain attributes such as entity type, domain, and source literature, while edges contain attributes such as relationship type, confidence level, and timestamp.

[0025] In this application, the vector retrieval channel is used to perform semantic matching on unstructured text through dense paragraph retrieval; the graph retrieval channel is used to perform structured semantic path retrieval in the knowledge graph through a graph neural network; wherein, the retrieval results of the vector retrieval channel and the graph retrieval channel are fused through a path attention mechanism to generate factual evidence context containing local semantic matching and global structural association for the current review dimension. Local semantic matching refers to the output result of the vector retrieval channel, that is, finding the document paragraphs in the unstructured text of the knowledge graph that are highly similar in semantic expression to the scientific claim to be under review. Global structural association refers to the output result of the graph retrieval channel, that is, tracing knowledge nodes that have indirect logical association with the current scientific claim along multiple paths in the entity relationship network of the knowledge graph through a graph neural network.

[0026] The specific process of vector retrieval includes: first, converting text such as document abstracts and paragraphs from the knowledge graph into vectors using an encoding model and storing them in a vector database; then, converting the scientific claim to be reviewed into a query vector; and finally, searching the vector database for the paragraphs most similar to the query vectors and returning them as the basis for review. This method is based on semantic matching rather than literal matching. For example, "significant difference" and "statistical significance" are different in expression but similar in meaning. Traditional keyword retrieval would miss the latter, while vector retrieval can identify both as related content and recall them together, thereby improving the coverage of the review basis.

[0027] The specific process of graph retrieval includes: encoding entities (such as papers, authors, concepts, methods, etc.) and their relationships (such as "citation," "support," "refutation," etc.) in the knowledge graph into structured vectors using a graph neural network, making semantically related nodes in the graph adjacent to each other in the vector space. Then, starting with the scientific claim to be examined, the corresponding entity node is located in the knowledge graph, and the search is performed step by step along the relationship edges to find semantic paths that are structurally related to the scientific claim (e.g., a certain argument → citing a certain document → the document supports a certain experimental conclusion). The retrieved paths and the node information on the paths are then summarized and returned as structured factual evidence for this examination dimension. This method can discover indirect relationships across nodes. For example, although a scientific claim does not directly cite a certain document, a logical contradiction can be found in the conclusion of that document through two-hop paths.

[0028] Specifically, a dual-channel retrieval mechanism is constructed using a knowledge graph retrieval subsystem. The vector retrieval channel performs semantic matching on unstructured text through dense paragraph retrieval (DPR); the graph retrieval channel performs structured semantic path retrieval in the knowledge graph using a graph neural network (GNN), supporting multi-hop reasoning. The retrieval results from the two channels are fused using a path attention mechanism to generate factual context that includes both local semantic matching and global structural relationships.

[0029] This application maintains a domain-adaptive scientific knowledge graph as an external source of factual evidence for the review process. During the review, based on the core entities and research topics in the analysis results, a dual-channel retrieval mechanism is used in the knowledge graph to construct targeted factual evidence contexts for each review dimension.

[0030] Furthermore, in one embodiment, the review agents include a logical consistency review agent, a methodological rationality review agent, a data reliability review agent, a document authenticity verification agent, and an innovativeness evaluation agent. Each review agent embeds a self-correcting "Retrieve-Evaluate-Refine" self-correction loop mechanism during the reasoning process. Furthermore, this application may also include an arbitration agent, which is used to acquire the outputs of each review agent, identify and handle logical contradictions, calculate a comprehensive score based on a weighted scoring tree, and then output the result.

[0031] After receiving their respective review content, the five review agents start the review tasks in parallel. Each review agent independently executes a self-correction loop and searches the knowledge graph using a dual-channel retrieval mechanism to generate review conclusions and scores for their respective review dimensions.

[0032] In this application, the logical consistency review agent is used to check whether there are contradictions between arguments, whether the claims in the abstract are adequately supported in the main text, and whether the experimental results support the conclusions drawn. Specifically, the logical consistency review agent calls the argument-evidence relationship graph to perform formal logical chain verification and generates location labels for the logical breakpoints found.

[0033] In this application, the methodological rationality review agent is used to evaluate the scientific validity of experimental designs, including whether the sample size is sufficient, whether the control group is reasonably set up, whether the statistical methods are appropriately selected, and whether the experimental procedure is reproducible. Specifically, the methodological rationality review agent uses a knowledge graph to search for standard methodologies used in published studies in the same field and uses them as a benchmark for comparative analysis.

[0034] In this application, the data reliability review agent is used to verify the internal consistency of data presented in scientific literature, including whether the textual descriptions and graphical data match, whether the statistical calculations are correct (such as the reasonableness of p-values, confidence intervals, and effect sizes), and whether the data trends are consistent with the claimed conclusions. The data reliability review agent is equipped with numerical calculation tools and can independently verify the statistical data in the scientific literature.

[0035] In this application, a document authenticity verification agent is used to verify the authenticity of references and the accuracy of citations. Specifically, the document authenticity verification agent uses external academic APIs (CrossRef, Semantic Scholar API) to verify one by one whether the DOI of each citation truly exists, whether the cited content is consistent with the original text, and whether there are any citations taken out of context.

[0036] In this application, an innovation assessment agent is used to evaluate the novelty of the contributions claimed in the paper. Specifically, the innovation assessment agent performs large-scale literature comparisons using a knowledge graph to retrieve existing works with highly similar methods and conclusions to the scientific literature under review, and assesses whether their incremental contributions have substantial innovation.

[0037] Furthermore, in one embodiment, the self-correcting loop mechanism specifically includes: S301: The review agent generates a retrieval strategy (such as a knowledge graph query statement) corresponding to the current review content based on the current review task, and retrieves relevant factual evidence from the knowledge graph; S302: Assess the sufficiency of the context of the retrieved factual evidence to determine whether the current factual evidence supports the review conclusion, and obtain the assessment result (including sufficient, partially sufficient, or insufficient): If the evaluation result is partially sufficient or insufficient, the search strategy is adjusted (such as expanding search keywords, adjusting the map traversal depth, switching search channels, etc.) and the search is performed again to obtain the evaluation result. This process is repeated until the evaluation result is sufficient, or the maximum number of iterations (default is 3) is exceeded, at which point the review conclusion is output and active processing is performed (e.g., the evidence is marked as insufficient, and manual review is recommended). If the assessment is deemed sufficient, the review conclusion will be output directly. See also Figure 2 The diagram shown is a flowchart illustrating the implementation of the self-correcting loop mechanism.

[0038] Furthermore, in one embodiment, the outputs of each review agent are acquired, logical contradictions are identified and processed, and a comprehensive score is calculated based on a weighted scoring tree, specifically including: S401: Obtain and summarize the outputs of the logical consistency review agent, the methodological rationality review agent, the data reliability review agent, the document authenticity verification agent, and the innovation evaluation agent; S402: Identify and handle logical contradictions in the review conclusions (e.g., the logical consistency review agent believes a certain conclusion is valid, but the data reliability review agent finds anomalies in the data supporting that conclusion); S403: Calculate the comprehensive score of the review conclusion based on the weighted scoring tree.

[0039] Furthermore, in one embodiment, generating a review report specifically includes: generating a structured review report based on the output of the review agent, the result of handling logical contradictions, and the comprehensive score, so that users can backtrack and verify each review opinion; that is, accepting the summary output of the arbitration agent to generate a structured review report; The scientific review report includes a comprehensive score (1-10 points), scores for each review dimension and detailed review comments, a list of key issues (graded by severity: fatal flaws, important issues, and improvement suggestions), an evidence traceability chain for each review comment (linked to specific nodes and edges in the knowledge graph), and an interactive, visual review conclusion dashboard.

[0040] The following example illustrates the scientific rigor review method based on agent collaboration and enhanced retrieval proposed in this application.

[0041] Implementation and verification were conducted in the following environment: the server was deployed on a computing cluster equipped with 8×NVIDIA A100 (80GB) GPUs, and the operating system was Ubuntu 22.04LTS; the core large language model used LLaMA-3.3-70B-Instruct as the inference engine for each agent (it can also be replaced by a commercial or open-source LLM with equivalent capabilities), the knowledge graph was stored in the Neo4j5.x graph database, vector retrieval was indexed based on FAISS, and the document parsing module was implemented based on LandingAI AgenticDocument Extraction.

[0042] Taking the biomedical field as an example, the knowledge graph construction process is as follows: (1) Data collection: Incrementally crawled literature abstracts and metadata from the PubMed database for the past 10 years, with an initial scale of about 35 million articles; crawled about 2.5 million computer science and interdisciplinary papers from arXiv. (2) Entity extraction: Using a domain-fine-tuned named entity recognition model (based on the BioBERT architecture), six types of entities are extracted from the literature abstract: Method, Concept, Material, Dataset, Metric, and Finding; the F1 score of entity extraction reaches 0.89; (3) Relation extraction: The relation extraction model is used to identify eight types of relations between entities: ImproveUpon, ComparedWith, DependsOn, ContradictedBy, Validates, Uses, Produces, and EquivalentTo. (4) Knowledge graph storage: The extracted entities and relations are stored in the Neo4j graph database, and a 768-dimensional semantic embedding vector is generated for each entity node and stored in the FAISS vector index to support dual-channel retrieval. (5) Incremental update: The incremental crawling and extraction process is automatically executed every day to incorporate newly published literature knowledge into the knowledge graph.

[0043] The specific implementation of the review agent will be illustrated using a logical consistency review agent as an example. The specific implementation process includes: (1) System Prompt Design: Clearly define the role of the logical consistency review agent (scientific logic review expert), the review task (check the logical consistency within the paper), and the output format (a structured list of questions, each containing a question description, severity, location information, and evidence citations). (2) Toolset configuration: The tools that the logical consistency review agent can call include: knowledge graph query tool (execute Cypher query statement to retrieve relevant facts from Neo4j), argument-evidence relationship graph traversal tool (perform path search on the Claim-Evidence Graph built by the document parsing engine), and natural language reasoning tool (determine the implication, contradiction, and neutral relationship between two text fragments). (3) Self-correcting loop example: The logical consistency review agent first generates an initial search query based on the core claims in the paper abstract, such as "MATCH (m:Method)-[:ImproveUpon]->(m2:Method) WHERE m.nameCONTAINS 'Method name to be reviewed' RETURN m, m2". If the returned result is empty, the evaluation module judges it as "insufficient", and the refinement module expands the query to a fuzzy match or synonym replacement of the method name to be reviewed and then retryes.

[0044] Finally, experimental verification was conducted. To verify the effectiveness, a test set of 500 scientific papers was constructed, covering the fields of biomedicine, computer science, and materials science. Among them, 250 papers were known to have scientific problems (the specific types and locations of the problems were manually marked), and 250 papers were high-quality papers that had passed normal peer review.

[0045] Three control groups were set up in the experiment: (A) direct review of a single LLM (without RAG, without multi-agent); (B) review of a single LLM enhanced with traditional RAG (based on FAISS vector retrieval); and (C) the application in question (KG-RAG + multi-agent + self-correcting loop). The main experimental results are shown in Table 1 below.

[0046] Table 1

[0047] Experimental results show that this application maintains high examination efficiency while significantly outperforming the control scheme in all aspects of examination quality.

[0048] It should be further noted that, for the scientific rigor review method based on agent collaboration and enhanced retrieval proposed in this application, in specific applications, the core large language model can be replaced with other models with equivalent reasoning capabilities, such as GPT-4 and Claude; the domain scope of the knowledge graph can be expanded to social science fields such as law and economics according to the application scenario; the number and type of review agents can be increased or decreased according to review needs (such as adding "ethics compliance review agents"); and the maximum number of iterations of the self-correction loop can be adjusted according to the review accuracy requirements.

[0049] The scientific rigor review method based on agent collaboration and enhanced retrieval in this application embodiment is as follows: (1) Multi-dimensional in-depth review capability: Through the parallel collaboration of dedicated review agents, a comprehensive in-depth review of scientific literature is achieved, covering logical consistency, methodological rationality, data reliability, authenticity of literature citations and innovation. The coverage of review dimensions far exceeds that of existing single model schemes. Experimental results show that the F1 score of this application on the multi-dimensional review task reaches 0.82, which is 23 percentage points higher than the baseline single LLM scheme. (2) Effective suppression of hallucinations and traceability of review conclusions: Through dual-channel RAG retrieval enhanced by knowledge graph, each review opinion is anchored to a specific fact node in the knowledge graph, achieving zero hallucination citation of review conclusions; In the experiment, the Unsupported Claims Rate of this application was only 3.2%, while the single LLM scheme of the control group was 27.8%; (3) Self-correction mechanism ensures the reliability of review: The self-correction loop mechanism enables each review agent to automatically iterate and optimize the retrieval strategy when there is insufficient evidence, effectively avoiding review omissions caused by a single retrieval failure; experiments show that after the self-correction loop, the accuracy of the review opinions increased from 71.5% in the first round to 86.3%; (4) Significantly improved review efficiency: The average time for this application to complete a full-dimensional review of a standard-length (approximately 8,000 words) scientific paper is 4.2 minutes, which is more than 20 times more efficient than the average 2-4 week cycle of human expert review. At the same time, the consistency between the review results and the human expert review opinions (measured by Cohen's Kappa coefficient) reaches 0.73, which is at the "basically consistent" level. (5) Domain Adaptability: The continuous updating mechanism of the knowledge graph enables this application to automatically adapt to the review requirements of different disciplines without the need for model retraining for each domain.

[0050] Secondly, embodiments of this application also provide a scientific rigor review system based on agent collaboration and enhanced retrieval.

[0051] In one embodiment, reference is made to Figure 3 , Figure 3 This is a diagram illustrating the overall architecture of the scientific rigor review system based on agent collaboration and enhanced retrieval, as described in this application. Figure 3 As shown, the scientific rigor review system based on agent collaboration and enhanced retrieval includes: a document parsing engine, a scientific knowledge graph module, a multi-dimensional review agent cluster, a self-correcting loop engine, and a review report generator.

[0052] The document parsing engine is used to perform structured parsing of scientific documents to be reviewed, obtaining parsing results, including chapter segmentation results, argument-evidence relationship diagrams, and a list of reference metadata. Specifically, the document parsing engine is responsible for converting the input scientific documents (supporting formats such as PDF, DOCX, and LaTeX) into a structured representation.

[0053] The scientific knowledge graph module is used to construct a knowledge graph and create a dual-channel retrieval mechanism for searching within the knowledge graph. This dual-channel retrieval mechanism includes a vector retrieval channel and a graph retrieval channel. Specifically, the scientific knowledge graph module is used to maintain a domain-adaptive scientific knowledge graph as an external source of factual evidence for the review process.

[0054] The multi-dimensional review agent cluster includes an arbitration agent and multiple review agents corresponding to different review dimensions. The review agents execute review tasks based on parsing results and a knowledge graph, and, combined with a self-correcting loop mechanism, output the review conclusion and score for the current review dimension. The arbitration agent acquires the outputs of each review agent, identifies and handles logical contradictions, and calculates a comprehensive score based on a weighted scoring tree. Specifically, the multi-dimensional review agent cluster is the core reasoning layer of the scientific review implementation system, consisting of five dedicated review agents and one arbitration agent. Each agent is based on a shared large language model kernel but is equipped with different prompts, dedicated toolsets, and domain knowledge retrieval strategies.

[0055] The self-correcting loop engine drives the review agents to perform review tasks based on a self-correcting loop mechanism. Specifically, each review agent embeds a "Retrieve-Evaluate-Refine" self-correcting loop mechanism during the inference process.

[0056] The review report generator is used to generate review reports based on the output of the arbitration agent.

[0057] The terms "comprising" and "having," and any variations thereof, in the specification, claims, and accompanying drawings of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such process, method, product, or apparatus. The terms "first," "second," and "third," etc., are used to distinguish different objects, etc., and do not indicate a sequence, nor do they limit "first," "second," and "third" to different types.

[0058] In the description of the embodiments of this application, terms such as "exemplary," "for example," or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplary," "for example," or "for instance" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary," "for example," or "for instance" is intended to present the relevant concepts in a concrete manner.

[0059] In the description of the embodiments of this application, unless otherwise stated, " / " means "or". For example, A / B can mean A or B. The "and / or" in the text is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist simultaneously, and B exists alone. In addition, in the description of the embodiments of this application, "multiple" means two or more.

[0060] In some processes described in the embodiments of this application, multiple operations or steps are included in a specific order. However, it should be understood that these operations or steps may not be executed in the order they appear in the embodiments of this application, or they may be executed in parallel. The sequence number of the operation is only used to distinguish different operations, and the sequence number itself does not represent any execution order. In addition, these processes may include more or fewer operations, and these operations or steps may be executed sequentially or in parallel, and these operations or steps may be combined.

[0061] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device to execute the methods described in the various embodiments of this application.

[0062] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A method for scientific rigor review based on agent collaboration and enhanced retrieval, characterized in that, The scientific rigor review method based on agent collaboration and enhanced retrieval includes: The scientific literature to be reviewed is obtained and structured parsed to obtain the parsing results, which include chapter segmentation results, argument-evidence relationship diagrams, and reference metadata lists. Construct a knowledge graph and create a dual-channel retrieval mechanism for searching within the knowledge graph, the dual-channel retrieval mechanism including a vector retrieval channel and a graph retrieval channel; Based on the parsing results and knowledge graph, the review agents corresponding to each review dimension execute review tasks, and combined with the self-correction loop mechanism, output the review conclusions and scores for each review dimension. The system acquires the outputs of each review agent, identifies and handles logical inconsistencies, calculates a comprehensive score based on a weighted scoring tree, and generates a review report.

2. The scientific rigor review method based on agent collaboration and enhanced retrieval as described in claim 1, characterized in that, The process of acquiring the scientific literature to be reviewed and performing structured analysis to obtain the analysis results specifically includes: The scientific literature to be reviewed is obtained, and the layout analysis model is used to detect and identify the chapter regions, which include the title, abstract, introduction, methods, results, discussion, and references. Semantic understanding of charts in scientific literature is performed using a multimodal large language model to extract data trends, statistical indicators, and conclusions claimed in the chart titles. Scientific claims in scientific literature are identified by natural language reasoning models, and the arguments on which these claims are based are traced. The arguments are then organized into a directed acyclic graph to obtain an argument-argument relationship graph. The arguments include experimental data, theoretical derivations, and literature citations. Extract each citation from the reference list and parse the DOI, author, journal, and year of each citation.

3. The scientific rigor review method based on agent collaboration and enhanced retrieval as described in claim 1, characterized in that: The knowledge graph is constructed by continuously crawling literature metadata and abstracts from open academic databases, using information extraction models to extract entities and relationships, and constructing a continuously updated knowledge graph. The vector retrieval channel is used to perform semantic matching of unstructured text through dense paragraph retrieval; The graph retrieval channel is used to perform structured semantic path retrieval in the knowledge graph through a graph neural network; The retrieval results from the vector retrieval channel and the graph retrieval channel are fused through a path attention mechanism to generate factual evidence context that includes local semantic matching and global structural association for the current review dimension.

4. The scientific rigor review method based on agent collaboration and enhanced retrieval as described in claim 3, characterized in that: The entities mentioned include research methods, theoretical concepts, experimental materials, and statistical indicators; The relationships include improvement upon, comparison with, dependence on, and contradiction with; The nodes of the knowledge graph include entity type, domain, and source literature; The edges of the knowledge graph include relation type, confidence level, and timestamp.

5. The scientific rigor review method based on agent collaboration and enhanced retrieval as described in claim 1, characterized in that: The review agents include a logical consistency review agent, a methodological rationality review agent, a data reliability review agent, a document authenticity verification agent, and an innovation evaluation agent.

6. The scientific rigor review method based on agent collaboration and enhanced retrieval as described in claim 5, characterized in that: The logical consistency review agent is used to check whether there are contradictions between the arguments, whether the claims in the abstract are adequately supported in the main text, and whether the experimental results support the conclusions drawn. The methodology rationality review agent is used to evaluate the scientific validity of experimental design, including whether the sample size is sufficient, whether the control group is set up reasonably, whether the statistical method is selected appropriately, and whether the experimental procedure is reproducible. The data reliability review agent is used to verify the internal consistency of data reported in scientific literature, including whether the text description and the chart data match, whether the statistical values ​​are calculated correctly, and whether the data change trend is consistent with the claimed conclusions. The document authenticity verification agent is used to verify the authenticity of references and the accuracy of citations. The innovative evaluation agent is used to evaluate the novelty of the contributions claimed in the paper.

7. The scientific rigor review method based on agent collaboration and enhanced retrieval as described in claim 6, characterized in that, The self-correcting loop mechanism specifically includes: The review agent generates a retrieval strategy corresponding to the current review content based on the current review task, and retrieves relevant factual evidence from the knowledge graph; A sufficiency assessment is performed on the context of the retrieved factual evidence to determine whether the current factual evidence can support the review conclusion, and the assessment result is obtained: If the evaluation result is partially sufficient or insufficient, the search strategy is adjusted and the search is performed again to obtain the evaluation result. This process is repeated until the evaluation result is sufficient, or the maximum number of iterations is exceeded, at which point the review conclusion is output and active processing and annotation are performed. If the assessment results are sufficient, the review conclusion will be output directly.

8. The scientific rigor review method based on agent collaboration and enhanced retrieval as described in claim 5, characterized in that, The process of acquiring the outputs of each review agent, identifying and handling logical contradictions, and calculating a comprehensive score based on a weighted scoring tree specifically includes: Obtain and summarize the outputs of the logical consistency review agent, the methodological rationality review agent, the data reliability review agent, the document authenticity verification agent, and the innovation evaluation agent; Identify and resolve logical contradictions in the various review conclusions; The overall score for the review conclusion is calculated based on a weighted scoring tree.

9. The scientific rigor review method based on agent collaboration and enhanced retrieval as described in claim 8, characterized in that, The generation of the review report specifically includes: Based on the output of the review agent, the results of handling logical contradictions, and the comprehensive score, a structured review report is generated; The scientific review report includes a comprehensive score, scores for each review dimension and detailed review comments, a list of key issues, an evidence traceability chain for each review comment, and an interactive, visual review conclusion dashboard.

10. A scientific rigor review system based on agent collaboration and enhanced retrieval, characterized in that, The scientific rigor review system based on agent collaboration and enhanced retrieval includes: The document parsing engine is used to perform structured parsing of scientific documents to be reviewed, and to obtain the parsing results, which include chapter segmentation results, argument-evidence relationship diagrams, and reference metadata lists. A scientific knowledge graph module is used to construct a knowledge graph and create a dual-channel retrieval mechanism for searching within the knowledge graph, the dual-channel retrieval mechanism including a vector retrieval channel and a graph retrieval channel; A multi-dimensional review agent cluster includes an arbitration agent and multiple review agents corresponding to review dimensions. The review agents are used to perform review tasks based on parsing results and knowledge graphs, and output the review conclusion and score of the current review dimension in combination with a self-correction loop mechanism. The arbitration agent is used to obtain the output of each review agent, identify and handle logical contradictions, and calculate a comprehensive score based on a weighted scoring tree. A self-correcting loop engine is used to drive the review agent to perform review tasks based on a self-correcting loop mechanism. A review report generator, which is used to generate review reports based on the output of the arbitration agent.

Images