An academic paper critical reading method based on multi-agent debate

Abstract

The application discloses a kind of based on multi-agent debate's academic paper critical reading method, it is related to academic paper analysis technical field, including: receiving paper PDF file, extracting paper full-text object;According to the structured field of pre-established extraction structured information, generate paper identifier and store in database;According to regular extraction, CrossRef API inquiry, three levels of large language model high confidence inference are tried in turn to find the digital object identifier of paper and store in database;Start multi-agent debate, generate strong type object of criticality;Convert into criticality evaluation report, persistent storage in database.The application extracts full-text structured information, multi-agent debate and persistent storage, improves the objectivity and efficiency of academic paper critical reading, can generate criticality evaluation report, suppresses the flattery deviation of single large language model.

Description

Technical Field

[0001] This application relates to the field of academic paper analysis technology, and in particular to a critical interpretation method for academic papers based on multi-agent debate. Background Technology

[0002] Peer review is the core mechanism for verifying the scientific validity of academic papers. In the standard process, editors invite 2-3 experts in the field to conduct anonymous reviews, with the process typically taking months to years. This mechanism has inherent flaws, including limited coverage (with over 2 million papers published globally each year, review resources are severely insufficient), strong subjectivity (different reviewers' conclusions may contradict each other), and being unfriendly to young researchers (who struggle to independently judge paper quality without expert guidance). Therefore, research into academic paper analysis techniques is necessary.

[0003] In recent years, Large Language Models (LLMs), represented by GPT-4, Claude, and DeepSeek, have been widely applied in academic scenarios, including automatic abstract generation, keyword extraction, literature review assistance, and format checking. The mainstream technical approach for existing LLM academic applications involves users inputting their paper text into a single LLM module, obtaining analysis results through one or more rounds of dialogue. The core flaw of this approach lies in the "sycophancy" inherent in LLMs without external constraints, which tends to provide positive, vague summaries without actively identifying flaws in the paper; furthermore, this single perspective fails to present the diverse evaluations of a paper within the academic community.

[0004] Multi-agent systems are an important research direction in the field of artificial intelligence. Their core idea is to decompose complex tasks and have multiple agents with different role definitions collaborate to complete them. Different agents possess different objective functions and prompts, enabling them to challenge and defend each other, generating a more comprehensive and objective output through "competition" than a single agent. Currently, multi-agent systems have applications in software development (such as AutoGPT and MetaGPT), but in the context of critical review of academic papers, there is still no mature solution combining multi-agent debate mechanisms with persistent literature databases.

[0005] Computational materials science (represented by density functional theory (DFT) calculations) is a highly specialized research field. Papers in this field are characterized by: a large number of quantitative results (bandgap energy, magnetic moment, phonon dispersion, etc.); complex methodological parameters (exchange-correlation functional types such as HSE06 / PBE, Hubbard U correction values, and computational software versions such as VASP6.3); and specialized nomenclature for material systems (e.g., MnO2, perovskite-type LaFeO3). General-purpose literature management tools (such as Zotero and EndNote) cannot provide structured storage and precise retrieval of these field-specific fields.

[0006] In summary, the objectivity and efficiency of critical interpretation of academic papers (especially those in the field of computational materials science) in existing technologies need to be improved, and the bias of a single large language model needs to be reduced. Summary of the Invention

[0007] This application provides a critical interpretation method for academic papers based on multi-agent debate, which addresses the problems in the existing technology where the objectivity and efficiency of critical interpretation of academic papers need to be improved, and the catering bias of a single large language model needs to be reduced.

[0008] On the one hand, this application provides a method for critical interpretation of academic papers based on multi-agent debate, including the following steps: Step 1: Receive the PDF file of the paper uploaded by the user, extract the full text and identify the chapter structure to obtain the full text object of the paper, including the title, abstract and content of each chapter.

[0009] Step 2: Input the full text of the paper into the large language model, extract structured information according to a number of preset structured fields, generate paper identifiers and store them in the database.

[0010] Step 3: Using the paper identifier as the operation object, attempt to find the paper's digital object identifier and store it in the database in sequence according to three levels: regular expression extraction, CrossRef API query, and high-confidence inference of large language model.

[0011] Step four: Extract keywords from the full text of the paper, accumulate the frequencies of existing keywords in the database, and establish the association between the paper and the keywords.

[0012] Step 5: Initiate multi-agent debate by calling on the advocate agent, the questioner agent, and the heuristic agent to conduct interactive debates according to the configured number of debate rounds, and the judge agent generates objects with strong critical types.

[0013] Step six: Convert the critically typed object into a structured critical evaluation report, persist it in the database, and associate it with the paper identifier.

[0014] Step 7: The browser client polls the task status, obtains a structured critical evaluation report, and displays it on the front end.

[0015] In one possible implementation, in step two, the structured fields include: publication year, author list, journal name, material system, calculation method parameters, physical phenomena, core conclusions, key quantitative results, research limitations, future research directions, and data sources.

[0016] In one possible implementation, step three, the regular expression extraction includes: using regular expressions to match numeric object identifiers that conform to a certain format in the full-text paper object.

[0017] The CrossRef API query includes: if the regular expression extraction does not find the numeric object identifier, calling the CrossRef API query with the paper title and publication year as parameters to extract the numeric object identifier based on the similarity score.

[0018] The high-confidence inference of the large language model includes: if neither the regular expression extraction nor the CrossRef API query finds a numeric object identifier, the paper title and author are sent to the large language model for high-confidence inference to extract the numeric object identifier.

[0019] If the regular expression extraction, the CrossRef API query, and the high-confidence inference of the large language model all fail to find the numeric object identifier, the numeric object identifier field will be left blank, which will not affect subsequent processes.

[0020] In one possible implementation, in step four, the following database operation is performed for each keyword: if the keyword does not exist in the database, a new row is inserted; if the keyword already exists in the database, the corresponding frequency is incremented by one.

[0021] In one possible implementation, in step five, during each round of debate, the advocate agent and the skeptic agent execute concurrently without waiting for each other, while simultaneously making independent requests to the large language model and returning their statements.

[0022] Then the heuristic agent is executed sequentially. The heuristic agent reads all the statements made by the advocate and the skeptic agent in this round and all previous rounds before making its own statement.

[0023] In one possible implementation, in step five, the advocate agent is prompted to list the core contributions of the paper, with each argument accompanied by evidence of original text citations. In subsequent rounds, the advocate agent is also required to respond to each criticism from the skeptic agent in the previous round.

[0024] The prompts for the questioner agent require it to list methodological defects, statistical problems, and logical loopholes. Each problem must be marked with its severity and accompanied by original evidence. In subsequent rounds, the questioner agent is also required to provide in-depth rebuttals to the defenses of the supporter agent.

[0025] The prompts for the heuristic agent require the heuristic agent to speak from the perspective of cross-domain value and future research directions. Each heuristic must be accompanied by a cross-domain anchor and marked with feasibility.

[0026] In one possible implementation, in step five, after all rounds of interactive debate have ended, the referee agent concatenates the first 3000 words of the full paper and all debate records in the message history list as input, sends them to the large language model, outputs the evaluation result JSON string according to the predefined JSON Schema format, and generates a critically typed object after parsing.

[0027] In one possible implementation, a critical interpretation method for academic papers based on multi-agent debate further includes: Step eight, full-text literature search: When the backend receives keywords input by the user, it performs a case-insensitive LIKE join query on the structured fields of the papers already stored in the database.

[0028] In one possible implementation, a critical interpretation method for academic papers based on multi-agent debate further includes: Step nine, PDF file access control: When receiving the uploaded PDF file of the paper in step one, the user's identity is parsed and the file is stored locally on the server. When a user requests to access the PDF file of the paper, the user token carried in the request is verified.

[0029] If the user identifier in the user token matches the user identifier corresponding to the paper PDF file, the paper PDF file is returned; otherwise, a 403 / 401 error is returned.

[0030] In one possible implementation, a method for critically interpreting academic papers based on multi-agent debate also includes: Step 10, a paper-specific AI dialogue: When a user's question is received, the structured fields of the corresponding paper and the first 3,000 words of the critical evaluation report are read from the database, and together with historical messages and the current question, they are used to construct the input message list for the large language model.

[0031] The large language model's streaming interface is called, and each generated token is written to the HTTP response stream in Server-Sent Events format. The front end then displays the response content word by word in real time.

[0032] The critical interpretation method for academic papers based on multi-agent debate in this application has the following advantages: By extracting structured information from the full text, engaging in multi-agent debate, and persistent storage, the objectivity and efficiency of critical interpretation of academic papers are improved. Critical evaluation reports can be generated, and the conformity bias of a single large language model can be suppressed.

[0033] By pre-setting structured fields, the information integrity and retrieval accuracy of the literature database are improved, making cross-paper comparison and refined retrieval possible.

[0034] By sequentially attempting to find and store the digital object identifiers of the paper in the database through three levels—regular expression extraction, CrossRef API query, and high-confidence inference from a large language model—the automation and success rate of digital object identifier retrieval are improved, avoiding manual searching by users and preventing spurious writes.

[0035] By performing frequency-accumulating database operations on keywords, the efficiency of automatically constructing domain terminology knowledge graphs is improved. High-frequency keywords are accumulated as the number of papers increases, supporting subsequent literature statistical analysis.

[0036] By scheduling the concurrent debate between advocate and skeptic agents and the sequential speaking of heuristic agents, the parallel efficiency and depth of viewpoint exchange in multi-agent debates are improved, saving approximately 50% of the waiting time.

[0037] By setting prompts for each agent that include original text citations, severity labels, and cross-domain anchor requirements, the traceability and inspiration of the debate output are improved, generating evaluation content with a complete chain of evidence.

[0038] By using a referee AI agent to stitch together the first 3,000 words of the full paper with all the debate records and outputting them in JSON Schema format, the standardization and parsability of subsequent critical evaluation reports are improved.

[0039] By performing case-insensitive LIKE join queries using the structured fields of papers already stored in the database, the dimensionality and flexibility of cross-document retrieval are improved, and retrieval capabilities are expanded.

[0040] By controlling PDF access through a token verification mechanism based on user identity, the copyright compliance and data security of the paper PDF are improved, allowing only the uploader to view it and preventing unauthorized access.

[0041] By constructing a list of input messages for a large language model from structured fields and the first 3,000 words of the critical evaluation report, along with historical messages and the current question, and using streaming return, the relevance and real-time performance of the paper-specific AI dialogue are improved, enabling multi-turn question answering with memory. Attached Figure Description

[0042] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0043] Figure 1 This is a flowchart illustrating a method for critically interpreting academic papers based on multi-agent debate, provided as an embodiment of this application. Detailed Implementation

[0044] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0045] like Figure 1 As shown in the figure, this application provides a method for critical interpretation of academic papers based on multi-agent debate, including the following steps: Step 1: Receive the PDF file of the paper uploaded by the user, extract the full text and identify the chapter structure to obtain the full text object of the paper, including the title, abstract and content of each chapter.

[0046] Step 2: Input the full text of the paper into the large language model, extract structured information according to several preset structured fields, generate paper identifiers and store them in the database.

[0047] Step 3: Using the paper identifier as the operation object, attempt to find the paper's digital object identifier and store it in the database in sequence according to three levels: regular expression extraction, CrossRef API query, and high-confidence inference of large language model.

[0048] Step four: Extract keywords from the full text of the paper, accumulate the frequencies of existing keywords in the database, and establish the association between the paper and the keywords.

[0049] Step 5: Initiate multi-agent debate by calling on the advocate agent, the questioner agent, and the heuristic agent to conduct interactive debates according to the configured number of debate rounds, and the judge agent generates objects with strong critical types.

[0050] Step six: Convert the critically typed object into a structured critical evaluation report, persist it in the database, and associate it with the paper identifier.

[0051] Step 7: The browser client polls the task status, obtains a structured critical evaluation report, and displays it on the front end.

[0052] Specifically, in this embodiment, in step one, the user uploads a PDF file via a browser client (only text-based PDFs are supported; scanned image PDFs are not supported). After receiving the file, the server performs the following processing: (1) Parse the user identity identifier (user_id) from the authentication field (Authorization Header), permanently save the file to uploads.pdf, and return the information that needs to be extracted to the browser.

[0053] (2) Use the PyMuPDF library to extract the text layer content from the PDF page by page and merge it into a complete full-text string (full_text); (3) Perform regular expression matching on the entire text according to the academic paper chapter title rules (such as "Introduction", "Methods", "Results", "Conclusion" etc.), identify chapter boundaries, and generate a structured object list (PaperSection list) containing chapter names and chapter text. (4) Extract the paper title and abstract separately; (5) Encapsulate the above results into a PaperContent data object, i.e., the full text of the paper, for use in subsequent steps.

[0054] For example, in step two, the structured fields include: publication year, author list, journal name, material system, calculation method parameters, physical phenomena, core conclusions, key quantitative results, research limitations, future research directions, and data sources.

[0055] Specifically, in this embodiment, in step two, the full text of the paper obtained in step one (without truncation, passed in completely) is constructed as prompt words and sent to the Large Language Model (LLM). The model is required to output 11 structured fields in JSON format, as follows: publication year, author list, journal name, material system, calculation method parameters (method_details, including 10 subfields: functional type, U value, calculation software, cutoff energy, etc.), physical phenomenon, core conclusion, key quantitative results, research limitations, future research direction, and data source.

[0056] After extraction, calculate the MD5 hash value of the PDF file and compare it with the existing record in the database: if the MD5 is the same, it is determined to be a duplicate upload, the existing record is reused and the existing paper identifier (paper_id) is returned, and it is not written again; if the MD5 is different, the above 11 fields, the file path, and the uploader user ID (uploader_user_id) are written to the papers table of the database (adding a new row), and the newly generated paper_id is returned.

[0057] For example, in step three, the regular expression extraction includes: using regular expressions to match numeric object identifiers that conform to a certain format in the full-text paper object.

[0058] The CrossRef API query includes: if the regular expression extraction does not find the numeric object identifier, calling the CrossRef API query with the paper title and publication year as parameters to extract the numeric object identifier based on the similarity score.

[0059] The high-confidence inference of the large language model includes: if neither the regular expression extraction nor the CrossRef API query finds a numeric object identifier, the paper title and author are sent to the large language model for high-confidence inference to extract the numeric object identifier.

[0060] If the regular expression extraction, the CrossRef API query, and the high-confidence inference of the large language model all fail to find the numeric object identifier, the numeric object identifier field will be left blank, which will not affect subsequent processes.

[0061] Specifically, in this embodiment, in step three, the first level (regular expression extraction, synchronous execution): in the full text of the paper extracted in step one, regular expressions are used to match numeric object identifier (DOI) strings that conform to the format 10.\d{4,} / \S+; if the match is successful, the source is recorded as "regex", and the writing to the database step is initiated.

[0062] Level 2 (CrossRef API query, asynchronous execution): If not found in Level 1, use the paper title and publication year as parameters to call the CrossRef public API (https: / / api.crossref.org / works) via HTTPS request; sort the returned results by similarity score, extract the DOI from the result with the highest score; record the source as "crossref", and proceed to the database writing step.

[0063] Level 3 (High-confidence inference for large language models, executed asynchronously): If the first two levels fail to find the paper, send the paper title and authors to the LLM and ask if they are aware of the paper's DOI; require the LLM to provide a confidence field in the JSON output, and only accept the result if the confidence value is "high"; record the source as "llm" and proceed to the database writing step.

[0064] If no DOI is found at any of the three levels, leave the DOI field blank; this will not affect subsequent processes. Once found, write the DOI and the original URL into the corresponding fields of the papers table.

[0065] For example, in step four, the following database operation is performed on each keyword: if the keyword does not exist in the database, a new row is inserted; if the keyword already exists in the database, the corresponding frequency is incremented by one.

[0066] Specifically, in this embodiment, in step four, the first 8000 characters of the full text of the paper are taken, prompt words are constructed and sent to LLM, which is required to extract up to 25 keywords, each of which is labeled with its category: material (material system), method (calculation method), property (physical property), phenomenon (physical phenomenon), general (general).

[0067] For each keyword, perform the following database operations: If the keyword text does not exist in the keywords table, insert a new row (frequency=1); if it already exists, increment the frequency field value by 1 (using the SQL INSERT ORIGNORE...ON CONFLICT DO UPDATE SET frequency=frequency+1 statement to achieve an atomic operation). Simultaneously, write the mapping between the current paper_id and each keyword ID into the paper_keywords related table.

[0068] For example, in step five, in each round of debate, the advocate agent and the skeptic agent execute concurrently without waiting for each other, and simultaneously send independent requests to the large language model and return their statements.

[0069] Then the heuristic agent is executed sequentially. The heuristic agent reads all the statements made by the advocate and the skeptic agent in this round and all previous rounds before making its own statement.

[0070] For example, in step five, the advocate agent is prompted to list the core contributions of the paper, and each argument must be accompanied by evidence of original text citations. In subsequent rounds, the advocate agent is also required to respond to each criticism from the questioner agent in the previous round.

[0071] The prompts for the questioner agent require it to list methodological defects, statistical problems, and logical loopholes. Each problem must be marked with its severity and accompanied by original evidence. In subsequent rounds, the questioner agent is also required to provide in-depth rebuttals to the defenses of the supporter agent.

[0072] The prompts for the heuristic agent require the heuristic agent to speak from the perspective of cross-domain value and future research directions. Each heuristic must be accompanied by a cross-domain anchor and marked with feasibility.

[0073] For example, in step five, after all rounds of interactive debate have ended, the referee agent concatenates the first 3000 words of the full text of the paper and all the debate records in the message history list as input, sends them to the large language model, outputs the evaluation result JSON string according to the predefined JSONSchema format, and generates a critically typed object after parsing.

[0074] Specifically, in this embodiment, step five consists of the following sub-steps: (5.1) Initialize the debate state object (DebateState) to record paper_id, paper title, full text context string of the paper, message history list (initially empty), and current round (initially 0).

[0075] (5.2) Perform the following operations in a loop according to the configured number of debate rounds (default 2 rounds): (5.2.1) Concurrent execution of the `respond()` method of the AdvocateAgent and SkepticAgent: Both simultaneously send independent requests to the LLM (using `asyncio.gather` for concurrent scheduling), without waiting for each other's results. The AdvocateAgent's prompt requires it to list the core contributions of the paper, with each argument accompanied by original citations as evidence, in the format of [citation: chapter · "key sentence"]. In subsequent rounds, it is required to respond to each of the SkepticAgent's criticisms from the previous round. The SkepticAgent's prompt requires it to list methodological defects, statistical problems, and logical flaws, with each problem marked with its severity (major or minor) and accompanied by original evidence. In subsequent rounds, it is required to provide a more in-depth rebuttal to the AdvocateAgent's arguments. The response texts of both are encapsulated as `DebateMessage` objects and appended to the message history list.

[0076] (5.2.2) Execute the respond() method of the InspirerAgent in sequence: The InspirerAgent can read all the statements made by the SupporterAgent and the SkepticAgent in this round and all previous rounds. Based on this, it makes a statement from the perspective of cross-domain value and future research direction. Each inspiration must be accompanied by a cross-domain anchor in the format of [Relationship: Domain·Concept] and marked with feasibility (high / medium / low). The response text is also appended to the message history list.

[0077] (5.3) After all rounds are completed, the JudgeAgent executes the judge() method: it concatenates the first 3000 words of the full paper and all debate records in the message history list as input, sends them to the LLM, and requires it to output the evaluation results in strict accordance with the predefined JSON Schema format, including: paper rating (five levels of enumeration values: S / A / B / C / D), rating reasons, a list of contributions confirmed by the debate (each with a contribution description, supporting evidence, and debate conclusion), a list of issues confirmed by the debate (each with an issue description, evidence, and severity), a list of cross-disciplinary inspirations, a recommended reader group, and a 200-word overall summary.

[0078] (5.4) The JSON string output by the referee agent is parsed to generate a strongly typed CriticalReport object. If JSON parsing fails, a downgrade process is performed: a fallback report object is created with a B-level rating, and the original LLM output is written to the summary field.

[0079] Specifically, in this embodiment, in step six, the CriticalReport object generated in step five is converted into a complete report text (report_md) in Markdown format through the report_builder module, while retaining the JSON string (report_json). Both are written to the debate_sessions table in the database, and the status field of the row is updated to "complete", the paper_id field is updated to the paper_id obtained in step two, and the completed_at field is updated to the current timestamp.

[0080] Specifically, in this embodiment, in step seven, the browser client polls the backend interface GET / api / tasks / {task_id} every 2.5 seconds, receiving a JSON response containing the current status (pending / running / complete / failed) and the latest progress log. When the status is detected as "complete", a GET / api / reports / {task_id} / json request is initiated to obtain structured report data, which is then rendered and displayed on the front end in five tabs: overall rating, advocate agent, skeptic agent, inspirer agent, and complete report.

[0081] For example, a method for critically interpreting academic papers based on multi-agent debate also includes: Step eight, full-text literature search: When the backend receives keywords input by the user, it performs a case-insensitive LIKE join query on the structured fields of the papers already stored in the database.

[0082] Specifically, in this embodiment, in step eight, when the user enters keywords in the search box, the backend performs a case-insensitive LIKE query on the eight fields (title, material system, method parameters, physical phenomena, core conclusions, key quantitative results, author, and journal) in the papers table; if the input contains a separator (semicolon or comma), it is split into multiple keywords and combined with AND logic for querying; the results are returned in reverse order of publication year, up to a maximum of 100 results.

[0083] For example, a method for critically interpreting academic papers based on multi-agent debate further includes: Step nine, controlling access to PDF files: When receiving the uploaded PDF file of the paper in step one, the user's identity is parsed and the file is stored locally on the server. When a user requests to access the PDF file of the paper, the user token carried in the request is verified.

[0084] If the user identifier in the user token matches the user identifier corresponding to the paper PDF file, the paper PDF file is returned; otherwise, a 403 / 401 error is returned.

[0085] Specifically, in this embodiment, in step nine, the PDF file is permanently stored locally on the server in the uploads / directory with a UUID name after step one is completed, and is not automatically deleted. When a user requests access to the PDF, the client sends a request to the GET / api / papers / {paper_id} / pdf interface, carrying Authorization:Bearer{token} in the request header; the server verifies the validity of the token from the sessions table, obtains the corresponding user_id, and compares it with the uploader_user_id field of the paper in the papers table: if they match, the PDF file stream is returned; if they do not match or the token is invalid, a 403 / 401 error is returned. After receiving the file stream, the client displays it embedded in an iframe using Blob URL technology to avoid exposing the authentication token in the URL.

[0086] For example, a method for critically interpreting academic papers based on multi-agent debate also includes: Step 10, a paper-specific AI dialogue: When a user's question is received, the structured fields of the corresponding paper and the first 3,000 words of the critical evaluation report are read from the database, and together with historical messages and the current question, they are used to construct the input message list for the large language model.

[0087] The large language model's streaming interface is called, and each generated token is written to the HTTP response stream in Server-Sent Events format. The front end then displays the response content word by word in real time.

[0088] Specifically, in this embodiment, in step ten, the user sends a JSON request containing the current prompt and a history list to the POST / api / papers / {paper_id} / chat interface; the server reads the structured fields of the paper from the papers table and the first 3000 words of the latest completed report from the debate_sessions table, and constructs an LLM input message list together with the history messages (the 6 most recent messages) and the current prompt; the server calls the LLM streaming interface (stream=True) to write each generated token into the HTTP response stream in Server-Sent Events (SSE) format (data:{content}\n\n); the client reads the streaming response block by block using fetch+ReadableStream.getReader() and updates the page display content in real time.

[0089] In one embodiment, the uploader re-views the critical interpretation report: the user sends a request to the GET / api / papers / {paper_id} / report interface, with the Authorization:'Bearer' header included. <token>'Token; The server parses the user_id from the token and performs permission verification, allowing only the uploader of the paper to access it; if the verification is successful, the server returns the latest record in the debate_sessions table, containing the report_md and report_json fields; if the visitor is not the uploader of the paper, the server returns a 403 permission insufficient error; if there is no critical interpretation report for the paper, the server returns a 404 resource not found error.'

[0090] Implementation Example 1: Critical Interpretation of a Single Paper.

[0091] Composition: The system consists of five parts: browser front-end, FastAPI back-end service, SQLite local database, multi-agent debate engine, and large language model API.

[0092] Implementation steps: Users open the system page (http: / / localhost:8000) in their browser. While logged in, they drag and drop a PDF file (12 pages, approximately 15,000 characters) about DFT calculation of MnO2 electronic structure to the upload area and click the "Start Multi-Agent Analysis" button.

[0093] The system executes steps one through seven of the critical interpretation method for academic papers based on multi-agent debate proposed in this application.

[0094] Full text extraction → LLM extracts 11 fields (year 2023, journal Physical Review B, key results "band gap 2.3eV, magnetic moment 3.1μB / Mn", limitation "not considering finite temperature effect", etc.) → written to database → CrossRef finds DOI → extracts 18 keywords and writes them to the thesaurus → two rounds of debate between proponents and skeptics → judges output rating B, 3 contribution points, and 2 serious issues → generate Markdown report → front-end displays 5 tabs of report.

[0095] Effects: Users receive a multi-faceted evaluation report containing evidence of original citations within approximately 8 minutes, gaining a clear understanding of the rationality of the functional selection in paper HSE06 and the methodological shortcomings of the lack of convergence testing in the selection of U-values, which helps in deciding whether to cite the paper.

[0096] Implementation Example 2: Construction of a shared literature knowledge base for research groups.

[0097] Composition: Browser clients on multiple computers within the same local area network connect to the system server running on a host within the research group's intranet (listening on 0.0.0.0:8000), sharing the same local SQLite database.

[0098] Implementation steps: Three graduate students in the research group registered accounts and uploaded 15 papers in their respective fields. After each upload, the system automatically executed steps one to six of the "A Critical Interpretation Method for Academic Papers Based on Multi-Agent Debate" proposed in this application, writing the structured information of all papers into the shared database. The keyword "DFT+U" was extracted from 4 out of the 15 papers, with the frequency accumulated to 4 times.

[0099] The PDF file of the paper uploaded by member A can only be viewed embedded by member A after logging in.

[0100] When other members search for this paper, only the structured information and DOI link are displayed. All three members can perform cross-document searches on public databases; for example, entering "DFT+U;MnO2" will simultaneously filter all papers involving both the DFT+U method and the MnO2 system.

[0101] Effects: After the 15 papers were uploaded, the research group established a dedicated literature knowledge base containing 15 complete records, about 180 keywords, and covering the research directions of all members. Through the frequency distribution of keywords, the research group intuitively understood that the DFT+U method is the main methodology of the group's research, and MnO2 is the most frequently studied material system, which helps to grasp the distribution of research directions and potential cooperation opportunities.

[0102] Implementation Example 3: AI-powered deep dialogue specific to academic papers.

[0103] Composition: Based on Implementation Example 1, after the analysis is completed, the user interacts with the system's built-in AI dialogue panel dedicated to the paper.

[0104] Implementation steps: After the analysis is completed, the user clicks the quick question "What are the main limitations of this paper?" in the dialog panel below the report. The system combines this question with the paper's structured fields (including the limitation field content "Finite temperature effect not considered") and the first 3000 words of the debate report as LLM input, and returns the answer through the SSE streaming interface, which is displayed word by word in real time in the browser. If the user further asks "How to improve the method for determining the U value?", the system carries the previous dialogue record as context, and the LLM provides a targeted answer based on the previous questions and answers.

[0105] Effects: Users can conduct multiple rounds of in-depth questioning based on the complete information of the paper and the content of the debate report without leaving the system. The AI ​​can answer by citing specific data and methodological details from the paper, helping users to deeply understand the content of the paper and assisting in subsequent research design.

[0106] The embodiments of this application improve the objectivity and efficiency of critical interpretation of academic papers through full-text structured information extraction, multi-agent debate, and persistent storage. They can generate critical evaluation reports and suppress the conformity bias of a single large language model.

[0107] By pre-setting structured fields, the information integrity and retrieval accuracy of the literature database are improved, making cross-paper comparison and refined retrieval possible.

[0108] By sequentially attempting to find and store the digital object identifiers of the paper in the database through three levels—regular expression extraction, CrossRef API query, and high-confidence inference from a large language model—the automation and success rate of digital object identifier retrieval are improved, avoiding manual searching by users and preventing spurious writes.

[0109] By performing frequency-accumulating database operations on keywords, the efficiency of automatically constructing domain terminology knowledge graphs is improved. High-frequency keywords are accumulated as the number of papers increases, supporting subsequent literature statistical analysis.

[0110] By scheduling the concurrent debate between advocate and skeptic agents and the sequential speaking of heuristic agents, the parallel efficiency and depth of viewpoint exchange in multi-agent debates are improved, saving approximately 50% of the waiting time.

[0111] By setting prompts for each agent that include original text citations, severity labels, and cross-domain anchor requirements, the traceability and inspiration of the debate output are improved, generating evaluation content with a complete chain of evidence.

[0112] By using a referee AI agent to stitch together the first 3,000 words of the full paper with all the debate records and outputting them in JSON Schema format, the standardization and parsability of subsequent critical evaluation reports are improved.

[0113] By performing case-insensitive LIKE join queries using the structured fields of papers already stored in the database, the dimensionality and flexibility of cross-document retrieval are improved, and retrieval capabilities are expanded.

[0114] By controlling PDF access through a token verification mechanism based on user identity, the copyright compliance and data security of the paper PDF are improved, allowing only the uploader to view it and preventing unauthorized access.

[0115] By constructing a list of input messages for a large language model from structured fields and the first 3,000 words of the critical evaluation report, along with historical messages and the current question, and using streaming return, the relevance and real-time performance of the paper-specific AI dialogue are improved, enabling multi-turn question answering with memory.

[0116] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0117] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.< / token>

Claims

1. A method for critically interpreting academic papers based on multi-agent debate, characterized in that, Includes the following steps: Step 1: Receive the PDF file of the paper uploaded by the user, extract the full text and identify the chapter structure to obtain the full text object of the paper, which includes the title, abstract and content of each chapter. Step 2: Input the full text of the paper into the large language model, extract structured information according to a number of preset structured fields, generate paper identifiers and store them in the database; Step 3: Using the paper identifier as the operation object, attempt to find the paper's digital object identifier and store it in the database in sequence according to three levels: regular expression extraction, CrossRef API query, and high-confidence inference of large language model. Step 4: Extract keywords from the full text of the paper, accumulate the frequencies of keywords already existing in the database, and establish the association between the paper and the keywords; Step 5: Initiate multi-agent debate by calling on the advocate agent, the questioner agent, and the heuristic agent to conduct interactive debate according to the configured number of debate rounds, and the judge agent generates objects with strong critical types. Step six: Convert the critically typed object into a structured critical evaluation report, persist it in the database, and associate it with the paper identifier; Step 7: The browser client polls the task status, obtains a structured critical evaluation report, and displays it on the front end.

2. The critical interpretation method for academic papers based on multi-agent debate as described in claim 1, characterized in that, In step two, the structured fields include: publication year, author list, journal name, material system, calculation method parameters, physical phenomena, core conclusions, key quantitative results, research limitations, future research directions, and data sources.

3. The critical interpretation method for academic papers based on multi-agent debate as described in claim 1, characterized in that, In step three, the regular expression extraction includes: using regular expressions to match numeric object identifiers that conform to a certain format in the full-text paper object; The CrossRef API query includes: if the regular expression extraction does not find the digital object identifier, calling the CrossRef API query with the paper title and publication year as parameters to extract the digital object identifier based on the similarity score; The high-confidence inference of the large language model includes: if neither the regular expression extraction nor the CrossRef API query finds a numeric object identifier, the paper title and author are sent to the large language model for high-confidence inference to extract the numeric object identifier; If the regular expression extraction, the CrossRef API query, and the high-confidence inference of the large language model all fail to find the numeric object identifier, the numeric object identifier field will be left blank, which will not affect subsequent processes.

4. The critical interpretation method for academic papers based on multi-agent debate as described in claim 1, characterized in that, In step four, perform the following database operations for each keyword: if the keyword does not exist in the database, insert a new row; if the keyword already exists in the database, increment the corresponding frequency by one.

5. The critical interpretation method for academic papers based on multi-agent debate according to claim 1, characterized in that, In step five, in each round of debate, the advocate agent and the skeptic agent execute concurrently without waiting for each other, and simultaneously send independent requests to the large language model and return their statements. Then the heuristic agent is executed sequentially. The heuristic agent reads all the statements made by the advocate and the skeptic agent in this round and all previous rounds before making its own statement.

6. The critical interpretation method for academic papers based on multi-agent debate according to claim 5, characterized in that, In step five, the advocate agent is prompted to list the core contributions of the paper, and each argument must be accompanied by evidence of original text citations. In subsequent rounds, the advocate agent is also required to respond to the criticisms of the skeptic agent in the previous round, one by one. The prompts for the questioner agent require the questioner agent to list methodological defects, statistical problems, and logical loopholes. Each problem must be marked with its severity and accompanied by original evidence. In subsequent rounds, the questioner agent is also required to provide in-depth rebuttals to the defenses of the supporter agent. The prompts for the heuristic agent require the heuristic agent to speak from the perspective of cross-domain value and future research directions. Each heuristic must be accompanied by a cross-domain anchor and marked with feasibility.

7. The critical interpretation method for academic papers based on multi-agent debate according to claim 1, characterized in that, In step five, after all rounds of interactive debate have ended, the referee agent concatenates the first 3000 words of the full paper and all debate records in the message history list as input, sends them to the large language model, and outputs the evaluation result JSON string according to the predefined JSON Schema format. After parsing, a highly critical typed object is generated.

8. The critical interpretation method for academic papers based on multi-agent debate according to claim 1, characterized in that, It also includes: Step eight, full-text literature search: When the backend receives keywords input by the user, it performs a case-insensitive LIKE join query on the structured fields of the papers already stored in the database.

9. The critical interpretation method for academic papers based on multi-agent debate according to claim 1, characterized in that, This also includes: Step Nine, PDF file access control: When receiving the paper PDF file uploaded by the user in step one, the user's identity is parsed and the file is stored locally on the server. When a user requests to access the paper PDF file, the user token carried in the request is verified. If the user identifier in the user token matches the user identifier corresponding to the paper PDF file, the paper PDF file is returned; otherwise, a 403 / 401 error is returned.

10. A method for critical interpretation of academic papers based on multi-agent debate as described in claim 1, characterized in that, It also includes: Step 10, AI dialogue specific to the paper: When a user's question is received, the structured fields of the corresponding paper and the first 3,000 words of the critical evaluation report are read from the database and constructed together with historical messages and the current question to form the input message list of the large language model. The large language model's streaming interface is called, and each generated token is written to the HTTP response stream in Server-Sent Events format. The front end then displays the response content word by word in real time.

Images