A literature review generation method based on a large language model and a question and answer system

Abstract

The application relates to a literature review generation method based on a large language model and a question and answer system, wherein the literature review generation method based on the large language model comprises the following steps: classifying a request input by a user, determining whether the request is a research request according to a classification result, performing interactive detail confirmation on the request to obtain a context, generating a review generation plan based on the context, decomposing the review generation plan into a plurality of independent chapter tasks, generating a chapter draft corresponding to each chapter task in parallel based on the context by using a large language model, performing chapter length adjustment, context logic fusion, chart and literature reference insertion on each chapter draft to obtain a review full-text draft, reviewing the review full-text draft, outputting modification suggestions, modifying the review full-text draft based on the modification suggestions to obtain a target review full text, and improving the generation quality and rigor of the review.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a literature review generation method and question-answering system based on a large language model. Background Technology

[0002] With the explosive growth of scientific research results, manually writing review reports in various fields faces challenges such as being time-consuming and labor-intensive, highly subjective, and having limited coverage.

[0003] Existing patent CN118278365B focuses on the medical field, forming a linear processing pipeline by constructing search queries, screening literature, extracting data (such as effect size and sample size), and summarizing and generating reports. Existing patent CN118278365B extracts the research background, methods, and conclusions of literature through an identification model, and calculates "literature importance" by integrating metadata such as citation count and journal impact factor, and then ranks and synthesizes the results.

[0004] Existing automated solutions mainly rely on text summarization and large language model (LLM) prompting engineering, but still have the following problems: (1) General large language models generate text based on probability, and the corresponding output may have "illusions", that is, generate seemingly reasonable but unverified or inconsistent conclusions. There is a lack of effective review and modification opinions. Relying on manual review for quality assessment is not only resource-intensive, but also lacks scalability. It is difficult to guarantee the logical integrity, credibility and technical insight of the generated review, resulting in low rigor of the generated review. (2) There is a lack of manual interaction confirmation. When the user input is unclear, the large language model is difficult to accurately capture the user's true intention, resulting in a large deviation between the generated review after a long period of reasoning and the user's goal, causing a waste of time and computing costs. (3) Existing large language model generation is usually a one-time process, and it is impossible to effectively control the length of each chapter and the length of the entire review.

[0005] There is currently no effective solution to the problem of low quality and rigor in the generated literature reviews in related technologies. Summary of the Invention

[0006] Therefore, it is necessary to provide a literature review generation method and question-answering system based on a large language model to address the aforementioned technical problems.

[0007] In a first aspect, embodiments of this application provide a method for generating literature reviews based on a large language model, the method comprising:

[0008] Classify user-input requests and determine whether they are research-related requests based on the classification results;

[0009] If so, the request is interactively reviewed to obtain context; and a summary generation plan is generated based on the context.

[0010] The review generation plan is decomposed into multiple independent chapter tasks; based on the context, a large language model is used to generate the chapter drafts corresponding to each chapter task in parallel;

[0011] The initial drafts of each chapter were adjusted in terms of chapter length, logical integration of context, and insertion of figures, tables and references to obtain the initial draft of the full review.

[0012] The initial draft of the full review article is reviewed, and revision suggestions are provided. Based on the revision suggestions, the initial draft of the full review article is revised to obtain the target full review article.

[0013] In one embodiment, adjusting the chapter length of each of the initial draft chapters includes:

[0014] Based on the target length of the full text and the preset weights of the tasks in each chapter, the preset length of each task in each chapter is determined.

[0015] The output length of the current chapter's first draft is adjusted based on the deviation between the output length of the current chapter's first draft and the preset length.

[0016] Determine the total output length and total preset length from the adjusted first draft of the first chapter to the adjusted first draft of the current chapter;

[0017] Based on the deviation between the total preset length and the total output length, the preset length of the initial draft of subsequent chapters is updated.

[0018] In one embodiment, after adjusting the chapter length of each draft chapter, the context logic fusion includes:

[0019] Input the initial drafts of adjacent chapters into the context window of the large language model to generate transitional paragraphs that connect the preceding and following chapters;

[0020] The transition paragraphs are inserted between the initial drafts of adjacent chapters to achieve contextual logical integration.

[0021] In one embodiment, after contextualizing the initial drafts of each chapter, inserting charts includes:

[0022] Scan the full draft of the review article to identify text regions suitable for visualization, which include at least one of data-intensive statements or pattern comparison descriptions;

[0023] Extract visualization data from the text region;

[0024] Based on the semantic context of the text region and the visualization data, a matching chart is generated and inserted into the corresponding position.

[0025] In one embodiment, after context fusion of the initial drafts of each chapter, the insertion of bibliographic citations includes:

[0026] Scan the full draft of the review and identify the statements that need to be cited;

[0027] The statements that need to be cited are converted into vectors, and evidence fragments that match them are searched from a literature database; wherein, the literature database is a collection of literature retrieved based on a large language model during the literature review generation process;

[0028] Based on the evidence fragments, the consistency of the statements that need to be cited is verified. If the verification passes, a citation is inserted in the corresponding position.

[0029] In one embodiment, the review of the initial full-text draft of the review and the output of revision suggestions based on the review results include:

[0030] The BERT model was used to identify the core claims in the initial full-text draft of the review, resulting in a core claim library.

[0031] Based on the aforementioned core claim library, the logical consistency of the initial draft of the full review is examined, and conflict correction suggestions are generated.

[0032] The depth of thought in the initial draft of the full review was evaluated, and suggestions for expanding the depth of thought were generated.

[0033] The core arguments of the initial draft of the full review were reviewed and supplementary comments were generated.

[0034] Based on the aforementioned conflict correction opinions, the aforementioned opinions on expanding the depth of thinking, and the aforementioned opinions on supplementing details, the proposed modifications are output.

[0035] In one embodiment, revising the initial draft of the full review based on the proposed modifications includes:

[0036] The initial drafts of each of the aforementioned chapters were revised based on the suggested modifications.

[0037] The revised drafts of each chapter were re-integrated with contextual logic and references were inserted to obtain the revised full-text draft of the review.

[0038] Based on the aforementioned suggestions, the revised full-text review was revised to obtain the target full-text review.

[0039] During the modification process, the semantic similarity between the modified content and the core claims in the core claim library is calculated. If the similarity meets the threshold, the modified content is updated in the first draft of the full review.

[0040] In one embodiment, classifying the user-input request and determining whether it is a research request based on the classification result includes:

[0041] The user-input requests are initially classified based on predefined rules to obtain preliminary classification results and confidence levels; it is then determined whether the confidence levels meet a preset threshold.

[0042] If so, then determine whether it is a research request based on the preliminary classification results;

[0043] If not, the request is reclassified based on the BERT semantic classification model to obtain the final classification result; based on the final classification result, it is determined whether it is a research request.

[0044] Secondly, embodiments of this application also provide a question-answering system based on a large language model, the system comprising:

[0045] The classification module is used to classify user-input requests and obtain classification results; the classification results include at least query requests, question-and-answer requests, and research requests.

[0046] The routing module is used to route the request to the corresponding unit for processing based on the classification result;

[0047] The query generation unit is used to call an interface to obtain factual answers when a query request is received, and to generate a query response based on the factual answers;

[0048] The knowledge retrieval and generation unit is used to retrieve basic explanations from a structured knowledge base when a question-and-answer request is received, and to generate an explanatory answer based on the basic explanations.

[0049] The literature review generation unit, upon receiving a research request, interactively confirms the details of the request to obtain context; generates a review generation plan based on the context; decomposes the review generation plan into multiple independent chapter tasks; generates initial drafts for each chapter task in parallel using a large language model based on the context; adjusts the chapter length, integrates contextual logic, and inserts figures, tables, and citations for each initial draft to obtain an initial full-text review; reviews the initial full-text review and outputs modification suggestions; and modifies the initial full-text review based on the modification suggestions to obtain the target full-text review.

[0050] Thirdly, embodiments of this application also provide a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method described in the first aspect above.

[0051] The aforementioned literature review generation method and question-answering system based on a large language model classifies user-input requests and determines whether they are research-related requests based on the classification results. If so, the request is interactively reviewed to obtain context, and a literature review generation plan is generated based on the context. The literature review generation plan is decomposed into multiple independent chapter tasks. Based on the context, a large language model is used to generate initial drafts for each chapter task in parallel. The initial drafts of each chapter are adjusted in length, contextual logic is integrated, and figures, tables, and citations are inserted to obtain an initial full-text review. The initial full-text review is reviewed, and modification suggestions are output. Based on the modification suggestions, the initial full-text review is modified to obtain the target full-text review. This embodiment of the application, through reflective review and iterative revision, can continuously deepen the content of the review, repair logical defects, realize the self-examination and revision of human authors, and improve the quality and rigor of the generated review.

[0052] Details of one or more embodiments of this application are set forth in the following drawings and description to make other features, objects and advantages of this application more readily apparent. Attached Figure Description

[0053] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0054] Figure 1 This is a hardware structure block diagram of a terminal device for a literature review generation method based on a large language model in one embodiment.

[0055] Figure 2 This is a flowchart illustrating a literature review generation method based on a large language model in one embodiment;

[0056] Figure 3 This is a flowchart illustrating the process of classifying user input requests in one embodiment;

[0057] Figure 4 This is a flowchart illustrating the full-text fusion process in one embodiment;

[0058] Figure 5 This is a flowchart illustrating a literature review generation method based on a large language model in another embodiment;

[0059] Figure 6This is a flowchart illustrating the chapter quality assessment and iterative modification process in one embodiment;

[0060] Figure 7 This is a schematic diagram of the structure of a question-answering system based on a large language model in one embodiment. Detailed Implementation

[0061] To make the objectives, technical solutions, and advantages of this application clearer, the application is described and illustrated below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application.

[0062] The method embodiments provided in this example can be executed on a terminal, computer, or similar computing device. For example, it can run on a terminal. Figure 1 This is a hardware structure block diagram of the terminal for the literature review generation method based on a large language model in this embodiment. For example... Figure 1 As shown, a terminal may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 and a memory 104 for storing data are also included. The processor 102 may be, but is not limited to, a microprocessor (MCU) or a programmable logic device (FPGA). The terminal may also include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that… Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the terminal described above. For example, the terminal may also include components that are larger than... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown are illustrated.

[0063] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the literature review generation method based on a large language model in this embodiment. The processor 102 executes various functional applications and data processing by running the computer programs stored in the memory 104, thereby implementing the above-described method. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0064] The transmission device 106 is used to receive or send data via a network. This network includes a wireless network provided by the terminal's communication provider. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 can be a Radio Frequency (RF) module for wireless communication with the Internet.

[0065] This application provides a method for generating literature reviews based on a large language model, which can be applied to... Figure 1 Taking the terminal in the example of this, for example... Figure 2 As shown, the method includes the following steps:

[0066] Step S201: Classify the user-inputted request and determine whether it is a research request based on the classification result.

[0067] The user's original question is the request. After receiving the user's request, the processing flow is as follows: Figure 3 As shown, user requests are first categorized using a classifier, and category labels are generated based on the categorization results. User requests typically include query requests, basic question-and-answer requests, and specialized research requests. This application configures corresponding processing strategies for different user request types. For simple query questions, external APIs are directly called to quickly retrieve and format factual answers, with a target response time of less than 1 second. For basic question-and-answer questions, relevant information is retrieved from a structured knowledge base, and a summarization algorithm is used to integrate it into a basic explanation or overview, with possible simple examples. For corresponding specialized research questions, an advanced research pipeline is activated for in-depth analysis, including subsequent detail confirmation, plan generation, and content production processes.

[0068] Step S202: If yes, then the request is interactively confirmed to obtain the context; and a summary generation plan is generated based on the context.

[0069] If the request is research-based, the process begins with interactive detail confirmation to precisely define the task scope and generate a preliminary review project plan. This includes the following sub-steps:

[0070] Step 1: Perform preliminary decomposition of user requests, identifying key sub-domains, required data types, and potential ambiguities.

[0071] Step 2: Based on the preliminary decomposition results, proactively generate and output clarifying queries to users, such as inquiring about specific research sub-directions, timeframes, key parameters, or comparison dimensions. This human feedback step helps eliminate long-tail noise and reduces computational redundancy in subsequent generation from the outset.

[0072] Step 3: Receive user feedback.

[0073] Step 4: Iteratively update the internal information retrieval instructions and content integration framework based on user feedback.

[0074] Step 5: Based on the confirmed clear context, generate a preliminary review generation plan, including suggested chapter outlines, key areas of coverage, and proposed literature sources, and then provide it to the user for final confirmation, thus laying the foundation for high-quality generation.

[0075] This application forms a closed loop through human-computer interaction and large language model planning, and solves the problems of demand mismatch and result generalization in professional scenarios through an interactive detail confirmation mechanism, ensuring that the generated overview closely matches the user's real and specific intentions.

[0076] In some embodiments, as the number of user research requests increases and a user profile is established, the user confirmation process can be designed as an option or a default configuration. The research fields that the user is interested in, as well as the recent research goals constructed from historical information, have the ability to replace user confirmation and are used to increase the response speed of requests.

[0077] Step S203: Decompose the review generation plan into multiple independent chapter tasks; based on the context, use a large language model to generate the chapter drafts corresponding to each chapter task in parallel.

[0078] After the user confirms the review generation plan, the content of each chapter is generated in parallel.

[0079] Step 1: Break down the review generation plan into relatively independent chapter tasks, such as the generation of chapters like introduction, methodology, current situation analysis, and challenges outlook.

[0080] Step 2: Utilize a distributed task scheduling mechanism to assign the generation tasks of different chapters to multiple processing instances or parallel threads for synchronous execution. Each instance, based on its chapter topic and combined with a meticulously confirmed global context, invokes a large language model for in-depth content writing and collects relevant references from academic search engines, website content searches, and user literature databases.

[0081] Step S204: Adjust the chapter length, integrate the contextual logic, and insert figures, tables, and references for each chapter's initial draft to obtain the full review article's initial draft.

[0082] After generating initial drafts for each chapter in parallel, full-text fusion and enhancement are initiated to obtain the initial draft of the full review article. For example... Figure 4 As shown, the full text integration and enhancement specifically includes: adjusting the chapter length, inserting transition paragraphs between adjacent chapters to integrate the contextual logic, inserting figures and tables, inserting reference citations, and finally, optimizing the format of the entire text in accordance with the target academic standards, including standardizing terminology, sorting the reference list, adjusting figure and table numbering, adjusting the heading level and font spacing, etc.

[0083] This application's embodiments utilize parallel generation combined with fusion technology to generate automatic transition paragraphs and multimodal enhancements in chart generation, significantly improving the readability, visualization, and logical coherence of the review, thus ensuring the quality of the review's content.

[0084] Step S205: Review the initial draft of the full review and output modification suggestions; modify the initial draft of the full review based on the modification suggestions to obtain the target full review.

[0085] Specifically, the initial draft of the full review is reviewed from multiple dimensions. Based on the review results, revision suggestions are proposed, specifically targeting issues in particular chapters and the entire text. These suggestions may include correcting inconsistencies in certain chapters, expanding the depth of chapters, adding details to chapters, and adjusting the overall structure. The revision suggestions are then fed back to the large language model, which modifies the initial drafts of each chapter and the entire text, iteratively optimizing the initial draft of the full review until it meets the preset quality threshold, resulting in the target full review.

[0086] Specifically, Figure 5 This is another flowchart illustrating the literature review generation method based on a large language model provided in this application. First, the user's question is understood and clarified. Then, a report generation plan is output for user confirmation. After user confirmation, chapter drafts are generated. These chapter drafts are then fully integrated and referenced to obtain a full-text review draft. The full-text review draft is then reviewed and scored. Modifications are made based on the review results, including chapter content modifications and full-text content modifications based on reviewer comments. When the review results meet a preset quality standard, such as a score meeting a threshold, the target full-text review is output.

[0087] The embodiments of this application, through reflective review and iterative revision methods, can continuously deepen the content of the review, fix logical defects, realize self-examination and modification by human authors, and improve the quality of the generated review.

[0088] To provide reviews that are both in-depth and readable, it is necessary to control their length. This application abstracts a chapter length scheduling strategy for reviews from a Google Congestion Control Algorithm for Real-Time Communication. Using the target length of the review as the object, the length of each chapter is adjusted sequentially, and the content fusion process is iteratively controlled to achieve adaptive adjustment of chapter lengths, ensuring effective control of the total review length. In one embodiment, adjusting the chapter length of each draft includes the following steps:

[0089] Step S301: Determine the preset length of each chapter task based on the target length of the full text and the preset weight of each chapter task.

[0090] The target length of the full text (total word count T) can be set in advance or by referring to the total word count of all chapter drafts.

[0091] The preset weights Wi∈(0,1) for chapter tasks can be set based on the top journal review template based on topic relevance, or user-defined templates can be set, Sum(Wi) = 1.

[0092] The preset length of each chapter task is Bi = Wi × T, and the list of preset lengths for each chapter task is: [B1, B2, ..., Bn], where n is the total number of chapters.

[0093] Step S302: Adjust the output length of the current chapter's first draft based on the deviation between the output length of the current chapter's first draft and the preset length.

[0094] Adjustments are made chapter-by-chapter. The output length Ai of the current chapter's initial draft is the length of the parallel-generated chapter's initial draft. The output length Ai of the current chapter's initial draft is adjusted based on the deviation between the current chapter's initial draft's output length and the preset length. For example: if the output length Ai is more than 10% longer than the preset length, the output length Ai of the current chapter's initial draft is adaptively compressed. If the output length Ai is more than 10% shorter than the preset length, the output length Ai of the current chapter's initial draft is adaptively expanded. If the difference between the output length Ai and the preset length is within 10%, no adjustment can be made.

[0095] In the i-th chapter, the output length of the current chapter is controlled according to the following formula based on the deviation between the output length of the initial draft of the current chapter and the preset length.

[0096]

[0097] in, The range can be set to [1.1, 1.15], indicating the current degree of expansion required; The range can be set to [0.85, 0.9], indicating the current degree of compression required.

[0098] Step S303: Determine the total output length and total preset length from the adjusted first chapter draft to the adjusted current chapter draft.

[0099] The first chapter refers to the first page of the first chapter. After adjusting the first page of the first chapter's initial draft, the total output length from the adjusted first page of the first chapter's initial draft to the adjusted current page of the first chapter's initial draft is the output length of the adjusted first page of the first chapter's initial draft. The preset total length from the adjusted first page of the first chapter's initial draft to the adjusted current page of the first chapter's initial draft is A1. 。

[0100] Step S304: Based on the deviation between the total preset length and the total output length, update the preset length of the initial draft of subsequent chapters.

[0101] After adjusting the first draft of the first chapter, the preset lengths of the subsequent chapter drafts are updated based on the deviation between the total preset length and the total output length. At this point, the preset lengths of the subsequent chapter drafts have been updated to A2, …, An. The update strategy can be as follows: if the total output length is more than 10% longer than the total preset length, the output length of the subsequent chapter drafts is adaptively compressed. If the total output length is more than 10% shorter than the total preset length, the output length of the subsequent chapter drafts is adaptively expanded. If the difference between the total output length and the total preset length is within 10%, no adjustment can be made.

[0102] For example, if the total output length of the current chapter is more than 10% longer than the total preset length, then the output length of the initial drafts of subsequent chapters will be adaptively compressed, including:

[0103] Calculate the number of characters T1 that the total output length of the current chapter exceeds the total preset length.

[0104] Calculate the proportion of the preset length of each subsequent chapter to the total preset length of all subsequent chapters.

[0105] Multiply the word count T1 by the ratio of the preset length of each subsequent chapter to the total preset length of all subsequent chapters to obtain the word count that needs to be compressed for each subsequent chapter.

[0106] If the total output length is more than 10% shorter than the total preset length, the method for adaptively expanding the output length of subsequent chapter drafts is the same as above:

[0107] Calculate the number of characters T2 that the total output length of the current chapter is less than the total preset length.

[0108] Calculate the proportion of the preset length of each subsequent chapter to the total preset length of all subsequent chapters.

[0109] Multiply the word count T2 by the ratio of the preset length of each subsequent chapter to the total preset length of all subsequent chapters to obtain the word count that needs to be expanded for each subsequent chapter.

[0110] Then, adjust the length of the first draft of the second chapter according to the strategies in steps S302 to S304 above, as follows:

[0111] Step 1: Adjust the output length of the current chapter's first draft based on the deviation between the output length of the current chapter's first draft and the preset length.

[0112] That is, the output length of the current chapter's first draft is adjusted based on the deviation between the output length B2 of the second chapter and the preset length of the second chapter (at this time, the preset length of the second chapter has been updated).

[0113] Step 2: Determine the total output length and total preset length from the adjusted first chapter draft to the adjusted current chapter draft.

[0114] The total output length from the first draft of the adjusted first chapter to the first draft of the adjusted current chapter: that is, the total output length from the first adjusted chapter to the second adjusted chapter.

[0115] The total preset length from the first draft of the adjusted first chapter to the current draft of the adjusted first chapter: that is, the total preset length from the preset length of the first chapter to the second chapter (here, the preset length of the second chapter has been updated).

[0116] Step 3: Based on the deviation between the total preset length and the total output length, update the preset length of the initial draft of subsequent chapters.

[0117] The preset length of the initial draft of subsequent chapters will be updated, that is, A3, …, An will be updated.

[0118] Then, adjust the length of the first draft of the third chapter according to the strategy of steps S302-S304 above, until the output length of all chapters is under control.

[0119] Existing methods based on large language models typically involve one-off generation, making it impossible to predict the final length and leading to performance degradation in the later stages of long reviews. This application's implementation overcomes the technical problem of "stateless feedback control in text generation" by introducing a chapter length scheduling strategy to address deviations between the initial draft length and the target length. Through iterative control of coefficients α and β, the cumulative length deviation is corrected in real-time during chapter-by-chapter length control, effectively solving common technical problems in long text generation such as loose beginnings and tight endings or severe word count overflow. This transforms discrete chapter generation into a closed-loop dynamic gain adjustment process.

[0120] In one embodiment, after adjusting the chapter length of each of the initial drafts, the context logic fusion includes the following: inputting the initial drafts of adjacent chapters into the context window of the large language model to generate a transitional paragraph; inserting the transitional paragraph between the initial drafts of adjacent chapters to perform context logic fusion.

[0121] In this embodiment, the content of adjacent chapters is placed in the context window of the large language model. The large language model is driven to act as an "academic editor" through customized "context-aware fusion prompt words". It automatically generates transitional paragraphs that connect the preceding and following chapters, seamlessly inserts them between chapters, eliminates the sense of splicing, ensures logical fluency, and achieves contextual logic fusion.

[0122] In one embodiment, after context fusion of the initial drafts of each chapter, the chart insertion includes the following: scanning the full draft of the review, identifying text regions suitable for visualization, wherein the text regions suitable for visualization include at least one of data-intensive statements or pattern comparison descriptions; extracting visualization data from the text regions; generating matching charts based on the semantic context of the text regions and the visualization data, and inserting them into the corresponding positions.

[0123] In this embodiment, text scanning identifies pattern comparison descriptions suitable for visualization (such as performance comparisons, process sequences) or data-intensive statements / paragraphs. LLM (Limited Language Management) is used to extract information, structurally extracting the relevant descriptions into data (e.g., JSON format: { "entity": ["Model A", "Model B"], "metric": "Accuracy", "values":[85.5, 92.3]}). Then, based on the semantic context, matching chart types (bar charts, flowcharts, etc.) are automatically recommended and generated. Based on the JSON content, drawing code is dynamically generated, running in an isolated environment to produce high-quality charts (vector graphics or high-resolution graphs, etc.). The generated charts are automatically inserted after the most logically relevant paragraphs, and chart captions are automatically generated based on the chart data and surrounding text, thus achieving chart generation and adaptation.

[0124] In one embodiment, after context fusion of the initial drafts of each chapter, the insertion of literature citations includes the following: scanning the full-text initial draft of the review and identifying the statements that need to be cited; converting the statements that need to be cited into vectors and searching for matching evidence fragments from a literature database; wherein, the literature database is a collection of literature retrieved based on a large language model during the literature review generation process; and performing consistency verification on the statements that need to be cited based on the evidence fragments, and if the verification passes, inserting a literature citation at the corresponding position.

[0125] Specifically, the statements requiring citation include data, definitions, key facts, and concluding statements. To enhance academic rigor, an "extraction-retrieval-verification" strategy is adopted for these statements. First, the statements requiring citation are vectorized, and a vector semantic search is performed against the literature database retrieved during the review generation process to obtain the most matching evidence fragment. Then, a fact-verification prompt is used to determine the support level. Once verification is successful, the corresponding citation is injected. For example, the following prompt is constructed: Given the original text fragment $D$ and the statement to be verified $S$, please determine whether $D$ sufficiently supports $S$? Is the logic consistent? Citation insertion is only performed when the verification result is True; if it is False, it is marked and a manual review is prompted.

[0126] This application's embodiments, through cross-referencing and a chain of fact verification, combined with the maintenance of claim consistency, curb the inherent illusion of the existence of large language models, overcome the probabilistic uncertainty of the content generated by large language models, construct a reliable network of citation evidence, and enhance the academic credibility and rigor of the review.

[0127] In one embodiment, reviewing the initial draft of the full review and outputting revision suggestions based on the review results includes the following steps:

[0128] Step S401: Use the BERT model to identify the core claims in the initial draft of the full review article to obtain a core claim library.

[0129] Claim identification in this application is based on training and fine-tuning of the BERT model. The training data is obtained by initially labeling the documents using a large language model and then manually checking and correcting the labeling. Claim identification is a common classification task with low complexity and a measured accuracy of over 95%.

[0130] The BERT model was used to identify the core claims in the initial draft of the full review, and the core claims were automatically identified and extracted to form a core claim library.

[0131] Step S402: Based on the core claim library, the logical consistency of the first draft of the full review is checked, and contradictory correction opinions are generated.

[0132] First, the initial draft of the review article was checked for logical coherence and contradictions. Sentences within and across chapters were scanned for logical contradictions, such as conflicting claims, and these contradictions were marked. One type of contradiction detection occurs when the core claim in the preceding text states "research in this direction is scarce," while the core claim in the following text states "there has been extensive research," in which case the contradiction is marked.

[0133] Step S403: Evaluate the depth of thought in the initial draft of the full review and generate suggestions for expanding the depth of thought.

[0134] Second, conduct a depth and insight assessment of the initial draft of the full review. Evaluate the analytical depth of the generated content and identify potential problems such as "merely describing the phenomenon without analyzing the reasons," "lacking discussion of limitations," and "failing to introduce comparative perspectives."

[0135] Step S404: Review the core claims of the first draft of the full review and generate supplementary comments on details.

[0136] Third, a follow-up questioning mechanism will be added to the initial draft of the review article. For key claims or conclusions presented in the article, a reviewer will be asked follow-up questions (implemented through the prompting engineering of the large language model), requesting more details or implementation mechanisms. One scenario for this follow-up questioning mechanism is: if the generated content in the article is "The application of large model technology in earth science research faces challenges in deployment and application," the follow-up question would be: What are the challenges in deployment and application? Are they theoretical bottlenecks or engineering implementation challenges?

[0137] Step S405: Based on the contradictory correction opinions, the opinions on expanding the depth of thinking, and the opinions on supplementing details, output the modification opinions.

[0138] By integrating review results from multiple dimensions, detailed revision suggestions and scores are generated, with the suggestions clearly targeting specific chapters or overall text issues. These suggestions (such as correcting local contradictions, expanding depth, adding details, and adjusting structure) are then fed back to the large language model for iterative optimization until the preset quality threshold is met.

[0139] Figure 6 This is a schematic diagram illustrating the process of chapter quality assessment and iterative modification in an embodiment of this application. For example... Figure 6 As shown, the initial drafts of each chapter are first input into the reflection and evaluation system for evaluation and defect identification. Based on the defects, the system conducts autonomous iterative research and generates precise questions. Then, the chapter content is revised and enhanced. The revised and enhanced chapter content is then checked to see if it meets the preset quality. If not, it is re-entered into the reflection and evaluation system for evaluation. If it does, the initial draft of the chapter is finalized.

[0140] In some embodiments, the review results of the report can adopt the peer review template of a top journal, and combine the aforementioned revision comments to produce a structured output, guiding the large language model to generate high-quality content in the next round. In one example embodiment, a structured output of review and revision comments is as follows:

[0141] 1. Overall Evaluation

[0142] Overall score: 55 points (out of 100).

[0143] Level: Requires major revisions.

[0144] Overall assessment: The content has a basic factual basis, but there are serious flaws in logical consistency, the analysis is superficial, and the conclusions are hasty and lack support.

[0145] 2. Summary of Key Issues

[0146] Logical contradiction: The optimistic conclusion at the end of the chapter that "it will be widely used and replace doctors within three years" directly conflicts with the fundamental challenges such as the "black box problem" and "data bias" that were discussed in the previous text, and the chain of reasoning is broken.

[0147] Insufficient depth:

[0148] (1) Explainability: Only the concept of "black box" is mentioned, without analyzing its technical roots (such as model complexity) and clinical impact (such as accountability).

[0149] (2) Fairness: The discussion on “data bias” remains at the level of describing the phenomenon, without going into the socio-technical causes and potential harms.

[0150] (3) Single perspective: lack of comparative analysis with related technologies (such as traditional diagnostic AI) or mainstream application paradigms (such as human-machine collaboration).

[0151] The evidence does not match the claim: The cited high-accuracy research cases (such as 95% in dermatology diagnosis) are improperly extended to the general conclusion that "the accuracy rate is generally higher than that of traditional systems", and the evidence does not support the claim.

[0152] 3. Specific modification suggestions, listed in order of priority.

[0153] Correction logic and conclusions (high priority):

[0154] (1) Rewrite the conclusion section to align it with the challenge analysis, shifting the focus of the discussion from “substitution” to “assistance” and “empowerment”.

[0155] (2) Clearly distinguish between “high performance in specific scenarios” and “general applicability” to avoid generalization.

[0156] Deepen key issues (high priority): Each of the challenges of "explainability" and "fairness" has a separate sub-chapter, which analyzes their technical mechanisms, clinical / ethical impacts and mitigation pathways in depth.

[0157] Enrich the analytical framework (medium priority):

[0158] (1) Introduce a comparative perspective to clarify the differentiated advantages and limitations of large language models compared with other diagnostic technologies.

[0159] (2) Adjust the chapter structure. It is recommended to reorganize the content according to the logic of "advantages-limitations (in-depth analysis)-feasible path".

[0160] 4. Iterative optimization instructions

[0161] Core instruction: Adjust the tone of the conclusion to ensure it is consistent with the logical consistency of the entire argument.

[0162] Extended instructions: Expand upon the identified superficial points of discussion in greater depth, requiring supplementary analysis of the technical root causes and discussion of their impact.

[0163] Structural adjustment instructions: Optimize chapter narrative logic and add comparative perspectives.

[0164] In one embodiment, modifying the initial draft of the full review based on the modification suggestions includes: modifying the initial drafts of each chapter based on the modification suggestions; re-integrating the contextual logic and inserting literature citations in the modified initial drafts of each chapter to obtain a modified initial draft of the full review; and modifying the modified initial draft of the full review based on the modification suggestions to obtain the target full review. During the modification process, the semantic similarity between the modified content and the core claims in the core claim library is calculated. If the similarity meets a threshold, the modified content is updated in the initial draft of the full review.

[0165] First, the initial drafts of each chapter were revised based on the feedback received. During the revision process, semantic consistency analysis was performed in real time. Newly added content was compared with the core argument database, semantic vectors of both were extracted, and the cosine distance between the semantic vectors was calculated. Similarity was determined based on the cosine distance. If the similarity met a threshold, the newly added content was used, and the revised content was updated in the initial draft of the full review. If obvious contradictions were detected, automatic correction suggestions (using the core argument database as recommended candidates) or strong prompts were provided; if it was a reasonable expansion or supplement, it was silently approved; if inconsistent terminology was used, standardization suggestions were provided, effectively maintaining the thematic focus and logical rigor of the entire text.

[0166] Then, the revised draft of the chapters was re-integrated with contextual logic and references were inserted to obtain the updated full-text review draft.

[0167] Next, the updated first draft of the review article was revised based on the feedback received. Similarly, during the revision process, semantic consistency analysis was performed in real time. The newly added content was compared with the claim database, semantic vectors of both were extracted, and the cosine distance between the semantic vectors was calculated. Similarity was obtained based on the cosine distance. If the similarity met a threshold, the newly added content was used to update the revised first draft of the review article. If obvious contradictions were detected, automatic correction suggestions (using the core claim database as recommended candidates) or strong prompts were provided; if it was a reasonable expansion or supplement, it was silently approved; if the terminology was inconsistent, standardization suggestions were provided, effectively maintaining the article's focus and logical rigor.

[0168] Next, the revised full-text draft of the review is reviewed and scored. When the score meets the preset threshold, the current full-text review is output. If the score does not meet the preset threshold, iterative revision continues.

[0169] In one embodiment, classifying the user-inputted request and determining whether it is a research request based on the classification result includes: performing an initial classification of the user-inputted request based on predefined rules to obtain a preliminary classification result and a confidence level; determining whether the confidence level meets a preset threshold; if yes, determining whether it is a research request based on the preliminary classification result; if no, reclassifying the request based on the BERT semantic classification model to obtain a final classification result; and determining whether it is a research request based on the final classification result.

[0170] Existing technologies typically employ single vector similarity matching for task distribution, which suffers from low efficiency and accuracy when handling complex instructions from specialized research requests in fields such as geosciences. This application utilizes a cascaded architecture combining rule engine-based initial screening with deep semantic recognition. First, a rapid initial screening is performed based on predefined rules in the rule engine to obtain classification results and corresponding confidence levels. These predefined rules include keyword hit rate, sentence length limits, and sentence structure complexity. For scenarios with insufficient confidence, a BERT-based semantic classification model (or a large model hinting engineering approach) is used to perform deep semantic analysis on the user's question to confirm the category. If the category remains undetermined, the default route is routed to a basic question-answering category or a user clarification is triggered. Experimental testing on the GeoGPT model in the fundamental geosciences domain demonstrates that this hybrid classification approach improves both speed and accuracy compared to a single strategy or model.

[0171] This application also provides a question-answering system based on a large language model, such as... Figure 7 As shown, the system includes a classification module 10, a routing module 20, and a generation module, wherein the generation module includes a query generation unit 30, a knowledge retrieval generation unit 40, and a literature review generation unit 50.

[0172] The classification module 10 is used to classify user-input requests and obtain classification results; the classification results include at least query requests, question-and-answer requests, and research requests. The routing module 20 is used to route the requests to the corresponding units for processing based on the classification results.

[0173] For simple query requests, such as "What will the weather be like in Hangzhou next week?", the classification module 10 identifies the request as a query request and then routes it to the query generation unit 30 for processing via the routing module 20.

[0174] For basic question-and-answer type requests, such as: What are the basic geological conditions of Southwest China?, the classification module 10 identifies the request as a question-and-answer type request and routes it to the knowledge retrieval and generation unit 40 for processing via the routing module 20.

[0175] Professional research request: What are the opportunities, challenges, and key technologies for using large language models in medical diagnosis? Classification module 10 identifies this request as a research request and routes it to literature review generation unit 50 for processing via routing module 20.

[0176] Specifically, the query generation unit 30 is used to call an interface to obtain factual answers when a query request is received, and generate a query response based on the factual answers. The knowledge retrieval generation unit 40 is used to retrieve basic explanations from a structured knowledge base when a question-and-answer request is received, and generate an explanation response based on the basic explanations. The literature review generation unit 50 is used to interactively confirm the details of a research request when a research request is received, obtain context, and generate a review plan based on the context; decompose the review plan into multiple independent chapter tasks; generate chapter drafts corresponding to each chapter task in parallel using a large language model based on the context; adjust the chapter length, integrate contextual logic, insert charts and literature citations for each chapter draft to obtain a full-text review draft; review the full-text review draft and output modification opinions; modify the full-text review draft based on the modification opinions to obtain the target full-text review.

[0177] This application embodiment achieves optimal matching between processing strategies and question complexity through intelligent routing. By implementing a cascaded architecture and adaptive model fine-tuning, it significantly improves the processing efficiency and answer accuracy for various types of questions, from simple queries to complex research.

[0178] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in any of the above embodiments of the literature review generation method based on a large language model.

[0179] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0180] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0181] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for generating literature reviews based on a large language model, characterized in that, The method includes: Classify user-input requests and determine whether they are research-related requests based on the classification results; If so, the request is interactively reviewed to obtain context; and a summary generation plan is generated based on the context. The review generation plan is decomposed into multiple independent chapter tasks; based on the context, a large language model is used to generate the chapter drafts corresponding to each chapter task in parallel; The initial drafts of each chapter were adjusted in terms of chapter length, logical integration of context, and insertion of figures, tables and references to obtain the initial draft of the full review. The initial draft of the full review article is reviewed, and revision suggestions are provided. Based on the revision suggestions, the initial draft of the full review article is revised to obtain the target full review article.

2. The method according to claim 1, characterized in that, Adjusting the chapter length of each draft includes: Based on the target length of the full text and the preset weights of the tasks in each chapter, the preset length of each task in each chapter is determined. The output length of the current chapter's first draft is adjusted based on the deviation between the output length of the current chapter's first draft and the preset length. Determine the total output length and total preset length from the adjusted first chapter draft to the adjusted current chapter draft; Based on the deviation between the total preset length and the total output length, the preset length of the initial draft of subsequent chapters is updated.

3. The method according to claim 1, characterized in that, After adjusting the chapter lengths of the initial drafts of each chapter, the contextual logic was integrated, including: Input the initial drafts of adjacent chapters into the context window of the large language model to generate transitional paragraphs that connect the preceding and following chapters; The transition paragraphs are inserted between the initial drafts of adjacent chapters to achieve contextual logical integration.

4. The method according to claim 3, characterized in that, After integrating the context of each chapter's initial draft, the insertion of charts and graphs includes: Scan the full draft of the review article to identify text regions suitable for visualization, which include at least one of data-intensive statements or pattern comparison descriptions; Extract visualization data from the text region; Based on the semantic context of the text region and the visualization data, a matching chart is generated and inserted into the corresponding position.

5. The method according to claim 4, characterized in that, After integrating the context of each chapter's initial draft, the following bibliographic citations were inserted: Scan the full draft of the review and identify the statements that need to be cited; The statements that need to be cited are converted into vectors, and evidence fragments that match them are searched from a literature database; wherein, the literature database is a collection of literature retrieved based on a large language model during the literature review generation process; Based on the evidence fragments, the consistency of the statements that need to be cited is verified. If the verification passes, a citation is inserted in the corresponding position.

6. The method according to claim 1, characterized in that, The review of the initial draft of the full review article, and the output of revision suggestions based on the review results, include: The BERT model was used to identify the core claims in the initial full-text draft of the review, resulting in a core claim library. Based on the aforementioned core claim library, the logical consistency of the initial draft of the full review is examined, and conflict correction suggestions are generated. The depth of thought in the initial draft of the full review was evaluated, and suggestions for expanding the depth of thought were generated. The core arguments of the initial draft of the full review were reviewed and supplementary comments were generated. Based on the aforementioned conflict correction opinions, the aforementioned opinions on expanding the depth of thinking, and the aforementioned opinions on supplementing details, the proposed modifications are output.

7. The method according to claim 6, characterized in that, The revisions made to the initial full-text draft of the review based on the aforementioned suggestions include: The initial drafts of each of the aforementioned chapters were revised based on the suggested modifications. The revised drafts of each chapter were re-integrated with contextual logic and references were inserted to obtain the revised full-text draft of the review. Based on the aforementioned suggestions, the revised full-text review was revised to obtain the target full-text review. During the modification process, the semantic similarity between the modified content and the core claims in the core claim library is calculated. If the similarity meets the threshold, the modified content is updated in the first draft of the full review.

8. The method according to claim 1, characterized in that, The process of classifying user-input requests and determining whether a request is a research-related request based on the classification results includes: The user-input requests are initially classified based on predefined rules to obtain preliminary classification results and confidence levels; it is then determined whether the confidence levels meet a preset threshold. If so, then determine whether it is a research request based on the preliminary classification results; If not, the request is reclassified based on the BERT semantic classification model to obtain the final classification result; based on the final classification result, it is determined whether it is a research request.

9. A question-answering system based on a large language model, characterized in that, The system includes: The classification module is used to classify user-input requests and obtain classification results; the classification results include at least query requests, question-and-answer requests, and research requests. The routing module is used to route requests to the corresponding units for processing based on the classification results; The query generation unit is used to call an interface to obtain factual answers when a query request is received, and to generate a query response based on the factual answers; The knowledge retrieval and generation unit is used to retrieve basic explanations from a structured knowledge base when a question-and-answer request is received, and to generate an explanatory answer based on the basic explanations. The literature review generation unit, upon receiving a research request, interactively confirms the details of the request to obtain context; generates a review generation plan based on the context; decomposes the review generation plan into multiple independent chapter tasks; generates initial drafts for each chapter task in parallel using a large language model based on the context; adjusts the chapter length, integrates contextual logic, and inserts figures, tables, and citations for each initial draft to obtain an initial full-text review; reviews the initial full-text review and outputs modification suggestions; and modifies the initial full-text review based on the modification suggestions to obtain the target full-text review.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 8.

Images