Educational academic literature tracing method and system based on retrieval enhancement generation

Abstract

The invention discloses an educational academic literature tracing method and system based on retrieval enhancement generation. The method comprises the following steps: performing fine-grained analysis on the multi-source literature, constructing a document object containing fields such as a title, an abstract, a methodology, an experimental result and a conclusion, and establishing a field-level index and a citation network feature score; identifying an academic intention queried by a user by utilizing a large model, dynamically loading a weight mapping table according to the academic intention, and performing weighted rearrangement on multiple paths of retrieval results to obtain candidate core literatures; generating a traceability text with an explicit label based on the candidate literature; and executing reference consistency verification by using the natural language inference model, and executing illusion correction according to a verification result. According to the method, through structured field analysis and dynamic rearrangement of intention driving, the problems of semantic fragmentation and reference fictition when a general RAG is used for processing complex academic literatures are solved, and the academic traceability preciseness and accuracy are remarkably improved.

Description

A Method and System for Tracing the Sources of Educational Academic Literature Based on Enhanced Search Technical Field

[0001] This invention relates to the interdisciplinary field of artificial intelligence and educational technology (EdTech), and in particular to an educational academic literature tracing technology that addresses the large language model illusion problem in academic writing by optimizing retrieval strategies and verifying citation consistency. Specifically, it is an educational academic literature tracing method and system based on retrieval enhancement generation. Background Technology

[0002] With the rapid advancements in deep learning technology, the application of large language models in education and scientific research is becoming increasingly sophisticated. Retrieval enhancement generation techniques, through the addition of external knowledge bases, attempt to alleviate model "illusion" and supplement timely information. However, when processing long and complex texts, particularly educational academic literature, existing general RAG solutions still suffer from insufficient structure awareness and limited citation constraint capabilities, primarily in the following three aspects:

[0003] First, the slicing mechanism based on the "fixed sliding window" undermines the logical integrity of academic argumentation.

[0004] Current mainstream RAG techniques typically employ a fixed character count combined with overlapping windows to brute-force segment long documents. For rigorously structured academic papers, this approach leads to severe semantic fragmentation. For example, the literature review section in an academic paper often heavily cites the viewpoints of others, while the experimental results section presents the author's original findings. The slicing mechanism of general RAG completely ignores section boundaries, easily conflating descriptions of others in the literature review with the author's own statements in the experimental results within the same vector slice. When a user asks "What is the innovative point of this paper?", the retrieval system is highly likely to incorrectly recall previous work mentioned in the review, resulting in misattribution errors in the generated content. This affects the accuracy and reliability of academic citation results.

[0005] Second, flattened vector retrieval cannot resolve the deep mapping relationship between intent and chapter.

[0006] Academic queries often have a strong structured intent. For example, a user searching for "statistics on the negative impacts of the flipped classroom" clearly intends to find negative data in the "Discussion" or "Results" sections. However, existing general search engines are flat, relying solely on textual semantic similarity matching. Because the introductions or theoretical background sections of academic papers often contain a large amount of high-frequency keyword stuffing, general search engines easily recall background descriptions with high semantic similarity but low information density, while missing the core sections that truly contain experimental data. This mismatch between search distribution and user intent results in search results filled with low-value noise, severely reducing the professionalism of the generated answers.

[0007] Third, distortion of references and logical breaks in long contextual reasoning.

[0008] In long documents of tens of thousands of words, traditional RAGs often directly extract fragments and feed them to large models. Due to the complex cross-paragraph logic in academic literature, such as: the data in the figure is on page 5, but the analysis conclusion is on page 8, simple fragment splicing breaks the context. In addition, general models have difficulty distinguishing the strong binding relationship between "citation mark [1]" and its corresponding statement, and citation mark drift or citation is often fabricated for the sake of fluency, which cannot meet the high standard requirements of education and scientific research for the absolute accuracy of citation placement.

[0009] In conclusion, existing general-purpose RAG technologies, when faced with long academic documents, lack the ability to perceive the fine-grained logical structure of documents and the dynamic matching mechanism for users' academic intentions, and thus cannot meet the needs of high-quality academic writing assistance. Summary of the Invention

[0010] The technical problem this invention aims to solve is to address the structural failures of existing general-purpose RAG technology when processing long educational academic literature. These failures are caused by the use of unstructured sliding window slicing and flat search strategies, which result in the inability to distinguish between others' viewpoints, literature reviews, author contributions, and experimental results; the inability to accurately locate functional chapters corresponding to specific academic intentions; and the resulting content citation attribution confusion. This invention provides a method and system for tracing the source of educational academic literature that can deeply analyze the logical structure of documents, dynamically adjust chapter weights based on intentions, and perform rigorous logical verification.

[0011] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0012] A method for tracing the source of educational academic literature based on retrieval enhancement includes the following steps:

[0013] Step 1, Database Construction:

[0014] A vectorized educational academic database is established to perform fine-grained structured parsing of multi-source heterogeneous educational literature, parsing the literature into document objects containing multiple structured fields. These structured fields include at least a title field, an abstract field, an introduction field, a methodology section field, an experimental results section field, and a conclusion section field. Each structured field is vectorized to establish a field-level independent vector index. Simultaneously, the citation network feature score of each document in the academic citation relationship graph is calculated, and the field vectors, the citation network feature scores, and the text index data used for keyword retrieval are stored together in the vector database.

[0015] Step 2, Optimize the search strategy:

[0016] The system receives user queries, analyzes the academic intent type of the user query using a large language model, and transforms the user query into a query vector. Semantic vector retrieval is performed in the field-level independent vector index, and keyword retrieval is performed in parallel to obtain literal matching documents. The semantic vector retrieval results and keyword retrieval results are merged and deduplicated to obtain a preliminary document list. Based on the academic intent type, the preliminary document list is reordered according to academic intent weights to obtain a set of candidate core documents.

[0017] Step 3, Source Text Generation:

[0018] The system uses a large language model to generate answers based on the candidate core literature set, and explicitly marks the sources cited in the generated content.

[0019] Step 4, reference consistency check:

[0020] Extract citation markers and their corresponding generated statements from the generated text, construct natural language inference input samples, take the generated statements as hypotheses and the original text fragments of the cited sources as premises, input them into a pre-trained natural language inference model, output the probability distribution of logical relationships, use the natural language inference model to determine whether the original text of the cited documents logically implies the generated statements; based on the judgment results, remove or correct invalid citations, and output the final academic text.

[0021] Furthermore, the fine-grained structured analysis of multi-source heterogeneous educational literature involves dividing the literature into a set of structured fields, which includes at least: title field, abstract field, introduction field, methodology section field, experimental results section field, and conclusion section field; the document object includes citation network features describing the position of the literature in the academic citation relationship graph; the citation network features include an academic authority score calculated by the network centrality algorithm and normalized by Min-Max.

[0022] Furthermore, in the retrieval strategy optimization step, the analysis of the academic intent type of the user query using the large language model includes: defining an intent analysis prompt template, the template containing the classification criteria for educational academic queries; calling the large language model interface to map the user query to a predefined academic intent type; the academic intent type includes at least: empirical data type, theoretical origin type, methodological type, and review type.

[0023] Furthermore, the weight-based reordering operation of the preliminary literature list based on the academic intent type includes: pre-setting a weight mapping table for each structured field of the literature based on different academic intent types; determining the weight coefficient of each structured field according to the identified academic intent type; calculating the similarity between each structured field vector and the user query vector; and calculating the relevance score between the literature and the query using a weighted scoring formula, wherein the weighted scoring formula is:

[0024] Total score = (Sum of the products of the similarity between each structured field and the user query and their corresponding weight coefficients) + Citation network feature score;

[0025]

[0026] in It is the total weighted score. It is the first The cosine similarity between a structured field vector and a user query vector. It is the first dynamic mapping of academic intent types The weight coefficients of each field, This refers to the citation network feature score of the literature, i.e., the normalized academic authority. The preliminary literature list is sorted in descending order based on the total score, and the top K items (either a preset number or a dynamically determined number) are selected as the candidate core literature set. The number K can be a fixed empirical value preset by the system, derived from the maximum context window limit of the large language model, and ranging from 5 to 10; or it can be a dynamically determined value by the system. In the dynamic determination method, K can be determined based on the difference in total scores between adjacent literatures after sorting, the number of literatures with total scores exceeding a preset threshold, or the context capacity allowed to be processed by the downstream generation model.

[0027] Furthermore, the weighting coefficient The determination rules include: when the academic intent type is "empirical data type", a first weight coefficient is assigned to the experimental results section field, and a second weight coefficient is assigned to each of the remaining fields, wherein the first weight coefficient is greater than any of the second weight coefficients, and the value range of the first weight coefficient is 0.4 to 0.6; when the academic intent type is "theoretical origin type", the sum of the weights of the abstract field and the introduction field is the maximum value among the weights of each field; when the academic intent type is "methodology type", the weight coefficient of the methodology section field is the maximum value among the weights of each field.

[0028] Furthermore, in the retrieval strategy optimization step, the semantic vector retrieval results and keyword retrieval results are sorted and fused using the Reciprocal Rank Fusion algorithm to improve the coverage and sorting stability of the preliminary document list.

[0029] Furthermore, the field weight coefficients are updated online based on user click feedback, the citation hit rate of generated text, or the natural language inference verification pass rate, to form a closed-loop adaptive optimization mechanism for retrieval ranking.

[0030] Furthermore, the judgment result is determined based on the probability distribution of the logical relationship in the output, which includes: implication, contradiction, and neutrality. If the probability of the "implication" label exceeds a preset threshold, the verification is deemed successful. If the probability of the "contradiction" or "neutral" label is the highest, or if the probability of the "implication" label does not exceed the preset threshold, it is determined to be a false citation, and an illusion correction mechanism is triggered. The illusion correction mechanism includes deleting the citation mark or rewriting the generated statement based on the original text of the document using a large language model, and performing natural language reasoning verification on the rewritten statement again.

[0031] This invention also provides a retrieval system for educational academic literature based on enhanced retrieval, comprising:

[0032] The database construction module is used to build a vectorized educational academic database, perform fine-grained structured parsing and vectorized indexing of documents, parse documents into document objects containing multiple structured fields, and build field-level independent vector indexes.

[0033] The retrieval optimization module is used to receive user queries, analyze the academic intent type of the user queries, and perform weight-based reordering of the multi-path recall retrieval results based on the academic intent type to obtain a set of candidate core documents.

[0034] The generation module is used to generate answers with explicit citations based on a set of candidate core documents using a large language model;

[0035] The verification module is used to perform citation consistency verification on the generated text using a natural language inference model, and to remove or correct invalid citations based on logical implication relationships.

[0036] The present invention has the following beneficial effects:

[0037] Significantly improves retrieval accuracy and recall: This invention parses documents into document objects containing multiple structured fields and establishes field-level independent vector indexes, enabling the retrieval process to focus on specific functional blocks of the document. Simultaneously, it introduces a dynamic weight mapping mechanism based on academic intent type, breaking away from the traditional retrieval's neglect of document structure. Addressing the specific needs of educational academic queries, such as differentiating between searching for "empirical data" and "theoretical reviews," it dynamically adjusts the weights of different chapters in the document, ensuring that the retrieval results are not only semantically relevant but also accurately match the user's academic intent in terms of structure and function, effectively solving the problems of missed core literature and unstable ranking.

[0038] A rigorous citation consistency verification mechanism: This invention uses a natural language reasoning model as an automated verifier to verify the logical relationship between the generated text and the cited sources one by one, and constructs a closed-loop correction process of "rewriting - re-verification - deletion" to effectively eliminate invalid citations that, although they exist, are unrelated to the generated content or have numerical distortions, thus eliminating citation fraud in academic writing from a logical perspective.

[0039] Enhanced fine-grainedness and system scalability in long document processing: Through field-level independent indexing and parallel similarity retrieval mechanism, the system can maintain high retrieval efficiency when facing large-scale long document sets; combined with multi-way hybrid retrieval recall and Reciprocal Rank Fusion fusion ranking algorithm, the coverage and ranking stability of the preliminary document list are improved, thereby enhancing the practicality and robustness of the system in complex educational and academic retrieval scenarios.

[0040] An adaptive optimization closed loop for retrieval ranking has been achieved: by introducing user click feedback, the citation hit rate of generated text, and the pass rate of natural language inference verification into the field weight update mechanism, the system can adjust the weight mapping table online according to the actual usage effect, avoiding the rigidity of ranking caused by purely manual experience rules, thereby continuously improving the overall quality of retrieval and generated results. Attached Figure Description

[0041] Figure 1 is a flowchart of the educational academic literature tracing method based on retrieval enhancement according to an embodiment of the present invention;

[0042] Figure 2 is a flowchart of multi-source academic data acquisition and preprocessing in an embodiment of the present invention;

[0043] Figure 3 is a schematic diagram illustrating the principle of fine-grained structured document parsing in an embodiment of the present invention;

[0044] Figure 4 is a schematic diagram of the data structure of the multi-field vector index database in an embodiment of the present invention;

[0045] Figure 5 is a flowchart of the query intent perspective based on the large language model in an embodiment of the present invention;

[0046] Figure 6 is a logical schematic diagram of multi-path hybrid retrieval and recall in an embodiment of the present invention;

[0047] Figure 7 is a logic diagram of the core algorithm of the intent-driven structured weighted sort in an embodiment of the present invention;

[0048] Figure 8 is a schematic diagram of the construction of prompt words for source text generation based on reference awareness in an embodiment of the present invention;

[0049] Figure 9 is a schematic diagram of the input construction of the natural language reasoning model in the reference consistency verification in an embodiment of the present invention;

[0050] Figure 10 is a flowchart of the determination logic and correction mechanism for hallucination reference in an embodiment of the present invention;

[0051] Figure 11 is a functional module structure diagram of the educational academic literature tracing system provided in an embodiment of the present invention. Detailed Implementation

[0052] The embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples.

[0053] Referring to Figure 1, Figure 1 is a flowchart of the educational academic literature tracing method based on retrieval enhancement provided in this embodiment of the invention. To address the problems of low retrieval intent matching and fictitious citations in existing technologies, the method performs fine-grained optimization of the traditional retrieval generation process, specifically including steps S1 to S7:

[0054] S1: Steps for Acquiring and Preprocessing Multi-Source Academic Data

[0055] As shown in Figure 2, academic literature data in the field of education is obtained, and the data sources include, but are not limited to, digital journals, conference proceedings, dissertations, and online academic resources. To ensure the quality of subsequent processing, the raw data needs to be cleaned and standardized preprocessed, specifically including: using regular expressions to remove non-textual interference items such as headers, footers, page numbers, and reference lists; identifying and removing documents with garbled characters and incorrect formats; and using algorithms such as SimHash to deduplicate duplicate documents to obtain a clean text data stream.

[0056] S2: Fine-grained structured analysis and vector library construction steps

[0057] A vectorized educational academic database was established, and in-depth structured analysis and index construction were carried out on the literature processed by S1.

[0058] First, perform document structure parsing.

[0059] As shown in Figure 3, a hybrid parsing strategy combining rule-based and deep learning is adopted:

[0060] Use PDF parsing tools, such as PyMuPDF or OCR technology, to extract the original text stream and font layout information of the document;

[0061] The algorithm for identifying the hierarchical structure of documents is based on visual features and directory tree structure recognition.

[0062] A pre-trained text classification model is used to perform semantic classification on text blocks under each level of heading, and these blocks are mapped to predefined structured fields.

[0063] The structured field set includes at least the following: title field, abstract field, introduction field, methodology section field, experimental results section field, and conclusion section field.

[0064] For paragraphs that cannot be clearly categorized, they should be placed in the "Other in the Main Text" field.

[0065] Secondly, as shown in Figure 4, citation network features are extracted.

[0066] Analyze the citation relationship graph of the literature, and construct a directed graph structure between the literature nodes based on the citation relationship graph;

[0067] The network centrality algorithm is used to calculate the academic authority score of each document, and the authority score is normalized by Min-Max to map it to the interval [0,1], which serves as a static indicator for measuring the academic quality of the document.

[0068] Finally, construct a multi-field vector index.

[0069] As shown in Figure 4, the pre-trained embedding model is used to vectorize each structured field of the document object to generate the corresponding field vector.

[0070] Each field vector is stored in a field-level independent vector index, which adopts a hybrid structure of inverted index and approximate nearest neighbor graph index, and supports parallel similarity retrieval of each structured field vector.

[0071] The field vectors, citation network feature scores, and text index data used for keyword retrieval are associated and stored in a vector database to establish an index structure that supports multi-field combined queries.

[0072] The field vectors, along with corresponding citation network feature scores, text content, and other metadata, are stored in a vector database to construct a multi-field combined query index that supports vector similarity retrieval, numerical range filtering, and full-text keyword matching.

[0073] S3: Steps for Analyzing Query Intent Based on a Large Model

[0074] Receive user queries and analyze the academic intent type of the user queries using a large language model.

[0075] This step aims to address the problem that traditional retrieval methods cannot distinguish user research objectives. As shown in Figure 5, it specifically includes: defining an "intent analysis prompt template" to request the large language model application programming interface. This template contains classification criteria for educational and academic intent, as well as classification and format constraints.

[0076] The classification criteria include at least the following:

[0077] Empirical data type: queries focus on specific experimental data, statistical results, significance proofs, p-values, t-values, or correlations between variables;

[0078] Theoretical Origin Category: Search for definitions of concepts, the historical evolution of theoretical frameworks, and the underlying theoretical support or paradigm shifts;

[0079] Methodology category: Search for specific research tools, scale design, experimental procedures, algorithmic processes, or intervention methods;

[0080] Review articles: Search for macro-level research status, development trends, research hotspots, or comprehensive reviews of literature.

[0081] By calling the interface, user queries are mapped to the predefined intent tags mentioned above, such as "empirical data category", which serve as the basis for subsequent weight reordering decisions.

[0082] S4: Multi-path hybrid retrieval and recall steps

[0083] As shown in Figure 6, the user query is transformed into a query vector, and a multi-way recall is performed in the vector database to obtain a preliminary list of literature.

[0084] To balance accuracy and coverage in the retrieval, this step performs the following operations in parallel:

[0085] On the one hand, vector similarity retrieval is performed on the user query vector and each structured field vector in the field-level independent vector index to obtain a list of semantically matched documents;

[0086] On the other hand, keyword retrieval is performed on the user query in the full-text inverted index to obtain a list of documents that provide an exact match. The document results returned by multiple retrievals are merged and deduplicated, and the Reciprocal RankFusion algorithm is used to sort and merge the semantically matching document list and the literal matching document list, resulting in a preliminary document list with higher coverage and more stable ranking, such as the Top-100 documents. At this stage, the weight differences of specific chapters within a document are not considered. After obtaining the preliminary document list, it can optionally be cached for quick access in subsequent intent-driven reordering steps, thereby reducing the computational overhead of repeated retrievals.

[0087] S5: Intent-driven structured weighted sorting steps.

[0088] Based on the academic intent type identified in step S3, the preliminary literature list obtained in step S4 is reordered based on weights to obtain a set of candidate core literature.

[0089] Figure 7 illustrates the core logic of the retrieval strategy optimization described in S5. This step aims to solve the semantic disambiguation problem of traditional retrieval algorithms. In general RAGs, when users query experimental data, the "Review" section containing the keyword "data" is often ranked higher due to its longer text and higher word frequency, causing the actual experimental results section to be buried. This invention introduces a dynamic weight mapping mechanism to forcibly improve the ranking score of functional fields that are strongly related to the current academic intent, thereby achieving a logical structure-based reordering without changing the original vector similarity. Specifically, it includes:

[0090] Weight configuration is implemented by pre-setting weight mapping tables for different academic intent types on various structured fields of the document. When the academic intent type is "Empirical Data," the experimental results section field in the document is assigned a higher weight coefficient; when the academic intent type is "Theoretical Origin," the abstract and introduction fields in the document are assigned higher weight coefficients; and when the academic intent type is "Methodology," the methodology section field in the document is assigned a higher weight coefficient. These field weight coefficients are updated online based on user click feedback, the citation hit rate of the generated text, or the natural language inference verification pass rate, forming a closed-loop adaptive optimization mechanism for retrieval ranking.

[0091] Perform weighted scoring calculation: Traverse each document in the preliminary document list, calculate the cosine similarity between its structured field vector and the user query vector, and combine the weight coefficients corresponding to the current intent and the normalized citation network features to calculate the total relevance score of each document using the weighted scoring formula, which is as follows:

[0092] Total score = (Sum of the products of the similarity between each structured field and the user query and their corresponding weight coefficients) + Citation network feature score;

[0093]

[0094] in It is the total weighted score. It is the first The cosine similarity between a structured field vector and a user query vector. It is the first dynamic mapping of academic intent types The weight coefficients of each field, It is the citation network feature score of the literature, that is, the normalized academic authority.

[0095] Execute sorting and truncation: Sort the preliminary literature list in descending order according to the total score, and truncate the top K items as the candidate core literature set, thereby ensuring that the search results accurately match the user's academic intent in terms of structure and function.

[0096] S6: Reference-aware source text generation steps.

[0097] The large language model is used to generate answers based on the candidate core literature set. As shown in Figure 8, this step constructs a "source-tracing prompt word template" and embeds the candidate core literature set obtained in S5 as contextual background information into the prompt words. The prompt words adopt a three-part structure of "system instruction + context embedding + user input" and explicitly separate different input blocks in a structured format.

[0098] In the source tracing prompt word template, the output format of the large language model is explicitly constrained, requiring the model to output only structured text consisting of natural language statements and corresponding citation tags, and prohibiting the output of assertive statements without source attribution. During the generation process, the decoding strategy of the large language model is parameter-controlled, including setting the temperature parameter to a preset low value range and limiting the maximum generation length, to reduce the probability of numerical distortion or overgeneralization in the generated text. Although the generation range is constrained by prompt words, due to the probabilistic generation characteristics of the large language model, overgeneralization or numerical deviation of the original text may still occur. Therefore, subsequent citation consistency verification steps are needed to logically verify and correct each line of the generated content.

[0099] S7: Logical implication verification and illusion correction steps.

[0100] The generated text and its citations are checked line by line using a Natural Language Inference (NLI) model to remove or correct invalid citations.

[0101] Figure 9 illustrates the detailed process of reference consistency verification described in S7, including:

[0102] Extracting Citation Pairs: Parsing the preliminary academic text generated by S6, using regular expressions to extract all citation markers and their corresponding generated statements, forming a check pair of <generated statement, cited original text excerpt>.

[0103] Constructing the inference input: The generated statement is used as an assumption, and the original text fragments of the cited source are used as premises, and then concatenated according to the input format of the natural language inference model.

[0104] Perform logical reasoning: Input the concatenated text into a pre-trained natural language reasoning model and output a probability distribution of logical relationships, including: implication, contradiction, and neutrality.

[0105] As shown in Figure 10, the judgment and correction are as follows:

[0106] If the probability of the "implication" tag exceeds the preset threshold, the verification is deemed successful, and the reference and statement are retained.

[0107] If the probability of the "contradictory" or "neutral" label is the highest, or the probability of the "implied" label is not high enough, it is judged as a false reference and the illusion correction mechanism is triggered.

[0108] Illusion correction mechanism: First, the large language model is called again, requiring it to rewrite the generated sentence strictly according to the original text of the document; the rewritten sentence is then checked again by natural language reasoning; if the rewritten sentence still fails the second check, a forced strategy is executed to delete the entire sentence containing the citation from the final text to ensure the absolute rigor of the output content.

[0109] All the above-mentioned optional technical solutions can be combined in any way to form optional embodiments of the present invention, and will not be listed in detail here.

[0110] Example

[0111] To make the objectives, technical solutions, and advantages of the present invention clearer and to further facilitate those skilled in the art to better understand the essence of the present invention, the embodiments of the present invention will be further described below in conjunction with the accompanying drawings.

[0112] Considering the wide range of applications involved in tracing educational academic literature, the following example focuses on "An Empirical Study on the Impact of Flipped Classrooms on Junior High School Students' Geometry Scores." In this embodiment, the core retrieval and ranking strategy relies on dynamic weighted scoring, as shown in Figure 7. Specifically, the weighted scoring formula for this academic retrieval problem has its set of operators and variables limited to the range in the table below, with the meanings of the corresponding symbols shown in the table:

[0113] Table 1. Symbol Meaning Table

[0114] Symbol Meaning: Sim_Ti: Cosine similarity between the title field and the query vector; Sim_Ab: Cosine similarity between the abstract field and the query vector; Sim_In: Cosine similarity between the introduction field and the query vector; Sim_Me: Cosine similarity between the methodology section field and the query vector; Sim_Re: Cosine similarity between the experimental results section field and the query vector; Sim_Co: Cosine similarity between the conclusion section field and the query vector; Net_Score: Citation network feature score of the literature, i.e., normalized academic authority; W_Ti: Dynamic weight coefficient corresponding to the title field; W_Ab: Dynamic weight coefficient corresponding to the abstract field; W_In: Dynamic weight coefficient corresponding to the introduction field; W_Me: Dynamic weight coefficient corresponding to the methodology section field; W_Re: Dynamic weight coefficient corresponding to the experimental results section field; W_Co: Dynamic weight coefficient corresponding to the conclusion section field. surface

[0115] The weighted scoring formula provided in this embodiment of the invention is used to calculate the final relevance score of a document. This expression consists of the similarity score of each structured field, dynamic weight coefficients, and citation network feature scores. During the optimization and execution of the retrieval strategy, by analyzing the academic intent of the user's query, the weight status of each field at the current moment is determined, and the numerical values ​​represented by the corresponding variables are obtained. These values ​​are then input into the weighted scoring formula to calculate a total relevance value.

[0116] Specifically, the weighted scoring formula in this embodiment can be expressed by the following formula:

[0117]

[0118] If, during score calculation, the system identifies the user's academic intent as "empirical data," the system automatically loads the weight configuration, assigning the highest weight to the experimental results section. Let's assume the configuration is as follows: If a document is found, the similarity scores of its fields with the query results are as follows: Furthermore, the citation network feature score of this document... Therefore, the final priority value corresponding to this document is:

[0119]

[0120] Figure 8 shows a schematic diagram of the construction of prompt words for source text generation based on reference awareness provided in an embodiment of the present invention.

[0121] The 'source-tracing and prompt word template' constructed in this step adopts a three-part structure of 'instruction + context + query', as detailed below:

[0122] System Prompt:

[0123] You are a rigorous academic writing assistant. Please answer the user's query based on the [reference excerpt] provided below.

[0124] Require:

[0125] Your answers must be strictly based on the provided reference passages; you are strictly prohibited from fabricating information using your training knowledge.

[0126] When citing opinions or data, the source must be explicitly indicated at the end of the sentence in the format [Doc_ID].

[0127] If the reference passage cannot answer the question, please state that you cannot answer it.

[0128] Context Injection:

[0129] [Doc_1]: ... (Document fragment 1 selected in step S5)...

[0130] [Doc_2]: ... (Document fragment 2 selected in step S5)...

[0131] ...

[0132] [Doc_K]: ... (Document fragment K selected in step S5)...

[0133] User Input:

[0134] [Question]: What are the specific data on the impact of the flipped classroom model on geometry grades?

[0135] Model Output:

[0136] The flipped classroom model significantly improved the geometry scores of junior high school students [Doc_1]. Experimental data showed that the experimental group had a mean geometry score of 85.4 in the post-test, while the control group had a mean score of 78.2 [Doc_3]. Independent samples t-test showed that the difference was statistically significant (p<0.05) [Doc_3].

[0137] Figure 9 shows a schematic diagram of reference consistency verification based on a large language model provided in an embodiment of the present invention.

[0138] Input pairs for the constructed Natural Language Inference (NLI) model:

[0139] Hypothesis, i.e., the statement that generates the hypothesis:

[0140] "The flipped classroom significantly improved the geometry scores of junior high school students (P<0.05)."

[0141] Premise, that is, citing excerpts from the original text:

[0142] "...the geometric mean score of the experimental group in the post-test was 85.4, while that of the control group was 78.2. The independent samples t-test showed a significant difference (t=2.45, p=0.032)..."

[0143] Task instructions:

[0144] Determine whether the 'premise' logically implies the 'assumption'. Output probability distribution: [Entailment, Contradiction, Neutral].

[0145] Output the probability distribution of logical relationships, with corresponding labels "[Implication, Contradiction, Neutral]".

[0146] Finally, logical relationships are extracted using regular expressions or rule-based text parsing. If the result is "contradictory" or "neutral," it is considered an invalid reference, triggering subsequent culling or rewriting mechanisms.

[0147] After identifying the intent, such as "empirical data", the system automatically calls the corresponding weight table, such as W_Re=0.4, calculates the score of each candidate document using the formula, and extracts the top K items according to the score.

[0148] Finally, regular expressions are used to parse the literature index, and the corresponding literature is extracted from the list to form a candidate core literature set.

[0149] Figure 10 shows a flowchart of the determination logic and correction mechanism for hallucination references provided in an embodiment of the present invention.

[0150] Following the probability distribution output by the NLI model above, the system executes the following judgment and correction logic:

[0151] The first step is threshold determination. The confidence threshold for the implication relationship is set to 0.8. If the model outputs an 'implication' probability...

[0152]

[0153] If the reference is found to be valid, the system will retain the statement and the reference mark (e.g., [1]).

[0154] The second step is illusion recognition. In this embodiment, if the NLI model outputs a "contradiction" with the highest probability, or if the "implication" probability is only 0.4, below the threshold, the system determines the reference to be an "illusory reference." For example, if the generated statement claims "significant difference," but the p-value calculated from the original data is actually greater than 0.05, the NLI model will identify a logical contradiction.

[0155] The third step is to correct the closed loop. Once a hallucination is determined, the system automatically triggers a correction mechanism:

[0156] Rewrite attempt: The system calls the large language model interface again and inputs the command: 'The statement is found to be inconsistent with the original text logic. Please rewrite the statement strictly according to the original text fragment. If the original text does not support it, remain silent.'

[0157] Secondary validation: Perform NLI inference on the rewritten statement again. If the rewritten statement passes the threshold check, replace the original statement.

[0158] Forced fallback: If the rewritten statement still fails the second validation, for example, if the original text did not mention the relevant conclusion, the system will execute a forced policy, directly deleting the entire sentence containing the quote from the final answer to ensure the absolute rigor of the output content.

[0159] In this embodiment, a vectorized educational academic database is established through database construction. Then, through retrieval strategy optimization, a large language model is used to analyze academic intent and rearrange the results to iteratively obtain candidate core documents. Finally, through citation consistency verification, an invalid citation is eliminated using a natural language inference model, which serves as the final output academic text.

[0160] Referring to Figure 11, which is a functional module structure diagram of the educational academic literature tracing system provided in an embodiment of the present invention, the system 11 of this embodiment includes:

[0161] The database construction module 111 is used for database construction, establishing a vectorized educational academic database, performing structured parsing and vectorized indexing of documents, and parsing documents into document objects containing multiple structured fields. Specifically, structured parsing can adopt a combination of rule-based and deep learning approaches. First, PDF parsing tools, such as PyMuPDF, are used to extract text and font features; a document tree is constructed using a heading level recognition algorithm; and a pre-trained text classification model is used to semantically classify text blocks under each heading level, mapping them to predefined fields such as 'methodology' and 'experimental results'.

[0162] The retrieval optimization module 112 is used for retrieval strategy optimization. It uses a large language model to analyze the academic intent type of user queries and reorders and optimizes the retrieval results based on the intent.

[0163] The generation module 113 is used for source text generation. It uses a large language model to generate answers based on a set of candidate core documents and explicitly marks the sources cited in the generated content.

[0164] The verification module 114 is used for citation consistency verification. It uses a natural language reasoning model to perform line-by-line logical implication verification on the generated text and its citations. Based on the judgment results, invalid citations are removed or corrected, and the final academic text is output.

[0165] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0166] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A method for tracing the source of educational academic literature based on retrieval enhancement, comprising the following steps: Step 1, Database Construction: Establish a vectorized educational academic database, perform fine-grained structured parsing on multi-source heterogeneous educational literature, and parse the literature into document objects containing multiple structured fields; vectorize each structured field separately and establish field-level independent vector indexes; at the same time, calculate the citation network feature score of each document in the academic citation relationship graph, and store the field vectors, the citation network feature scores, and the text index data used for keyword retrieval in the vector database. Step 2, retrieval strategy optimization: Receive user queries, analyze the academic intent type of the user query using a large language model, and transform the user query into a query vector; perform semantic vector retrieval in the field-level independent vector index, and perform keyword retrieval in parallel to obtain literal matching documents; The semantic vector retrieval results and keyword retrieval results are merged and deduplicated to obtain a preliminary literature list. Based on the academic intent type, the preliminary literature list is rearranged according to academic intent weights to obtain a candidate core literature set. Step 3: Source text generation: A large language model is used to generate answers based on the candidate core literature set, and the citation sources are explicitly marked in the generated content. Step 4: Citation consistency verification: Citation markers and their corresponding generated statements are extracted from the generated text, and natural language inference input samples are constructed. The generated statements are used as hypotheses, and the original text fragments corresponding to the citation sources are used as premises. These are input into a pre-trained natural language inference model, which outputs a logical relationship probability distribution. The natural language inference model is used to determine whether the original text of the cited literature logically implies the generated statements. Invalid citations are removed or corrected based on the judgment results, and the final academic text is output.

2. The method for tracing educational academic literature based on retrieval enhancement as described in claim 1, characterized in that, The fine-grained structured analysis of multi-source heterogeneous educational literature involves dividing the literature into a set of structured fields, which includes at least: title field, abstract field, introduction field, methodology section field, experimental results section field, and conclusion section field.

3. The method for tracing educational academic literature based on retrieval enhancement as described in claim 1, characterized in that, The document object includes citation network features that describe the position of the document in the academic citation relationship graph. The citation network features include an academic authority score calculated by the network centrality algorithm and normalized by Min-Max.

4. The method for tracing educational academic literature based on retrieval enhancement as described in claim 1, characterized in that, The process of using a large language model to analyze the academic intent type of the user query includes: defining an intent analysis prompt template, which contains classification criteria for educational academic queries; calling the large language model interface to map the user query to a predefined academic intent type; the academic intent type includes at least: empirical data type, theoretical origin type, methodological type, and review type.

5. The method for tracing educational academic literature based on retrieval enhancement as described in claim 1, characterized in that, The weighted rearrangement of the preliminary literature list based on academic intent type includes: pre-setting a weight mapping table for each structured field of the literature based on different academic intent types; determining the weight coefficient of each structured field according to the identified academic intent type; calculating the similarity between each structured field vector and the user query vector; and calculating the relevance score between the literature and the query using a weighted scoring formula, wherein the weighted scoring formula is: Total score = (sum of the products of the similarity between each structured field and the user query and the corresponding weight coefficient) + citation network feature score; ;in It is the total weighted score. It is the first The cosine similarity between a structured field vector and a user query vector. It is the first dynamic mapping of academic intent types The weight coefficients of each field, It is the citation network feature score of the literature, that is, the normalized academic authority; the preliminary literature list is sorted in descending order according to the total score, and the top K items are selected as the candidate core literature set.

6. The method for tracing educational academic literature based on retrieval enhancement as described in claim 5, characterized in that, The weighting coefficient The determination rules include: when the academic intent type is "empirical data type", a first weight coefficient is assigned to the experimental results section field, and a second weight coefficient is assigned to each of the remaining fields, wherein the first weight coefficient is greater than any of the second weight coefficients, and the value range of the first weight coefficient is 0.4 to 0.6; when the academic intent type is "theoretical origin type", the sum of the weights of the abstract field and the introduction field is the maximum value among the weights of each field; when the academic intent type is "methodology type", the weight coefficient of the methodology section field is the maximum value among the weights of each field.

7. The method for tracing educational academic literature based on retrieval enhancement as described in claim 1, characterized in that, Step 2 involves merging and deduplicating the semantic vector retrieval results and keyword retrieval results using the Reciprocal RankFusion fusion algorithm to improve the coverage and ranking stability of the preliminary document list.

8. The method for tracing educational academic literature based on retrieval enhancement as described in claim 5, characterized in that, The field weight coefficients are updated online based on user click feedback, the citation hit rate of generated text, or the natural language inference verification pass rate, to form a closed-loop adaptive optimization mechanism for retrieval ranking.

9. The method for tracing educational academic literature based on retrieval enhancement as described in claim 1, characterized in that, The judgment result is determined based on the probability distribution of the logical relationship in the output, which includes: implication, contradiction, and neutrality. If the probability of the "implication" label exceeds a preset threshold, the verification is deemed successful. If the probability of the "contradiction" or "neutral" label is the highest, or the probability of the "implication" label does not exceed the preset threshold, it is determined to be a false citation, and an illusion correction mechanism is triggered. The illusion correction mechanism includes deleting the citation mark or rewriting the generated statement based on the original text of the document using a large language model, and performing natural language reasoning verification on the rewritten statement again.

10. A retrieval system for educational academic literature based on enhanced retrieval, comprising: Database construction module: used to build a vectorized educational academic database, perform fine-grained structured parsing and vectorized indexing of documents, parse documents into document objects containing multiple structured fields, and build field-level independent vector indexes; Search optimization module: used to receive user queries, analyze the academic intent type of user queries, and perform weight-based reordering of multi-path recall search results based on the academic intent type to obtain a set of candidate core documents; The generation module is used to generate answers with explicit citations based on a set of candidate core documents using a large language model. Verification module: Used to perform citation consistency verification on the generated text using a natural language inference model, and to remove or correct invalid citations based on logical implication relationships.

Images