Retrieval fragment reference labeling method and device based on streaming big language model and medium

Abstract

The invention discloses a retrieval fragment reference marking method and device based on a streaming big language model and a medium, belongs to the technical field of natural language processing, and aims to solve the problems that reference marks cannot be stably analyzed in a streaming output stage of an existing model, a unified reference format and a reference list required by front-end display are difficult to synchronously output from a streaming text, and the data processing efficiency is low. Therefore, the technical problem of insufficient interpretability of large language model reply is solved. The method comprises the following steps: in a large language model streaming generation process, acquiring a streaming text which is generated step by step and a retrieved reference fragment; constructing a short window buffer pool, identifying a source identifier of the streaming text in real time, and performing alignment processing on the source identifier and a fragment identifier corresponding to the reference fragment; based on the aligned streaming text and reference fragment, generating an incremental text and an effective reference list; and based on the reference fragment and the answer-side context in the effective reference list, positioning a text range directly related to the answer in the reference fragment, and performing highlight labeling.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology, and in particular to a method, device and medium for retrieving fragment citation annotation based on a streaming large language model. Background Technology

[0002] With the widespread adoption of large language models, Retrieval-Augmented Generation (RAG) technology has become a mainstream paradigm for improving the controllability and timeliness of responses. A typical RAG includes: an offline knowledge base; online retrieval and ranking; and using recalled fragments (hereinafter referred to as "retrieval fragments" or "nodes") as contextual input to generate answers and provide source citations.

[0003] In practical applications, citations are not only used to enhance interpretability and verifiability, but are also frequently used for compliance audits and user trust endorsements. Existing systems, without citations and highlighting, suffer from the following main problems: 1. Insufficient Explainability and User Trust: Without providing any source citations or highlighted locations, the generated answers are completely decoupled from their evidence chain, making it difficult for users to determine which document, paragraph, or sentence the conclusion originates from. In serious scenarios such as regulations, policies, medical information, financial systems, and contract terms, the lack of verifiable sources significantly weakens the acceptability and persuasiveness of the answers, leading to increased subsequent communication costs. Users often need to conduct secondary searches or manual verification, lengthening the decision-making chain and reducing the overall system experience and conversion rate. Furthermore, the lack of highlighted "source invisibility" obscures the boundary between the model's inferences and factual basis, making it impossible for readers to distinguish between original facts and model inductions or extrapolations, thus compromising both objective correctness and subjective credibility.

[0004] 2. Both engineering operations and user experience suffer: On the operations side, the lack of citations and highlighting significantly increases the time required for fault location and problem reproduction. When encountering irrelevant answers, outdated information, or factual errors, engineers cannot quickly jump to the specific evidence area that triggered the conclusion for comparison. Troubleshooting is forced to be done manually by searching through logs and large sections of the original text, increasing the average processing time. After the knowledge base is updated, it is also difficult to promptly assess whether existing answers are still supported by the latest version, creating the potential risk of knowledge obsolescence and answer drift. On the user side, long text reading and mobile scenarios are particularly affected: readers cannot jump to the evidence location with one click for secondary verification and can only manually search for keywords in the entire document. When answers involve numbers, formulas, legal provisions, or table parameters, the lack of character-level or sentence-level highlighting makes it easier to cause ambiguity and repeated communication, further reducing satisfaction and overall efficiency.

[0005] In summary, in existing RAG dialogues, the model's streaming output stage cannot reliably parse citation tags, and post-processing of entire segments leads to high latency and misplacement. The unified citation format and citation list required for front-end display are difficult to generate synchronously from the streaming text, ultimately resulting in insufficient interpretability of the large language model's response and difficulty in determining the highlighted range of citation content, thus affecting the user experience of the large language model. Summary of the Invention

[0006] This invention provides a method, device, and medium for retrieval fragment citation annotation based on a streaming large language model, which addresses the following technical problem: In existing RAG dialogues, the model cannot stably parse citation tags during the streaming output stage, and the unified citation format and citation list required for front-end display are difficult to generate synchronously from the streaming text, resulting in insufficient interpretability of the large language model's response.

[0007] The embodiments of the present invention adopt the following technical solutions: On one hand, embodiments of the present invention provide a method for retrieving citation annotation of fragments based on a streaming large language model. The method includes: during the streaming generation process of the large language model, acquiring the streaming text gradually generated by the large language model and the retrieved citation fragments; wherein, the citation fragments contain corresponding fragment identifiers; Construct a short window buffer pool, identify the source identifier of the streaming text in real time, and align the source identifier with the segment identifier corresponding to the referenced segment; Based on the aligned streaming text and cited fragments, an incremental body text and a list of valid citations are generated. Based on the cited fragments in the valid citation list and the answer-side context directly related to this citation, the text range directly related to the answer in the cited fragment is located and highlighted.

[0008] In one feasible implementation, during the streaming generation process of the large language model, the streaming text gradually generated by the large language model and the retrieved reference fragments are obtained, specifically including: In a scenario where a user uses a large language model for streaming generation, the streaming text generated step by step during the streaming generation process of the large language model is read; wherein, the streaming text contains at least unresolved reference placeholders; Simultaneously, the source document data retrieved during the streaming generation process is read, and the referenced fragments in the source document data are identified; Extract the fragment identifier of the referenced fragment and construct a fragment identifier index; Extract the location metadata of the referenced fragment in the source document data and the document identifier of the source document data.

[0009] In one feasible implementation, a short-window buffer pool is constructed to identify the source identifier of the streaming text in real time, and to align the source identifier with the segment identifier corresponding to the referenced segment. Specifically, this includes: Based on a preset window length, a short window buffer pool is constructed; the streaming text is received through the short window buffer pool to obtain streaming text fragments; Identify reference placeholders in the current streaming text segment within the short window buffer pool; When a reference placeholder is detected in the short window buffer pool, the source identifier corresponding to the reference placeholder is extracted, and the source identifier is aligned with the fragment identifier index.

[0010] In one feasible implementation, based on the aligned streaming text and cited fragments, incremental text and a list of valid citations are generated, specifically including: After successful alignment, source features are established based on the combination of the document identifier and the fragment identifier; According to the order in which the source feature first appears, assign corresponding footnote numbers. When the same source appears multiple times, reuse the already assigned footnote numbers. The footnote numbers involved in the same document are aggregated into a compact superscript representation, and only the currently identified reference placeholders are replaced; Once a complete pair of reference placeholders is identified and both are replaced, the streaming text fragment between the complete reference placeholders is output as incremental text. After the streaming session ends, the accumulated reference data during the streaming generation is organized into a structured list and output externally, thus obtaining the effective reference list.

[0011] In one feasible implementation, after the streaming session ends, the accumulated reference data during the streaming generation period is organized into a structured list and output externally to obtain the effective reference list, which specifically includes: At the end of the streaming session, the last remaining segment of streaming text in the short window buffer is output. The accumulated reference fragments during the streaming generation process, along with their corresponding document identifiers, fragment identifiers, footnote numbers, and interval information used for front-end positioning, are compiled into a structured list and output externally, resulting in the effective reference list.

[0012] In one feasible implementation, based on the cited fragments in the valid citation list and the answer-side context directly related to the current citation, the text range directly related to the answer within the cited fragments is located, specifically including: Based on the current business scenario, a text location scheme is determined; wherein, the text location scheme includes a first text location scheme and a second text location scheme; If the current business scenario is a high-concurrency scenario, then the first text location scheme is used to locate the range of text in the quoted fragment that is directly related to the answer; If the current business scenario is a high-precision scenario, then the second text positioning scheme is used to locate the range of text in the quoted fragment that is directly related to the answer.

[0013] In one feasible implementation, the first text localization scheme is used to locate the text range within the cited fragment that is directly related to the answer, specifically including: The reference fragments in the valid reference list and the answer-side context are input into the preset algorithm corresponding to the first text location scheme; The segmentation step size is determined based on the text length of the context surrounding the answer. Based on the segmentation step size, the reference fragment is slid-segmented to obtain an n-ary segmentation set; wherein the n-ary segmentation set includes at least unary segments, binary segments, ternary segments, and quaternary segments; The segments in the n-gram segment set are projected onto the answer-side context into a unified semantic representation space, and the cosine similarity between the two is calculated to obtain the semantic relevance score of each segment. Penalty scores are assigned to segments whose length is lower than a first preset threshold or whose length is higher than a second preset threshold; the coverage ratio of the answer-side context in the segment is calculated using inverse document frequency as weight, and reward scores are assigned to the segment based on the coverage ratio; The semantic relevance score, the penalty score, and the reward score are fused according to a preset weight to obtain the comprehensive semantic relevance score for each segment. Among all segments, the segment with the highest comprehensive semantic relevance score is selected, and its character-level start and end positions in the source fragment are extracted as the text range directly related to the answer.

[0014] In one feasible implementation, the second text localization scheme is used to locate the text range within the quoted fragment that is directly related to the answer, specifically including: The reference fragments in the valid reference list and the answer-side context are input into the preset algorithm corresponding to the second text location scheme; The answer-side context is used as the query semantics, and the quoted fragment is used as the candidate corpus; the query semantics and the candidate corpus are concatenated into an input sequence according to a preset format; The input sequence is encoded into a token sequence that can be processed by a large language model using a word segmenter; Mark the answer-side context portion of the token sequence as 0 and the reference fragment portion as 1 to construct a segment identifier; mark the real token in the token sequence as valid and the padding position as invalid to construct a valid bitmask; Assign a unique position index to each token in the token sequence to express the token's position information; establish a mapping relationship between each token and the original text character range; The encoded token sequence, the segment identifier, and the effective bitmask are input into the large language model to predict the start and end token positions of the highlighted range, respectively. The predicted start and end token positions are mapped to the character-level start and end intervals of the reference fragment, thus obtaining the text range in the reference fragment that is directly related to the answer.

[0015] On the other hand, embodiments of the present invention also provide a retrieval fragment citation annotation device based on a streaming large language model, the device comprising: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, enabling the at least one processor to execute the described method for retrieving fragment reference annotation based on a streaming large language model.

[0016] Finally, this embodiment of the invention also provides a storage medium, which is a non-volatile computer-readable storage medium storing at least one program, each program including instructions, which, when executed by a terminal, cause the terminal to execute the aforementioned method for retrieving fragment reference annotation based on a streaming large language model.

[0017] Compared with existing technologies, the retrieval fragment citation annotation method, device, and medium based on a streaming large language model provided in this invention have the following beneficial effects: 1. Significantly improved efficiency and stability of citation annotation: Through short-window buffering and a secure prefix confirmation mechanism, only fully received citation placeholders are parsed and replaced, avoiding interruptions to streaming output and ensuring that incremental text is output continuously in chronological order. This solves the high latency and misalignment problems caused by post-processing. A stable footnote number is assigned to each source based on the first occurrence of the search fragment, and the number is reused for multiple occurrences, ensuring consistent understanding of citations on both the reading and writing ends. Simultaneously, a structured citation list is built to unify citation standards between the front-end and back-end, facilitating traceability, reproduction, and compliance auditing, thereby improving system maintainability and reliability.

[0018] 2. Evidence Highlighting Balancing Accuracy, Efficiency, and Interpretability: This invention provides two complementary highlighting algorithm schemes, which can be flexibly switched according to business scenarios to meet different needs: Scheme 1 performs n-gram segmentation on the retrieval fragment, projects the segment and the answer-side context onto a unified semantic space to calculate similarity, and introduces length regularization and vocabulary coverage as auxiliary constraints to obtain a comprehensive score; the segment with the highest score is selected as the highlighting range. This scheme has low computational overhead and strong interpretability, and is suitable for high-concurrency scenarios with high response speed requirements. Scheme 2 uses the answer-side context as the query and the retrieval fragment as candidate corpus, constructs an input sequence containing segment identifiers and effective bitmasks, predicts the token-level start and end positions of the cited content through a large language model, and then restores it to the character-level interval through a position mapping table; when the relevance is insufficient, a threshold backoff is triggered, and the entire sentence or segment is output. This scheme has high accuracy and reproducible results, and is suitable for core business scenarios with high highlighting accuracy requirements.

[0019] 3. Improved overall system experience and usability: Through real-time citation annotation and precise evidence highlighting, users can quickly locate the source of evidence corresponding to the answer without secondary retrieval or manual verification, significantly improving trust and efficiency; engineers can quickly locate fault evidence through the structured citation list, shortening troubleshooting time and reducing maintenance costs; at the same time, the unified citation format and list support rapid evaluation after knowledge base updates, avoiding the risk of knowledge obsolescence and answer drift. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings: Figure 1 A flowchart of a retrieval fragment citation annotation method based on a streaming large language model is provided for an embodiment of the present invention; Figure 2 This is a schematic diagram of the structure of a retrieval fragment citation annotation device based on a streaming large language model, provided in an embodiment of the present invention. Detailed Implementation

[0021] To enable those skilled in the art to better understand the technical solutions of this invention, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this invention.

[0022] This invention provides a method for tagging retrieval fragment references based on a streaming large language model, such as... Figure 1 As shown, the retrieval fragment citation annotation method based on the streaming large language model specifically includes steps S101-S104: S101. During the streaming generation process of the large language model, obtain the streaming text generated step by step by the large language model and the retrieved reference fragments; wherein, the reference fragments contain the corresponding fragment identifiers.

[0023] Specifically, in scenarios where users use a large language model for streaming generation, the system reads the streaming text generated step by step during the streaming generation process of the large language model; wherein the streaming text contains at least unresolved reference placeholders.

[0024] Simultaneously, the source document data retrieved during the streaming generation process is read, and the referenced fragments within the source document data are identified. Fragment identifiers of the referenced fragments are extracted, and a fragment identifier index is constructed. Location metadata of the referenced fragments within the source document data, as well as the document identifier of the source document data, are extracted.

[0025] As a feasible implementation method, in streaming generation scenarios, reference placeholders (such as "[segment_id: idx1, idx2]") in the model output are identified in real time. Validity checks, deduplication, and stable number allocation are performed, and the original placeholders are replaced in place with a unified display format. Simultaneously, a reference list is built for front-end rendering and auditing. The placeholder syntax supports standard form: [doc:<doc_id> ; seg:<idx|idx1-idx2,...> (Including document identifiers and fragment identifiers) and compatible forms [segment_id:<idx,...> (This only includes fragment identifiers, which can be automatically completed with the corresponding document identifier doc_id based on the session context). The main text uses superscript footnotes (such as text). 1 ,text 1,4 The final output is a structured list of references, including footnote number stable_no, document identifier doc_id, reference placeholders segments, and UI positioning metadata.

[0026] S102. Construct a short window buffer pool to identify the source identifier of the streaming text in real time, and align the source identifier with the segment identifier corresponding to the quoted segment.

[0027] Specifically, a short window buffer pool is constructed based on a preset window length; the short window buffer pool receives the streaming text to obtain streaming text fragments.

[0028] Furthermore, the system identifies reference placeholders in the current streaming text segment within the short window buffer pool. When a reference placeholder is identified in the short window buffer pool, the source identifier corresponding to the reference placeholder is extracted, and the source identifier is aligned with the segment identifier index.

[0029] As a feasible implementation, a short-window buffer pool for streaming text is first maintained to receive newly arrived text and ensure that truncated placeholders can still be fully identified across segments. When candidate placeholders appear in the window, the system extracts their source identifiers and aligns them with the index established on the retrieval side, filtering out non-existent or invalid sources.

[0030] S103. Based on the aligned streaming text and cited fragments, generate incremental text and a list of valid citations.

[0031] Specifically, after successful alignment, a source feature is established based on the combination of document identifier and fragment identifier. Footnote numbers are assigned according to the chronological order of the first appearance of the source feature; when the same source appears multiple times, the assigned footnote numbers are reused.

[0032] Furthermore, footnote numbers within the same document are aggregated into compact superscript representations, and only the currently identified citation placeholders are replaced. Once a complete pair of citation placeholders is identified and both are replaced, the flowing text fragment between the complete citation placeholders is output as incremental text.

[0033] Furthermore, after the streaming session ends, the accumulated citation data during the streaming generation period is organized into a structured list and output externally, resulting in a valid citation list, which specifically includes: At the end of the streaming session, the last remaining segment of streaming text in the short window buffer is output. The accumulated reference segments during the streaming generation process, along with their corresponding document identifiers, segment identifiers, footnote numbers, and interval information used for front-end positioning, are compiled into a structured list and output externally, resulting in a valid reference list.

[0034] As a feasible implementation method, after successful alignment, a deduplication relationship is established based on the combination of "document identifier and fragment identifier," and a stable footnote number is assigned to each source according to the order of its first appearance. When the same source appears multiple times, the existing number is reused to maintain consistency in understanding between the reader and writer. Subsequently, the numbers involved in this instance under the same document are aggregated into a compact superscript representation, and only the currently hit placeholder is replaced to avoid premature processing of incomplete text. After converting the source placeholders into a unified and front-end recognizable display format, they are output segment by segment in chronological order as incremental text output, while the end of the window is retained to receive the next batch of streaming characters. At the end of the streaming session, the system outputs the remaining text and compiles the accumulated citations into a structured list for external output. If there are anomalies such as incomplete placeholders or invalid sources during the process, the process mainly ignores and delays the handling, without interrupting incremental output; when the proportion of anomalies is too high, only the text can be output temporarily and the anomalies can be recorded.

[0035] S104. Based on the cited fragments in the valid citation list and the answer-side context directly related to this citation, locate the text range directly related to the answer in the cited fragments and highlight the text range.

[0036] Specifically, a text localization scheme is determined based on the current business scenario; the text localization scheme includes a first text localization scheme and a second text localization scheme. If the current business scenario is a high-concurrency scenario, the first text localization scheme is used to locate the range of text in the referenced segment that is directly related to the answer; if the current business scenario is a high-precision scenario, the second text localization scheme is used to locate the range of text in the referenced segment that is directly related to the answer.

[0037] As a feasible implementation method, the first text localization scheme locates the text range in the quoted fragment that is directly related to the answer, specifically including: The reference fragments in the valid reference list and the answer-side context are input into the preset algorithm corresponding to the first text localization scheme. The segmentation step size is determined according to the text length of the answer-side context; based on the segmentation step size, the reference fragments are slidably segmented to obtain an n-gram segmentation set; wherein the n-gram segmentation set includes at least unary segments, binary segments, ternary segments and quaternary segments.

[0038] The segments in the n-gram segment set are projected onto the answer-side context into a unified semantic representation space, and the cosine similarity between the two is calculated to obtain the semantic relevance score of each segment. Segments with a length lower than a first preset threshold or a length higher than a second preset threshold are given a penalty score. The coverage ratio of the answer-side context in the segment is calculated using inverse document frequency as the weight, and a reward score is given to the segment based on the coverage ratio. The semantic relevance score, penalty score, and reward score are fused according to preset weights to obtain the comprehensive semantic relevance score of each segment.

[0039] Among all segments, the segment with the highest comprehensive semantic relevance score is selected, and its character-level start and end positions in the source fragment are extracted as the text range directly related to the answer.

[0040] In one embodiment, the first text localization scheme is a semantic similarity calculation method based on n-gram segmentation, suitable for high-concurrency scenarios and computationally sensitive environments, such as general question-answering systems. In this scheme, the source fragment is first segmented, with the segment size adaptively adjusted according to the length of the answer context to achieve a balance between recall coverage and computational cost. Subsequently, an n-gram segment set is formed from the source fragment, constructing unary, binary, ternary, and quaternary segments respectively. A binary segment represents the merging of two adjacent segments, a ternary segment represents the merging of three adjacent segments, and so on. Next, these n-gram segments and the answer-side context are projected together into a unified semantic representation space, and their semantic proximity is calculated to obtain a semantic relevance score for each segment. To avoid bias caused by extremely short or long segments, lightweight length regularization and lexical coverage are introduced as auxiliary constraints in addition to the semantic score, ensuring that high-scoring candidates are both semantically consistent and have good readability, better meeting business interpretation requirements. The overall score obtained in this way is comparable among different n-ary segments. The system ultimately selects the segment with the highest score from all candidates and uses its corresponding start and end positions as the preferred highlight range.

[0041] As another feasible implementation method, the second text location scheme locates the text range within the quoted fragment that is directly related to the answer, specifically including: The citation fragments from the valid citation list and the answer-side context are input into the preset model corresponding to the second text localization scheme. The answer-side context is used as the query semantics, and the citation fragments are used as candidate corpora. The query semantics and candidate corpora are concatenated into an input sequence according to a preset format. The input sequence is then encoded into a token sequence that can be processed by a large language model using a token segmenter.

[0042] Mark the answer-side context portion of the token sequence as 0 and the reference fragment portion as 1 to construct a segment identifier; mark the actual tokens in the token sequence as valid and the padding positions as invalid to construct a valid bitmask. Assign a unique position index to each token in the token sequence to express the token's position information; establish a mapping relationship between each token and the original text character range.

[0043] The encoded token sequence, segment identifier, and valid bitmask are input into the large language model to predict the start and end token positions of the highlighted range. These predicted start and end token positions are then mapped to character-level start and end intervals of the referenced segment, yielding the text range within the referenced segment directly related to the answer.

[0044] The input data for the preset model corresponding to the second text localization scheme are the answer-side context and the quoted fragment, and the output is the start and end positions in the quoted fragment.

[0045] The training method for this model is as follows: The data required for training the model consists of the answer-side context, the quoted fragments, and the start and end positions within the quoted fragments. The start and end positions within the quoted fragments in this dataset do not need to be manually labeled, but are handled by another large language model through system prompts, thereby achieving weakly supervised training.

[0046] In one embodiment, the second text localization scheme is an extractive question-answering model based on a large language model. It is suitable for core scenarios with high highlighting accuracy requirements, such as medical science popularization and legal interpretation. The core idea is to use extractive question-answering modeling based on a large language model, allowing the large language model to directly predict the start and end positions of text related to the answer in the source segment, achieving character-level precise localization. When the relevance is insufficient, a threshold backoff is triggered, outputting longer semantic units (such as entire sentences or paragraphs). Specifically, the answer-side context is first treated as query semantics, and the cited fragment is treated as candidate corpus, and the two are concatenated into an input sequence according to model conventions. Next, it is encoded into a token sequence that the model can process through a token segmenter. To enable the model to distinguish between question segments and corpus segments, segment identifiers need to be constructed, marking the context part as 0 and the source segment as 1. A valid bitmask also needs to be provided, marking the real token as valid and the padding position as invalid, to avoid the attention mechanism wasting computation on invalid positions.

[0047] During input preparation, a position index is required to represent the token's location information, and a mapping (offset_mapping) between the token and the original text character range needs to be established to accurately restore the prediction results to the original text. During the inference phase, the model outputs the predicted start and end positions within the source segment, and then uses offset_mapping to losslessly fill the prediction results back into the character start and end ranges of the source segment, which serves as the final highlighted range.

[0048] This invention addresses the output scenarios of streaming large language models by implementing real-time citation parsing and evidence highlighting during the generation process, significantly improving the overall system stability and user experience. Through short-window buffering and a "confirmation of security prefix" mechanism, only complete placeholders are replaced, ensuring uninterrupted output. Simultaneously, stable numbers are assigned to citation sources according to their first appearance sequence, generating a unified structured citation list. This ensures consistency between the front-end and back-end, facilitates tracing, reproduction, and auditing, and improves the system's maintainability and reliability.

[0049] In the evidence highlighting section, this invention provides two complementary solutions: a lightweight n-gram segmentation scheme that uses semantic similarity and term frequency-inverse document frequency coverage fusion scoring, offering interpretability and low computational overhead; and a scheme based on extractive question-answering modeling that uses start-point and end-point cross-entropy training, supplemented by perfect matching rate and token-level F1 evaluation, to output accurate and reproducible character-level positions, and supports threshold fallback to the sentence level, thus balancing accuracy and readability. The two schemes can be flexibly switched according to business scenarios, meeting the performance requirements of high-concurrency scenarios while ensuring the high accuracy and consistency requirements of core business operations.

[0050] In addition, embodiments of the present invention also provide a retrieval fragment citation annotation device based on a streaming large language model, such as... Figure 2 As shown, the retrieval fragment citation annotation device based on the streaming large language model specifically includes: At least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by at least one processor, so that at least one processor can perform the following: During the streaming generation process of the large language model, the streaming text generated step by step by the large language model and the retrieved reference fragments are obtained; wherein, the reference fragments contain corresponding fragment identifiers; Construct a short window buffer pool, identify the source identifier of the streaming text in real time, and align the source identifier with the segment identifier corresponding to the referenced segment; Based on the aligned streaming text and cited fragments, an incremental body text and a list of valid citations are generated. Based on the cited fragments in the valid citation list and the answer-side context directly related to this citation, the text range directly related to the answer in the cited fragment is located and highlighted.

[0051] Finally, the present invention also provides a storage medium, which is a non-volatile computer-readable storage medium storing at least one program, each program including instructions that, when executed by a terminal, cause the terminal to perform: During the streaming generation process of the large language model, the streaming text generated step by step by the large language model and the retrieved reference fragments are obtained; wherein, the reference fragments contain corresponding fragment identifiers; Construct a short window buffer pool, identify the source identifier of the streaming text in real time, and align the source identifier with the segment identifier corresponding to the referenced segment; Based on the aligned streaming text and cited fragments, an incremental body text and a list of valid citations are generated. Based on the cited fragments in the valid citation list and the answer-side context directly related to this citation, the text range directly related to the answer in the cited fragment is located and highlighted.

[0052] The various embodiments in this invention are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments of apparatus, devices, and non-volatile computer storage media are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0053] The foregoing has described specific embodiments of the present invention. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired results. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0054] The above description is merely an embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, various modifications and variations can be made to the embodiments of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of the embodiments of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for tagging citations of retrieved fragments based on a streaming large language model, characterized in that, The method includes: During the streaming generation process of the large language model, the streaming text generated step by step by the large language model and the retrieved reference fragments are obtained; wherein, the reference fragments contain corresponding fragment identifiers; Construct a short window buffer pool, identify the source identifier of the streaming text in real time, and align the source identifier with the segment identifier corresponding to the referenced segment; Based on the aligned streaming text and cited fragments, an incremental body text and a list of valid citations are generated. Based on the cited fragments in the valid citation list and the answer-side context directly related to this citation, the text range directly related to the answer in the cited fragment is located and highlighted.

2. The retrieval fragment citation annotation method based on a streaming large language model according to claim 1, characterized in that, During the streaming generation process of the large language model, the streaming text generated step by step by the large language model and the retrieved citation fragments are obtained, specifically including: In a scenario where a user uses a large language model for streaming generation, the streaming text generated step by step during the streaming generation process of the large language model is read; wherein, the streaming text contains at least unresolved reference placeholders; Simultaneously, the source document data retrieved during the streaming generation process is read, and the referenced fragments in the source document data are identified; Extract the fragment identifier of the referenced fragment and construct a fragment identifier index; Extract the location metadata of the referenced fragment in the source document data and the document identifier of the source document data.

3. The retrieval fragment citation annotation method based on a streaming large language model according to claim 2, characterized in that, Construct a short-window buffer pool, identify the source identifier of the streaming text in real time, and align the source identifier with the segment identifier corresponding to the referenced segment. Specifically, this includes: Based on a preset window length, a short window buffer pool is constructed; the streaming text is received through the short window buffer pool to obtain streaming text fragments; Identify reference placeholders in the current streaming text segment within the short window buffer pool; When a reference placeholder is detected in the short window buffer pool, the source identifier corresponding to the reference placeholder is extracted, and the source identifier is aligned with the fragment identifier index.

4. The retrieval fragment citation annotation method based on a streaming large language model according to claim 3, characterized in that, Based on the aligned streaming text and cited fragments, an incremental body text and a list of valid citations are generated, specifically including: After successful alignment, source features are established based on the combination of the document identifier and the fragment identifier; According to the order in which the source feature first appears, assign corresponding footnote numbers. When the same source appears multiple times, reuse the already assigned footnote numbers. The footnote numbers involved in the same document are aggregated into a compact superscript representation, and only the currently identified reference placeholders are replaced; Once a complete pair of reference placeholders is identified and both are replaced, the streaming text fragment between the complete reference placeholders is output as incremental text. After the streaming session ends, the accumulated reference data during the streaming generation is organized into a structured list and output externally, thus obtaining the effective reference list.

5. The retrieval fragment citation annotation method based on a streaming large language model according to claim 4, characterized in that, After the streaming session ends, the accumulated reference data during the streaming generation is organized into a structured list and output externally, resulting in the effective reference list, which specifically includes: At the end of the streaming session, the last remaining segment of streaming text in the short window buffer is output. The accumulated reference fragments during the streaming generation process, along with their corresponding document identifiers, fragment identifiers, footnote numbers, and interval information used for front-end positioning, are compiled into a structured list and output externally, resulting in the effective reference list.

6. The retrieval fragment citation annotation method based on a streaming large language model according to claim 1, characterized in that, Based on the cited fragments in the valid citation list and the answer-side context directly related to this citation, the text range directly related to the answer within the cited fragments is located, specifically including: Based on the current business scenario, a text location scheme is determined; wherein, the text location scheme includes a first text location scheme and a second text location scheme; If the current business scenario is a high-concurrency scenario, then the first text location scheme is used to locate the range of text in the quoted fragment that is directly related to the answer; If the current business scenario is a high-precision scenario, then the second text positioning scheme is used to locate the range of text in the quoted fragment that is directly related to the answer.

7. The retrieval fragment citation annotation method based on a streaming large language model according to claim 6, characterized in that, Using the first text localization scheme, the text range directly related to the answer within the quoted fragment is located, specifically including: The reference fragments in the valid reference list and the answer-side context are input into the preset algorithm corresponding to the first text location scheme; The segmentation step size is determined based on the text length of the context surrounding the answer. Based on the segmentation step size, the reference fragment is slid-segmented to obtain an n-ary segmentation set; wherein the n-ary segmentation set includes at least unary segments, binary segments, ternary segments, and quaternary segments; The segments in the n-gram segment set are projected onto the answer-side context into a unified semantic representation space, and the cosine similarity between the two is calculated to obtain the semantic relevance score of each segment. Penalty scores are assigned to segments whose length is lower than a first preset threshold or whose length is higher than a second preset threshold; the coverage ratio of the answer-side context in the segment is calculated using inverse document frequency as weight, and reward scores are assigned to the segment based on the coverage ratio; The semantic relevance score, the penalty score, and the reward score are fused according to a preset weight to obtain the comprehensive semantic relevance score for each segment. Among all segments, the segment with the highest comprehensive semantic relevance score is selected, and its character-level start and end positions in the source fragment are extracted as the text range directly related to the answer.

8. The retrieval fragment citation annotation method based on a streaming large language model according to claim 6, characterized in that, Using the second text localization scheme, the text range directly related to the answer within the quoted fragment is located, specifically including: The reference fragments in the valid reference list and the answer-side context are input into the preset algorithm corresponding to the second text location scheme; The answer-side context is used as the query semantics, and the quoted fragment is used as the candidate corpus; the query semantics and the candidate corpus are concatenated into an input sequence according to a preset format; The input sequence is encoded into a token sequence that can be processed by a large language model using a word segmenter; Mark the answer-side context portion of the token sequence as 0 and the reference fragment portion as 1 to construct a segment identifier; mark the real token in the token sequence as valid and the padding position as invalid to construct a valid bitmask; Assign a unique position index to each token in the token sequence to express the token's position information; establish a mapping relationship between each token and the original text character range; The encoded token sequence, the segment identifier, and the effective bitmask are input into the large language model to predict the start and end token positions of the highlighted range, respectively. The predicted start and end token positions are mapped to the character-level start and end intervals of the reference fragment, thus obtaining the text range in the reference fragment that is directly related to the answer.

9. A retrieval fragment citation annotation device based on a streaming large language model, characterized in that, The device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, enabling the at least one processor to perform a retrieval fragment reference annotation method based on a streaming large language model according to any one of claims 1-8.

10. A storage medium, characterized in that, The storage medium is a non-volatile computer-readable storage medium that stores at least one program, each program including instructions that, when executed by a terminal, cause the terminal to perform a retrieval fragment reference annotation method based on a streaming large language model according to any one of claims 1-8.

Images