Answer information generation method, question and answer system, and electronic device

Abstract

The present disclosure provides a reply information generation method, a question and answer system and an electronic device, and relates to the technical field of artificial intelligence. The method comprises the following steps: determining a current search result in a preset knowledge base according to a search generation request and a generated token; generating a first candidate token according to the generated token and the search generation request, generating a second candidate token according to the generated token, the search generation request and the current search result, forming first candidate content according to the first candidate token and the generated token, and forming second candidate content according to the second candidate token and the generated token; determining a first semantic similarity between the first candidate content and the second candidate content and a second semantic similarity between the current search result and the second candidate content; determining a target token in the first candidate token and the second candidate token according to the first semantic similarity and the second semantic similarity, and setting the target token as the generated token. The embodiment of the present disclosure can improve the accuracy of the content generated by search enhancement.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of artificial intelligence, in particular to a reply information generation method, a question and answer system and an electronic device. BACKGROUND

[0002] Retrieval-Augmented Generation (RAG) technology is a text generation technology that combines external retrieved relevant information and language model knowledge to enhance the accuracy and richness of generated content. Because it can supplement external knowledge for large language models and alleviate the problems of outdated knowledge and hallucination, it has been widely used. However, the existing retrieval augmented generation technology may contain noise, irrelevant information or incorrect information in the retrieval results, which may mislead the generated text, cause logical contradictions or redundancy, and the quality and accuracy of the generated text need to be further improved.

[0003] It should be noted that the information disclosed in the above background section is only used to strengthen the understanding of the background of the present disclosure, and therefore can include information that does not constitute prior art known to those of ordinary skill in the art. SUMMARY

[0004] The purpose of the present disclosure is to provide a reply information generation method, a question and answer system and an electronic device for improving the accuracy of generated information in retrieval augmented generation technology.

[0005] According to a first aspect of an embodiment of the present disclosure, a reply information generation method is provided, comprising: determining a current retrieval result in a preset knowledge base according to a retrieval generation request and a generated token, the initial value of the number of generated tokens being zero; generating a first candidate token according to the generated token and the retrieval generation request, generating a second candidate token according to the generated token, the retrieval generation request and the current retrieval result, forming a first candidate content according to the first candidate token and the generated token, and forming a second candidate content according to the second candidate token and the generated token; determining a first semantic similarity between the first candidate content and the second candidate content, and a second semantic similarity between the current retrieval result and the second candidate content; determining the target token in the first candidate token and the second candidate token according to the first semantic similarity and the second semantic similarity, and setting the target token as a generated token.

[0006] According to a second aspect of the embodiments of the present disclosure, a question and answer system is provided for performing the answer information generation method according to any one of the preceding embodiments, comprising: a preset knowledge base configured to store semantic blocks and a semantic block association graph; a graph retrieval module configured to perform graph retrieval on the preset knowledge base according to a retrieval generation request and a generated token to obtain a current retrieval result; a large model configured to generate a first candidate token according to the retrieval generation request and the generated token, and generate a second candidate token according to the current retrieval result, the retrieval generation request and the generated token; and a collaborative screening module configured to screen a target token from the first candidate token and the second candidate token as a next generated token.

[0007] According to a third aspect of the embodiments of the present disclosure, an electronic device is provided, comprising: a memory; and a processor coupled to the memory, the processor being configured to perform the method according to any one of the preceding embodiments based on instructions stored in the memory.

[0008] According to a fourth aspect of the embodiments of the present disclosure, a computer readable storage medium is provided, having a program stored thereon, the program being executed by a processor to implement the answer information generation method according to any one of the preceding embodiments.

[0009] According to a fifth aspect of the embodiments of the present disclosure, a computer program product is provided, comprising a computer program, wherein the computer program is executed by a processor to implement the steps of the method according to any one of the preceding embodiments.

[0010] The embodiments of the present disclosure can supplement external knowledge by obtaining a retrieval result in response to a retrieval generation request, generate a first candidate token based on the knowledge of the model itself and a second candidate token fused with the retrieval result in parallel, dynamically select a target token according to the similarity of the two, and iteratively update the generated token, thereby enhancing the content richness by utilizing external knowledge, reducing the impact of false information by comparing the two tokens, ensuring the coherence of the generated content and the context, and improving the accuracy and reliability of the generated result.

[0011] It should be understood that the foregoing general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0012] The accompanying drawings, which are incorporated into and form part of the specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure. Obviously, the drawings in the following description are only some embodiments of the present disclosure, and other drawings can be obtained by those skilled in the art without creative labor based on these drawings.

[0013] Figure 1is a flowchart of a reply information generation method in the example embodiment of the present disclosure.

[0014] Figure 2 is a flowchart of a formation process of a preset knowledge base in the example embodiment of the present disclosure.

[0015] Figure 3 is a sub-flowchart of step S11 in the example embodiment of the present disclosure.

[0016] Figure 4 is a sub-flowchart of step S14 in the example embodiment of the present disclosure.

[0017] Figure 5 is a schematic diagram of one retrieval augmentation generation process in the example embodiment of the present disclosure.

[0018] Figure 6 is a schematic diagram of a question answering system in the example embodiment of the present disclosure.

[0019] Figure 7 is a block diagram of an electronic device in the example embodiment of the present disclosure. DETAILED DESCRIPTION

[0020] Example embodiments now will be described more fully hereinafter with reference to the accompanying drawings. Example embodiments, however, can be implemented in many different forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the example embodiments to those skilled in the art. Like reference numerals refer to like elements throughout the description. One or more specific embodiments of the present disclosure will be described below.

[0021] In addition, the drawings provided herein are diagrammatic in nature and comprise various optional steps utilized in the example embodiments of the present disclosure, and thus the specific order or hierarchy of steps shown in the figures should not be taken as representative of the specific steps of the example embodiments as being strictly required. It should be recognized that some of the steps could in fact be performed in a different order, concurrently, or omitted altogether. Additionally, various elements of the example embodiments described herein can be implemented in hardware, software, firmware, or a combination thereof. Further, the example embodiments described herein can be implemented as a system, method, apparatus or article of manufacture using standard devices.

[0022] The example embodiments of the present disclosure will now be described in detail below with reference to the accompanying drawings.

[0023] The example embodiments of the present disclosure will now be described in detail below with reference to the accompanying drawings.Figure 1 is a flowchart of a reply information generation method in an example embodiment of the present disclosure.

[0024] Reference Figure 1 The reply information generation method 100 can include:

[0025] Step S11, determining a current search result in a preset knowledge base according to a search generation request and a generated token, the initial value of the number of the generated token being zero;

[0026] Step S12, generating a first candidate token according to the generated token and the search generation request, generating a second candidate token according to the generated token, the search generation request and the current search result, forming a first candidate content according to the first candidate token and the generated token, and forming a second candidate content according to the second candidate token and the generated token;

[0027] Step S13, determining a first semantic similarity between the first candidate content and the second candidate content, and a second semantic similarity between the current search result and the second candidate content;

[0028] Step S14, determining the target token in the first candidate token and the second candidate token according to the first semantic similarity and the second semantic similarity, and setting the target token as the generated token;

[0029] Step S15, repeating the above process until the generated token meets the complete semantic requirements of the search generation request.

[0030] The embodiment of the present disclosure can obtain search results in response to a search generation request to supplement external knowledge, generate a first candidate token based on the model's own knowledge and a second candidate token fused with the search results in parallel, dynamically select a target token according to the similarity of the two, and iteratively update the generated token. This can enhance the richness of the generated content while reducing the impact of false information through double token comparison and selection, ensuring the coherence of the generated content and the context, and improving the accuracy and reliability of the generated results.

[0031] The method of the embodiment of the present disclosure can be implemented by a large language model (LLM), or more specifically, by a retrieval augmented generation model or a question and answer system. The question and answer system can include a large language model, a preset knowledge base and other parts for executing the method 100 of the embodiment of the present disclosure.

[0032] Next, each step of the reply information generation method 100 will be described in detail.

[0033] At step S11, the current retrieval result is determined in the preset knowledge base according to the retrieval generation request and the generated token, and the initial value of the number of the generated token is zero.

[0034] The retrieval generation request (RAGR), which can also be referred to as a user query "q", refers to the original demand text input by the user to trigger the retrieval augmented generation process, such as "explain the basic principles of quantum computing" and "summarize the important breakthroughs in the field of artificial intelligence in 2023", etc., which functions to clarify the generation target and the retrieval direction, and provides a basis for subsequent search for relevant information in the preset knowledge base.

[0035] The generated token refers to the text segment (in the basic unit of token, which can be understood as a word, a character or a subword) that has been output in the retrieval augmented generation process, and the initial number is zero. With the advancement of the generation process, the target token determined each time will be added to the generated token to form a coherent context sequence. For example, after generating "quantum computing is based on the principles of quantum mechanics", "quantum computing", "based on", "quantum mechanics" and "principles" are all generated tokens, which are used to provide text context constraints for the next round of token generation.

[0036] The preset knowledge base refers to a structured, semi-structured or unstructured knowledge collection that is pre-constructed or accessed, which contains various information related to the potential retrieval generation request, such as academic paper library, encyclopedia, industry report, text database, etc., which functions as an external knowledge source to provide reference factual and knowledge content for the retrieval augmented generation process to make up for the limitations of the language model's own knowledge.

[0037] The current retrieval result refers to the information set related to the current generation task retrieved from the preset knowledge base based on the retrieval generation request and the generated token at step S11. In the embodiments of the present disclosure, the current retrieval result contains multiple semantic blocks (such as sentences, paragraphs or knowledge point units), for example, for the request "basic principles of quantum computing", the retrieval result may contain semantic blocks of related content such as "quantum superposition state" and "quantum entanglement".

[0038] In the embodiments of the present disclosure, in order to overcome the problem that the retrieval content has a negative impact on the generation result in related retrieval augmented generation technologies, the following four aspects are mainly considered: optimizing the construction of the preset knowledge base, optimizing the retrieval method, optimizing the influence of the retrieval content on the generated token, and utilizing the original knowledge of the large model to generate tokens in parallel and perform token screening.

[0039] Among them, the construction of the optimized preset knowledge base reduces the inclusion of noise and low-quality information by structuring the organization of knowledge, which is the basis for improving the quality of retrieval content from the source.

[0040] Optimizing the retrieval method, through methods such as lexical-semantic fusion retrieval, multi-hop sub-graph mining using graph convolution network, etc., can accurately locate the knowledge block highly related to user demand, reduce irrelevant or low-correlation information retrieval recall, and reduce the risk of noise introduction from the process level, which is an important means to improve the accuracy of target information acquisition.

[0041] Using the original knowledge of the large model to generate tokens in parallel and selecting tokens, through double-path token comparison and selection, when the retrieved information is suspicious, relying on reliable model knowledge first to avoid error transmission is the main way to ensure the reliability of the generated results.

[0042] In order to better understand the subsequent steps, first introduce the construction of the optimized preset knowledge base of the embodiments of the present disclosure.

[0043] Figure 2 is a flowchart of the formation process of the preset knowledge base in the exemplary embodiments of the present disclosure.

[0044] Reference Figure 2 In exemplary embodiments, the formation process of the preset knowledge base can include:

[0045] Step S21, obtaining an original document, and forming reference information after data cleaning of the original document;

[0046] Step S22, decomposing the reference information into a plurality of semantic blocks according to semantics;

[0047] Step S23, forming a semantic block association graph according to the association relationship of the semantic blocks within the original document and the reference relationship between the original documents, wherein the nodes of the semantic block association graph are the semantic blocks, and the attributes of the nodes are the contents of the semantic blocks; the edges of the semantic block association graph are formed according to the association relationship between the semantic blocks, and the weights of the edges are determined according to the association strength between the semantic blocks.

[0048] In step S21, the original document refers to the initial information carrier for constructing the preset knowledge base, including but not limited to academic papers, book chapters, news reports, industry reports, web texts, etc. Various types of text materials containing factual knowledge, logical reasoning or opinion discussion. These documents may contain redundant content, format errors, duplicate information or irrelevant noise when not processed, and are the basis for subsequent data cleaning and decomposition into semantic blocks. The quality directly affects the effectiveness of subsequent knowledge base construction.

[0049] The data cleaning of the original document includes removing redundant content, repeated information, irrelevant symbols and noise data (such as format error characters, meaningless garbled code, etc.) in the original document, ensuring that the retained reference information has basic accuracy and integrity, providing high-quality basic data for subsequent semantic block decomposition, and avoiding the negative impact of low-quality original information on knowledge base construction.

[0050] In the process of constructing and using the preset knowledge base, the original documents can be continuously collected, and the collected original documents can be classified according to Figure 2 The operation processing shown in the figure is used to add a new semantic block; at the same time, the semantic blocks formed in the preset knowledge base are updated (such as adjusting the edge weight or hierarchical label of the association graph) in combination with the association relationship of the added semantic block, so as to dynamically optimize the knowledge base structure.

[0051] In the embodiments of the present disclosure, the cleaned original document is referred to as reference information.

[0052] In step S22, the reference information can be split into the smallest unit (i.e. semantic block) with independent semantics based on the semantic logic of the reference information (such as the subject-predicate-object structure of the sentence, the core point of the paragraph, the independent expression of the knowledge point, etc.), each semantic block can correspond to a complete concept, fact, point of view, reasoning step or argument unit, so that the reference information is converted from continuous text to a semantic unit that can be processed independently, laying a foundation for constructing the association graph.

[0053] In the exemplary embodiments, the length of the semantic block can be determined based on a unified division standard (such as a fixed token number interval, a sentence group or a paragraph unit) to balance the needs of structured processing and semantic coherence, and to avoid semantic fragmentation caused by forced cutting.

[0054] In step S23, first, the semantic blocks are used to form nodes of the semantic block association graph, wherein each node corresponds to a semantic block, and the attribute of the node is the content of the semantic block.

[0055] Then, according to the association relationship between the semantic blocks, the edges between the nodes are formed, thereby forming the semantic block association graph.

[0056] The embodiments of the present disclosure divide the association relationship between the semantic blocks into two categories: intra-document association relationship and inter-document association relationship.

[0057] The intra-document association relationship refers to a logical relationship between different semantic blocks in a same original document, including but not limited to: a total-part relationship (such as an association between a general semantic block and a specific explanatory semantic block), a premise-conclusion relationship (such as an association between a semantic block serving as a reasoning basis and a conclusion semantic block derived), a cause-effect relationship (such as an association between a semantic block describing a cause and a semantic block describing a result), a parallel relationship (such as an association between different sub-viewpoint semantic blocks under a same topic), and the like, which can be determined according to a context structure (such as a chapter order, a logical relationship between sentences) of the original document.

[0058] The inter-document association relationship refers to a reference or derivation relationship between semantic blocks in different original documents, including but not limited to: a direct reference relationship (such as a semantic block of document B explicitly referencing content of a semantic block of document A or directly referencing document A), a supplementary extension relationship (such as a semantic block of document B supplementing details or extending a range based on a semantic block of document A), a comparison and refutation relationship (such as a semantic block of document B holding a different viewpoint from a semantic block of document A and forming a comparison), and the like, which can be determined according to a reference identifier, a content association degree, and the like in the original document.

[0059] According to the above two types of association relationships, it is determined whether there is an association relationship between two semantic blocks, and when there is an association relationship, an edge is added between nodes corresponding to the two semantic blocks. Next, the weight of the edge between the two nodes is determined according to the strength of the association relationship.

[0060] In an exemplary embodiment, the edge weight between the two nodes can be determined according to the morphology and semantics.

[0061] For example, a bag-of-words model representation can be first performed on text contents of the two semantic blocks by using a term frequency-inverse document frequency (TF-IDF) or the like morphological feature calculation method, a term overlap degree and an importance weight are counted, and a morphological layer association score is obtained, which quantitatively reflects a matching degree of the semantic blocks at a vocabulary table level.

[0062] Then, a pre-trained language model (such as BERT) is used to encode the texts of the two semantic blocks, to generate dense semantic vectors, and to calculate a semantic closeness in a vector space by using a cosine similarity or the like measurement method, to obtain a semantic layer association score, which quantitatively reflects an association degree of the semantic blocks at a deep semantic connotation.

[0063] Finally, the morphological layer association score and the semantic layer association score are linearly weighted and fused to calculate an association strength between the two semantic blocks, wherein the weights corresponding to the two association scores can be pre-set according to an application scenario, or determined through machine learning training.

[0064] The process of setting edge weights according to the correlation strength can involve normalization, that is, after calculating the correlation strength of different edges, the correlation strength of each edge can be mapped into a unified numerical interval (such as [0, 1]). For example, using the maximum-minimum normalization method, the correlation strength of all edges is subtracted from the minimum value and divided by the difference between the maximum value and the minimum value, so that the weight of each edge is in the same order of magnitude, avoiding the over-amplification or weakening of the weight of some edges in the graph calculation due to the too large difference in the original correlation strength numerical range. After normalization, the edge weight can not only retain the relative difference of the correlation strength between different semantic blocks, but also ensure that in subsequent graph convolution network calculation, multi-hop subgraph retrieval and other links, the weight of each edge can be fairly and effectively involved in the operation, further improving the reliability of the semantic block correlation graph in knowledge retrieval and feature fusion.

[0065] By constructing the weight of the edge between the nodes corresponding to the two semantic blocks in the semantic block correlation graph according to the correlation strength between the two semantic blocks, a structured basis can be provided for subsequent correlation graph-based retrieval and feature extraction.

[0066] Finally, the constructed semantic block correlation graph can be represented as:

[0067] (1)

[0068] wherein, is a semantic block node set, is an edge set, and C is a semantic block text content set.

[0069] After constructing the semantic block correlation graph, if the graph is directly used for retrieval or generation, it can only rely on the surface text matching and single correlation weight of the semantic block, which can easily lead to the problem of missing detection (such as ignoring important semantic blocks with indirect correlation) or false detection (such as over-reliance on lexical matching and ignoring deep semantic correlation) in complex knowledge scenarios.

[0070] Therefore, in the embodiments of the present disclosure, in step S11, a lexical-semantic graph retrieval algorithm is set to retrieve a multi-hop subgraph related to the retrieval generation request: this process extracts a context set containing key semantic blocks and their neighborhoods to capture the structured correlation (such as citation chains, semantic dependency relationships) between documents; at the same time, by integrating sparse lexical matching (such as TF-IDF) and dense semantic embedding (such as BERT) through a graph convolution network, the ability to jointly utilize structural information and semantic information in retrieval is enhanced.

[0071] Figure 3 is a sub-process flowchart of step S11 in the exemplary embodiments of the present disclosure.

[0072] Reference Figure 3 In exemplary embodiments, step S11 can include:

[0073] Step S111: Determine multiple first semantic blocks based on the lexical relevance and semantic relevance between the retrieval generation request and the generated tokens and semantic blocks;

[0074] Step S112: Based on the edges and weights of the first semantic block in the semantic block association graph, search for the second semantic block that has a direct association or a multi-hop indirect association with the first semantic block along the high-weight edges.

[0075] Step S113: Determine the hierarchical information of the first semantic block and the second semantic block through a graph convolutional network;

[0076] Step S114: Based on the hierarchical information, determine the target semantic block in the first semantic block and the second semantic block that corresponds to the semantics of the retrieval generation request and the generated token, and determine the target semantic block as the current retrieval result.

[0077] exist Figure 3 In the embodiment shown, in step S111, the retrieval generation request (i.e., user query q) and the generated token are first segmented to obtain the vocabulary units of both; then, a joint word set is constructed based on these vocabulary units, and the lexical relevance with each semantic block is calculated based on this.

[0078] In this embodiment of the disclosure, the lexical relevance between a semantic block and the joint word set can be determined using a sparse lexical matching method (such as TF-IDF). For example, a global vocabulary can be generated using all semantic block texts from a preset knowledge base as a corpus. The joint word set containing the retrieval request and the generated tokens can be mapped to this vocabulary. The term frequency (TF) and inverse document frequency (IDF) of each word can be calculated to construct the TF-IDF vector of the user input. Similarly, the text of each semantic block can be converted into a corresponding TF-IDF vector. By calculating the cosine similarity between the user input vector and the semantic block vector, the lexical relevance can be obtained, quantifying the degree of matching between the two at the vocabulary level.

[0079] Meanwhile, in order to capture deep semantic relationships, the joint text of the retrieval request and the generated token is input into a pre-trained language model (such as BERT) to obtain a dense vector that can represent its overall semantics. The text of each semantic block is also encoded in the same way to generate corresponding semantic vectors. By calculating the similarity between the two types of vectors, the semantic relevance is obtained, which reflects the degree of connection between the two in deep meaning, such as the semantic consistency in cases of synonym substitution, sentence transformation and other situations.

[0080] Finally, the comprehensive correlation score is obtained by combining the lexical correlation and semantic correlation, and a batch of semantic blocks with higher scores are selected as the first semantic blocks. Thus, in step S111, by setting the matching of the lexical and semantic double dimensions, the direct correlation between the initial candidate semantic blocks and the user demand in the vocabulary level is ensured, and important knowledge that is missed due to different expression methods is avoided through deep semantic matching, thereby laying a precise foundation for subsequent multi-hop retrieval.

[0081] Next, in step S112, the first semantic block determined in step S111 is taken as the starting point, and according to the edges and their weight information that have been constructed in the semantic block association graph, the expansion retrieval is preferentially performed along the edges with higher weights: for the semantic blocks that have direct connections (i.e., one-hop association) with the first semantic block, they are directly included in the candidate range; for the semantic blocks that need to be indirectly connected through other semantic blocks (i.e., two-hop or more association), they are mined by continuously tracking the path of high-weight edges, and finally the semantic blocks that have direct association or multi-hop indirect association with the first semantic block are selected as the second semantic blocks.

[0082] For example, when a first semantic block is connected to multiple semantic blocks through multiple edges respectively, the semantic blocks corresponding to the x connections with higher edge weights (x is a self-defined value) are preferentially selected and included in the initial candidate set of the second semantic blocks, and the remaining connections with lower weights are secondarily evaluated (such as supplemented screening in combination with semantic correlation) to determine whether the semantic blocks corresponding to the connections that pass the secondary evaluation (for example, the semantic correlation is greater than a preset threshold) are included in the initial candidate set of the second semantic blocks.

[0083] When a second semantic block included in the initial candidate set of the second semantic blocks is connected to multiple semantic blocks through multiple edges respectively, the second semantic block is taken as a new starting point, and multi-hop expansion is continued along high-weight edges to a deeper layer, while the expansion depth is limited (such as a maximum of three hops) to avoid excessive divergence of the retrieval range, and to ensure that the newly added indirectly associated semantic blocks still maintain theme consistency with the retrieval generation request.

[0084] Next, the second semantic blocks in the initial candidate set of the second semantic blocks are screened based on the semantic association strength with the retrieval generation request and the generated tokens, the hierarchical position (such as the core level or the support level) in the association graph, and the continuity of the multi-hop path, to determine the final second semantic blocks. Specifically, the semantic blocks that have high semantic matching degree with the retrieval generation request, are in the key logical level (such as the core view layer), and maintain coherent association with the first semantic block in the multi-hop path can be preferentially retained, and redundant semantic blocks that deviate from the semantics, are in the edge level, or have broken paths are removed, thereby ensuring that the final second semantic blocks are both rich and accurate, and providing a high-quality candidate set for subsequent hierarchical information determination.

[0085] Thus, by traversing the high-weight connections in the association graph, the search range is narrowed (reducing missed detection) and the relevance of the expanded semantic blocks to the core topic is ensured (reducing noise introduction), thereby capturing more structured association information.

[0086] In step S113, the hierarchical information of the first semantic block and the second semantic block is first determined by a graph convolution network. A graph convolution network can be pre-trained to perform this step, and then the trained, parameter-fixed graph convolution network is embedded into a model or a question-answering system implementing the answer information generation method of the embodiments of the present disclosure.

[0087] In the embodiments of the present disclosure, step S113 can include: determining, by the graph convolution network, a logical level of the first semantic block or the second semantic block according to a logical dependency relationship (such as a premise-conclusion, cause-effect, etc.) of the first semantic block or the second semantic block in the original document, in combination with an edge weight and a structural relationship in the semantic block association graph, and aggregating neighbor node features of the first semantic block or the second semantic block; determining a semantic level of the first semantic block or the second semantic block according to an abstract degree (such as an association between general and detailed content) of the first semantic block or the second semantic block, by distinguishing a high-level abstraction, a middle-level transition, and a low-level detail through multi-layer feature aggregation; determining a reference chain level of the first semantic block or the second semantic block according to a position of the first semantic block or the second semantic block in a cross-document reference chain and a weight of a reference edge (such as an association strength of a source and a derivative, an extension); and fusing and encoding information of the logical level, the semantic level, and the reference chain level by the graph convolution network to form the hierarchical information of the first semantic block or the second semantic block.

[0088] The logical level of the semantic block refers to a positional relationship of the semantic block in logical reasoning or narrative structure within the same original document. The logical level of the semantic block is determined according to a logical dependency relationship of the semantic block in the original document, for example: in an experimental paper, an “experimental hypothesis” semantic block is a logical premise layer, an “experimental design” semantic block is a logical execution layer, and an “experimental conclusion” semantic block is a logical result layer, which form a logical level chain of premise→execution→result.

[0089] The semantic hierarchy of a semantic block refers to the difference in the degree of abstraction of the semantic block, i.e., the semantic relationship of who is the core summary and who is the detail supplement. The semantic hierarchy of a semantic block is determined according to the degree of abstraction of the semantic block, for example: in the document about artificial intelligence, "artificial intelligence is a technology that simulates human intelligence" is a high-level abstract semantic block (core definition), "machine learning is a branch of artificial intelligence" is a middle-level transitional semantic block (branch description), and "parameter setting of a certain machine learning algorithm" is a low-level specific semantic block (detail description), which form a semantic hierarchy from abstract to specific.

[0090] The reference chain hierarchy of a semantic block refers to the position of the semantic block in the cross-document reference relationship, i.e., the reference logic between the source and the derivative. The reference chain hierarchy of a semantic block is determined according to the position of the semantic block in the cross-document reference chain, for example: the semantic block "definition of quantum entanglement" in document A is referenced by document B, document B extends the semantic block "experimental verification of quantum entanglement" based on this, and document C derives the semantic block "application scenarios of quantum entanglement" based on the semantic block of document B, then the semantic block "definition of quantum entanglement" is the reference chain source layer, the semantic block "experimental verification of quantum entanglement" is the reference chain derivative layer, and the semantic block "application scenarios of quantum entanglement" is the reference chain extension layer.

[0091] The hierarchical information of the first semantic block and the second semantic block is determined by the graph convolution network. First, the first semantic block and the second semantic block selected based on steps S111 and S112 and the corresponding edge weight of the first semantic block and the second semantic block in the semantic block feature map can be used as the initial input features of the graph convolution network with the original features of the first semantic block node and the second semantic block node (such as pre-processed text encoding, lexical-semantic matching score).

[0092] The trained graph convolution network can gradually aggregate the neighbor node information of each node (semantic block) through multi-layer iterative calculation, so as to capture the hierarchical features of the node in the overall association structure.

[0093] In an exemplary embodiment, the graph convolution network fuses sparse lexical matching (such as TF-IDF) and dense semantic embedding (such as BERT) to enhance the joint utilization of structural information and semantic information. The feature of the k+1 layer of the graph convolution network is exemplarily as follows:

[0094] (2)

[0095] wherein, is the lexical relevance (such as TF-IDF score) of the retrieval generation request to the semantic block; is the semantic relevance between semantic blocks, which is calculated by a multi-layer perceptron to calculate the feature difference, 、 is a text semantic block node; is the feature representation of the k-th layer semantic block The CU is a graph convolution update function, which performs a nonlinear transformation on the aggregated features in the graph convolution, and converts the aggregation results of the fused lexical and semantic information into feature representations suitable for subsequent processes.

[0096] For example, in the first layer of calculation, the graph convolutional network fuses the features of each semantic block with the features of its directly connected high-weight neighbors (such as the directly associated semantic blocks of the first semantic block, and the one-hop neighbors of the second semantic block), and preliminarily captures the direct support or direct dependency relationship, providing a basis for the preliminary division of logical levels and citation chain levels (for example, the "experimental design" semantic block directly connected with high weight to the "experimental hypothesis" semantic block will reflect the dependency on the former in the features).

[0097] As the network depth increases (such as the second layer, the third layer), the semantic block features will further aggregate the information of multi-hop neighbors, at this time the high-level semantic blocks (such as abstract and general semantic blocks) will present stronger generalization in the feature vector due to the aggregation of more low-level detailed semantic block features; while the low-level semantic blocks will retain more specific details in the features due to more dependence on the guidance of high-level semantic blocks, which provides a feature basis for the division of semantic levels.

[0098] It should be noted that in the training process of the graph convolutional network, the graph convolutional network needs to focus on the role of edge weight in the aggregation process, and give higher aggregation weight to the features of neighbor nodes connected by high-weight edges, so that the core association (such as direct citation relationship, strong logical dependency relationship) has a more significant impact on the hierarchical features. For example, in the citation chain, the derived layer semantic block connected to the source layer semantic block through the high-weight citation edge will more obviously retain the imprint of the source information in its features, so as to be identified as the next level of the citation chain.

[0099] After multi-layer aggregation, the output features of each semantic block formed by encoding through the graph convolutional network contain its hierarchical attributes in the dimensions of logical dependency, abstraction level, and citation chain position. By analyzing these features (such as extracting hierarchical identifiers through fully connected layers or attention mechanisms), the logical level (premise layer / execution layer / result layer), semantic level (high-level abstraction / medium-level transition / low-level detail), and citation chain level (source layer / derived layer / extended layer) of each semantic block can be determined, forming complete hierarchical information.

[0100] Based on this hierarchical information, the target semantic block is further screened in step S114: preferentially selecting semantic blocks that are in the core reasoning chain in the logical level, match the abstraction level of the retrieval generation request in the semantic level (such as selecting high-level semantic blocks if the retrieval generation request requires generalization answers), and are authoritative sources or effective derivatives in the citation chain level, and finally determining them as the current retrieval result.

[0101] In Figure 3 In the embodiment shown, by first mining a multi-hop subgraph (composed of a first semantic block and a second semantic block) that is relevant in both vocabulary and semantics to the retrieval generation request in the semantic block association graph, and then performing deep mining on the structured association (such as citation chains, semantic dependency relationships) of the multi-hop subgraph according to the key semantic blocks (the first semantic block and the second semantic block) and their neighborhood context set by the graph convolution network, it can be ensured that the retrieval results not only meet the semantic requirements, but also have logical coherence and knowledge reliability.

[0102] Therefore, Figure 3 The optimized retrieval method shown can enhance the joint utilization capability of the retrieval process for structural information and semantic information by using a graph convolution network (GCN) to fuse sparse lexical matching (such as TF-IDF) and dense semantic embedding (such as BERT) for lexical-semantic graph retrieval, accurately locate knowledge blocks highly relevant to user requirements, reduce retrieval recall of irrelevant or low-correlation information, reduce the risk of noise introduction from the process level, and solve the problems of missed detection and false detection in traditional retrieval in complex structured data.

[0103] In step S12, a first candidate token is generated according to the generated token and the retrieval generation request, a second candidate token is generated according to the generated token, the retrieval generation request, and the current retrieval result, a first candidate content is formed according to the first candidate token and the generated token, and a second candidate content is formed according to the second candidate token and the generated token.

[0104] To solve the problem that the retrieval result may contain noise or error information in the existing retrieval enhancement generation technology, leading to the generated content being misled and the accuracy being reduced, in the embodiment of the present disclosure, the token generated based on the current retrieval result is not directly output, but two candidate tokens are generated in parallel by a large model, and the token with the highest reliability is selected from the two candidate tokens after comparison.

[0105] Among the two candidate tokens generated in parallel, the first candidate token is a token generated by the large model based on the original knowledge only according to the input data (retrieval generation request) of the user and the generated token; the second candidate token is a token generated by the large model after referring to the current retrieval result. That is, the second candidate token is affected by the current retrieval result, and the first candidate token is not affected by the current retrieval result.

[0106] Then, a first candidate content is formed according to the first candidate token and the generated token, i.e., a token sequence not affected by the current search result at the latest step; and a second candidate content is generated according to the second candidate token and the generated token, i.e., a token sequence affected by the current search result at the latest step.

[0107] At step S13, a first semantic similarity of the first candidate content and the second candidate content is determined, and a second semantic similarity of the current search result and the second candidate content is determined.

[0108] In an exemplary embodiment, the first semantic similarity and the second semantic similarity are, for example, cosine similarities, and the first similarity can be expressed as, for example, and the second similarity can be expressed as, for example, . Wherein, is a weighted word embedding of the second candidate content distribution; is a weighted word embedding of the current search result distribution; is a weighted word embedding of the first candidate content distribution. are token probabilities of the corresponding distributions, respectively, is a word embedding vector of the token.

[0109] The first semantic similarity is used to measure the semantic consistency between the content generated by the large model itself (the first candidate content) and the content generated by fusing the search result (the second candidate content), so as to determine whether the information introduced by the search result deviates from the original knowledge of the model: if the first semantic similarity is high, it means that the search result does not significantly interfere with the generated content, and the model's own knowledge and the search information are consistent at the current step; if the first semantic similarity is low, it means that the search result brings new information or different information to the generated content, and the reliability of the search result needs to be further verified.

[0110] The second semantic similarity is used to evaluate the close relationship between the current search result and the second candidate content generated by fusing it, for judging whether the search result is effectively and reasonably utilized: if the second semantic similarity is high, it means that the second candidate content accurately absorbs the information in the search result, and the search result positively supports the generation process; if the second semantic similarity is low, there may be a problem that the search result does not match the generation target or the generation process does not effectively fuse the search information, i.e., there is a situation of misuse or ineffective introduction of the search result.

[0111] Through the similarity evaluation of these two dimensions, the nature of the influence of the search result on the generated content (consistent / different) and the effectiveness of the influence (related / irrelevant) can be comprehensively judged, providing a double basis for the selection of subsequent target tokens, and reducing the deviation of the generated content caused by the error or mismatch of the search result.

[0112] At step S14, the target token is determined in the first candidate token and the second candidate token according to the first semantic similarity and the second semantic similarity, and the target token is set as a generated token.

[0113] In an example embodiment, the first candidate token can be set as the target token when the first semantic similarity is greater than the second semantic similarity, and the second candidate token can be set as the target token when the first semantic similarity is less than or equal to the second semantic similarity.

[0114] When the first semantic similarity is greater than the second semantic similarity, it indicates that the content generated by the large model based on its own knowledge is more consistent with the content generated by the fusion retrieval result, and the association between the retrieval result and the content generated by the fusion retrieval result is relatively weak. This situation may be due to the fact that the retrieval result has noise, low association with the current generation target, or the introduced information conflicts with the inherent knowledge verified by the model. At this time, the first candidate token is preferred, which can reduce the interference of unreliable retrieval information on the generation process and ensure the coherence and accuracy of the content.

[0115] When the first semantic similarity is less than or equal to the second semantic similarity, it indicates that the content generated by the fusion retrieval result is more closely associated with the retrieval result, and the influence of the retrieval information on the generation process is more in line with expectations. This situation shows that the retrieval result effectively integrates into the generated content, and there is no significant conflict with the model knowledge (or the conflict degree is within an acceptable range). At this time, the second candidate token is selected, which can fully utilize the retrieved external information to enhance the richness and pertinence of the generated content, and realize the core value of retrieval enhancement.

[0116] In some embodiments of the present disclosure, it is also provided that the relationship between the current retrieval result and the inherent knowledge of the large model is first evaluated, and the relationship is used to determine how to select the target token.

[0117] Figure 4 FIG. 14 is a subflowchart of step S14 in an example embodiment of the present disclosure.

[0118] Reference Figure 4 In an example embodiment, step S14 can include:

[0119] Step S141, determining a first probability distribution corresponding to the first candidate token, a second probability distribution corresponding to the second candidate token, and a third probability distribution corresponding to the current retrieval result;

[0120] Step S142, calculating the KL divergence of the first probability distribution and the third probability distribution to determine the harmful value of the current retrieval result, and calculating the KL divergence of the second probability distribution and the third probability distribution to determine the beneficial value of the current retrieval result;

[0121] Step S143, determining a net influence value of the current search result on the second candidate token according to the difference between the beneficial value and the harmful value;

[0122] Step S144, determining a target token from the first candidate token and the second candidate token according to the net influence value, the first semantic similarity, and the second semantic similarity.

[0123] The beneficial value refers to a quantitative indicator of the amount of effective information in the current search result that is associated with the context knowledge and can provide supplementary information, enhance content richness, or improve accuracy for the generation process. The higher the value, the greater the positive contribution of the search result to the generation. The harmful value refers to a quantitative indicator of the amount of invalid or incorrect information in the current search result that conflicts with the context knowledge, may mislead the generation direction, or reduces the consistency of the content. The higher the value, the greater the negative impact of the search result on the generation.

[0124] For example, if the second candidate token generated according to the current search result is considered as a fusion of the context knowledge distribution p() from the LLM and the distribution P R of the current search, the prediction probability of the second candidate token can be represented as:

[0125] (3)

[0126] where R is the current search result, x r is the target token to be predicted, i is the sequence position index of the target token.

[0127] In step S141, the first probability distribution corresponding to the first candidate token generated based on only the context knowledge distribution of the LLM is determined as p(r∣z ); the second probability distribution corresponding to the second candidate token generated after the model fuses the current search result is determined as p(r∣z); and the third probability distribution corresponding to the current search result is determined as P R (r). Where r is the token to be predicted.

[0128] In steps S142 and S143, the net influence value v(z) of the current search result on the generation of the second candidate token is determined by calculating the KL divergence (Kullback-Leibler divergence) to decompose the benefits and risks as follows:

[0129] (4)

[0130] where, ​is the KL divergence of the third probability distribution and the second probability distribution, used to measure the difference between the probability distribution of the current retrieval result and the probability distribution of the second candidate token. The larger the value, the more the current retrieval result is the "out-of-domain knowledge" supplemented by the LLM, and the higher the positive gain to the generated content. Therefore, it is used to represent the beneficial value (benefit) of the current retrieval result.

[0131] is the KL divergence of the third probability distribution and the first probability distribution, used to measure the difference between the probability distribution of the current retrieval result and the probability distribution of the inherent knowledge of the LLM. The larger the value, the more obvious the conflict between the current retrieval result and the inherent knowledge of the LLM, and the higher the risk of misleading generation. Therefore, it is used to represent the harmful value (risk) of the current retrieval result.

[0132] Therefore, by constructing a distribution fusion model, the gain (such as out-of-domain knowledge supplement) and risk (such as noise misleading) of external knowledge are decomposed into calculable distribution differences (KL divergence) at the token level, solving the "black box" problem of traditional RAG technology, making the answer generation process interpretable and quantifiable.

[0133] In step S144, the net impact value can be used as the main factor, and the semantic comparison result can be used as verification reinforcement to determine the final target token.

[0134] For example, when the net impact value is a harmful value, the first candidate token can be set as the target token; when the net impact value is a beneficial value, and the first semantic similarity is less than or equal to the second semantic similarity, the second candidate token can be set as the target token; when the net impact value is a beneficial value, and the first semantic similarity is greater than the second semantic similarity, the first candidate token can be set as the target token.

[0135] In addition, the size of the net impact value and the semantic similarity situation can also be comprehensively judged.

[0136] For example, the first semantic similarity is greater than the second semantic similarity (according to the semantic similarity, the first candidate token should be selected), but the beneficial value of the net impact value is large (such as the beneficial value is 0.8, which is much higher than the preset threshold 0.5). At this time, although the first semantic similarity is higher, since the positive impact brought by the second candidate token is significant, the second candidate token can be selected preferentially to make full use of the high-value information of the retrieval result.

[0137] Or, the first semantic similarity is greater than the second semantic similarity (according to the semantic similarity, the first candidate token should be selected), and the net impact value is a beneficial value but small (such as the beneficial value is 0.3, which is lower than the general threshold 0.5). In this case, the weight of the semantic similarity is more prominent, and the first candidate token is still selected to avoid sacrificing the semantic coherence of the content due to the small beneficial impact.

[0138] If the first semantic similarity is less than or equal to the second semantic similarity (the second candidate token should be selected according to the semantic similarity), the net impact value is a harmful value but small (for example, the harmful value is 0.1, which is much lower than the risk threshold 0.3). Although there is a certain harmful effect, the degree is slight, and the semantic matching degree of the second candidate token with the related content is higher, so the second candidate token can be selected.

[0139] If the first semantic similarity is less than or equal to the second semantic similarity (the second candidate token should be selected according to the semantic similarity), the net impact value is a harmful value and large (for example, the harmful value is 0.6, which exceeds the risk threshold 0.3). At this time, the large harmful effect dominates, and even if the semantic similarity supports the second candidate token, the first candidate token should be selected to avoid risks.

[0140] If the first semantic similarity and the second semantic similarity are close (for example, 0.51 and 0.5 respectively), the net impact value is a beneficial value and large. Since the difference between the semantic similarities of the two is very small, and the beneficial effect of the second candidate token is significant, selecting the second candidate token is more beneficial to improve the quality of the generated content.

[0141] If the first semantic similarity and the second semantic similarity are close (for example, 0.5 and 0.49 respectively), the net impact value is a harmful value and small. Although the net impact value is harmful, the degree is low, and the second candidate token has a slight advantage in semantic similarity, so the second candidate token can be selected.

[0142] By comprehensively considering the size of the net impact value and the situation of the semantic similarity, the target token can be selected more flexibly to cope with different scenarios, so that the selection of the target token can take into account both semantic coherence and fully play the value or avoid the risk of the retrieval result.

[0143] In step S14, by setting a large model to generate two candidate tokens based on retrieval results and not based on retrieval results in parallel, and comparing and selecting the next token, the embodiment of the disclosure can dynamically balance the weight of the model's own knowledge and external retrieval information, maximize the retrieval enhancement effect while ensuring the reliability of generation, and avoid problems such as "retrieval information overload", "wrong knowledge transfer" or "model's own reliable knowledge is diluted" caused by directly using retrieval results in related technologies.

[0144] For example, when the retrieval results contain noise, outdated information, or irrelevant content, the anchoring effect of the first candidate token can prevent incorrect information from being directly incorporated into the generated sequence. When the retrieval results contain fresh knowledge or precise details that the model does not master, the enhancement effect of the second candidate token can allow external information to be efficiently integrated into the generation process. This dynamic adjustment mechanism not only preserves the stability of the large model's own knowledge, but also breaks through the boundaries of knowledge by selectively introducing retrieval information. Ultimately, it achieves a better balance between the accuracy, richness, and adaptability of generated content, especially in complex scenarios that require consideration of factual rigor and timeliness of knowledge (such as professional question answering and dynamic information aggregation).

[0145] At step S15, the above process is repeated until it is determined that the generated token meets the complete semantic requirements of the retrieval generation request.

[0146] On this basis, the final selected token sequence (i.e., the generated token) is fed back to the image retrieval step for iterative retrieval to supplement context information that better fits the current generation progress. This process forms a dynamic optimization closed loop of "retrieval → generation → re-retrieval," which adjusts the retrieval direction by real-time adaptation to the generation progress and selects a more reliable output path, significantly improving the accuracy and coherence of knowledge utilization in complex tasks - avoiding the problem of outdated or mismatched information from a single retrieval, and continuously calibrating the generation direction through iterative feedback, so that the model can always call the most suitable knowledge resources in long sequence generation.

[0147] Finally, when it is determined that the generated token sequence completely covers the core semantics of the retrieval generation request (e.g., contains all key information points, the logic chain is closed, and meets the user's requirements for output format or depth), the retrieval generation process is ended, the generated content corresponding to the retrieval generation request is formed, and the output is output.

[0148] In an exemplary embodiment, the generated token can also be output in real time as it is determined, to realize the streaming interaction effect of generating and outputting in parallel, reducing the user's waiting time for the full amount of results. At the same time, this token-by-token output method can also provide more timely feedback for subsequent iterative retrieval, such as adjusting the retrieval and generation strategy in the next round based on real-time user feedback (such as correction instructions and supplementary requirements), further improving the flexibility of interaction and the adaptability of results. By setting "closed-loop iteration + on-demand output," the embodiments of the present disclosure not only efficiently complete complex knowledge generation tasks, but also take into account the user's demand for response speed and result controllability, and are more adaptable to scenarios compared to the retrieval-enhanced generation scheme with a fixed process.

[0149] To sum up, in the retrieval enhancement generation process based on the graph structure, first, the multi-hop subgraph related to the user query is retrieved through the lexical-semantic graph retrieval algorithm. This process extracts the context set containing the key semantic blocks and their neighborhoods, thereby capturing the structured association between documents (such as citation chains, semantic dependency relationships). Subsequently, the retrieved semantic blocks are summarized and spliced with the original query q (retrieval generation request) to form an enhanced query q'. Finally, the enhanced query information is input into the large language model, and the token-level collaborative strategy is combined to dynamically select the optimal generation path at each token generation step, thereby realizing context-aware accurate answer generation. Through the closed loop of "subgraph retrieval → query enhancement → token-level generation", the answer accuracy and explainability of LLM in complex multi-hop reasoning tasks are effectively improved.

[0150] Figure 5 FIG. 1 is a schematic diagram of a retrieval enhancement generation process in an exemplary embodiment of the present disclosure.

[0151] Reference Figure 5 In step S500, a preset knowledge base is constructed and maintained daily. For example, original documents (such as various types of domain texts, knowledge base materials, etc.) can be collected, and the original documents are converted into semantic block feature graphs (for example, semantic fragments are represented by vectors, and knowledge graph association relationships are constructed) based on text semantic segmentation and feature extraction techniques, and stored in the preset knowledge base.

[0152] In step S501, the system receives a retrieval generation request triggered by a user / service (for example, the user inputs "text generation requirement for introducing the principle of quantum computing"), and obtains the generated token corresponding to the request (initially empty when first executed, and then the token sequence of the generated text fragment in subsequent iterative generation).

[0153] In step S502 (LLM independently generates a first candidate token), a large language model (LLM) is called to directly generate a first candidate token based on the retrieval generation request and the generated token. For example, the LLM generates an initial token fragment "quantum computing is a quantum bit of superposition state characteristics…" based on its own training knowledge, representing the content candidate independently generated by the model.

[0154] In step S503, the first candidate token is spliced with the generated token to form a first candidate content. If the generated token is empty (first generation), the first candidate content is the first candidate token itself; if it is in iterative generation (such as "core concept of quantum computing" has been generated), it is spliced into "core concept of quantum computing quantum computing is a quantum bit of superposition state characteristics…" for subsequent semantic comparison.

[0155] At step S504 (knowledge base retrieval current result), according to the retrieval generation request (such as "introduce the principle of quantum computing") and the generated token (as a retrieval context supplement, optional), perform retrieval in the preset knowledge base (S500 constructed semantic block feature map), match the knowledge fragments strongly related to the request, and obtain the current retrieval result. For example, the professional knowledge content "quantum bits can be in 0 and 1 state at the same time, which is due to the principle of quantum superposition, and is the basis of quantum computing parallelism..." is retrieved from the knowledge base, which is prepared for the fusion of external knowledge.

[0156] At step S505 (generate enhanced retrieval request), the current retrieval result is abstracted (such as extracting the key sentence "quantum bits rely on superposition state to realize parallel computing" by using a text summary model), and is spliced with the original retrieval generation request to form an enhanced retrieval generation request. The purpose is to make the subsequent generation integrate knowledge base information and strengthen the knowledge depth of the content. The spliced enhanced retrieval generation request is, for example, changed to "introduce the principle of quantum computing, combined with knowledge: quantum bits rely on superposition state to realize parallel computing".

[0157] At step S506 (fusion retrieval generates second candidate token), according to the enhanced retrieval generation request (new instruction integrated with knowledge base summary) and the generated token (maintaining context coherence), the model (LLM can be reused and combined with retrieval information guidance) is called again to generate a second candidate token. For example, the model generates the second candidate token "quantum computing through the superposition state of quantum bits..." in combination with retrieval knowledge.

[0158] At step S507 (construct second candidate content), the second candidate token is spliced with the generated token to form the second candidate content, which has the same structure as the first candidate content logic. If it is generated for the first time, the second candidate content is "quantum computing through the superposition state of quantum bits..."; if it is iteratively generated (such as "core concept of quantum computing"), it is spliced into "core concept of quantum computing quantum computing through the superposition state of quantum bits...". The semantic chain of "existing content + retrieval enhanced new segment" is constructed for subsequent multi-dimensional semantic comparison.

[0159] The steps S502-S503 and S504-S507 are executed in parallel.

[0160] At step S508 (calculate second semantic similarity), based on a text semantic matching algorithm (such as cosine similarity, Sentence-Bert vector distance), calculate the second semantic similarity between the second candidate content and the current retrieval result, and verify the "penetration degree" of the generated content. For example, if the second candidate content largely reuses the professional expressions of the retrieval result, the similarity may be 0.8, indicating that the retrieval knowledge is effectively integrated; if there is only a slight association, the similarity may be 0.3, and the knowledge fusion effect needs to be evaluated.

[0161] At step S509 (calculate the first semantic similarity), the first semantic similarity between the first candidate content (pure LLM generated chain) and the second candidate content (retrieval enhanced generated chain) is calculated using the semantic matching algorithm, measuring the semantic difference between the two generated paths. For example, if the first candidate content focuses on basic explanation of the principle, and the second candidate content incorporates professional terms, the similarity may be 0.6, indicating that retrieval enhancement brings content gain.

[0162] At step S510 (calculate the net impact value), after step S506, the net impact value of the current retrieval result is calculated according to the second candidate token according to the KL divergence calculation method introduced in the above embodiment. For example: if the second candidate token introduces new and non-conflicting knowledge such as "quantum bit parallelism", the net impact value is positive (beneficial), representing that retrieval enhancement improves content value; if the second candidate token has knowledge conflict (such as error reuse of obsolete theory) with LLM, the net impact value is negative (harmful), which needs to be alert to the risk of knowledge fusion; if the content is redundant (such as repeated expression of existing knowledge), the net impact value may tend to 0, indicating that retrieval has not effectively gained.

[0163] Steps S508-S510 can be executed in parallel.

[0164] At step S511 (target token decision and iteration), the net impact value (judging the value / risk of retrieval knowledge), the first semantic similarity (comparing the difference between the two generated chains), and the second semantic similarity (verifying the degree of knowledge integration) are comprehensively considered to decide to select the target token (first or second candidate token) and add it to the generated token sequence to promote content iteration generation. For example: if the net impact value is positive and the second semantic similarity is high (retrieval knowledge is effectively integrated), the second candidate token is preferred to strengthen professional content; if the net impact value is negative (knowledge conflict), the first candidate token is preferred to avoid incorrect information; if the semantic similarity difference is small (retrieval has not significantly gained), the more fluent token is selected as needed. For detailed logic, please refer to the above embodiment.

[0165] At step S512 (generation termination judgment), the complete content after splicing the generated tokens is checked to see if it meets the "complete semantic requirement" of the retrieval generation request (for example, whether it covers the definition, characteristics, technical basis, application value, etc. of quantum computing principle in all dimensions, and whether the logic is closed loop). If it meets the requirement, execute "end generation"; if it does not meet the requirement (such as only generating the definition of the principle, lacking the development of characteristics), go back to step S502 and step S504 to iteratively generate the next token until the content completely covers the requirement.

[0166] Through the above process, the embodiment of the present disclosure can realize a closed loop of "knowledge base pre-construction → pure model generation → retrieval enhancement generation → multi-dimensional semantic / value evaluation → iterative optimization and completion", which not only has the flexibility of LLM generation, but also injects professional knowledge through retrieval to ensure content quality and knowledge accuracy, and adapts to the generation needs of complex scenarios (such as professional document writing, knowledge question and answer enhancement, etc.).

[0167] Figure 6 is a schematic diagram of a question and answer system in an exemplary embodiment of the present disclosure.

[0168] Figure 6 The question and answer system 600 shown can be used for the reply information generation method of any embodiment of the present disclosure.

[0169] Reference Figure 6 In an exemplary embodiment, the question and answer system 600 can include:

[0170] A preset knowledge base 61 for storing semantic blocks and semantic block association graphs;

[0171] A graph retrieval module 62 for performing graph retrieval on the preset knowledge base 61 according to a retrieval generation request and a generated token to obtain a current retrieval result;

[0172] A large model 63 for generating a first candidate token according to the retrieval generation request and the generated token, and generating a second candidate token according to the current retrieval result, the retrieval generation request and the generated token;

[0173] A collaborative screening module 64 for screening a target token from the first candidate token and the second candidate token as a next generated token.

[0174] The collaborative screening module 64 includes:

[0175] A risk assessment unit 641 for determining a net impact value of the current retrieval result;

[0176] A semantic similarity calculation unit 642 for forming first candidate content according to the first candidate token and the generated token, forming second candidate content according to the second candidate token and the generated token, determining a first semantic similarity between the first candidate content and the second candidate content, and a second semantic similarity between the current retrieval result and the second candidate content.

[0177] The collaborative screening module 64 is configured to screen the target token from the first candidate token and the second candidate token as the next generated token according to the net impact value of the current retrieval result, the first semantic similarity and the second semantic similarity.

[0178] Figure 6 The functions of each module shown in the method 100 have corresponding embodiments, which will not be repeated here.

[0179] The embodiment of the present disclosure deeply integrates the graph structure retrieval and the token-level collaborative filtering, provides structured context through the graph retrieval module 62, and filters noise by using the token-level filtering, so that the graph retrieval and the token-level collaborative filtering form an efficient closed loop through bidirectional interaction: on the one hand, the structured context (such as the semantic block association in the multi-hop subgraph) retrieved by the semantic block feature graph retrieval provides more accurate retrieval signals for the token-level collaborative filtering, so that the large model 63 can make decisions based on the filtered high-quality information when generating each token, effectively reducing noise interference; on the other hand, the token sequence dynamically filtered by the token-level collaborative filtering can be fed back to the graph retrieval module 62 as a new prefix, guiding the graph retrieval to perform iterative retrieval to supplement the context that is more suitable for the current generation progress, thereby forming a dynamic optimization closed loop of “retrieval→generation→retrieval again”, significantly improving the accuracy and coherence of knowledge utilization in complex tasks, and bringing significant improvements in multiple aspects for the RAG system, effectively solving the key problems existing in the existing system.

[0180] The embodiment of the present disclosure at least has the following beneficial effects in performance, efficiency, resource utilization and application expansion:

[0181] (1) Fine-grained generation control is realized based on token-level benefit-risk assessment in the collaborative process of LLM and RAG. Through theoretical analysis, the generation process of RAG is decomposed into “gain” and “misleading risk” of external knowledge, and the tokens generated by pure LLM and RAG are dynamically compared by using semantic similarity, so that the more reliable output path is selected in real time, solving the black box problem of traditional methods relying on data-driven.

[0182] (2) The hierarchical reference graph model containing intra-document chapter association and inter-document reference relationship is constructed, and the “lexical-semantic” retrieval of traditional RAG is upgraded to a “structure-semantic-relation” three-in-one retrieval framework. Through the graph convolution network, the text lexical matching signal and the semantic embedding are fused, supporting the capture of complex structural information such as multi-hop reference chain and cross-document semantic association, significantly improving the retrieval depth and comprehensiveness of structured data.

[0183] (3) Through the design of the dynamic interaction mechanism of “graph retrieval providing structured context→token-level generation filtering noise→generation result feeding back to retrieval optimization”, the graph retrieval function provides high-quality context containing multi-hop association for the generation process of the large model, reducing invalid information interference; and the output prefix (such as part of the answer content) of the token-level collaborative generation can be used as a new clue to guide the graph retrieval function to perform iterative query, forming a closed loop of “retrieval enhancement→accurate generation→information completion”, breaking through the performance bottleneck of the traditional RAG module running independently.

[0184] It should be noted that although several modules or units of the devices for action execution are mentioned in the above detailed description, such division is not mandatory. Indeed, according to embodiments of the present disclosure, features and functionalities of two or more modules or units described above can be embodied in one module or unit. Conversely, features and functionalities of one module or unit described above can be further divided into embodied by multiple modules or units.

[0185] In an exemplary embodiment of the present disclosure, an electronic device capable of implementing the above-described method is also provided.

[0186] Those skilled in the art can understand that various aspects of the present disclosure can be implemented as a system, a method or a program product. Therefore, various aspects of the present disclosure can be embodied as a whole hardware embodiment, a whole software embodiment (including firmware, microcode, etc.), or an embodiment combining hardware and software aspects, which can be collectively referred to as "circuitry", "module" or "system".

[0187] The electronic device 700 according to this embodiment of the present disclosure will be described below with reference to Figure 7 Figure 7 The electronic device 700 shown is merely an example and should not limit the function and scope of use of embodiments of the present disclosure.

[0188] As shown in Figure 7 The electronic device 700 is in the form of a general computing device. The components of the electronic device 700 can include, but are not limited to, at least one processor 710, at least one memory 720, and a bus 730 connecting different system components, including the memory 720 and the processor 710.

[0189] The memory stores program codes that can be executed by the processor 710, so that the processor 710 performs the steps described in the above "Exemplary Method" section according to various exemplary embodiments of the present disclosure. For example, the processor 710 can perform the method as shown in the embodiments of the present disclosure.

[0190] The memory 720 can include a readable medium in the form of volatile memory, such as random access memory (RAM) 7201 and / or cache 7202, and can further include read-only memory (ROM) 7203.

[0191] ​The memory 720 can also include the programs / utilities 7204 having a set (at least one) of program modules 7205, such as an operating system, one or more application programs, other program modules, and program data, and each or a combination thereof can include an implementation of a network environment.

[0192] The bus 730 can be representative of one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor or local bus using any of a variety of bus structures.

[0193] The electronic device 700 can also communicate with one or more external devices 800 such as a keyboard or pointing device, a Bluetooth device, etc.; other devices that enable a user to interact with the electronic device 700; and / or one or more devices that enable the electronic device 700 to communicate with one or more other computing devices. Such communication can be via the input / output (I / O) interface(s) 750. Further, the electronic device 700 can communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN), and / or the public network such as the Internet) through a network adapter 760. As depicted, the network adapter 760 is in communication with the other modules of the electronic device 700 through the bus 730. It should be appreciated that other hardware and / or software modules can be used in conjunction with the electronic device 700, including but not limited to microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archival storage systems, etc.

[0194] Those skilled in the art will readily understand that the example embodiments described herein can be implemented by software and / or by software in combination with the necessary hardware. Thus, the technical solutions according to the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash disk, a mobile hard disk, etc.) or a network, and includes a number of instructions to make a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) execute the methods according to the embodiments of the present disclosure.

[0195] In exemplary embodiments of the present disclosure, a computer readable storage medium having stored thereon a program product capable of implementing the above-described methods of the specification is also provided. In some possible implementations, various aspects of the present application can also be implemented in the form of a program product including a program code for causing an end device to perform the steps described in the above "Exemplary Methods" section according to various exemplary embodiments of the present application when the program product is run on the end device.

[0196] The program product for implementing the above-described methods according to embodiments of the present application can take the form of a portable compact disc read-only memory (CD-ROM) and include a program code, and can be run on an end device, such as a personal computer. However, the program product of the present application is not limited thereto, and in the present document, a readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0197] The program product can take any combination of one or more readable media. The readable media can be a readable signal medium or a readable storage medium. The readable storage medium, for example, can be, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (non-exhaustive list) of the readable storage medium include an electrical connection having one or more wires, a portable disc, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0198] The computer readable signal medium can include a data signal propagated in baseband or propagated as a carrier wave in a propagated data signal, in which the readable program code is embodied. Such propagated data signal can take multiple forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination thereof. The readable signal medium can also be any readable medium that is not a readable storage medium and that can transmit, propagate, or transport the program for use by or in connection with an instruction execution system, apparatus, or device.

[0199] The program code contained on the readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0200] The program code may, for example, be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C++ or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's computing device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device or entirely on the remote computing device or server. In the latter scenario, the remote computing device can be connected to the user's computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computing device, for example, through the Internet using an Internet Service Provider.

[0201] Furthermore, the above-described diagrams are merely schematic illustrations of the processes included in the method according to the exemplary embodiments of the present application, and are not intended to be limiting. It is readily understood that the processes shown in the above-described diagrams do not indicate or limit the time sequence of the processes. In addition, it is readily understood that the processes can be executed synchronously or asynchronously, for example, in a plurality of modules.

[0202] Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the features disclosed herein. It is intended that the specification and examples be considered as exemplary only, with the true scope and spirit of the disclosure being indicated by the following claims.

Claims

1. A reply information generation method characterized by comprising: The method comprises: determining a current search result in a preset knowledge base according to a search generation request and generated tokens, an initial value of the number of the generated tokens being zero; generating a first candidate token according to the generated tokens and the search generation request, generating a second candidate token according to the generated tokens, the search generation request and the current search result, forming first candidate content according to the first candidate token and the generated tokens, and forming second candidate content according to the second candidate token and the generated tokens; determining a first semantic similarity between the first candidate content and the second candidate content, and a second semantic similarity between the current search result and the second candidate content; determining a target token in the first candidate token and the second candidate token according to the first semantic similarity and the second semantic similarity, and setting the target token as a generated token; repeating the above process until the generated tokens meet the complete semantic requirements of the search generation request.

2. The reply information generating method of claim 1, wherein The forming process of the preset knowledge base comprises: obtaining an original document, and forming reference information after data cleaning of the original document; decomposing the reference information into a plurality of semantic blocks according to semantics; forming a semantic block association graph according to the association relationship of the semantic blocks in the original document and the reference relationship between the original documents, wherein the nodes of the semantic block association graph are the semantic blocks, and the attributes of the nodes are the contents of the semantic blocks; the edges of the semantic block association graph are formed according to the association relationship between the semantic blocks, and the weights of the edges are determined according to the association strength between the semantic blocks.

3. The reply information generating method of claim 2, wherein Determining a current search result in a preset knowledge base according to a search generation request and generated tokens comprises: determining a plurality of first semantic blocks according to the lexical and semantic correlation between the search generation request and the generated tokens and the semantic blocks; finding second semantic blocks that have direct association or multi-hop indirect association with the first semantic blocks along high-weight edges in the semantic block association graph according to the edges of the first semantic blocks in the semantic block association graph and the weights of the edges; determining hierarchical information of the first semantic blocks and the second semantic blocks through a graph convolution network; determining a target semantic block corresponding to the semantic of the search generation request and the generated tokens in the first semantic blocks and the second semantic blocks according to the hierarchical information, and determining the target semantic block as the current search result.

4. The reply information generating method of claim 3, wherein Determining hierarchical information of the first semantic blocks and the second semantic blocks through a graph convolution network comprises: combining the edge weights and structural relationships in the semantic block association graph, aggregating neighbor node features of the first semantic blocks or the second semantic blocks according to logical dependency relationships of the first semantic blocks or the second semantic blocks in the original document to determine logical levels of the first semantic blocks or the second semantic blocks through the graph convolution network; determining semantic levels of the first semantic blocks or the second semantic blocks according to the degrees of abstraction of the first semantic blocks or the second semantic blocks through the graph convolution network. determine, by the graph convolution network, a reference propagation path corresponding to the first semantic block or the second semantic block according to a position of the first semantic block or the second semantic block in the cross-document reference chain and a weight of a reference edge, to determine a reference chain level of the first semantic block or the second semantic block; encode, by the graph convolution network, information of the logical level, the semantic level, and the reference chain level, to form level information of the first semantic block or the second semantic block.

5. The reply information generating method of claim 1, wherein determining the target token from the first candidate token and the second candidate token according to the first semantic similarity and the second semantic similarity includes: setting the first candidate token as the target token when the first semantic similarity is greater than the second semantic similarity, and setting the second candidate token as the target token when the first semantic similarity is less than or equal to the second semantic similarity.

6. The reply information generating method of claim 1, wherein determining the target token from the first candidate token and the second candidate token according to the first semantic similarity and the second semantic similarity includes: determining a first probability distribution corresponding to the first candidate token, a second probability distribution corresponding to the second candidate token, and a third probability distribution corresponding to the current retrieval result; calculating a KL divergence of the first probability distribution and the third probability distribution to determine a harmful value of the current retrieval result, and calculating a KL divergence of the second probability distribution and the third probability distribution to determine a beneficial value of the current retrieval result; determining a net influence value of the current retrieval result on the second candidate token according to a difference between the beneficial value and the harmful value; determining the target token from the first candidate token and the second candidate token according to the net influence value, the first semantic similarity, and the second semantic similarity.

7. The reply information generating method of claim 6, wherein determining the target token from the first candidate token and the second candidate token according to the net influence value, the first semantic similarity, and the second semantic similarity includes: setting the first candidate token as the target token when the net influence value is a harmful value; setting the second candidate token as the target token when the net influence value is a beneficial value and the first semantic similarity is less than or equal to the second semantic similarity; setting the first candidate token as the target token when the net influence value is a beneficial value and the first semantic similarity is greater than the second semantic similarity.

8. A question answering system, characterized by A method for performing the answer information generation method according to any one of claims 1-7, comprising: a preset knowledge base for storing semantic blocks and semantic block association graphs; a graph retrieval module for performing graph retrieval on the preset knowledge base according to a retrieval generation request and a generated token to obtain a current retrieval result; a large model for generating a first candidate token according to the retrieval generation request and the generated token, and generating a second candidate token according to the current retrieval result, the retrieval generation request, and the generated token; a collaborative screening module for screening a target token from the first candidate token and the second candidate token as a next generated token.

9. The question answering system of claim 8, wherein, The cooperative screening module comprises: a risk assessment unit configured to determine a net impact value of the current search result; a semantic similarity calculation unit configured to form first candidate content according to the first candidate token and the generated token, form second candidate content according to the second candidate token and the generated token, determine a first semantic similarity between the first candidate content and the second candidate content, and a second semantic similarity between the current search result and the second candidate content; the cooperative screening module is configured to screen a target token from the first candidate token and the second candidate token as the next generated token according to the net impact value of the current search result, the first semantic similarity and the second semantic similarity.

10. An electronic device, comprising: comprise: a memory; and a processor coupled to the memory, the processor configured to perform the method of any of claims 1-7 based on instructions stored in the memory.

Images