A method and system for retrieving enhanced knowledge base based on RAG

Abstract

The application discloses a kind of based on RAG retrieval enhanced knowledge base technical method and system, it is related to knowledge base technical field, method includes: source document is parsed into structured text segment and generates semantic vector;Text segment is split into minimum knowledge unit, and concept and behavior trigger word are extracted to build concept association graph, and save session summary in fixed length annular structure;Receive user query and generate query vector and retrieve most relevant text segment, combine concept association graph hierarchical diffusion and annular summary matching result, build and deduplicate candidate knowledge unit, according to atlas path depth splicing original paragraph, generate dynamic prompt box, call generation model output preliminary answer after execution verification, and verify state is marked in final answer.By building concept association graph, real-time update graph edge activity weight and time sequence fingerprint, realize the accurate capture of query intent deep semantic and logical relationship.

Description

Technical Field

[0001] This invention relates to the field of knowledge base technology, specifically to a knowledge base technology method and system based on RAG retrieval enhancement. Background Technology

[0002] Existing knowledge base retrieval systems typically rely on keyword matching or sparse retrieval models to segment and match user queries, then search for records containing the corresponding terms in a static document database. This approach struggles to grasp the deep semantic meaning of the query intent, and when faced with synonyms, ambiguous terms, or complex search conditions, the search results often lack sufficient satisfaction. Furthermore, traditional retrieval systems often fail to effectively integrate external real-time data or user history dialogue information with the current query, resulting in responses that lack timeliness and coherence.

[0003] To improve question-answering quality, researchers have recently attempted to combine vectorized semantic retrieval with large-scale language model (LLM) generation, proposing a retrieval-enhanced generation (RAG) scheme. However, existing RAG implementations mostly remain at the level of simply concatenating retrieval fragments and then processing them by the generative model, lacking dynamic scoring of retrieval results and management of relational graphs. Such schemes struggle to achieve temporal awareness of changes in document hotspots and cannot continuously optimize retrieval and generation strategies based on user feedback.

[0004] Furthermore, many cloud-based knowledge base services require the uploading of sensitive enterprise documents to third-party platforms, making it difficult to meet the stringent requirements for data privatization and compliance. Therefore, there is an urgent need for an innovative technical solution that, in a locally deployed environment, generates verifiable and highly reliable answers through deep structured document parsing, fusion of vector retrieval and concept graph retrieval, and integration of multi-dimensional session memory and user feedback loops. Summary of the Invention

[0005] In view of the shortcomings of the prior art described above, the purpose of this invention is to provide a knowledge base technology method and system based on RAG retrieval enhancement to solve the above-mentioned technical problems.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a knowledge base technology method based on RAG retrieval enhancement, comprising: S1: Parse the source document into structured text fragments, input them into a pre-trained semantic encoder to generate semantic vectors, and store them in a vector database; S2: Break down the text fragment into the smallest knowledge unit, extract concepts and behavioral trigger words, construct a concept association graph with concepts as nodes and behavioral trigger words as edges, and save the query summary and answer summary of the preset session in a fixed-length ring structure; S3: After receiving the user's query, generate a semantic vector for the query text, and retrieve a preset number of text fragments that are most relevant to the query vector from the vector database; S4: Based on the summaries matched in the hierarchical diffusion and circular structure of the concept association graph according to the concepts of the query text fragments, merge the vector retrieval results and graph retrieval results, construct a candidate knowledge unit set, remove duplicates, and splice the original text paragraphs in order according to the path depth of the knowledge units in the concept association graph, and insert logical bridging sentences between adjacent paragraphs. S5: Based on the concatenated paragraphs and the user's original query, construct a dynamic prompt box that includes key interaction points and external verification prompts. After generating an initial answer based on the dynamic prompt box using a generative model, perform verification and mark the verification result as a verification status in the final answer.

[0007] The present invention is further configured such that, in the concept association graph, edges are assigned activity weights and temporal fingerprints, and the update logic includes: By segmenting text fragments according to dependency relationships, multiple smallest knowledge units can be obtained; Identify the set of concept words and the set of behavior trigger words in each knowledge unit. Based on the ratio of the number of concept words to the number of behavior trigger words, calculate the initial correlation degree of concept pairs according to the edit distance decay between concept words. For each edge to be updated in the concept association graph, the original activity weight is decayed exponentially according to a predetermined decay exponent, and the activity weight is updated by accumulating the initial association degree from multiple rounds. The difference between the last update timestamp and the current processing timestamp is recorded on the same side and merged with the original time fingerprint in proportion using an adjustable memory coefficient to generate a new time fingerprint; In the concept association graph, if an edge does not exist, the initial association degree and temporal fingerprint are set as the initial activity weight and temporal fingerprint of the newly created edge.

[0008] The present invention is further configured to store the query summary and answer summary of a preset session in a fixed-length ring structure, including: A fixed number of recent session rounds are stored in a circular queue. Each round enqueues a query summary and a response summary in pairs. When the queue is full, the first summary is removed from the queue. Calculate the matching degree by performing intersection and union calculations on the concept word set of each summary and the current query; Set the normalized matching degree as a score, correspond it to the summary, and write it back to the circular queue in an atomic transaction manner.

[0009] The present invention is further configured such that step S3 includes: After receiving a user query, the query text is input into a pre-trained semantic encoder to obtain the query semantic vector; A set of candidate vectors is obtained from the vector database in a preset batch size, and a retrieval score is calculated for each candidate vector. The candidate vectors are sorted in descending order of retrieval scores, compared with dynamic score thresholds, and retained sequentially until a preset number of fragments is reached. The text fragment identifiers corresponding to the retained candidate vectors are then output.

[0010] The present invention is further configured such that the construction logic of the candidate knowledge unit set includes: Obtain the retrieval score, sum of summary injection values, and total hierarchical influence for each text segment in the vector retrieval result set; The retrieval score of each text fragment is increased by a specified index, the total influence of the corresponding level is mapped to the same numerical domain according to the depth decay factor, and the sum of the summary injection values ​​is weighted by a power after logarithmic mapping to generate a comprehensive candidate score. All text fragments are sorted in descending order of comprehensive candidate scores, and text fragment identifiers that are greater than the score threshold and less than or equal to a preset number are retained to construct a candidate knowledge unit set.

[0011] The present invention is further configured to calculate the similarity of each unit in the candidate knowledge unit set based on the XOR distance of semantic fingerprint, and remove redundant units whose similarity exceeds a preset threshold. For the remaining units, an innovation score is calculated based on the number of concept words and the shortest path depth in the concept association graph, combined with vector retrieval scores and graph diffusion effects. The list of units scored by innovation is first sorted in ascending order by path depth, and then in descending order by innovation score for stable sorting. The original text paragraphs of the sorted units are extracted in sequence and sent to the output buffer. For each pair of adjacent paragraphs, the concept with the highest frequency is selected from the shared concept set. Combined with the difference between the two concepts, a concise bridging sentence is generated through high-order mapping and inserted between the paragraphs to output continuous paragraph text.

[0012] The present invention is further configured such that the construction logic of the dynamic prompt box includes: Extract keywords from each core knowledge unit in the spliced ​​paragraph and calculate the unit importance based on keyword frequency and paragraph position; Set the most important preset quantity unit as the interactive key point, and generate an index identifier and summary text for each key point; Identify the entities and technical terms contained in all sentences in the concatenated paragraph, calculate the verification demand strength based on entity rarity and sentence length, and mark sentences with verification demand strength greater than a threshold as sentences requiring external verification. Place the original user query at the beginning of the prompt box, and list the generated interaction points and the sequence number of sentences requiring external verification in sequence.

[0013] The present invention is further configured to extract propositional assertions from the preliminary answers output by the generative model and construct a list of fact triples; For each fact triple, retrieve the corresponding entity node and path in the concept association graph, and obtain the internal consistency index based on the path and time fingerprint. Parallel calls to external knowledge sources perform evidence retrieval on the same fact triple, and obtain external credibility indicators based on external results and assertion text; A comprehensive verification score is generated based on internal consistency indicators and external credibility indicators; The comprehensive verification score is compared with the preset verification threshold. If the score is greater than or equal to the threshold, it is marked as verified; if the score is less than the threshold, it is marked as pending verification.

[0014] The present invention is further configured to include: Analyze user feedback information to obtain the intensity of summary interaction; For each summary in the circular session memory, a power-law retention operation is performed on the summary score. Logarithmic mapping is performed based on the summary interaction strength signal, and the result is combined with the power-law result to obtain the updated summary score.

[0015] This invention also provides a knowledge base technology system based on RAG retrieval enhancement, used to implement the above-mentioned knowledge base technology method based on RAG retrieval enhancement, the system comprising: Parsing module: Parses the source document into structured text fragments, inputs them into a pre-trained semantic encoder to generate semantic vectors, and stores them in a vector database; Building Module: The text fragments are broken down into the smallest knowledge units, and concepts and behavioral trigger words are extracted. A concept association graph is built with concepts as nodes and behavioral trigger words as edges. The query summary and answer summary of the preset session are stored in a fixed-length circular structure. Query module: After receiving a user query, it generates a semantic vector for the query text and retrieves a preset number of text fragments that are most relevant to the query vector from the vector database; The splicing module: Based on the summaries matched in the hierarchical diffusion and circular structure of the concept association graph of the query text fragment, it merges the vector retrieval results and graph retrieval results, constructs a set of candidate knowledge units, performs deduplication, splices the original text paragraphs in order of the path depth of the knowledge units in the concept association graph, and inserts logical bridging sentences between adjacent paragraphs; The generation module constructs a dynamic prompt box based on the concatenated paragraphs and the user's original query, including key interaction points and external verification prompts. After generating an initial answer based on the dynamic prompt box through the generation model, it is verified, and the verification result is marked as a verification status in the final answer.

[0016] This invention provides a knowledge base technology method and system based on RAG retrieval enhancement. The method parses source documents into structured text fragments, inputs them into a pre-trained semantic encoder to generate semantic vectors, and stores them in a vector database. The text fragments are then split into the smallest knowledge units, and concepts and behavioral trigger words are extracted. A concept association graph is constructed with concepts as nodes and behavioral trigger words as edges. A fixed-length circular structure is used to store query and answer summaries for a preset session. Upon receiving a user query, a semantic vector is generated from the query text. A preset number of text fragments most relevant to the query vector are retrieved from the vector database. Based on the summaries matched in the hierarchical diffusion and circular structure of the concept association graph according to the concepts of the query text fragments, the vector retrieval results and graph retrieval results are merged to construct a candidate knowledge unit set. Deduplication is performed, and the original text paragraphs are sequentially concatenated according to the path depth of the knowledge units in the concept association graph, with logical bridging sentences inserted between adjacent paragraphs. Based on the concatenated paragraphs and the user's original query, a dynamic prompt box including interactive points and external verification hints is constructed. After generating a preliminary answer based on the dynamic prompt box using a generative model, verification is performed, and the verification result is marked as a verification status in the final answer. The beneficial effects include: 1. Significantly improved retrieval accuracy: Through vectorized retrieval and concept association graph retrieval, the deep semantic and logical relationships of the query intent are accurately captured, effectively reducing interference from irrelevant information; 2. Significantly enhanced credibility of answers: The introduction of multi-dimensional conversation memory and triangulation ensures that the generated answers are supported by historical context and verified and labeled by multiple sources, reducing the risk of "illusions". 3. Dynamic time-series awareness capability: The activity weights and time-series fingerprints of the edges in the concept association graph are updated in real time, which can automatically capture changes in document hotspots, so that the search results can smoothly connect old and new information and highlight hot content.

[0017] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings: Figure 1 A flowchart illustrating a knowledge base technology method based on RAG retrieval enhancement, as shown in an exemplary embodiment of the present invention; Figure 2This is a schematic diagram illustrating the structure of a knowledge base technology system based on RAG retrieval enhancement, as an exemplary embodiment of the present invention. Detailed Implementation

[0019] The embodiments of the present invention will be described below with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred embodiments are only for illustrating the present invention and not for limiting the scope of protection of the present invention.

[0020] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Therefore, the drawings only show the components related to the present invention and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0021] In the following description, numerous details are explored to provide a more thorough explanation of embodiments of the invention. However, it will be apparent to those skilled in the art that embodiments of the invention may be practiced without these specific details. In other embodiments, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring embodiments of the invention.

[0022] Example 1 A knowledge base technology method based on RAG retrieval enhancement, such as Figure 1 As shown, it includes: S1: Parse the source document into structured text fragments, input them into a pre-trained semantic encoder to generate semantic vectors, and store them in a vector database; S2: Break down the text fragment into the smallest knowledge unit, extract concepts and behavioral trigger words, construct a concept association graph with concepts as nodes and behavioral trigger words as edges, and save the query summary and answer summary of the preset session in a fixed-length ring structure; S3: After receiving the user's query, generate a semantic vector for the query text, and retrieve a preset number of text fragments that are most relevant to the query vector from the vector database; S4: Based on the summaries matched in the hierarchical diffusion and circular structure of the concept association graph according to the concepts of the query text fragments, merge the vector retrieval results and graph retrieval results, construct a candidate knowledge unit set, remove duplicates, and splice the original text paragraphs in order according to the path depth of the knowledge units in the concept association graph, and insert logical bridging sentences between adjacent paragraphs. S5: Based on the concatenated paragraphs and the user's original query, construct a dynamic prompt box that includes key interaction points and external verification prompts. After generating an initial answer based on the dynamic prompt box using a generative model, perform verification and mark the verification result as a verification status in the final answer.

[0023] Specifically, in step S1, the source document is input into the parser. The source document includes text. The original text is segmented into several structured text fragments with context labels, including headings, paragraphs, lists, table items, and metadata location identifiers. These structured text fragments are then used as input to call a finely tuned pre-trained semantic encoder, including a Transformer-based deep bidirectional language model, to extract a high-dimensional semantic vector for each fragment. The semantic vector represents the fragment's position in the semantic space as a continuous real-number matrix, capturing contextual dependencies and implicit topics through a self-attention mechanism. Each generated semantic vector and its corresponding text fragment identifier are encapsulated as an index item. The text fragment identifier includes a document identifier, fragment number, and location information, and these are batch-written into a local vector database. The database constructs a multi-level index structure based on an approximate nearest neighbor algorithm. In step S2, relying on syntactic analysis and semantic boundary detection, each structured text segment is divided into the smallest knowledge unit, which typically corresponds to a sentence or paragraph with complete semantics. After segmentation, each unit can independently carry complete concept-behavior information, avoiding ambiguity caused by cross-sentence semantic fusion. Named entity recognition and part-of-speech tagging are applied to each knowledge unit to extract all nouns or terms as concepts, and combined with verb phrases or action phrases to identify behavior trigger words, obtaining a pair of concept sets and behavior sets, laying the foundation for graph construction. Each concept is mapped to a graph node, and the corresponding behavior trigger word is mapped to an edge connecting the nodes. For all concept pairs in a knowledge unit, an edge is created or updated, and the latest activity index and temporal fingerprint are recorded on the edge to reflect the frequency and time distribution of the association in local documents and historical queries. A preset number of recent dialogue rounds are stored in a circular queue, with each round enqueuing a "query summary - answer summary" pair. When the queue is full, the oldest entry is discarded, always maintaining the latest historical context. Each summary is accompanied by a matching score. The present invention is further configured such that, in the concept association graph, edges are assigned activity weights and temporal fingerprints, and the update logic includes: Text fragments are segmented according to dependency relations to obtain multiple minimum knowledge units; specifically, syntactic dependency relation parsing is performed on each structured text fragment, and several minimum knowledge units are segmented with verbs and core semantic touchpoints as boundaries to ensure that the semantics within each unit are complete and can independently reflect the correspondence between concepts and behaviors. Identify the set of concept words and the set of action trigger words in each knowledge unit. Based on the ratio of the number of concept words to the number of action trigger words, calculate the initial relevance of concept pairs according to the edit distance decay between concept words; specifically, for the first... Any binary concept in a knowledge unit Calculate the initial relevance with the set of behavior trigger words : , The number of concept words in a knowledge unit. The number of action trigger words in a knowledge unit; lev Conceptual terms and The edit distance between them; the ratio of the total number of concepts to "number of actions + 1" is raised to the power of two, which significantly amplifies the contribution of high concept density units, while avoiding division by zero errors caused by zero count of actions; the edit distance is half-decimated by an exponential function, and the relevance is multiplied by about 0.607 for every 1 increase in distance; effectively distinguishing between similar and dissimilar concepts; For each edge in the concept association graph to be updated, the original activity weight is decayed exponentially according to a predetermined decay exponent, and the updated activity weight is then added based on the cumulative initial association degree over multiple rounds; specifically, for each edge in the concept association graph to be updated... (connect and First, adopt historical forgetting, then integrate new connections: ; The activity weight before the update; The difference in timestamps between this update and the last update, in days; The initial relevance from all new knowledge units is accumulated; the original weights are non-linearly decayed by a power of 0.9, and then combined with an exponential factor. By implementing a time-series fade-out mechanism, a memory mechanism prioritizing recent connections and decreasing historical connections is constructed. The initial relevance of all knowledge units to this side is accumulated, and the two parts are summed to set the final activity level. This balances the fading of old connections with the strengthening of new connections, enabling the graph to continuously capture the evolution of hotspots. For the same edge, the difference between the last update timestamp and the current processing timestamp is recorded and merged with the original temporal fingerprint proportionally using an adjustable memory coefficient to generate a new temporal fingerprint; specifically, a temporal fingerprint is maintained for each edge. To depict the distribution of time: ; The time sequence fingerprint before the update; The time difference is the same as the activity update time difference; historical time-series fingerprints are linearly preserved with a memory retention factor of 0.8 to simulate the gradual decay of the old time distribution; then the time difference is... Increasing the value to the second power highlights the strengthening effect of a longer time interval on the freshness fingerprint. After proportional superposition, a new temporal fingerprint is obtained. If no edge exists in the concept association graph, the initial association degree and temporal fingerprint are set as the initial activity weight and temporal fingerprint of the newly created edge. Specifically, if a pair of concepts is found to have no edge connection in the concept association graph, the initial association degree and preliminary temporal fingerprint of the pair in the current unit are used together as the initial activity weight and temporal fingerprint of the newly created edge to ensure that the first concept association has basic support.

[0024] The present invention is further configured to store the query summary and answer summary of a preset session in a fixed-length ring structure, including: A fixed number of recent session rounds are stored in a circular queue. Each round, a query summary and a response summary are enqueued in pairs. The head summary is removed from the queue when it is full. Specifically, a fixed length... Build a circular queue to store the most recently... The round consists of "query summary - answer summary" pairs. Whenever a new round of summary pairs is enqueued, if the queue is full, the element at the head of the queue is automatically removed, thus ensuring that the queue always retains the latest context. For each summary and the set of terms for the current query, the intersection and union of the terms are used to calculate the matching degree; specifically, for each summary's set of terms... With the set of concept words currently being queried First, find the size of their intersection and the size of their union, then proceed according to... Calculate matching degree The common concept number is obtained by taking the intersection of the concept word sets extracted from the summary and the query, which reflects the degree of semantic overlap between the two. The common concept number is raised to the power of two to non-linearly amplify the matching effect between the summary and the query with high overlap, so that the matching degree increases sharply when the key information is fully covered. The total number of concepts in the union is used as the divisor to normalize the matching degree to the range of [0,+∞), ensuring that the matching degree is comparable between sets of different sizes. The normalized matching score is set as a rating, mapped to the summary, and written back to the circular queue in an atomic transaction manner; specifically, the matching score of each summary is... Set the rating to correspond to the summary, and write it back to the corresponding slot in the circular queue in an atomic transaction manner to ensure update consistency.

[0025] The present invention is further configured such that step S3 includes: After receiving a user query, the query text is input into a pre-trained semantic encoder to obtain a query semantic vector; specifically, the user-input query text is passed to the aforementioned pre-trained semantic encoder to output the query semantic vector. semantic vector Locate the core features of query intent in a high-dimensional semantic space to provide a basic representation for similarity retrieval; A candidate vector set is obtained from the vector database with a preset batch size, and a retrieval score is calculated for each candidate vector; specifically, the process is based on the system's preset batch size. Parallel acquisition from vector database with Most likely related to the front candidate vectors and its corresponding time series factors (Indicating the freshness or novelty of the content). A retrieval score is then calculated for each candidate vector. : , A non-linear decay is applied to the vector distance; for each increase in distance, the similarity is multiplied by approximately exp(-0.2) ≈ 0.82. When the distance is large, it rapidly approaches zero, ensuring that only semantically similar vectors receive significant scores. With time series factor The power-law mapping appropriately amplifies the priority of recent content, when (Latest) This item is 1.5; when (Oldest), this item is 1.0, which ensures that there is a difference between the old and new content but does not completely lose historical information; the two parts are multiplied to obtain the final score, which can accurately assess semantic similarity and take into account the newness of the content, thereby achieving efficient candidate vector ranking; Candidate vectors are sorted in descending order of retrieval score, compared with dynamic score thresholds, and retained sequentially until a preset number of fragments is reached. The text fragment identifiers corresponding to the retained candidate vectors are then output. Specifically, all... After sorting in descending order, compare with the dynamic threshold in turn. The comparison is performed, and only scores higher than the threshold and not exceeding a preset number are retained. The former The final output consists of candidate vectors. The corresponding text segment identifiers are then used as input for subsequent retrieval enhancement generation. A threshold for retaining the percentage of scores is set, with a value range of [0.6, 0.9].

[0026] In step S4, for each text fragment returned by vector retrieval, the set of core concept words is identified. Each concept word is then marked as a starting level node in the global concept association graph, forming a hierarchical mapping queue. For each level of mapping node, expand sequentially according to the preset maximum diffusion depth: each time starting from the current node Spread to all its directly adjacent nodes ; Calculate the path impact value This is calculated by exponentiating all edge activity weights on the current path, multiplying them by their respective powers, and then multiplying by the depth exponent of the hierarchy decay factor. After diffusion ends, the influence values ​​of each concept word on all paths are summarized to obtain the total hierarchy influence of each concept. , For concept nodes The level affects the total amount. For all from the query core to A set of simple paths, each path Contains edge sequence , ,…, And the length does not exceed the maximum depth , For path length, For the first on the path The activity weight of the edge; for all summaries in the circular memory, the set of concept words for the current text segment. Perform an intersection operation and calculate a summary injection value for each matching summary, which is equal to the product of the power mapping of the number of matching concepts and the multidimensional score of that summary, divided by the exponential function of the total number of concepts in that segment: For the current text fragment concept set With all summaries in circular memory Calculate each individual injection value separately and then sum them up to get the total. , Inject the sum of values ​​into the summary. For the first A collection of abstract concepts. The summary is scored; more specifically, the score is the relevance of each summary. As has been explained in detail above, it will not be repeated here.

[0027] The present invention is further configured such that the construction logic of the candidate knowledge unit set includes: Obtain the retrieval score, sum of summary injection values, and total hierarchical influence for each text segment in the vector retrieval result set; specifically, receive the retrieval score of each candidate vector in each text segment of the vector retrieval result set. The retrieval score for each text segment is obtained by summing and averaging. Summary of injected values The path influence of text fragments obtained through hierarchical diffusion in the concept association graph. ; The retrieval score of each text segment is boosted by a specified index, and the total influence of the corresponding level is mapped to the same numerical domain according to a depth decay factor. The sum of the abstract injection values ​​is then weighted by a power factor after logarithmic mapping to generate a comprehensive candidate score. Specifically, for each segment... Based on search scores Summary of injected values and path influence Calculate candidate scores ,in, For fragments Candidate scores, for The second power is used to significantly widen the gap in high similarity scores after mapping, prioritizing the retention of segments that are closest to the query; exponential decay is first performed based on path depth, and then the power is used to strengthen high-impact values, balancing shallow and deep associations; the logarithm of historical matching scores is taken to compress extreme values, and then the second power is used to highlight the contribution of highly credible summaries to candidates; the product of the three can simultaneously reflect semantic relevance, logical support, and historical credibility, forming a unified comprehensive score; All text fragments are sorted in descending order of comprehensive candidate score, and text fragment identifiers with scores greater than the score threshold and less than or equal to a preset number are retained to construct a candidate knowledge unit set; specifically, all Sort in descending order and filter accordingly to find those that meet the criteria. The previous score was greater than the scoring threshold. The text fragments are identified to form the final candidate knowledge unit set, and this set and its corresponding identifiers are passed to the subsequent generation module. value.

[0028] The present invention is further configured to calculate the similarity of each unit in the candidate knowledge unit set based on the XOR distance of semantic fingerprints, and remove redundant units whose similarity exceeds a preset threshold; specifically, for any two units Obtain its binary semantic fingerprint vector Calculate Hamming distance Similarity scores are generated based on the power-exponential mapping. ,like Greater than the preset threshold (0.9), then the one with the lower initial value of innovation is removed from the two. The Hamming distance is the binary vector distance. Perform a bitwise XOR operation on the two fingerprint vectors and count the number of 1s in the result to obtain the Hamming distance, which reflects the degree of dispersion of the two units in the semantic fingerprint space. For the remaining units, an innovation score is calculated based on the number of concept words and the shortest path depth in the concept association graph, combined with vector retrieval scores and graph diffusion effects; specifically, for each remaining unit... An innovation score is generated by combining four dimensions in parallel. : ,in, The number of concept words in the unit; The shortest path length from the unit in the concept graph to the query core; Score for vector retrieval; To determine the total impact of the graph's layered diffusion; to raise the number of concept words to a power of two, significantly improving the innovation score of information-rich units; to use the shortest path hop count as the denominator of the first power, ensuring that units further from the query core have lower innovation scores; to raise the vector retrieval score to a power of two, further distinguishing between highly similar and moderately similar units; and to raise the graph diffusion impact to a power of 1.5, enhancing units connected to multiple high-weight paths in a non-linear manner. The list of units scored by innovation is first sorted in ascending order by path depth, then stably sorted in descending order by innovation score. The original text paragraphs of each sorted unit are extracted sequentially and added to the output buffer. For each pair of adjacent paragraphs, the most frequent concept is selected from the shared concept set, and a concise bridging sentence is generated by combining the difference between the two concepts through a high-order mapping and inserted between the paragraphs, outputting continuous paragraph text. Specifically, the original text paragraphs are extracted sequentially according to the sorting results and appended to the output buffer; for adjacent paragraph pairs... Extracting shared concept sets Difference between concepts Select the concept that appears most frequently among the shared concepts. Generate bridging sentences.

[0029] The present invention is further configured such that the construction logic of the dynamic prompt box includes: Keywords are extracted from each core knowledge unit in the concatenated paragraphs, and the unit importance is calculated based on keyword frequency and paragraph position. Specifically, from the concatenated context paragraphs, the set of keywords for each core knowledge unit is extracted, and the frequency of each keyword within the unit is counted. Simultaneously, the unit's position within the paragraph is recorded, including the first paragraph (1.2), middle paragraph (1.0), and last paragraph (0.8). Information density is primarily measured by keyword frequency, while positional weights reflect user attention preferences, and the importance of each unit is calculated. Higher importance indicates that the unit's content is more concentrated and closer to the beginning of the paragraph, making it more likely to become the focus of user interaction. The most important preset number of units are set as interactive key points, and an index identifier and summary text are generated for each key point. Specifically, based on the above importance values, the top preset number of units are selected as interactive key points. A unique index identifier (e.g., "Key Point-1", "Key Point-2") is generated for each key point, and the first sentence or core summary text of the unit is extracted to form a list of key point entries, making it easy for users to quickly locate and expand the corresponding content when clicking or referencing it. The system identifies entities and technical terms contained in all sentences within the concatenated paragraph. It calculates the verification demand strength based on entity rarity and sentence length, marking sentences with a verification demand strength exceeding a threshold as requiring external verification. Specifically, the concatenated paragraph is split into independent sentences, and entity recognition and technical term extraction are performed on each sentence. The verification demand strength is then evaluated based on the entity rarity in the entire knowledge base and the sentence's character or vocabulary length. If the strength exceeds a preset threshold, the sentence is determined to potentially contain model illusions or requests for the latest dynamic information, is marked as requiring external verification, and its sequence number within the paragraph is recorded. Further, the invention quantifies entity rarity using frequency of occurrence. The original user query is placed at the beginning of the prompt box, followed by a list of generated interaction points and the sequence numbers of sentences requiring external verification. Specifically, a structured prompt box data object is constructed, with the user's original query text at the beginning as a question context identifier. Below this, a list of interaction points (including point indexes and corresponding summaries) and a list of sentences requiring external verification are listed in sequence. An additional field indicates the verification interface type or external resource call method. This structured object is output in JSON or a similar data format for rendering by the generated model or front-end interface.

[0030] The prompt box construction logic automatically generates interactive key points by sorting the importance of key units, which significantly improves the efficiency of users locating the information they need. At the same time, it automatically marks potentially uncertain or emerging facts that require external verification, ensuring the transparency of the generated answers and providing precise trigger points for downstream calls to external data interfaces.

[0031] The present invention is further configured to extract propositional assertions from the preliminary responses output by the generative model and construct a list of fact triples; specifically, it performs deep analysis on the natural language responses output by the generative model to identify propositional assertions and structures each assertion into a list of fact triples in the form of <subject, predicate, object>. This structuring process ensures that the subsequent verification module can process various factual statements in a uniform format; For each fact triple, the corresponding entity node and path are retrieved in the concept association graph. An internal consistency index is obtained based on the path and temporal fingerprint. Specifically, for each fact triple in the list, the nodes corresponding to the triple's subject and object are first located in the concept association graph, and then all paths connecting these two nodes are retrieved. The internal consistency index is evaluated based on the activity weights and temporal fingerprint distribution of these paths. This index reflects the strength of the support between the stated facts and the existing knowledge graph. The system performs evidence retrieval on the same fact triples by calling external knowledge sources in parallel, and obtains external credibility indicators based on the external results and the original assertion text. Specifically, it triggers external evidence retrieval for each triple in parallel, extracting corresponding facts or data from authoritative APIs, literature databases, or knowledge bases. The external evidence is matched with the original assertion text and the differences are evaluated to form external credibility indicators, thereby quantifying the reliability of the factual basis for the assertion. A comprehensive validation score is generated based on internal consistency and external credibility indicators. Specifically, the internal consistency and external credibility indicators are fused to obtain the comprehensive validation score for each triple. The fusion strategy considers both the strength of internal graph support and the matching degree of external data corroboration to ensure that the score balances the contributions of both. The comprehensive verification score is compared with a preset verification threshold. Those scores exceeding the threshold are marked as verified, while those below are marked as pending verification. This labeling information is then integrated into the final answer text. Specifically, the comprehensive verification score is compared with the preset verification threshold; those scores exceeding the threshold are marked "verified," and those below are marked "pending verification." Subsequently, the verification labels for each assertion are systematically integrated back into the original answer text. This ensures that the final output includes both the answer content and explicitly displays the verification status of each assertion, thereby improving transparency and auditability.

[0032] The present invention is further configured to include: The user feedback information is analyzed to obtain the summary interaction intensity; specifically, the number of times users click on each summary in the prompt box is collected and mapped to the summary interaction intensity signal. For each summary in the circular session memory, a power-law retention operation is performed on the summary score. Logarithmic mapping is then performed based on the summary interaction strength signal, and the result is combined with the power-law result to obtain the updated summary score. Specifically, for each summary in the circular session memory, its existing score is first subjected to a power-law retention operation to preserve historical evaluation differences. Then, logarithmic mapping is applied based on the summary interaction strength signal, and the mapping result is combined with the power-law retention result to form a new summary credibility score. This allows high-value summaries that are frequently clicked by users to receive higher priority in subsequent searches.

[0033] Example 2 Please see Figure 2 This exemplary knowledge base technology system based on RAG retrieval enhancement is used to implement the aforementioned knowledge base technology method based on RAG retrieval enhancement, and includes: Parsing module: Parses the source document into structured text fragments, inputs them into a pre-trained semantic encoder to generate semantic vectors, and stores them in a vector database; Building Module: The text fragments are broken down into the smallest knowledge units, and concepts and behavioral trigger words are extracted. A concept association graph is built with concepts as nodes and behavioral trigger words as edges. The query summary and answer summary of the preset session are stored in a fixed-length circular structure. Query module: After receiving a user query, it generates a semantic vector for the query text and retrieves a preset number of text fragments that are most relevant to the query vector from the vector database; The splicing module: Based on the summaries matched in the hierarchical diffusion and circular structure of the concept association graph of the query text fragment, it merges the vector retrieval results and graph retrieval results, constructs a set of candidate knowledge units, performs deduplication, splices the original text paragraphs in order of the path depth of the knowledge units in the concept association graph, and inserts logical bridging sentences between adjacent paragraphs; The generation module constructs a dynamic prompt box based on the concatenated paragraphs and the user's original query, including key interaction points and external verification prompts. After generating an initial answer based on the dynamic prompt box through the generation model, it is verified, and the verification result is marked as a verification status in the final answer.

[0034] It should be noted that the knowledge base technology system based on RAG retrieval enhancement provided in the above embodiments and the knowledge base technology method based on RAG retrieval enhancement provided in the above embodiments belong to the same concept. The specific ways in which each module and unit performs operations have been described in detail in the method embodiments, and will not be repeated here. In practical applications, the knowledge base technology system based on RAG retrieval enhancement provided in the above embodiments can be assigned to different functional modules as needed, that is, the internal structure of the system can be divided into different functional modules to complete all or part of the functions described above, and this is not a limitation here.

[0035] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0036] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0037] In this application, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or multiple items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0038] It should be understood that in the various embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0039] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0040] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0041] In the several embodiments provided in this application, it should be understood that the disclosed system can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0042] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0043] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0044] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0045] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A knowledge base technology method based on RAG retrieval enhancement, characterized in that, include: S1: Parse the source document into structured text fragments, input them into a pre-trained semantic encoder to generate semantic vectors, and store them in a vector database; S2: Break down the text fragment into the smallest knowledge unit, extract concepts and behavioral trigger words, construct a concept association graph with concepts as nodes and behavioral trigger words as edges, and save the query summary and answer summary of the preset session in a fixed-length ring structure; S3: After receiving the user's query, generate a semantic vector for the query text, retrieve a preset number of text fragments that are most relevant to the query vector from the vector database, and obtain the vector retrieval results; S4: Extract core concepts from text fragments in the vector retrieval results, and use the core concepts as starting nodes to perform hierarchical diffusion in the concept association graph to obtain graph retrieval results; obtain summaries that match the concepts in the text fragments from a fixed-length circular structure to obtain summary matching results; construct a set of candidate knowledge units based on the vector retrieval results, graph retrieval results, and summary matching results, remove duplicates, and concatenate the original text paragraphs in order of the path depth of the knowledge units in the concept association graph, inserting logical bridging sentences between adjacent paragraphs; S5: Based on the concatenated paragraphs and the user's original query, construct a dynamic prompt box that includes key interaction points and external verification prompts. After generating an initial answer based on the dynamic prompt box using a generative model, perform verification and mark the verification result as a verification status in the final answer.

2. The knowledge base technology method based on RAG retrieval enhancement according to claim 1, characterized in that, In the concept association graph, edges are assigned activity weights and time-series fingerprints, and the update logic includes: By segmenting text fragments according to dependency relationships, multiple smallest knowledge units can be obtained; Identify the set of concept words and the set of behavior trigger words in each knowledge unit. Based on the ratio of the number of concept words to the number of behavior trigger words, calculate the initial correlation degree of concept pairs according to the edit distance decay between concept words. For each edge to be updated in the concept association graph, the original activity weight is decayed exponentially according to a predetermined decay exponent, and the activity weight is updated by accumulating the initial association degree from multiple rounds. The difference between the last updated timestamp and the current processing timestamp is recorded on the same side and merged with the original time-series fingerprint proportionally using an adjustable memory coefficient to generate a new time-series fingerprint. In the concept association graph, if an edge does not exist, the initial association degree and temporal fingerprint are set as the initial activity weight and temporal fingerprint of the newly created edge.

3. The knowledge base technology method based on RAG retrieval enhancement according to claim 2, characterized in that, The query summary and answer summary of the preset session are stored in a fixed-length ring structure, including: A fixed number of recent session rounds are stored in a circular queue. Each round enqueues a query summary and a response summary in pairs. When the queue is full, the first summary is removed from the queue. Calculate the matching degree by performing intersection and union calculations on the concept word set of each summary and the current query; Set the normalized matching degree as a score, correspond it to the summary, and write it back to the circular queue in an atomic transaction manner.

4. The knowledge base technology method based on RAG retrieval enhancement according to claim 1, characterized in that, Step S3 includes: After receiving a user query, the query text is input into a pre-trained semantic encoder to obtain the query semantic vector; Obtain a set of candidate vectors from the vector database with a preset batch size, and calculate a vector retrieval score for each candidate vector; The candidate vectors are sorted in descending order of vector retrieval scores. After comparing with the dynamic score threshold, they are retained in sequence until the preset number of segments is reached. The text segment identifiers corresponding to the retained candidate vectors are then output.

5. The knowledge base technology method based on RAG retrieval enhancement according to claim 4, characterized in that, The construction logic of the candidate knowledge unit set includes: Receive the vector retrieval score corresponding to each text segment in the vector retrieval result set; Using the core concept in the text fragment as the starting node, the concept association graph is diffused in layers according to the preset diffusion depth, and the total graph diffusion influence is obtained based on the edge activity weight and path depth in the diffusion path. Extract summaries that match the concepts of text fragments from a fixed-length circular structure, and obtain the summary injection value based on the multidimensional score of the matching summaries; A comprehensive candidate score is generated based on the vector retrieval score, the total graph diffusion impact, and the sum of the abstract injection value. All text fragments are sorted in descending order of comprehensive candidate scores, and text fragment identifiers that are greater than the score threshold and less than or equal to a preset number are retained to construct a candidate knowledge unit set.

6. The knowledge base technology method based on RAG retrieval enhancement according to claim 5, characterized in that, Calculate the similarity of each unit in the candidate knowledge unit set based on the XOR distance of semantic fingerprint, and remove redundant units whose similarity exceeds a preset threshold; For the remaining units, an innovation score is calculated based on the number of concept words and the shortest path depth in the concept association graph, combined with vector retrieval scores and the total graph diffusion influence. The list of units scored by innovation is first sorted in ascending order by path depth, and then in descending order by innovation score for stable sorting. The original text paragraphs of the sorted units are extracted in sequence and sent to the output buffer. For each pair of adjacent paragraphs, the concept with the highest frequency is selected from the shared concept set. Combined with the difference between the two concepts, a concise bridging sentence is generated through high-order mapping and inserted between the paragraphs to output continuous paragraph text.

7. The knowledge base technology method based on RAG retrieval enhancement according to claim 1, characterized in that, The construction logic for dynamic tooltips includes: Extract keywords from each core knowledge unit in the spliced ​​paragraph and calculate the unit importance based on keyword frequency and paragraph position; Set the most important preset quantity unit as the interactive key point, and generate an index identifier and summary text for each key point; Identify the entities and technical terms contained in all sentences in the concatenated paragraph, calculate the verification demand strength based on entity rarity and sentence length, and mark sentences with verification demand strength greater than a threshold as sentences requiring external verification. Place the original user query at the beginning of the prompt box, and list the generated interaction points and the sequence number of sentences requiring external verification in sequence.

8. The knowledge base technology method based on RAG retrieval enhancement according to claim 7, characterized in that, Extract propositional assertions from the initial responses output by the generative model and construct a list of fact triples; For each fact triple, retrieve the corresponding entity node and path in the concept association graph, and obtain the internal consistency index based on the path and time fingerprint. Parallel calls to external knowledge sources perform evidence retrieval on the same fact triple, and obtain external credibility indicators based on external results and assertion text; A comprehensive verification score is generated based on internal consistency indicators and external credibility indicators; The comprehensive verification score is compared with the preset verification threshold. If the score is greater than or equal to the threshold, it is marked as verified; if the score is less than the threshold, it is marked as pending verification.

9. A knowledge base technology method based on RAG retrieval enhancement according to claim 1, characterized in that, Also includes: Analyze user feedback information to obtain the intensity of summary interaction; For each summary in the circular session memory, a power-law retention operation is performed on the summary score. Logarithmic mapping is performed based on the summary interaction strength signal, and the result is combined with the power-law result to obtain the updated summary score.

10. A knowledge base technology system based on RAG retrieval enhancement, used to implement the knowledge base technology method based on RAG retrieval enhancement as described in any one of claims 1-9, characterized in that, include: Parsing module: Parses the source document into structured text fragments, inputs them into a pre-trained semantic encoder to generate semantic vectors, and stores them in a vector database; Building Module: The text fragments are broken down into the smallest knowledge units, and concepts and behavioral trigger words are extracted. A concept association graph is built with concepts as nodes and behavioral trigger words as edges. The query summary and answer summary of the preset session are stored in a fixed-length circular structure. Query module: After receiving a user query, it generates a semantic vector for the query text, retrieves a preset number of text fragments most relevant to the query vector from the vector database, and obtains vector retrieval results; The splicing module extracts core concepts from text fragments in the vector retrieval results, uses these core concepts as starting nodes to perform hierarchical diffusion in the concept association graph, and obtains graph retrieval results. It also obtains summaries that match the concepts in the text fragments from a fixed-length circular structure, resulting in summary matching results. Based on the vector retrieval results, graph retrieval results, and summary matching results, it constructs a set of candidate knowledge units, removes duplicates, and splices the original text paragraphs sequentially according to the path depth of the knowledge units in the concept association graph, inserting logical bridging sentences between adjacent paragraphs. The generation module constructs a dynamic prompt box based on the concatenated paragraphs and the user's original query, including key interaction points and external verification prompts. After generating an initial answer based on the dynamic prompt box through the generation model, it is verified, and the verification result is marked as a verification status in the final answer.

Images