Knowledge base technical method and system based on RAG retrieval enhancement

Abstract

The invention discloses a knowledge base technical method and system based on RAG retrieval enhancement, and relates to the technical field of knowledge bases. The method comprises the steps that a source document is analyzed into structured text fragments, and semantic vectors are generated; splitting the text segments into minimum knowledge units, extracting concepts and behavior trigger words to construct a concept association graph, and storing session abstracts in a fixed-length annular structure; the method comprises the following steps: receiving user query, generating a query vector, retrieving a most relevant text fragment, constructing and de-duplicating candidate knowledge units by combining hierarchical diffusion of a concept association graph and an annular abstract matching result, deeply splicing original text paragraphs according to a graph path, generating a dynamic prompt box, calling a generative model to generate a preliminary answer, and executing verification. And the verification state is marked on the final answer. By constructing a concept association map, the activeness weight and the time sequence fingerprint of a map edge are updated in real time, and accurate capture of deep semantics and logical relationships of query intentions is realized.

Description

Technical Field

[0001] The present invention relates to the field of knowledge base technology, and in particular to a knowledge base technology method and system based on RAG retrieval enhancement. Background Art

[0002] Existing knowledge base retrieval systems typically rely on keyword matching or methods based on sparse retrieval models, performing word segmentation and Boolean matching on user queries, and then searching for records containing the corresponding terms in static document repositories. This approach struggles to grasp the deeper semantics of query intent, and when faced with synonyms, ambiguous terms, or complex search conditions, the search results are often insufficiently satisfying. Furthermore, traditional retrieval often fails to effectively integrate external real-time data or historical user conversation information with the current query, resulting in answers that lack timeliness and coherence.

[0003] To improve question-answering quality, researchers have recently attempted to combine vectorized semantic retrieval with large-scale language model (LLM) generation, proposing retrieval-augmented generation (RAG) solutions. However, existing RAG implementations often simply concatenate retrieval segments and pass them to a generative model for processing. They lack dynamic scoring of search results and graph-based management of associations. These solutions struggle to understand the temporal nature of document hotspot changes and are unable to continuously optimize retrieval and generation strategies based on user feedback.

[0004] Furthermore, many cloud-based knowledge base services require uploading sensitive corporate documents to third-party platforms, making it difficult to meet stringent data privacy and compliance requirements. Therefore, an innovative technical solution is urgently needed that can generate verifiable and highly reliable answers in a local deployment environment through deep structured document parsing, integrating vector search and concept graph search, and combining multi-dimensional conversational memory with a closed-loop user feedback loop. Summary of the Invention

[0005] Based on the above-mentioned shortcomings of the prior art, the purpose of the present invention is to provide a knowledge base technology method and system based on RAG retrieval enhancement to solve the above-mentioned technical problems.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a knowledge base technology method based on RAG retrieval enhancement, comprising: S1: Parse the source document into structured text fragments, input them into the pre-trained semantic encoder to generate semantic vectors, and store them in the vector database; S2: Split the text fragment into the smallest knowledge units, extract concepts and action triggers, build a concept association graph with concepts as nodes and action triggers as edges, and save the query and answer summaries of the preset session in a fixed-length ring structure; S3: After receiving the user query, generate a semantic vector for the query text and retrieve a preset number of text segments that are most relevant to the query vector from the vector database; S4: Based on the abstracts of the concepts of the query text fragment that match the hierarchical diffusion and ring structures in the concept association graph, the vector retrieval results and the graph retrieval results are merged to construct a set of candidate knowledge units. Duplicates are removed and the original paragraphs are sequentially spliced ​​according to the path depth of the knowledge units in the concept association graph. Logical bridge sentences are inserted between adjacent paragraphs. S5: Based on the concatenated paragraphs and the user's original query, a dynamic prompt box is constructed, including interaction points and external verification prompts. After the generative model generates a preliminary answer based on the dynamic prompt box, it is verified and the verification result is marked on the final answer with a verification status.

[0007] The present invention is further configured such that, in the concept association graph, edges are provided with activity weights and temporal fingerprints, and the update logic includes: Split text segments according to dependency relationships to obtain multiple minimum knowledge units; Identify the concept word set and behavior trigger word set in each knowledge unit, and calculate the initial correlation of concept pairs based on the ratio of the number of concept words to the number of behavior trigger words and the decay of the edit distance between concept words; For each edge to be updated in the concept association graph, the original activity weight is exponentially decayed according to the predetermined decay exponent, and the activity weight is updated based on the cumulative superposition of multiple rounds of initial associations; For the same side, the difference between the last updated timestamp and the current processing timestamp is recorded, and the new time series fingerprint is generated by proportionally merging it with the original time series fingerprint using an adjustable memory coefficient. In the concept association graph, if an edge does not exist, the initial association and temporal fingerprint are set to the initial activity weight and temporal fingerprint of the newly created edge.

[0008] The present invention is further configured to store the query summary and the answer summary of the preset session in a fixed-length ring structure, including: A fixed number of recent conversation rounds are stored in a circular queue. In each round, query summaries and answer summaries are added to the queue in pairs. When the queue is full, the first summary is evicted. Calculate the matching degree by performing intersection and union on each abstract and the concept word set of the current query; The normalized matching degree is set as the score, corresponds to the summary, and is written back to the circular queue in an atomic transaction manner.

[0009] The present invention is further configured such that step S3 includes: After receiving the user query, the query text is input into the pre-trained semantic encoder to obtain the query semantic vector; Obtain a set of candidate vectors from the vector database in a preset batch size and calculate a retrieval score for each candidate vector; The candidate vectors are sorted in descending order according to the search score, and after comparing the dynamic score threshold, they are retained in sequence until the preset number of segments is reached, and the text segment identifiers corresponding to the retained candidate vectors are output.

[0010] The present invention is further configured such that the construction logic of the candidate knowledge unit set includes: Obtain the search score, summary injection value sum, and hierarchical influence total of each text segment in the vector search result set; The retrieval score of each text fragment is increased by a specified exponent, the corresponding total level influence is mapped to the same numerical domain according to the depth attenuation factor, and the sum of the summary injection values ​​is logarithmically mapped and then power-weighted to generate a comprehensive candidate score; All text fragments are sorted in descending order according to the comprehensive candidate scores, and the text fragment identifiers with scores greater than the score threshold and less than or equal to the preset number are retained to construct a set of candidate knowledge units.

[0011] The present invention is further configured to calculate similarity of each unit in the candidate knowledge unit set based on the XOR distance of the semantic fingerprint, and remove redundant units whose similarity exceeds a preset threshold; For the remaining units, the innovation score is calculated based on the number of concept words and the shortest path depth in the concept association graph, combined with the vector retrieval score and graph diffusion influence; The unit list that has been scored by innovation is first sorted in ascending order by path depth, and then stably sorted in descending order by innovation score. The original paragraphs of the sorted units are extracted in sequence to the output buffer. For each pair of adjacent paragraphs, the concept with the highest frequency is selected from the shared concept set. The difference between the two concepts is combined through high-order mapping to generate a concise bridge sentence that is inserted between the paragraphs, and the continuous paragraph text is output.

[0012] The present invention is further configured such that the construction logic of the dynamic prompt box includes: Extract keywords from each core knowledge unit in the spliced ​​paragraphs and calculate the unit importance based on keyword frequency and paragraph position; Set a preset number of units with the highest importance as interaction points, and generate an index mark and summary text for each point; Identify the entities and technical terms contained in all sentences in the concatenated paragraphs, calculate the verification requirement strength based on the entity rarity and sentence length, and mark sentences with verification requirement strength greater than a threshold as requiring external verification. Place the original user query at the head of the prompt box, and list the generated interaction points and the sequence numbers of the sentences requiring external verification in sequence.

[0013] The present invention is further configured to extract propositional assertions from the preliminary answers output by the generative model and construct a list of fact triples; For each fact triple, the corresponding entity node and path are retrieved in the concept association graph, and the internal consistency index is obtained based on the path and temporal fingerprint; Parallel calls to external knowledge sources to perform evidence retrieval on the same fact triples, and obtain external credibility indicators based on external results and assertion original text; Generate a composite validation score based on internal consistency indicators and external credibility indicators; The comprehensive verification score is compared with the preset verification threshold. If it is greater than or equal to the threshold, it is marked as verified; if it is less than the threshold, it is marked as pending verification.

[0014] The present invention is further configured to include: Parse user feedback information to obtain summary interaction strength; For each summary in the ring conversation memory, a power-preserving operation is performed on the summary score, logarithmically mapped according to the summary interaction strength signal, and combined with the power result to obtain the updated summary score.

[0015] The present invention also provides a knowledge base technology system based on RAG retrieval enhancement, which is used to implement the above-mentioned knowledge base technology method based on RAG retrieval enhancement. The system includes: Parsing module: parses the source document into structured text fragments, inputs them into a pre-trained semantic encoder to generate semantic vectors, and stores them in a vector database; Construction module: Split text snippets into minimal knowledge units, extract concepts and action triggers, construct a concept association graph with concepts as nodes and action triggers as edges, and save the query and answer summaries of the preset session in a fixed-length ring structure; Query module: After receiving a user query, it generates a semantic vector for the query text and retrieves a preset number of text segments that are most relevant to the query vector from the vector database; The splicing module: Based on the abstracts matching the concepts of the query text fragment in the hierarchical diffusion and ring structure of the concept association graph, it merges the vector retrieval results and the graph retrieval results to construct a set of candidate knowledge units, removes duplicates, and splices the original paragraphs in sequence according to the path depth of the knowledge units in the concept association graph, inserting logical bridge sentences between adjacent paragraphs. Generation module: Based on the spliced ​​paragraphs and the user's original query, a dynamic prompt box is constructed, including interaction points and external verification prompts. After the generation model generates a preliminary answer based on the dynamic prompt box, it is verified and the verification result is marked on the final answer with a verification status.

[0016] The present invention provides a knowledge base technology method and system based on RAG retrieval enhancement. The method parses source documents into structured text fragments, inputs them into a pre-trained semantic encoder to generate semantic vectors, and stores them in a vector database; splits the text fragments into minimum knowledge units, extracts concepts and behavior trigger words, constructs a concept association graph with concepts as nodes and behavior trigger words as edges, and saves query summaries and answer summaries of preset sessions in a fixed-length ring structure; after receiving a user query, generates a semantic vector for the query text, and retrieves a preset number of text fragments most relevant to the query vector from the vector database; merges vector retrieval results and graph retrieval results based on the hierarchical diffusion and ring structure matching summaries of the concepts of the query text fragments in the concept association graph, constructs a set of candidate knowledge units, removes duplicates, sequentially splices original text paragraphs according to the path depth of the knowledge units in the concept association graph, and inserts logical bridging sentences between adjacent paragraphs; constructs a dynamic prompt box including interaction points and external verification prompts based on the spliced ​​paragraphs and the user's original query, generates a preliminary answer based on the dynamic prompt box through a generative model, and then verifies it, and annotates the verification result with a verification status on the final answer. The beneficial effects produced include: 1. Significantly improved retrieval accuracy: Through vectorized retrieval and concept association graph retrieval, the deep semantics and logical relationships of query intent are accurately captured, effectively reducing interference from irrelevant information; 2. Significantly enhanced answer credibility: Multi-dimensional conversation memory and triangulation verification are introduced to ensure that generated answers are supported by historical context and verified and annotated by multiple sources, reducing the risk of "hallucination"; 3. Dynamic time series perception capability: The activity weights and time series fingerprints of the edges in the concept association graph are updated in real time, which can automatically capture changes in document hot spots, so that the search results can smoothly connect new and old information and highlight hot content.

[0017] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without inventive efforts. In the drawings: Figure 1 A flowchart of a knowledge base technology method based on RAG retrieval enhancement is shown as an exemplary embodiment of the present invention; Figure 2The diagram is a structural diagram of a knowledge base technology system based on RAG retrieval enhancement, showing an exemplary embodiment of the present invention. DETAILED DESCRIPTION

[0019] The following describes the embodiments of the present invention with reference to the accompanying drawings and preferred embodiments. Those skilled in the art will readily appreciate the other advantages and benefits of the present invention from the disclosure herein. The present invention may also be implemented or applied through various other specific embodiments, and the various details in this specification may be modified or altered based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred embodiments are intended only to illustrate the present invention and are not intended to limit the scope of protection of the present invention.

[0020] It should be noted that the illustrations provided in the following embodiments are merely schematic illustrations of the basic concept of the present invention. Therefore, the illustrations only show components related to the present invention and are not drawn according to the number, shape, and size of components in actual implementation. In actual implementation, the type, quantity, and proportion of each component may be changed arbitrarily, and the component layout may also be more complex.

[0021] In the following description, numerous details are discussed to provide a more thorough explanation of the embodiments of the present invention. However, it will be apparent to those skilled in the art that the embodiments of the present invention may be practiced without these specific details. In other embodiments, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring the embodiments of the present invention.

[0022] Example 1 A knowledge base technology method based on RAG retrieval enhancement, such as Figure 1 As shown, including: S1: Parse the source document into structured text fragments, input them into the pre-trained semantic encoder to generate semantic vectors, and store them in the vector database; S2: Split the text fragment into the smallest knowledge units, extract concepts and action triggers, build a concept association graph with concepts as nodes and action triggers as edges, and save the query and answer summaries of the preset session in a fixed-length ring structure; S3: After receiving the user query, generate a semantic vector for the query text and retrieve a preset number of text segments that are most relevant to the query vector from the vector database; S4: Based on the abstracts of the concepts of the query text fragment that match the hierarchical diffusion and ring structures in the concept association graph, the vector retrieval results and the graph retrieval results are merged to construct a set of candidate knowledge units. Duplicates are removed and the original paragraphs are sequentially spliced ​​according to the path depth of the knowledge units in the concept association graph. Logical bridge sentences are inserted between adjacent paragraphs. S5: Based on the concatenated paragraphs and the user's original query, a dynamic prompt box is constructed, including interaction points and external verification prompts. After the generative model generates a preliminary answer based on the dynamic prompt box, it is verified and the verification result is marked on the final answer with a verification status.

[0023] Specifically, in step S1, the source document is input into the parser. The source document includes text. The original text is divided into several structured text segments with context labels. The context labels include titles, paragraphs, lists, table items, and metadata location identifiers. The above structured text segments are used as input, and the fine-tuned pre-trained semantic encoder is called, including a deep bidirectional language model based on Transformer, to extract the high-dimensional semantic vector of each segment. The semantic vector represents the position of the segment in the semantic space in the form of a continuous real number matrix, and captures contextual dependencies and implicit topics through the self-attention mechanism; each generated semantic vector and its corresponding text segment identifier are encapsulated as an index item. The text segment identifier includes a document identifier, a segment number, and location information, and is written in batches to the local vector database. The database constructs a multi-level index structure based on the approximate nearest neighbor algorithm; In step S2, relying on syntactic analysis and semantic boundary detection, each structured text fragment is segmented into the smallest knowledge unit, which usually corresponds to a complete semantic sentence or paragraph fragment. After segmentation, each unit can independently carry complete concept-behavior information, avoiding the ambiguity caused by cross-sentence semantic fusion; named entity recognition and part-of-speech tagging are applied to each knowledge unit, and all nouns or terms are extracted as concepts, combined with verb phrases or action phrases to identify them as behavior trigger words, and a pair of concept sets and behavior sets are obtained, laying the foundation for graph construction; each concept is mapped to a graph node, and the corresponding behavior trigger words are mapped to edges connecting the nodes. For all concept pairs in a knowledge unit, an edge is created or updated, and the latest activity index and time series fingerprint are recorded on the edge to reflect the frequency and time distribution of the association in local documents and historical queries; a preset number of recent conversation rounds are saved in the form of a circular queue, and each round is queued with a "query summary-answer summary" pair. When the queue is full, the earliest entry is eliminated to always keep the latest historical context. Each summary is accompanied by a matching score; The present invention is further configured such that, in the concept association graph, edges are provided with activity weights and temporal fingerprints, and the update logic includes: Segment text fragments according to dependency relationships to obtain multiple minimum knowledge units. Specifically, perform syntactic dependency analysis on each structured text fragment, segmenting it into several minimum knowledge units based on verbs and core semantic points, ensuring that each unit has complete internal semantics and can independently reflect the correspondence between concepts and behaviors. Identify the concept word set and behavior trigger word set in each knowledge unit, and calculate the initial correlation between concept pairs based on the ratio of the number of concept words to the number of behavior trigger words and the edit distance decay between concept words; specifically, for the first Any binary concept pair in a knowledge unit and behavioral trigger word collection Calculate the initial correlation : , is the number of concept words in the knowledge unit, is the number of behavioral trigger words in the knowledge unit; lev Concept words and The edit distance between them is calculated; the ratio of the total number of concepts to the number of behaviors + 1 is raised to the power of two, significantly amplifying the contribution of high-concept density units while avoiding division by zero errors caused by zero behavior counts; the edit distance is halved using an exponential function, with the correlation multiplied by approximately 0.607 for every increase in the distance; this effectively distinguishes similar and dissimilar concepts; For each edge to be updated in the concept association graph, the original activity weight is exponentially decayed according to the established decay exponent, and the activity weight is updated based on the cumulative superposition of multiple rounds of initial associations; specifically, for each edge to be updated in the concept association graph, (connect and ), first adopt historical forgetfulness and then integrate new associations: ; is the activity weight before the update; : The difference between the timestamp of this update and the last update, in days; The initial correlation degree accumulation from all new knowledge units; the original weight is nonlinearly decayed with a power of 0.9, and then combined with the exponential factor Performing time-order fading, a memory mechanism with recent priority and historical decrement is constructed. The initial relevance of all knowledge units to the edge is accumulated, and the sum of the two parts is set as the final activity. This balances the fading of old relevance and the strengthening of new relevance, allowing the graph to continuously capture the evolution of hot spots. For the same edge, the difference between the last updated timestamp and the current processing timestamp is recorded, and the new time series fingerprint is generated by proportionally merging it with the original time series fingerprint using an adjustable memory coefficient. Specifically, a time series fingerprint is maintained for each edge. , to characterize the time distribution: ; is the temporal fingerprint before updating; is the fingerprint memory coefficient; The time difference is the same as the activity update; the historical time series fingerprint is linearly retained with a memory retention coefficient of 0.8 to simulate the gradual decay of the old time distribution; then the time difference is Raising it to the second power highlights the strengthening effect of longer time intervals on freshness fingerprints, and a new temporal fingerprint is obtained by proportional superposition; If an edge does not exist in the concept association graph, the initial association degree and temporal fingerprint are set as the initial activity weight and temporal fingerprint of the newly created edge; specifically, if a pair of concepts is found to have no edge connection in the concept association graph, the initial association degree and preliminary temporal fingerprint of the pair in the current unit are used together as the initial activity weight and temporal fingerprint of the newly created edge to ensure that the first concept association has basic support.

[0024] The present invention is further configured to store the query summary and the answer summary of the preset session in a fixed-length ring structure, including: A fixed number of recent conversation rounds are stored in a circular queue. In each round, query summaries and answer summaries are added to the queue in pairs. When the queue is full, the first summary is removed. Specifically, a fixed length Build a circular queue to store the most recent Each time a new round of summary pairs is added to the queue, if the queue is full, the first element is automatically removed to ensure that the latest context is always retained in the queue; The intersection and union of each abstract and the concept word set of the current query are used to calculate the matching degree; specifically, the concept word set of each abstract in the team The concept word set related to the current query , first find the size of the intersection and the size of the union, then press Calculate matching degree The intersection of the concept word sets extracted from the abstract and the query is used to obtain the number of shared concepts, reflecting the degree of semantic overlap between the two. The number of shared concepts is raised to the power of two to nonlinearly amplify the matching effect between abstracts and queries with high overlap, so that the matching degree increases sharply when the key information is fully covered. The total number of concepts in the union is used as the divisor to normalize the matching degree to the range [0, +∞) to ensure that the matching degree is comparable between collections of different sizes. The normalized matching degree is set as a score, which is matched with the summary and written back to the circular queue in an atomic transaction mode; specifically, the matching degree of each summary is set as Set it as a score, correspond to the summary, and write it back to the corresponding slot in the circular queue in an atomic transaction to ensure update consistency.

[0025] The present invention is further configured such that step S3 includes: After receiving the user query, the query text is input into the pre-trained semantic encoder to obtain the query semantic vector; specifically, the query text input by the user is passed to the aforementioned pre-trained semantic encoder, and the query semantic vector is output. , semantic vector Locate the core features of query intent in high-dimensional semantic space and provide a basic representation for similarity retrieval; Obtain a set of candidate vectors in the vector database with a preset batch size, and calculate the search score for each candidate vector; specifically, according to the system preset batch size , parallel acquisition from the vector database Most likely related Candidate vectors and its corresponding timing factor (Indicates the newness of the content). Then calculate the retrieval score for each candidate vector : , The vector distance is nonlinearly attenuated. For every increase in distance, the similarity is multiplied by approximately exp(-0.2)≈0.82. When the distance is large, it quickly approaches zero, ensuring that only vectors with similar semantics can obtain significant scores. Time series factor The power mapping moderately amplifies the priority of recent content. (latest), this item is 1.5; when (oldest), which is 1.0, ensures that new and old content are distinguishable without completely losing historical information. The final score is obtained by multiplying the two parts, which can accurately assess semantic similarity while taking into account the newness of the content, thereby achieving efficient candidate vector sorting; The candidate vectors are sorted in descending order according to the search score, and after comparing the dynamic score threshold, they are retained in order until the preset number of segments is reached, and the text segment identifiers corresponding to the retained candidate vectors are output; specifically, all After sorting in descending order, follow the dynamic threshold Compare and keep only the scores above the threshold and no more than the preset number Before Finally, the text segment identifiers corresponding to these candidate vectors are output as input for subsequent retrieval enhancement generation. The score retention ratio threshold is in the range of [0.6, 0.9].

[0026] In step S4, for each text segment returned by the vector search, identify the core concept word set in it And mark each concept word as the starting level node in the global concept association graph to form a hierarchical mapping queue ; Expand each level mapping node in turn according to the preset maximum diffusion depth: each time from the current node Spread to all its directly adjacent nodes ; Calculate path impact value , which is the cumulative multiplication of the activity weights of all edges on the current path after exponentiation, and then multiplied by the depth index of the hierarchical attenuation factor; after the diffusion is completed, the influence value of each concept word on all paths is summarized to obtain the total hierarchical influence of each concept: , Concept Node The total amount of hierarchical impact, For all queries from the core to A collection of simple paths, each path Contains edge sequence , ,…, And the length does not exceed the maximum depth , is the path length, For the path The activity weight of the edge; the concept word set of all summaries in the ring memory and the current text segment Perform an intersection operation and calculate the summary injection value for each matching summary, which is equal to the power mapping of the number of matching concepts multiplied by the multidimensional score of the summary, and then divided by the exponential function of the total number of concepts in the segment: , the concept set of the current text segment With all the summaries in the ring memory , calculate the single injection value and add them up to get the total , Injects the sum of the values ​​into the digest, For the A collection of concepts for abstracts, The score of the abstract; further, the score of the abstract is specifically the matching degree of each abstract , which has been explained in detail in the previous article and will not be repeated here.

[0027] The present invention is further configured such that the construction logic of the candidate knowledge unit set includes: Obtain the retrieval score, summary injection value sum and hierarchical influence of each text segment in the vector retrieval result set; specifically, receive the retrieval score of each candidate vector in each text segment in the vector retrieval result set , add and average to get the retrieval score of each text segment , Sum of summary injection values The path influence obtained by hierarchical diffusion of text fragments in the concept association graph ; The retrieval score of each text fragment is increased by a specified exponent, the corresponding total level influence is mapped to the same numerical domain according to the depth attenuation factor, the sum of the summary injection values ​​is logarithmically mapped and then weighted by power to generate a comprehensive candidate score; specifically, for each fragment , based on the retrieval score , Sum of summary injection values and path influence , calculate the candidate score ,in, For fragments Candidate ratings for Taking a quadratic power significantly spreads out high similarity scores after mapping, prioritizing the fragments closest to the query. First, exponential decay is performed based on path depth, and then exponentially strengthens high-impact scores, balancing shallow and deep associations. Taking the logarithm of the historical match score to suppress extreme values, and then taking a quadratic power to highlight the contribution of highly credible summaries to the candidate. The product of these three factors simultaneously reflects semantic relevance, logical support, and historical credibility, forming a unified comprehensive score. Arrange all text fragments in descending order according to the comprehensive candidate scores, retain the text fragment identifiers that are greater than the score threshold and less than or equal to the preset number, and construct a candidate knowledge unit set; specifically, Sort in descending order and filter out the items that meet the requirements Before the score threshold The text fragments are identified to form the final candidate knowledge unit set, and the set and corresponding value.

[0028] The present invention is further configured to calculate the similarity of each unit in the candidate knowledge unit set based on the XOR distance of the semantic fingerprint, and remove the redundant units whose similarity exceeds the preset threshold; specifically, for any two units Get its binary semantic fingerprint vector , calculate the Hamming distance And generate similarity scores based on power and exponential mapping ,like Greater than the preset threshold (0.9), then the one with the lower initial value of innovation is eliminated, among which, is the Hamming distance of binary vectors; perform bitwise XOR on two fingerprint vectors and count the number of 1s in the result to get the Hamming distance, which reflects the degree of dispersion of the two units in the semantic fingerprint space; For the remaining units, the innovation score is calculated based on the number of concept words and the shortest path depth in the concept association graph, combined with the vector retrieval score and the graph diffusion effect; specifically, for each remaining unit Generate innovation scores based on the parallel combination of four-dimensional indicators : ,in, is the number of concept words in the unit; is the shortest path length from the unit to the query core in the concept graph; Score vector retrieval; The total amount of graph layer diffusion influence is calculated; the number of concept words is raised to the power of two to significantly improve the innovation score of information-rich units; the number of shortest path hops is used as the denominator of the first power to ensure that the farther the unit is from the query core, the lower the innovation score; the vector retrieval score is raised to the power of two to further distinguish between high-similarity and medium-similarity units; the graph diffusion influence is raised to the power of 1.5 to nonlinearly enhance units connected to multiple high-weight paths; The unit list with innovation score is first sorted in ascending order by path depth, and then in descending order by innovation score, and the original paragraphs of the sorted units are extracted in sequence to the output buffer. For each pair of adjacent paragraphs, the concept with the highest frequency is selected from the shared concept set, and a concise bridge sentence is generated by high-order mapping based on the difference between the two concepts and inserted between the paragraphs, and the continuous paragraph text is output; specifically, the original paragraphs are extracted in sequence according to the sorting results and appended to the output buffer; for adjacent paragraph pairs, the concept with the highest frequency is selected from the shared concept set, and the concept with the highest frequency is inserted between the paragraphs through high-order mapping, and the continuous paragraph text is output; specifically, the original paragraphs are extracted in sequence according to the sorting results and appended to the output buffer; for adjacent paragraph pairs, the concept with the highest frequency is selected from the shared concept set, and the concept with the highest frequency is inserted between the paragraphs. Extracting shared concept sets Differentiation from the concept . Select the shared concept with the highest frequency , generate bridge sentences.

[0029] The present invention is further configured such that the construction logic of the dynamic prompt box includes: Extract keywords from each core knowledge unit in the spliced ​​paragraphs and calculate the unit importance based on keyword frequency and paragraph position. Specifically, first extract the keyword set from the spliced ​​context paragraphs for each core knowledge unit and count the frequency of each keyword in the unit. At the same time, record the unit's position in the paragraph, including the first paragraph (1.2), the middle paragraph (1.0), and the last paragraph (0.8). Use keyword frequency as the main measure of information density, use position weight to reflect user attention preferences, and calculate the importance of each unit. A higher importance means that the unit content is more concentrated and closer to the beginning of the paragraph, making it more likely to become the focus of user interaction. Set a preset number of units with the highest importance as interactive points, and generate an index identifier and summary text for each key point. Specifically, select the top preset number of units according to the aforementioned importance value ranking as interactive points. Generate a unique index identifier for each key point (e.g., "Key Point 1," "Key Point 2") and extract the first sentence or core summary text of the unit to form a key point list entry, making it easier for users to quickly locate and expand the corresponding content when clicking or quoting. Identify the entities and technical terms contained in all sentences in the spliced ​​paragraph, calculate the verification requirement strength based on the entity rarity and sentence length, and mark sentences with verification requirement strength greater than a threshold as sentences requiring external verification; specifically, split the spliced ​​paragraph into independent sentences, perform entity recognition and technical term extraction on each sentence, and then evaluate the verification requirement strength based on the entity rarity of the identified entity in the entire knowledge base and the character or vocabulary length of the sentence. If the strength exceeds a preset threshold, it is determined that the sentence may contain model hallucinations or the latest dynamic information needs, marked as requiring external verification, and its serial number in the paragraph is recorded. The present invention is further configured to quantitatively calculate the entity rarity by the frequency of occurrence; Place the original user query at the top of the prompt box, then list the generated interaction points and the numbers of the sentences requiring external verification. Specifically, construct a structured prompt box data object, placing the user's original query text at the top as the question context identifier. Below this object, list the interaction points (including the key point index and corresponding summary) and the numbers of the sentences requiring external verification. Also included are fields indicating the verification interface type or external resource call method. This structured object is output in JSON or a similar data format for model generation or front-end interface call rendering.

[0030] The prompt box construction logic automatically generates interactive key points by sorting the importance of key units, significantly improving the efficiency of users in locating the information they need. At the same time, it automatically marks potential uncertainties or emerging facts as requiring external verification to ensure the transparency of generated answers and provide precise trigger points for downstream calls to external data interfaces.

[0031] The present invention further extracts propositional assertions from the initial answers generated by the generative model and constructs a list of fact triples. Specifically, the natural language answers generated by the generative model are deeply parsed to identify the propositional assertions, and each assertion is structured into a list of fact triples in the form of <subject, predicate, object>. This structuring process ensures that the subsequent verification module can process various factual statements in a unified format. For each fact triple, the corresponding entity node and path are retrieved in the concept association graph, and the internal consistency index is obtained based on the path and temporal fingerprint. Specifically, for each fact triple in the list, the nodes corresponding to the subject and object of the triple are first located in the concept association graph, and then all paths connecting these two nodes are retrieved in the graph. The internal consistency index is evaluated based on the activity weight and temporal fingerprint distribution of these paths. This index reflects the degree of support for the facts described in the answer and the existing knowledge graph. Parallel calls to external knowledge sources perform evidence retrieval for the same fact triple, obtaining external credibility indicators based on the external results and the original assertion text. Specifically, external evidence retrieval is triggered for each triple in parallel, extracting corresponding facts or data from authoritative APIs, document libraries, or knowledge bases. The external evidence is matched with the original assertion text and the differences are evaluated to form external credibility indicators to quantify the reliability of the real-world basis for the assertion in the answer. Generate a comprehensive verification score based on internal consistency indicators and external credibility indicators. Specifically, the internal consistency indicators are fused with the external credibility indicators to obtain a comprehensive verification score for each triple. The fusion strategy considers both the strength of internal graph support and the degree of match verified by external data to ensure that the score can balance the contributions of both. The comprehensive verification score is compared against a preset verification threshold. Those above the threshold are marked as verified, while those below are marked as pending verification. This annotation information is then integrated into the final answer text. Specifically, the comprehensive verification score is compared against a preset verification threshold. Those above the threshold are marked as "verified," while those below are marked as "pending verification." The verification annotations for each assertion are then sequentially integrated back into the original answer text, ensuring that the final output not only includes the answer content but also explicitly displays the verification status of each assertion, thereby improving transparency and auditability.

[0032] The present invention is further configured to include: Parse user feedback to obtain summary interaction strength. Specifically, collect the number of user clicks on each summary in the prompt box and map it to a summary interaction strength signal. For each summary in the circular conversation memory, a power-preserving operation is performed on the summary score, logarithmic mapping is performed based on the summary interaction strength signal, and the result is combined with the power-preserving result to obtain an updated summary score. Specifically, for each summary in the circular conversation memory, a power-preserving operation is first performed on its existing score to preserve historical evaluation differences, and then logarithmic mapping is applied based on the summary interaction strength signal and the mapping result is combined with the power-preserving result to form a new summary credibility score, so that high-value summaries that users frequently click on will receive higher priority in subsequent searches.

[0033] Example 2 See also Figure 2 The exemplary RAG-based retrieval enhancement-based knowledge base technology system is used to implement the aforementioned RAG-based retrieval enhancement-based knowledge base technology method, including: Parsing module: parses the source document into structured text fragments, inputs them into a pre-trained semantic encoder to generate semantic vectors, and stores them in a vector database; Construction module: Split text snippets into minimal knowledge units, extract concepts and action triggers, construct a concept association graph with concepts as nodes and action triggers as edges, and save the query and answer summaries of the preset session in a fixed-length ring structure; Query module: After receiving a user query, it generates a semantic vector for the query text and retrieves a preset number of text segments that are most relevant to the query vector from the vector database; The splicing module: Based on the abstracts matching the concepts of the query text fragment in the hierarchical diffusion and ring structure of the concept association graph, it merges the vector retrieval results and the graph retrieval results to construct a set of candidate knowledge units, removes duplicates, and splices the original paragraphs in sequence according to the path depth of the knowledge units in the concept association graph, inserting logical bridge sentences between adjacent paragraphs. Generation module: Based on the spliced ​​paragraphs and the user's original query, a dynamic prompt box is constructed, including interaction points and external verification prompts. After the generation model generates a preliminary answer based on the dynamic prompt box, it is verified and the verification result is marked on the final answer with a verification status.

[0034] It should be noted that the knowledge base technology system based on RAG retrieval enhancement provided by the above embodiment and the knowledge base technology method based on RAG retrieval enhancement provided by the above embodiment belong to the same concept, wherein the specific manner in which each module and unit performs operations has been described in detail in the method embodiment and will not be repeated here. In actual applications, the knowledge base technology system based on RAG retrieval enhancement provided by the above embodiment can allocate the above functions to different functional modules as needed, that is, divide the internal structure of the system into different functional modules to complete all or part of the functions described above, and this is not limited here.

[0035] The above embodiments can be implemented in whole or in part by software, hardware, firmware or any other combination. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer program are loaded or executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center via a wired (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state drive.

[0036] It should be understood that the term "and / or" as used herein simply describes a relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A alone, A and B together, or B alone. A and B can be singular or plural. Furthermore, the character " / " as used herein generally indicates an "or" relationship between the associated objects, but it may also indicate an "and / or" relationship. For specific understanding, please refer to the context.

[0037] In this application, "at least one" means one or more, and "plurality" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or plural.

[0038] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0039] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0040] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0041] In the several embodiments provided in this application, it should be understood that the disclosed system can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0042] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0043] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0044] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0045] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A knowledge base technology method based on RAG retrieval enhancement, characterized in that: include: S1: Parse the source document into structured text fragments, input them into the pre-trained semantic encoder to generate semantic vectors, and store them in the vector database; S2: Split the text fragment into the smallest knowledge units, extract concepts and action triggers, build a concept association graph with concepts as nodes and action triggers as edges, and save the query and answer summaries of the preset session in a fixed-length ring structure; S3: After receiving the user query, generate a semantic vector for the query text and retrieve a preset number of text segments that are most relevant to the query vector from the vector database; S4: Based on the abstracts of the concepts of the query text fragment that match the hierarchical diffusion and ring structures in the concept association graph, the vector retrieval results and the graph retrieval results are merged to construct a set of candidate knowledge units. Duplicates are removed and the original paragraphs are sequentially spliced ​​according to the path depth of the knowledge units in the concept association graph. Logical bridge sentences are inserted between adjacent paragraphs. S5: Based on the concatenated paragraphs and the user's original query, a dynamic prompt box is constructed, including interaction points and external verification prompts. After the generative model generates a preliminary answer based on the dynamic prompt box, it is verified and the verification result is marked on the final answer with a verification status.

2. The knowledge base technology method based on RAG retrieval enhancement according to claim 1 is characterized in that: In the concept association graph, edges are set with activity weights and temporal fingerprints. The update logic includes: Split text segments according to dependency relationships to obtain multiple minimum knowledge units; Identify the concept word set and behavior trigger word set in each knowledge unit, and calculate the initial correlation of concept pairs based on the ratio of the number of concept words to the number of behavior trigger words and the decay of the edit distance between concept words; For each edge to be updated in the concept association graph, the original activity weight is exponentially decayed according to the predetermined decay exponent, and the activity weight is updated based on the cumulative superposition of multiple rounds of initial associations; For the same side, the difference between the last updated timestamp and the current processing timestamp is recorded, and the new time series fingerprint is generated by proportionally merging it with the original time series fingerprint using an adjustable memory coefficient. In the concept association graph, if an edge does not exist, the initial association and temporal fingerprint are set to the initial activity weight and temporal fingerprint of the newly created edge.

3. The knowledge base technology method based on RAG retrieval enhancement according to claim 2 is characterized in that: Save the query summary and answer summary of the preset session in a fixed-length ring structure, including: A fixed number of recent conversation rounds are stored in a circular queue. In each round, query summaries and answer summaries are added to the queue in pairs. When the queue is full, the first summary is evicted. Calculate the matching degree by performing intersection and union on each abstract and the concept word set of the current query; The normalized matching degree is set as the score, corresponds to the summary, and is written back to the circular queue in an atomic transaction manner.

4. The knowledge base technology method based on RAG retrieval enhancement according to claim 1 is characterized in that: Step S3 includes: After receiving the user query, the query text is input into the pre-trained semantic encoder to obtain the query semantic vector; Obtain a set of candidate vectors from the vector database in a preset batch size and calculate a retrieval score for each candidate vector; The candidate vectors are sorted in descending order according to the search score, and after comparing the dynamic score threshold, they are retained in sequence until the preset number of segments is reached, and the text segment identifiers corresponding to the retained candidate vectors are output.

5. The knowledge base technology method based on RAG retrieval enhancement according to claim 1 is characterized in that: The construction logic of the candidate knowledge unit set includes: Obtain the search score, summary injection value sum, and hierarchical influence total of each text segment in the vector search result set; The retrieval score of each text fragment is increased by a specified exponent, the corresponding total level influence is mapped to the same numerical domain according to the depth attenuation factor, the sum of the summary injection values ​​is logarithmically mapped and then power-weighted to generate a comprehensive candidate score; All text fragments are sorted in descending order according to the comprehensive candidate scores, and the text fragment identifiers with scores greater than the score threshold and less than or equal to the preset number are retained to construct a set of candidate knowledge units.

6. The knowledge base technology method based on RAG retrieval enhancement according to claim 5 is characterized in that: Calculate the similarity of each unit in the candidate knowledge unit set based on the XOR distance of the semantic fingerprint, and remove redundant units whose similarity exceeds the preset threshold; For the remaining units, the innovation score is calculated based on the number of concept words and the shortest path depth in the concept association graph, combined with the vector retrieval score and graph diffusion influence; The unit list that has been scored by innovation is first sorted in ascending order by path depth, and then stably sorted in descending order by innovation score. The original paragraphs of the sorted units are extracted in sequence to the output buffer. For each pair of adjacent paragraphs, the concept with the highest frequency is selected from the shared concept set. The difference between the two concepts is combined through high-order mapping to generate a concise bridge sentence that is inserted between the paragraphs, and the continuous paragraph text is output.

7. The knowledge base technology method based on RAG retrieval enhancement according to claim 1 is characterized in that: The construction logic of the dynamic prompt box includes: Extract keywords from each core knowledge unit in the spliced ​​paragraphs and calculate the unit importance based on keyword frequency and paragraph position; Set a preset number of units with the highest importance as interaction points, and generate an index mark and summary text for each point; Identify the entities and technical terms contained in all sentences in the concatenated paragraphs, calculate the verification requirement strength based on the entity rarity and sentence length, and mark sentences with verification requirement strength greater than a threshold as requiring external verification. Place the original user query at the head of the prompt box, and list the generated interaction points and the sequence numbers of the sentences requiring external verification in sequence.

8. The knowledge base technology method based on RAG retrieval enhancement according to claim 7 is characterized in that: Extract the propositional assertions from the output of the generative model and construct a list of fact triples; For each fact triple, the corresponding entity node and path are retrieved in the concept association graph, and the internal consistency index is obtained based on the path and temporal fingerprint; Parallel calls to external knowledge sources to perform evidence retrieval on the same fact triples, and obtain external credibility indicators based on external results and assertion original text; Generate a composite validation score based on internal consistency indicators and external credibility indicators; The comprehensive verification score is compared with the preset verification threshold. If it is greater than or equal to the threshold, it is marked as verified; if it is less than the threshold, it is marked as pending verification.

9. The knowledge base technology method based on RAG retrieval enhancement according to claim 1 is characterized in that: Also includes: Parse user feedback information to obtain summary interaction strength; For each summary in the ring conversation memory, a power-preserving operation is performed on the summary score, logarithmically mapped according to the summary interaction strength signal, and combined with the power result to obtain the updated summary score.

10. A knowledge base technology system based on RAG retrieval enhancement, used to implement the knowledge base technology method based on RAG retrieval enhancement according to any one of claims 1 to 9, characterized in that: include: Parsing module: parses the source document into structured text fragments, inputs them into a pre-trained semantic encoder to generate semantic vectors, and stores them in a vector database; Construction module: Split text snippets into minimal knowledge units, extract concepts and action triggers, construct a concept association graph with concepts as nodes and action triggers as edges, and save the query and answer summaries of the preset session in a fixed-length ring structure; Query module: After receiving a user query, it generates a semantic vector for the query text and retrieves a preset number of text segments that are most relevant to the query vector from the vector database; The splicing module: Based on the abstracts matching the concepts of the query text fragment in the hierarchical diffusion and ring structure of the concept association graph, it merges the vector retrieval results and the graph retrieval results to construct a set of candidate knowledge units, removes duplicates, and splices the original paragraphs in sequence according to the path depth of the knowledge units in the concept association graph, inserting logical bridge sentences between adjacent paragraphs. Generation module: Based on the spliced ​​paragraphs and the user's original query, a dynamic prompt box is constructed, including interaction points and external verification prompts. After the generation model generates a preliminary answer based on the dynamic prompt box, it is verified and the verification result is marked on the final answer with a verification status.

Images