Transform deep learning-based multi-hop RAG information reasoning method and system

Abstract

The invention provides a multi-hop RAG information reasoning method and system based on transform deep learning. The method comprises the following steps: S101, carrying out data collection in a specific field through modes of automatic capture, API integration, database integration and manual input; s102, aiming at requirements and characteristics of a specific field, a multi-hop RAG model architecture is constructed in a customized manner, and a Transform deep learning architecture is adopted as a core construction module of the model; s103, a knowledge graph of the specific field is constructed, and the knowledge graph comprises entities, relationships and attribute information and is used for representing concepts and associations in the specific field; s104, training the collected data model architecture by using a pre-training strategy, and optimizing the parameters and the structure of the model; and S105, regularly evaluating the model through accuracy and recall rate indexes.

Description

Technical Field

[0001] The present invention relates to retrieval-augmented generation (RAG), and in particular to a multi-hop RAG information reasoning method and system based on transformer deep learning. Background Art

[0002] In today's rapidly developing information age, more and more paper documents are being digitized, and even many businesses and institutions are promoting paperless offices. With the widespread adoption of paperless offices, a series of drawbacks have also emerged. A prominent issue is the lack of standardized management of electronic files, which means users need to spend a considerable amount of time searching for specific documents or answers when they want to access them. This is especially true for academic researchers and scientific researchers, who rely on extensive research resources to support their projects.

[0003] Based on the existing industry pain points, this invention uses RAG, file storage and other technologies to store a large number of files in a designated cloud disk or network disk, uses RAG to parse the file content and fill it into the knowledge graph, and then obtains the user's intention through dialogue. The information related to the user's intention and the answer source file are pushed directly to the user, achieving the purpose of quickly answering questions and retrieving files. Summary of the Invention

[0004] The present invention aims to at least solve the technical problems existing in the prior art, and in particular innovatively proposes a multi-hop RAG information reasoning method and system based on transformer deep learning.

[0005] In order to achieve the above-mentioned object of the present invention, the present invention provides a multi-hop RAG information reasoning method based on transformer deep learning, comprising the following steps:

[0006] S101 collects data in specific fields through automatic crawling, API integration, database integration, and manual entry;

[0007] S102, based on the needs and characteristics of specific fields, customizes the multi-hop RAG model architecture and uses the TranSformer deep learning architecture as the core building module of the model;

[0008] S103, constructing a knowledge graph for a specific domain. The knowledge graph includes entities, relationships, and attribute information, and is used to represent concepts and associations within the specific domain.

[0009] S104, using a pre-training strategy to train the collected data model architecture to optimize the parameters and structure of the model;

[0010] S105 regularly evaluates the model using precision and recall metrics.

[0011] In the above technical solution, preferably, S101 includes:

[0012] Perform multimodal data fusion and set the original data set to

[0013] D={D auto ,D API ,D DB ,D manual};

[0014] Among them, D auto For automatically captured web pages / text data, D API For structured data obtained through third-party API, D DB Time series data integrated into the database, D manual Field-specific terminology entered manually;

[0015] After acquiring multimodal data, the original data set is formed and data cleaning operations are performed.

[0016]

[0017] Among them, Regex is a regular expression to filter unstructured noise; NER is named entity recognition to retain key entities, d is a separate data element, and classification operations are performed through classification label formulas. Among them, the classification label data C is defined as {c1, c2, ..., c i ,...,c n}.

[0018] In the above technical solution, preferably, S102 includes:

[0019] Based on the Transformer architecture, an encoder-decoder model is used, in which the encoder is used to understand the entities of the input knowledge graph. User query information must be converted into vectors through the encoder, and the decoder is used to generate output. That is, the output information generated by RAG must be decoded by the decoder before it can become natural language that users can understand.

[0020] In the above technical solution, preferably, the S102 further includes:

[0021] Design a retriever based on the Transformer architecture to retrieve relevant documents or fragments from the knowledge base. The retriever can receive questions and return the most relevant knowledge fragments.

[0022] In the above technical solution, preferably, the S102 further includes:

[0023] A generator model is designed based on the Transformer architecture to generate answers based on the information provided by the encoder and retriever. The retriever and generator are integrated. When a user asks a question, the retriever searches for relevant knowledge, and the generator uses this knowledge to generate answers.

[0024] In the above technical solution, preferably, S103 further includes:

[0025] The TransE algorithm is used to convert each entity and relationship in the knowledge graph into a vector representation. When the entities and relationships are encoded into vectors, a high-dimensional vector space is constructed, and a KD tree is built based on the vector representation. When a query request is received, the query is converted into a vector representation, and then a search is performed in the index structure to find the entity or relationship vector closest to the query vector.

[0026] In the above technical solution, preferably, S103 further includes:

[0027] Vectorizing entities and relationships through the TransE model

[0028]

[0029] Among them, σ, τ is the vector representation of the head entity, relation, and tail entity respectively, and L1 and L2 are the norms.

[0030] In the above technical solution, preferably, S103 further includes:

[0031] By combining the inverted index with the KD tree,

[0032] The inverted index formula is all triples involved in the entity or relationship m

[0033]

[0034] KD tree query is

[0035] Where q is the query vector, and V is the vector set of all entities or relationships in the knowledge graph;

[0036] When a user query arrives, it interacts with the knowledge graph through multi-hop RAG.

[0037] First, q = Encoder (Query) to perform knowledge graph retrieval;

[0038] K KD =KD-Tree(q)∪I(NER(Query));

[0039] Search K by text text=Retriever(q); the final fusion is generated

[0040]

[0041] The present invention also discloses a multi-hop RAG information reasoning system based on tranSformer deep learning, which includes the logical steps executed by the method described.

[0042] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:

[0043] Through multimodal data fusion and meticulous preprocessing, data quality and diversity are ensured, providing a solid foundation for model training. The customized multi-hop RAG model architecture, combined with the integration of encoder-decoder models, retrievers, and generators, and a multi-hop reasoning mechanism, significantly improves the relevance and quality of text generation, while preventing infinite multi-hops and ensuring reasoning efficiency. The combination of knowledge graphs and vector space models enables semantic representation and efficient retrieval of entities and relationships, enhancing the comprehensiveness and accuracy of answers. Model parameters and structure are optimized through training techniques such as pre-training and fine-tuning, further improving model performance. Finally, regular evaluation and debugging of the model using indicators such as accuracy and recall ensures continuous optimization and improvement. The overall process forms a complete closed loop from data collection and processing to model training and evaluation, providing an efficient and accurate solution for knowledge question-answering systems in specific fields.

[0044] Additional aspects and advantages of the present invention will be set forth in part in the description which follows and, in part, will be obvious from the description which follows, or may be learned by practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments with reference to the accompanying drawings, in which:

[0046] Figure 1 This is a diagram illustrating the implementation of the RAG model training function of the present invention.

[0047] Figure 2 It is a business flow chart of the present invention. DETAILED DESCRIPTION

[0048] The following describes embodiments of the present invention in detail. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended only to explain the present invention and are not to be construed as limiting the present invention.

[0049] To make the above-mentioned technologies, features and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments:

[0050] like Figure 1 and 2 The present invention discloses a multi-hop RAG information reasoning method and system based on tranSformer deep learning, comprising the following steps:

[0051] RAG model training function implementation diagram (such as Figure 1 )

[0052] Step 101: Collect data in a specific field through automatic crawling, API integration, database integration, and manual entry. Then, use natural language processing technology, relationship extraction, and time extraction to pre-process the collected data, including cleaning, labeling, and classification operations, so that the model can better understand and process the data.

[0053] Perform multimodal data fusion and set the original data set to

[0054] D={D auto ,D API ,D DB ,D manual};

[0055] Among them, D auto For automatically captured web pages / text data, D API For structured data obtained through third-party API, D DB Time series data integrated into the database, D manual Field-specific terminology entered manually;

[0056] After acquiring multimodal data, the original data set is formed and data cleaning operations are performed.

[0057]

[0058] Among them, Regex is a regular expression that filters unstructured noise (such as HTML tags and special symbols); NER is named entity recognition that retains key entities (such as names of people, organizations, and domain terms); d is a separate data element that is classified using the classification label formula. Among them, the classification label data C is defined as {c1, c2, ..., c i ,...,c n}, for example: scientific literature, medical text, θ is the parameter of the BERT-based text classification model.

[0059] In step S102, a customized multi-hop RAG model architecture is constructed to meet the needs and characteristics of specific domains. The TranSformer deep learning architecture is used as the core building block of the model to implement RAG's text generation and retrieval capabilities. The retriever and generator are further combined to improve the relevance and quality of the generated content.

[0060] Use the Transformer deep learning architecture to build a multi-hop RAG model:

[0061] 1. Based on the Transformer architecture, an encoder-decoder model is used. The encoder is used to understand the entities in the input knowledge graph. User query information must be converted into vectors through the encoder, and the decoder is used to generate output. That is, the output information generated by RAG must be decoded by the decoder before it can become natural language that users can understand.

[0062] Set the encoder to

[0063] h=Encoder(x j )=MultiHeadAttn(x j W Q ,x j W K ,x j W V ), x j is the jth token of the sequence input to the encoder, W Q ,W K ,W V are respectively trainable projection matrices;

[0064] Set the decoder to

[0065] s t =Decoder(h,y ＜t )=FFN(CrossAttn(h,s t-1 ), where h is the context vector output by the encoder, y ＜t is the first t-1 tokens generated, s t is the hidden state of the decoder for the tth token.

[0066] 2. Design a retriever based on the Transformer architecture to retrieve relevant documents or fragments from the knowledge base. The retriever can receive questions and return the most relevant knowledge fragments.

[0067] By setting the retriever q = Encoder(Query)∈R e ,

[0068] Among them, Query is the user query, k l is the vector representation of the lth document in the knowledge base, R e is the space composed of all e-dimensional real vectors,

[0069] Assume e = 300, then:

[0070] Query = "What is the physical principle of quantum entanglement?"

[0071] The encoder converts it into q = (0.12, -0.35, 0.89, ..., 0.04) ∈ R 300 .

[0072] Each dimension (e.g. 0.12 for the first dimension, -0.35 for the second dimension) corresponds to a certain semantic feature learned by the model (e.g. “whether it involves the basic concepts of quantum mechanics”, “whether it requires mathematical formula explanation”, etc.)

[0073] 3. Design a generator model based on the Transformer architecture to generate answers based on the information provided by the encoder and retriever.

[0074] In the generator P(y t |y ＜t ,Query,k)=Softmax(Z g [s t ;a t ]),

[0075] Among them, a t =Attn(s t ,{k l}) is the attention weight of the retrieval result, k is the set of all documents retrieved by the retriever. In the multi-hop RAG model, k is an important basis for the generator to answer the question. This information is combined with the current state of the decoder st through the attention mechanism to know the generator to generate the next token;

[0076] If the user query is "the physical principles of quantum entanglement", k contains the following document fragments:

[0077] k1: "Quantum entanglement is a phenomenon in quantum mechanics that refers to two or more particles..."

[0078] k2: "The Bell inequality experiment shows that quantum entanglement is non-local..."

[0079] Z g is the weight matrix that can be trained in the generator, and the subscript g is the weight matrix Z g Belongs to the generator,

[0080] 4. Integrate the retriever and generator. When a user asks a question, the retriever will search for relevant knowledge, and the generator will use this knowledge to generate an answer.

[0081] By setting the initial search result k1, generating the preliminary answer R1, the second jump search is

[0082]

[0083] R final =Generate(Query,k1∪k2)

[0084] in, It is the concatenation operation of answer and query;

[0085] Step S103: To support more efficient retrieval and generation, a domain-specific knowledge graph is constructed. The knowledge graph contains entity, relationship, and attribute information, representing concepts and associations within the domain. The knowledge graph is automatically extracted from the retrieved documents to ensure its quality and integrity.

[0086] The combined operation of the knowledge graph and the vector space model first uses the TransE algorithm to convert each entity and relationship in the knowledge graph into a vector representation. Once the entities and relationships are encoded as vectors, a high-dimensional vector space is constructed, where each vector represents an entity or relationship. Next, different metrics are used in the vector space to calculate the similarity between entities or relationships using Euclidean distance. When a user issues a query, the query is also converted into a vector representation and then compared with the entity or relationship vector in the vector space to identify the most relevant entities or relationships.

[0087] Each entity and relationship is automatically assigned an identifier when it is created, allowing for easy reference in the index. Next, a mapping is created from the entity or relationship to all edges in which it appears, recording all triples (subject-verb-object) in which it participates. An inverted index table is then created, with the key being the entity or relationship and the value being a list of triples containing that entity or relationship.

[0088] When a user queries, he only needs to query a certain entity and its related triples without traversing the entire knowledge graph.

[0089] Combining the knowledge graph with nearest neighbor search, the TransE model maps entities and relationships into a continuous vector space. These vectors reflect the semantic connections and structural properties between entities and relationships. A KD tree is constructed based on this vector representation. When a query is received, the query is converted into a vector representation, and then a search is performed within the index structure to find the entity or relationship vector closest to the query vector.

[0090] Vectorizing entities and relationships through the TransE model

[0091]

[0092] Among them, σ, τ is the vector representation of the head entity, relation, and tail entity respectively, L1 and L2 are the norms,

[0093] In TransE, the L1 norm measures the Manhattan distance between h+r and t, generating sparse vectors (many dimensions are 0) and is insensitive to outliers. If h+rt = (2, -3, 0), then the L1 distance is |2|+|-3|+|0|=5.

[0094] In TransE, the L2 norm measures the Euclidean distance between h+r and t. It is sensitive to the overall size of the vector and even more sensitive to outliers. If h+rt=(2,-3,0), then the L2 distance is

[0095] By combining the inverted index with the KD tree,

[0096] The inverted index formula is all triples involved in the entity or relationship m

[0097]

[0098] KD tree query is

[0099] Where q is the query vector, and V is the vector set of all entities or relationships in the knowledge graph;

[0100] When a user query arrives, it interacts with the knowledge graph through multi-hop RAG.

[0101] First, q = Encoder (Query) to perform knowledge graph retrieval;

[0102] K KD =KD-Tree(q)∪I(NER(Query));

[0103] Search K by text text =Retriever(q); the final fusion is generated

[0104]

[0105] In order to prevent infinite hops, a termination condition is set. When the confidence of the generated answer R exceeds the set threshold ω, or reaches the preset maximum number of hops N maxWhen , the reasoning process stops,

[0106] If Confidence(R)>ω or n=N max , then stop;

[0107] Among them, the confidence (Rfinal) can be calculated through the output probability distribution of the generator.

[0108] Where β1 represents the KD tree retrieval result K KD The weight of (the documents retrieved based on KD tree query q) when generating the final answer realizes the degree of trust in the inverted index to KD tree retrieval results. If β1 is high, it means that the model believes that K KD The information contained is crucial for generating accurate answers; if it is low, the model relies on subsequent retrieval results.

[0109] β1=Sigmoid(W β [q;K KD ;K text ]+b β )1Wherein, W β and b β are the trainable weights and biases, [q; K KD ;K text ] is the vector obtained by concatenating the enhanced query, KD tree and text retrieval results. The Sigmoid function limits β1 to the range of (0,1).

[0110] Among them, β2 represents the text retrieval result K text The weight when generating the final answer, the importance of the retrieval results after iterative optimization. If β2 is high, it means that the model believes that K text It provides more relevant or deeper information, which is crucial for generating high-quality answers; q′ is the enhanced query vector for q;

[0111] β2=Sigmoid(W β [q′;K KD ;K text ]+b β )2

[0112] Similar to β1, β2 is calculated through the same gating mechanism, and the concatenated vector is another component, which is also restricted to the range of (0,1).

[0113] Assume the user query is "explain the phenomenon of quantum entanglement":

[0114] The retriever finds KD tree documents K based on Query KD (Such as "Introduction to Quantum Mechanics").

[0115] Generate preliminary answer R1: "Quantum entanglement is a correlation between particles, whereby even if they are far apart, their states can affect each other instantaneously."

[0116] Then perform text retrieval, the retriever finds more professional documents K based on the enhanced query q′ (including the attention-weighted concatenation of R1) text (Such as "Bell's inequality and quantum entanglement experiment").

[0117] Dynamic weight allocation:

[0118] If K text Contains key experimental data, model settings β2 = 0.7, β1 = 0.3, emphasizing K text The final answer R will be combined with K KD Basic concepts and K text The experimental details of the experiment were analyzed to generate a more comprehensive explanation.

[0119] Step S104 uses a pre-training strategy to train the collected data model architecture to optimize the parameters and structure of the model;

[0120] Step S105 regularly evaluates the model through the accuracy and recall rate indicators, and compares and analyzes it with other benchmark models to find out the advantages and disadvantages of the model and perform corresponding debugging and improvements.

[0121] The debugging and improvement steps are as follows: After confirming that the data set is intact, if the indicator performance is poor, you can debug and improve it by following the steps below:

[0122] By switching the decoding strategy Top-K sampling, the best balance between generation quality and diversity is found.

[0123] Business process diagram (such as Figure 2 )

[0124] In order to better illustrate the business process, the present invention will be described in an illustrative manner in combination with a business process diagram.

[0125] Step [S201]: The user uploads a file named "Introduction to the Forbidden City".txt to the knowledge base.

[0126] Step [S202]: Through RAG retrieval model analysis, the file content obtained is: "The Forbidden City in Beijing is the imperial palace of the Ming and Qing dynasties in China. It was formerly known as the Forbidden City. It is located at the center of Beijing's central axis and is the essence of ancient Chinese palace architecture." Then, combined with information extraction technology, the entity is extracted, including the alias, type, location, architectural style, and other attributes of the Forbidden City in Beijing, and is populated into the knowledge graph.

[0127] Step [S203] The user asks the question "Where is the Forbidden City in Beijing?" through a dialogue. Using the Transformer model, by analyzing the keywords in the input sequence (such as "Forbidden City in Beijing" and "where") and their relationship in the context, it is inferred that the user's main intention is to query the location of an entity.

[0128] Step [S204] constructs a multi-hop RAG model architecture in the knowledge graph through a vector space model and customization according to the user's intention, and uses the TranSformer deep learning architecture as the core building module of the model to find the key information that the entity is the Forbidden City in Beijing and the attribute is the location.

[0129] Step [S205] The retrieved key information is subjected to data fusion and text generation post-processing optimization to form a complete reply content "The Forbidden City in Beijing is located at the center of Beijing's central axis"

[0130] First, the document encoding vector is stored in a searchable index for quick retrieval [S202]. Then, the same Transformer encoder is used to encode the user's query question to obtain the query vector [S203]. Finally, cosine similarity is used as a similarity metric in the index to find the document most relevant to the query vector [S204]. The retrieved document vector is merged with the query vector by splicing [S205]. The fused vector is used as an additional input and fed into the Transformer decoder together with the query. It is decoded into natural language that the user can understand through the multi-hop RAG architecture [S206].

[0131] Step [S206] Reply to the user: "The Forbidden City is located at the center of Beijing's central axis." Data source: "Introduction to the Forbidden City".txt.

[0132] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to the embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the claims and their equivalents.

Claims

1. A multi-hop RAG information reasoning method based on transformer deep learning, characterized by: The steps include: S101 collects data in specific fields through automatic crawling, API integration, database integration, and manual entry; S102, based on the needs and characteristics of specific fields, customizes the multi-hop RAG model architecture and uses the Transformer deep learning architecture as the core building block of the model; S103, constructing a knowledge graph for a specific domain. The knowledge graph includes entities, relationships, and attribute information, and is used to represent concepts and associations within the specific domain. S104, using a pre-training strategy to train the collected data model architecture to optimize the parameters and structure of the model; S105 regularly evaluates the model using precision and recall metrics.

2. The multi-hop RAG information reasoning method based on transformer deep learning according to claim 1 is characterized in that The S101 includes: Perform multimodal data fusion and set the original data set to D={D auto ,D API ,D DB ,D manual }; Among them, D auto For automatically captured web pages / text data, D API For structured data obtained through third-party API, D DB Time series data integrated into the database, D manual Field-specific terminology entered manually; After acquiring multimodal data, the original data set is formed and data cleaning operations are performed. Among them, Regex is a regular expression to filter unstructured noise; NER is named entity recognition to retain key entities, d is a separate data element, and classification operations are performed through classification label formulas. Among them, the classification label data C is defined as {c1, c2, ..., c i ,...,c n }.

3. The multi-hop RAG information reasoning method based on tranSformer deep learning according to claim 1 is characterized in that The S102 includes: Based on the Transformer architecture, an encoder-decoder model is used, in which the encoder is used to understand the entities of the input knowledge graph. User query information must be converted into vectors through the encoder, and the decoder is used to generate output. That is, the output information generated by RAG must be decoded by the decoder before it can become natural language that users can understand.

4. The multi-hop RAG information reasoning method based on transformer deep learning according to claim 1 is characterized in that The S102 further includes: Design a retriever based on the Transformer architecture to retrieve relevant documents or fragments from the knowledge base. The retriever can receive questions and return the most relevant knowledge fragments.

5. The multi-hop RAG information reasoning method based on transformer deep learning according to claim 1 is characterized in that The S102 further includes: A generator model is designed based on the Transformer architecture to generate answers based on the information provided by the encoder and retriever. The retriever and generator are integrated. When a user asks a question, the retriever searches for relevant knowledge, and the generator uses this knowledge to generate answers.

6. The multi-hop RAG information reasoning method based on transformer deep learning according to claim 1, characterized in that The S103 further includes: The TransE algorithm is used to convert each entity and relationship in the knowledge graph into a vector representation. When the entities and relationships are encoded into vectors, a high-dimensional vector space is constructed, and a KD tree is built based on the vector representation. When a query request is received, the query is converted into a vector representation, and then a search is performed in the index structure to find the entity or relationship vector closest to the query vector.

7. The multi-hop RAG information reasoning method based on transformer deep learning according to claim 6 is characterized in that The S103 further includes: Vectorizing entities and relationships through the TransE model in, They are the vector representations of the head entity, relation, and tail entity respectively, and L1 and L2 are the norms.

8. The multi-hop RAG information reasoning method based on transformer deep learning according to claim 7 is characterized in that The S103 further includes: By combining the inverted index with the KD tree, The inverted index formula is all triples involved in the entity or relationship m KD tree query is Where q is the query vector, and V is the vector set of all entities or relationships in the knowledge graph; When a user query arrives, it interacts with the knowledge graph through multi-hop RAG. First, q = Encoder (Query) to perform knowledge graph retrieval; K KD =KD-Tree(q)∪I(NER(Query)); Search K by text text =Retriever(q); the final fusion is generated 9. A multi-hop RAG information reasoning system based on transformer deep learning, characterized by: The system includes the logical steps performed by the method according to any one of claims 1-8.

Images