Question and answer processing method and device fusing intention recognition and retrieval enhancement generation technology, equipment, readable storage medium and program product

Abstract

The invention relates to the technical field of artificial intelligence, and provides a question and answer processing method and device fusing intention recognition and retrieval enhancement generation technologies, equipment, a readable storage medium and a program product. The method comprises the following steps: generating an embedded fine tuning model according to a target query problem input by a user and retrieval enhancement, and retrieving a first text vector matched with the target query problem in a document content vector knowledge base; according to a target query question input by a user, retrieving a question vector which is similar to the semantic of the target query question in the question vector knowledge base to obtain a target question vector; according to an index mechanism between the document content vector knowledge base and the problem vector knowledge base, obtaining a text vector associated with the target problem vector to obtain a second text vector; and inputting the first text vector, the second text vector and the target query question into a large language model, so that the large language model generates a target answer corresponding to the target query question. By adopting the method, the question and answer processing accuracy can be improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a question-answering processing method, apparatus, computer device, computer-readable storage medium, and computer program product that integrates intent recognition and retrieval enhancement generation technology. Background Technology

[0002] In recent years, Retrieval-Augmented Generation (RAG) technology, as an innovative technique combining information retrieval and text generation, has demonstrated enormous potential in the field of intelligent question-answering. By retrieving relevant information from large-scale knowledge bases and generating answers based on this information, RAG technology effectively improves the accuracy and diversity of question-answering processing.

[0003] Traditional search-enhanced generation techniques often focus on retrieving content from the knowledge base that is literally similar to the query question when processing user queries, neglecting a deeper understanding of the user's true intent behind the query. This literal matching retrieval method can easily lead to retrieved information that is far removed from the user's actual needs when faced with complex or ambiguous queries, thus affecting the accuracy of question-and-answer processing. Summary of the Invention

[0004] Therefore, it is necessary to provide a question-answering processing method, apparatus, computer equipment, computer-readable storage medium, and computer program product that integrates intent recognition and retrieval enhancement generation technologies to address the aforementioned technical problems.

[0005] Firstly, this application provides a question-answering processing method that integrates intent recognition and retrieval enhancement generation technologies, including:

[0006] Based on the user-input target query question and retrieval enhancement, an embedding fine-tuning model is generated, and a first text vector matching the target query question is retrieved from the document content vector knowledge base.

[0007] Based on the user's input of the target query question, retrieve question vectors that are semantically similar to the target query question from the question vector knowledge base to obtain the target question vector;

[0008] Based on the indexing mechanism between the document content vector knowledge base and the question vector knowledge base, a text vector associated with the target question vector is obtained to obtain a second text vector;

[0009] The first text vector, the second text vector, and the target query question are input into a large language model, so that the large language model integrates the first text vector, the second text vector, and the target query question to generate the target answer corresponding to the target query question.

[0010] In one embodiment, before retrieving a first text vector matching the target query question from a document content vector knowledge base based on the user-input target query question, the method further includes:

[0011] Obtain retrieval enhancements to generate embedded fine-tuning models;

[0012] Based on the retrieval enhancement, an embedded fine-tuning model is generated, and relevant documents in the domain of the target query question are vectorized to obtain several text vectors, which are then stored in the document content vector knowledge base.

[0013] Based on relevant documents in the domain of the target query question, we obtain the original question and the enhanced question;

[0014] Based on the retrieved enhancement generation embedding fine-tuning model, the original question and the enhanced question are vectorized to obtain several question vectors, which are then stored in a question vector knowledge base.

[0015] In one embodiment, obtaining the original question and the enhanced question based on relevant documents in the domain to which the target query question belongs includes:

[0016] Based on the large language model, relevant documents in the domain of the target query question are traversed, each paragraph or sentence in the relevant documents is analyzed, and content containing key information is identified.

[0017] Based on the content containing key information, generate the corresponding original question;

[0018] Based on the large language model, the original problem is augmented to obtain the augmented problem.

[0019] In one embodiment, the method further includes:

[0020] Based on the original question and the content containing key information corresponding to the original question, a preliminary question-answer pair set is obtained;

[0021] Based on the relationship between the original question and the enhanced question, the enhanced question is added to the preliminary question-answer pair set to obtain the question-answer pair set;

[0022] An indexing mechanism is constructed between the document content vector knowledge base and the question vector knowledge base based on the question-answer pair set.

[0023] In one embodiment, obtaining the retrieval enhancement generation embedding fine-tuning model includes:

[0024] The original problem and the enhancement problem are combined into multiple sets of semantically similar problem pairs to obtain the fine-tuned dataset;

[0025] Based on the sentence converter model, the underlying embedding model to be trained is obtained;

[0026] Based on the fine-tuning dataset, the base embedding model to be trained is trained to obtain the retrieval enhancement generative embedding fine-tuning model.

[0027] In one embodiment, the step of inputting the first text vector, the second text vector, and the target query question into a large language model, so that the large language model integrates the first text vector, the second text vector, and the target query question to generate a target answer corresponding to the target query question, includes:

[0028] The first text vector and the second text vector are merged to obtain a merged text vector;

[0029] The merged text vector and the target query question are input into a large language model, so that the large language model merges the merged text vector and the target query question to generate the target answer corresponding to the target query question.

[0030] Secondly, this application also provides a question-answering processing apparatus that integrates intent recognition and retrieval enhancement generation technologies, comprising:

[0031] The first text vector acquisition module is used to generate an embedding fine-tuning model based on the user-input target query question and retrieval enhancement, and to retrieve a first text vector that matches the target query question from the document content vector knowledge base;

[0032] The target question vector acquisition module is used to retrieve question vectors that are semantically similar to the target query question from the question vector knowledge base based on the target query question input by the user, and obtain the target question vector;

[0033] The second text vector acquisition module is used to obtain a text vector associated with the target question vector based on the indexing mechanism between the document content vector knowledge base and the question vector knowledge base, so as to obtain the second text vector;

[0034] The target answer acquisition module is used to input the first text vector, the second text vector, and the target query question into a large language model, so that the large language model integrates the first text vector, the second text vector, and the target query question to generate a target answer corresponding to the target query question.

[0035] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the above-described method.

[0036] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, the computer program being executed by a processor using the methods described above.

[0037] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that is executed by a processor using the methods described above.

[0038] The aforementioned question-answering processing method, apparatus, computer device, computer-readable storage medium, and computer program product, which integrates intent recognition and retrieval enhancement generation technologies, retrieves a first text vector matching the target query question from a document content vector knowledge base based on the user-input target query question and a retrieval enhancement generation embedded fine-tuning model; retrieves question vectors semantically similar to the target query question from a question vector knowledge base based on the user-input target query question to obtain a target question vector; obtains a text vector associated with the target question vector based on the indexing mechanism between the document content vector knowledge base and the question vector knowledge base to obtain a second text vector; and inputs the first text vector, the second text vector, and the target query question into a large language model so that the large language model integrates the first text vector, the second text vector, and the target query question to generate a target answer corresponding to the target query question. This application uses a user-input target query question and a retrieval enhancement generation embedding fine-tuning model with high query question recognition accuracy. It retrieves a first text vector matching the target query question from a document content vector knowledge base, compensating for the shortcomings of traditional retrieval enhancement generation techniques in intent understanding. Based on the user-input target query question, it retrieves question vectors semantically similar to the target query question from a question vector knowledge base to obtain a target question vector, which in turn generates a second text vector. The first text vector, the second text vector, and the target query question are then input into a large language model, which integrates these elements to generate the target answer. By employing a combined retrieval strategy of direct and indirect text retrieval, it can more accurately obtain text related to the target query question, further compensating for the shortcomings of traditional retrieval enhancement generation techniques in intent understanding, thereby significantly improving the accuracy of question answering. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 This is an application environment diagram of a question-answering processing method that integrates intent recognition and retrieval enhancement generation techniques in one embodiment.

[0041] Figure 2 This is a flowchart illustrating a question-answering processing method that integrates intent recognition and retrieval enhancement generation techniques in one embodiment.

[0042] Figure 3 This is a flowchart illustrating a question-answering processing method that integrates intent recognition and retrieval enhancement generation techniques in another embodiment.

[0043] Figure 4 This is a structural block diagram of a question-answering processing device that integrates intent recognition and retrieval enhancement generation technology in one embodiment;

[0044] Figure 5 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0045] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0046] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.

[0047] The question-answering method that integrates intent recognition and retrieval enhancement generation technology provided in this application can be applied to, for example... Figure 1In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104 or placed on a cloud or other network server. Terminal 102 may include an intelligent question-answering system. This system can generate an embedded fine-tuning model based on the user-input target query question and retrieval enhancements. It retrieves a first text vector matching the target query question from a document content vector knowledge base; based on the user-input target query question, it retrieves question vectors semantically similar to the target query question from a question vector knowledge base to obtain a target question vector, which in turn yields a second text vector; the first text vector, the second text vector, and the target query question are input into a large language model to obtain the target answer corresponding to the target query question. Terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, drones, low-altitude aircraft, IoT devices, and portable wearable devices. Server 104 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0048] In one exemplary embodiment, such as Figure 2 As shown, a question-answering processing method integrating intent recognition and retrieval enhancement generation technology is provided, which is then applied to... Figure 1 The following explanation uses the intelligent question-answering system in terminal 102 as an example, including the following steps S201 to S204. Wherein:

[0049] Step S201: Generate an embedding fine-tuning model based on the user-input target query question and retrieval enhancement, and retrieve the first text vector that matches the target query question from the document content vector knowledge base.

[0050] The retrieval enhancement generative embedding fine-tuning model is an embedding model trained on a fine-tuned dataset, which has high accuracy in recognizing query questions.

[0051] Users can enter their query questions through the intelligent question-and-answer system's interface. The system can retrieve the query questions entered by the user during the current query and use them as the target query.

[0052] It can generate an embedded fine-tuning model based on retrieval enhancement, identify the target query question, and retrieve the first text vector that matches the target query question from the document content vector knowledge base.

[0053] For example, an embedding fine-tuning model can be generated based on retrieval enhancement to identify keywords in the target query question. Based on the keywords in the target query question, text vectors containing the same or highly similar literal features can be retrieved from the document content vector knowledge base. The similarity between the text vectors containing the same or highly similar literal features and the target query question can be calculated. The text vector with the highest similarity among the text vectors containing the same or highly similar literal features is taken as the first text vector that matches the target query question.

[0054] Step S202: Based on the target query question input by the user, retrieve question vectors with semantic similarity to the target query question from the question vector knowledge base to obtain the target question vector.

[0055] Based on the user's input of the target query question, the system can retrieve question vectors with semantically similar meanings to the target query question from the question vector knowledge base, thus obtaining the target question vector.

[0056] For example, the user-input target query can be vectorized to obtain a target query vector. Based on the vector similarity between the target query vector and other question vectors in a question vector knowledge base, question vectors in the knowledge base that are semantically similar to the target query can be determined. The question vector with the highest vector similarity among the semantically similar question vectors in the knowledge base can then be used as the target query vector.

[0057] Step S203: Based on the indexing mechanism between the document content vector knowledge base and the question vector knowledge base, obtain the text vector associated with the target question vector to obtain the second text vector.

[0058] The indexing mechanism between the document content vector knowledge base and the question vector knowledge base includes the association between text vectors in the document content vector knowledge base and question vectors in the question vector knowledge base. For example, there is an association between text vector A in the document content vector knowledge base and question vectors B and C in the question vector knowledge base; that is, the answer corresponding to question vectors B and C is text vector A, and the query question corresponding to text vector A is question vectors B and C.

[0059] Based on the indexing mechanism between the document content vector knowledge base and the question vector knowledge base, the text vector associated with the target question vector can be quickly located and used as the second text vector.

[0060] Step S204: Input the first text vector, the second text vector, and the target query question into the large language model so that the large language model can integrate the first text vector, the second text vector, and the target query question to generate the target answer corresponding to the target query question.

[0061] The first text vector, the second text vector, and the target query question can be input into the large language model together, so that the large language model can leverage its powerful generation capabilities to integrate the first text vector, the second text vector, and the target query question to generate the target answer corresponding to the target query question.

[0062] In the aforementioned question-answering method that integrates intent recognition and retrieval enhancement generation techniques, based on the user-input target query question and a retrieval enhancement generation embedding fine-tuning model with high query question recognition accuracy, a first text vector matching the target query question is retrieved from the document content vector knowledge base. This compensates for the shortcomings of traditional retrieval enhancement generation techniques in intent understanding. Based on the user-input target query question, question vectors semantically similar to the target query question are retrieved from the question vector knowledge base to obtain the target question vector, which in turn generates a second text vector. The first text vector, the second text vector, and the target query question are then input into a large language model, allowing the large language model to integrate the first text vector, the second text vector, and the target query question to generate the target answer corresponding to the target query question. By employing a combined retrieval strategy of direct and indirect text retrieval, text related to the target query question can be obtained more accurately, further compensating for the shortcomings of traditional retrieval enhancement generation techniques in intent understanding, thereby significantly improving the accuracy of question-answering processing.

[0063] In one embodiment, before retrieving a first text vector matching the target query question from the document content vector knowledge base based on the user-input target query question, the method provided in this application further includes: obtaining a retrieval enhancement generation embedding fine-tuning model; vectorizing relevant documents in the domain to which the target query question belongs based on the retrieval enhancement generation embedding fine-tuning model to obtain several text vectors, and storing them in the document content vector knowledge base; obtaining the original question and the enhanced question based on the relevant documents in the domain to which the target query question belongs; and vectorizing the original question and the enhanced question based on the retrieval enhancement generation embedding fine-tuning model to obtain several question vectors, and storing them in the question vector knowledge base.

[0064] A retrieval enhancement generative embedding fine-tuning model can be trained based on the fine-tuned dataset.

[0065] The retrieval enhancement model can be used to generate an embedded fine-tuning model, which can vectorize relevant documents in the domain of the target query question to obtain several text vectors, and store these text vectors in a document content vector knowledge base.

[0066] The system can retrieve relevant documents in the domain of the target question to obtain the original question and the enhanced question; it can also generate an embedded fine-tuning model based on the retrieval enhancement, vectorize the original question and the enhanced question to obtain several question vectors, and store these question vectors in a question vector knowledge base.

[0067] In this embodiment, relevant documents in the domain of the target query question are vectorized to obtain several text vectors, which are stored in a document content vector knowledge base. This allows for direct retrieval of the most relevant text during subsequent question-answering processing. The original question and the enhanced question are also vectorized to obtain several question vectors, which are stored in a question vector knowledge base. This results in a relatively complete document content vector knowledge base and question vector knowledge base. During question-answering processing, the target query question is retrieved from the question vector knowledge base, and then the retrieved target question vector is used to trace back to the corresponding text vector, thus achieving a question-guided retrieval path to the text.

[0068] In one embodiment, the original question and the enhanced question are obtained based on relevant documents in the domain to which the target query question belongs. The specific steps are as follows: based on the large language model, the relevant documents in the domain to which the target query question belongs are traversed, each paragraph or sentence in the relevant documents is analyzed, and the content containing key information is identified; based on the content containing key information, the corresponding original question is generated; based on the large language model, the original question is augmented to obtain the enhanced question.

[0069] The data augmentation capabilities of the Large Language Model (LLM) can be trained to obtain a pre-trained large language model.

[0070] A pre-trained large language model can be used as an intelligent engine to traverse relevant documents in the domain of the target query question, analyze each paragraph or sentence in the relevant documents, and identify content containing key information.

[0071] Based on content containing key information, a corresponding original question can be generated. For example, for a fragment describing the time of occurrence of a historical event, event A, the original question that might be generated is "When did event A occur?"

[0072] Data augmentation can be performed on the original question based on the semantic understanding capabilities of the large language model, resulting in augmented questions. Specifically, operations such as synonym substitution and sentence transformation can be performed on the original question based on the large language model to generate multiple augmented questions that are semantically similar to the original question but have different expressions. For example, the original question "What is the background of event A?" can be transformed into the augmented question "Please briefly describe the cause and background of event A?".

[0073] In this embodiment, based on a large language model, each paragraph or sentence in relevant documents belonging to the domain of the target query question is analyzed to identify content containing key information, thereby generating the corresponding original question. Then, based on the large language model, the original question is augmented to obtain an augmented question. This enriches the data in the question vector knowledge base, compensates for the shortcomings of traditional retrieval augmentation generation techniques in intent understanding, and thus significantly improves the accuracy of question-answering processing.

[0074] In one embodiment, the method provided by this application further includes: combining the original question with the content containing key information corresponding to the original question to obtain a preliminary question-answer pair set; adding the enhanced question to the preliminary question-answer pair set according to the association between the original question and the enhanced question to obtain a question-answer pair set; and constructing an indexing mechanism between the document content vector knowledge base and the question vector knowledge base based on the question-answer pair set.

[0075] By combining each original question with its corresponding content containing key information, several preliminary question-answer pairs are obtained; based on these preliminary question-answer pairs, a preliminary question-answer pair set is obtained.

[0076] The relationship between the original problem and the augmented problem can be determined by the data augmentation association between them. For example, if augmented problem E is obtained by augmenting the original problem D, then there is a relationship between augmented problem E and the original problem D.

[0077] Based on the relationship between the original question and the enhanced question, the enhanced question can be added to the preliminary question-answer pair corresponding to each original question in the preliminary question-answer pair set, thus obtaining the question-answer pair set; based on the question-answer pair set, the relationship between the text vector in the document content vector knowledge base and the question vector in the question vector knowledge base can be determined; based on the relationship between the text vector in the document content vector knowledge base and the question vector in the question vector knowledge base, an indexing mechanism can be built between the document content vector knowledge base and the question vector knowledge base.

[0078] In this embodiment, a preliminary question-answer pair set is obtained by combining the original question with the content containing key information corresponding to the original question; based on the relationship between the original question and the enhanced question, the enhanced question is added to the preliminary question-answer pair set to obtain a question-answer pair set, so as to build an indexing mechanism between the document content vector knowledge base and the question vector knowledge base, so as to quickly locate the text vector associated with the target question vector in the subsequent question-answering process.

[0079] In one embodiment, the retrieval enhancement generative embedding fine-tuning model is obtained through the following steps: combining the original question and the enhancement question into multiple sets of semantically similar question pairs to obtain a fine-tuning dataset; obtaining the basic embedding model to be trained based on the sentence converter model; and training the basic embedding model to be trained based on the fine-tuning dataset to obtain the retrieval enhancement generative embedding fine-tuning model.

[0080] Based on the data augmentation association between the original question and the augmented question, the original question and the augmented question can be combined into multiple sets of semantically similar question pairs. For example, if augmented question E is obtained by data augmentation of the original question D, then augmented question E and the original question D can be combined into a question pair.

[0081] Multiple sets of semantically similar question pairs can be used as a fine-tuning dataset.

[0082] The sentence transformer model can be used as the base embedding model to be trained. The base embedding model to be trained can be trained based on the fine-tuning dataset, that is, the base embedding model to be trained can be finely tuned. By optimizing the parameters and training strategy of the base embedding model to be trained, the retrieval enhancement generative embedding fine-tuning model can be obtained.

[0083] In this embodiment, the original question and the augmentation question are combined into multiple sets of semantically similar question pairs to obtain a fine-tuning dataset. Based on the fine-tuning dataset, the basic embedding model to be trained is trained to obtain the retrieval augmentation generative embedding fine-tuning model, which can improve the retrieval augmentation generative embedding fine-tuning model's ability to identify query questions and the accuracy of retrieving text vectors in the document content vector knowledge base.

[0084] In one embodiment, a first text vector, a second text vector, and a target query question are input into a large language model so that the large language model can fuse the first text vector, the second text vector, and the target query question to generate a target answer corresponding to the target query question. The specific steps are as follows: the first text vector and the second text vector are merged to obtain a merged text vector; the merged text vector and the target query question are input into the large language model so that the large language model can fuse the merged text vector and the target query question to generate a target answer corresponding to the target query question.

[0085] The first and second text vectors can be merged to obtain a merged text vector. The merged text vector and the target query question can be input into a large language model, so that the large language model can leverage its powerful generation capabilities to fuse the merged text vector and the target query question to generate the target answer corresponding to the target query question.

[0086] In this embodiment, the first text vector and the second text vector are merged to obtain a merged text vector. This merged text vector and the target query question are then input into a large language model, enabling the model to generate the target answer corresponding to the query. This reduces the amount of data input to the large language model, thereby improving its processing efficiency.

[0087] To better understand the above method, the following describes in detail an application example of the question-answering processing method that integrates intent recognition and retrieval enhancement generation technology.

[0088] In recent years, Retrieval-Augmented Generation (RAG), as an innovative technology combining information retrieval and text generation, has demonstrated enormous potential in the field of intelligent question answering. RAG effectively improves the accuracy and diversity of question answering systems by retrieving relevant information from large-scale knowledge bases and generating answers based on this information. However, despite its significant advantage in knowledge coverage, RAG suffers from a marked deficiency in intent recognition capabilities.

[0089] Traditional search-enhanced generation techniques, when processing user queries, often focus on retrieving content from the knowledge base that is literally similar to the query question (or query text), while neglecting a deeper understanding of the user's true intent behind the query. This literal matching retrieval method can easily lead to retrieved information that is far removed from the user's actual needs when faced with complex or ambiguous queries, thus affecting the accuracy of question-and-answer processing.

[0090] To address the aforementioned issues, this embodiment proposes a question-answering method that integrates intent recognition and retrieval enhancement generation techniques. By embedding a fine-tuned model using retrieval enhancement generation and employing a combined retrieval strategy, it overcomes the shortcomings of traditional retrieval enhancement generation techniques in intent understanding, thereby significantly improving the accuracy of question-answering processing and user experience. This method is then applied to… Figure 1 Taking the intelligent question-answering system in terminal 102 as an example, the specific technical solution is as follows: Figure 3 As shown:

[0091] Step 1: Document Preprocessing and Original Question Generation

[0092] 1. Document Collection: Collect relevant documents from multiple channels related to the field of the target query. For example, when the target query is related to high school history, relevant documents can be collected from textbooks, supplementary teaching materials, historical documents, and online resources, including textbook content and descriptions of historical events.

[0093] 2. Intelligent Engine Application: A pre-trained large-scale language model is used as an intelligent engine to traverse relevant documents in the domain of the target query question. Each paragraph or sentence in the relevant documents is analyzed to identify content containing key information, and the corresponding original question is generated accordingly. For example, for a fragment describing the time of occurrence of a historical event, event A, the original question might be "When did event A occur?"

[0094] 3. Question-answer pair set formation: Combine the generated original questions with the corresponding content (or summary) containing key information to form a preliminary question-answer pair set, providing a data foundation for subsequent steps.

[0095] Step 2: Fine-tuning the dataset construction:

[0096] 1. Data Augmentation: For each original question generated in Step 1, data augmentation is performed on each original question based on the semantic understanding capabilities of the large language model, resulting in augmented questions. Specifically, operations such as synonym replacement and sentence transformation can be performed on the original questions based on the large language model to generate multiple augmented questions that are semantically similar to the original questions but have different expressions. For example, the original question "What is the background of event A?" can be transformed into the augmented question "Please briefly describe the cause and background of event A".

[0097] 2. Fine-tuning Dataset Construction: The original questions and augmentation questions are combined into multiple semantically similar question pairs, such as {“What is the background of event A?”, “Please briefly describe the cause and background of event A.”}. For example, for each original question M generated in step one, based on the semantic understanding capability of the large language model, multiple semantically similar but differently expressed questions N, Z, etc., are generated, forming multiple semantically similar question pairs {M, N}, {M, Z}. Based on the question pairs, a fine-tuning dataset is formed. The fine-tuning dataset will be used for subsequent targeted training of the underlying embedding model to be trained.

[0098] Step 3: Fine-tuning the basic embedding model:

[0099] 1. Model selection: The Sentence Transformers model was selected as the base embedding model to be trained. This model has good text representation ability and certain retrieval ability.

[0100] 2. Model Training: Using the fine-tuning dataset constructed in step two, the basic embedding model to be trained is fine-tuned. By optimizing the parameters and training strategy of the basic embedding model, a retrieval-enhanced generative embedding fine-tuning model is obtained. This improves the retrieval-enhanced generative embedding fine-tuning model's ability to identify query questions and its accuracy in retrieving text vectors from the document content vector knowledge base, effectively enhancing its ability to recognize user intent.

[0101] Step 4: Constructing the Vector Knowledge Base

[0102] 1. Construction of Document Content Vector Knowledge Base: Based on the retrieval enhancement generation embedding fine-tuning model, the relevant documents in the domain of the target query question are vectorized to obtain several text vectors, which are stored in the document content vector knowledge base so that the most relevant text to the target query question can be directly retrieved in the subsequent question answering process.

[0103] 2. Question Vector Knowledge Base Construction: Based on the retrieval enhancement generation embedding fine-tuning model, the original question and the enhanced question are vectorized to obtain several question vectors, which are stored in the question vector knowledge base. In the question-answering process, the target query question is retrieved from the question vector knowledge base, and then the retrieved target question vector is used to trace back to the corresponding text vector, realizing the retrieval path from question to text.

[0104] 3. Index Construction: Based on the original question and its corresponding content containing key information, a preliminary question-answer pair set is obtained. Based on the relationship between the original question and the enhanced question, the enhanced question is added to the preliminary question-answer pair set to obtain the question-answer pair set. Based on the question-answer pair set, an efficient indexing mechanism is built between the document content vector knowledge base and the question vector knowledge base so that the text vector associated with the target question vector can be quickly located in the subsequent question-answering process.

[0105] Step 5: Combined Search and Response Generation:

[0106] 1. User Query Input: Users can input their query questions through the intelligent question-and-answer system's interface. The system can retrieve the user's input question for this query and use it as the target query. For example, if a user inputs a question about history, such as "What is the significance of event B?", this question can be used as the target query.

[0107] 2. Combined Retrieval: Based on the user-input target query and the retrieval enhancement model, an embedded fine-tuning model is generated. A first text vector matching the target query is retrieved from the document content vector knowledge base. Simultaneously, based on the user-input target query, question vectors semantically similar to the target query are retrieved from the question vector knowledge base to obtain the target question vector. Using the indexing mechanism between the document content vector and question vector knowledge bases, text vectors associated with the target question vector are obtained, and these associated text vectors are used as the second text vector. The first and second text vectors obtained from the two retrieval methods are merged to obtain a combined text vector. This combined retrieval method helps improve retrieval accuracy.

[0108] 3. Response generation: Input the merged text vector and the target query question into the large language model, so that the large language model can use its powerful generation capabilities to merge the merged text vector and the target query question to generate a high-quality target response to the target query question.

[0109] 4. Output the answer: The generated target answer will be displayed to the user through the intelligent question-answering system's interface, while providing relevant document links or summaries for the user to further learn and understand.

[0110] The question-answering method provided in this embodiment, which integrates intent recognition and retrieval enhancement generation technologies, generates original questions based on relevant documents in the domain of the target query question; it constructs a fine-tuned dataset using data augmentation strategies on the generated original questions; it fine-tunes the basic embedding model to be trained to improve its recognition accuracy of the query question; it constructs a complementary vector knowledge base based on document content and question orientation to achieve efficient and accurate retrieval; and it combines the text vectors obtained from the two retrieval paths with the target query question, using a large language model to generate high-quality target answers, thereby achieving accurate and efficient intelligent question-answering services.

[0111] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0112] Based on the same inventive concept, this application also provides a question-and-answer processing apparatus for implementing the question-and-answer processing method described above. The solution provided by this apparatus is similar to the solution described in the above method; therefore, the specific limitations in one or more question-and-answer processing apparatus embodiments provided below can be found in the limitations of the question-and-answer processing method described above, and will not be repeated here.

[0113] In one exemplary embodiment, such as Figure 4 As shown, a question-answering processing device integrating intent recognition and retrieval enhancement generation technology is provided, wherein:

[0114] The first text vector acquisition module 401 is used to generate an embedding fine-tuning model based on the target query question and retrieval enhancement input by the user, and to retrieve the first text vector that matches the target query question from the document content vector knowledge base;

[0115] The target question vector acquisition module 402 is used to retrieve question vectors that are semantically similar to the target query question from the question vector knowledge base based on the target query question input by the user, and obtain the target question vector;

[0116] The second text vector acquisition module 403 is used to obtain a text vector associated with the target question vector according to the indexing mechanism between the document content vector knowledge base and the question vector knowledge base, so as to obtain a second text vector;

[0117] The target answer acquisition module 404 is used to input the first text vector, the second text vector and the target query question into the large language model, so that the large language model integrates the first text vector, the second text vector and the target query question to generate the target answer corresponding to the target query question.

[0118] In one embodiment, the apparatus further includes a vector knowledge base construction module, configured to: acquire a retrieval enhancement generation embedding fine-tuning model; vectorize relevant documents in the domain of the target query question according to the retrieval enhancement generation embedding fine-tuning model to obtain several text vectors, and store them in a document content vector knowledge base; obtain the original question and the enhanced question according to relevant documents in the domain of the target query question; vectorize the original question and the enhanced question according to the retrieval enhancement generation embedding fine-tuning model to obtain several question vectors, and store them in a question vector knowledge base.

[0119] In one embodiment, the vector knowledge base construction module is further configured to: traverse relevant documents in the domain to which the target query question belongs based on a large language model, analyze each paragraph or sentence in the relevant documents, and identify content containing key information; generate a corresponding original question based on the content containing key information; and perform data augmentation on the original question based on the large language model to obtain an augmented question.

[0120] In one embodiment, the vector knowledge base construction module is further configured to: combine the original question with the content containing key information corresponding to the original question to obtain a preliminary question-answer pair set; add the enhanced question to the preliminary question-answer pair set according to the association between the original question and the enhanced question to obtain a question-answer pair set; and construct an indexing mechanism between the document content vector knowledge base and the question vector knowledge base according to the question-answer pair set.

[0121] In one embodiment, the vector knowledge base construction module is further configured to: combine the original question and the enhancement question into multiple sets of semantically similar question pairs to obtain a fine-tuning dataset; obtain a basic embedding model to be trained based on the sentence converter model; and train the basic embedding model to be trained based on the fine-tuning dataset to obtain a retrieval enhancement generation embedding fine-tuning model.

[0122] In one embodiment, the target answer acquisition module 404 is further configured to: merge the first text vector and the second text vector to obtain a merged text vector; input the merged text vector and the target query question into a large language model, so that the large language model integrates the merged text vector and the target query question to generate a target answer corresponding to the target query question.

[0123] The modules in the aforementioned question-answering processing device that integrates intent recognition and retrieval enhancement generation technologies can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0124] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 5 As shown, the computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores data for embodiments of a question-answering processing method that integrates intent recognition and retrieval enhancement generation technology. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements a question-answering processing method that integrates intent recognition and retrieval enhancement generation technology.

[0125] Those skilled in the art will understand that Figure 5The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0126] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0127] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0128] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0129] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0130] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0131] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0132] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A question-answering processing method integrating intent recognition and retrieval enhancement generation technologies, characterized in that, The method includes: Based on the user-input target query question and retrieval enhancement, an embedding fine-tuning model is generated, and a first text vector matching the target query question is retrieved from the document content vector knowledge base. Based on the user's input target query question, retrieve question vectors that are semantically similar to the target query question from the question vector knowledge base to obtain the target question vector; Based on the indexing mechanism between the document content vector knowledge base and the question vector knowledge base, a text vector associated with the target question vector is obtained to obtain a second text vector; The first text vector, the second text vector, and the target query question are input into a large language model, so that the large language model integrates the first text vector, the second text vector, and the target query question to generate the target answer corresponding to the target query question.

2. The method according to claim 1, characterized in that, Before retrieving a first text vector matching the target query question from the document content vector knowledge base based on the user-input target query question, the method further includes: Obtain retrieval enhancements to generate embedded fine-tuning models; Based on the retrieval enhancement, an embedded fine-tuning model is generated, and relevant documents in the domain of the target query question are vectorized to obtain several text vectors, which are then stored in the document content vector knowledge base. Based on relevant documents in the domain of the target query question, we obtain the original question and the enhanced question; Based on the retrieved enhancement generation embedding fine-tuning model, the original question and the enhanced question are vectorized to obtain several question vectors, which are then stored in a question vector knowledge base.

3. The method according to claim 2, characterized in that, The process of obtaining the original question and enhanced questions based on relevant documents in the domain of the target query question includes: Based on the large language model, relevant documents in the domain of the target query question are traversed, each paragraph or sentence in the relevant documents is analyzed, and content containing key information is identified. Based on the content containing key information, generate the corresponding original question; Based on the large language model, the original problem is augmented to obtain the augmented problem.

4. The method according to claim 2, characterized in that, The method further includes: Based on the original question and the content containing key information corresponding to the original question, a preliminary question-answer pair set is obtained; Based on the relationship between the original question and the enhanced question, the enhanced question is added to the preliminary question-answer pair set to obtain the question-answer pair set; An indexing mechanism is constructed between the document content vector knowledge base and the question vector knowledge base based on the question-answer pair set.

5. The method according to claim 2, characterized in that, The process of obtaining the retrieval enhancement generation embedding fine-tuning model includes: The original problem and the enhancement problem are combined into multiple sets of semantically similar problem pairs to obtain the fine-tuned dataset; Based on the sentence converter model, the underlying embedding model to be trained is obtained; Based on the fine-tuning dataset, the base embedding model to be trained is trained to obtain the retrieval enhancement generative embedding fine-tuning model.

6. The method according to claim 1, characterized in that, The step of inputting the first text vector, the second text vector, and the target query question into a large language model, so that the large language model integrates the first text vector, the second text vector, and the target query question to generate a target answer corresponding to the target query question, includes: The first text vector and the second text vector are merged to obtain a merged text vector; The merged text vector and the target query question are input into a large language model, so that the large language model merges the merged text vector and the target query question to generate the target answer corresponding to the target query question.

7. A question-answering processing device integrating intent recognition and retrieval enhancement generation technology, characterized in that, The device includes: The first text vector acquisition module is used to generate an embedding fine-tuning model based on the user-input target query question and retrieval enhancement, and to retrieve a first text vector that matches the target query question from the document content vector knowledge base; The target question vector acquisition module is used to retrieve question vectors that are semantically similar to the target query question from the question vector knowledge base based on the target query question input by the user, and obtain the target question vector; The second text vector acquisition module is used to obtain a text vector associated with the target question vector based on the indexing mechanism between the document content vector knowledge base and the question vector knowledge base, so as to obtain the second text vector; The target answer acquisition module is used to input the first text vector, the second text vector, and the target query question into a large language model, so that the large language model integrates the first text vector, the second text vector, and the target query question to generate a target answer corresponding to the target query question.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images