Method and device for generating answer text, storage medium and electronic device

Abstract

The application discloses a kind of answering text generation method and device, storage medium and electronic equipment, it is related to artificial intelligence technical field, comprising: obtaining first question, and according to first question generates first answering text;At least one reference question based on containing first question generates predicted question, wherein, reference question is used to indicate the question of having generated corresponding answering text;According to predicted question, retrieve first query result in text database, wherein, first query result is the data matched with predicted question;Second question is obtained, and in the case where second question and first query result match, first query result is used to generate second answering text.The technical problem that the current retrieval generation method exists lower efficiency in the case of processing continuous question and answer is solved, and the technical effect of improving question and answer efficiency is achieved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence, and in particular to a method and device for generating an answer text, a storage medium and an electronic device. BACKGROUND

[0002] In practical applications, such as enterprise-level customer service systems, professional knowledge Q&A platforms, and news media content recommendation systems, retrieval augmented generation (RAG) systems need to be able to quickly respond to a large number of continuous input prompts. These inputs may cover similar topics, but may also vary in details.

[0003] However, the RAG architecture of the prior art tends to execute each module linearly, i.e., starting from the reception of the user prompt, the processing flow of indexing, retrieval, and generation is executed one by one. This design, when faced with a large number of continuous inputs, results in significant time delays and waste of computing resources, as each query needs to be retrieved and processed from scratch. And since large inference models usually take a long time to generate high-quality answers, this linear execution mode often fails to provide immediate responses when users ask questions quickly and continuously, affecting the user experience. That is, the retrieval generation method in the related art has the problem of low efficiency in handling continuous Q&A. SUMMARY

[0004] The present application provides a method and device for generating an answer text, a storage medium and an electronic device to at least solve the problem of low efficiency of the retrieval generation method in the related art in handling continuous Q&A.

[0005] The present application provides a method for generating an answer text, comprising: obtaining a first question and generating a first answer text according to the first question;

[0006] Generating a predicted question based on at least one reference question containing the first question, wherein the reference question is used to indicate a question for which an answer text has been generated;

[0007] Retrieving a first query result in the text database according to the predicted question, wherein the first query result is data matching the predicted question;

[0008] Obtaining a second question and generating a second answer text using the first query result if the second question matches the first query result.

[0009] The present application also provides a device for generating an answer text, comprising: a first generation module for obtaining a first question and generating a first answer text according to the first question;

[0010] The prediction question generation module is configured to generate a prediction question based on at least one reference question including the first question, wherein the reference question is used to indicate a question for which the answer text has been generated;

[0011] The query module is configured to retrieve a first query result from the text database according to the prediction question, wherein the first query result is data matched with the prediction question;

[0012] The second generation module is configured to obtain a second question, and generate a second answer text by using the first query result in a case where the second question matches the first query result.

[0013] The application further provides an electronic device, including a memory configured to store a computer program, and a processor configured to execute the computer program to implement the steps of the method for generating an answer text.

[0014] The application further provides a computer readable storage medium, which stores a computer program, wherein the computer program is executed by a processor to implement the steps of the method for generating an answer text.

[0015] The application further provides a computer program product, which includes a computer program, and the computer program is executed by a processor to implement the steps of the method for generating an answer text.

[0016] According to the application, the first question is obtained, and the first answer text is generated according to the first question; the prediction question is generated based on at least one reference question including the first question, wherein the reference question is used to indicate a question for which the answer text has been generated; the first query result is retrieved from the text database according to the prediction question, wherein the first query result is data matched with the prediction question; the second question is obtained, and the second answer text is generated by using the first query result in a case where the second question matches the first query result. After the first answer text for the first question is generated, the question in the future can be predicted based on at least one reference question including the first question and the generated answer text, so that the prediction question is generated. Then, the corresponding first query result can be retrieved from the text database based on the prediction question. After the second question is newly input by the user, if the second question matches the first query result found by the prediction question, the retrieval process of the second question can be omitted, and the second answer text is generated by using the first query result. In this way, the prediction of the question is performed between two questions input by the user, and the retrieval is performed in advance in the text database according to the prediction question, so that the retrieval time is reduced in a case where the second question matches the first query result. Therefore, the problem that the retrieval generation method in the related art has low efficiency in processing continuous questions and answers can be solved. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments. Obviously, the drawings described below are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained from these drawings without creative labor.

[0018] Figure 1 is a schematic diagram of a hardware environment of an optional answer text generation method according to an embodiment of the present application;

[0019] Figure 2 is a flow chart of an optional answer text generation method according to an embodiment of the present application;

[0020] Figure 3 is a schematic diagram of an optional answer text generation method according to an embodiment of the present application;

[0021] Figure 4 is a schematic diagram of another optional answer text generation method according to an embodiment of the present application;

[0022] Figure 5 is a schematic diagram of an optional predicted answer text generation method according to an embodiment of the present application;

[0023] Figure 6 is a schematic diagram of an optional second answer text generation method according to an embodiment of the present application;

[0024] Figure 7 is a schematic diagram of another optional second answer text generation method according to an embodiment of the present application;

[0025] Figure 8 is a structural block diagram of an optional answer text generation device according to an embodiment of the present application. DETAILED DESCRIPTION

[0026] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the protection scope of the present application.

[0027] It should be noted that in the description of the present application, the term "comprising" or "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or apparatus including a series of elements includes not only those elements, but also other elements not explicitly listed, or inherent to such a process, method, article or apparatus. The terms "first", "second" and the like in the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence.

[0028] In order for those skilled in the art to better understand the present application, the present application will be further described in detail below in conjunction with the drawings and specific embodiments.

[0029] According to an aspect of an embodiment of the present application, a method for generating an answer text is provided. As an optional implementation, the method for generating an answer text can be applied to a system for generating an answer text in a hardware environment as shown in Figure 1 , but is not limited thereto. The system for generating an answer text can include, but is not limited to, a terminal device 102, a network 110, a server 112, and a database 114. The terminal device 102 runs a target client (for example, as shown in Figure 1 , the target client is a client that can generate a text answer to a question). The terminal device 102 includes a display 108, a processor 106, and a memory 104. The display 108 can be used to display a system interface and the like, and also to provide a human-computer interaction interface to receive human-computer interaction operations of operations on the interface and touch controls on different controls. The processor is used to generate an interaction instruction in response to the human-computer interaction operation, and send the interaction instruction to the server. The memory is used to store system files.

[0030] In addition, the server 112 includes a processing engine 116, which is used to perform storage or reading operations on the database 114. Specifically, the processing engine 116 reads a reference question from the database 114.

[0031] Suppose Figure 1The terminal device 102 in the above embodiment runs a client for generating a text answer. The specific process of the embodiment is as follows: as shown in step S102, the terminal device 102 sends a first question to the server 112 through the network 110. The server 112 performs steps S104-S108 to generate a first answer text according to the first question; generates a predicted question based on at least one reference question containing the first question, wherein the reference question is used to indicate a question for which a corresponding answer text has been generated; and retrieves a first query result in a text database according to the predicted question, wherein the first query result is data matched with the predicted question. Then, the terminal device 102 performs step S110 to send a second question to the server 112. Then, the server 112 performs step S112 to generate a second answer text using the first query result in the case that the second question matches the first query result.

[0032] Optionally, in the embodiment, the terminal device 102 can be a terminal device configured with a target client, which can include but is not limited to at least one of the following: a mobile phone (such as an Android mobile phone, an iOS mobile phone, etc.), a notebook computer, a tablet computer, a palm computer, a MID (Mobile Internet Device), a PAD, a desktop computer, a smart television, etc. The target client can be a client that supports providing a text answer generation function. The network can include but is not limited to a wired network and a wireless network, wherein the wired network includes a local area network, a metropolitan area network, and a wide area network, and the wireless network includes Bluetooth, WIFI, and other wireless communication networks. The server can be a single server, a server cluster composed of multiple servers, or a cloud server. The above is only an example, and the embodiment does not make any limitation on this.

[0033] In an optional implementation, the system first receives a first question from a user, and the RAG system performs a search in the knowledge base based on the question and generates a detailed first answer text. Based on the first question and / or the questions for which answers have been generated in the historical dialogue, the system intelligently predicts the next possible question, and these predicted questions are used to search for possible relevant information for subsequent use. The predicted questions are searched to obtain a first query result matched with the predicted questions, which can include a preliminary answer or relevant information to the predicted question. When the user proposes a second question, if the question matches the first query result of the predicted question, the system directly generates a second answer text using the result, avoiding redundant search and explanation processes, thereby greatly accelerating the response speed of the system.

[0034] Specifically, the scheme can be applied in a scenario supporting question and answer, for example:

[0035] In an online customer service system, users often ask a series of related questions. For example, a user may first ask, "How do I set up my Wi-Fi router?" and then may ask, "How should I change the Wi-Fi password?" or "How can I improve the Wi-Fi signal strength?" The present solution generates predicted questions (such as password change and signal optimization) based on the first question (such as Wi-Fi setup), and quickly generates answers using the retrieved first query results when the user raises the relevant second question, without the need for re-retrieval, significantly improving the speed and efficiency of customer service responses.

[0036] For an intelligent educational assistance platform, students may ask a series of coherent questions during the learning process, such as when learning calculus, they may start with basic concepts and gradually ask more complex questions. For example, the first question is "What is the basic definition of derivative?" The subsequent question may be "How do you find the derivative of a quadratic function?" or "What are the applications of derivatives in physics?" By generating predicted questions based on the first question, the platform can retrieve relevant first query results (such as the calculation method of the derivative of a quadratic function and examples of physical applications) in advance, and when the student raises the second question, generate a high-quality second answer text immediately, promoting learning continuity and efficiency.

[0037] In a medical information query system, a user may first ask a basic question, such as "What are the symptoms of a cold?" and then may ask more specific questions, such as "How can I distinguish between a common cold and the flu?" or "What are the preventive measures for a cold?" Through the present solution, the system generates predicted questions related to the cold after the user raises the first question, retrieves relevant literature and materials to form the first query result. When the user raises the second question, the system can quickly identify the relevance of the question to the first query result and directly generate a second answer text, providing immediate and accurate medical information to support users in making more informed health decisions.

[0038] The application scenarios of the present solution are not limited to the above examples, and the present method can be used in any dialogue system or question-answering system.

[0039] An embodiment of the present application provides a method for generating an answer text, Figure 2 An optional flowchart of a method for generating an answer text according to an embodiment of the present application is shown in FIG. Figure 2 The method for generating an answer text includes the following steps:

[0040] In step S202, a first question is obtained, and a first answer text is generated based on the first question;

[0041] It is noted that the first question can be the first or the most recent inquiry or request from the user in the interactive process, which is the starting point or the intermediate point of the entire dialogue or interactive process. The first answer text is the first response generated based on the first question through the internal processing flow, aiming to provide an answer or relevant information to the user's inquiry.

[0042] In an optional embodiment, the system needs to receive the first question from the user, which usually occurs when the user interacts with the system through a user interface such as a chat window or voice input. After receiving the user's question, the system generates the first text answer for the first question through a series of internal processes, including but not limited to retrieving the knowledge base, refining the content using the Refiner LLM, and reasoning and answer generation using the Reasoning LLM. The specific generation process is described later, and will not be repeated here.

[0043] It is noted that in the answer text generation system, step S202 is the cornerstone of the entire interactive process. When the user raises the first question, the system immediately enters the working state, and its internal components begin to work together to generate the most relevant and accurate first answer text.

[0044] In an optional embodiment, the user can input the first question through the interface, which can be a text query or a voice command, marking the beginning of the interaction. After the user interface receives the question, it passes the information to the query engine. The query engine uses the text database to retrieve relevant documents and content blocks based on the keywords or semantics of the first question. The large amount of retrieved content can exceed the processing capacity of the downstream reasoning model. Therefore, the Refiner LLM can be used to refine the retrieval results, compress and reorganize the text, retain the most critical information, and reduce redundancy, ensuring that the information input to the reasoning model is both concise and informative. The refined information is passed to the Reasoning LLM, which is a large-scale parameter language model with stronger processing capacity, used to understand and generate high-quality answers. It will combine historical information in the text repository to conduct in-depth analysis on the first question and generate the first answer text.

[0045] Figure 3 is a schematic diagram of an optional answer text generation method according to an embodiment of the present application; as Figure 3As shown, the user can start a question-answering session through the user interface 302. The user interface 302 represents the interface through which the user interacts with the system, where the user inputs queries or prompts. The user input question 304 is the textual information input by the user, which serves as the starting point for the system to process and generate answers (e.g., the first question). The query engine 306 is responsible for retrieving relevant documents and content blocks from the database based on the user input question. The query results 308 are the set of documents and content blocks retrieved by the query engine from the text database based on the question. The refinement model 312 can be a specially designed model for compressing and optimizing the query results to adapt to the context length limit of the reasoning language model. The filtered query results 314 can be the query results processed by the refinement model, which have been compressed and reordered to retain key information. The generation model 316 can be a high-performance language model that generates the final answer based on the filtered results and the context. The textual answer 318 is the detailed answer output by the system to the user prompt.

[0046] At step S204, a predicted question is generated based on at least one reference question including the first question, wherein the reference question is used to indicate a question for which a corresponding answer text has been generated;

[0047] It should be noted that the reference question refers to a question that has been processed by the system and for which a corresponding answer text has been generated. These questions are part of the system's knowledge base and are used to guide and optimize the subsequent query and answer generation process. The predicted question is a question that the user is likely to ask based on the current dialogue context and historical data. These predicted questions can be used to optimize the retrieval and generation process, and to prepare possible answers in advance.

[0048] In an optional implementation, the system uses the current user query (the first question) and a set of previously processed questions (reference questions) to generate a series of predicted questions at step S204. These reference questions are not limited to the first question, but can also include multiple questions in the previous dialogue, as long as the system has generated corresponding answers.

[0049] When the user asks the first question, the system combines this question with the reference questions in the history, which constitute the patterns of user queries and system responses known to the system. Based on this information, the model attempts to predict the subsequent questions that the user is likely to ask in the current dialogue context, generating predicted questions that cover a variety of possibilities to improve the response speed and accuracy of the generated results. The reference question is a question in the same dialogue as the first question. If the first question is the first question in the current dialogue, the reference question only includes the first question. If the first question is an intermediate question in the current continuous question-answering session, multiple reference questions including the first question can be used to generate predicted questions.

[0050] In optional implementations, the history of the current conversation can be collected and analyzed, including all the user's questions (reference questions) and the system's generated responses. Key features are extracted from the historical conversation, including the topic of the questions, the type of questions (such as information queries, operation guide requests, concept explanation needs, etc.), and the user's question patterns (such as continuous questions, repeated inquiries on the same topic, etc.). The information in the historical conversation is integrated to form a comprehensive context containing the context of the conversation, the topic background, and the user's intention. The above generation model or a specific prediction model can be used to make deep reasoning based on the integrated context to predict the user's possible subsequent question direction.

[0051] During the generation process, the system may use question template matching techniques to generate a series of possible question templates based on the types and patterns of historical questions. In order to cover possible angles of questioning, the system will generate multiple predicted questions that start from different angles but are closely related to the current conversation topic.

[0052] After generation, each predicted question can be evaluated to determine its relevance score to the current conversation context, ensuring the relevance and usefulness of the predicted questions. The feasibility of the predicted questions is also checked to ensure that they are reasonable and answerable questions, avoiding the generation of invalid or meaningless questions.

[0053] Step S206, according to the predicted question, a first query result is retrieved from the text database, wherein the first query result is data matched with the predicted question;

[0054] It should be noted that the text database can be a database storing a large amount of text information, including documents, content blocks, etc., for retrieving information matching user queries. The first query result is a set of data matched with the predicted question retrieved from the text database.

[0055] The system retrieves information related to the predicted question from the text database based on the predicted possible question. The retrieval process will produce a series of data that matches the predicted question, constituting the first query result.

[0056] In optional implementations, once the predicted question is generated, the query engine immediately performs retrieval in the text database to find data matching the predicted question. The text database may contain various knowledge documents and content blocks, and the query engine will use vector similarity, keyword matching, etc. to locate the most relevant information. After the retrieval process is completed, the system will collect the data paragraphs or documents that highly match the predicted question and encapsulate them as the first query result. These results may be directly used as system preprocessing resources, or further refined to adapt to the context length limit of the downstream LLM.

[0057] Step S208, the second question is obtained, and in the case that the second question matches the first query result, the first query result is used to generate a second answer text.

[0058] In optional embodiments, the second question can be a follow-up question raised by the user after receiving the first answer text or during the process of generating the first answer text, usually based on the understanding of the first answer or the need for further clarification. After the second question is raised, the system will evaluate its relevance to the first query result, and if the second question directly or indirectly involves a part of the first query result, it is considered that the two match. The second answer text can be the second answer generated by the system based on the matching of the second question and the first query result, aiming to further answer the user's doubts or provide additional information.

[0059] In optional embodiments, when the user raises the second question, the system first checks whether the question matches the previously retrieved first query result. This check can be based on semantic similarity evaluation, possibly using word embedding, syntactic analysis or other NLP techniques. If the match is successful, the system will directly use the existing information in the first query result related to the second question to generate the second answer text, which reduces repeated retrieval and speeds up response.

[0060] Embodiment 1:

[0061] Suppose a user of a technology company is seeking help on the company's newly launched smart device (X). Through the smart customer service platform, the user interacts with AI, hoping to understand the device's settings, common problem solutions, and advanced function usage methods through a series of questions. Then the process of using the above answer text generation method is:

[0062] S202: Obtain the first question and generate the first answer text according to the first question;

[0063] The user sends the first question "How to initialize X?" through the smart customer service platform. After the customer service system receives the question, it first retrieves the information related to "initializing X" in the text database through the query engine. Suppose a set of detailed initialization steps are found in the database. Then, the generation model generates a detailed first answer text based on the retrieved information, which contains all the steps of device initialization.

[0064] S204: Generate a predicted question based on at least one reference question including the first question;

[0065] After generating the first answer text, the system predicts the subsequent question that the user can ask based on the first question and the existing knowledge base content. Here, the predicted questions that the system can predict include: "How to connect to WiFi after initialization?", "How to set the security password of the device?", and "What are the advanced functions of the device?"

[0066] S206: Retrieving the first query result in the text database according to the predicted question;

[0067] The system retrieves in the text database for each predicted question to find the matching data and answers. For example, the question "How to connect to WiFi?" can retrieve the WiFi connection tutorial of X.

[0068] S208: Obtaining the second question and generating the second answer text using the first query result in the case that the second question matches the first query result;

[0069] After receiving the first answer, the user continues to ask: "After initialization, how do I connect to the WiFi at home?" The customer service system quickly identifies that the new question of the user matches one of the previously predicted questions. The question "connect to WiFi" corresponds to the first query result (i.e., the WiFi connection tutorial). At this time, the system can directly generate the second answer text using the first query result without performing comprehensive retrieval again, thereby greatly accelerating the response speed.

[0070] According to the present application, the first question is obtained, and the first answer text is generated according to the first question. The predicted question is generated based on at least one reference question including the first question, wherein the reference question is used to indicate the question for which the corresponding answer text has been generated. The first query result is retrieved in the text database according to the predicted question, wherein the first query result is the data matched with the predicted question. The second question is obtained, and the second answer text is generated using the first query result in the case that the second question matches the first query result. After generating the first answer text for the first question, the question that can be asked in the future can be predicted based on at least one reference question including the first question for which the answer text has been generated, thereby generating the predicted question. Then, the corresponding first query result can be retrieved from the text database based on the predicted question. After the user inputs the second question, if the second question matches the first query result found by the predicted question, the retrieval process of the second question can be omitted, and the second answer text is generated using the first query result. In this way, the prediction of the question is performed between the two questions input by the user, and the text database is retrieved in advance according to the predicted question. Therefore, the retrieval time is reduced in the case that the second question matches the first query result. Therefore, the problem that the retrieval and generation method in the related art has low efficiency in handling continuous question and answer can be solved.

[0071] In an optional embodiment, after obtaining the first query result from the text database according to the predicted question, the method comprises: storing the first query result in a text storage library, wherein the text storage library is used to store reference questions, answer texts corresponding to the reference questions, and query results corresponding to the reference questions.

[0072] After obtaining the second question, the method comprises: retrieving a second query result from the first text database according to the second question; and in the case that the second question matches the first query result, generating a second answer text using the first query result, which comprises: generating the second answer text using the first query result, the second query result, and text data in the text storage library.

[0073] It should be noted that the text storage library is a database for storing historical queries, corresponding question answers, and retrieval results, which are used for reference in subsequent queries and generation.

[0074] Figure 4 is a schematic diagram of another optional answer text generation method according to an embodiment of the present application; as shown in Figure 4 The text storage library 402 is used to maintain and manage the dialogue history between the user and the system, as well as the relevant document information retrieved. Through the integration of these information, the system can better understand the dialogue context, and thus generate more coherent and relevant answers. That is, the question 304 input by the user, the query result 308, and the text answer 318 can all be input into the text storage library 402 for storage.

[0075] In an optional embodiment, the text storage library can be used to store the retrieved documents and content blocks, and can be a traditional database, a collection of text files, or a storage system specially designed for text data. For example, a NoSQL database such as MongoDB or Cassandra can be used, which is good at handling large amounts of structured and unstructured data, and can efficiently store and query text documents. In the process of indexing the content blocks, an inverted index structure can be used to quickly locate specific text segments. The above embodiments are only examples, and the specific implementation of the text storage library can be set as needed.

[0076] The task of the Refiner model 312 is to filter and compress the content chunks retrieved from the knowledge base to meet the input limitations of downstream models. Natural language processing techniques such as text summarization, key information extraction, etc. can be used. For example, using sequence-to-sequence based learning models: using pre-trained models such as T5 or BART (Bidirectional and Auto-Regressive Transformers), which perform well on text summarization and translation tasks, can effectively convert long texts into refined summaries that retain key information. Custom refinement models can also be used: it is possible to build a refinement model based on the Transformer architecture but with fewer parameters, tailored for the refinement task to achieve a balance between efficient compression and retention of key information. The above implementations are only examples, and the implementation of specific refinement models can be set as needed.

[0077] The generation model 316 is responsible for generating answers based on the refined context, which may involve complex reasoning, association, and creativity. For example, using generation models such as GPT-3 or GPT-4, based on the Transformer architecture, with tens of billions or even hundreds of billions of parameters, can generate high-quality text in a wide range of tasks. Custom reasoning models can also be used: it is possible to build a reasoning model for a specific field (such as law, medicine, or technology) by fine-tuning on relevant domain corpus to improve its understanding and generation capabilities in specific topics. The above implementations are only examples, and the implementation of specific generation models can be set as needed.

[0078] In optional implementations, when the user starts interacting with the system, the text store can be in an empty state, without storing any historical information. As the user asks questions and the system generates answers, the text store begins to store the conversation history, including questions and answers, as well as the retrieved relevant document content chunks. Whenever new content chunks are retrieved, the Refiner LLM (refinement model) is called to refine these chunks, compressing unnecessary information and retaining the most critical parts, and then storing the refined chunks.

[0079] To avoid excessive redundancy of information, the text store can have a mechanism to clean up outdated or irrelevant context information, ensuring the effectiveness of the storage and the efficiency of the system operation. For example, a time window or question relevance threshold can be set, and when the information is outdated or the relevance is reduced, it will be removed from the context.

[0080] It should be noted that the text database can include a first text database, which can be a database with relatively fixed data, and the data in the database can be updated periodically. It needs personnel to maintain the data therein. The first text database can be a vector database and a knowledge database.

[0081] The main function of the vector database is to store and retrieve the vector representation of documents or knowledge segments, so as to quickly find the most relevant information when receiving user questions. Each document or content block in the vector database is converted into its vector representation form. This vectorization process is usually completed through a pre-trained deep learning model (such as BERT, RoBERTa, etc.), which maps text into a dense vector space, so that similar content has a similar vector representation. Through the distance measurement (such as cosine similarity, Euclidean distance, etc.) in the vector space, the vector database can quickly locate the content block most matching the user query in a large number of documents, greatly reducing the retrieval time. In order to support efficient retrieval, the vector database often uses highly optimized data structures and algorithms (such as inverted index, HNSW, ball tree, etc.), which ensures fast access in high-dimensional vector space.

[0082] When a user submits a question, the system will convert the question into its vector representation, and then search for the nearest neighbor vectors in the vector database. The documents corresponding to these vectors are the information related to the question. The vector representation of the most relevant document or content block retrieved will be returned to the subsequent processing module.

[0083] Unlike the vector database that stores vector representations, the knowledge database saves the complete content of the documents, including various formats of text, charts, videos, etc. The knowledge database contains a large number of documents and information blocks, which can be structured (such as database tables), semi-structured (such as XML files) or unstructured (such as plain text, web content). The documents in the knowledge database cover a wide range of topics and fields, aiming to provide the RAG system with the most comprehensive knowledge coverage possible to meet the consultation and query needs of various users. The knowledge database needs to be updated and maintained regularly to reflect the latest information and data. Updates can be achieved through manual addition of documents, automated scraping of resources on the Internet, or cooperation with third-party data providers. When the vector database locates the vectors related to the user's question, the knowledge database will be queried to obtain the specific document content, which will be passed to the subsequent processing stage of the RAG system.

[0084] In optional embodiments, when the user actually asks the second question, the system retrieves the second query result in the first text database. If the second question matches or is related to the first query result already existing in the text repository, the system will directly use the stored query result, combine the currently retrieved second query result, and the text data in the text repository to generate a richer and faster second answer text.

[0085] Through the above embodiments of the present application, the predictive retrieval is performed when the system is idle and the results are stored in the text repository, which can efficiently handle the user's continuous questioning. In particular, when the second question is related to the previous predicted question, the stored query result can be directly used to accelerate the answer generation, thereby improving the user experience and system efficiency.

[0086] In optional embodiments, after retrieving the second query result in the first text database according to the second question, the following steps are included: in the case that the second question does not match the first query result, calculating the similarity between the second query result and a reference query result, wherein the reference query result is the query result corresponding to a reference question; in the case that the similarity between the second query result and the reference query result is greater than a first similarity, generating the second answer text by using the second query result, the reference query result, and the text data in the text repository.

[0087] It should be noted that the reference query result is a query result generated based on the reference question and used as a comparison benchmark. The reference query result can be the result obtained by querying the text database in the process of generating the reference answer text using the reference question. After the query is completed, the reference query result can be saved to the text repository. In the case that the second question does not match the first query result, the reference query result generated in the current round of dialogue can be queried in the text repository.

[0088] When the second question does not match the first query result, the system first evaluates the similarity between the second query result and the previously stored reference query result. If the similarity between the second query result and the reference query result exceeds a set threshold (the first similarity), the system will use these results and the information in the text repository to comprehensively generate the second answer text.

[0089] In an optional embodiment, the system retrieves information for the second question in the first text database, obtaining a second query result. Meanwhile, the system checks whether the second query result matches or is highly similar to a reference query result from history (possibly from a predictive question or the processing of a similar question before). In the case that there is a significant similarity between the second query result and the reference query result, the system compares the similarity between the two sets of results. This step quantifies the degree of overlap between the information by calculating a similarity score between the texts, such as cosine similarity or Jaccard similarity. If the calculated similarity is higher than a preset first similarity threshold, it indicates that the second query result is sufficiently related to the reference query result and can be used to generate an answer. At this point, the system combines the second query result, the reference query result, and other text data in the text repository to generate a second answer text based on these information.

[0090] Through the above embodiments of the present application, predictive and historical data are effectively utilized, reducing the repetition of information retrieval, accelerating the answering time of questions, and improving the user experience. In this way, even if the new question raised by the user is different from the historical question or the predictive question in literal terms, the essential connection between the questions can be accurately captured, and timely and high-quality answers can be provided.

[0091] In an optional embodiment, after calculating the similarity between the second query result and the reference query result, the following is included: in the case that the similarity between the second query result and the reference query result is less than or equal to the first similarity, a third query result is retrieved according to the second question in the second text database; and the second answer text is generated based on the second query result, the third query result, and the text data in the text repository.

[0092] It should be noted that the second text database can be another information source different from the first text database, which can contain more specific or comprehensive data to provide supplementary information when the first database retrieval is insufficient.

[0093] The second text database can be an external data source. Similar to a web search engine, its retrieval range in the RAG system is more limited to a specific document set or knowledge field. Based on the keywords in the user's question, the search engine scans the knowledge library or the documents on the Internet to find information containing these keywords. In addition to the keywords, the search engine can also perform full-text retrieval to find complete sentences or paragraphs related to the question. The second text database as an external data source can help the system integrate the latest or most relevant external information, enhancing the timeliness and comprehensiveness of the answer.

[0094] The first text database can be an internal database maintained by the system, and the second text database can be an external data source connected to the network. By using the two different databases, the scope of the query can be expanded, and the accuracy of the final text answer can be ensured.

[0095] The third query result is an additional query result retrieved from the second text database when the similarity between the second query result and the reference query result is low.

[0096] In optional embodiments, in the RAG system, when processing consecutive questions, it is crucial to ensure the accuracy and relevance of the answers. When the user raises a second question, the system automatically retrieves information from the first text database to obtain a second query result. In the preliminary processing stage, the system calculates the similarity between the second query result and the reference query result stored in the text repository to determine whether the reference query result can be reused to quickly generate an answer. If the calculated similarity is lower than the first similarity threshold, it means that the second query result and the reference query result have a large difference in content or low relevance. In this case, the system will take additional retrieval steps, i.e., conduct a more in-depth or extensive query from the second text database to obtain a third query result to supplement or correct the understanding of the second question. Subsequently, the system will integrate the second query result, the third query result, and other text data in the text repository to generate a more detailed and accurate second answer text, ensuring that the user obtains the most relevant and useful information.

[0097] Through the above-mentioned embodiments of the present application, when constructing an information system, different database types and search tools can be selected or combined according to the specific needs of the application program to achieve the best performance and function balance. For example, the vector database can quickly locate similar text fragments, the knowledge database provides in-depth and structured information, and the search engine can be used to expand the knowledge base or obtain real-time data. Through this process, even if there is a content jump or expansion in the user's consecutive questions, appropriate and comprehensive information can be provided in a timely manner, and the accumulation of the text repository can enhance the system's ability to handle complex and diverse problems.

[0098] In optional embodiments, when the second question matches the first query result, generating the second answer text using the first query result includes: calculating the similarity between the first query result and the second query result; and when the similarity between the first query result and the second query result is greater than the first similarity, determining that the second question matches the first query result, and generating the second answer text using the first query result, the second query result, and the text data in the text repository.

[0099] It is to be noted that when the system receives the second question, it compares the first query result with the query result of the second question, and makes a matching judgment by calculating the similarity. If the similarity exceeds the set first similarity threshold, it is considered that the two questions are related to a certain extent, and the system can directly use the information in the first query result, combined with the second query result and the relevant data in the text storage library, to quickly generate the second answer text.

[0100] In an optional embodiment, the second query result is compared with the first query result stored in the text storage library, and the text similarity or information relevance between the two is calculated. If the calculated similarity is higher than the pre-set first similarity threshold, the system determines that the second question has sufficient relevance with the first query result, and can use the information in the first query result to generate an answer. After confirming the match, the system will combine the first query result, the second query result and other text data in the text storage library, comprehensively analyze and generate the second answer text, thereby saving the time for retrieving and processing information from scratch and speeding up the response speed.

[0101] It is to be noted that, as mentioned before, the first text database can be an internal architecture database, and the second text database can be an external data source. Obviously, in the process of querying the database, the querying process of the second text database will take a long time. The reference query result and the first query result queried by the reference question and the prediction question will be stored in the text storage library, so in the case of querying the second question, it can be determined from the text storage library whether there is relevant information for generating an answer to reduce the query time.

[0102] In an optional embodiment, when querying based on the second question, if the second query result queried in the first text database has been stored in the text storage library, the system will immediately interrupt the querying of the second text database which takes the longest time, to avoid unnecessary calculation and time consumption.

[0103] In an optional embodiment, after obtaining the second question, similar vectors are searched in the vector database (Vector DB), and relevant documents are located in the knowledge database (Knowledge DB). When the query engine starts to return the second query result, the text storage library checks whether these query results match the information stored in the past. It compares the document content, source ID and other key features of the new query result with the filtered results stored in the history.

[0104] If the second query result highly matches the filtering result stored in the text database, it means that the current question can quickly generate an answer based on the existing information without the need for time-consuming full retrieval or building an answer from scratch. After confirming the match, the system will immediately interrupt the longest time-consuming query process in the second text database, avoiding further time and resource waste.

[0105] Through the above-mentioned embodiments of the present application, by calculating the similarity between the query results and reusing the previous query results to generate answers when the similarity meets the conditions, the RAG system can significantly improve the response efficiency of user questions on continuous or similar topics. And by interrupting repeated and time-consuming external queries, the system can generate answers faster, significantly improving response speed. Thus, shorter waiting time, more coherent dialogue process, and better user interaction experience are achieved.

[0106] In an optional embodiment, generating the second answer text using the first query result comprises: vectorizing the first query result and the second query result to obtain a first vector and a second vector, wherein the first vector is used to indicate the key features of the first query result and the second query result, and the second vector is used to indicate the context information of the first query result and the second query result; generating a third vector based on the second question, wherein the third vector is used to indicate the information of the second question; and calculating the second answer text based on the first vector, the second vector, and the third vector.

[0107] It is necessary to explain that vectorization is a process of converting text information into numerical vectors, which facilitates the understanding and processing of machine learning models. The first vector can be a Key vector, which represents the information source that the Query will focus on in the attention mechanism. In the RAG system, the Key vector is usually associated with the retrieved documents or content blocks in the knowledge base (query results). Each content block is converted into one or a set of Key vectors to help the model determine which parts are most relevant to the current query. The second vector can be a Value vector, which contains the actual information content extracted from the knowledge base and is the direct data source used by the model to generate answers. Corresponding to the Key vector, each Value vector is associated with a Key vector, representing the information of a specific content block. The Value vector plays the role of a "storage warehouse" or "information library" in the attention mechanism, and its importance lies in that it carries the specific information that the model will use to construct answers. The third vector can be a Query vector, which is the vector representation of the user input or system-generated query. When the user asks a question or the system predicts a possible follow-up question, the question is converted into a Query vector through a language model. This vector represents the semantic and content features of the question. The Query vector is the query signal in the attention mechanism, and its main responsibility is to compare with the Key vector to determine which Value vector contains the information needed to answer the question. The generation and optimization of the Query vector are the key to ensuring that the RAG system can accurately understand the user's intention and quickly locate relevant knowledge.

[0108] In optional implementations, the interaction between the Query vector and the Key vector can help the model decide which information to extract from the Value vector. This mechanism allows the model to focus on the most relevant information fragments when processing large amounts of text data, rather than traversing all possible texts, thereby significantly improving the speed and quality of answer generation.

[0109] It is necessary to explain that the system converts the first query results retrieved previously and the second query results obtained by the current query into numerical vector form, respectively called the first vector and the second vector, for further analysis and processing. The new question raised by the user is converted into a numerical vector, namely the third vector, to indicate the information content of the question.

[0110] Through the above-mentioned embodiments of the present application, through such vectorization and calculation, the system can not only give accurate answers based on the second question of the user, but also organically integrate the relevant information of the historical retrieval result into the answer, further improving the completeness and reliability of the answer. This method is especially suitable for scenarios of processing complex queries and generating high-quality answers, because it can effectively utilize historical data and real-time retrieval information to provide users with more personalized and in-depth answers.

[0111] In an optional embodiment, after retrieving the first query result in the text database according to the predicted question, the method comprises: generating a first prediction vector and a second prediction vector based on the first query result, and generating a prediction answer text based on the first prediction vector and the second prediction vector.

[0112] Generating the second answer text using the first query result comprises: if the similarity between the predicted question and the second question is greater than the second similarity, generating a third vector based on the second question, and calculating the second answer text based on the first prediction vector, the second prediction vector and the third vector.

[0113] It should be noted that the first prediction vector is a vector representation converted from the first query result, which is used for subsequent vector operations and information processing. The second prediction vector is another set of vector representations that the system may generate after generating the first prediction vector to enhance the generation of the prediction answer text. The prediction answer text is a predicted answer of the system to the predicted question, which is generated based on the first query result and the relevant prediction vector. The second similarity is a quantitative indicator for measuring the similarity between the predicted question and the second question, which is used to determine whether the prediction answer text or the first query result can be directly used to generate the second answer text.

[0114] It should be noted that when the generation model (which can be LLM) first processes a certain specific question, such as prompt_A, it will go through a series of calculation processes to generate Key vector (first vector) and Value vector (second vector). These vectors will leave traces in the cache inside the model, which is K, V cache. Next time when a very similar question, such as prompt_B, is encountered, the model can skip the repeated calculation of K and V vectors, and instead directly use the vectors stored in K, V cache last time. This is because similar prompts are likely to point to the same or similar information sources, and the vector representations (K and V) of these information sources are essentially the same.

[0115] After obtaining the first query result, the system first converts the first query result into a first prediction vector and a second prediction vector, and then performs vector operation or information fusion using the two sets of vectors to generate a prediction answer text for the prediction question. When the system detects a high similarity between the prediction question and the second question actually proposed by the user, it generates a third vector for the second question, and then combines the first prediction vector and the second prediction vector to generate a second answer text more relevant to the second question through certain calculation or information reconstruction process.

[0116] In an optional embodiment, the system first performs retrieval in the text database according to the prediction question to obtain the first query result. Subsequently, the query results are converted into a first prediction vector and a second prediction vector, which contain the key information and semantic features of the retrieval results, and prepare for further information processing and text generation. By using the first prediction vector and the second prediction vector for vector operation or information fusion, the system can generate a prediction answer text for the prediction question. These texts are based on the historical information stored in the context manager and the current retrieval results, and are intended to prepare answers in advance for the questions that the user may propose. When the user actually proposes a second question, the system first evaluates the similarity between the second question and the previous prediction question. If the similarity exceeds the preset second similarity threshold, it means that the two have a high correlation, and the previously generated prediction answer text or the first query result can be used as a reference to reduce repeated retrieval and processing work. On the basis of similarity judgment, the system generates a third vector based on the second question, and performs operation or information reconstruction using the first prediction vector, the second prediction vector and the third vector to generate a second answer text more accurate and relevant to the second question.

[0117] Through the above embodiments of the present application, by avoiding repeated calculation, the system can significantly reduce the processing time, especially when processing long texts or large amounts of data. For similar or repeated queries, the response speed of the system is greatly improved, improving the user experience. Especially when processing continuous user input, the response efficiency and the relevance of information of the system can be significantly improved.

[0118] In an optional embodiment, generating the second answer text using the first query result includes: in a case where the similarity between the prediction question and the second question is less than or equal to the second similarity, calculating the similarity between the second question and the reference question; in a case where the similarity between the second question and the reference question is greater than the second similarity, generating a third vector based on the second question; and calculating the second answer text based on the first reference vector, the second reference vector and the third vector corresponding to the reference question.

[0119] It should be noted that the first reference vector can be a vector representation related to the reference question, used for information retrieval and similarity calculation. The second reference vector can be another vector representation corresponding to the same reference question or related historical question, used for further information analysis and fusion.

[0120] In an optional embodiment, when the user raises a second question, the system first checks the similarity between the predicted question and the second question, and if it is not close enough, further assesses the similarity between the second question and the historical reference question. By calculating the similarity, the system can determine whether to reuse the previous query result or need to conduct a new query. When the second question is identified to have high similarity with the reference question, the system utilizes vector fusion technology to integrate the historical query result and the information of the current question to generate a more specific and high-quality second answer text.

[0121] Through the above-mentioned embodiments of the present application, the similarity between the historical query result and the predicted question can be intelligently evaluated to determine whether to reuse the information, and the second answer text for the subsequent question of the user can be efficiently generated through vector fusion technology. The accuracy and speed of the answer are improved by using existing information resources, thereby providing a better user experience.

[0122] In an optional embodiment, storing the first query result in the text storage library includes: determining a first mark of the first query result, wherein the first mark is used to indicate the source of the first query result; and storing the first query result in the text storage library in a case where the reference mark corresponding to each of the at least one text data in the text storage library is different from the first mark.

[0123] It should be noted that the first mark is a unique identifier used to mark the source of the first query result, such as a specific database of retrieval, a document ID, or a content creation date. The reference mark can be a unique identifier associated with the source of each text data stored in the text storage library. When the reference mark of the existing data in the text storage library is the same as the first mark of the newly obtained first query result, it indicates that the information already exists and does not need to be stored repeatedly.

[0124] As mentioned earlier, the text storage library is responsible for managing the context of the dialogue, including historical dialogue records, retrieved document content (query result), and output of the large model (LLM) (answer text). One of its main functions is to update the content stored in itself according to the final answer result, ensuring that the knowledge and information in the system are the latest and most relevant.

[0125] When the system generates the answer text, the text repository analyzes the query result and the text blocks of the answer text in this answering process, especially paying attention to the source ID of these blocks. The source ID is an identifier used to track the original location of each content block or document, whether the information is obtained from the knowledge database, the vector database or the external search engine. If the content or reference material corresponding to this ID has not been stored in the text repository, the corresponding data will be written to enrich its internal knowledge base. Through this mechanism, its storage can be automatically updated to ensure that each time the user asks a question, the system can generate an answer based on the most recent and relevant information, thereby improving the accuracy and timeliness of the answer.

[0126] In an optional embodiment, whenever the system retrieves information from the knowledge base or database to generate the first query result, a first mark is assigned to this result to record its source. This mark can be based on the unique ID of the document, the version number of the content block, or a specific code of the retrieval time point, aiming to ensure that each retrieval can be uniquely identified. Before attempting to store the first query result, the system will traverse the text repository to compare the first mark of the new result with the reference mark of each item of text data in the repository. If it is found that there is no reference mark of any item of text data that is the same as the first mark, it means that the current query result is new and has not been stored before, at which time the system will add the first query result to the text repository to expand its knowledge range.

[0127] Through the above embodiments of the present application, repeated processing of the same information is avoided, and the computing cost is reduced. It is ensured that the dialogue is based on the latest knowledge state, and the quality of user interaction is improved. And it is ensured that each storage is a beneficial supplement to the knowledge base of the system, while unnecessary repeated occupation of resources is avoided, effective use of data and continuous optimization of the system are realized.

[0128] In an optional embodiment, after generating the first answer text according to the first question, the method further includes: storing the first answer text in the text repository; performing data compression on the first answer text to obtain a simplified first answer text, and replacing the first answer text in the text repository with the simplified first answer text.

[0129] It should be noted that both the query results and the answer texts generated in the question and answer dialogue can be compressed by the refining large model. The refining large model (Refiner LLM) can optimize the query results during the generation of answers, ensuring that the information finally input into the reasoning large model (generation model) can meet the context length limit and maintain the integrity of the key content. Refiner LLM compresses and reorders a large number of documents and content blocks retrieved, removes redundant information, and extracts the most relevant and valuable parts to provide refined input for subsequent reasoning processes. Refiner LLM can significantly reduce the number of tokens in the search results, thereby avoiding processing exceptions caused by exceeding the context length limit of the downstream large model, while ensuring that the compressed information still has sufficient semantic coherence and information density. During compression, Refiner LLM needs to have the ability to understand the context and identify which information is the key point in the current dialogue or query context to determine the retention or discard strategy.

[0130] In optional embodiments, the refining model is designed as a model with fewer parameters. Due to fewer parameters, the refining model can run with lower computational resource requirements, quickly compress the retrieved document fragments and context history, reduce the number of tokens, and retain key information. This method not only speeds up information processing, but also reduces overall operating costs.

[0131] In optional embodiments, the refining model can use attention mechanisms to filter out the most relevant information fragments. This method can identify the parts of the text that have the greatest impact on the current problem, and then effectively compress them without losing key information. The length and content of the context can also be dynamically adjusted according to the characteristics of the current problem, ensuring that each compression meets the context length limit of the large model while maintaining the coherence of the context.

[0132] In optional embodiments, the system stores the initial, uncompressed first answer text in the text repository as a historical record for future reference and learning. The system uses specialized data compression techniques to refine the first answer text, eliminating unnecessary redundant information and retaining core content and key data to generate a refined first answer text. The refined first answer text is used to replace the original first answer text in the text repository, ensuring the integrity and usability of the information in the repository while significantly reducing storage space requirements and improving the management and retrieval efficiency of the repository.

[0133] Through the above-mentioned embodiments of the present application, through such embodiments, the RAG system can not only provide detailed answers, but also ensure efficient management and use of the text repository, quickly locate and reuse relevant information even in the face of a large number of historical answers, and further improve the response speed and quality of the system to subsequent user questions.

[0134] Embodiment 2:

[0135] Figure 5 is a schematic diagram of an optional method for generating a predicted answer text according to an embodiment of the present application; as Figure 5 shown, after the generation model 502 generates the first answer text of the first question, step S502 can be performed to input the first answer text into the text repository 504; the text repository 504 can perform step S504-1 to compress the first answer text through the refining model, i.e., send the first answer text to the refining model 508, which can refine (compress) it, and perform step S506-1 to store the compressed first answer text to the text repository 504. After receiving the first answer text, the text repository 504 can immediately perform step S504-2 to prompt the generation model 502 to generate a predicted question, and the generation model 502 can perform step S506-2 to generate a predicted question based on the reference question including the first question, and send the generated predicted question to the text repository 504. The text repository 504 can perform step S508 to query the text database based on the predicted question, i.e., send the predicted question to the text database 506 to enable the text database 506 to query the relevant text data of the predicted question. The text database 506 can perform step S510 to return the first query result to the text repository 504. Then, the text repository 504 performs step S512-1 to send the first query result and the predicted question to the generation model, and the generation model 502 generates a corresponding predicted answer text, and performs step S514-1 to send the predicted answer text to the text repository. In synchronization or asynchronization with the above process, the text repository 504 can perform step S512-2 to send the first query result and the predicted question to the refining model 508 for compression, and after the refining model 508 refines it, perform step S514-2 to send the compressed first query result and the predicted question to the text repository 504.

[0136] In an optional embodiment, the system resources are effectively utilized during the Idle period of the RAG system, and the response speed of subsequent user questions is accelerated by generating predictive prompts (predicted questions) and pre-processing. Figure 5Detailed steps of predictive cue processing during Idle time are shown. When the generation model generates the final answer, the answer (first answer text) along with its internal knowledge and source ID are written to the text repository, preparing for the follow-up question. The system is in a waiting state between the user question and the final answer generation, or between two rounds of questions. During the system Idle time, the system generates multiple predictive cues (predicted questions) that are based on the future possible questions predicted from the last round of user questions. The refinement model receives the predictive cues and reads the relevant information (such as the previously retrieved document snippets and the dialog history) from the text repository, performs a quick filtering and compression to reduce the number of tokens while retaining the key information. After the generation model processes the predictive cues, the generated filtered results are logged in the query log as a reference for the follow-up questions. The content chunks filtered and compressed by the refinement model are written to the text repository, overwriting the original unfiltered query results. If there are false cues that are not verified in the predictive cues, they are also processed by the refinement model and the processing results are logged so that the system can learn and improve its prediction algorithm.

[0137] Embodiment 3

[0138] Figure 6 is a schematic diagram of an optional second answer text generation method according to an embodiment of the present application. As shown in Figure 6 the process of generating a predictive answer text shown in Figure 5 If the user interface 602 receives a second question from the user after step S510 during the process of generating a predictive answer text based on the predictive question, the user interface 602 will perform step S602 to send the second question to the text database 506, and the text database 506 will query the second query result based on the second question. If the second question matches the first query result, no third query result will be generated, and step S604 will be performed to send the second query result to the text repository 504. Then, the text repository 504 will perform step S606-1 to send the first query result, the second query result and the second question to the generation model 502, and the generation model will perform step S608-1 to send the second answer text to the text repository 504 after generating the second answer text. During this process, synchronously or asynchronously, the text repository 504 can perform step S606-2 to send the first query result, the second query result and the second question to the refinement model 508 for compression, and the refinement model 508 will perform step S608-2 to send the compressed first query result, the second query result and the second question to the text repository 504 after refining them.

[0139] When the second prompt_B (second question) appears, since it is very short time interval from prompt_A (first question), the system is likely to have not fully released the resources of the model. At this time, the system will immediately start the RAG process and enter the retrieval stage, but because prompt_B and the predicted question may have semantic similarity, the system can directly find the query results related or identical to prompt_B from the text repository. In the case of similar prompts, directly obtaining query results from the text repository means that the system does not need to call the external search engine (second text database) again, thereby avoiding the time and resources required for re-retrieval. The query of the external search engine is usually one of the most time-consuming links, so this change greatly improves the response speed of the system to prompt_B.

[0140] In the context of short time interval between two prompts, by fully utilizing the stored query results, the RAG system can significantly reduce the retrieval time and improve the response speed to the user's continuous questions. This mechanism not only saves computing resources, but also improves user experience, especially in the case of fast and continuous answering of similar questions, such as multi-round dialogue or real-time query scenarios. In this way, even in a short time, the system can efficiently handle the user's new questions without repeating the time-consuming retrieval process, thereby achieving fast and accurate information retrieval and generation.

[0141] In an optional implementation, generating the second answer text using the first query result includes: retrieving a second query result in the first text database according to the second question; in a case where a similarity between the first query result and the second query result is greater than the first similarity and a similarity between the predicted question and the second question is greater than the second similarity, obtaining a first predicted vector and a second predicted vector, wherein the first predicted vector and the second predicted vector are obtained by vectorizing the first query result; generating a third vector for the second question, and calculating the second answer text based on the first predicted vector, the second predicted vector, and the third vector.

[0142] In an optional embodiment, the system first retrieves the second query result in the database based on the new second question. The historical first query result and the current second query result are converted into vector format, and the similarity between them is calculated; at the same time, the second question is vectorized, and the similarity between the predicted question is calculated. This process processes information through mathematical processing, and judges the close relationship between the two groups of query results and questions. When the system detects that the similarity of the two groups of query results and the predicted similarity between the questions both meet the threshold condition, the system considers that the first query result has contribution value to generate the second answer text. Therefore, combined with the first predicted vector, the second predicted vector and the third vector, the most appropriate answer text to the second question is calculated through a specific algorithm (such as weighting, fusion, etc.). This strategy ingeniously uses historical data, avoids duplication of work, and greatly improves the generation efficiency and quality of the answer.

[0143] Embodiment 4:

[0144] Figure 7 is a schematic diagram of another optional second answer text generation method according to an embodiment of the application; as Figure 7 shown, after the process of generating the predicted answer text shown in Figure 5 is completed, if the user sends a second question to the user interface 602, the user interface 602 will execute step S702, and send the second question to the text database 506. Then the text database 506 finds the second query result related to the second question, and in the case that the second query result is similar to the first query result, the query for the third query result can be stopped, and step S704 is directly executed to send the second query result to the text storage 504. The text storage 504 can then execute step S706-1, and send the first query result, the second query result and the second question to the generation model 502. In the case that the second question is similar to the predicted question or the reference question, the generation model 502 internally executes step S708-1, and generates the second answer text based on the first predicted vector and the second predicted vector. Then step S710 is executed to send the second answer text to the text storage 504. In this process, synchronously or asynchronously, the text storage 504 can execute step S706-2, and send the first query result, the second query result and the second question to the compression model 508 for compression. After the compression model 508 compresses them, step S708-2 is executed to send the compressed first query result, the second query result and the second question to the text storage.

[0145] In an optional embodiment, in a RAG (Retrieval-Augmented Generation) system, when the generation model finishes the final result generation for one predictive prompt (predictive question), the system state is ready for the new user input. At this stage, the state of the system components is as follows: the generation model stores the Key and Value vectors related to the previous predictive prompt (predictive question) in its internal K, V (Key-Value) Cache. These vectors are the product of the model encoding the relevant documents and content chunks during the generation process, used to accelerate the processing of subsequent similar requests.

[0146] Suppose in this case, the user inputs a new prompt (second question), which has a certain degree of similarity or relevance to the previously processed predictive prompt. At this time, the RAG system will use the existing optimization mechanism to speed up the processing flow: the system first queries the text repository, which is a component that stores historical dialogue records, query results, and other related context information. Since the previous predictive question and its query results have been written into the text repository, the system can quickly retrieve the filtered information related to the new prompt. After obtaining the filtered information, the text database does not need to perform time-consuming external database queries (such as search engines). This means that for new prompts with overlapping information with historical queries, the system can almost instantaneously provide query results, greatly reducing retrieval time. When the generation model receives the new prompt, it can quickly find the Key and Value vectors associated with the new question by calling the internal K, V Cache. This is because similar questions often point to the same or similar information sources, and these information sources have been encoded and stored in the Cache in previous queries. By directly using the information in the Cache instead of regenerating, the generation model can significantly speed up its generation of the final answer, reducing the consumption of computing resources.

[0147] Through the above embodiments of the present application, as can be seen from this example, the RAG system effectively reuses previously retrieved information through vectorization and similarity calculation, greatly accelerating the answering process of subsequent related questions while ensuring the quality and accuracy of the answers. This mechanism is very effective for handling continuous and related query scenarios, significantly improving user experience and system efficiency.

[0148] In an optional implementation, before obtaining the second question, the process includes: generating M prediction questions based on at least one reference question including the first question, where M is an integer greater than 0; iterating through the M prediction questions and performing the following operations: retrieving the i-th first query result from the text database based on the i-th prediction question, and generating the i-th predicted answer text based on the i-th first query result; storing the i-th prediction question, the i-th first query result, and the i-th predicted answer text in a text repository, where i is an integer greater than 0 and less than or equal to M.

[0149] In an optional implementation, predicting potential follow-up questions from the user is a crucial step in optimizing the response generation speed of the RAG system. Before acquiring a second question, the system generates M possible predicted questions based on historical dialogues, especially the most recent user question (the first question). The value of M can be adjusted according to system design and resource availability, but it must be a positive integer, indicating that the system will attempt to generate at least one predicted question.

[0150] For each generated predicted question (the i-th predicted question), the system performs the following operations: It queries the text database to find information matching the predicted question, obtaining the i-th first query result. Based on the retrieved information, the system generates a predictive answer text, the i-th predicted answer text, to preview possible user question-and-answer scenarios. The i-th predicted question, the corresponding first query result, and the predicted answer text are stored in a text repository, providing a foundation for rapid retrieval and answer generation for subsequent user questions.

[0151] like Figure 7 As shown, after executing step S502, steps S504-1 to S514-1 can be repeated to continuously generate prediction questions and store the corresponding prediction answer text and the first query result in a text repository. Steps S702-S710 will only be executed when the user inputs the second question into the user interface 602.

[0152] In optional embodiments, in RAG systems, when there is a long time interval between two user questions, the system faces unique challenges and opportunities. The long idle time provides an opportunity for the system to perform deep optimization and preprocessing, but at the same time, the system needs to be able to efficiently manage large volumes of data to meet the demand for rapid response to subsequent questions. The system can generate multiple sets of predictive prompt words (predicted questions) during the long idle time of the user based on historical interactions and content in the knowledge base. Each set of predictive prompt words represents a possible direction or pattern of user questioning. For each set of predictive prompt words, the system performs a complete RAG process, i.e., retrieves the text database, obtains query results, and uses the generation model to generate the final answer (predicted answer text). These results are then stored in the text repository for subsequent use. Before storing each round of question and answer results, the refinement model filters and compresses them, reducing the original 128K tokens to 64K tokens, i.e., the compression rate is 50%. In this way, not only does it reduce the demand for storage space, but it also improves the processing efficiency of the downstream generation model.

[0153] During the long idle period, the system can store the results of up to 30 rounds of predictive question and answer, which significantly increases the amount of information compared to the designed 15 rounds. This means that the text repository contains more extensive knowledge and contextual information, increasing the probability of finding relevant information when subsequent questions are asked. When the user asks a question again, if its prompt word is related to any of the predictive question and answer stored in the text repository, the system can directly retrieve the optimized material from the inside without having to start the time-consuming external search engine again, greatly shortening the information retrieval and processing time.

[0154] Example 5:

[0155] Suppose the RAG system is interacting with a researcher interested in "new energy vehicles". The user first asks "What are the types of new energy vehicles?" (first question). While waiting for the user's possible second question, the system predicts the following M (e.g., M=5) possible subsequent questions based on the first question:

[0156] 1. "What are the advantages of new energy vehicles compared to traditional fuel vehicles?";

[0157] 2. "What are the charging technologies for new energy vehicles?";

[0158] 3. "How is the sales situation of new energy vehicles in the current market?";

[0159] 4. "What is the future development trend of new energy vehicles?";

[0160] 5. "How does the maintenance cost of new energy vehicles compare to fuel vehicles?".

[0161] The system will iterate through these 5 prediction questions:

[0162] For the first prediction question, the system performs a search in the text database to obtain information about the advantages of new energy vehicles, generates a predicted answer text, and stores the question, result, and text in the text repository. Similarly, the above steps are performed for each of the other prediction questions until all M prediction questions have been processed and all information has been stored in the text repository. When the user actually poses a second question that matches one of the prediction questions, such as asking "What are the charging technologies for new energy vehicles?", the system can directly retrieve the previously generated predicted answer text from the text repository and quickly provide a detailed and accurate answer without having to perform time-consuming database searches and answer generation processes again, thereby significantly improving the response speed and efficiency of the system and enhancing the user's interactive experience.

[0163] Through the above embodiments of the present application, the prediction questions are generated, searched, and results stored in advance before the user poses a second question, and the RAG system can quickly respond to the user's subsequent queries using the information in the text repository, achieving more intelligent and efficient information processing and answer generation. This strategy not only improves the efficiency of the system, but also provides a smooth and instant interactive experience for the user, especially when dealing with continuous or highly related questions, the effect is particularly significant.

[0164] In optional embodiments, multi-level refinement processing can be performed, and the dynamicity and intelligence of the context management mechanism can be enhanced. Through more detailed information filtering and dynamic adjustment of the context, the system can provide higher quality dialogue services under limited resources.

[0165] A primary Refiner (primary refinement model) can be implemented to quickly preprocess high-volume query results, retaining only the most relevant information units. A secondary Refiner (advanced refinement model) is introduced to further refine the output of the primary Refiner, further optimizing content quality and context length. The amount and type of historical records stored in the Context Manager (text repository) can be dynamically adjusted according to the complexity of the current question and the relevance of the previous and subsequent questions. In the case of high user query frequency and high similarity of questions, the context window is automatically expanded, and the number of retained historical dialogues and Refiner LLM (refinement model) outputs is increased.

[0166] In optional implementations, the primary Refiner focuses on high-speed, large-batch query result refinement, using simpler algorithms and looser refinement criteria to quickly filter out irrelevant information. The secondary Refiner, based on the output of the primary, uses more complex and refined algorithms to perform in-depth analysis on the refined data, ensuring that the information presented to the Reasoning LLM (generative model) is both comprehensive and accurate. A set of context management algorithms is developed to intelligently adjust the content storage strategy in the Context Manager based on the topic, complexity of the question, and user interaction habits. For example, in the technical consultation field, the algorithm tends to retain more professional terminology and contextual concepts; while in the field of daily conversation, it may focus on maintaining the smoothness of the conversation and emotional connection. Combined with user historical conversation records and predictive cue words, a predictive model is built to estimate the likelihood and relevance of future questions. When the predicted question matches the actual question highly, not only the previously generated answer text is reused, but also the stored answer text is used as part of the answer, further reducing the burden on the Reasoning LLM and accelerating the generation speed.

[0167] Example 6:

[0168] In an education tutoring application, a student may ask multiple questions about the same mathematical concept in succession. For example, from "What is a quadratic equation?" to "How to solve a quadratic equation?" to "How to check if the roots of a quadratic equation are correct?", these questions are closely related and the student's question pattern is predictable.

[0169] The primary Refiner quickly filters out content unrelated to "quadratic equations", and the secondary Refiner refines the documents of the solution method and the verification steps to ensure that the information ultimately delivered to the student is accurate and hits the point. When the student's conversation is identified to be focused on a single topic, the Context Manager automatically adjusts to retain more historical conversations and refinement results about "quadratic equations" for subsequent queries. Based on the student's learning progress and question pattern, the system predicts that the next question will be about the application examples or exercises of quadratic equations. It uses the Refiner LLM to retrieve and refine relevant information in advance during the system's idle time, and once the student actually asks the question, the system can immediately call these preprocessed information from the Context Manager, greatly reducing the waiting time and providing a seamless learning experience.

[0170] By implementing the multi-level Refiner LLM architecture, dynamic context management, and adaptive predictive prompt generation, the RAG system can effectively handle different types and complexities of user queries, significantly improve the coherence and response speed of the dialogue. This solution provides a more intelligent and efficient solution for handling massive information and intensive dialogue, significantly improving user satisfaction and the overall performance of the dialogue system.

[0171] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by means of software on a general hardware platform, of course, it can also be implemented by hardware, but in many cases the former is a better implementation.

[0172] Embodiments of the present application also provide a response text generation device, Figure 8 is a structural block diagram of an optional response text generation device according to an embodiment of the present application, as Figure 8 shown, the device comprises:

[0173] The first generation module 802 is configured to obtain a first question and generate a first response text according to the first question;

[0174] The prediction question generation module 804 is configured to generate a prediction question based on at least one reference question including the first question, wherein the reference question is used to indicate a question for which a corresponding response text has been generated;

[0175] The query module 806 is configured to retrieve a first query result in a text database according to the prediction question, wherein the first query result is data matched with the prediction question;

[0176] The second generation module 808 is configured to obtain a second question and generate a second response text using the first query result if the second question matches the first query result.

[0177] Optionally, the query module 806 is further configured to store the first query result in a text storage library, wherein the text storage library is used to store reference questions, response texts corresponding to the reference questions, and query results corresponding to the reference questions; the second generation module 806 is further configured to retrieve a second query result in the first text database according to the second question; and generate the second response text using the first query result, the second query result, and text data in the text storage library.

[0178] Optionally, the second generation module 808 is further configured to: in a case where the second question does not match the first query result, calculate a similarity between the second query result and a reference query result, wherein the reference query result is a query result corresponding to a reference question; and in a case where the similarity between the second query result and the reference query result is greater than the first similarity, generate the second answer text by using the second query result, the reference query result, and text data in the text repository.

[0179] Optionally, the second generation module 808 is further configured to: in a case where the similarity between the second query result and the reference query result is less than or equal to the first similarity, retrieve a third query result from the second text database according to the second question; and generate the second answer text by using the second query result, the third query result, and the text data in the text repository.

[0180] Optionally, the second generation module 808 is further configured to: calculate a similarity between the first query result and the second query result; and in a case where the similarity between the first query result and the second query result is greater than the first similarity, determine that the second question matches the first query result, and generate the second answer text by using the first query result, the second query result, and the text data in the text repository.

[0181] Optionally, the second generation module 808 is further configured to: vectorize the first query result and the second query result to obtain a first vector and a second vector, wherein the first vector is used to indicate key features of the first query result and the second query result, and the second vector is used to indicate context information of the first query result and the second query result; generate a third vector based on the second question, wherein the third vector is used to indicate information of the second question; and calculate the second answer text based on the first vector, the second vector, and the third vector.

[0182] Optionally, the second generation module 808 is further configured to: generate a first prediction vector and a second prediction vector based on the first query result, and generate a predicted answer text based on the first prediction vector and the second prediction vector; in a case where a similarity between the predicted question and the second question is greater than a second similarity, generate a third vector based on the second question, and calculate the second answer text based on the first prediction vector, the second prediction vector, and the third vector.

[0183] Optionally, the second generation module 808 is further configured to: in a case where the similarity between the predicted question and the second question is less than or equal to the second similarity, calculate a similarity between the second question and a reference question; in a case where the similarity between the second question and the reference question is greater than the second similarity, generate a third vector based on the second question; and calculate the second answer text based on a first reference vector, a second reference vector corresponding to the reference question, and the third vector.

[0184] Optionally, the query module 806 is further configured to determine a first mark of the first query result, wherein the first mark is used to indicate a source of the first query result; and store the first query result in the text storage library in a case where each corresponding reference mark of the at least one text data in the text storage library is different from the first mark.

[0185] Optionally, the first generation module 802 is further configured to store the first answer text in the text storage library; perform data compression on the first answer text to obtain a simplified first answer text, and replace the first answer text in the text storage library with the simplified first answer text.

[0186] Optionally, the second generation module 808 is further configured to retrieve a second query result from the first text database according to the second question; in a case where a similarity between the first query result and the second query result is greater than the first similarity and a similarity between the predicted question and the second question is greater than the second similarity, obtain a first predicted vector and a second predicted vector, wherein the first predicted vector and the second predicted vector are obtained by vectorizing the first query result; generate a third vector according to the second question, and calculate a second answer text based on the first predicted vector, the second predicted vector and the third vector.

[0187] Optionally, the second generation module 808 is further configured to generate M predicted questions based on at least one reference question containing the first question, wherein M is an integer greater than 0; traverse the M predicted questions, and perform the following operations: retrieve an i th first query result from the text database according to an i th predicted question, and generate an i th predicted answer text according to the i th first query result; and store the i th predicted question, the i th first query result and the i th predicted answer text in the text storage library, wherein i is an integer greater than 0 and less than or equal to M.

[0188] The features of the embodiments of the answer text generation apparatus can be referred to the related descriptions of the embodiments of the answer text generation method, which will not be repeated here.

[0189] The embodiments of the present application further provide an electronic device, comprising a memory and a processor, the memory stores a computer program, and the processor is configured to run the computer program to perform the steps in any of the embodiments of the answer text generation method.

[0190] The embodiments of the present application further provide a computer readable storage medium, which stores a computer program, wherein the computer program is configured to perform the steps in any of the embodiments of the answer text generation method when running.

[0191] In an example embodiment, the computer readable storage medium described above can include, but is not limited to, a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store computer programs.

[0192] Embodiments of the present application also provide a computer program product, which comprises a computer program, and the computer program, when executed by a processor, implements the steps in any of the above-mentioned method embodiments for generating an answer text.

[0193] Embodiments of the present application also provide another computer program product, which comprises a non-volatile computer readable storage medium, and the non-volatile computer readable storage medium stores a computer program, and the computer program, when executed by a processor, implements the steps in any of the above-mentioned method embodiments for generating an answer text.

[0194] The skilled in the art can further realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be realized by electronic hardware, computer software or a combination of both. In order to clearly illustrate the interchangeability of hardware and software, the components and steps of the examples have been described in the above description in general terms. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. The skilled in the art can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0195] The above provides a method and device for generating an answer text, a storage medium and an electronic device. The principles and implementation modes of the present application are described by applying specific examples. The above description of the examples is only used to help understand the method of the present application and its core idea. It should be pointed out that for those skilled in the art, without departing from the principles of the present application, some improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of the present application.

Claims

1. A method for generating response text, characterized in that, include: Obtain the first question and generate the first answer text based on the first question; A prediction question is generated based on at least one reference question, including the first question, wherein the reference question is used to indicate a question for which a corresponding answer text has been generated; A first query result is obtained by retrieving data from a text database based on the prediction question, wherein the first query result is data that matches the prediction question; Obtain a second question, and if the second question matches the first query result, retrieve a second query result from the first text database based on the second question; If the similarity between the first query result and the second query result is greater than the first similarity and the similarity between the predicted question and the second question is greater than the second similarity, the first query result, the second query result, and the second question are sent to the generative model to obtain the second answer text.

2. The method according to claim 1, characterized in that, After obtaining the first query result by retrieving the text database based on the prediction question, the process includes: The first query result is stored in a text repository, wherein the text repository is used to store the reference question, the answer text corresponding to the reference question, and the query result corresponding to the reference question; Following the acquisition of the second question, the following is included: The second query result is obtained by retrieving data from the first text database based on the second question. The step of sending the first query result, the second query result, and the second question to the generation model to obtain the second answer text includes: The second answer text is generated using the first query result, the second query result, and the text data in the text repository.

3. The method according to claim 2, characterized in that, After obtaining the second query result from the first text database based on the second question, the process includes: If the second question does not match the first query result, the similarity between the second query result and the reference query result is calculated, wherein the reference query result is the query result corresponding to the reference question; If the similarity between the second query result and the reference query result is greater than the first similarity, the second answer text is generated using the second query result, the reference query result, and the text data in the text repository.

4. The method according to claim 3, characterized in that, After calculating the similarity between the second query result and the reference query result, the following steps are included: If the similarity between the second query result and the reference query result is less than or equal to the first similarity, a third query result is retrieved from the second text database based on the second question. The second answer text is generated using the second query result, the third query result, and the text data in the text repository.

5. The method according to claim 3, characterized in that, The step of sending the first query result, the second query result, and the second question to the generation model to obtain the second answer text includes: Calculate the similarity between the first query result and the second query result; If the similarity between the first query result and the second query result is greater than the first similarity, the second question is determined to match the first query result, and the second answer text is generated using the first query result, the second query result, and the text data in the text repository.

6. The method according to claim 2, characterized in that, The step of sending the first query result, the second query result, and the second question to the generation model to obtain the second answer text includes: The first query result and the second query result are vectorized to obtain a first vector and a second vector, wherein the first vector is used to indicate the key features of the first query result and the second query result, and the second vector is used to indicate the context information of the first query result and the second query result; A third vector is generated based on the second question, wherein the third vector is used to indicate information about the second question; The second answer text is calculated based on the first vector, the second vector, and the third vector.

7. The method according to claim 6, characterized in that, After obtaining the first query result from the text database based on the prediction question, the process includes: A first prediction vector and a second prediction vector are generated based on the first query result, and a predicted answer text is generated based on the first prediction vector and the second prediction vector. The step of sending the first query result, the second query result, and the second question to the generation model to obtain the second answer text includes: If the similarity between the predicted question and the second question is greater than the second similarity, a third vector is generated based on the second question, and the second answer text is calculated based on the first predicted vector, the second predicted vector, and the third vector.

8. The method according to claim 7, characterized in that, The step of sending the first query result, the second query result, and the second question to the generation model to obtain the second answer text includes: If the similarity between the predicted problem and the second problem is less than or equal to the second similarity, the similarity between the second problem and the reference problem is calculated. If the similarity between the second problem and the reference problem is greater than the second similarity, a third vector is generated based on the second problem; The second answer text is calculated based on the first reference vector, the second reference vector, and the third vector corresponding to the reference question.

9. The method according to claim 2, characterized in that, The step of storing the first query result in a text repository includes: A first flag is determined for the first query result, wherein the first flag is used to indicate the source of the first query result; If the reference flag corresponding to at least one text data in the text repository is different from the first flag, the first query result is stored in the text repository.

10. The method according to claim 2, characterized in that, After generating the first answer text based on the first question, the process includes: The first answer text is stored in the text repository; The first answer text is compressed to obtain a simplified first answer text, and the simplified first answer text is used to replace the first answer text in the text repository.

11. The method according to any one of claims 1 to 10, characterized in that, The step of sending the first query result, the second query result, and the second question to the generation model to obtain the second answer text includes: The second query result is obtained by retrieving data from the first text database based on the second question. If the similarity between the first query result and the second query result is greater than the first similarity and the similarity between the prediction question and the second question is greater than the second similarity, a first prediction vector and a second prediction vector are obtained, wherein the first prediction vector and the second prediction vector are obtained by vectorizing the first query result; A third vector is generated for the second question, and the second answer text is calculated based on the first prediction vector, the second prediction vector, and the third vector.

12. The method according to claim 11, characterized in that, Before obtaining the second question, the following are included: M prediction questions are generated based on at least one reference question, including the first question, where M is an integer greater than 0; The M prediction problems are traversed, and the following operations are performed: The i-th prediction question is used to retrieve the i-th first query result from the text database, and the i-th predicted answer text is generated based on the i-th first query result. The i-th prediction question, the i-th first query result, and the i-th prediction answer text are stored in a text repository, where i is an integer greater than 0 and less than or equal to M.

13. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the method for generating response text as described in any one of claims 1 to 12 when executing the computer program.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, it implements the steps of the method for generating response text as described in any one of claims 1 to 12.

15. A computer program product comprising computer instructions, characterized in that, When the computer instructions are executed by the processor, they implement the steps of the method for generating the response text as described in any one of claims 1 to 12.

Images