Question and answer method and device based on large language model, medium and product

Abstract

The embodiment of the invention provides a question answering method and device based on a large language model, a medium and a product, and relates to the technical field of natural language process.The method comprises the steps that a target question is input into the large language model, so that the target question is decomposed into multiple sub-questions through the large language model; in response to the fact that the large language model is detected to generate a sub-problem, asynchronously calling a retrieval service module to retrieve the generated current sub-problem to obtain a corresponding retrieval result; and inputting the plurality of retrieval results into a large language model to obtain a question and answer result corresponding to the target question. According to the technical scheme provided by the embodiment of the invention, the degree of parallelism in the calculation generation process can be improved and the calculation time can be saved on the premise of not changing the network structure of the large language model.

Description

Technical Field

[0001] This application relates to the field of natural language processing technology, and in particular to a question-answering method based on a large language model, an electronic device, a question-answering system based on a large language model, a computer-readable storage medium, and a computer program product, which can be applied to the field of artificial intelligence. Background Technology

[0002] With the rapid development of natural language processing technology, Large Language Models (LLMs) have become a core tool for acquiring, generating, and improving text content. LLMs can understand and generate natural language and are widely used in chatbots, automatic summarization, translation systems, and other language understanding tasks. Despite the significant progress made by LLMs in content generation, the generated content still faces limitations in accuracy and timeliness. Summary of the Invention

[0003] This application provides a question-answering method based on a large language model, an electronic device, a question-answering system based on a large language model, a computer-readable storage medium, and a computer program product to alleviate or solve one or more technical problems existing in the prior art.

[0004] In a first aspect, embodiments of this application provide a question-answering method based on a large language model, comprising: inputting a target question into a large language model, so that the large language model decomposes the target question into multiple sub-questions; in response to detecting that the large language model generates a sub-question, asynchronously calling a retrieval service module to retrieve the generated current sub-question and obtain corresponding retrieval results; inputting multiple retrieval results into the large language model to obtain question-answering results corresponding to the target question.

[0005] Secondly, embodiments of this application provide an electronic device, including a memory, a processor, and a computer program stored in the memory, wherein the processor implements any of the methods of embodiments of this application when executing the computer program.

[0006] Thirdly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the method of any one of the embodiments of this application.

[0007] Fourthly, embodiments of this application provide a computer program product, including a computer program, which, when executed by a processor, implements any of the methods described in the embodiments of this application.

[0008] According to the technical solution of this application, when generating answers, the large language model no longer relies solely on the knowledge learned during pre-training. It can also dynamically call an external retrieval service module to obtain the latest and most relevant information, thereby improving the accuracy and timeliness of the generated content. Furthermore, by asynchronously calling the retrieval service module—that is, whenever the large language model generates a sub-question, it directly inputs the generated current sub-question into the retrieval service module for retrieval—the computation time for the large language model to generate the next sub-question is hidden within the retrieval time of the retrieval service module for the current sub-question, thus significantly saving computation time and improving the efficiency of question-and-answer result generation. Moreover, by decoupling the large language model and the retrieval service module, the parallelism in the computation generation process can be improved without changing the network structure of the large language model, saving computation time.

[0009] Optionally, the retrieval service module includes a vector generation submodule and a vector retrieval submodule. Each time the LLM generates a sub-question, it asynchronously inputs it into the vector generation submodule, thus hiding the generation time of the LLM for the next sub-question within the vector computation time of the vector generation submodule for the current sub-question. Similarly, each time the vector generation submodule generates a sub-question vector, it asynchronously inputs it into the vector retrieval submodule, thus hiding the generation time of the vector generation submodule for the next sub-question vector within the retrieval time of the vector retrieval submodule for the current sub-question vector. Based on this, sub-question generation, sub-question vector computation, and sub-question vector retrieval can be asynchronous, decoupling the LLM, vector generation submodule, and vector retrieval submodule. This maximizes the parallelization of sub-question decomposition, vector generation, and vector retrieval, thereby saving hardware idle time and improving the efficiency of question-answering result generation.

[0010] Optionally, during the computation and generation process, shared prefixes can be pre-stored. For example, attention for decomposition guide words can be pre-filled during the query decomposition stage, attention for sub-answer guide words can be pre-filled when generating each sub-answer, and attention for question-answer result guide words can be pre-filled when generating the final question-answer result. Thus, when attention is calculated on the input sequence each time, the attention calculation results of shared prefixes can be pre-filled, avoiding the waste of resources caused by repeatedly calculating the attention of long guide word parts.

[0011] Optionally, the advantages of multi-core CPUs or multi-core GPUs, as well as the parallel processing capabilities provided by the computing platform, can be fully utilized to make the LLM process of generating multiple sub-answers in parallel a parallel processing process, thereby improving the computational efficiency of answer generation.

[0012] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application, it can be implemented according to the contents of the specification. In order to make the above and other objects, features and advantages of this application more obvious and understandable, specific embodiments of this application are given below. Attached Figure Description

[0013] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the various drawings denote the same or similar parts or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings depict only some embodiments according to this application and should not be construed as limiting the scope of this application.

[0014] Figure 1 An exemplary architecture diagram of a question-answering system based on a large language model, as shown in an embodiment of this application, is illustrated.

[0015] Figure 2 A timing diagram illustrating an application example of this question-answering system is shown;

[0016] Figure 3 A flowchart of the question-answering method based on a large language model in an embodiment of this application is shown;

[0017] Figure 4 A schematic diagram of the sub-problem vector generation process according to an embodiment of this application is shown;

[0018] Figure 5 Another schematic diagram of the sub-problem vector generation process according to an embodiment of this application is shown;

[0019] Figure 6 A timing diagram of the question-answering method based on a large language model in an embodiment of this application is shown;

[0020] Figure 7 The diagram illustrates an application example of the question-answering method based on a large language model in this embodiment of the application.

[0021] Figure 8 A block diagram of an electronic device provided in an embodiment of this application is shown. Detailed Implementation

[0022] In the following description, only certain exemplary embodiments are briefly described. As those skilled in the art will recognize, the described embodiments can be modified in various ways without departing from the concept or scope of this application. Therefore, the drawings and description are considered to be exemplary in nature and not restrictive.

[0023] To facilitate understanding of the technical solutions of the embodiments of this application, the relevant technologies of the embodiments of this application are described below. The following relevant technologies are optional solutions and can be combined with the technical solutions of the embodiments of this application in any way, and all of them fall within the protection scope of the embodiments of this application.

[0024] The following terms will be used in the following text:

[0025] Large Language Model (LLM): Often shortened to large model, it is a large-scale machine learning model used in Natural Language Processing (NLP) to understand or generate human language. LLMs are trained on very large text datasets using deep learning techniques, enabling them to capture the complex patterns and diversity of natural language.

[0026] The Retrieval Augmented Generation (RAG) framework is a natural language processing model framework that integrates retrieval and generation mechanisms. By introducing an information retrieval module, RAG enhances the knowledge base of LLM. LLM no longer relies solely on the knowledge learned during pre-training but can also dynamically retrieve the latest and most relevant information from external databases.

[0027] Embedding Model: A model that maps high-dimensional sparse data (such as words, sentences, images, etc.) to a low-dimensional dense vector space. This low-dimensional dense vector is called an "embedding vector." The basic idea of ​​an embedding model is to learn a function that transforms input data into a fixed-dimensional dense vector. This function is usually obtained through training a neural network, with the aim of ensuring that similar input data have similar vector representations in the embedding space, while dissimilar data have relatively distant vector representations.

[0028] In the field of natural language processing, large language models can understand and generate natural language, and are widely used in chatbots, automatic summarization, question answering systems, and other language understanding tasks. Despite the significant progress made by large language models in content generation, the accuracy of the generated content still faces limitations.

[0029] In one implementation, queries and user-input documents can be embedded in the LLM, and the retrieved documents can be concatenated into the context, allowing the LLM to incorporate external knowledge information during the response generation process. However, the external knowledge information obtained by this implementation method is very limited.

[0030] In another implementation, a RAG framework can be introduced on top of LLM. The RAG framework incorporates a retrieval mechanism, searching for relevant information in an external knowledge base before generating an answer. This makes the LLM's answer generation more accurate and comprehensive, thereby improving the quality and relevance of the generated content and enhancing the question-answering performance of LLM. However, the RAG framework requires very high computational resources and time when handling complex queries. Both vector generation and retrieval, as well as reasoning in large models, require significant computational resources and time. With the continuous growth of information volume, existing RAG framework implementations for LLM content generation struggle to meet the demands of real-time, efficient content retrieval, thus limiting the performance of RAG frameworks in practical LLM applications.

[0031] In view of this, embodiments of this application provide a question-answering method and system based on a large language model, which can improve the parallelism of the RAG framework and save computation time without changing the model network structure. To facilitate understanding of the embodiments of this application, the system architecture to which the embodiments of this application are applicable is first briefly described. Figure 1 An exemplary architecture diagram of a question-answering system based on a large language model, as shown in an embodiment of this application, is illustrated. Figure 2 A timing diagram illustrating an application example of this question-answering system is shown. Figure 1 and Figure 2 As shown, the question-answering system based on a large language model in this application embodiment includes a client 110 and a server 120, wherein the server 120 is equipped with a large language model and a retrieval service module.

[0032] The large language model and retrieval service modules can be deployed on the same entity or multiple entities. This entity can be an application, service, instance, functional module in software form, virtual machine, container, or cloud server, or a hardware device or chip with data processing capabilities. The server-side 120 can be deployed on local computing devices or cloud computing platforms or clusters providing computing power, storage, and network resources. The specific application architecture can be built according to service requirements. The client-side 110 can be hardware, such as mobile phones, personal computers, tablets, wearable devices, or other electronic devices; the client-side 110 can also be an application (APP) or software module deployed on an electronic device.

[0033] In one application example, such as Figure 2 As shown, users can input target questions in natural language form through the interactive interface provided by client 110, such as the target question "Who is the author of this article, and what dataset was used?" Client 110 then sends the target question to server 120.

[0034] Server 120 invokes a large language model, inputting the target question into it. The large language model then decomposes the target question into multiple sub-questions, such as sub-question 1 "Who is the author of this article?" and sub-question 2 "What dataset did this article use?". These sub-questions can be output in text format. Server 120 then invokes the retrieval service module to search the database for each sub-question, obtaining the corresponding search results. For example, sub-question 1 corresponds to search result 1, and sub-question 2 corresponds to search result 2. These search results can also be output in text format.

[0035] Server 120 invokes a large language model, which infers from multiple search results and the target question, outputting the question-and-answer results corresponding to the target question. Server 120 then returns the question-and-answer results to client 110, allowing the user to obtain the question-and-answer results corresponding to the target question.

[0036] In this embodiment, the calls made by the server 120 to the large language model and the retrieval service module are asynchronous. That is, for each sub-question generated by the large language model, the server 120 asynchronously calls the retrieval service module, inputs the generated sub-question into the retrieval service module, and the retrieval service module retrieves the corresponding search results from the database. In other words, the generation of the next sub-question by the large language model and the retrieval of the current sub-question by the retrieval service module are executed asynchronously.

[0037] In related technologies, large language models need to generate all sub-problems and use the text containing all sub-problems to retrieve the results from the database. This approach can meet user needs in terms of computational performance in short text scenarios, but if the user inputs long text, it will waste a lot of computation time and make it difficult to meet the needs of real-time and efficient retrieval.

[0038] In the technical solution of this application embodiment, the retrieval service module is called asynchronously. That is, whenever the large language model generates a sub-problem, the generated current sub-problem is directly input to the retrieval service module for retrieval. Therefore, the calculation time of the large language model for generating the next sub-problem is hidden in the retrieval time of the retrieval service module for the current sub-problem, which can significantly save calculation time and improve the generation efficiency of the entire system.

[0039] The question-answering system of this application embodiment can be applied to various application scenarios such as human-computer dialogue, machine translation, document analysis, data organization, writing assistance, knowledge learning, and information retrieval, thereby meeting users' needs for real-time and efficient retrieval generation in different scenarios.

[0040] It should be noted that the application scenarios or examples provided in the embodiments of this application are for ease of understanding, and the embodiments of this application do not specifically limit the application of the technical solutions. In addition, the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0041] The technical solution of this application and how it solves the aforementioned technical problems are described in detail below with specific embodiments. The listed specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0042] Figure 3 A flowchart illustrating a question-answering method based on a large language model according to an embodiment of this application is shown. This question-answering method can be applied to and executed by server 120. Figure 3 As shown, the question-and-answer method may include steps S301, S302 and S303.

[0043] Step S301: Input the target problem into the large language model so that the large language model decomposes the target problem into multiple sub-problems.

[0044] The target question is in natural language form, such as a question input by the user. The core structure of the Large Language Model (LLM) is based on the Transformer architecture (a neural network architecture based on a self-attention mechanism), which can understand and generate natural language. In this embodiment, the LLM can understand the target question and decompose it into multiple sub-questions. For example, the LLM processes the original target question input by the user and guides the LLM to generate more specific or more general sub-questions through prompt word engineering.

[0045] For example, such as Figure 4As shown, the target question is a long text, i.e., a complex query containing multiple sub-questions, such as "Who is the author of this article; what dataset was used?" This input sequence is fed into the LLM. After receiving this input sequence, the LLM preprocesses it using word embeddings, converting the text into word vectors of multiple words (such as "author", "writer", "is", "who") that the model can understand. Positional encoding is added to the word vectors according to the order of the words in the input sequence. Based on a self-attention mechanism, the LLM can pay attention to information about other words in the input sequence while processing the current word, thereby capturing contextual dependencies, identifying different parts and relationships in the target question, and progressively generating the corresponding text for each sub-question.

[0046] It should be noted that the embodiments of this application do not specifically limit the network structure of LLM, as long as a problem can be decomposed into multiple sub-problems.

[0047] Step S302: In response to the detection that a sub-question has been generated by the large language model, the retrieval service module is asynchronously invoked to retrieve the generated current sub-question and obtain the corresponding retrieval results.

[0048] In this embodiment, the calls to the LLM and the retrieval service module are asynchronous. That is, each time the LLM generates a sub-question, it is asynchronously input into the retrieval service module, which then retrieves the corresponding search results from the database. Therefore, the process of the LLM generating the next sub-question and the process of the retrieval service module retrieving the current sub-question are asynchronous.

[0049] For example, such as Figure 4 As shown, when the LLM generates the subquestion "Who is the author?", the subquestion is immediately input into the retrieval service module for retrieval, without waiting for the LLM to complete the generation of the next subquestion "What dataset did this article use?".

[0050] The retrieval service module accesses a database to search for sub-questions and outputs one or more search segments corresponding to the sub-questions as search results. For example, each sub-question corresponds to a subquery, and the database is typically an LLM add-on (external) database containing a large amount of knowledge information, which can consist of multiple documents or data items. The retrieval service module queries the database to retrieve the information most relevant to each sub-question. For example, based on the similarity to the vector representation of the sub-question, it extracts the k highest-ranking results from the retrieved information as the search results for that sub-question.

[0051] The retrieval service module can use the RAG framework. It can be a software module integrated into the application as part of the application, or it can be deployed as a standalone service on the server and provide functionality through a network interface.

[0052] Step S303: Input multiple search results into the large language model to obtain the question-and-answer results corresponding to the target question.

[0053] In this application, multiple search results can be input into the LLM sequentially or together after all results have been generated, or they can be input asynchronously as each search result is generated. The LLM performs reasoning on the multiple search results and the target question, and outputs the question-and-answer results corresponding to the target question.

[0054] In one example, multiple search results can be aggregated as an information-enriched contextual input to the LLM. Using this contextual information, the LLM constructs and generates a comprehensive and accurate question-and-answer result. In another example, for each search result, the LLM generates a corresponding sub-answer, such as combining the search result with a sub-question to generate a sub-answer in natural language. All sub-answers are then aggregated, and their contextual semantic understanding capabilities are leveraged to ensure semantic and logical coherence, ultimately generating a complete question-and-answer result.

[0055] Compared to searching a database using text containing all sub-questions, the technical solution of this application embodiment asynchronously calls the retrieval service module. That is, whenever the large language model generates a sub-question, it directly inputs the generated current sub-question into the retrieval service module for retrieval. This hides the computation time of the large language model generating the next sub-question within the retrieval service module's retrieval time for the current sub-question, thereby significantly saving computation time and improving the efficiency of question-and-answer result generation.

[0056] In one implementation, the retrieval service module includes a vector generation submodule and a vector retrieval submodule. In step S302, in response to detecting that the large language model outputs a sub-question, the retrieval service module is asynchronously invoked to retrieve the generated current sub-question and obtain the corresponding retrieval result. This includes: in response to detecting that the large language model generates a sub-question, inputting the generated current sub-question into the vector generation submodule for vector calculation, wherein the generation of the next sub-question by the large language model and the vector calculation of the current sub-question by the vector generation submodule are performed asynchronously; in response to detecting that the vector generation submodule generates a sub-question vector, inputting the generated current sub-question vector into the vector retrieval submodule for retrieval and obtaining the corresponding retrieval result, wherein the generation of the next sub-question vector by the vector generation submodule and the retrieval of the current sub-question vector by the vector retrieval submodule are performed asynchronously.

[0057] Both the vector generation submodule and the vector retrieval submodule can be software modules, integrated into the application as part of it, or deployed as independent services on a server, providing functionality through a network interface. Specifically, the vector generation submodule can be deployed as an independent service, for example, on the same server as the LLM, or deployed separately to improve performance. Alternatively, it can be embedded within the LLM as an embedded model. The vector retrieval submodule can be deployed on a dedicated server for efficient retrieval, tightly integrated with the database, facilitating the retrieval of large amounts of data. For example, the LLM can serve as a core large model, while the vector generation and vector retrieval submodules can be deployed as auxiliary modules in a distributed manner. Communication between the LLM, the vector generation submodule, and the vector retrieval submodule can be achieved through network application programming interfaces (APIs) or remote procedure calls (RPCs).

[0058] In other words, for each subproblem generated by the LLM, its input vector is asynchronously generated by a submodule, thus hiding the generation time of the next subproblem by the LLM within the vector computation time of the current subproblem by the vector generation submodule. Similarly, for each subproblem vector generated by the vector generation submodule, its input vector is asynchronously retrieved by a submodule, thus hiding the generation time of the next subproblem vector by the vector generation submodule within the retrieval time of the current subproblem vector by the vector retrieval submodule.

[0059] Based on this, sub-problem generation, sub-problem vector computation, and sub-problem vector retrieval can be made asynchronous, decoupling the LLM, vector generation submodule, and vector retrieval submodule. This allows for maximum parallelization of sub-problem decomposition, vector generation, and vector retrieval, thereby saving hardware idle time and improving the efficiency of question-answering result generation.

[0060] In one implementation, in response to detecting that a sub-problem is generated by the large language model, the vector generation module for the generated current sub-problem input vector performs vector computation. This may include: in response to detecting that the large language model generates a sequence start identifier, the vector generation module for the sub-problem sequence input vector corresponding to the sequence start identifier performs vector computation to obtain the sub-problem vector corresponding to the current sub-problem. The sub-problem sequence includes a sequence start identifier, the current sub-problem, and a sequence end identifier. When the sequence end identifier is detected, the vector generation module outputs the generated sub-problem vector.

[0061] To facilitate the identification of the LLM's division of each subproblem, each subproblem is wrapped into a structured sequence of subproblems through model fine-tuning or prompt engineering. Each subproblem sequence sequentially contains a sequence start identifier, the subproblem itself, and a sequence end identifier. For example... Figure 4 As shown, <s> "Who is the author?"< / s> "Forms a sequence of subproblems, where," <s> "Each sequence begins with a marker to indicate that what follows is a subproblem,"< / s> The "" symbol serves as a sequence end marker, indicating that the subproblem output has been completed. Furthermore, when a sequence start marker is detected in the content generated by the LLM, it can be assumed that the content generated by the LLM contains structured data, and vector calculations can be performed synchronously in the input vector generation submodule of the subsequently generated subproblems.

[0062] For example, such as Figure 5 As shown, the user's complete query is input into the LLM. The LLM starts by calculating the token. For each token output by the LLM, the system checks whether it is a special token, specifically whether it is a sequence start identifier (such as "..."). <s> If it is a special token, an asynchronous task is started, and the tokens generated after LLM are sequentially sent to the vector generation submodule. Therefore, the subquery and the sequence end marker (such as "") are involved.< / s> All inputs ("") are fed into the vector generation submodule. The vector generation submodule asynchronously computes the vector representation of the subproblem and generates the subproblem vector. If an end-of-sequence marker is detected, the computed subproblem vector is output.

[0063] For example, the vector generation submodule can adopt a transformer architecture, using a hidden layer of the transformer network as output and employing a key-value cache (kv-cache) mechanism to accelerate computation. Specifically, the main function of the transformer network's self-attention mechanism is to generate weighted representations for each input word. These weights are determined by the relationship between the current word and all words in the input sequence (i.e., the sub-problem sequence). For each input word, a corresponding query vector, key vector, and value vector are generated through a linear transformation. For each query vector, its dot product with all key vectors is calculated to obtain an attention score. The score is scaled and normalized to obtain attention weights. The value vectors are then weighted and summed using these attention weights to generate the output vector. During computation, the current sub-problem can be parsed into multiple words, and the attention of each word is calculated based on its query vector, key vector, and value vector. The calculated key vectors and value vectors are cached for later use when calculating the attention of the word. Then, the sub-problem vector corresponding to the current sub-problem is determined based on the attention of each word. Therefore, by using the kv-cache mechanism, the key vectors and value vectors calculated for each word can be cached during the processing of the sub-problem sequence in the vector generation submodule, so that they can be accessed quickly in the attention calculation of subsequent word segmentation.

[0064] Based on this, not only can the asynchronous generation of subproblems and subproblem vectors be achieved, but by detecting the start identifier of the sequence, the successively generated tokens can be asynchronously sent to the vector generation submodule one by one, thereby further improving the parallelism between the LLM and the vector generation submodule. In addition, the detection of the end identifier of the sequence also facilitates the system's recognition of the completion of a complete subproblem vector.

[0065] In one implementation, in step S301, the target problem is input into a large language model so that the large language model decomposes the target problem into multiple sub-problems. This includes: inputting the target problem and preset decomposition guide words into the large language model so that the large language model determines the first attention corresponding to the decomposition guide words from the pre-stored attention calculation results, and calculates the second attention corresponding to the target problem, so as to generate multiple sub-problems based on the first attention and the second attention.

[0066] The self-attention mechanism is calculated using three main vectors: query vector, key vector, and value vector. The attention score is determined by calculating the dot product of the query and key vectors, and this score is then normalized using the softmax function to weight the corresponding value.

[0067] When guiding an LLM to decompose a target problem into multiple subproblems, a prompt project can be used. This involves adding pre-defined decomposition prompts before the target problem (usually at the beginning, but can also be at the end or elsewhere) to illustrate and define the output structure of the subproblems, thus guiding the LLM to output structured subproblems. These decomposition prompts are the same across multiple requests; that is, the decomposition prompts are shared prefixes, the target problem is a unique prefix, and the sequence input into the LLM is a combination of the shared prefix and the target problem. The shared prefix remains unchanged across multiple requests, while the target problem is the part that changes each time. The length of the decomposition prompts far exceeds the length of the target problem itself; repeatedly calculating this part would waste computational resources.

[0068] In this embodiment, the attention calculation results (Key and Value) of the shared prefix (decomposition guide word) are pre-calculated and pre-stored, such as in the processor's shared memory, so that these calculation results can be directly accessed on subsequent requests. Therefore, whenever the LLM needs to perform attention calculation on the input sequence, it can pre-fill the first attention (Key and Value) corresponding to the decomposition guide word, thereby quickly obtaining the attention of the decomposition guide word. Only the second attention corresponding to the target problem needs to be calculated to quickly output the subproblem.

[0069] In one implementation, in step S303, multiple search results are input into a large language model to obtain the question-and-answer results corresponding to the target question. This includes: inputting multiple search results into the large language model respectively, so that the large language model processes multiple search results in parallel and generates multiple corresponding sub-answers; and obtaining the question-and-answer results generated by the large language model based on the multiple sub-answers.

[0070] For each search result, LLM generates a corresponding sub-answer. For example, it combines the search result with the sub-question to generate a sub-answer in natural language. Then, it aggregates all sub-answers and uses its contextual semantic understanding capabilities to ensure that all sub-answers are semantically and logically coherent, and finally generates a complete question-and-answer result.

[0071] In LLM, the processing of each search result is performed in parallel. For example, LLM starts multiple processes or threads, each processing one search result to generate a corresponding sub-answer, thus achieving the parallel generation of multiple sub-answers. Based on this, the advantages of multi-core CPUs or multi-core GPUs, as well as the parallel processing capabilities provided by the computing platform, can be fully utilized to improve the computational efficiency of answer generation.

[0072] In one implementation, multiple search results are input into a large language model to enable the large language model to process the multiple search results in parallel and generate multiple corresponding sub-answers. This includes: in response to detecting that a search service module generates a search result, the generated current search result is input into the large language model so that the large language model generates corresponding sub-answers based on the current search result. The generation of the next search result by the search service module and the processing of the current search result by the large language model are executed asynchronously.

[0073] Whenever the retrieval service module generates a retrieval result for a sub-question, it directly inputs the generated current retrieval result into the LLM for sub-answer reasoning, without waiting for all retrieval results. Based on this, by making sub-question retrieval and sub-answer generation asynchronous, the retrieval time of the retrieval service module can be hidden in the time of the LLM generating sub-answers. Furthermore, by decoupling the retrieval service module and the LLM, the parallelism of retrieval and sub-answer generation can be maximized, thereby significantly saving computation time and improving the efficiency of sub-answer generation.

[0074] In one implementation, the current search result includes multiple search segments. The generated current search result is input into a large language model so that the large language model generates a corresponding sub-answer based on the current search result. This includes: determining the total length of the multiple search segments; in response to the total length not exceeding the length threshold of the large language model, inputting the multiple search segments into the large language model so that the large language model generates a sub-answer corresponding to the current search result based on the multiple search segments; in response to the total length exceeding the length threshold of the large language model, sequentially inputting multiple search information corresponding to the current search result into the large language model so that the large language model generates a corresponding sub-answer based on the multiple search information, wherein each search information includes a search segment and a target question.

[0075] In other words, the number and length of search segments can be considered, and multiple search segments can be selectively fed into the LLM (Local Level Management) together to generate a corresponding sub-answer. Specifically, for multiple search segments in the search results, the system first calculates the total length of the multiple search segments. In one scenario, if the total length is within the LLM's context length limit (length threshold), the generated multiple search segments are directly fed into the LLM for calculation. As explained above regarding the kv-cache mechanism, this direct aggregation method can quickly generate the final question-and-answer result. In another scenario, if the total length of multiple search segments exceeds the LLM's context limit, each search segment is combined with the target question to form a search message. Then, the multiple search messages corresponding to a search result are sequentially input into the LLM to obtain the sub-answer corresponding to that search result, thus adapting to situations where the search results are long texts or contain excessive content. Furthermore, the generation of multiple sub-answers by the LLM can be executed in parallel. For example, multiple processes can be started in parallel, each process generating a corresponding sub-answer based on a search result (including multiple search segments) or multiple search messages corresponding to a search result.

[0076] In one implementation, the generated current search result is input into a large language model so that the large language model generates a corresponding sub-answer based on the current search result. This includes: inputting the current search result and a preset sub-answer guide word into the large language model so that the large language model determines the third attention corresponding to the sub-answer guide word from the pre-stored attention calculation results, calculates the fourth attention corresponding to the current search result, and generates a corresponding sub-answer based on the third attention and the fourth attention.

[0077] When guiding an LLM to generate a sub-answer based on search results, a prompt project can be used. This involves inputting the search results along with predefined sub-answer prompts into the LLM. The sub-answer prompts serve as examples of the sub-answer output and define its structure. Therefore, the content of the sub-answer prompts remains unchanged across multiple requests; that is, the sub-answer prompts share a prefix, while the search results have unique prefixes. The length of the sub-answer prompts far exceeds the total length of the search segments in the search results. Repeatedly calculating the attention of this portion would waste computational resources.

[0078] Therefore, in this embodiment, the attention calculation results of the shared prefix (sub-answer guide word) are pre-calculated and pre-stored, such as in the processor's shared memory, so that these calculation results can be directly accessed each time a sub-answer generation request is made. Based on this, each time the LLM needs to perform attention calculation on the input sequence, it can pre-fill the third attention corresponding to the sub-answer guide word to quickly obtain the attention of the sub-answer guide word. Only the fourth attention corresponding to the retrieval result needs to be calculated to quickly output the sub-answer, thereby avoiding the waste of computing resources caused by repeatedly calculating the attention of the sub-answer guide word.

[0079] The following is an example of a sub-response guide word:

[0080] "{__BASE_HEADER}# Dialogue Guide Words"

[0081] Here is the user's question:

[0082] {question}

[0083] Here is some context to help you answer the question:

[0084] {context}”.

[0085] Here, the {context} part is the search segment. For example, the search segment can be placed at the very end of the input sequence to make the length of the shared prefix as long as possible, thereby maximizing the speedup effect brought by the attention of the pre-padded shared prefix without affecting the LLM generation quality.

[0086] In one implementation, obtaining the question-and-answer results generated by the large language model based on multiple sub-answers may include: inputting multiple sub-answers and preset question-and-answer result guide words into the large language model, so that the large language model determines the fifth attention corresponding to the question-and-answer result guide words from the pre-stored attention calculation results, and calculates multiple sixth attentions corresponding to the multiple sub-answers, so as to generate question-and-answer results based on the fifth attention and multiple sixth attentions.

[0087] When guiding an LLM to generate the final question-and-answer result based on multiple sub-answers, a prompt project can be used. This involves using multiple sub-answers as unique suffixes and pre-defined question-and-answer result prompts as shared prefixes, inputting them together into the LLM. The question-and-answer result prompts are used to demonstrate the output of the question-and-answer result and define its output structure. Since the length of the question-and-answer result prompts far exceeds the total length of the multiple sub-answers, repeatedly calculating the attention for this part would waste computational resources. However, in this embodiment, the attention calculation results for the shared prefix can be pre-calculated and pre-stored, such as in the processor's shared memory. Each time the LLM needs to perform attention calculations on the input sequence, it can pre-fill the sixth attention corresponding to the question-and-answer result prompts, quickly obtaining the attention of the prompts. Only multiple sixth attentions corresponding to multiple sub-answers need to be calculated to quickly output the final question-and-answer result, thus avoiding the waste of computational resources caused by repeatedly calculating the attention of the question-and-answer result prompts.

[0088] Figure 6 and Figure 7 An application example diagram of an embodiment of this application is shown. Exemplarily, as... Figure 6 and Figure 7 As shown, the question-answering method based on a large language model in this application embodiment may include:

[0089] Step S701: The LLM receives the target question input by the user. For example... Figure 6 As shown, LLM can compute the target problem, including compute some irrelevant characters, which can be format or mood characters that are not very relevant to the target problem.

[0090] Step S702: Determine whether it is necessary to decompose the target problem into subproblems. If decomposition is required, proceed to step S703; otherwise, proceed to step S707. For example, if the target problem may contain multiple subproblems, it is determined that decomposition is necessary, and the process proceeds to step S703. If the target problem represents only one problem, it is determined that decomposition is not necessary, and the process proceeds to step S707.

[0091] Step S703: Extract sub-problems, that is, the LLM extracts multiple sub-problems from the target problem. This step can adopt the same or similar implementation as step S301. Optionally, attention can be pre-filled during the sub-problem extraction process, that is, the target problem and preset decomposition guide words are input into the large language model, so that the large language model determines the first attention corresponding to the decomposition guide words from the pre-stored attention calculation results, and calculates the second attention corresponding to the target problem, so as to generate multiple sub-problems based on the first attention and the second attention. Figure 6 As shown, when generating multiple subproblems, LLM can format and encapsulate each subproblem, that is, each subproblem generates a sequence of subproblems, including a sequence start identifier (such as "..."). <s> Sub-problems, sequence end markers (such as "")< / s> ”).

[0092] Step S704: Retrieve from the database. This step can be implemented in the same or similar manner as step S302. Optionally, as... Figure 6 As shown, during the retrieval process, the LLM, vector generation submodule, and vector retrieval submodule can be called asynchronously. Specifically, in response to the detection that the large language model generates a sub-question, the generated current sub-question is input into the vector generation submodule for vector computation. The generation of the next sub-question by the large language model and the vector computation of the current sub-question by the vector generation submodule are executed asynchronously. Optionally, when a sequence start marker is detected in the content generated by the LLM, it can be considered that the content generated by the LLM contains structured data, and the vector computation of the subsequently generated sub-questions can be performed synchronously in the vector generation submodule. Further, in response to the detection that the vector generation submodule generates a sub-question vector, the generated current sub-question vector is input into the vector retrieval submodule for retrieval to obtain the corresponding retrieval result. The generation of the next sub-question vector by the vector generation submodule and the retrieval of the current sub-question vector by the vector retrieval submodule are executed asynchronously. Before generating sub-questions, the vector generation submodule can rewrite the target question and calculate the vector of the rewritten target question, which is then input into the LLM for inference.

[0093] Optionally, such as Figure 6As shown, during the retrieval phase, the vector retrieval submodule can employ a hierarchical index structure. Specifically, at the start of the retrieval, the vector retrieval submodule first determines the database's primary index. The primary index is a coarse-grained partition, dividing the vector data in the entire database into multiple large blocks. Then, a query rewriting method is used to extract the summary information of the target question. This summary information is then embedded to obtain a query vector, which is used to determine the database's primary index list. The vector retrieval submodule selects several large blocks most relevant to the query vector from the primary index. For example, it selects the most relevant large blocks by comparing the similarity between the query vector and the centers of various blocks. The selected blocks determine the initial scope of the database retrieval, thereby narrowing the subsequent search scope. After determining the primary index, the system can prefetch data from these blocks into the cache for fast access. Subsequent queries no longer need to traverse the entire database, significantly improving retrieval speed. Then, within the blocks selected by the primary index, secondary or multi-level indexes are used to further narrow the search scope until the most relevant vector data is finally located as the retrieval result.

[0094] Step S705: Concatenate sub-problems, that is, concatenate each search result with its corresponding sub-problem and input it into the LLM.

[0095] Step S706: LLM generates sub-answers for each search result, resulting in multiple sub-answers. For example... Figure 6 As shown, on the one hand, the LLM generates each sub-answer in parallel; on the other hand, in response to detecting that the retrieval service module generates a retrieval result, the generated current retrieval result is input into the large language model, so that the large language model generates the corresponding sub-answer based on the current retrieval result. Here, the retrieval service module's generation of the next retrieval result and the large language model's processing of the current retrieval result are executed asynchronously. Optionally, when generating sub-answers, attention can be pre-filled, that is, the current retrieval result and preset sub-answer guide words are input into the large language model, so that the large language model determines the third attention corresponding to the sub-answer guide word from the pre-stored attention calculation results, calculates the fourth attention corresponding to the current retrieval result, and generates the corresponding sub-answer based on the third and fourth attention.

[0096] Step S707: Calculate the question-answering results, such as obtaining the question-answering results generated by the large language model based on multiple sub-answers. Optionally, attention can be pre-filled, that is, multiple sub-answers and preset question-answering result guide words are input into the large language model, so that the large language model determines the fifth attention corresponding to the question-answering result guide words from the pre-stored attention calculation results, and calculates multiple sixth attentions corresponding to multiple sub-answers, so as to generate question-answering results based on the fifth attention and multiple sixth attentions.

[0097] like Figure 6 and Figure 7 As shown, by asynchronously calling the LLM, vector generation submodule, and vector retrieval submodule, the LLM, vector generation submodule, and vector retrieval submodule can be decoupled without changing the LLM network structure. On the one hand, this maximizes the parallelism of subproblem decomposition, subproblem vector generation, subproblem vector retrieval, and sub-answer generation. On the other hand, it hides the subproblem generation time within the subproblem vector generation time, the subproblem vector generation time within the subproblem vector retrieval time, and the subproblem vector retrieval time within the sub-answer generation time, thereby saving computation time.

[0098] Optionally, during the computation and generation process, shared prefixes (decomposition guide words, sub-answer guide words, or question-answer result guide words) can be pre-stored. This allows the attention computation results of the shared prefixes to be pre-filled each time attention is calculated on the input sequence, avoiding the waste of resources caused by repeatedly calculating the attention of long guide word parts.

[0099] Optionally, LLM executes the generation process of multiple sub-answers in parallel, thereby making full use of the advantages of multi-core CPUs or multi-core GPUs, as well as the parallel processing capabilities provided by the computing platform, to improve the computational efficiency of generating answers.

[0100] Alternatively, the structure of the input sequence of LLM can be optimized by placing the guide word template part at the beginning and distinguishing between shared prefixes and unique suffixes, thereby maximizing the speedup effect of pre-filled shared prefixes.

[0101] Corresponding to the application scenarios and methods provided in the embodiments of this application, the embodiments of this application also provide a question-answering device based on a large language model. This device includes: a target question input module, used to input a target question into a large language model, so that the large language model decomposes the target question into multiple sub-questions; an asynchronous invocation module, used to asynchronously invoke a retrieval service module to retrieve the generated current sub-question in response to detecting that the large language model generates a sub-question, and obtain corresponding retrieval results; and a question-answering result determination module, used to input multiple retrieval results into the large language model to obtain the question-answering result corresponding to the target question.

[0102] In one implementation, the retrieval service module includes a vector generation submodule and a vector retrieval submodule; the asynchronous invocation module includes: a first invocation submodule, used to, in response to detecting that the large language model generates a sub-question, input the generated current sub-question into the vector generation submodule for vector calculation, wherein the generation of the next sub-question by the large language model and the vector calculation of the current sub-question by the vector generation submodule are executed asynchronously; a second invocation submodule, used to, in response to detecting that the vector generation submodule generates a sub-question vector, input the generated current sub-question vector into the vector retrieval submodule for retrieval to obtain the corresponding retrieval result, wherein the generation of the next sub-question vector by the vector generation submodule and the retrieval of the current sub-question vector by the vector retrieval submodule are executed asynchronously.

[0103] In one implementation, the first calling submodule is configured to: in response to detecting the start identifier of the sequence generated by the large language model, input the sub-problem sequence corresponding to the start identifier of the sequence into the vector generation submodule for vector calculation to obtain the sub-problem vector corresponding to the current sub-problem, wherein the sub-problem sequence includes the start identifier of the sequence, the current sub-problem and the end identifier of the sequence, and the vector generation submodule outputs the generated sub-problem vector when the end identifier of the sequence is detected.

[0104] In one implementation, the target question input module is used to: input the target question and a preset decomposition guide word into the large language model, so that the large language model determines a first attention corresponding to the decomposition guide word from the pre-stored attention calculation results, and calculates a second attention corresponding to the target question, so as to generate the plurality of sub-questions based on the first attention and the second attention.

[0105] In one implementation, the question-and-answer result determination module includes: a sub-answer generation sub-module, used to input multiple search results into the large language model respectively, so that the large language model processes multiple search results in parallel and generates multiple corresponding sub-answers; and a question-and-answer result generation module, used to obtain the question-and-answer results generated by the large language model based on the multiple sub-answers.

[0106] In one implementation, the sub-answer generation submodule is configured to: in response to detecting that the retrieval service module generates a retrieval result, input the generated current retrieval result into the large language model, so that the large language model generates a corresponding sub-answer based on the current retrieval result, wherein the generation of the next retrieval result by the retrieval service module and the processing of the current retrieval result by the large language model are executed asynchronously.

[0107] In one implementation, the current search result includes multiple search segments. The generated current search result is input into the large language model so that the large language model generates a corresponding sub-answer based on the current search result. This includes: determining the total length of the multiple search segments; and, in response to the total length not exceeding the length threshold of the large language model, inputting the multiple search segments into the large language model so that the large language model generates a sub-answer corresponding to the current search result based on the multiple search segments.

[0108] In one implementation, the current search result includes multiple search segments. The generated current search result is input into the large language model so that the large language model generates a corresponding sub-answer based on the current search result. This includes: determining the total length of the multiple search segments; and in response to the total length exceeding the length threshold of the large language model, sequentially inputting multiple search information corresponding to the current search result into the large language model so that the large language model generates a corresponding sub-answer based on the multiple search information, wherein each search information includes a search segment and the target question.

[0109] In one implementation, the sub-answer generation submodule is used to: input the current search result and the preset sub-answer guide words into the large language model, so that the large language model determines the third attention corresponding to the sub-answer guide words from the pre-stored attention calculation results, calculates the fourth attention corresponding to the current search result, and generates the corresponding sub-answer based on the third attention and the fourth attention.

[0110] In one implementation, the question-and-answer result generation submodule is used to: input the plurality of sub-answers and preset question-and-answer result guide words into the large language model, so that the large language model determines the fifth attention corresponding to the question-and-answer result guide words from the pre-stored attention calculation results, and calculates the plurality of sixth attentions corresponding to the plurality of sub-answers, so as to generate the question-and-answer result based on the fifth attention and the plurality of sixth attentions.

[0111] The functions of each module in each device in the embodiments of this application can be found in the corresponding description in the above method, and they have corresponding beneficial effects, which will not be repeated here.

[0112] Figure 8 This is a block diagram of an electronic device used to implement embodiments of this application. For example... Figure 8As shown, the electronic device includes a memory 801 and a processor 802. The memory 801 stores a computer program that can run on the processor 802. When the processor 802 executes the computer program, it implements the method described in the above embodiments. The number of memories 801 and processors 802 can be one or more. In a specific implementation, the electronic device may also include a communication interface 803 for communicating with external devices and performing data exchange and transmission.

[0113] In practical implementation, if the memory 801, processor 802, and communication interface 803 are implemented independently, they can be interconnected via a bus to communicate with each other. This bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 8 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0114] Optionally, in a specific implementation, if the memory 801, the processor 802, and the communication interface 803 are integrated on a single chip, then the memory 801, the processor 802, and the communication interface 803 can communicate with each other through an internal interface.

[0115] This application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method provided in this application.

[0116] This application provides a computer program product, including a computer program that, when executed by a processor, implements the method provided in this application.

[0117] This application also provides a chip including a processor for calling and executing instructions stored in a memory, causing a communication device with the chip installed to perform the method provided in this application.

[0118] This application also provides a chip, including: an input interface, an output interface, a processor, and a memory. The input interface, output interface, processor, and memory are connected through an internal connection path. The processor is used to execute code in the memory. When the code is executed, the processor is used to execute the method provided in the application embodiment.

[0119] It should be understood that the aforementioned processor can be a Central Processing Unit (CPU), or other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. General-purpose processors can be microprocessors or any conventional processor. It is worth noting that the processor can be a processor supporting Advanced Reduced Instruction Set Machines (ARM) architecture.

[0120] Further, optionally, the aforementioned memory may include read-only memory and random access memory. The memory may be volatile memory or non-volatile memory, or may include both. Non-volatile memory may include read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory may include random access memory (RAM), which serves as an external cache. By way of example, but not limitation, many forms of RAM are available. Examples include Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), and Direct Rambus RAM (DR RAM).

[0121] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions according to this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another.

[0122] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples.

[0123] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified.

[0124] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process. Furthermore, the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functionality involved.

[0125] The logic and / or steps described in the flowchart or otherwise herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus or device (such as a computer-based system, a processor-included system or other system that can fetch and execute instructions from, an instruction execution system, apparatus or device).

[0126] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. All or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware, the program being stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiments.

[0127] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. This storage medium can be a read-only memory, a disk, or an optical disk, etc.

[0128] The above description is merely an exemplary embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various variations or substitutions within the technical scope described in this application, and these should all be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A question-answering method based on a large language model, comprising: The target problem is input into a large language model, which then decomposes the target problem into multiple sub-problems. In response to the detection that the large language model has generated a sub-question, the retrieval service module is asynchronously invoked to retrieve the generated current sub-question and obtain the corresponding retrieval results; The multiple search results are input into the large language model to obtain the question-and-answer results corresponding to the target question.

2. The question-and-answer method according to claim 1, wherein, The retrieval service module includes a vector generation submodule and a vector retrieval submodule; in response to detecting that the large language model outputs a sub-question, the retrieval service module is asynchronously invoked to retrieve the generated current sub-question, and the corresponding retrieval results are obtained, including: In response to detecting that the large language model generates a sub-problem, the generated current sub-problem is input into the vector generation submodule for vector computation, wherein the generation of the next sub-problem by the large language model and the vector computation of the current sub-problem by the vector generation submodule are executed asynchronously; In response to the detection that the vector generation submodule generates a sub-question vector, the generated current sub-question vector is input into the vector retrieval submodule for retrieval to obtain the corresponding retrieval result. The generation of the next sub-question vector by the vector generation submodule and the retrieval of the current sub-question vector by the vector retrieval submodule are executed asynchronously.

3. The question-and-answer method according to claim 2, wherein, In response to detecting that the large language model generates a sub-question, the generated current sub-question is input into the vector generation submodule for vector computation, including: In response to the detection of the start identifier of the sequence generated by the large language model, the sub-problem sequence corresponding to the start identifier is input into the vector generation submodule for vector calculation to obtain the sub-problem vector corresponding to the current sub-problem. The sub-problem sequence includes the start identifier, the current sub-problem, and the end identifier. When the end identifier is detected, the vector generation submodule outputs the generated sub-problem vector.

4. The method according to any one of claims 1 to 3, wherein, The target problem is input into a large language model, which then decomposes the target problem into multiple sub-problems, including: The target question and the preset decomposition guide words are input into the large language model, so that the large language model determines the first attention corresponding to the decomposition guide words from the pre-stored attention calculation results, and calculates the second attention corresponding to the target question, so as to generate the multiple sub-questions based on the first attention and the second attention.

5. The question-and-answer method according to any one of claims 1 to 3, wherein, Inputting multiple search results into the large language model yields the question-and-answer results corresponding to the target question, including: The multiple search results are respectively input into the large language model so that the large language model processes the multiple search results in parallel and generates multiple corresponding sub-answers; Obtain the question-and-answer results generated by the large language model based on the multiple sub-answers.

6. The question-and-answer method according to claim 5, wherein, Multiple search results are input into the large language model to enable the large language model to process the multiple search results in parallel and generate multiple corresponding sub-answers, including: In response to detecting that the retrieval service module has generated a retrieval result, the generated current retrieval result is input into the large language model so that the large language model generates a corresponding sub-answer based on the current retrieval result. The generation of the next retrieval result by the retrieval service module and the processing of the current retrieval result by the large language model are executed asynchronously.

7. The question-and-answer method according to claim 6, wherein, The current search results include multiple search segments. The generated current search results are input into the large language model so that the large language model generates corresponding sub-answers based on the current search results, including: Determine the total length of the plurality of search segments; In response to the total length not exceeding the length threshold of the large language model, the multiple search segments are input into the large language model so that the large language model generates a sub-answer corresponding to the current search result based on the multiple search segments.

8. The question-and-answer method according to claim 6, wherein, The current search results include multiple search segments. The generated current search results are input into the large language model so that the large language model generates corresponding sub-answers based on the current search results, including: Determine the total length of the plurality of search segments; In response to the total length exceeding the length threshold of the large language model, multiple search information corresponding to the current search result are sequentially input into the large language model so that the large language model generates corresponding sub-answers based on the multiple search information, wherein each search information includes a search passage and the target question.

9. The question-and-answer method according to claim 6, wherein, The generated current search results are input into the large language model, so that the large language model generates corresponding sub-answers based on the current search results, including: The current search result and the preset sub-answer guide words are input into the large language model, so that the large language model determines the third attention corresponding to the sub-answer guide words from the pre-stored attention calculation results, calculates the fourth attention corresponding to the current search result, and generates the corresponding sub-answer based on the third attention and the fourth attention.

10. The question-and-answer method according to claim 5, wherein, Obtaining the question-answering results generated by the large language model based on the multiple sub-answers includes: The multiple sub-answers and the preset question-and-answer result guide words are input into the large language model, so that the large language model determines the fifth attention corresponding to the question-and-answer result guide word from the pre-stored attention calculation results, and calculates multiple sixth attentions corresponding to the multiple sub-answers, so as to generate the question-and-answer result based on the fifth attention and the multiple sixth attentions.

11. An electronic device comprising a memory, a processor, and a computer program stored in the memory, wherein the processor, when executing the computer program, implements the method of any one of claims 1 to 10.

12. A computer-readable storage medium storing a computer program that, when executed by a processor, implements the method of any one of claims 1 to 10.

13. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1 to 10.

Images