Question-answering method based on large language model, and device, medium and product

Abstract

The embodiments of the present disclosure relate to the technical field of natural language processing. Provided are a question-answering method based on a large language model, and a device, a medium and a product. The method comprises: inputting a target question into a large language model, such that the large language model decomposes the target question into a plurality of sub-questions; in response to having detected that the large language model has generated a sub-question, asynchronously calling a retrieval service module to perform retrieval on the current sub-question that has been generated, so as to obtain a corresponding retrieval result; and inputting a plurality of retrieval results into the large language model to obtain a question-answering result corresponding to the target question. The technical solution in the embodiments of the present disclosure can improve the degree of parallelism during calculation generation and can save the calculation time without changing the network structure of large language models.

Description

Question answering method, device, medium and product based on large language model Cross-reference to Related Applications

[0001] The present disclosure claims priority to Chinese Patent Application No. 202411259912.0, filed on September 9, 2024, the entire contents of which are incorporated herein by reference. TECHNICAL FIELD

[0002] The present disclosure relates to the technical field of natural language processing, in particular to a question answering method based on a large language model, an electronic device, a question answering system based on a large language model, a computer readable storage medium and a computer program product, which can be applied to the field of artificial intelligence. BACKGROUND

[0003] With the rapid development of natural language processing technology, large language models (LLM) have become a core tool for obtaining, generating and improving text content. Large language models can understand and generate natural language and are widely used in chatbots, automatic abstracts, translation systems and other language understanding tasks. Although large language models have made great progress in content generation, the content generated by them still faces limitations in accuracy and knowledge timeliness. SUMMARY

[0004] The embodiments of the present disclosure provide a question answering method based on a large language model, an electronic device, a question answering system based on a large language model, a computer readable storage medium and a computer program product to alleviate or solve one or more technical problems in the prior art.

[0005] In a first aspect, the embodiments of the present disclosure provide a question answering method based on a large language model, comprising: inputting a target question into a large language model to make the large language model decompose the target question into a plurality of sub-questions; in response to detecting that the large language model generates one sub-question, asynchronously calling a retrieval service module to retrieve the generated current sub-question to obtain a corresponding retrieval result; inputting a plurality of the retrieval results into the large language model to obtain a corresponding question answering result of the target question.

[0006] In a second aspect, the embodiments of the present disclosure provide an electronic device, comprising a memory, a processor and a computer program stored in the memory, wherein the processor implements the method of any one of the embodiments of the present disclosure when executing the computer program.

[0007] In a third aspect, the embodiments of the present disclosure provide a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the method of any one of the embodiments of the present disclosure.

[0008] In a fourth aspect, the embodiments of the present disclosure provide a computer program product comprising a computer program which, when executed by a processor, implements the method of any of the embodiments of the present disclosure.

[0009] According to the technical solution of the embodiments of the present disclosure, when generating an answer, the large language model no longer only relies on the knowledge learned during pre-training, but can also dynamically call an external retrieval service module to obtain the latest and most relevant information, thereby improving the accuracy and timeliness of the generated content. Furthermore, by asynchronously calling the retrieval service module, that is, whenever the large language model generates a sub-question, the generated current sub-question is directly input to the retrieval service module for retrieval. Therefore, the generation and calculation time of the next sub-question by the large language model is hidden in the retrieval time of the current sub-question by the retrieval service module, thereby significantly saving the calculation time and improving the generation efficiency of the question and answer result. Moreover, by decoupling the large language model and the retrieval service module, the parallelism in the calculation and generation process can be improved without changing the network structure of the large language model, thereby saving the calculation time.

[0010] Optionally, the retrieval service module comprises a vector generation submodule and a vector retrieval submodule. The large language model asynchronously inputs each generated sub-question to the vector generation submodule, thereby hiding the generation time of the next sub-question by the large language model in the vector calculation time of the current sub-question by the vector generation submodule. The vector generation submodule asynchronously inputs each generated sub-question vector to the vector retrieval submodule, thereby hiding the generation time of the next sub-question vector by the vector generation submodule in the retrieval time of the current sub-question vector by the vector retrieval submodule. Based on this, the sub-question generation, sub-question vector calculation, and sub-question vector retrieval can be asynchronous, and the large language model, the vector generation submodule, and the vector retrieval submodule can be decoupled, so that the sub-question decomposition, vector generation, and vector retrieval are maximally parallelized, thereby saving the idle time of the hardware and improving the generation efficiency of the question and answer result.

[0011] Optionally, during the calculation and generation process, shared prefixes can be pre-stored, such as pre-filling the attention of the decomposition guide word in the query decomposition stage, pre-filling the attention of the sub-answer guide word when generating each sub-answer, and pre-filling the attention of the question and answer result guide word when generating the final question and answer result. Therefore, when calculating the attention of the input sequence each time, the attention calculation result of the shared prefix can be pre-filled, thereby avoiding the waste of resources caused by repeatedly calculating the attention of the long guide word part.

[0012] Optionally, the advantages of multi-core central processing units (CPUs) or multi-core graphics processing units (GPUs) and the parallel processing capabilities provided by the computing platform can be fully utilized, so that the large language model generates multiple sub-answers in parallel as a parallel processing process, thereby improving the calculation efficiency of the generated answer.

[0013] The above description is only a summary of the technical solutions of the present disclosure. In order to enable a more clear understanding of the technical means of the present disclosure, the description can be implemented according to the content of the specification, and in order to enable the above and other purposes, characteristics and advantages of the present disclosure to be more obvious and easy to understand, the specific embodiments of the present disclosure are described below. BRIEF DESCRIPTION OF DRAWINGS

[0014] In the drawings, like reference numerals refer to like elements throughout the various drawings. The drawings are not necessarily to scale, the emphasis instead being placed upon illustrating the principles of the present disclosure. It should be understood that the drawings only depict some embodiments of the present disclosure and should not be considered limiting of its scope.

[0015] FIG. 1 shows an exemplary architecture diagram of a large language model-based question answering system in embodiments of the present disclosure;

[0016] FIG. 2 shows a timing diagram of an application example of the question answering system;

[0017] FIG. 3 shows a flowchart of a large language model-based question answering method in embodiments of the present disclosure;

[0018] FIG. 4 shows one schematic diagram of a sub-question vector generation process in embodiments of the present disclosure;

[0019] FIG. 5 shows another schematic diagram of a sub-question vector generation process in embodiments of the present disclosure;

[0020] FIG. 6 shows a timing diagram of a large language model-based question answering method in embodiments of the present disclosure;

[0021] FIG. 7 shows an application example diagram of a large language model-based question answering method in embodiments of the present disclosure;

[0022] FIG. 8 shows a block diagram of an electronic device provided in embodiments of the present disclosure. DETAILED DESCRIPTION

[0023] In the following, only some exemplary embodiments are described briefly. As those skilled in the art can recognize, the described embodiments can be modified in various different ways without departing from the concept or scope of the present disclosure. Therefore, the drawings and the description are considered to be exemplary in nature, rather than limiting.

[0024] In order to facilitate understanding of the technical solutions of the embodiments of the present disclosure, the related technologies of the embodiments of the present disclosure are described below. The following related technologies can be combined with the technical solutions of the embodiments of the present disclosure in any way as optional solutions, which all belong to the protection scope of the embodiments of the present disclosure.

[0025] In the following, the following terms will be used.

[0026] Large Language Model (LLM): A large-scale machine learning model used in the field of Natural Language Processing (NLP) to understand or generate human language. LLM is trained on very large text datasets through deep learning techniques, enabling it to capture the complexity and diversity of natural language.

[0027] Retrieval Augmented Generation (RAG) framework: A natural language processing model framework that integrates retrieval and generation mechanisms. By introducing an information retrieval module, RAG enhances the knowledge base of LLM. LLM no longer relies solely on the knowledge learned during pre-training, but can also dynamically retrieve the latest and most relevant information from external databases.

[0028] Embedding Model: A model that maps high-dimensional sparse data (such as words, sentences, pictures, etc.) to a low-dimensional dense vector space. This low-dimensional dense vector is called an "embedding vector". The basic idea of embedding model is to learn a function to convert input data into fixed-dimensional dense vectors. This function is usually trained through neural networks, with the goal of making similar input data have similar vector representations in the embedding space, and dissimilar data have relatively distant vector representations.

[0029] In the field of natural language processing, large language models can understand and generate natural language, and are widely used in chat robots, automatic summaries, question and answer systems, and other language understanding tasks. Although large language models have made great progress in content generation, the accuracy of the generated content is still limited.

[0030] In one implementation, the query and user input document can be embedded in the LLM, and the retrieved document can be spliced into the context, so that the LLM combines external knowledge information in the process of generating answers. However, the external knowledge information obtained by this implementation method is very limited.

[0031] In another implementation, the RAG framework can be introduced on the basis of the LLM. The RAG framework combines a retrieval mechanism to retrieve relevant information from an external knowledge base before generating an answer, so that the LLM is more accurate and rich when generating an answer, thereby improving the quality and relevance of the content generated by the LLM and enhancing the question and answer effect of the LLM. However, the RAG framework requires a very high computational resource and time cost when processing complex queries, and both vector generation and retrieval and large model inference require a large amount of computational resource and time. With the continuous growth of information, the existing RAG framework implementation for LLM content generation is difficult to meet the real-time and efficient content retrieval requirements, and also limits the performance of the RAG framework in the practical application of the LLM.

[0032] Therefore, the embodiments of the present disclosure provide a large language model-based question and answer method and a question and answer system, which can improve the parallelism of the RAG framework and save computing time without changing the model network structure. In order to facilitate the understanding of the embodiments of the present disclosure, first, the system architecture to which the embodiments of the present disclosure are applied is simply described. FIG. 1 shows an exemplary architecture diagram of a large language model-based question and answer system in the embodiments of the present disclosure, and FIG. 2 shows a timing diagram of an application example of the question and answer system. As shown in FIG. 1 and FIG. 2, the large language model-based question and answer system in the embodiments of the present disclosure includes a client 110 and a server 120, wherein the server 120 is deployed with a large language model and a retrieval service module.

[0033] The large language model and the retrieval service module can be deployed on the same subject or multiple subjects, which can be an application program, a service, an instance, a functional module in the form of software, a virtual machine, a container, or a cloud server, etc., or a hardware device or a hardware chip with data processing function, etc. The server 120 can be deployed on a local computing device or a cloud computing platform or cluster providing computing power, storage and network resources, and the specific application architecture can be built according to the service requirements. The client 110 can be hardware such as a mobile phone, a personal computer, a tablet computer, a wearable device, etc. The client 110 can also be an application program (APP) or a software module deployed on an electronic device.

[0034] In one application example, as shown in FIG. 2, a user can input a target question in natural language form through an interactive interface provided by the client 110, for example, the target question "Who is the author of this article, and what dataset is used?". The client 110 sends the target question to the server 120.

[0035] The server 120 calls the large language model, inputs the target question into the large language model, and decomposes the target question into multiple sub-questions, such as sub-question 1 "Who is the author of this article?" and sub-question 2 "What dataset is used in this article?" The multiple sub-questions can be output in the form of text. The server 120 calls the retrieval service module to retrieve each sub-question from the database to obtain the corresponding retrieval result, for example, sub-question 1 corresponds to retrieval result 1, and sub-question 2 corresponds to retrieval result 2. The multiple retrieval results can be output in the form of text.

[0036] The server 120 calls the large language model, and the large language model infers the multiple retrieval results and the target question to output the question and answer result corresponding to the target question. The server 120 returns the question and answer result to the client 110, so that the user can obtain the question and answer result corresponding to the target question.

[0037] In the embodiment of the present disclosure, the server 120 calls the large language model and the retrieval service module asynchronously, that is, the server 120 asynchronously calls the retrieval service module to input the generated sub-question into the retrieval service module to retrieve from the database to obtain the corresponding retrieval result as soon as the large language model generates a sub-question. That is, the generation of the next sub-question by the large language model and the retrieval of the current sub-question by the retrieval service module are executed asynchronously.

[0038] In the related art, the large language model needs to generate all sub-questions, and retrieve from the database using text containing all sub-questions to obtain retrieval results. This implementation can meet the user's demand for computing performance in the short text scenario, but if the user inputs a long text, a lot of computing time will be wasted, and it is difficult to meet the real-time and efficient retrieval demand.

[0039] In the technical solution of the embodiment of the present disclosure, the retrieval service module is called asynchronously, that is, the current generated sub-question is directly input into the retrieval service module for retrieval as soon as the large language model generates a sub-question. Therefore, the computing time of the large language model for generating the next sub-question is hidden in the retrieval time of the retrieval service module for retrieving the current sub-question, so that the computing time can be obviously saved, and the generation efficiency of the entire system can be improved.

[0040] The question and answer system of the embodiment of the present disclosure can be applied to various application scenarios such as human-computer dialogue, machine translation, document analysis, data organization, writing assistance, knowledge learning, and information retrieval, so as to meet the real-time and efficient generation and retrieval demand of the user in different scenarios.

[0041] It should be noted that the above application scenarios or application examples provided in the embodiments of the present disclosure are for the convenience of understanding, and the application of the technical solutions of the embodiments of the present disclosure is not specifically limited. In addition, the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present disclosure are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant national and regional laws, regulations and standards, and provide corresponding operation portal for user to choose authorization or refusal.

[0042] The technical solutions of the present disclosure and how the technical solutions of the present disclosure solve the foregoing technical problems will be described in detail below with specific embodiments. Several specific embodiments listed can be combined with each other, and for the same or similar concepts or processes, some embodiments can not be described again. The embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.

[0043] FIG. 3 shows a flowchart of a large language model-based question answering method according to an embodiment of the present disclosure, which can be applied to and executed by the server 120. As shown in FIG. 3, the question answering method can include steps S301, S302 and S303.

[0044] Step S301: inputting a target question into a large language model to make the large language model decompose the target question into a plurality of sub-questions.

[0045] The target question is in natural language form, such as a question input by a user. The core structure of the large language model LLM is based on the Transformer architecture (a neural network architecture based on self-attention mechanism), which can understand and generate natural language. In the embodiments of the present application, the LLM can understand the target question and decompose it into a plurality of sub-questions. For example, the LLM processes the original target question input by the user, and through prompt word (prompt, also called guide word) engineering, guides the LLM to generate more specific or more general sub-questions from the target question.

[0046] Exemplarily, as shown in FIG. 4, the target question is a long text, that is, a complex query containing multiple sub-questions, such as "Who is the author of this article? What data set is used?", and this input sequence is input into the LLM. After the LLM receives the input sequence, the input sequence is preprocessed by word embedding to convert the text into word vectors of multiple words (such as "make", "make", "is", "who"...) that the model can understand, and positional encoding is added to the word vectors according to the order of the words in the input sequence; based on the self-attention mechanism, the LLM can pay attention to the information of other words in the input sequence when processing the current word, thereby capturing the dependency relationship of the context, identifying different parts and relationships in the target question, and generating the corresponding text of each sub-question step by step.

[0047] It should be noted that the network structure of the LLM is not specifically limited in the embodiments of the present disclosure, as long as it can decompose one question into multiple sub-questions.

[0048] Step S302: In response to detecting that the large language model generates a sub-question, the retrieval service module is asynchronously called to retrieve the generated current sub-question to obtain the corresponding retrieval result.

[0049] In the embodiments of the present disclosure, the calling of the LLM and the retrieval service module is asynchronous, that is, the LLM generates a sub-question, and the retrieval service module is asynchronously input to retrieve the corresponding retrieval result from the database. Therefore, the generation process of the next sub-question by the LLM and the retrieval process of the current sub-question by the retrieval service module are asynchronous.

[0050] Exemplarily, as shown in FIG. 4, when the LLM generates the sub-question "Who is the author?", the sub-question will be input to the retrieval service module for retrieval, without waiting for the LLM to complete the generation of the next sub-question "What data set is used for this article?".

[0051] Among them, the retrieval service module retrieves the sub-question by accessing the database, and outputs one or more retrieval text segments corresponding to the sub-question as the retrieval result. Exemplarily, each sub-question corresponds to a sub-query, and the database is usually an external database of the LLM and contains a large amount of knowledge information, which can be composed of multiple documents or data items. The retrieval service module queries the most relevant information for each sub-question from the database, for example, extracts the top k results from the retrieved information according to the similarity of the vector representation of the sub-question, as the retrieval result of the sub-question.

[0052] The retrieval service module can use the RAG framework, can be a software module, integrated in the application, as part of the application, or deployed on a server as an independent service, and provide functions through a network interface.

[0053] Step S303: inputting the plurality of retrieval results into the large language model to obtain a question and answer result corresponding to the target question.

[0054] Among them, the plurality of retrieval results can be input into the LLM one by one or together after all the plurality of retrieval results are generated, or each retrieval result is input into the LLM asynchronously, and the embodiments of the present disclosure do not limit this. The LLM reasons the plurality of retrieval results and the target question, and outputs a question and answer result corresponding to the target question.

[0055] In one example, the plurality of retrieval results can be summarized as an information-rich context to input the LLM. Using these context information, the LLM constructs and generates a comprehensive and accurate question and answer result. In another example, for each retrieval result, the LLM generates a corresponding sub-answer, for example, combines the retrieval result with a sub-question to generate a sub-answer in natural language form, then summarizes all sub-answers, and uses its context semantic understanding ability to ensure that all sub-answers are semantically and logically coherent, and finally generates a complete question and answer result.

[0056] Compared with retrieving from the database using text containing all sub-questions, the technical solution of the embodiments of the present disclosure hides the generation and calculation time of the next sub-question of the large language model in the retrieval time of the current sub-question of the retrieval service module by asynchronously calling the retrieval service module, that is, each time the large language model generates a sub-question, the generated current sub-question is directly input into the retrieval service module for retrieval, so that the generation and calculation time of the next sub-question of the large language model can be hidden in the retrieval time of the current sub-question of the retrieval service module, thereby significantly saving the calculation time and improving the generation efficiency of the question and answer result.

[0057] In one embodiment, the retrieval service module includes a vector generation submodule and a vector retrieval submodule; in step S302, in response to detecting that the large language model outputs a sub-question, the retrieval service module is asynchronously called to retrieve the generated current sub-question to obtain the corresponding retrieval result, including: in response to detecting that the large language model generates a sub-question, the generated current sub-question is input into the vector generation submodule for vector calculation, wherein the generation of the next sub-question by the large language model and the vector calculation of the current sub-question by the vector generation submodule are executed asynchronously; in response to detecting that the vector generation submodule generates a sub-question vector, the generated current sub-question vector is input into the vector retrieval submodule for retrieval to obtain the corresponding retrieval result, wherein the generation of the next sub-question vector by the vector generation submodule and the retrieval of the current sub-question vector by the vector retrieval submodule are executed asynchronously.

[0058] The vector generation submodule and the vector retrieval submodule can be software modules integrated in an application as part of an application program, or deployed on a server as an independent service to provide functions through a network interface. Specifically, the vector generation submodule can be deployed as an independent service, for example, deployed on the same server as the LLM, or deployed separately to improve performance, or the vector generation submodule can be embedded in the LLM in the form of an embedded model. The vector retrieval submodule can be deployed on a server dedicated to efficient retrieval, closely integrated with a database to facilitate retrieval of a large amount of data. Illustratively, the LLM can be a core large model, and the vector generation submodule and the vector retrieval submodule can be distributed as auxiliary modules. Communication between the LLM, the vector generation submodule, and the vector retrieval submodule can be achieved through a network application programming interface (API) or remote procedure call (RPC).

[0059] That is, the LLM asynchronously inputs each generated sub-question to the vector generation submodule, thereby hiding the generation time of the next sub-question by the LLM in the vector calculation time of the current sub-question by the vector generation submodule. The vector generation submodule asynchronously inputs each generated sub-question vector to the vector retrieval submodule, thereby hiding the generation time of the next sub-question vector by the vector generation submodule in the retrieval time of the current sub-question vector by the vector retrieval submodule.

[0060] Based on this, sub-question generation, sub-question vector calculation, and sub-question vector retrieval can be asynchronous, and the LLM, the vector generation submodule, and the vector retrieval submodule can be decoupled, so that sub-question decomposition, vector generation, and vector retrieval are maximally parallelized, thereby saving hardware idle time and improving the generation efficiency of the question and answer results.

[0061] In an implementation, in response to detecting that the large language model generates a sub-question, inputting the generated current sub-question to the vector generation submodule for vector calculation can include: in response to detecting that the large language model generates a sequence start identifier, inputting a sub-question sequence corresponding to the sequence start identifier to the vector generation submodule for vector calculation, to obtain a sub-question vector corresponding to the current sub-question, wherein the sub-question sequence includes the sequence start identifier, the current sub-question, and a sequence end identifier, and the vector generation submodule outputs the generated sub-question vector when detecting the sequence end identifier.

[0062] To facilitate the identification of the LLM's division of each sub-problem, each sub-problem is wrapped to form a structured sub-problem sequence through model fine-tuning or prompt engineering, and each sub-problem sequence sequentially contains a sequence start identifier, a sub-problem, and a sequence end identifier of the sub-problem sequence. As shown in FIG. 4, " <s>", "Who is the author?", "< / s> <s>" are the sequence start identifiers, which are used to identify that the following is a subquery,< / s> " constitutes a sub-problem sequence, wherein " <s>" are the sequence start identifiers, which are used to identify that the following is a subquery,< / s> " is a sequence end identifier used to indicate that the sub-problem output is complete. Further, when the sequence start identifier is detected in the content generated by the LLM, it is considered that the structured data appears in the content generated by the LLM, and the input vector generation submodule for the subsequent generated sub-problem can be calculated synchronously.

[0063] For example, as shown in FIG. 5, the user's target question (query) is completely input into the LLM, and the LLM starts token calculation. The system detects whether the token output by the LLM is a special token, i.e., whether the token is a sequence start identifier (such as " <s>", "Who is the author?", "< / s> <s>") are special tokens, which will start an asynchronous task and send the tokens generated by LLM into the vector generation submodule one by one. Therefore, the subquery and the sequence end identifier (such as< / s> "), and inputs the token into the vector generation submodule. The vector generation submodule asynchronously calculates the vector representation of the sub-problem and generates the sub-problem vector. If the sequence end identifier is detected, the calculated sub-problem vector is output.

[0064] For example, the vector generation submodule can use the transformer architecture, use a certain hidden layer of the transformer network as the output, and use the key-value cache (kv-cache) mechanism to accelerate the calculation. Specifically, the main role of the self-attention mechanism of the transformer network is to generate a weighted representation for each input word, and the weights are determined by the relationship between all words in the input sequence (i.e., the sub-problem sequence) and the current word. For each input word, a corresponding query vector, key vector, and value vector are generated through linear transformation; for each query vector, the dot product of it and all key vectors is calculated to obtain an attention score: the score is scaled and normalized to obtain an attention weight; the value vector is weighted and summed using the attention weight to generate an output vector. In the calculation process, the current sub-problem can be parsed into multiple tokens, and the attention of each token can be calculated according to the query vector, key vector, and value vector of each token, wherein the calculation of each key vector and value vector is cached for calling when calculating the attention of the token. Then, the sub-problem vector corresponding to the current sub-problem is determined based on the attention of each token. Therefore, using the kv-cache mechanism, the key vector and value vector calculated for each word can be cached during the processing of the sub-problem sequence in the vector generation submodule, so as to be quickly accessed in the subsequent token attention calculation.

[0065] ​Based on this, not only can the asynchronous generation of sub-questions and sub-question vectors be realized, but also the tokens generated successively can be sent into the vector generation sub-module one by one in an asynchronous manner through detection of the sequence start identifier, so as to further improve the parallelism of the LLM and the vector generation sub-module. In addition, detection of the sequence end identifier also facilitates the system to identify a complete sub-question vector.

[0066] In an implementation, in step S301, the target question is input into the large language model, so that the large language model decomposes the target question into a plurality of sub-questions, including: inputting the target question and a preset decomposition guide word into the large language model, so that the large language model determines a first attention corresponding to the decomposition guide word from a pre-stored attention calculation result, and calculates a second attention corresponding to the target question, to generate a plurality of sub-questions according to the first attention and the second attention.

[0067] The self-attention mechanism calculates through three main vectors: query vector (Query), key vector (Key) and value vector (Value), and the attention score is determined by calculating the dot product of Query and Key, and the score is normalized by the softmax function to weight the corresponding Value.

[0068] When guiding the LLM to decompose the target question into a plurality of sub-questions, prompt engineering can be used, that is, a preset decomposition guide word is added in front of (usually in front, but also behind or other positions) the target question, which is used to demonstrate and agree on the output structure of the sub-questions, so as to guide the LLM to output structured sub-questions. These decomposition guide words are the same in multiple requests, that is, the decomposition guide word is a shared prefix, the target question is a unique prefix, and the sequence input into the LLM is a combination of the shared prefix and the target question. The shared prefix is unchanged in multiple requests, while the target question is the part that changes each time. The length of the decomposition guide word far exceeds the length of the target question itself, and if the attention of this part of content is repeatedly calculated, it will cause waste of computing resources.

[0069] In the embodiments of the present disclosure, the attention calculation results (Key and Value) of the shared prefix (decomposition guide word) are pre-calculated and pre-stored, such as in the shared memory of the processor, so that these calculation results can be directly accessed for subsequent each request. Therefore, when the LLM needs to perform attention calculation on the input sequence each time, the first attention (Key and Value) corresponding to the decomposition guide word can be pre-filled, so that the attention of the decomposition guide word can be quickly obtained, and only the second attention corresponding to the target question needs to be calculated, so that the sub-questions can be quickly output.

[0070] In an implementation, in step S303, the plurality of search results are input into the large language model to obtain the question and answer result corresponding to the target question, including: inputting the plurality of search results into the large language model respectively, so that the large language model processes the plurality of search results in parallel to generate a plurality of corresponding sub-answers; and obtaining the question and answer result generated by the large language model based on the plurality of sub-answers.

[0071] For each search result, the LLM generates a corresponding sub-answer, for example, combines the search result with a sub-question to generate a sub-answer in natural language form, then aggregates all sub-answers and uses its contextual semantic understanding ability to ensure that all sub-answers are semantically and logically coherent, and finally generates a complete question and answer result.

[0072] Among them, the processing of each search result by the LLM is performed in parallel. For example, the LLM starts multiple processes or threads, each of which is used to process a search result to generate a corresponding sub-answer, thereby achieving parallel generation of multiple sub-answers. Based on this, the advantages of multi-core central processing units (CPUs) or multi-core graphics processing units (GPUs) can be fully utilized, as well as the parallel processing capabilities provided by the computing platform, to improve the computational efficiency of generating answers.

[0073] In an implementation, the plurality of search results are input into the large language model respectively, so that the large language model processes the plurality of search results in parallel to generate a plurality of corresponding sub-answers, including: in response to detecting that the search service module generates one search result, inputting the generated current search result into the large language model, so that the large language model generates a corresponding sub-answer based on the current search result, wherein the generation of the next search result by the search service module and the processing of the current search result by the large language model are performed asynchronously.

[0074] Whenever the search service module generates a search result for a sub-question, the generated current search result is directly input to the LLM for sub-answer reasoning, without waiting for all search results. Based on this, by asynchronousizing sub-question search and sub-answer generation, the search time of the search service module can be hidden in the time of LLM generating sub-answers, and by decoupling the search service module and the LLM, the parallelization of search and sub-answer generation can be maximized, thereby significantly saving computing time and improving the generation efficiency of sub-answers.

[0075] In an implementation, the current search result includes a plurality of search paragraphs, and inputting the generated current search result into the large language model to enable the large language model to generate a corresponding sub-answer based on the current search result includes: determining a total length of the plurality of search paragraphs; in response to the total length not exceeding a length threshold of the large language model, inputting the plurality of search paragraphs into the large language model to enable the large language model to generate a sub-answer corresponding to the current search result based on the plurality of search paragraphs; and in response to the total length exceeding the length threshold of the large language model, inputting a plurality of search information corresponding to the current search result into the large language model in sequence to enable the large language model to generate a corresponding sub-answer based on the plurality of search information, wherein each search information includes a search paragraph and the target question.

[0076] That is, the number and length of the search paragraphs can be considered to selectively send the obtained plurality of search paragraphs to the LLM to generate a corresponding sub-answer by the LLM. Specifically, for the plurality of search paragraphs in the search result, the system will first count the total length of the plurality of search paragraphs. In one case, the total length is within the context length limit (length threshold) of the LLM, and the generated plurality of search paragraphs are directly sent to the LLM for calculation. Through the introduction of the above kv-cache mechanism, it can be understood that this direct summary generation method can quickly generate the final question and answer result. In another case, if the total length of the plurality of search paragraphs exceeds the context limit of the LLM, each search paragraph is combined with the target question to form a search information, and then the plurality of search information corresponding to one search result is input into the LLM in sequence to obtain a sub-answer corresponding to the search result, thereby adapting to the case of long text or excessive content of the search result. In addition, the generation of a plurality of sub-answers by the LLM can be performed in parallel, for example, a plurality of processes are started in parallel, and each process is used to generate a corresponding sub-answer based on one search result (including a plurality of search paragraphs) or a plurality of search information corresponding to one search result.

[0077] In an implementation, inputting the generated current search result into the large language model to enable the large language model to generate a corresponding sub-answer based on the current search result includes: inputting the current search result and a preset sub-answer guide word into the large language model to enable the large language model to determine a third attention corresponding to the sub-answer guide word from a pre-stored attention calculation result, calculate a fourth attention corresponding to the current search result, and generate a corresponding sub-answer according to the third attention and the fourth attention.

[0078] In guiding the LLM to generate a sub-answer based on the retrieval result, prompt engineering can be adopted, that is, the retrieval result is combined with a preset sub-answer guide word and input to the LLM. The sub-answer guide word is used to demonstrate the output of the sub-answer and to agree on the output structure of the sub-answer. Therefore, the content of the sub-answer guide word is unchanged in multiple requests, that is, the sub-answer guide word is a shared prefix, and the retrieval result is a unique prefix. The length of the sub-answer guide word far exceeds the total length of the retrieval text segments in the retrieval result, and the attention of this part of the content will be repeatedly calculated, which will cause waste of computing resources.

[0079] Therefore, in the embodiments of the present disclosure, the attention calculation result of the shared prefix (sub-answer guide word) is pre-calculated and pre-stored, such as pre-stored in the shared memory of the processor, so that these calculation results can be directly accessed every time a sub-answer generation request is made subsequently. Based on this, the LLM can pre-fill the third attention corresponding to the sub-answer guide word every time it needs to perform attention calculation on the input sequence, quickly obtain the attention of the sub-answer guide word, and only need to calculate the fourth attention corresponding to the retrieval result, so as to quickly output the sub-answer, thereby avoiding the waste of computing resources caused by repeated calculation of the attention of the sub-answer guide word.

[0080] The following shows an example of a sub-answer guide word: "{__BASE_HEADER}#dialog guide word Here is the user's question: {question} Here is some context to help you answer the question: {context}".

[0081] Among them, the {context} part is the retrieval text segment. Exemplarily, the position of the retrieval text segment can be placed at the end of the input sequence, so that the length of the shared prefix is as long as possible, thereby maximizing the acceleration effect brought by the pre-filled attention of the shared prefix without affecting the quality of the LLM generation.

[0082] In an implementation, obtaining a question and answer result generated by the large language model based on a plurality of sub-answers can include: inputting the plurality of sub-answers and a preset question and answer result guide word into the large language model, so that the large language model determines a fifth attention corresponding to the question and answer result guide word from the pre-stored attention calculation result, and calculates a plurality of sixth attentions corresponding to the plurality of sub-answers, to generate the question and answer result according to the fifth attention and the plurality of sixth attentions.

[0083] In guiding the LLM to generate the final question and answer result based on multiple sub-answers, prompt engineering can be adopted, that is, multiple sub-answers are input as unique suffixes, and a preset question and answer result guide word is input as a shared prefix to the LLM. The question and answer result guide word is used to demonstrate the output of the question and answer result and to agree on the output structure of the question and answer result. Since the length of the question and answer result guide word is much longer than the total length of the multiple sub-answers, if the attention of this part of content is repeatedly calculated, it will cause waste of computing resources. In the embodiments of the present disclosure, the attention calculation result of the shared prefix can be pre-calculated and pre-stored, such as pre-stored in the shared memory of the processor. When the LLM needs to perform attention calculation on the input sequence each time, the sixth attention corresponding to the question and answer result guide word can be pre-filled to quickly obtain the attention of the question and answer result guide word, and only the multiple sixth attentions corresponding to the multiple sub-answers need to be calculated, so that the final question and answer result can be quickly output, thereby avoiding the waste of computing resources caused by repeated calculation of the attention of the question and answer result guide word.

[0084] FIGS. 6 and 7 show application examples of the embodiments of the present disclosure. As shown in FIGS. 6 and 7, the question and answer method based on the large language model of the embodiments of the present disclosure can include steps S701 to S707.

[0085] Step S701: The LLM receives a target question input by a user. As shown in FIG. 6, the LLM can calculate the target question, including calculating some irrelevant characters, which can be characters of format or tone categories that are not very relevant to the target question.

[0086] Step S702: It is judged whether the sub-problems need to be decomposed, that is, whether the target question needs to be decomposed into multiple sub-questions, if the decomposition is needed, step S703 is entered, if the decomposition is not needed, step S707 is entered. For example, if the target question may contain multiple sub-questions, it is determined that decomposition is needed, and step S703 is entered. If the target question only represents one question, it is determined that decomposition is not needed, and step S707 is entered.

[0087] Step S703: Extracting sub-questions, that is, extracting multiple sub-questions from the target question by the LLM. This step can adopt the same or similar implementation as step S301. Alternatively, attention can also be pre-filled in the process of extracting sub-questions, that is, the target question and the preset decomposition guide word are input into the large language model, so that the large language model determines the first attention corresponding to the decomposition guide word from the pre-stored attention calculation result, and calculates the second attention corresponding to the target question, to generate multiple sub-questions according to the first attention and the second attention. As shown in FIG. 6, when generating multiple sub-questions, the LLM can format and encapsulate each sub-question, that is, generate a sub-question sequence for each sub-question, including a sequence start identifier (such as “<s>") are special tokens, which will start an asynchronous task and send the tokens generated by LLM into the vector generation submodule one by one. Therefore, the subquery and the sequence end identifier (such as ") are special tokens, which will start an asynchronous task and send the tokens generated by LLM into the vector generation submodule one by one. Therefore, the subquery and the sequence end identifier (such as ") are special tokens, which will start an asynchronous task and send the tokens generated by LLM into the vector< / s>

[0088] Step S704: Retrieving from the database, which can adopt the same or similar implementation as step S302. Alternatively, as shown in FIG. 6, during the retrieval process, the LLM, the vector generation submodule and the vector retrieval submodule can be called asynchronously, that is, in response to detecting that the large language model generates a sub-question, the generated current sub-question is input into the vector generation submodule for vector calculation, wherein the generation of the next sub-question by the large language model and the vector calculation of the current sub-question by the vector generation submodule are performed asynchronously. Alternatively, when it is detected that the content generated by the LLM appears a sequence starting mark, it is considered that the content generated by the LLM appears structured data, and the input vector of the sub-question generated thereafter can be input into the vector generation submodule for vector calculation synchronously. Further, in response to detecting that the vector generation submodule generates a sub-question vector, the generated current sub-question vector is input into the vector retrieval submodule for retrieval to obtain the corresponding retrieval result, wherein the generation of the next sub-question vector by the vector generation submodule and the retrieval of the current sub-question vector by the vector retrieval submodule are performed asynchronously. Wherein, the vector generation submodule can rewrite the target question before generating the sub-question, calculate the vector of the rewritten result of the target question, and facilitate subsequent input into the LLM for reasoning.

[0089] Alternatively, as shown in FIG. 6, during the retrieval phase, the vector retrieval submodule can use a hierarchical index structure for retrieval. Specifically, at the beginning of retrieval, the vector retrieval submodule first needs to determine the primary index of the database, which is a coarse-grained division that divides the vector data in the entire database into multiple large blocks, and then the query rewriting method extracts the summary information of the target question, performs embedding representation on the summary information to obtain a query vector, which is used to determine the primary index list of the database; the vector retrieval submodule selects several large blocks most relevant to the query vector in the primary index, for example, by comparing the similarity between the query vector and the center of each block to select several large blocks most relevant to the query vector, and the selected blocks determine the preliminary range of database retrieval, thereby narrowing the subsequent search range. After determining the primary index, the system can prefetch the data in these blocks into the cache for fast access, and subsequent queries no longer need to traverse the entire database, which can significantly improve the retrieval speed, and then in the blocks selected by the primary index, the secondary index or multi-level index is used to further narrow the search range until the most relevant vector data is located as the retrieval result.

[0090] Step S705: Splicing sub-questions, that is, splicing each retrieval result and the corresponding sub-question to input into the LLM.

[0091] ​Step S706: The LLM generates a sub-answer for each search result, obtaining a plurality of sub-answers. As shown in FIG. 6, on the one hand, the generation of each sub-answer by the LLM is parallel computing, and on the other hand, in response to detecting that the search service module generates a search result, the generated current search result is input into the large language model to enable the large language model to generate a corresponding sub-answer based on the current search result, wherein the generation of the next search result by the search service module and the processing of the current search result by the large language model are executed asynchronously. Optionally, when generating the sub-answer, the attention can be pre-filled, that is, the current search result and a preset sub-answer guide word are input into the large language model to enable the large language model to determine a third attention corresponding to the sub-answer guide word from a pre-stored attention calculation result, calculate a fourth attention corresponding to the current search result, and generate a corresponding sub-answer according to the third attention and the fourth attention.

[0092] Step S707: Calculate the question and answer result, such as obtaining the question and answer result generated by the large language model based on the plurality of sub-answers. Optionally, the attention can be pre-filled, that is, the plurality of sub-answers and a preset question and answer result guide word are input into the large language model to enable the large language model to determine a fifth attention corresponding to the question and answer result guide word from a pre-stored attention calculation result, and calculate a plurality of sixth attentions corresponding to the plurality of sub-answers, to generate the question and answer result according to the fifth attention and the plurality of sixth attentions.

[0093] As shown in FIGS. 6 and 7, it can be seen that by asynchronously calling the LLM, the vector generation submodule and the vector retrieval submodule, the LLM, the vector generation submodule and the vector retrieval submodule can be decoupled without changing the network structure of the LLM, on the one hand, the parallelism of sub-problem decomposition, sub-problem vector generation, sub-problem vector retrieval and sub-answer generation can be maximized, on the other hand, the generation time of the sub-problem can be hidden in the generation time of the sub-problem vector, the generation time of the sub-problem vector can be hidden in the retrieval time of the sub-problem vector, and the retrieval time of the sub-problem vector can be hidden in the generation time of the sub-answer, thereby saving the calculation time.

[0094] Optionally, during the calculation generation process, a shared prefix (decomposition guide word or sub-answer guide word or question and answer result guide word) can be pre-stored, so that when attention calculation is performed on the input sequence each time, the attention calculation result of the shared prefix can be pre-filled, thereby avoiding the waste of resources caused by repeatedly calculating the attention of the guide word part with a very long length.

[0095] Optionally, the generation process of the LLM for the plurality of sub-answers is parallel execution, so that the advantages of multi-core central processing unit (CPU) or multi-core graphics processing unit (GPU) and the parallel processing capability provided by the computing platform can be fully utilized, and the calculation efficiency of generating the answer is improved.

[0096] Optionally, the structure of the input sequence of the LLM can be optimized, the leading word template part is placed in front, the shared prefix and the unique suffix are distinguished, and the acceleration effect of pre-filling the shared prefix is maximized.

[0097] Corresponding to the application scenario of the method and the method provided by the embodiments of the present disclosure, the embodiments of the present disclosure also provide a large language model-based question and answer device, which comprises: a target question input module configured to input a target question into a large language model, so that the large language model decomposes the target question into a plurality of sub-questions; an asynchronous calling module configured to, in response to detecting that the large language model generates a sub-question, asynchronously call a retrieval service module to retrieve a generated current sub-question to obtain a corresponding retrieval result; and a question and answer result determination module configured to input a plurality of the retrieval results into the large language model to obtain a question and answer result corresponding to the target question.

[0098] In an implementation, the retrieval service module comprises a vector generation submodule and a vector retrieval submodule; the asynchronous calling module comprises: a first calling submodule configured to, in response to detecting that the large language model generates a sub-question, input a generated current sub-question into the vector generation submodule for vector calculation, wherein the generation of a next sub-question by the large language model and the vector calculation of the current sub-question by the vector generation submodule are performed asynchronously; and a second calling submodule configured to, in response to detecting that the vector generation submodule generates a sub-question vector, input a generated current sub-question vector into the vector retrieval submodule for retrieval to obtain a corresponding retrieval result, wherein the generation of a next sub-question vector by the vector generation submodule and the retrieval of the current sub-question vector by the vector retrieval submodule are performed asynchronously.

[0099] In an implementation, the first calling submodule is configured to: in response to detecting that the large language model generates a sequence start identifier, input a sub-question sequence corresponding to the sequence start identifier into the vector generation submodule for vector calculation to obtain a sub-question vector corresponding to the current sub-question, wherein the sub-question sequence comprises the sequence start identifier, the current sub-question and a sequence end identifier, and the vector generation submodule outputs a generated sub-question vector in a case where the sequence end identifier is detected.

[0100] In an implementation, the target question input module is configured to: input the target question and a preset decomposition guide word into the large language model, so that the large language model determines a first attention corresponding to the decomposition guide word from a pre-stored attention calculation result, calculates a second attention corresponding to the target question, and generates the plurality of sub-questions according to the first attention and the second attention.

[0101] In an implementation, the question and answer result determination module comprises: a sub-answer generation sub-module configured to input the plurality of search results into the large language model respectively, so that the large language model processes the plurality of search results in parallel to generate a plurality of corresponding sub-answers; and a question and answer result generation module configured to obtain a question and answer result generated by the large language model based on the plurality of sub-answers.

[0102] In an implementation, the sub-answer generation sub-module is configured to, in response to detecting that the search service module generates one search result, input the generated current search result into the large language model, so that the large language model generates a corresponding sub-answer based on the current search result, wherein the generation of the next search result by the search service module is performed asynchronously with the processing of the current search result by the large language model.

[0103] In an implementation, the current search result comprises a plurality of search text segments, and the inputting of the generated current search result into the large language model, so that the large language model generates a corresponding sub-answer based on the current search result, comprises: determining a total length of the plurality of search text segments; and in response to the total length not exceeding a length threshold of the large language model, inputting the plurality of search text segments into the large language model, so that the large language model generates a sub-answer corresponding to the current search result based on the plurality of search text segments.

[0104] In an implementation, the current search result comprises a plurality of search text segments, and the inputting of the generated current search result into the large language model, so that the large language model generates a corresponding sub-answer based on the current search result, comprises: determining a total length of the plurality of search text segments; and in response to the total length exceeding a length threshold of the large language model, inputting a plurality of search information corresponding to the current search result into the large language model in sequence, so that the large language model generates a corresponding sub-answer based on the plurality of search information, wherein each search information comprises one search text segment and the target question.

[0105] In an implementation, the sub-answer generation sub-module is configured to input the current search result and a preset sub-answer guide word into the large language model, so that the large language model determines a third attention corresponding to the sub-answer guide word from a pre-stored attention calculation result, calculates a fourth attention corresponding to the current search result, and generates a corresponding sub-answer according to the third attention and the fourth attention.

[0106] In an implementation, the question and answer result generation submodule is configured to: input the plurality of sub-answers and the preset question and answer result guide word into the large language model, so that the large language model determines a fifth attention corresponding to the question and answer result guide word from a pre-stored attention calculation result, and calculates a plurality of sixth attentions corresponding to the plurality of sub-answers, to generate the question and answer result according to the fifth attention and the plurality of sixth attentions.

[0107] The functions of each module in each device of the embodiments of the present disclosure can be referred to the corresponding description in the above method, and have the corresponding beneficial effects, which will not be repeated here.

[0108] FIG. 8 is a block diagram of an electronic device for implementing the embodiments of the present disclosure. As shown in FIG. 8, the electronic device includes a memory 801 and a processor 802, and the memory 801 stores a computer program executable on the processor 802. The processor 802 implements the method in the above embodiments when executing the computer program. The number of the memory 801 and the processor 802 can be one or more. In a specific implementation, the electronic device can further include a communication interface 803 for communicating with external devices and transmitting data.

[0109] In a specific implementation, if the memory 801, the processor 802 and the communication interface 803 are independently implemented, the memory 801, the processor 802 and the communication interface 803 can be connected to each other through a bus and complete communication between each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the convenience of representation, only one thick line is used in FIG. 8, but it does not mean that there is only one bus or only one type of bus.

[0110] Optionally, in a specific implementation, if the memory 801, the processor 802 and the communication interface 803 are integrated on a chip, the memory 801, the processor 802 and the communication interface 803 can complete communication between each other through an internal interface.

[0111] The embodiments of the present disclosure provide a computer readable storage medium, which stores a computer program, and the program is executed by a processor to implement the method provided in the embodiments of the present disclosure.

[0112] The embodiment of the present disclosure provides a computer program product, comprising a computer program, which, when executed by a processor, implements the method provided in the embodiment of the present disclosure.

[0113] The embodiment of the present disclosure further provides a chip, comprising a processor, which is used for calling and running instructions stored in a memory, so that a communication device installed with the chip executes the method provided in the embodiment of the present disclosure.

[0114] The embodiment of the present disclosure further provides a chip, comprising an input interface, an output interface, a processor and a memory, the input interface, the output interface, the processor and the memory are connected through internal connection paths, and the processor is used for executing code in the memory, when the code is executed, the processor is used for executing the method provided in the embodiment of the present disclosure.

[0115] It should be understood that the processor described above can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc. It should be noted that the processor can be a processor supporting an advanced RISC machine (ARM) architecture.

[0116] Further, the memory can include a read-only memory and a random access memory, optionally. The memory can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memory. The non-volatile memory can include a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically EPROM (EEPROM), or a flash memory, among others. The volatile memory can include a random access memory (RAM), which is used as an external cache. By way of example, and not limitation, many forms of RAM are available. The RAM can include a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate SDRAM (DDR SDRAM), an enhanced SDRAM (ESDRAM), a Sync link DRAM (SLDRAM), and a direct Rambus RAM (DR RAM), among others.

[0117] In the above-described embodiments, all or part of the embodiments can be implemented by software, hardware, firmware, or any combination thereof. When implemented by software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions according to the present disclosure are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable apparatus. The computer instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another computer-readable storage medium.

[0118] In the description of the present disclosure, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" and the like means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present disclosure. Moreover, the specific features, structures, materials or characteristics described can be combined in any appropriate manner in any one or more embodiments or examples. In addition, different embodiments or examples described in the present disclosure and the features of different embodiments or examples can be combined and combined by those skilled in the art without contradiction, if necessary.

[0119] In addition, the terms "first", "second" are only for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined with "first", "second" can explicitly or implicitly include at least one of the features. In the description of the present disclosure, the meaning of "a plurality of" is two or more, unless otherwise specifically limited.

[0120] Any process or method described in the flowchart or otherwise described herein can be understood as a representation of code, including one or more executable instructions for implementing the specific logical functions or steps, modules, segments or portions. And the scope of the preferred embodiments of the present disclosure includes additional implementations, in which the functions can be performed in the order shown or discussed, including in a substantially simultaneous manner or in reverse order according to the functions involved.

[0121] The logic and / or steps described in the flowchart or otherwise described herein, for example, can be considered as a list of executable instructions for implementing the logic function, which can be embodied in any computer-readable medium for use by or in conjunction with an instruction execution system, device or apparatus, such as a computer-based system, a system including a processor or other system that can fetch and execute instructions from the instruction execution system, device or apparatus.

[0122] It should be understood that parts of the present disclosure can be implemented in hardware, software, firmware or a combination thereof. In the above-described embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. All or part of the steps of the above-described embodiment method can be completed by a program instructing the relevant hardware, which can be stored in a computer-readable storage medium, and the program includes one or a combination of the steps of the method embodiment when executed.

[0123] In addition, each function unit in each embodiment of the present disclosure can be integrated in one processing module, or each unit can be physically present separately, or two or more units can be integrated in one module. The integrated module can be realized in the form of hardware or in the form of a software function module. When the integrated module is realized in the form of a software function module and sold or used as an independent product, it can also be stored in a computer readable storage medium. The storage medium can be a read-only memory, a magnetic disk or an optical disk, etc.

[0124] The above is only an exemplary embodiment of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any person skilled in the art can easily think of various changes or replacements within the technical range disclosed in the present disclosure, and these should be covered within the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be subject to the protection scope of the claims.

Claims

1. A question-answering method based on a large language model, comprising: The target problem is input into a large language model, which then decomposes the target problem into multiple sub-problems. In response to the detection that the large language model has generated a sub-question, the retrieval service module is asynchronously invoked to retrieve the generated current sub-question and obtain the corresponding retrieval results; The multiple search results are input into the large language model to obtain the question-and-answer results corresponding to the target question.

2. The question-and-answer method according to claim 1, wherein, The retrieval service module includes a vector generation submodule and a vector retrieval submodule; in response to detecting that the large language model outputs a sub-question, the retrieval service module is asynchronously invoked to retrieve the generated current sub-question, and the corresponding retrieval results are obtained, including: In response to detecting that the large language model generates a sub-problem, the generated current sub-problem is input into the vector generation submodule for vector computation, wherein the generation of the next sub-problem by the large language model and the vector computation of the current sub-problem by the vector generation submodule are executed asynchronously; In response to the detection that the vector generation submodule generates a sub-question vector, the generated current sub-question vector is input into the vector retrieval submodule for retrieval to obtain the corresponding retrieval result. The generation of the next sub-question vector by the vector generation submodule and the retrieval of the current sub-question vector by the vector retrieval submodule are executed asynchronously.

3. The question-and-answer method according to claim 2, wherein, In response to detecting that the large language model generates a sub-question, the generated current sub-question is input into the vector generation submodule for vector computation, including: In response to the detection of the start identifier of the sequence generated by the large language model, the sub-problem sequence corresponding to the start identifier is input into the vector generation submodule for vector calculation to obtain the sub-problem vector corresponding to the current sub-problem. The sub-problem sequence includes the start identifier, the current sub-problem, and the end identifier. When the end identifier is detected, the vector generation submodule outputs the generated sub-problem vector.

4. The method according to any one of claims 1 to 3, wherein, The target problem is input into a large language model, which then decomposes the target problem into multiple sub-problems, including: The target question and the preset decomposition guide words are input into the large language model, so that the large language model determines the first attention corresponding to the decomposition guide words from the pre-stored attention calculation results, and calculates the second attention corresponding to the target question, so as to generate the multiple sub-questions based on the first attention and the second attention.

5. The question-and-answer method according to any one of claims 1 to 3, wherein, Inputting multiple search results into the large language model yields the question-and-answer results corresponding to the target question, including: The multiple search results are respectively input into the large language model so that the large language model processes the multiple search results in parallel and generates multiple corresponding sub-answers; Obtain the question-and-answer results generated by the large language model based on the multiple sub-answers.

6. The question-and-answer method according to claim 5, wherein, Multiple search results are input into the large language model to enable the large language model to process the multiple search results in parallel and generate multiple corresponding sub-answers, including: In response to detecting that the retrieval service module has generated a retrieval result, the generated current retrieval result is input into the large language model so that the large language model generates a corresponding sub-answer based on the current retrieval result. The generation of the next retrieval result by the retrieval service module and the processing of the current retrieval result by the large language model are executed asynchronously.

7. The question-and-answer method according to claim 6, wherein, The current search results include multiple search segments. The generated current search results are input into the large language model so that the large language model generates corresponding sub-answers based on the current search results, including: Determine the total length of the plurality of search segments; In response to the total length not exceeding the length threshold of the large language model, the multiple search segments are input into the large language model so that the large language model generates a sub-answer corresponding to the current search result based on the multiple search segments.

8. The question-and-answer method according to claim 6, wherein, The current search results include multiple search segments. The generated current search results are input into the large language model so that the large language model generates corresponding sub-answers based on the current search results, including: Determine the total length of the plurality of search segments; In response to the total length exceeding the length threshold of the large language model, multiple search information corresponding to the current search result are sequentially input into the large language model so that the large language model generates corresponding sub-answers based on the multiple search information, wherein each search information includes a search passage and the target question.

9. The question-and-answer method according to claim 6, wherein, The generated current search results are input into the large language model, so that the large language model generates corresponding sub-answers based on the current search results, including: The current search result and the preset sub-answer guide words are input into the large language model, so that the large language model determines the third attention corresponding to the sub-answer guide words from the pre-stored attention calculation results, calculates the fourth attention corresponding to the current search result, and generates the corresponding sub-answer based on the third attention and the fourth attention.

10. The question-and-answer method according to claim 5, wherein, Obtaining the question-answering results generated by the large language model based on the multiple sub-answers includes: The multiple sub-answers and the preset question-and-answer result guide words are input into the large language model, so that the large language model determines the fifth attention corresponding to the question-and-answer result guide word from the pre-stored attention calculation results, and calculates multiple sixth attentions corresponding to the multiple sub-answers, so as to generate the question-and-answer result based on the fifth attention and the multiple sixth attentions.

11. An electronic device comprising a memory, a processor, and a computer program stored in the memory, wherein the processor, when executing the computer program, implements the method of any one of claims 1 to 10.

12. A computer-readable storage medium storing a computer program that, when executed by a processor, implements the method of any one of claims 1 to 10.

13. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1 to 10.

Images