Question and answer method and device based on large language model, medium and product

HK40137285APending Publication Date: 2026-09-11CLOUD INTELLIGENCE ASSETS HOLDING (SINGAPORE) PTE LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
HK42026126147
Authority / Receiving Office
HK · HK
Patent Type
Applications
Current Assignee / Owner
Filing Date
2026-07-14
Publication Date
2026-09-11

Smart Images

  • Figure 00000000_0000_ABST
    Figure 00000000_0000_ABST
Patent Text Reader

Abstract

The embodiment of the invention provides a question answering method and device based on a large language model, a medium and a product, and relates to the technical field of natural language process.The method comprises the steps that a target question is input into the large language model, so that the target question is decomposed into multiple sub-questions through the large language model; in response to the fact that the large language model is detected to generate a sub-problem, asynchronously calling a retrieval service module to retrieve the generated current sub-problem to obtain a corresponding retrieval result; and inputting the plurality of retrieval results into a large language model to obtain a question and answer result corresponding to the target question. According to the technical scheme provided by the embodiment of the invention, the degree of parallelism in the calculation generation process can be improved and the calculation time can be saved on the premise of not changing the network structure of the large language model.
Need to check novelty before this filing date? Find Prior Art

Description

(19) State Intellectual Property Office (12) Invention Patent Application (10) Application Publication Number (43) Application Publication Date (21) Application Number 202411259912.0 (22) Application Date 2024.09.09 (71) Applicant Alibaba Cloud Computing Co., Ltd. Address Room 1-2-A06, Yung Valley Park, No. 1008, Dengcai Street, Sandun Town, Xihu District, Hangzhou City, Zhejiang Province, 310030 (72) Inventors Wang Nan, Ling Xingyu, Zhou Peng (74) Patent Agency Beijing Chenquan Intellectual Property Agency Co., Ltd. 11619 Patent Attorney Bao Lili (51) Int.Cl. G06N 5 / 04 (2023.01) G06F 16 / 3329 (2025.01) G06F 16 / 334 (2025.01) (54) Invention Title: Question Answering Method, Device, Medium, and Product Based on Large Language Model (57) Abstract: This application provides a question answering method, device, medium, and product based on a large language model, relating to the field of natural language processing technology. The method includes: inputting a target question into a large language model, so that the large language model decomposes the target question into multiple sub-questions; in response to detecting that the large language model generates a sub-question, asynchronously calling a retrieval service module to retrieve the generated current sub-question and obtain the corresponding retrieval results; inputting multiple retrieval results into the large language model to obtain the question answering results corresponding to the target question. The technical solution of this application can improve the parallelism in the computation generation process and save computation time without changing the network structure of the large language model. Claims (2 pages), Description (15 pages), Drawings (5 pages), CN 121660059 A, 2026.03.13, CN 1 21 66 00 59 A. 1. A question-answering method based on a large language model, comprising: inputting a target question into a large language model, so that the large language model decomposes the target question into multiple sub-questions; responding to detecting that the large language model generates a sub-question, asynchronously calling a retrieval service module to retrieve the generated current sub-question, and obtaining corresponding retrieval results; inputting multiple retrieval results into the large language model to obtain question-answering results corresponding to the target question.2. The question-answering method according to claim 1, wherein the retrieval service module includes a vector generation submodule and a vector retrieval submodule; in response to detecting that the large language model outputs a sub-question, the retrieval service module is asynchronously invoked to retrieve the generated current sub-question and obtain the corresponding retrieval result, including: in response to detecting that the large language model generates a sub-question, inputting the generated current sub-question into the vector generation submodule for vector calculation, wherein the generation of the next sub-question by the large language model and the vector calculation of the current sub-question by the vector generation submodule are executed asynchronously; in response to detecting that the vector generation submodule generates a sub-question vector, inputting the generated current sub-question vector into the vector retrieval submodule for retrieval and obtaining the corresponding retrieval result, wherein the generation of the next sub-question vector by the vector generation submodule and the retrieval of the current sub-question vector by the vector retrieval submodule are executed asynchronously. 3. The question-answering method according to claim 2, wherein, in response to detecting that the large language model generates a sub-question, the generated current sub-question is input into the vector generation submodule for vector calculation, comprising: in response to detecting that the large language model generates a sequence start identifier, the sub-question sequence corresponding to the sequence start identifier is input into the vector generation submodule for vector calculation to obtain the sub-question vector corresponding to the current sub-question, wherein the sub-question sequence includes the sequence start identifier, the current sub-question, and the sequence end identifier, and the vector generation submodule outputs the generated sub-question vector when the sequence end identifier is detected. 4. The method according to any one of claims 1 to 3, wherein, inputting a target question into a large language model to decompose the target question into multiple sub-questions by the large language model, comprising: inputting the target question and a preset decomposition guide word into the large language model to determine a first attention corresponding to the decomposition guide word from pre-stored attention calculation results, and calculating a second attention corresponding to the target question, so as to generate the multiple sub-questions based on the first attention and the second attention. 5. The question-answering method according to any one of claims 1 to 3, wherein inputting multiple search results into the large language model to obtain question-answering results corresponding to the target question includes: inputting multiple search results into the large language model respectively, so that the large language model processes multiple search results in parallel to generate multiple corresponding sub-answers; and obtaining question-answering results generated by the large language model based on the multiple sub-answers.6. The question-answering method according to claim 5, wherein inputting multiple search results into the large language model to enable the large language model to process multiple search results in parallel and generate multiple corresponding sub-answers includes: in response to detecting that the search service module generates a search result, inputting the generated current search result into the large language model to enable the large language model to generate a corresponding sub-answer based on the current search result, wherein the generation of the next search result by the search service module and the processing of the current search result by the large language model are executed asynchronously. 7. The question-answering method according to claim 6, wherein the current search result includes multiple search segments, and inputting the generated current search result into the large language model to enable the large language model to generate a sub-answer corresponding to claim 1 / 2 page 2 CN 121660059 A based on the current search result includes: determining the total length of the multiple search segments; in response to the total length not exceeding the length threshold of the large language model, inputting the multiple search segments into the large language model to enable the large language model to generate a sub-answer corresponding to the current search result based on the multiple search segments. 8. The question-answering method according to claim 6, wherein the current search result includes multiple search segments, and inputting the generated current search result into the large language model to enable the large language model to generate a corresponding sub-answer based on the current search result includes: determining the total length of the multiple search segments; in response to the total length exceeding the length threshold of the large language model, sequentially inputting multiple search information corresponding to the current search result into the large language model to enable the large language model to generate a corresponding sub-answer based on the multiple search information, wherein each search information includes a search segment and the target question. 9. The question-answering method according to claim 6, wherein inputting the generated current search result into the large language model to enable the large language model to generate a corresponding sub-answer based on the current search result includes: inputting the current search result and preset sub-answer guide words into the large language model to enable the large language model to determine a third attention corresponding to the sub-answer guide word from pre-stored attention calculation results, calculate a fourth attention corresponding to the current search result, and generate a corresponding sub-answer based on the third attention and the fourth attention.10. The question-answering method according to claim 5, wherein obtaining the question-answering result generated by the large language model based on the plurality of sub-answers includes: inputting the plurality of sub-answers and preset question-answering result guide words into the large language model, so that the large language model determines a fifth attention corresponding to the question-answering result guide words from pre-stored attention calculation results, and calculates a plurality of sixth attentions corresponding to the plurality of sub-answers, so as to generate the question-answering result based on the fifth attention and the plurality of sixth attentions. 11. An electronic device, comprising a memory, a processor, and a computer program stored in the memory, wherein the processor, when executing the computer program, implements the method according to any one of claims 1 to 10. 12. A computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the method according to any one of claims 1 to 10. 13. A computer program product, comprising a computer program, wherein the computer program, when executed by a processor, implements the method according to any one of claims 1 to 10. Claims 2 / 2 Page 3 CN 121660059 A Question Answering Method, Device, Medium and Product Based on Large Language Model Technical Field

[0001] This application relates to the field of natural language processing technology, and in particular to a question answering method, electronic device, question answering system based on large language model, computer-readable storage medium and computer program product, which can be applied to the field of artificial intelligence. Background Art

[0002] With the rapid development of natural language processing technology, large language model (LLM) has become a core tool for acquiring, generating and improving text content. Large language model can understand and generate natural language and is widely used in chatbots, automatic summarization, translation systems and other language understanding tasks. Although large language model has made great progress in content generation, the content it generates still faces limitations in accuracy and timeliness of knowledge. Summary of the Invention

[0003] The embodiments of this application provide a question answering method, electronic device, question answering system based on large language model, computer-readable storage medium and computer program product to alleviate or solve one or more technical problems existing in the prior art.

[0004] In a first aspect, embodiments of this application provide a question-answering method based on a large language model, comprising: inputting a target question into a large language model so that the large language model decomposes the target question into multiple sub-questions; in response to detecting that the large language model generates a sub-question, asynchronously calling a retrieval service module to retrieve the generated current sub-question and obtain corresponding retrieval results; inputting multiple retrieval results into the large language model to obtain question-answering results corresponding to the target question.

[0005] In a second aspect, embodiments of this application provide an electronic device, including a memory, a processor, and a computer program stored in the memory, wherein the processor, when executing the computer program, implements the method of any one of the embodiments of this application.

[0006] In a third aspect, embodiments of this application provide a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the method of any one of the embodiments of this application.

[0007] In a fourth aspect, embodiments of this application provide a computer program product, including a computer program, wherein the computer program, when executed by a processor, implements the method of any one of the embodiments of this application.

[0008] According to the technical solution of the embodiments of this application, when generating answers, the large language model no longer relies solely on the knowledge learned during its pre-training, but can also dynamically call an external retrieval service module to obtain the latest and most relevant information, thereby improving the accuracy and timeliness of the generated content. Furthermore, by asynchronously calling the retrieval service module, that is, whenever the large language model generates a sub-question, it directly inputs the generated current sub-question into the retrieval service module for retrieval. Therefore, the computation time for the large language model to generate the next sub-question is hidden in the retrieval time of the retrieval service module for the current sub-question, thereby significantly saving computation time and improving the efficiency of question-and-answer result generation. Moreover, by decoupling the large language model and the retrieval service module, the parallelism in the computation generation process can be improved without changing the network structure of the large language model, saving computation time.

[0009] Optionally, the retrieval service module includes a vector generation submodule and a vector retrieval submodule. Each time the LLM generates a sub-question, it asynchronously inputs it into the vector generation submodule, thus hiding the generation time of the LLM for the next sub-question within the vector calculation time of the vector generation submodule for the current sub-question. Each time the vector generation submodule generates a sub-question vector, it asynchronously inputs it into the vector retrieval submodule, thus hiding the generation time of the vector generation submodule for the next sub-question vector within the retrieval time of the vector retrieval submodule for the current sub-question vector. Based on this, sub-question generation, sub-question vector calculation, and sub-question vector retrieval can be asynchronous, decoupling the LLM, vector generation submodule, and vector retrieval submodule, maximizing the parallelization of sub-question decomposition, vector generation, and vector retrieval, thereby saving hardware idle time and improving the efficiency of question-and-answer result generation.

[0010] Optionally, during the calculation and generation process, shared prefixes can be pre-stored. For example, attention for decomposition guide words can be pre-filled during the query decomposition stage, attention for sub-answer guide words can be pre-filled when generating each sub-answer, and attention for question-answer result guide words can be pre-filled when generating the final question-answer result. Thus, when attention calculation is performed on the input sequence each time, the attention calculation result of the shared prefix can be pre-filled, avoiding the waste of resources caused by repeatedly calculating the attention of long guide words.

[0011] Optionally, the advantages of multi-core CPUs or multi-core GPUs can be fully utilized, as well as the parallel processing capabilities provided by the computing platform, so that the parallel generation of multiple sub-answers by LLM becomes a parallel processing process, improving the computational efficiency of answer generation.

[0012] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application, it can be implemented according to the contents of the specification. In order to make the above and other objects, features and advantages of this application more obvious and understandable, specific embodiments of this application are given below.

[0013] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the various drawings denote the same or similar parts or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings only depict some embodiments according to this application and should not be regarded as limiting the scope of this application.

[0014] FIG1 shows an exemplary architecture diagram of a question-answering system based on a large language model in an embodiment of this application;

[0015] FIG2 shows a timing diagram of an application example of the question-answering system;

[0016] FIG3 shows a flowchart of a question-answering method based on a large language model in an embodiment of this application;

[0017] FIG4 shows a schematic diagram of a sub-question vector generation process in an embodiment of this application;

[0018] FIG5 shows another schematic diagram of a sub-question vector generation process in an embodiment of this application;

[0019] FIG6 shows a timing diagram of a question-answering method based on a large language model in an embodiment of this application;

[0020] FIG7 shows an application example diagram of a question-answering method based on a large language model in an embodiment of this application;

[0021] FIG8 shows a block diagram of an electronic device provided in an embodiment of this application. Detailed Description of Embodiments

[0022] In the following, only certain exemplary embodiments are briefly described. As those skilled in the art will recognize, the described embodiments can be modified in various ways without departing from the concept or scope of this application. Therefore, the drawings and descriptions are considered to be exemplary in nature and not restrictive.

[0023] To facilitate understanding of the technical solutions of the embodiments of this application, the related technologies of the embodiments of this application are described below. The following related technologies are optional solutions and can be arbitrarily combined with the technical solutions of the embodiments of this application, and all of them fall within the protection scope of the embodiments of this application.

[0024] The following terms will be used hereinafter: Specification 2 / 15 page 5 CN 121660059 A

[0025] Large Language Model (LLM): A large model is a large-scale machine learning model used in the field of Natural Language Processing (NLP) to understand or generate human language. LLMs are trained on very large text datasets using deep learning techniques, enabling them to capture the complex patterns and diversity of natural language.

[0026] Retrieval Augmented Generation (RAG) Framework: A natural language processing model framework that integrates retrieval and generation mechanisms. By introducing an information retrieval module, RAG enhances the knowledge base of LLMs. LLMs no longer rely solely on the knowledge learned during pre-training but can also dynamically retrieve the latest and most relevant information from external databases.

[0027] Embedding Model: A model that maps high-dimensional sparse data (such as words, sentences, images, etc.) to a low-dimensional dense vector space. This low-dimensional dense vector is called an "embedding vector". The basic idea of ​​the embedding model is to learn a function to convert input data into a dense vector of fixed dimensions. This function is usually obtained through neural network training, with the aim of making similar input data have similar vector representations in the embedding space, while dissimilar data have relatively distant vector representations.

[0028] In the field of natural language processing, large language models can understand and generate natural language and are widely used in chatbots, automatic summarization, question answering systems, and other language understanding tasks. Although large language models have made great progress in content generation, the accuracy of the content they generate still faces limitations.

[0029] In one implementation, query and user-input documents can be embedded in the LLM, and the retrieved documents can be concatenated into the context, so that the LLM can combine external knowledge information in the process of generating answers. However, the external knowledge information obtained by this implementation method is very limited.

[0030] In another implementation, the RAG framework can be introduced on top of LLM. The RAG framework combines a retrieval mechanism, searching for relevant information in an external knowledge base before generating an answer, making the LLM more accurate and richer in generating answers, thereby improving the quality and relevance of the content generated by the LLM and enhancing the question-answering effect of the LLM. However, the RAG framework requires very high computational resources and time costs when handling complex queries. Both vector generation and retrieval, as well as reasoning in large models, require a lot of computational resources and time.With the continuous growth of information, the existing RAG framework implementation for LLM content generation is difficult to meet the requirements of real-time and efficient content retrieval, and also limits the performance of the RAG framework in practical LLM applications.

[0031] In view of this, the present application provides a question answering method and question answering system based on a large language model, which can improve the parallelism of the RAG framework and save computation time without changing the model network structure. To facilitate the understanding of the embodiments of the present application, the system architecture applicable to the embodiments of the present application is first briefly described. Figure 1 shows an exemplary architecture diagram of the question answering system based on a large language model in the embodiments of the present application, and Figure 2 shows a timing diagram of an application example of the question answering system. As shown in Figures 1 and 2, the question answering system based on a large language model in the embodiments of the present application includes a client 110 and a server 120, wherein the server 120 is deployed with a large language model and a retrieval service module.

[0032] The large language model and the retrieval service module can be deployed on the same subject or multiple subjects. The subject can be an application, service, instance, functional module of software form, virtual machine, container or cloud server, or hardware device or hardware chip with data processing function. The server 120 can be deployed on a local computing device or a cloud computing platform or cluster that provides computing power, storage, and network resources. The specific application architecture can be built according to service requirements. The client 110 can be hardware, such as mobile phones, personal computers, tablets, wearable devices, and other electronic devices; the client 110 can also be an application (APP) or software module deployed on an electronic device.

[0033] In an application example, as shown in Figure 2, the user can input a target question in natural language form through the interactive interface provided by the client 110, such as the target question "Who is the author of this article, and what dataset was used?" The client manual 3 / 15 page 6 CN 121660059 A 110 sends the target question to the server 120.

[0034] The server 120 calls the large language model, inputs the target question into the large language model, and the large language model decomposes the target question into multiple sub-questions, such as sub-question 1 "Who is the author of this article?" and sub-question 2 "What dataset was used in this article?". Among them, multiple sub-questions can be output in text form. The server 120 calls the retrieval service module to retrieve each sub-question from the database and obtain the corresponding retrieval results. For example, sub-question 1 corresponds to retrieval result 1, and sub-question 2 corresponds to retrieval result 2. Multiple retrieval results can be output in text form.

[0035] The server 120 calls the large language model, which performs reasoning on the multiple retrieval results and the target question, and outputs the question-and-answer results corresponding to the target question.The server 120 returns the question-and-answer results to the client 110, so that the user can obtain the question-and-answer results corresponding to the target question.

[0036] In this embodiment of the application, the server 120 calls the large language model and the retrieval service module asynchronously. That is, for each sub-question generated by the large language model, the server 120 asynchronously calls the retrieval service module, inputs the generated sub-question into the retrieval service module, and the retrieval service module retrieves the corresponding retrieval results from the database. In other words, the generation of the next sub-question by the large language model and the retrieval of the current sub-question by the retrieval service module are executed asynchronously.

[0037] In related technologies, the large language model needs to generate all sub-questions and use the text containing all sub-questions to retrieve the results from the database. This implementation method can meet the user's needs in the short text scenario, but if the user inputs long text, it will waste a lot of computing time and it is difficult to meet the real-time and efficient retrieval requirements.

[0038] In the technical solution of this application embodiment, by asynchronously calling the retrieval service module, that is, whenever the large language model generates a sub-question, the generated current sub-question is directly input to the retrieval service module for retrieval. Therefore, the computation time of the large language model for generating the next sub-question is hidden in the retrieval time of the retrieval service module for the current sub-question, thereby significantly saving computation time and improving the generation efficiency of the entire system.

[0039] The question-answering system of this application embodiment can be applied to various application scenarios such as human-computer dialogue, machine translation, document analysis, data organization, writing assistance, knowledge learning, and information retrieval, thereby meeting the user's needs for real-time and efficient generation retrieval in different scenarios.

[0040] It should be noted that the above application scenarios or application examples provided in the embodiments of this application are for ease of understanding, and the application of the technical solution in the embodiments of this application is not specifically limited. Furthermore, the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data need to comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entry points for users to choose to authorize or refuse.

[0041] The technical solution of this application and how the technical solution of this application solves the aforementioned technical problems are described in detail below with specific embodiments. Several specific embodiments listed can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0042] Figure 3 shows a flowchart of the question-answering method based on a large language model according to an embodiment of this application. This question-answering method can be applied to the server 120 and executed by the server 120.As shown in Figure 3, the question-answering method may include steps S301, S302, and S303.

[0043] Step S301: Input the target question into the large language model so that the large language model decomposes the target question into multiple sub-questions. Specification 4 / 15 pages 7 CN 121660059 A

[0044] Wherein, the target question is in natural language form, such as the question input by the user. The core structure of the large language model LLM is based on the Transformer architecture (a neural network architecture based on self-attention mechanism), which can understand and generate natural language. In this embodiment of the application, the LLM can understand the target question and decompose the target question into multiple sub-questions. For example, the LLM processes the original target question input by the user and guides the LLM to generate more specific or more general sub-questions through prompt word (also called guide word) engineering.

[0045] For example, as shown in Figure 4, the target question is a long text, that is, a complex query containing multiple sub-questions, such as "Who is the author of this article; what dataset was used", this input sequence will be input into the LLM. After receiving the input sequence, the LLM preprocesses the input sequence through word embedding, converting the text into word vectors of multiple words (such as "author", "writer", "is", "who") that the model can understand, and adds positional encoding to the word vectors according to the order of words in the input sequence; based on the self-attention mechanism, the LLM can pay attention to the information of other words in the input sequence while processing the current word, thereby capturing the contextual dependencies, identifying different parts and relationships in the target question, and gradually generating the corresponding text for each sub-question.

[0046] It should be noted that the network structure of the LLM is not specifically limited in this embodiment of the application, as long as it can decompose a problem into multiple sub-questions.

[0047] Step S302: In response to detecting that the large language model generates a sub-question, the retrieval service module is asynchronously called to retrieve the generated current sub-question and obtain the corresponding retrieval results.

[0048] In this embodiment of the application, the calls to the LLM and the retrieval service module are asynchronous. That is, each time the LLM generates a sub-question, it is asynchronously input into the retrieval service module, which then retrieves the corresponding search results from the database. Therefore, the process of the LLM generating the next sub-question and the process of the retrieval service module retrieving the current sub-question are asynchronous.

[0049] For example, as shown in Figure 4, when the LLM generates the sub-question "Who is the author?", this sub-question is immediately input into the retrieval service module for retrieval, without waiting for the LLM to complete the generation of the next sub-question "What dataset did this article use?".

[0050] The retrieval service module accesses the database to retrieve sub-questions and outputs one or more retrieval segments corresponding to the sub-questions as retrieval results. For example, each sub-question corresponds to a sub-query. The database is usually an LLM plug-in (external) database containing a large amount of knowledge information and can be composed of multiple documents or data items. The retrieval service module will query the database to retrieve the information most relevant to each sub-question. For example, based on the similarity with the vector representation of the sub-question, the top k results are extracted from the retrieved information as the retrieval results for that sub-question.

[0051] The retrieval service module can use the RAG framework. It can be a software module integrated into the application as part of the application, or it can be deployed as an independent service on the server and provide functions through a network interface.

[0052] Step S303: Input multiple retrieval results into the large language model to obtain the question-and-answer results corresponding to the target question.

[0053] Multiple retrieval results can be input into the LLM sequentially or together after all multiple retrieval results have been generated, or they can be input into the LLM asynchronously after each retrieval result is generated. This application embodiment does not limit this. LLM infers from multiple search results and a target question, outputting a question-and-answer result corresponding to the target question.

[0054] In one example, multiple search results can be aggregated as an information-enriched context input to the LLM. Using this contextual information, the LLM constructs and generates a comprehensive and accurate question-and-answer result. In another example, for each search result, the LLM generates a corresponding sub-answer, for example, by combining the search result with a sub-question to generate a sub-answer in natural language form. Then, all sub-answers are aggregated, and their contextual semantic understanding capabilities are used to ensure that all sub-answers are semantically and logically coherent, finally generating a complete question-and-answer result.

[0055] Compared with using the text containing all sub-questions to search the database, the technical solution of this application embodiment, by asynchronously calling the retrieval service module, that is, whenever the large language model generates a sub-question, it directly inputs the generated current sub-question to the retrieval service module for retrieval, which can hide the large language model's calculation time for generating the next sub-question in the retrieval service module's retrieval time for the current sub-question, thereby significantly saving calculation time and improving the efficiency of question and answer result generation.

[0056] In one embodiment, the retrieval service module includes a vector generation submodule and a vector retrieval submodule; in step S302, in response to detecting that the large language model outputs a sub-question, the retrieval service module is asynchronously invoked to retrieve the generated current sub-question and obtain the corresponding retrieval result, including: in response to detecting that the large language model generates a sub-question, the generated current sub-question is input into the vector generation submodule for vector calculation, wherein the generation of the next sub-question by the large language model and the vector calculation of the current sub-question by the vector generation submodule are executed asynchronously; in response to detecting that the vector generation submodule generates a sub-question vector, the generated current sub-question vector is input into the vector retrieval submodule for retrieval and obtain the corresponding retrieval result, wherein the generation of the next sub-question vector by the vector generation submodule and the retrieval of the current sub-question vector by the vector retrieval submodule are executed asynchronously.

[0057] Both the vector generation submodule and the vector retrieval submodule can be software modules, integrated into the application as part of the application, or deployed as independent services on a server, providing functionality through a network interface. Specifically, the vector generation submodule can be deployed as an independent service, for example, on the same server as the LLM, or it can be deployed separately to improve performance. Alternatively, the vector generation submodule can be embedded in the LLM as an embedded model. The vector retrieval submodule can be deployed on a server dedicated to efficient retrieval, tightly integrated with the database, to facilitate the retrieval of large amounts of data. For example, the LLM can be a core large model, and the vector generation submodule and the vector retrieval submodule can be distributed as auxiliary modules. Communication between the LLM, the vector generation submodule, and the vector retrieval submodule can be achieved through network application programming interfaces (APIs) or remote procedure calls (RPCs).

[0058] That is, each time the LLM generates a subproblem, it asynchronously inputs it into the vector generation submodule, thereby hiding the generation time of the LLM for the next subproblem in the vector computation time of the vector generation submodule for the current subproblem. Each time the vector generation submodule generates a sub-question vector, it asynchronously inputs it into the vector retrieval submodule, thus hiding the generation time of the vector generation submodule for the next sub-question vector within the retrieval time of the vector retrieval submodule for the current sub-question vector.

[0059] Based on this, sub-question generation, sub-question vector calculation, and sub-question vector retrieval can be asynchronous, decoupling the LLM, the vector generation submodule, and the vector retrieval submodule, maximizing the parallelization of sub-question decomposition, vector generation, and vector retrieval, thereby saving hardware idle time and improving the generation efficiency of question-answering results.

[0060] In one embodiment, in response to detecting that the large language model generates a sub-problem, the generated current sub-problem input vector generation submodule performs vector calculation, which may include: in response to detecting that the large language model generates a sequence start identifier, the sub-problem sequence input vector generation submodule corresponding to the sequence start identifier performs vector calculation to obtain the sub-problem vector corresponding to the current sub-problem, wherein the sub-problem sequence includes a sequence start identifier, the current sub-problem, and a sequence end identifier, and the vector generation submodule outputs the generated sub-problem vector when the sequence end identifier is detected.

[0061] To facilitate the identification of the LLM's division of each sub-problem, each sub-problem is wrapped into a structured sub-problem sequence through model fine-tuning or prompt engineering. Each sub-problem sequence sequentially contains the sequence start identifier, the sub-problem, and the sequence end identifier of the sub-problem sequence. As shown in Figure 4, " <s>"Who is the author?"< / s> "Forms a sequence of subproblems, where," <s> "Each sequence begins with a marker to indicate that what follows is a subproblem,"< / s> " is the end of sequence marker, page 6 / 15 of the specification, CN 121660059 A, used to indicate that the sub-problem output is complete. Further, when the content generated by LLM is detected to have a sequence start marker, it can be considered that the content generated by LLM has structured data, and the input vector generation submodule of the subsequently generated sub-problems can be synchronously used for vector calculation.

[0062] For example, as shown in Figure 5, the user's target question (query) is completely input into LLM, and LLM starts to calculate tokens. For each token output by LLM, the system will detect whether the token is a special token, that is, detect whether the token is a sequence start marker (such as " <s> If it is a special token, an asynchronous task is started, and the tokens generated after LLM are sequentially sent to the vector generation submodule. Therefore, the subquery and the sequence end marker (such as "") are involved.< / s>All of these will be input to the vector generation submodule. The vector generation submodule asynchronously calculates the vector representation of the subproblem and generates the subproblem vector. If the end-of-sequence marker is detected, the calculated subproblem vector is output.

[0063] For example, the vector generation submodule can adopt a transformer architecture, using a hidden layer of the transformer network as the output, and using a key-value cache (kv-cache) mechanism to accelerate the calculation. Specifically, the main function of the transformer network using the self-attention mechanism is to generate a weighted representation for each input word. These weights are determined by the relationship between all words in the input sequence (i.e., the subproblem sequence) and the current word. For each input word, a corresponding query vector, key vector, and value vector are generated through linear transformation; for each query vector, the dot product of it and all key vectors is calculated to obtain the attention score; the score is scaled and normalized to obtain the attention weight; the attention weight is used to perform a weighted summation of the value vector. Generate output vectors. During the calculation process, the current subproblem can be parsed into multiple words, and the attention of each word can be calculated based on the query vector, key vector, and value vector of each word. The calculated key vector and value vector are cached for use when calculating the attention of the word. Then, the subproblem vector corresponding to the current subproblem is determined based on the attention of each word. Therefore, by using the kv-cache mechanism, the key vector and value vector calculated for each word can be cached during the processing of the subproblem sequence in the vector generation submodule, so that they can be accessed quickly in the attention calculation of subsequent words.

[0064] Based on this, not only can the asynchronous generation of subproblem and subproblem vector be realized, but also the tokens generated one by one can be asynchronously sent to the vector generation submodule by detecting the start identifier of the sequence, thereby further improving the parallelism between LLM and the vector generation submodule. In addition, the detection of the end identifier of the sequence also facilitates the system to identify the completion of a complete subproblem vector.

[0065] In one implementation, in step S301, the target problem is input into a large language model so that the large language model decomposes the target problem into multiple sub-problems, including: inputting the target problem and preset decomposition guide words into the large language model so that the large language model determines the first attention corresponding to the decomposition guide words from the pre-stored attention calculation results, and calculates the second attention corresponding to the target problem, so as to generate multiple sub-problems based on the first attention and the second attention.

[0066] The self-attention mechanism is calculated through three main vectors: query vector, key vector, and value vector. The attention score is determined by calculating the dot product of query and key, and this score is then normalized by the softmax function to weight the corresponding value.

[0067] When guiding the LLM to decompose the target problem into multiple subproblems, a prompt project can be used, that is, a preset decomposition guide word is added before the target problem (usually before, but it can also be after or in other positions) to illustrate and define the output structure of the subproblems, thereby guiding the LLM to output structured subproblems. These decomposition guide words are the same in multiple requests, that is, the decomposition guide word is a shared prefix, the target problem is a unique prefix, and the sequence input into the LLM is a combination of the shared prefix and the target problem. The shared prefix remains unchanged in multiple requests, while the target problem is the part that changes each time. The length of the decomposition guide word is much longer than the length of the target problem itself. If the attention of this part is repeatedly calculated, it will cause a waste of computing resources.

[0068] In the embodiments of this application, the attention calculation results of the shared prefix (decomposition guide word) (Key and specification page 7 / 15 10 CN 121660059 A Value) are pre-calculated and pre-stored, such as pre-stored in the processor's shared memory, so that these calculation results can be directly accessed in each subsequent request. Therefore, each time the LLM needs to perform attention calculation on the input sequence, it can pre-fill the first attention (Key and Value) corresponding to the decomposition guide word, so that the attention of the decomposition guide word can be obtained quickly. Only the second attention corresponding to the target question needs to be calculated, and the sub-question can be output quickly.

[0069] In one embodiment, in step S303, multiple search results are input into the large language model to obtain the question-and-answer result corresponding to the target question, including: inputting multiple search results into the large language model respectively, so that the large language model processes multiple search results in parallel and generates multiple corresponding sub-answers; obtaining the question-and-answer result generated by the large language model based on multiple sub-answers.

[0070] For each search result, the LLM will generate a corresponding sub-answer, for example, combining the search result with the sub-question to generate a sub-answer in natural language form, then summarizing all sub-answers, and using its contextual semantic understanding ability to ensure that all sub-answers are semantically and logically coherent, and finally generating a complete question-and-answer result.

[0071] Wherein, the LLM processes each search result in parallel. For example, LLM achieves parallel generation of multiple sub-answers by launching multiple processes or threads, each processing a search result to generate a corresponding sub-answer. Based on this, the advantages of multi-core CPUs or multi-core GPUs, as well as the parallel processing capabilities provided by the computing platform, can be fully utilized to improve the computational efficiency of answer generation.

[0072] In one embodiment, multiple search results are input into a large language model (LLM) to enable the LLM to process the multiple search results in parallel and generate multiple corresponding sub-answers. This includes: in response to detecting that the search service module generates a search result, the generated current search result is input into the LLM to enable the LLM to generate corresponding sub-answers based on the current search result. The generation of the next search result by the search service module and the processing of the current search result by the LLM are executed asynchronously.

[0073] Whenever the search service module generates a search result for a sub-question, it directly inputs the generated current search result into the LLM for sub-answer reasoning without waiting for all search results. Based on this, by making the sub-question retrieval and sub-answer generation asynchronous, the retrieval time of the search service module can be hidden within the time of the LLM generating sub-answers. Furthermore, by decoupling the search service module and the LLM, the parallelization of retrieval and sub-answer generation can be maximized, thereby significantly saving computation time and improving the efficiency of sub-answer generation.

[0074] In one embodiment, the current search result includes multiple search segments. The generated current search result is input into a large language model (LLM) so that the LLM generates a corresponding sub-answer based on the current search result. This includes: determining the total length of the multiple search segments; in response to the total length not exceeding the length threshold of the LLM, inputting the multiple search segments into the LLM so that the LLM generates a sub-answer corresponding to the current search result based on the multiple search segments; in response to the total length exceeding the length threshold of the LLM, sequentially inputting multiple search information corresponding to the current search result into the LLM so that the LLM generates a corresponding sub-answer based on the multiple search information, wherein each search information includes a search segment and a target question.

[0075] That is, the number and length of the search segments can be considered, and multiple search segments can be selectively sent into the LLM together, and the LLM generates a corresponding sub-answer. Specifically, for multiple search segments in the search result, the system will first count the total length of the multiple search segments. In one case, if the total length is within the context length limit (length threshold) of the LLM, the generated multiple search segments are directly sent into the LLM for calculation. Based on the above introduction to the key-value cache mechanism, it's clear that this direct aggregation method can quickly generate the final question-and-answer results. Another scenario is that if the total length of multiple search segments exceeds the context limit of the LLM, each search segment is combined individually with the target question to form a search query. Then, the multiple search queries corresponding to a single search result are sequentially input into the LLM to obtain the sub-answers corresponding to that search result, thus adapting to situations where the search results are long texts or contain excessive amounts of content.Additionally, the generation of multiple sub-answers can be performed in parallel, as described on page 8 / 15 of the LLM specification (CN 121660059 A). For example, multiple processes can be started in parallel, each process generating a corresponding sub-answer based on a search result (including multiple search segments) or multiple search information corresponding to a search result.

[0076] In one embodiment, the generated current search result is input into a large language model so that the large language model generates a corresponding sub-answer based on the current search result. This includes: inputting the current search result and preset sub-answer guide words into the large language model so that the large language model determines the third attention corresponding to the sub-answer guide words from the pre-stored attention calculation results, calculates the fourth attention corresponding to the current search result, and generates a corresponding sub-answer based on the third attention and the fourth attention.

[0077] When guiding the LLM to generate a sub-answer based on the search result, a prompt project can be used, that is, the search result and the preset sub-answer guide words are combined and input into the LLM. Among them, the sub-answer guide words are used to illustrate the output of the sub-answer and define the output structure of the sub-answer. Therefore, the content of the sub-answer guide word remains unchanged across multiple requests; that is, the sub-answer guide word is a shared prefix, while the search result is a unique prefix. The length of the sub-answer guide word far exceeds the total length of the search segment in the search result. If the attention of this part of the content is repeatedly calculated, it will waste computing resources.

[0078] Therefore, in this embodiment of the application, the attention calculation result of the shared prefix (sub-answer guide word) is pre-calculated and pre-stored, such as pre-stored in the processor's shared memory, so that these calculation results can be directly accessed each time a sub-answer is generated. Based on this, each time the LLM needs to perform attention calculation on the input sequence, it can pre-fill the third attention corresponding to the sub-answer guide word to quickly obtain the attention of the sub-answer guide word. Only the fourth attention corresponding to the search result needs to be calculated to quickly output the sub-answer, thereby avoiding the waste of computing resources caused by the repeated calculation of the attention of the sub-answer guide word.

[0079] The following is an example of a sub-answer guide word:

[0080] “{__BASE_HEADER}#dialogue guide word

[0081] Here is the user's question:

[0082] {question}

[0083] Here is some context to help you answer the question:

[0084] {context}”.

[0085] Wherein, the {context} part is the search text segment.For example, the position of the retrieval segment can be placed at the very end of the input sequence to make the length of the shared prefix as long as possible, thereby maximizing the acceleration effect brought by the attention of the pre-filled shared prefix without affecting the quality of LLM generation.

[0086] In one embodiment, obtaining the question-and-answer results generated by the large language model based on multiple sub-answers may include: inputting multiple sub-answers and preset question-and-answer result guide words into the large language model, so that the large language model determines the fifth attention corresponding to the question-and-answer result guide words from the pre-stored attention calculation results, and calculates multiple sixth attentions corresponding to multiple sub-answers, so as to generate question-and-answer results based on the fifth attention and multiple sixth attentions.

[0087] When guiding the LLM to generate the final question-and-answer results based on multiple sub-answers, a prompt project can be used, that is, multiple sub-answers are used as unique suffixes, and preset question-and-answer result guide words are used as shared prefixes, and input together into the LLM. Among them, the question-and-answer result guide words are used to illustrate the output of question-and-answer results and define the output structure of question-and-answer results. Since the length of the question-and-answer result guide words is much longer than the total length of multiple sub-answers, if the attention of this part is repeatedly calculated, it will cause a waste of computing resources. In this embodiment, the attention calculation results of the shared prefix can be pre-calculated and pre-stored, such as pre-stored in the processor's shared memory. Each time the LLM needs to perform attention calculation on the input sequence, it can pre-fill the sixth attention corresponding to the question-and-answer result guide word, quickly obtain the attention of the question-and-answer result guide word, and only need to calculate multiple sixth attentions corresponding to multiple sub-answers to quickly output the final question-and-answer result, thereby avoiding the waste of computing resources caused by the repeated calculation of the attention of the question-and-answer result guide word.

[0088] Figures 6 and 7 show application example diagrams of the embodiments of this application. Exemplarily, as shown in Figures 6 and 7, the question-and-answer method based on the large language model of this application embodiment may include:

[0089] Step S701: The LLM receives the target question input by the user. As shown in Figure 6, the LLM can calculate the target question, including calculating some irrelevant characters, which may be format or tone characters that are not very relevant to the target question.

[0090] Step S702: Determine whether it is necessary to decompose the target problem into subproblems. If decomposition is required, proceed to step S703; otherwise, proceed to step S707. For example, if the target problem may contain multiple subproblems, it is determined that decomposition is required, and the process proceeds to step S703. If the target problem represents only one problem, it is determined that decomposition is not required, and the process proceeds to step S707.

[0091] Step S703: Extract subproblems, that is, the LLM extracts multiple subproblems from the target problem.This step can be implemented in the same or similar manner as step S301. Optionally, attention can be pre-filled during the extraction of sub-problems, that is, the target problem and the preset decomposition guide words are input into the large language model, so that the large language model determines the first attention corresponding to the decomposition guide words from the pre-stored attention calculation results, and calculates the second attention corresponding to the target problem, so as to generate multiple sub-problems based on the first attention and the second attention. As shown in Figure 6, when generating multiple sub-problems, LLM can format and encapsulate each sub-problem, that is, each sub-problem generates a sub-problem sequence, including a sequence start identifier (such as ""). <s> Sub-problems, sequence end markers (such as "")< / s>

[0092] Step S704: Retrieve from the database. This step can be implemented in the same or similar manner as step S302. Optionally, as shown in FIG6, during the retrieval process, the LLM, vector generation submodule and vector retrieval submodule can be called asynchronously. That is, in response to the detection that the large language model generates a sub-problem, the generated current sub-problem is input into the vector generation submodule for vector calculation. The generation of the next sub-problem by the large language model and the vector calculation of the current sub-problem by the vector generation submodule are executed asynchronously. Optionally, when the content generated by the LLM is detected to have a sequence start identifier, it can be considered that the content generated by the LLM is... With structured data present, vector computation can be synchronously performed in the input vector generation submodule for subsequently generated subproblems. Furthermore, in response to detecting that the vector generation submodule has generated a subproblem vector, the generated current subproblem vector is input into the vector retrieval submodule for retrieval, obtaining the corresponding retrieval result. The generation of the next subproblem vector by the vector generation submodule and the retrieval of the current subproblem vector by the vector retrieval submodule are executed asynchronously. Before generating subproblems, the vector generation submodule can rewrite the target problem and calculate the vector of the rewritten target problem, facilitating subsequent input into the LLM for inference.

[0093] Optionally, as shown in Figure 6, the vector retrieval submodule can employ a hierarchical index structure for retrieval during the retrieval phase. Specifically, at the start of the retrieval, the vector retrieval submodule first determines the primary index of the database. The primary index is a coarse-grained partition, dividing the vector data in the entire database into multiple large blocks. Then, a query rewriting method is used to extract the summary information of the target question, and the summary information is embedded to obtain the query vector. This query vector is used to determine the list of primary indexes of the database. The vector retrieval submodule selects several large blocks most relevant to the query vector from the primary index. For example, it selects the most relevant large blocks by comparing the similarity between the query vector and the centers of each block. The selected blocks determine the initial scope of the database retrieval, thereby narrowing the subsequent search scope. After determining the primary index, the system can prefetch data from these blocks into the cache for fast access. Subsequent queries no longer need to traverse the entire database, thus significantly improving retrieval speed. Then, within the blocks selected by the primary index, secondary or multi-level indexes are used to further narrow the search scope until the most relevant vector data is finally located as the retrieval result.

[0094] Step S705: Concatenate sub-questions, that is, concatenate each search result with its corresponding sub-question and input it into the LLM. Specification 10 / 15 page 13 CN 121660059 A

[0095] Step S706: The LLM generates sub-answers for each search result, resulting in multiple sub-answers. As shown in Figure 6, on the one hand,The LLM generates each sub-answer in parallel. On the other hand, in response to the detection that the retrieval service module generates a retrieval result, the generated current retrieval result is input into the large language model so that the large language model generates the corresponding sub-answer based on the current retrieval result. The generation of the next retrieval result by the retrieval service module and the processing of the current retrieval result by the large language model are executed asynchronously. Optionally, when generating sub-answers, attention can be pre-filled, that is, the current retrieval result and the preset sub-answer guide words are input into the large language model so that the large language model determines the third attention corresponding to the sub-answer guide words from the pre-stored attention calculation results, calculates the fourth attention corresponding to the current retrieval result, and generates the corresponding sub-answer based on the third attention and the fourth attention.

[0096] Step S707: Calculate the question-answer result, such as obtaining the question-answer result generated by the large language model based on multiple sub-answers. Optionally, attention can be pre-filled, that is, multiple sub-answers and preset question-and-answer result guide words are input into the large language model, so that the large language model determines the fifth attention corresponding to the question-and-answer result guide words from the pre-stored attention calculation results, and calculates multiple sixth attentions corresponding to multiple sub-answers, so as to generate question-and-answer results based on the fifth attention and multiple sixth attentions.

[0097] As shown in Figures 6 and 7, it can be seen that by asynchronously calling the LLM, vector generation submodule and vector retrieval submodule, the LLM, vector generation submodule and vector retrieval submodule can be decoupled without changing the LLM network structure. On the one hand, the parallelism of sub-question decomposition, sub-question vector generation, sub-question vector retrieval and sub-answer generation can be maximized. On the other hand, the generation time of sub-questions can be hidden in the generation time of sub-question vectors, the generation time of sub-question vectors can be hidden in the retrieval time of sub-question vectors, and the retrieval time of sub-question vectors can be hidden in the generation time of sub-answers, thereby saving computation time.

[0098] Optionally, during the calculation and generation process, shared prefixes (decomposition guide words, sub-answer guide words, or question-answer result guide words) can be pre-stored, so that when attention calculation is performed on the input sequence each time, the attention calculation results of the shared prefixes can be pre-filled, avoiding the waste of resources caused by the repeated calculation of the attention of long guide word parts.

[0099] Optionally, the generation process of multiple sub-answers in LLM is executed in parallel, so as to make full use of the advantages of multi-core CPUs or multi-core GPUs, as well as the parallel processing capabilities provided by the computing platform, and improve the computational efficiency of answer generation.

[0100] Optionally, the structure of the input sequence of LLM can be optimized by placing the guide word template part at the beginning and distinguishing between shared prefixes and unique suffixes, thereby maximizing the acceleration effect of pre-filling shared prefixes.

[0101] Corresponding to the application scenarios and methods provided in the embodiments of this application, the embodiments of this application also provideA question-answering device based on a large language model includes: a target question input module for inputting a target question into a large language model, so that the large language model decomposes the target question into multiple sub-questions; an asynchronous invocation module for asynchronously invoking a retrieval service module to retrieve the generated current sub-question in response to detecting that the large language model generates a sub-question, thereby obtaining the corresponding retrieval result; and a question-answering result determination module for inputting multiple retrieval results into the large language model to obtain the question-answering result corresponding to the target question.

[0102] In one embodiment, the retrieval service module includes a vector generation submodule and a vector retrieval submodule; the asynchronous call module includes: a first call submodule, used to respond to the detection that the large language model generates a sub-question, input the generated current sub-question into the vector generation submodule for vector calculation, wherein the generation of the next sub-question by the large language model and the vector calculation of the current sub-question by the vector generation submodule are executed asynchronously; a second call submodule, used to respond to the detection that the vector generation submodule generates a sub-question vector, input the generated current sub-question vector into the vector retrieval submodule for retrieval, and obtain the corresponding retrieval result, wherein the generation of the next sub-question vector by the vector generation submodule and the retrieval of the current sub-question vector by the vector retrieval submodule are executed asynchronously.

[0103] In one embodiment, the first calling submodule is configured to: in response to detecting the start identifier of the sequence generated by the large language model, input the sub-question sequence corresponding to the start identifier of the sequence into the vector generation submodule for vector calculation to obtain the sub-question vector corresponding to the current sub-question, wherein the sub-question sequence includes the start identifier of the sequence, the current sub-question, and the end identifier of the sequence, and the vector generation submodule outputs the generated sub-question vector when the end identifier of the sequence is detected.

[0104] In one embodiment, the target question input module is configured to: input the target question and a preset decomposition guide word into the large language model, so that the large language model determines the first attention corresponding to the decomposition guide word from the pre-stored attention calculation results, and calculates the second attention corresponding to the target question, so as to generate the plurality of sub-questions based on the first attention and the second attention.

[0105] In one embodiment, the question-and-answer result determination module includes: a sub-answer generation sub-module, used to input multiple search results into the large language model respectively, so that the large language model processes the multiple search results in parallel and generates multiple corresponding sub-answers; and a question-and-answer result generation module, used to obtain the question-and-answer results generated by the large language model based on the multiple sub-answers.

[0106] In one embodiment, the sub-answer generation submodule is configured to: in response to detecting that the retrieval service module generates a retrieval result, input the generated current retrieval result into the large language model, so that the large language model generates a corresponding sub-answer based on the current retrieval result, wherein the generation of the next retrieval result by the retrieval service module and the processing of the current retrieval result by the large language model are executed asynchronously.

[0107] In one embodiment, the current retrieval result includes multiple retrieval segments. Inputting the generated current retrieval result into the large language model, so that the large language model generates a corresponding sub-answer based on the current retrieval result, includes: determining the total length of the multiple retrieval segments; in response that the total length does not exceed the length threshold of the large language model, inputting the multiple retrieval segments into the large language model, so that the large language model generates a sub-answer corresponding to the current retrieval result based on the multiple retrieval segments.

[0108] In one embodiment, the current search result includes multiple search segments. The generated current search result is input into the large language model so that the large language model generates a corresponding sub-answer based on the current search result. This includes: determining the total length of the multiple search segments; in response to the total length exceeding the length threshold of the large language model, sequentially inputting multiple search information corresponding to the current search result into the large language model so that the large language model generates a corresponding sub-answer based on the multiple search information, wherein each search information includes a search segment and the target question.

[0109] In one embodiment, the sub-answer generation submodule is used to: input the current search result and preset sub-answer guide words into the large language model so that the large language model determines a third attention corresponding to the sub-answer guide word from pre-stored attention calculation results, calculates a fourth attention corresponding to the current search result, and generates a corresponding sub-answer based on the third attention and the fourth attention.

[0110] In one embodiment, the question-and-answer result generation submodule is used to: input the plurality of sub-answers and preset question-and-answer result guidance words into the large language model, so that the large language model determines the fifth attention corresponding to the question-and-answer result guidance words from the pre-stored attention calculation results, and calculates the plurality of sixth attentions corresponding to the plurality of sub-answers, so as to generate the question-and-answer result based on the fifth attention and the plurality of sixth attentions.

[0111] The functions of each module in each device of the embodiments of this application can be referred to the corresponding description in the above method, and have the corresponding beneficial effects as described on pages 12 / 15 of the specification 15 CN 121660059 A, which will not be repeated here.

[0112] FIG8 is a block diagram of an electronic device used to implement the embodiments of this application. As shown in FIG8, the electronic device includes: storageThe electronic device includes a memory 801 and a processor 802. The memory 801 stores a computer program that can run on the processor 802. When the processor 802 executes the computer program, it implements the method described in the above embodiments. The number of memory 801 and processor 802 can be one or more. In a specific implementation, the electronic device may also include a communication interface 803 for communicating with external devices and performing data exchange and transmission.

[0113] In a specific implementation, if the memory 801, processor 802, and communication interface 803 are implemented independently, the memory 801, processor 802, and communication interface 803 can be interconnected through a bus to complete mutual communication. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in Figure 8, but this does not mean that there is only one bus or one type of bus.

[0114] Optionally, in a specific implementation, if the memory 801, processor 802, and communication interface 803 are integrated on a single chip, then the memory 801, processor 802, and communication interface 803 can communicate with each other through an internal interface.

[0115] This application embodiment provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the method provided in this application embodiment.

[0116] This application embodiment provides a computer program product including a computer program, which, when executed by a processor, implements the method provided in this application embodiment.

[0117] This application embodiment also provides a chip including a processor for calling and running instructions stored in a memory, causing a communication device with the chip installed to execute the method provided in this application embodiment.

[0118] This application embodiment also provides a chip including an input interface, an output interface, a processor, and a memory. The input interface, output interface, processor, and memory are connected through an internal connection path. The processor is used to execute code in the memory. When the code is executed, the processor is used to execute the method provided in the application embodiment.

[0119] It should be understood that the processor mentioned above may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), or application-specific integrated circuits (ASICs).Specific Integrated Circuit (ASIC), Field Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. It is worth noting that the processor can be a processor supporting Advanced Reduced Instruction Set Machines (ARM) architecture.

[0120] Further, optionally, the above-mentioned memory may include read-only memory and random access memory. The memory may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. Non-volatile memory may include read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory may include random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available. For example, Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate Synchronous DRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), and Direct Rambus RAM (DR RAM).

[0121] In the above embodiments, it can be implemented wholly or partially by software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented wholly or partially in the form of a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to this application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other similar computer.It is a programmable device. Computer instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another.

[0122] In the description of this specification, references to the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this application. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Furthermore, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples without contradiction.

[0123] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified.

[0124] Any process or method described in the flowcharts or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process. Furthermore, the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the function involved.

[0125] The logic and / or steps described in the flowcharts or otherwise herein, for example, can be considered as a ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, instruction execution systems, apparatuses, or devices (such as computer-based systems, processor-included systems, or other systems that can fetch and execute instructions from and from such instruction execution systems, apparatuses, or devices).

[0126] It should be understood that various parts of this application can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. All or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. This program can be stored in a computer-readable storage medium. When executed, the program includes one or a combination of the steps of the method embodiments.

[0127] Furthermore, the functional units in the various embodiments of this application can be integrated into a single processing module, or...Each unit can exist physically separately, or two or more units can be integrated into one module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. The storage medium can be a read-only memory, a disk, or an optical disk, etc.

[0128] The above description is only an exemplary embodiment of this application, but the protection scope of this application is not limited thereto. Any person skilled in the art can easily conceive of various variations or substitutions within the technical scope recorded in this application, and these should all be covered within the protection scope of this application. Therefore, the protection scope of this application should be determined by the protection scope of the claims. Instruction Manual 15 / 15 Page 18 CN 121660059 A Figure 1 Figure 2 Instruction Manual Drawings 1 / 5 Page 19 CN 121660059 A Figure 3 Figure 4 Instruction Manual Drawings 2 / 5 Page 20 CN 121660059 A Figure 5 Figure 6 Instruction Manual Drawings 3 / 5 Page 21 CN 121660059 A Figure 7 Instruction Manual Drawings 4 / 5 Page 22 CN 121660059 A Figure 8 Instruction Manual Drawings 5 / 5 Page 23 CN 121660059 A Abstract The embodiment of the invention provides a question answering method and device based on a large language model, a medium and a product, and relates to the technical field of natural language processing. The method comprises the steps that a target question is input into the large language model, so that the target question is decomposed into multiple sub-questions through the large language model; in response to the fact that the large language modelis detected to generate a sub-problem, asynchronously calling a retrieval service module to retrieve the generated current sub-problem to obtain a corresponding retrieval result; and inputting the plurality of retrieval results into a large language model to obtain a question and answer result corresponding to the target question. According to the technical scheme provided by the embodiment of the invention, the degree of parallelism in the calculation generation process can be improved and the calculation time can be saved on the premise of not changing the network structure of the large language model.

Claims

1. A question-answering method based on a large language model, comprising: The target problem is input into a large language model, which then decomposes the target problem into multiple sub-problems. In response to the detection that the large language model has generated a sub-question, the retrieval service module is asynchronously invoked to retrieve the generated current sub-question and obtain the corresponding retrieval results; The multiple search results are input into the large language model to obtain the question-and-answer results corresponding to the target question.

2. The question-and-answer method according to claim 1, wherein, The retrieval service module includes a vector generation submodule and a vector retrieval submodule; in response to detecting that the large language model outputs a sub-question, the retrieval service module is asynchronously invoked to retrieve the generated current sub-question, and the corresponding retrieval results are obtained, including: In response to detecting that the large language model generates a sub-problem, the generated current sub-problem is input into the vector generation submodule for vector computation, wherein the generation of the next sub-problem by the large language model and the vector computation of the current sub-problem by the vector generation submodule are executed asynchronously; In response to the detection that the vector generation submodule generates a sub-question vector, the generated current sub-question vector is input into the vector retrieval submodule for retrieval to obtain the corresponding retrieval result. The generation of the next sub-question vector by the vector generation submodule and the retrieval of the current sub-question vector by the vector retrieval submodule are executed asynchronously.

3. The question-and-answer method according to claim 2, wherein, In response to detecting that the large language model generates a sub-question, the generated current sub-question is input into the vector generation submodule for vector computation, including: In response to the detection of the start identifier of the sequence generated by the large language model, the sub-problem sequence corresponding to the start identifier is input into the vector generation submodule for vector calculation to obtain the sub-problem vector corresponding to the current sub-problem. The sub-problem sequence includes the start identifier, the current sub-problem, and the end identifier. When the end identifier is detected, the vector generation submodule outputs the generated sub-problem vector.

4. The method according to any one of claims 1 to 3, wherein, The target problem is input into a large language model, which then decomposes the target problem into multiple sub-problems, including: The target question and the preset decomposition guide words are input into the large language model, so that the large language model determines the first attention corresponding to the decomposition guide words from the pre-stored attention calculation results, and calculates the second attention corresponding to the target question, so as to generate the multiple sub-questions based on the first attention and the second attention.

5. The question-and-answer method according to any one of claims 1 to 3, wherein, Inputting multiple search results into the large language model yields the question-and-answer results corresponding to the target question, including: The multiple search results are respectively input into the large language model so that the large language model processes the multiple search results in parallel and generates multiple corresponding sub-answers; Obtain the question-and-answer results generated by the large language model based on the multiple sub-answers.

6. The question-and-answer method according to claim 5, wherein, Multiple search results are input into the large language model to enable the large language model to process the multiple search results in parallel and generate multiple corresponding sub-answers, including: In response to detecting that the retrieval service module has generated a retrieval result, the generated current retrieval result is input into the large language model so that the large language model generates a corresponding sub-answer based on the current retrieval result. The generation of the next retrieval result by the retrieval service module and the processing of the current retrieval result by the large language model are executed asynchronously.

7. The question-and-answer method according to claim 6, wherein, The current search results include multiple search segments. The generated current search results are input into the large language model so that the large language model generates corresponding sub-answers based on the current search results, including: Determine the total length of the plurality of search segments; In response to the total length not exceeding the length threshold of the large language model, the multiple search segments are input into the large language model so that the large language model generates a sub-answer corresponding to the current search result based on the multiple search segments.

8. The question-and-answer method according to claim 6, wherein, The current search results include multiple search segments. The generated current search results are input into the large language model so that the large language model generates corresponding sub-answers based on the current search results, including: Determine the total length of the plurality of search segments; In response to the total length exceeding the length threshold of the large language model, multiple search information corresponding to the current search result are sequentially input into the large language model so that the large language model generates corresponding sub-answers based on the multiple search information, wherein each search information includes a search passage and the target question.

9. The question-and-answer method according to claim 6, wherein, The generated current search results are input into the large language model, so that the large language model generates corresponding sub-answers based on the current search results, including: The current search result and the preset sub-answer guide words are input into the large language model, so that the large language model determines the third attention corresponding to the sub-answer guide words from the pre-stored attention calculation results, calculates the fourth attention corresponding to the current search result, and generates the corresponding sub-answer based on the third attention and the fourth attention.

10. The question-and-answer method according to claim 5, wherein, Obtaining the question-answering results generated by the large language model based on the multiple sub-answers includes: The multiple sub-answers and the preset question-and-answer result guide words are input into the large language model, so that the large language model determines the fifth attention corresponding to the question-and-answer result guide word from the pre-stored attention calculation results, and calculates multiple sixth attentions corresponding to the multiple sub-answers, so as to generate the question-and-answer result based on the fifth attention and the multiple sixth attentions.

11. An electronic device comprising a memory, a processor, and a computer program stored in the memory, wherein the processor, when executing the computer program, implements the method of any one of claims 1 to 10.

12. A computer-readable storage medium storing a computer program that, when executed by a processor, implements the method of any one of claims 1 to 10.

13. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1 to 10.