Information generation method, electronic device, storage medium, and program product

Abstract

This application provides an information generation method, electronic device, storage medium, and program product. The method includes: in response to receiving target question information, invoking a question rewriting model to rewrite the target question information into at least one sub-question; invoking a generation model to determine the answer generation method corresponding to each sub-question in the at least one sub-question; generating sub-answer information corresponding to each sub-question using the corresponding answer generation method; and obtaining answer information corresponding to the target question information based on each sub-answer information. This application, by rewriting the target question information and using answer generation methods adapted to each sub-question to generate the answer information corresponding to each sub-question, ensures the accuracy and recall of answer generation.

Description

Technical Field

[0001] This application relates to the field of natural language processing technology, and in particular to an information generation method, electronic device, storage medium, and program product. Background Technology

[0002] With the rapid development of natural language processing technology, Large Language Models (LLMs) have been widely applied in various fields such as finance, healthcare, and cloud security. Especially in the cloud security field, there are already security knowledge question-answering products based on LLMs. However, most existing products in this field uniformly use knowledge base retrieval or direct generation to produce answers, resulting in low accuracy and recall rates. Summary of the Invention

[0003] This application provides an information generation method, electronic device, storage medium, and program product to alleviate or solve one or more technical problems existing in the prior art.

[0004] In a first aspect, embodiments of this application provide an information generation method, which includes: in response to receiving target question information, invoking a question rewriting model to rewrite the target question information into at least one sub-question; invoking a generation model to determine the answer generation method corresponding to each of the at least one sub-question; generating sub-answer information corresponding to each of the sub-questions using the corresponding answer generation method; and obtaining answer information corresponding to the target question information based on each sub-answer information.

[0005] Secondly, embodiments of this application provide an electronic device, including a memory, a processor, and a computer program stored in the memory, wherein the processor implements any of the methods of embodiments of this application when executing the computer program.

[0006] Thirdly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the method of any one of the embodiments of this application.

[0007] Fourthly, embodiments of this application provide a computer program product, including a computer program, which, when executed by a processor, implements any of the methods described in the embodiments of this application.

[0008] In this embodiment, when generating answer information corresponding to the target question information, a question rewriting model is invoked to rewrite the target question information into at least one sub-question. For each sub-question, corresponding sub-answer information is generated, and then the answer information corresponding to the target question information is generated based on each sub-answer information. First, by rewriting the target question information into multiple more specific and smaller sub-questions and generating corresponding answers for each, the depth of understanding of the question is improved, thereby increasing the accuracy of the generated answers. This is especially true when dealing with complex questions, where the accuracy of the generated answers can be significantly improved. Furthermore, after rewriting the target question information into at least one sub-question, a generation model is invoked to determine the answer generation scheme corresponding to each sub-question in the rewritten at least one sub-question. The corresponding answer generation method is then used to generate the sub-answer information corresponding to each sub-question in the at least one sub-question. That is, when generating the sub-answer information corresponding to each sub-question, an appropriate answer generation method is used to generate the answer information for each sub-question, improving the fit between the question and the answer generation method, thereby improving the recall and accuracy of the generated answers.

[0009] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application, it can be implemented according to the contents of the specification. In order to make the above and other objects, features and advantages of this application more obvious and understandable, specific embodiments of this application are given below. Attached Figure Description

[0010] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the various drawings denote the same or similar parts or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings depict only some embodiments according to this application and should not be construed as limiting the scope of this application.

[0011] Figure 1 This illustration shows an application scenario diagram of the information generation method provided in the embodiments of this application;

[0012] Figure 2 One of the flowcharts of the information generation method provided in the embodiments of this application is shown;

[0013] Figure 3 A second schematic flowchart of the information generation method provided in an embodiment of this application is shown;

[0014] Figure 4 This document shows a flowchart of the information generation method provided in an embodiment of this application.

[0015] Figure 5 This paper shows a schematic diagram of the module composition of the information generation device provided in an embodiment of this application;

[0016] Figure 6 A block diagram of an electronic device provided in an embodiment of this application is shown. Detailed Implementation

[0017] In the following description, only certain exemplary embodiments are briefly described. As those skilled in the art will recognize, the described embodiments can be modified in various ways without departing from the concept or scope of this application. Therefore, the drawings and description are considered to be exemplary in nature and not restrictive.

[0018] To facilitate understanding of the technical solutions of the embodiments of this application, the relevant technologies of the embodiments of this application are described below. The following relevant technologies are optional solutions and can be combined with the technical solutions of the embodiments of this application in any way, and all of them fall within the protection scope of the embodiments of this application.

[0019] The following terms will be used in the following text:

[0020] Large Language Models (LLMs), or simply large models, are large-scale machine learning models used in Natural Language Processing (NLP) to understand or generate human language. LLMs are trained on very large text datasets using deep learning techniques, enabling them to capture the complex patterns and diversity of natural language.

[0021] The Retrieval Augmented Generation (RAG) framework is a natural language processing model framework that integrates retrieval and generation mechanisms. By introducing an information retrieval module, RAG enhances the knowledge base of LLM. LLM no longer relies solely on the knowledge learned during pre-training but can also dynamically retrieve the latest and most relevant information from external databases.

[0022] With the rapid development of natural language processing technology, large language models have been widely applied in various fields such as finance, healthcare, and cloud security. Particularly in cloud security, large language models have been used for question answering related to security knowledge. However, existing technologies already include security knowledge question answering products based on large language models, providing services such as product consultation, security knowledge question answering, and frequently asked questions. When providing services based on large language models, these products often generate answers for each question directly from the user's input using a uniform method, such as knowledge base retrieval combined with large model reasoning or directly generating answers from the large language model. However, in real-world applications, users may ask a wide range of questions or multiple questions simultaneously in a single question-and-answer session. In such cases, it is often necessary to use multiple tools to arrive at the correct answer. Therefore, relying on a single answer generation method may be insufficient to handle various situations, leading to the inability to provide a correct answer or generating an inaccurate answer. Thus, existing technologies suffer from low accuracy and recall rates in the generated answers.

[0023] In view of this, embodiments of this application provide an information generation method. When generating answer information corresponding to target question information, the target question information is rewritten into at least one sub-question, and sub-answer information for each sub-question is generated. By rewriting the target question information into multiple more specific and smaller sub-questions and generating corresponding answers for each, the depth of understanding of the question is improved, thereby increasing the accuracy of the generated answers. This is especially beneficial when dealing with complex questions, significantly improving the accuracy of the generated answers. Furthermore, after rewriting the target question information into at least one sub-question, a generation model is invoked to determine the answer generation scheme corresponding to each sub-question in the rewritten at least one sub-question. The corresponding answer generation method is then used to generate the sub-answer information corresponding to each sub-question in the at least one sub-question. That is, when generating the sub-answer information corresponding to each sub-question, an appropriate answer generation method is used to generate the answer information for each sub-question, improving the compatibility between the question and the answer generation method, thereby improving the recall and accuracy of the generated answers. Through the above series of measures, the accuracy and recall of answer generation are significantly improved.

[0024] To facilitate understanding of the embodiments of this application, the application scenarios of the information generation method provided in the embodiments of this application will be briefly described first. Figure 1 A schematic diagram illustrating an application scenario of the information generation method provided in this application embodiment is shown. For example... Figure 1As shown, this application scenario includes a server 110 and a client 120. A large language model is deployed on the server 110, and the client 120 communicates with the server 110. The client 120 can be deployed on computing devices such as mobile phones, computers, and tablets. For example, the aforementioned server can be a cloud security center server.

[0025] In one application example, a user enters the target question they wish to consult in a dialog box displayed on the client 120, and then sends this target question information to the server 110 through the client 120. After receiving the target question information from the client 120, the server 110 invokes a question rewriting model to rewrite the target question information into at least one sub-question. For example, if the target question information is "What vulnerabilities exist on my device and what risks exist in the cloud security configuration?", this target question information actually contains two sub-questions. Therefore, it can be rewritten as "Sub-question 1: Please query the vulnerability information of my device; 2: Please query the risk points of my device in the cloud security configuration." After rewriting to obtain at least one sub-question, the server 110 invokes a generation model to determine the answer generation method corresponding to each sub-question, and generates the corresponding sub-answer information for each sub-question using the corresponding answer generation method. Then, based on each sub-answer information, it obtains the answer information corresponding to the target question information and returns this answer information to the client 120. The client 120 displays the answer information returned by the server 110.

[0026] It should be noted that the application scenarios or examples provided in the embodiments of this application are for ease of understanding, and the embodiments of this application do not specifically limit the application of the technical solutions. In addition, the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0027] The technical solution of this application and how it solves the aforementioned technical problems are described in detail below with specific embodiments. The listed specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0028] Figure 2 This illustration shows one of the flowcharts of the information generation method provided in an embodiment of this application. Figure 2 The method shown can be applied to Figure 1 The server 110 in the application scenario shown is, for example Figure 2 As shown, the method may include steps S201, S202, S203 and S204.

[0029] Step S201: In response to receiving the target problem information, the problem rewriting model is invoked to rewrite the target problem information into at least one sub-problem.

[0030] Step S202: Invoke the generative model to determine the answer generation method for each sub-problem in at least one of the above sub-problems.

[0031] Step S203: Generate the sub-answer information corresponding to each of the above sub-questions using the corresponding answer generation method.

[0032] Step S204: Obtain the answer information corresponding to the target question information based on the sub-answer information.

[0033] In this embodiment, the target question information can be sent by the client. In one implementation, when a user needs to ask a question, they can enter the target question information in the client's dialogue interface. After receiving the target question information entered by the user, the client sends the target question information to the server 110. For example, a smart assistant (Agent) can be integrated on the client. The user can enter the target question information through the Agent's question-and-answer interface and send it to the server, and the Agent's question-and-answer interface can display the answer information corresponding to the target question information.

[0034] In one implementation, the aforementioned question rewriting model can be deployed on server 110. This model can be a custom-trained model based on a real-world application scenario, or it can be an open-source model provided within an existing technical framework. After receiving the target question information sent by the client, server 110 invokes the question rewriting model to rewrite the target question information into one or more sub-questions. The number of sub-questions obtained by rewriting the target question information depends on the actual number of questions contained in the target question information. For example, if the target question information is "What vulnerabilities exist on my device and what risks exist in my cloud security configuration?", although this target question information consists of only one sentence, the user has actually asked two questions within it. Therefore, the target question information can be rewritten as "Sub-question 1: Please query the vulnerability information of my device; 2: Please query the risk points of my device in cloud security configuration."

[0035] The problem rewriting model mentioned above can adopt different models, such as a large language model or a lightweight small model, such as a deep learning-based model.

[0036] For example, when the question rewriting model is a large language model, in step S201, calling the question rewriting model to rewrite the target question information into at least one sub-question can be implemented as follows: the target question information is filled into a pre-set prompt word template to obtain corresponding prompt words; these prompt words are then input into the large language model, utilizing its powerful text understanding and generation capabilities to rewrite the target question information into at least one sub-question based on the prompt words. The pre-set prompt word template contains guiding information instructing the large language model to decompose the filled target question information.

[0037] It should be noted that in step S201 above, rewriting the target problem information into at least one sub-problem information can be achieved by decomposing the target problem information into at least one sub-problem, or by further converting the format or language description of each sub-problem after decomposing the target problem information. In practical applications, it is possible to choose whether to further process the decomposed sub-problems according to actual needs. This is merely an illustrative description of some possible implementations of rewriting the target problem information into at least one sub-problem, and is not intended to limit the embodiments of this application.

[0038] After rewriting the target question information into at least one sub-question, this sub-question may contain sub-questions of different question types. For example, some sub-questions may be factual or common-sense questions, some may be explanatory or analytical questions, or some may be query questions. For sub-questions of different question types, different answer generation methods are needed to generate their corresponding sub-answers. For example, for factual questions, the answer may be generated directly by calling a large language model, while for explanatory questions, it may be necessary to rely on a preset knowledge base for retrieval and combine it with a large language model for reasoning to generate the corresponding answer. Based on this consideration, in this embodiment, after rewriting the target question into at least one sub-question, it is also necessary to determine the answer generation method corresponding to each sub-question within the at least one sub-question.

[0039] The aforementioned generative model can be a large language model or a lightweight small model, such as a generative model.

[0040] For example, when the above-mentioned generative model is a large language model, one possible implementation of step S202, which involves calling the generative model to determine the answer generation method corresponding to each sub-question in at least one sub-question, could be: filling at least one sub-question into a pre-set prompt word template to obtain corresponding prompt words, and inputting the prompt words into the large language model so that the large language model determines the answer generation method corresponding to each sub-question in at least one sub-question based on the prompt words. The pre-set prompt word template contains guiding information that instructs the large language model to analyze the filled at least one sub-question to obtain the answer generation method for each sub-question.

[0041] After obtaining the answer generation method corresponding to each sub-problem in at least one sub-problem based on the above step S202, the corresponding answer generation method is used to generate sub-answer information corresponding to each sub-problem in at least one sub-problem. After obtaining the sub-answer information corresponding to each sub-problem, the sub-answers can be concatenated together to obtain the answer information corresponding to the target question information. In one embodiment, the sub-answers can be combined according to the order of the dependencies between the sub-problems, or they can be combined according to the order of the sub-problems in the target question information. When implementing this scheme, the combination method of each sub-answer information can be determined according to actual needs. This is only an example listing several possible implementation methods and is not a limitation on the embodiments of this application.

[0042] The method provided in this application, when generating answer information corresponding to target question information, calls a question rewriting model to rewrite the target question information into at least one sub-question, generates corresponding sub-answer information for each sub-question, and then generates the answer information corresponding to the target question information based on each sub-answer information. First, by rewriting the target question information into multiple more specific and smaller sub-questions and generating corresponding answers for each, it helps to improve the depth of understanding of the question, thereby improving the accuracy of the generated answers, especially when dealing with complex questions, it can significantly improve the accuracy of the generated answers. Furthermore, after rewriting the target question information into at least one sub-question, the generation model is called to determine the answer generation scheme corresponding to each sub-question in the at least one rewritten sub-question, and the corresponding answer generation method is used to generate the sub-answer information corresponding to each sub-question in the at least one sub-question. That is, when generating the sub-answer information corresponding to each sub-question, an appropriate answer generation method is used to generate the answer information for each sub-question, improving the adaptability between the question and the answer generation method, thereby improving the recall and accuracy of the generated answers.

[0043] In some Q&A scenarios, users may engage in multiple rounds of question-and-answer sessions after opening the interface, with each round potentially addressing the same topic. For example, in the first round, the user's target question might be "What vulnerabilities exist on my device?" After receiving the answer, the user might ask a second question: "How do I fix this vulnerability?". When generating the answer to the second round's question, simply considering the user's input in the second round wouldn't reveal which vulnerability "this vulnerability" refers to. Therefore, it's crucial to correlate the information with the previous round's questions to determine the specific vulnerability. This approach allows for more accurate and reliable answers to the second round's question.

[0044] Therefore, the method provided in this application embodiment further includes: obtaining the preceding question and answer information of the target question information; correspondingly, when the above-mentioned question rewriting model is a large language model, the above-mentioned step S201, calling the question rewriting model to rewrite the target question information into at least one sub-question, may include the following steps: combining the target question information and the above-mentioned preceding question and answer information according to the information combination rules of the first preset prompt word template to obtain a first prompt word; calling the large language model to decompose the target question information according to the first prompt word, and performing standardization processing on the decomposed question to obtain at least one sub-question, the standardization processing including format standardization processing and / or description language standardization processing.

[0045] In this embodiment, the acquired preceding question-and-answer information can be question-and-answer information from a target number of rounds prior to the target question information, such as two or three rounds of question-and-answer information between the target question information. The acquired preceding question-and-answer information can also be question-and-answer information within a target time period prior to the target question information, such as question-and-answer information within two minutes prior to the target question information. It should be noted that this is merely an illustrative description of the preceding question-and-answer information for ease of understanding and is not intended to limit the scope of this application.

[0046] The information combination rules of the aforementioned first preset prompt word template can refer to filling in the corresponding information according to the positions indicated by the first preset prompt word template. In one embodiment, combining the target question information and the preceding question and answer information to obtain the first prompt word according to the information combination rules of the first preset prompt word template can be achieved by filling the target question information and the preceding question and answer information into the corresponding positions in the first preset prompt word template according to the instructions of the first preset prompt word template, thereby obtaining the aforementioned first prompt word. Alternatively, in another embodiment, it can also be achieved by extracting key information from the preceding question and answer information as the preceding question and answer information, and filling the target question information and the aforementioned key information into the corresponding positions in the first preset prompt word template according to the instructions of the first preset prompt word template, thereby obtaining the aforementioned first prompt word.

[0047] For example, the aforementioned key information may be information missing from the target question information, or it may be key information related to the target question information. In implementation, key information can be extracted from the preceding question and answer information according to actual needs; this is merely an example listing two possible implementation methods.

[0048] In this embodiment, the first preset prompt template includes instruction information for instructing the large language model to rewrite the target question information. After obtaining the first prompt, the first prompt is input into the large language model. Utilizing the powerful text understanding and generation capabilities of the large language model, the target question information is decomposed based on the first prompt, and the decomposed questions are standardized to obtain at least one sub-question.

[0049] The standardization process described above for the decomposed problems can take several forms. First, it can standardize the format of the problems, such as adjusting the way they are described. Second, it can standardize the descriptive language of the problems; for example, if the decomposed problems have a language description, such as English, then the decomposed problems can be uniformly converted into Chinese. Third, it can also involve both format standardization and language description standardization.

[0050] In one implementation, the description of the problem obtained by the above-mentioned adjustment and decomposition may include one or more of the following adjustments: removing irrelevant characters in the decomposed problem, such as extra spaces, line breaks, etc.; replacing each word in the problem with predefined standard terms with the same meaning to achieve consistency in terminology; and reorganizing and transforming the decomposed problem according to the preset problem framework to ensure the standardization and normalization of the problem format.

[0051] The large language model used in this application embodiment can be a model based on the Transformer architecture. In one implementation, after the first prompt word is input into the large language model, the large language model first performs word segmentation on the target question information, decomposes the target question information into a series of words or tokens, and converts the discrete words or tokens into vector sequence representations through the embedding layer. Then, it is input to the decoder layer, and the vector sequence representations are iterated through multiple layers through the causal self-attention mechanism and feedforward network of the decoder layer, and finally outputs each token of all sub-questions corresponding to the target question information in sequence.

[0052] In this embodiment, when rewriting the target question information into at least one sub-question, the preceding question and answer information corresponding to the target question information is taken into account. This enables the large language model to more accurately identify the core intent of the target question information, thereby improving the accuracy of the rewritten sub-questions. Furthermore, by associating with the preceding question and answer information, the missing information in the target question information can be supplemented, improving the comprehensiveness of the rewritten sub-questions and thus increasing the accuracy and recall of the generated answers. In addition, when rewriting the target question information into at least one sub-question, the questions obtained from the decomposition of the target question information are standardized, enabling more accurate identification of user intent and ensuring the accuracy of the generated answers.

[0053] In one implementation, when the above-mentioned generation model is a large language model, step S202, which involves calling the generation model to determine the answer generation method corresponding to each sub-question in at least one sub-question, may include the following steps: combining the above-mentioned at least one sub-question according to the information combination rules of the second preset prompt word template to obtain a second prompt word; calling the large language model to determine the question type of each sub-question based on the second prompt word, and determining the corresponding answer generation method based on the question type.

[0054] For example, the information combination rule of the second preset prompt word template may refer to filling in the corresponding information according to the positions of each piece of information indicated by the second preset prompt word template. In one implementation, the at least one sub-question can be filled into the corresponding positions in the second preset prompt word template according to the instructions of the second preset prompt word template, and then the second prompt word is input into the large language model. In the embodiments of this application, the second preset prompt word template includes instruction information for instructing the large language model to analyze the filled at least one sub-question. This instruction information is used to instruct the large language model to determine the question type of each sub-question and determine the corresponding answer generation method according to the question type.

[0055] The aforementioned question types represent a classification of questions, which can be derived from classifying questions based on their nature or the thought processes required to answer them. In this application embodiment, different question types require different thinking methods and answering strategies. For example, the aforementioned question types may include factual questions, explanatory questions, reasoning questions, or query questions, etc.

[0056] In this embodiment, the intent of each sub-question can be identified through the understanding capabilities of a large language model, thereby obtaining the aforementioned question types. Then, based on the question type, the corresponding answer generation method is obtained. This answer generation method can be based on RAG retrieval in a preset knowledge base, or it can directly call the large language model to generate the answer, or it can rely on a retrieval tool to generate the answer, etc.

[0057] In this embodiment of the application, considering that different question types require different strategies to generate answers, by analyzing the question type of each sub-question, a suitable answer generation method can be determined for each sub-question, thereby ensuring the accuracy of answer generation.

[0058] In some practical application scenarios, the at least one sub-problem obtained through step S201 may have dependencies on each other. For example, it may be necessary to generate the answer information for another sub-problem based on the answer information for one sub-problem. For instance, if the target problem is "What vulnerabilities exist in my device and how can I fix them?", the sub-problems corresponding to this target problem could be "Sub-problem 1: Query the vulnerability information of my device; Sub-problem 2: How to fix these vulnerabilities." In this case, when generating the sub-answer information for sub-problem 2, it is necessary to know the vulnerability information of the device, that is, it depends on the sub-answer information for sub-problem 1.

[0059] Based on the above considerations, after obtaining at least one sub-problem, it is necessary to determine the order in which the answers to at least one sub-problem are generated. Therefore, in one embodiment, the method provided by this application further includes the following steps: calling a sequence generation model to determine the order in which the answers to at least one sub-problem are generated; correspondingly, step S203 above, generating sub-answer information corresponding to each sub-problem using the corresponding answer generation method, may include the following steps: generating sub-answer information corresponding to each sub-problem using the corresponding answer generation method according to the above answer generation order.

[0060] The answer generation order represents the order in which at least one sub-answer is generated for each sub-question. In one implementation, for sub-questions with dependencies, the answer generation order can be determined based on the dependencies between the sub-questions to ensure the accuracy and recall of the generated answers; for sub-questions without dependencies, to improve answer generation efficiency, the sub-answers corresponding to these sub-questions can be generated in parallel.

[0061] The dependency relationship between at least one subproblem is used to characterize whether generating the sub-answer corresponding to any one subproblem depends on the sub-answer corresponding to other subproblems. If generating the sub-answer corresponding to one subproblem depends on the sub-answer corresponding to other subproblems, then there is a dependency relationship between the two subproblems.

[0062] For at least one sub-problem derived from the target problem information, if all sub-problems are dependent on each other, the above answer generation order only includes a first order representing the serial generation of the answers corresponding to each sub-problem. If none of the sub-problems are dependent on each other, the above answer generation order only includes a second order representing the parallel generation of the answers corresponding to each sub-problem. If the above at least one sub-problem contains both dependent and non-dependent sub-problems, the above answer generation order includes both the first and second orders.

[0063] For example, in one implementation, the target problem information is broken down into sub-problems 1, 2, 3, 4, and 5. Generating the sub-answer corresponding to sub-problem 2 depends on the sub-answer corresponding to sub-problem 1, and generating the sub-answer corresponding to sub-problem 3 depends on the sub-answer corresponding to sub-problem 2. There is no dependency between sub-problems 4 and 5. In this scenario, the order of answer generation could be: Generate the sub-answers corresponding to sub-problems 1, 2, and 3 sequentially according to the order of sub-problems 1, 2, and 3; generate the sub-answers corresponding to sub-problems 4 and 5 in parallel. Furthermore, the generation process of the sub-answers corresponding to sub-problems 4 and 5 can be executed in parallel with the generation process of the answer to any one of the sub-problems in the above sequential order.

[0064] In one implementation, for each sub-question whose answer generation order indicates that the answer needs to be generated serially, the sub-answers corresponding to each sub-question are generated serially according to the order of each sub-question in the answer generation order. For each sub-question whose answer generation order indicates that the answer needs to be generated in parallel, the sub-answers corresponding to each sub-question are generated in parallel. When generating the sub-answer corresponding to any sub-question, the answer generation method corresponding to that sub-question is adopted.

[0065] The aforementioned generative model can be a large language model or a lightweight small model, such as a generative model.

[0066] In this embodiment of the application, when generating the sub-answer information corresponding to each sub-question, the order in which the answers are generated between each sub-question is also considered. The answer information corresponding to each sub-question is generated in accordance with the answer generation method adapted to each sub-question, which further improves the accuracy of the generated answers.

[0067] In one implementation, when the above-mentioned order determination model is a large language model, the above-mentioned call to the order determination model to determine the answer generation order of at least one sub-question may include the following steps: combining the above-mentioned at least one sub-question to obtain a third prompt word according to the information combination rules of the third preset prompt word template; calling the large language model to determine the dependency relationship between at least one sub-question based on the third prompt word, and determining the answer generation order of at least one sub-question based on the dependency relationship.

[0068] For example, the information combination rule of the third preset prompt word template may refer to filling in the corresponding information according to the positions of each piece of information indicated by the third preset prompt word template. In one implementation, the at least one sub-question can be filled into the corresponding positions in the third preset prompt word template according to the instructions of the third preset prompt word template, and then the third prompt word is input into the large language model. In the embodiments of this application, the third preset prompt word template includes instruction information for instructing the large language model to analyze the filled at least one sub-question. This instruction information is used to instruct the large language model to determine the dependencies between the sub-questions and determine the corresponding answer generation method based on the dependencies.

[0069] In this embodiment of the application, by determining the dependency relationship between each sub-question and generating the answer generation order of at least one sub-question based on the dependency relationship, the dependency relationship between each sub-question is taken into account when generating the sub-answer corresponding to each sub-question. This enables the accurate capture of the true intent of each sub-question and the provision of a reasonable answer, ensuring the accuracy and recall of the answer.

[0070] Typically, after rewriting the target problem information into at least one subproblem, the subproblems in the rewritten at least one subproblem may all have dependencies on each other. In this case, it is necessary to determine the order in which the answers to each subproblem are generated based on the dependencies, and then generate the corresponding sub-answers serially according to this order. Alternatively, the rewritten at least one subproblem may not have any dependencies on each other. In this case, the corresponding sub-answers can be generated in parallel. Or, some of the rewritten at least one subproblem may have dependencies on each other, while others may not have dependencies on any other subproblem. In this case, for the subproblems with dependencies, the order in which the answers to each subproblem are generated is determined based on the dependencies, and the corresponding sub-answers are generated serially according to this order. For the subproblems without dependencies, the corresponding sub-answers can be generated in parallel. Therefore, in this embodiment, the answer generation order determined by the above step S202 includes a first sort and / or a second sort. The first sort refers to the sequence of subproblems whose answers need to be generated serially, and the second sort refers to the sequence of subproblems whose answers are generated in parallel.

[0071] Accordingly, generating sub-answer information for each sub-problem in at least one sub-problem according to the above-mentioned answer generation order and using the corresponding answer generation method can include the following process: For the i-th sub-problem in the first sort, add the sub-answer information corresponding to the (i-1)-th sub-problem to the question information of the i-th sub-problem, and generate the answer information corresponding to the i-th sub-problem according to the answer generation method corresponding to the i-th question, where i is a positive integer greater than or equal to 2; For the second sort, generate the sub-answer information corresponding to the sub-problem in parallel using the corresponding answer generation method.

[0072] It should be noted that in the first sorting described above, the sub-problems can be sorted according to the order in which the answers are generated. For example, if the first sorting includes sub-problems 1, 2, and 3, then the sub-answers corresponding to sub-problems 1 are generated first, followed by the sub-answers corresponding to sub-problems 2, and finally the sub-answers corresponding to sub-problems 3. That is, generating the sub-answers corresponding to sub-problems 2 depends on the sub-answers corresponding to sub-problems 1, and generating the sub-answers corresponding to sub-problems 3 may depend on the answer corresponding to sub-problems 2.

[0073] For the i-th sub-question in the first ranking, generating the sub-answer corresponding to the i-th sub-question depends on the sub-answer corresponding to the (i-1)-th sub-question. In one implementation, the sub-answer information corresponding to the (i-1)-th sub-questions can be added to the question information of the i-th sub-question as part of the question information of the i-th sub-question. For example, if the (i-1)-th sub-question is "What vulnerability exists in my device?", and the i-th question is "How to resolve this vulnerability?", assuming the sub-answer corresponding to the (i-1)-th sub-question is "Your device has vulnerability XX", then "vulnerability XX" from the sub-answer information of the (i-1)-th sub-question can be added to the i-th sub-question. That is, the i-th question can be modified to "How to resolve vulnerability XX".

[0074] For example, when adding the sub-answers corresponding to each (i-1)th sub-question to the question information of the ith sub-question, the answer information of the (i-1)th sub-question can be added directly to the ith sub-question, or the missing information in the ith sub-question can be filtered from the sub-answer information of the (i-1)th sub-question and added to the ith sub-question.

[0075] For each subproblem in the second sort, since the sub-answers corresponding to these subproblems do not depend on the answer information of any other subproblems, the sub-answer information corresponding to each subproblem in the second sort can be generated in parallel to improve the efficiency of answer generation.

[0076] The first and second sorting subproblems can be executed in parallel.

[0077] In this embodiment, for sub-problems with dependencies, the sub-answer information of the dependent sub-problem is first generated and used as the question information of the sub-problem dependent on that answer. Then, the answer information of the sub-problem dependent on that sub-answer is generated. In this way, the completeness of the question information of each sub-problem with dependencies is ensured, thereby improving the precision and recall of the generated answer. In addition, for sub-problems without dependencies, the sub-answers corresponding to each sub-problem are generated in parallel, which ensures both the accuracy and efficiency of answer generation.

[0078] In this embodiment, a corresponding answer generation method can be determined for sub-questions of different question types. For example, for explanatory or reasoning sub-questions, generating answer information for such questions may require the use of RAG retrieval and large-scale model reasoning. Therefore, in one implementation, the above-mentioned answer generation method may rely on retrieval from a preset knowledge base to generate the answer. In this case, generating sub-answer information corresponding to any sub-question may include: retrieving knowledge document information matching the sub-question from the preset knowledge base; if knowledge document information is retrieved, combining the sub-question and the aforementioned knowledge document according to the information combination rules of the fourth preset prompt word template to obtain the fourth prompt word; and calling the large-scale language model to generate the sub-answer information corresponding to the sub-question based on the fourth prompt word.

[0079] The aforementioned preset knowledge base can be an external knowledge base, that is, it comes from outside the system, rather than being built-in or pre-programmed inside the system or server.

[0080] For example, knowledge documents can be retrieved from a preset knowledge base using RAG retrieval. This preset knowledge base stores a large amount of knowledge document information, which is converted into document vectors and stored in vector form. In one implementation, the knowledge documents stored in the preset knowledge base may be quite long; to facilitate retrieval, any given knowledge document can be divided into multiple document fragments and stored separately. Therefore, in this embodiment, the knowledge document information retrieved from the preset knowledge base can be document fragments or complete documents.

[0081] In one implementation, when retrieving knowledge document information matching a sub-question from a preset knowledge base, the sub-question information can be converted into a vector form. Then, the question vector corresponding to the sub-question is matched with the vectors of various knowledge documents stored in the preset knowledge base to obtain a document vector matching the question vector. The number of document vectors matching the question vector can be one or more.

[0082] After obtaining the document vector that matches the question vector, it is necessary to convert the document vector into text. For example, algorithms such as autoregressive models, natural language generation methods, and Transformer models can be used to convert the document vector into text.

[0083] When using a large language model to generate answers, it is usually necessary to first construct prompt words and input them into the large language model so that the model can reason according to the prompt words to arrive at the corresponding answer. Therefore, in this embodiment, after obtaining the aforementioned knowledge document information, the knowledge document information and the sub-question can be combined according to the information combination rules of the fourth preset prompt word template to obtain the fourth prompt word.

[0084] For example, the information combination rules of the aforementioned fourth preset prompt word template may refer to the positions of each piece of information within the fourth preset prompt word template. In one implementation, the aforementioned sub-questions and the retrieved knowledge documents are filled into the corresponding positions in the fourth preset prompt word template according to the information combination rules of the fourth preset prompt word template, thereby obtaining the fourth prompt word.

[0085] The fourth preset prompt template also includes guidance information to help the large language model infer and generate the corresponding sub-answer based on the filled sub-question and knowledge document information. After the fourth prompt obtained above is input into the large language model, the powerful understanding and reasoning capabilities of the large language model are used to infer and generate the corresponding sub-answer information based on the fourth prompt.

[0086] It should also be noted that if no knowledge document information matching the sub-question is found in the aforementioned preset knowledge base, the sub-answer information corresponding to the sub-question will be directly generated through the large language model. For example, corresponding prompt words can be generated based on the sub-question. These prompt words are used to guide the large language model to generate the answer information corresponding to the sub-question. The sub-question is then input into the large language model, which leverages its powerful understanding capabilities to generate the answer information corresponding to the sub-question.

[0087] In this embodiment, retrieving a preset knowledge base via RAG can provide additional information sources for the large language model, thereby enhancing its generation capabilities, significantly improving the accuracy of answers, and effectively reducing the error rate generated by the large language model. Furthermore, retrieving a preset knowledge base via RAG can also obtain the latest information, ensuring the timeliness of the information and further guaranteeing the accuracy of the generated answers.

[0088] In some application scenarios, users may need to query information within the system. For example, for a cloud security platform, a user might need to query device vulnerability information or alarm information. For this type of question, it may be necessary to use the cloud security platform's internal retrieval tools to search the internal database to obtain the corresponding answer. Therefore, in one implementation, the above-mentioned answer generation method can rely on retrieval tools to generate answers. In this case, for any sub-question, generating the sub-answer information corresponding to that sub-question may include: if the parameter information of the sub-question is complete, calling the target retrieval tool corresponding to the sub-question, and using the target retrieval tool to query the target database for answer-related information based on the above parameter information; calling a large language model to summarize the answer-related information to generate the sub-answer information corresponding to the sub-question.

[0089] For example, in an application scenario like a cloud security platform, the aforementioned search tools may include vulnerability information search tools and alarm information search tools. Vulnerability information search tools are used to retrieve vulnerability information, while alarm information search tools are used to query alarm information. Typically, cloud security platforms deploy internal databases to store platform-related data. The corresponding internal search tools can be used to retrieve relevant answers from this internal database.

[0090] The parameter information of the sub-problem refers to the necessary information such as specific data, conditions, or details required to answer the sub-problem. Therefore, the completeness of the parameter information of the sub-problem indicates whether the sub-problem contains all the necessary information to answer it. For example, suppose the sub-problem is "Please query vulnerability information." This sub-problem lacks vulnerability identification information. Without vulnerability identification information, the sub-problem cannot be answered; therefore, the parameter information of this sub-problem is incomplete.

[0091] For example, if the sub-problem is related to a vulnerability, the vulnerability retrieval tool is invoked; if the sub-problem is related to an alarm, the alarm retrieval tool is invoked. In one implementation, the target retrieval tool corresponding to the sub-problem can be determined by matching the parameter information of the sub-problem with the tool information of each retrieval tool. The tool information of the retrieval tool can be any one or more of the following: the name information of the retrieval tool, the functional description information of the retrieval tool, and the introductory information of the retrieval tool.

[0092] After identifying the target retrieval tool corresponding to the aforementioned sub-question, the target retrieval tool is invoked to query the answer-related information for the sub-question from the target database. This target database is an internal system database. For example, if this embodiment of the application is applied to a cloud security platform, then the target database is an internal database of the cloud security platform.

[0093] After retrieving the answer-related information corresponding to the sub-question from the target database, to ensure the fluency and conciseness of the answer returned to the user, a large language model can be invoked to summarize the answer information. For example, the sub-question and the retrieved answer-related information can be filled into a corresponding prompt word template to obtain the corresponding prompt words. These prompt words are then input into the large language model, allowing it to summarize the answer-related information according to the prompt words, thereby obtaining the sub-answer information corresponding to the sub-question. The prompt word template contains guiding phrases to instruct the large language model to summarize the answer-related information filled into the prompt word template.

[0094] In this embodiment, by calling the platform's internal retrieval tool to search for answers in the platform's internal database, relevant answers can be retrieved quickly, improving the answer generation speed. Furthermore, after retrieving the answer-related information corresponding to the sub-question, the large language model is used to summarize the answer-related information, which helps to extract key information from the answer-related information and generate more concise and high-quality answers. In this embodiment, by combining the target retrieval tool retrieval with the large language model summary, the efficiency of answer generation is improved, and the accuracy and quality of the generated answers are guaranteed.

[0095] In this embodiment, the completeness of the parameter information of a sub-question can be detected by calling a large language model. Therefore, in one implementation, before generating the sub-answer information corresponding to the sub-question, the method provided in this embodiment may further include: constructing a fifth prompt word according to the sub-question and a fifth preset prompt word template; calling a large language model to determine whether the parameter information of the sub-question is complete based on the fifth prompt word; and determining the missing parameter information of the sub-question when the parameter information of the sub-question is incomplete.

[0096] For example, constructing the fifth prompt word according to the fifth preset prompt word template can be achieved by filling the sub-questions into the corresponding positions of the fifth prompt word template to obtain the fifth prompt word. The fifth prompt word template contains guiding information to instruct the large language model to analyze whether the parameter information of the sub-questions filled into the prompt word template is complete. After obtaining the fifth prompt word, it is input into the large language model so that the model analyzes whether the parameter information of the sub-question is complete according to the fifth prompt word. If the parameter information of the sub-question is complete, the model outputs a conclusion that the parameter information is complete; if the parameter information of the sub-question is incomplete, further analysis is needed to determine which parameter information is missing, and the missing parameter information is output.

[0097] Typically, in situations where answers rely on search tools, the search tool cannot be invoked if the parameter information of the sub-question is incomplete. Therefore, in one implementation, the completeness analysis of parameter information can be performed only on the sub-question in such cases.

[0098] In one implementation, if it is determined that the parameters of the aforementioned sub-problem are incomplete, a corresponding prompt word can be generated based on the sub-problem and its missing parameter information. The prompt word is then input into a large language model, which generates a sub-answer to instruct the user to complete the parameter information. Specifically, when generating the prompt word based on the sub-problem and its missing parameter information, the missing parameter information can be filled into a corresponding prompt word template to obtain the prompt word. This prompt word template contains guiding phrases that instruct the large language model to generate instructions for the user to complete the parameter information based on the sub-problem and its missing parameters.

[0099] In this embodiment, before calling the retrieval tool to generate the corresponding sub-answer, analyzing the completeness of the sub-question's parameter information ensures that all necessary parameters are provided before generating the answer. This reduces the likelihood of inability to call the retrieval tool due to incomplete information. Furthermore, ensuring the completeness of the sub-question's parameter information improves the accuracy of the generated answer. Additionally, this embodiment utilizes the understanding capabilities of a large language model to analyze the completeness of the sub-question's parameter information, enhancing the intelligence and accuracy of the analysis.

[0100] For some common-sense or factual questions, there's no need to rely on internal retrieval tools or external knowledge base searches using RAG; the answer can be generated directly through the large language model. Therefore, in one implementation where the answer generation method uses the large language model, generating sub-answer information for any sub-question can include the following steps: generating a sixth prompt word based on the sub-question; and calling the large language model to generate the sub-answer information based on the sixth prompt word.

[0101] The sixth prompt word is used to guide the large language model to generate the sub-answer information corresponding to the sub-question. In one implementation, a corresponding prompt word template can be pre-set, and the sub-question can be filled into the corresponding position in the prompt word template to obtain the sixth prompt word.

[0102] After obtaining the sixth prompt word, the sixth prompt word is input into the large language model. The large language model's understanding and reasoning capabilities are used to generate the sub-answer information corresponding to the sub-question based on the sixth prompt word.

[0103] In this embodiment of the application, a sixth prompt word specifically for the sub-question is generated, which can guide the large language model to generate the answer corresponding to the sub-question. This can reduce the ambiguity of the large language model when understanding the sub-question, thereby improving the certainty of the generated answer.

[0104] In some application scenarios, the target question information entered by the user may contain sensitive words. In such cases, the corresponding answer can be refused. Therefore, in one implementation, the method provided in this application embodiment further includes: if the above sub-question contains sensitive information, calling a large language model to generate sub-answer information indicating that the sub-question should not be answered.

[0105] For example, when determining the answer generation method for each sub-question in at least one sub-question, it is possible to simultaneously identify whether each sub-question contains sensitive information. For this implementation, a guiding phrase for instructing the large language model to identify whether each sub-question contains sensitive words can be included in the second preset prompt template. Alternatively, in another implementation, after obtaining at least one sub-question corresponding to the target question information through step S201, the large language model can be invoked to identify whether each sub-question contains sensitive information. For sub-questions that do not contain sensitive information, step S202 is executed; for sub-questions that contain sensitive information, the large language model is invoked to generate sub-answer information indicating refusal to answer the sub-question. Still another implementation, after identifying the answer generation method for each sub-question in at least one sub-question through the above steps, the large language model is further invoked to identify whether each sub-question contains sensitive information.

[0106] In one implementation, if a subquestion contains sensitive information, the prompt word can be filled into a pre-defined prompt word template to obtain the corresponding prompt word. This prompt word template contains guiding information to instruct the large language model to generate a statement refusing to answer the subquestion. After obtaining the prompt word, it is input into the large language model, causing the model to generate a statement refusing to answer the subquestion according to the prompt word's guidance. This statement is then returned to the client as the sub-answer information corresponding to the subquestion, along with the sub-answer information corresponding to other subquestions.

[0107] In this embodiment of the application, when a sub-question is identified as containing sensitive information, the sub-question is refused to be answered, which helps to protect user privacy, protects user privacy and platform security, and also improves the compliance of content management.

[0108] To facilitate understanding of the information generation method provided in the embodiments of this application, the solution provided in the embodiments of this application will be described below in conjunction with the corresponding embodiments. Figure 3 This illustrates a second flowchart of the information generation method provided in an embodiment of this application, as shown below. Figure 3As shown, when a user inputs target question information on the client side, the client sends the target question information to the server. Upon receiving the target question information, the server calls a large language model to rewrite the target question information into at least one sub-question, and then uses the large language model to perform intent recognition on each sub-question, determining the answer generation order and corresponding answer generation method for each sub-question. The server then generates sub-answer information for each sub-question according to the aforementioned answer generation order and corresponding answer generation method. If the answer generation method for a sub-question is to use the large language model, then the server uses the large language model to generate the corresponding sub-answer information. If the answer generation method for a sub-question is to use an internal retrieval tool to search for relevant answers in the internal database, the server first calls the large language model to check if the parameter information of the sub-question is complete. If it is complete, the server calls the corresponding internal retrieval tool to search for the relevant answer information in the internal database. After obtaining the relevant answer information, the server further calls the large language model to summarize the answer information to obtain the sub-answer information for that sub-question. If the parameter information of the sub-problem is found to be incomplete after detection, the large language model is invoked again. Based on the sub-problem and the missing parameter information, the large language model generates guiding utterances to help the user complete the parameter information.

[0109] If the answer to a sub-question is generated by retrieving an external database using RAG to enhance the large language model, then RAG is invoked to search the external knowledge base to obtain the relevant knowledge document information. The large language model is then invoked to reason from the retrieved knowledge document to obtain the sub-answer corresponding to the sub-question. If no relevant knowledge document is found in the external knowledge base using RAG, then the large language model is invoked to directly generate the sub-answer information corresponding to the sub-question.

[0110] In one implementation, when calling the large language model to determine the order of answer generation for at least one sub-question and the answer generation method corresponding to each sub-question, it is also possible to detect whether each sub-question contains sensitive information. If sensitive information is detected in a sub-question, the large language model is called to generate a rejection statement indicating refusal to answer the sub-question.

[0111] After obtaining the sub-answer information corresponding to each of the above sub-questions, the sub-answer information is returned to the client for display. The user can then proceed to the next round of dialogue and process the same procedure again.

[0112] In this embodiment, after obtaining the sub-answer information corresponding to each sub-question through the above steps, the sub-answer information can be combined to obtain the answer information corresponding to the target question information and returned to the client. In one implementation, the sub-answer information can be directly concatenated together as the answer information corresponding to the target question information, such as a sub-answer information being a paragraph or a sentence in the answer information. In some implementations, to facilitate user reading of the generated answer information, the above answer information can also be generated according to some pre-set rules. Therefore, in this embodiment, the above step S204, obtaining the answer information corresponding to the target question information based on each sub-answer information, can include the following steps: arranging each sub-answer information according to a preset generation rule to obtain the answer information corresponding to the target question information, wherein the preset generation rule includes the answer generation order and / or the arrangement order of each sub-question in the target question information.

[0113] For example, the above-mentioned preset generation rules may include only the order of answer generation, or only the order of arrangement of each sub-question in the target question information, or both the order of answer generation and the order of arrangement of each sub-question in the target question information.

[0114] For cases where the aforementioned preset generation rules include both the order of answer generation and the order of arrangement of each sub-question in the target question information, the sub-answer information corresponding to each sub-question with a dependency relationship can be sorted according to the order of answer generation, and the sub-answer information corresponding to each sub-question without a dependency relationship can be sorted according to the order of arrangement of each sub-question in the target question information.

[0115] To facilitate understanding, examples will be provided below.

[0116] For example, in one implementation, the target problem information is "What vulnerabilities exist in my device and how can these vulnerabilities be fixed? What resources are needed during the fixing process? What alarm information does my device currently have? What is the configuration information of my device?" Rewriting this target problem information yields the following sub-problems:

[0117] Sub-question 1: What vulnerabilities exist in my device?

[0118] Sub-question 2: How to fix these vulnerabilities?

[0119] Sub-question 3: What resources are needed during the repair process?

[0120] Sub-question 4: What alarm information does my device currently have?

[0121] Sub-question 5: What are the configuration details of my device?

[0122] When generating the sub-answer for subproblem 3, it depends on the sub-answer for subproblem 2. When generating the sub-answer for subproblem 2, it depends on the sub-answer for subproblem 1. Therefore, the order in which the answers for subproblems 1, 2, and 3 are generated is subproblem 1, subproblem 2, and subproblem 3. However, when generating the sub-answers for subproblems 4 and 5, it does not depend on other subproblems. Therefore, the sub-answers for subproblems 4 and 5 are generated in parallel.

[0123] After obtaining the sub-answer information corresponding to each of the above sub-questions, the sub-answer information corresponding to sub-question 1, sub-question 2, and sub-question 3 can be sorted in the order of the answers to sub-question 1, sub-question 2, and sub-question 3. The sub-answers corresponding to sub-question 4 and sub-question 5 can be sorted according to their order in the target question information. In the target question information, sub-question 4 is after sub-question 3, and sub-question 5 is after sub-question 4. Therefore, the sub-answer information corresponding to sub-question 4 can be placed after the sub-answer information corresponding to sub-question 3, and the sub-answer information corresponding to sub-question 5 can be placed after the sub-answer information corresponding to sub-question 4. Therefore, the final answer information corresponding to the target question information can be generated in the order of the sub-answer information corresponding to sub-question 1, sub-question 2, sub-question 3, sub-question 4, and sub-question 5.

[0124] In one implementation, the answer information can be obtained by combining a sub-answer information as a paragraph according to the sorting obtained above. Furthermore, in order to help users identify which paragraph corresponds to which sub-question, the key information in the sub-question or sub-question information can be marked above or in front of the corresponding paragraph.

[0125] For example, continuing with the previous example, one possible way to display the answer information to the user is as follows:

[0126] Subproblem 1:

[0127] XXXXXXXXXXXXXXXXXXXX

[0128] Subproblem 2:

[0129] XXXXXXXXXXXXXXXXXXXX

[0130] Sub-problem 3:

[0131] XXXXXXXXXXXXXXXXXXXX

[0132] Sub-problem 4:

[0133] XXXXXXXXXXXXXXXXXXXX

[0134] Subproblem 5:

[0135] XXXXXXXXXXXXXXXXXXXX

[0136] In this embodiment, the generated answer information of the target question is made clearer and easier to understand by the order in which the answers to at least one sub-question are generated and / or the order in which each sub-question is arranged in the target question information, thus ensuring the quality and readability of the generated answers and making it easier for users to extract information from the answers, thereby improving the user experience.

[0137] To facilitate understanding of the methods provided in the embodiments of this application, the methods provided in the embodiments of this application will be described below in conjunction with corresponding embodiments. Figure 4 The third schematic flowchart of the information generation method provided in this application embodiment is shown, as follows: Figure 4 As shown, the method includes the following steps:

[0138] Step S401: In response to receiving the target question information, obtain the preceding question and answer information of the target question information.

[0139] Step S402: According to the information combination rules of the first preset prompt word template, combine the above target question information and the previous question and answer information to obtain the first prompt word.

[0140] Step S403: Call the large language model to decompose the target question information based on the first prompt word, and standardize the decomposed questions to obtain at least one sub-question.

[0141] Step S404: According to the information combination rules of the target preset prompt word template, combine at least one sub-question to obtain the target prompt word.

[0142] The information combination rules of the aforementioned target preset prompt template can refer to filling in the corresponding information according to the positions indicated by the target preset prompt template. In one embodiment, at least one sub-question can be filled into the corresponding positions in the target preset prompt template according to the instructions of the target preset prompt template to obtain the target prompt. For example, the aforementioned target preset prompt template includes instruction information for instructing the large language model to analyze the filled at least one sub-question. This instruction information is used to instruct the large language model to determine the dependencies between at least one sub-question and to determine the question type of each sub-question, and to generate an answer generation order according to the aforementioned dependencies, and to generate a corresponding answer generation method according to the question type.

[0143] Step S405: Call the large language model to determine the dependency relationship between at least one sub-question and the question type of each sub-question based on the target prompt words, and determine the answer generation order of at least one sub-question based on the above dependency relationship, and determine the corresponding answer generation method based on the question type.

[0144] Step S406: Generate sub-answer information corresponding to each sub-question in at least one sub-question according to the above answer generation order and the corresponding answer generation method.

[0145] Step S407: Arrange the sub-answer information according to the preset generation rules to obtain the answer information corresponding to the target question information. The preset generation rules include the answer generation order and / or the arrangement order of each sub-question in the target question information.

[0146] Step S408: Send the obtained answer information to the client.

[0147] Corresponding to the method provided in the embodiments of this application, and based on the same idea, the embodiments of this application also provide an information generation apparatus. Figure 5 This illustration shows a schematic diagram of the module composition of an information generation apparatus provided in an embodiment of this application. The apparatus can be configured in... Figure 1 In the application scenario shown, the server 110 can execute the embodiments of this application. Figures 2 to 4 The method provided in any of the embodiments shown. Figure 5 As shown, the information generation device includes: a first invocation module 501, used to invoke a question rewriting model to rewrite the target question information into at least one sub-question in response to receiving target question information; a second invocation module 502, used to invoke a generation model to determine the answer generation method corresponding to each of the at least one sub-question; a generation module 503, used to generate sub-answer information corresponding to each sub-question using the corresponding answer generation method; and a determination module 504, used to obtain the answer information corresponding to the target question information based on each sub-answer information.

[0148] In one embodiment, the apparatus provided in this application further includes an acquisition module, used to acquire the preceding question and answer information of the target question information; when the question rewriting model is a large language model, the first invocation module 501 is specifically used to: combine the target question information and the preceding question and answer information according to the information combination rules of the first preset prompt word template to obtain the first prompt word; invoke the large language model to decompose the target question information according to the first prompt word, and perform standardization processing on the decomposed question to obtain the at least one sub-question, wherein the standardization processing includes format standardization processing and / or description language standardization processing.

[0149] In one implementation, when the above-mentioned generation model is a large language model, the second calling module 502 is specifically used to: combine the at least one sub-question to obtain a second prompt word according to the information combination rules of the second preset prompt word template; call the large language model to determine the question type of each sub-question according to the second prompt word, and determine the corresponding answer generation method according to the question type.

[0150] In one embodiment, the apparatus provided in this application further includes: a third calling module, used to call the order determination model to determine the answer generation order of the at least one sub-question; correspondingly, the above-mentioned generation module 503 is specifically used to: generate sub-answer information corresponding to each sub-question according to the answer generation order and using the corresponding answer generation method.

[0151] In one implementation, when the above-mentioned order determination model is a large language model, the third calling module is specifically used to: combine the at least one sub-question to obtain a third prompt word according to the information combination rules of the third preset prompt word template; call the large language model to determine the dependency relationship between the at least one sub-question based on the third prompt word, and determine the answer generation order of the at least one sub-question based on the dependency relationship.

[0152] In one implementation, the answer generation order determined by the second calling module 502 includes a first sort and / or a second sort. The first sort refers to a sequence of sub-questions for which answers need to be generated sequentially, and the second sort refers to a sequence of sub-questions for which answers are generated in parallel. Accordingly, the generation module 503 is specifically used to: for the i-th sub-question in the first sort, add the sub-answer information corresponding to the (i-1)-th sub-question to the question information of the i-th sub-question; and generate the sub-answer information corresponding to the i-th sub-question according to the answer generation method corresponding to the i-th sub-question, where i is a positive integer greater than or equal to 2; and for the second sort, generate the sub-answer information corresponding to the sub-question in parallel using the corresponding answer generation method.

[0153] In one implementation, when the answer generation method relies on retrieval from a preset knowledge base, the generation module 503 is further specifically used to: retrieve knowledge document information matching the sub-question from the preset knowledge base; if the knowledge document information is retrieved, combine the sub-question and the knowledge document according to the information combination rules of the fourth preset prompt word template to obtain a fourth prompt word; and call the large language model to generate sub-answer information corresponding to the sub-question based on the fourth prompt word.

[0154] In one implementation, where the answer generation method relies on a retrieval tool, the generation module 503 is further specifically configured to: if the parameter information of the sub-question is complete, invoke the target retrieval tool corresponding to the sub-question, and use the target retrieval tool to query the answer-related information for answering the sub-question from the target database based on the parameter information; invoke a large language model to summarize the answer-related information to generate sub-answer information corresponding to the sub-question.

[0155] In one embodiment, the generation module 503 is further specifically used to: construct a fifth prompt word according to the fifth preset prompt word template based on the sub-problem; call the large language model to determine whether the parameter information of the sub-problem is complete based on the fifth prompt word, and determine the missing parameter information of the sub-problem when the parameter information of the sub-problem is incomplete.

[0156] In one implementation, where the answer generation method is to generate the answer through a large language model, the generation module 503 is further specifically used to: generate a sixth prompt word based on the sub-question; and call the large language model to generate sub-answer information corresponding to the sub-question based on the sixth prompt word.

[0157] In one implementation, the generation module 503 is further configured to, if the sub-question contains sensitive information, invoke a large language model to generate sub-answer information indicating that the sub-question should not be answered.

[0158] In one embodiment, the determining module 504 is specifically used to: arrange the sub-answer information according to a preset generation rule to obtain the answer information corresponding to the target question information, wherein the preset generation rule includes the answer generation order and / or the arrangement order of each sub-question in the target question information.

[0159] The functions of each module in each device in the embodiments of this application can be found in the corresponding description in the above method, and they have corresponding beneficial effects, which will not be repeated here.

[0160] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative, and the modules described as separate components may or may not be physically separate. The components illustrated as modules may or may not be physical modules, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this application according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0161] Understandable, Figure 5 The division of each module is merely a logical functional division. In actual implementation, the functions of these modules can be integrated into the hardware entity implementation on the server side. For example, the functions of the first calling module 501, the first calling module 502, the generation module 503, and the determination module 504 can be integrated into the processor implementation on the server side.

[0162] Figure 6 This is a block diagram of an electronic device used to implement embodiments of this application. For example... Figure 6 As shown, the electronic device includes a memory 601 and a processor 602. The memory 601 stores a computer program that can run on the processor 602. When the processor 602 executes the computer program, it implements the method described in the above embodiments. The number of memories 601 and processors 602 can be one or more. In a specific implementation, the electronic device may also include a communication interface 603 for communicating with external devices and exchanging data.

[0163] In practical implementation, if the memory 601, processor 602, and communication interface 603 are implemented independently, they can be interconnected via a bus to communicate with each other. This bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 6 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0164] Optionally, in a specific implementation, if the memory 601, processor 602 and communication interface 603 are integrated on a single chip, the memory 601, processor 602 and communication interface 603 can communicate with each other through an internal interface.

[0165] This application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method provided in this application.

[0166] This application provides a computer program product, including a computer program that, when executed by a processor, implements the method provided in this application.

[0167] This application also provides a chip including a processor for calling and executing instructions stored in a memory, causing a communication device with the chip installed to perform the method provided in this application.

[0168] This application also provides a chip, including: an input interface, an output interface, a processor, and a memory. The input interface, output interface, processor, and memory are connected through an internal connection path. The processor is used to execute code in the memory. When the code is executed, the processor is used to execute the method provided in the application embodiment.

[0169] It should be understood that the aforementioned processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. General-purpose processors can be microprocessors or any conventional processor. It is worth noting that the processor can be a processor supporting Advanced Reduced Instruction Set Machines (ARM) architecture.

[0170] Further, optionally, the aforementioned memory may include read-only memory and random access memory. The memory may be volatile memory or non-volatile memory, or may include both. Non-volatile memory may include read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory may include random access memory (RAM), which serves as an external cache. By way of example, but not limitation, many forms of RAM are available. Examples include Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), and Direct Rambus RAM (DR RAM).

[0171] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions according to this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another.

[0172] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples.

[0173] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified.

[0174] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process. Furthermore, the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functionality involved.

[0175] The logic and / or steps described in the flowchart or otherwise herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus or device (such as a computer-based system, a processor-included system or other system that can fetch and execute instructions from, an instruction execution system, apparatus or device).

[0176] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. All or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware, the program being stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiments.

[0177] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. This storage medium can be a read-only memory, a disk, or an optical disk, etc.

[0178] The above description is merely an exemplary embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various variations or substitutions within the technical scope described in this application, and these should all be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An information generation method, characterized in that, The method includes: In response to receiving target problem information, the problem rewriting model is invoked to rewrite the target problem information into at least one sub-problem; Invoke the generative model to determine the answer generation method corresponding to each of the at least one sub-problem; The corresponding answer information for each sub-question is generated using the corresponding answer generation method; The answer information corresponding to the target question information is obtained based on the sub-answer information.

2. The method according to claim 1, characterized in that, The method further includes: obtaining the preceding question and answer information of the target question; When the problem rewriting model is a large language model, the step of invoking the problem rewriting model to rewrite the target problem information into at least one sub-problem includes: According to the information combination rules of the first preset prompt word template, the target question information and the preceding question and answer information are combined to obtain the first prompt word; The large language model is invoked to decompose the target question information based on the first prompt word, and the decomposed questions are standardized to obtain at least one sub-question. The standardization process includes format standardization and / or description language standardization.

3. The method according to claim 1, characterized in that, When the generative model is a large language model, the step of invoking the generative model to determine the answer generation method corresponding to each of the at least one sub-question includes: According to the information combination rules of the second preset prompt word template, the at least one sub-question is combined to obtain the second prompt word; The large language model is invoked to determine the question type of each sub-question based on the second prompt word, and the corresponding answer generation method is determined based on the question type.

4. The method according to any one of claims 1-3, characterized in that, The method further includes: The call order determination model determines the order in which the answers to the at least one sub-question are generated. The step of generating sub-answer information corresponding to each sub-question using the corresponding answer generation method includes: The sub-answer information corresponding to each sub-question is generated according to the order in which the answers are generated and the corresponding answer generation method.

5. The method according to claim 4, characterized in that, In the case that the order determination model is a large language model, the call order determination model determines the order in which the answers to the at least one sub-question are generated, including: According to the information combination rules of the third preset prompt word template, the at least one sub-question is combined to obtain the third prompt word; The large language model is invoked to determine the dependencies between the at least one sub-question based on the third prompt word, and the order in which the answers to the at least one sub-question are generated is determined based on the dependencies.

6. The method according to claim 4, characterized in that, The answer generation order includes a first sorting and / or a second sorting. The first sorting refers to a sequence of sub-questions whose answers are generated sequentially, and the second sorting refers to a sequence of sub-questions whose answers are generated in parallel. Generating sub-answer information corresponding to each sub-question in the at least one sub-question using the corresponding answer generation method according to the answer generation order includes: For the i-th sub-problem in the first sorting, the sub-answer information corresponding to the (i-1)-th sub-problem is added to the problem information of the i-th sub-problem. The sub-answer information corresponding to the i-th sub-problem is generated according to the answer generation method corresponding to the i-th sub-problem, where i is a positive integer greater than or equal to 2. For the second sorting, the corresponding answer generation method is used to generate the sub-answer information corresponding to the sub-question in parallel.

7. The method according to claim 6, characterized in that, When the answer generation method relies on searching a preset knowledge base to generate the answer, for any sub-question, sub-answer information corresponding to the sub-question is generated, including: Retrieve knowledge document information that matches the sub-problem from the preset knowledge base; If the knowledge document information is retrieved, the sub-question and the knowledge document are combined according to the information combination rules of the fourth preset prompt word template to obtain the fourth prompt word; The large language model is invoked to generate sub-answer information corresponding to the sub-question based on the fourth prompt word.

8. The method according to claim 6, characterized in that, When the answer generation method relies on a retrieval tool, for any sub-question, sub-answer information corresponding to the sub-question is generated, including: If the parameter information of the sub-question is complete, the target retrieval tool corresponding to the sub-question is invoked, and the target retrieval tool queries the target database for answer-related information based on the parameter information. The large language model is invoked to summarize the relevant information of the answer in order to generate sub-answer information corresponding to the sub-question.

9. The method according to claim 8, characterized in that, Before generating the sub-answer information corresponding to the sub-question, the process also includes: Construct the fifth prompt word according to the fifth preset prompt word template based on the sub-question; The large language model is invoked to determine whether the parameter information of the sub-problem is complete based on the fifth prompt word, and if the parameter information of the sub-problem is incomplete, the missing parameter information of the sub-problem is determined.

10. The method according to claim 6, characterized in that, When the answer generation method uses a large language model to generate answers, for any sub-question, sub-answer information corresponding to the sub-question is generated, including: Generate a sixth prompt word based on the sub-question; The large language model is invoked to generate sub-answer information corresponding to the sub-question based on the sixth prompt word.

11. The method according to any one of claims 1-3, characterized in that, The step of obtaining the answer information corresponding to the target question information based on each sub-answer information includes: The sub-answer information is arranged according to a preset generation rule to obtain the answer information corresponding to the target question information. The preset generation rule includes the answer generation order and / or the arrangement order of each sub-question in the target question information.

12. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory, wherein the processor, when executing the computer program, implements the method of any one of claims 1 to 11.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method of any one of claims 1 to 11.

14. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the method according to any one of claims 1 to 11.

Images