Answer generation method and device, equipment, storage medium and product

Abstract

The invention discloses an answer generation method and device, equipment, a storage medium and a product, relates to the technical field of natural language processing, and discloses a method for obtaining a question corresponding to a dialogue instruction in response to the dialogue instruction; taking the question as a search word, and recalling a plurality of search results; reasoning is carried out based on the question through the first large language model, answer guidance information is generated, and the answer guidance information indicates an answer generation strategy; and through the second large language model, based on the answer guidance information and the plurality of search results, generating an answer corresponding to the question. The method can ensure the accuracy of the generated answer and improve the answer quality.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of natural language processing, and particularly relates to a question answering generation method and device, equipment, a storage medium and a product. BACKGROUND

[0002] With the rapid development of the Internet and the sharp increase in information, users' demand for quickly and accurately obtaining required information is growing, and therefore a question answering system emerges as the times require. The question answering system is a natural language processing technology, which aims to automatically answer questions in the form of natural language proposed by users. Unlike traditional search engines, the question answering system can directly provide answers to users' questions, rather than only relevant web page links.

[0003] In the related art, the question answering system searches for relevant information in a large number of documents by analyzing users' questions, and attempts to extract the most suitable answer for users' requirements from the information. However, due to the diversity and complexity of information sources, the quality of the generated answers is uneven, and the accuracy of the answers needs to be improved.

[0004] The above content is only used to assist in understanding the technical solutions of the present application, and does not represent the acknowledgement of the above content as prior art. SUMMARY

[0005] The main purpose of the present application is to provide a question answering generation method, device, equipment, storage medium and product, which can ensure the accuracy of the generated answers and improve the quality of the answers.

[0006] To achieve the above purpose, the present application provides a question answering generation method, which comprises:

[0007] In response to a dialogue instruction, a question corresponding to the dialogue instruction is obtained;

[0008] The question is used as a search term to recall a plurality of search results;

[0009] A first large language model is used to infer based on the question to generate answer guidance information, which indicates a strategy for generating an answer;

[0010] A second large language model is used to generate an answer corresponding to the question based on the answer guidance information and the plurality of search results.

[0011] Optionally, the answer guidance information comprises a plurality of inference steps of the question; and the second large language model is used to generate an answer corresponding to the question based on the answer guidance information and the plurality of search results, which comprises:

[0012] The second large language model is used to obtain the result of each reasoning step in sequence based on the plurality of search results according to the order of the plurality of reasoning steps until the result of the last reasoning step is obtained, and the result of the last reasoning step is taken as the answer corresponding to the question.

[0013] Optionally, the answer guidance information further comprises result reference information of each reasoning step.

[0014] In the process of obtaining the result of each reasoning step in sequence by the second large language model, the result of each reasoning step is obtained based on corresponding result reference information.

[0015] Optionally, the method further comprises:

[0016] In the case where the result of the previous reasoning step is multiple, the second large language model is used to perform reasoning based on each result of the previous reasoning step to obtain a plurality of results of the current reasoning step.

[0017] In the case where the result of the last reasoning step is multiple, the second large language model is used to determine the result with the highest matching degree to the question from the plurality of results as the answer corresponding to the question.

[0018] Optionally, the recalling of the plurality of search results by taking the question as a search term comprises:

[0019] Multi-level intent recognition is performed on the search term, and a target intent of the highest level is determined from the plurality of levels of intents identified, wherein the intent of the high level belongs to the sub-intent of the intent of the low level.

[0020] The plurality of search results are recalled based on the target intent.

[0021] Optionally, the recalling of the plurality of search results based on the target intent comprises:

[0022] Based on the target intent, a target recall manner corresponding to the target intent is determined from the correspondence between the intent and the recall manner.

[0023] The plurality of search results are recalled according to the target recall manner based on the target intent.

[0024] Optionally, the recalling of the plurality of search results based on the target intent comprises:

[0025] Based on the target intent, a target sorting manner corresponding to the target intent is determined from the correspondence between the intent and the sorting manner.

[0026] The plurality of search results recalled are ranked according to the target ranking mode.

[0027] Optionally, the multi-level intent recognition on the search term comprises:

[0028] The multi-level intent recognition on the search term is performed by a plurality of intent recognition models, and a highest-level intent is determined from the plurality of levels of intents recognized by each intent recognition model.

[0029] The target intent is selected as the highest-level intent from the plurality of highest-level intents.

[0030] Optionally, the multi-level intent recognition on the search term comprises:

[0031] Based on the search term, intents recognized by each intent recognition model are sequentially obtained in the order of the plurality of intent recognition models until an intent recognized by a last intent recognition model is obtained, and the intent recognized by the last intent recognition model is taken as the highest-level target intent; wherein, except for a first intent recognition model, each of the remaining intent recognition models performs intent recognition based on an intent recognized by a previous intent recognition model, and the obtained intent is a sub-intent of the intent recognized by the previous intent recognition model.

[0032] Optionally, the recalling of the plurality of search results comprises:

[0033] At least one new search term is expanded based on the search term.

[0034] Information search is performed based on each of the search term before expansion and the search term after expansion, and the plurality of search results are obtained.

[0035] Optionally, the expanding of the at least one new search term based on the search term comprises at least one of:

[0036] The search term is split to obtain a plurality of new search terms after splitting.

[0037] The search term is rewritten to obtain at least one new search term after rewriting.

[0038] The search term is split, and at least one new search term is reconstructed based on a splitting result.

[0039] The search term is corrected to obtain a new search term after correction.

[0040] In addition, to achieve the above object, the application further provides an answer generation device, which comprises:

[0041] a question obtaining module configured to obtain, in response to the dialogue instruction, a question corresponding to the dialogue instruction;

[0042] an information recalling module configured to recall a plurality of search results by taking the question as a search word;

[0043] an information generating module configured to generate, by a first large language model, answer guidance information based on the question, the answer guidance information indicating a strategy for generating an answer;

[0044] an answer generating module configured to generate, by a second large language model, an answer corresponding to the question based on the answer guidance information and the plurality of search results.

[0045] Optionally, the answer guidance information comprises a plurality of reasoning steps of the question.

[0046] The answer generating module is configured to obtain, by the second large language model, results of each reasoning step in sequence based on the plurality of search results in an order of the plurality of reasoning steps, until a result of a last reasoning step is obtained, and take the result of the last reasoning step as the answer corresponding to the question; wherein the result of each reasoning step other than the first reasoning step is obtained based on a result of a previous reasoning step.

[0047] Optionally, the answer guidance information further comprises result reference information of each reasoning step.

[0048] In the process of obtaining the result of each reasoning step in sequence by the second large language model, the result of each reasoning step is obtained based on corresponding result reference information.

[0049] Optionally, the answer generating module is configured to, in a case where the result of the previous reasoning step is a plurality of results, obtain, by the second large language model, a plurality of results of the current reasoning step based on each result of the previous reasoning step; and in a case where the result of the last reasoning step is a plurality of results, determine, by the second large language model, a result with a highest matching degree with the question from the plurality of results as the answer corresponding to the question.

[0050] Optionally, the information recalling module comprises:

[0051] an intent recognition unit configured to perform multi-level intent recognition on the search word, and determine a target intent of a highest level from a plurality of levels of recognized intents, wherein an intent of a high level is a sub-intent of an intent of a low level;

[0052] an information recalling unit configured to recall a plurality of search results based on the target intent.

[0053] Optionally, the information retrieval unit is configured to determine, based on the target intent, a target retrieval manner corresponding to the target intent from a correspondence between intents and retrieval manners; and retrieve a plurality of search results based on the target intent and in the target retrieval manner.

[0054] Optionally, the information retrieval unit is configured to determine, based on the target intent, a target sorting manner corresponding to the target intent from a correspondence between intents and sorting manners; and sort the retrieved plurality of search results in the target sorting manner.

[0055] Optionally, the intent recognition unit is configured to perform multi-level intent recognition on the search term by using a plurality of intent recognition models, determine a highest-level intent from a plurality of levels of intents recognized by each of the intent recognition models, and select, from the plurality of highest-level intents, an intent with the highest frequency of occurrence as the target intent.

[0056] Optionally, the intent recognition unit is configured to, based on the search term, sequentially obtain an intent recognized by each of a plurality of intent recognition models in an order of the plurality of intent recognition models until an intent recognized by a last intent recognition model is obtained, and take the intent recognized by the last intent recognition model as a highest-level target intent; wherein each of the intent recognition models other than a first intent recognition model performs intent recognition based on an intent recognized by a previous intent recognition model, and an obtained intent is a sub-intent of the intent recognized by the previous intent recognition model.

[0057] Optionally, the information retrieval module comprises:

[0058] a search term expansion unit configured to expand at least one new search term based on the search term;

[0059] an information search unit configured to perform information search based on each of the search term before expansion and the at least one new search term after expansion to obtain a plurality of search results.

[0060] Optionally, the search term expansion unit is configured to perform at least one of the following operations:

[0061] split the search term to obtain a plurality of new search terms after splitting;

[0062] rewrite the search term to obtain at least one new search term after rewriting;

[0063] split the search term, and reconstruct at least one new search term based on a splitting result;

[0064] correct the search term to obtain a new search term after correction.

[0065] In addition, to achieve the above object, the application further provides an answering generation device, which comprises a memory, a processor and a computer program stored in the memory and executable on the processor, and the computer program is configured to implement the steps of the answering generation method.

[0066] In addition, to achieve the above object, the application further provides a storage medium, which is a computer readable storage medium, and the storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of the answering generation method.

[0067] In addition, to achieve the above object, the application further provides a computer program product, which comprises a computer program, and the computer program is executed by a processor to implement the steps of the answering generation method.

[0068] The one or more technical solutions provided by the application have at least the following technical effects:

[0069] The application provides a solution, in response to a dialogue instruction, a question corresponding to the dialogue instruction is obtained; the question is taken as a search word to recall a plurality of search results. Then, through a first large language model, the question is reasoned based on the question to generate answer guidance information. Since the answer guidance information indicates the strategy for generating an answer, it can clearly indicate how a second large language model generates an answer corresponding to the question, so that the second large language model generates an answer corresponding to the question based on the answer guidance information and the plurality of search results. The second large language model can more accurately distinguish which information in the search results is key based on the answer guidance information, thereby avoiding the interference of redundant and irrelevant information, to ensure that the generated answer is more accurate and improve the quality of the answer. BRIEF DESCRIPTION OF DRAWINGS

[0070] The accompanying drawings, which are incorporated into and form part of the specification, illustrate embodiments consistent with the application and, together with the specification, serve to explain the principles of the application.

[0071] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the accompanying drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, for those skilled in the art, other drawings can also be obtained without creative labor based on these drawings.

[0072] Figure 1 A schematic diagram of an implementation environment of the answering generation method of the application;

[0073] Figure 2 A flowchart provided by the first embodiment of the answering generation method of the application;

[0074] Figure 3 A flowchart of the second embodiment of the answer generation method of this application is provided;

[0075] Figure 4 A flowchart of the third embodiment of the answer generation method of this application is provided;

[0076] Figure 5 A flowchart of the fourth embodiment of the answer generation method of this application is provided;

[0077] Figure 6 This is a schematic diagram of the module structure of the answer generation device according to an embodiment of the present application;

[0078] Figure 7 A schematic diagram of the device structure of the hardware operating environment involved in the answer generation method in the embodiment of the present application.

[0079] The purpose, features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0080] It should be understood that the specific embodiments described herein are merely used to explain the technical solutions of the present application and are not intended to limit the present application.

[0081] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0082] Figure 1 This is a schematic diagram of an implementation environment provided by an embodiment of the present disclosure. Figure 1 The implementation environment includes a terminal 101 and a server 102. The terminal 101 and the server 102 are connected via a wireless or wired network. For example, the terminal 101 is installed with a target application provided by the server 102, and the terminal 101 can implement functions such as data transmission and message exchange through the target application.

[0083] Exemplarily, terminal 101 is a computer, mobile phone, tablet computer, or other terminal. Exemplarily, the target application is a target application in the operating system of terminal 101, or a target application provided by a third party. For example, the target application is a search application, a short video application, a shopping application, a navigation application, etc. Exemplarily, server 102 is the backend server corresponding to the target application. Accordingly, server 102 is a search application server, a short video application server, a shopping application server, a navigation application server, etc.

[0084] In the present application, the terminal 101 is used to obtain a question corresponding to the dialogue instruction in response to the dialogue instruction. The question is sent to the server 102. The server 102 receives the question, recalls a plurality of search results as search words, and then generates answer guidance information by a first large language model based on the question. The answer guidance information indicates the strategy for generating an answer. Then, the second large language model is used to generate an answer corresponding to the question based on the answer guidance information and the plurality of search results. The answer is then sent to the terminal 101. The terminal 101 receives and displays the answer.

[0085] Alternatively, the above-mentioned answer generation process can also be completed by the terminal 101 alone. Alternatively, the terminal 101 can complete it through a target application installed, such as a search application. The embodiments of the present application do not limit this.

[0086] The answer generation method provided by the present application is applicable to various scenarios. For example, in an e-commerce scenario, a user can ask questions about product information he or she needs to know, such as product specifications, usage methods, etc. The terminal can quickly find the product information the user needs by using the method provided by the present application and display the product information to the user in the form of an answer, thereby improving the shopping experience and conversion rate. For example, in an education scenario, the terminal can use the method provided by the present application to answer questions for students and assist students in their learning, becoming an intelligent tutor for students. For example, in a tourism scenario, a user can ask questions about scenic spot information, flight information, accommodation information, etc. The terminal can answer the user by using the method provided by the present application and assist the user in making travel arrangements.

[0087] Figure 2 The flowchart of the first embodiment of the answer generation method of the present application is shown in FIG. 1. Referring to FIG. 1, Figure 2 Taking the terminal as an example, the answer generation method includes the following steps S10-S30:

[0088] Step S10, in response to a dialogue instruction, obtaining a question corresponding to the dialogue instruction.

[0089] Dialogue instruction: a request or command issued by a user to a system in the form of voice, text, etc. It can be a direct question, such as "What is the weather in Beijing today?", or an indirect expression of demand, such as "I want to know the recent movie schedule". Regardless of the specific content in the dialogue instruction, the content expressed in the dialogue instruction is collectively referred to as a question.

[0090] For example, the terminal displays a dialogue interface, and the user inputs any dialogue content through the dialogue interface. The terminal generates a dialogue instruction in response to the dialogue content input by the user, and takes the dialogue content input by the user as a question corresponding to the dialogue instruction.

[0091] Step S20, recalling a plurality of search results as search words.

[0092] The search words are keywords or phrases used to find information in databases, knowledge bases, or the Internet. The present application uses the question as the search word to recall a plurality of search results, such as document fragments, web links, database records, etc., as the information source for the subsequent model to generate answers.

[0093] Optionally, the terminal can recall search results from various information sources based on the search words. For example, the information sources include: pre-built knowledge base, web content, knowledge graph of specific field, community-generated content such as social media platform, forum, etc. User-contributed content, third-party provided data services such as weather forecast service, stock market service, etc.

[0094] Step S30, reasoning based on the question through the first large language model to generate answer guidance information, which indicates the strategy for generating answers.

[0095] Large language model (Large Language Model, LLM) is an artificial intelligence model based on deep learning, aiming to understand and generate natural language. Through training on a large amount of text data, it has mastered the grammar, semantics and even contextual relationships of language, and can complete various tasks from simple question answering, translation to complex creative writing. And can answer questions, write articles and develop reasoning according to the context.

[0096] The large language model in the present application can be any type of model. For example, the large language model is BERT (Bidirectional Encoder Representations from Transformers). Unlike traditional unidirectional language models, BERT can consider left and right context information simultaneously. It can be used to understand the semantics and intent of the question, and to extract answers from given documents or paragraphs. Alternatively, the large language model is a generative pre-training model, such as GPT (a generative pre-training model). Due to its strong contextual understanding and reasoning ability, the GPT model is very suitable for generative question answering tasks, i.e. automatically generating complete answers based on questions.

[0097] The present application involves two large language models, namely the first large language model and the second large language model. The first large language model is used to reason based on the question to generate answer guidance information. The second large language model is used to generate answers corresponding to the question based on the answer guidance information. The first large language model and the second large language model can be the same model or different models, and the present application embodiments do not limit this.

[0098] Exemplarily, the first large language model is inputted with the question, and the first large language model can infer based on the question to generate answer guidance information, so as to output the answer guidance information.

[0099] The answer guidance information indicates a strategy for generating an answer, which includes a series of instructions or suggestions for guiding how to construct an answer to the specific question. The answer guidance information is generated by the first large language model based on the understanding and analysis of the question, aiming to help the second large language model generate an accurate, comprehensive and useful answer. Exemplarily, the answer guidance information can include at least one of the following: an explanation of the core content and intention of the question to ensure that the second large language model accurately grasps the key points of the question; the type of information or data required to answer the question; suggestions for the structure of the answer, such as whether an introduction, main body and conclusion are needed, and what content each part should contain; suggestions for the language style of the answer, such as formal or informal, and how to make the answer clear, concise and easy to understand; sensitive content that needs to be avoided to ensure that the answer of the second large language model complies with regulations; and multiple reasoning steps for the question, i.e., multiple solution steps that need to be taken to solve the question, which can also be understood as the process of solving the question.

[0100] In step S40, the second large language model is used to generate an answer corresponding to the question based on the answer guidance information and the plurality of search results.

[0101] Exemplarily, the answer guidance information and the plurality of search results are inputted into the second large language model, and the second large language model can generate an answer corresponding to the question based on the answer guidance information and the plurality of search results, so as to output the answer. Then, the terminal can display the answer in the dialogue interface for the user to view. Of course, in addition to the answer guidance information and the plurality of search results, other information can also be inputted into the second large language model to assist the second large language model in making the answer. For example, the question, the answer guidance information and the plurality of search results are inputted into the second large language model together.

[0102] The scheme provided in the present application responds to the dialogue instruction, obtains a question corresponding to the dialogue instruction, and recalls a plurality of search results using the question as a search term. Then, the first large language model is used to infer based on the question to generate answer guidance information. Since the answer guidance information indicates a strategy for generating an answer, it can explicitly indicate how the second large language model generates an answer corresponding to the question, so that the second large language model can more accurately identify which information in the search results is key based on the answer guidance information, thereby avoiding the interference of redundant and irrelevant information, to ensure that the generated answer is more accurate and improve the quality of the answer.

[0103] Based on the first embodiment, the second embodiment of the present application is proposed. The same or similar contents as the first embodiment can be referred to the above description, and the subsequent will not be described again. Refer to Figure 3 In the second embodiment, the step S40 includes a step S401:

[0104] In the step S401, the second large language model obtains the result of each reasoning step in sequence based on the plurality of search results according to the order of the plurality of reasoning steps in the answer guidance information until the result of the last reasoning step is obtained, and takes the result of the last reasoning step as the answer corresponding to the question; wherein the result of each reasoning step other than the first reasoning step is obtained based on the result of the previous reasoning step.

[0105] The answer guidance information includes a plurality of reasoning steps of the question, that is, a plurality of solution steps needed to solve the question, which can also be understood as a process of solving the question. For example, what needs to be done in the first step, what needs to be done in the second step, and what needs to be done in the third step.

[0106] For example, the question is "good and affordable restaurants near Chongqing Railway Station". The plurality of reasoning steps in the answer guidance information are "first, determine which restaurants near Chongqing Railway Station; second, determine which of these restaurants have a taste score above a threshold; third, sort the restaurants determined in the second step in order of their per capita consumption from low to high. Correspondingly, the second large language model first determines the result of the first reasoning step, that is, "there are restaurant A, restaurant B, restaurant C, restaurant D and restaurant E near Chongqing Railway Station". Then, based on the result of the first reasoning step, the result of the second reasoning step is determined, that is, "among these restaurants, there are restaurant B, restaurant C, restaurant D and restaurant E with a taste score above a threshold". Then, based on the result of the second reasoning step, the result of the third reasoning step is determined, that is, "the result of sorting the restaurants determined in the second step in order of their per capita consumption from low to high is restaurant D, restaurant C, restaurant B and restaurant E". Then, the second large language model takes the result of the last reasoning step as the answer corresponding to the question, which is "restaurant D, restaurant C, restaurant B and restaurant E".

[0107] For example, the question is "The school originally has 23 sports equipment, 20 of which are lent out, and then two batches of sports equipment are purchased, the first batch contains 10, and the second batch contains 15, how many sports equipment are left in the school now?" The multiple reasoning steps in the answer guidance information are "first, calculate how many sports equipment are left in the school after 20 sports equipment are lent out; second, calculate how many sports equipment are left in the school after the first batch of sports equipment is purchased; third, calculate how many sports equipment are left in the school after the second batch of sports equipment is purchased." Correspondingly, the second large language model determines the result of the first reasoning step first, that is, "after 20 sports equipment are lent out, 3 sports equipment are left in the school." Then, based on the result of the first reasoning step, the result of the second reasoning step is determined, that is, "after purchasing the first batch of sports equipment, 13 sports equipment are left in the school." Then, based on the result of the second reasoning step, the result of the third reasoning step is determined, that is, "after purchasing the second batch of sports equipment, 28 sports equipment are left in the school." Then, the second large language model takes the result of the last reasoning step as the answer corresponding to the question, that is, "28 sports equipment are left in the school."

[0108] In the embodiments of the present application, the second large language model obtains the result of each reasoning step in order according to the multiple reasoning steps in the answer guidance information, and solves the problem step by step, so that the problem solving process becomes more structured and systematic, which helps the model to gradually delve into the core of the problem and improve the accuracy and depth of the answer. Moreover, since each reasoning step after the first reasoning step is based on the result of the previous reasoning step, the logical coherence of the entire reasoning process is ensured, which can ensure that the final answer is more reasonable.

[0109] Optionally, the result of each reasoning step other than the first reasoning step is obtained based on the result of any one or more of the previous reasoning steps. That is, when determining the result of the current reasoning step, it is not limited to being based on the result of the previous reasoning step. For example, the second reasoning step is based on the result of the first reasoning step. The result of the third reasoning step is based on the results of the first reasoning step and the second reasoning step.

[0110] Optionally, the answer guidance information further includes result reference information of each reasoning step. For example, when obtaining the answer guidance information by the first large language model, the question and the multiple search results are input into the first large language model. The first large language model not only generates multiple reasoning steps of the question, but also determines the result of each reasoning step based on the multiple search results, and then outputs the answer guidance information containing multiple reasoning steps of the question and result reference information of each reasoning step. The result reference information of each reasoning step is the result of each reasoning step determined by the first large language model.

[0111] Correspondingly, in the process of sequentially obtaining the result of each reasoning step by the second large language model, the result of each reasoning step is obtained based on the corresponding result reference information. That is, the second large language model can refer to the result reference information given by the first large language model when determining the result of each reasoning step, which not only helps the second large language model to more quickly determine the result of each reasoning step, but also helps the second large language model to check and correct the result of each reasoning step, thereby ensuring the accuracy of the final answer and improving the quality of the answer.

[0112] Optionally, in the case that the result of the previous reasoning step is multiple, the second large language model reasons based on each result of the previous reasoning step to obtain multiple results of the current reasoning step; in the case that the result of the last reasoning step is multiple, the second large language model determines the result with the highest matching degree with the question from the multiple results as the answer corresponding to the question.

[0113] It can be understood that for a certain reasoning step, when the result of the reasoning step is obtained, since the information is uncertain or changing, the reasoning result can be more than one. In this case, the subsequent reasoning step reasons based on each result, thereby obtaining multiple results. By analogy, the last reasoning step will reason to obtain multiple results. Since these multiple results are obtained by reasonable reasoning, each result can be used as an answer to the question. In order to further improve the accuracy of the answer, the second large language model takes the multiple results as candidate answers, and then determines the candidate answer with the highest matching degree with the question as the answer corresponding to the question. For example, the second large language model calculates the similarity between the question and each candidate answer, and takes the candidate answer with the highest similarity as the answer corresponding to the question.

[0114] In the embodiments of the present application, when the second large language model determines the result of any reasoning step, it faces uncertain or changing information, generates multiple results of the reasoning step, and the subsequent reasoning step considers the multiple results of the previous reasoning step. In this way, multiple aspects of the question can be explored, the diversity of the answer can be increased, and the bias that may be caused by a single reasoning path can be avoided. Then, the multiple results of the last reasoning step are compared, and the result with the highest matching degree with the question is selected, which can improve the accuracy of the answer and ensure that the answer is highly relevant to the user's question. This scheme of generating an answer through multi-angle reasoning has higher robustness, and even if it faces a complex question, it can also guarantee the quality of the answer.

[0115] Based on the first embodiment of the present application, the third embodiment of the present application is proposed. The same or similar contents as the first embodiment can be referred to the above introduction, and will not be described in detail hereinafter. Refer to Figure 4In the third embodiment, the step S20 includes a step S201 and a step S202.

[0116] In the step S201, the question is taken as a search word, multi-level intent recognition is performed on the search word, and a highest-level target intent is determined from the multiple levels of intents recognized.

[0117] Intent recognition refers to inferring the specific needs or purposes of a user by analyzing the user's search word. This is not just simple keyword matching, but a deep understanding of the meaning behind the user's query. In many cases, a user's expression reflects not only a single intent, but also a series of complex and interrelated intents. The goal of multi-level intent recognition is to deeply mine the user's multi-level intent to provide more accurate information for the user.

[0118] The number of levels of intent recognition can be set as needed. For example, three levels of intent recognition are performed on the search word, namely primary intent recognition, intermediate intent recognition, and high-level intent recognition. The intent recognized by the intermediate intent recognition is a sub-intent of the intent recognized by the primary intent recognition, and the intent recognized by the high-level intent recognition is a sub-intent of the intent recognized by the intermediate intent recognition. For example, for the search word "write an article about A winning the championship", the result of the primary intent recognition may be "writing", the result of the intermediate intent recognition may be "writing a news article", and the result of the high-level intent recognition may be "writing a news article about athlete A winning the championship". For example, for the search word "best Chinese restaurant in A area", the result of the primary intent recognition may be "finding a restaurant", the result of the intermediate intent recognition may be "finding the best restaurant in A area", and the result of the high-level intent recognition may be "finding the highest-rated Chinese restaurant in A area".

[0119] It can be understood that, compared with the low-level intent, the highest-level intent contains more details and is more accurate in intent expression, and therefore taking the highest-level intent as the target intent and subsequently recalling multiple search results based on the target intent can ensure the accuracy of the recalled search results.

[0120] In the embodiments of the present application, the multi-level intent recognition is performed on the search word, and the highest-level target intent is determined from the multiple levels of intents recognized. The following two implementation modes are included:

[0121] First, the terminal performs multi-level intent recognition on the search word through multiple intent recognition models, respectively determines the highest-level intent from the multiple levels of intents recognized by each intent recognition model, and selects the highest frequency as the target intent from the multiple highest-level intents determined.

[0122] The multiple intent recognition models work in parallel. Each intent recognition model independently performs intent recognition on the search word, and the obtained intent recognition result is independent. For example, five intent recognition models independently perform three-level intent recognition on the search word, and each obtains the highest-level intent. Then the terminal selects the highest frequency from the five highest-level intents as the target intent.

[0123] By working in parallel, the multiple intent recognition models perform multi-level intent recognition on the search word, which can capture the user's intent from different angles and levels, and improve the accuracy of intent recognition. Selecting the highest frequency from multiple highest-level intents as the target intent can avoid the bias or limitations of a single model, so that even if the search word is complex, the reliability of the intent recognition result can be ensured.

[0124] Secondly, the terminal obtains the intent recognized by each intent recognition model in sequence according to the order of the multiple intent recognition models based on the search word, until the intent recognized by the last intent recognition model is obtained, and the intent recognized by the last intent recognition model is taken as the highest-level target intent; wherein, except for the first intent recognition model, each of the remaining intent recognition models performs intent recognition based on the intent recognized by the previous intent recognition model, and the obtained intent is a sub-intent of the intent recognized by the previous intent recognition model.

[0125] The multiple intent recognition models work in sequence. The output of a previous intent recognition model is taken as the input of the next intent recognition model. The next intent recognition model is used to refine the intent recognized by the previous intent recognition model, so that through multiple intent recognition models, the intent can be gradually refined, and the accurate intent output by the last intent recognition model is obtained.

[0126] In the embodiments of the present application, by sequentially using multiple intent recognition models, the user's intent can be gradually and deeply understood, from general to specific, and a hierarchical intent recognition system can be constructed. Each model further refines the intent based on the previous model, which helps to capture more specific and refined user needs. And each model has the opportunity to correct the possible errors of the previous model, thereby improving the accuracy of the final intent recognition.

[0127] It should be noted that the multiple intent recognition models involved in the present application can be the same or different. And each intent recognition model can be any type of model, such as a convolutional neural network, a deep learning model, etc., and the embodiments of the present application do not limit this. In addition, the data used for training each intent recognition model can be self-defined intent classification data. The intent classification data includes a large number of search words and corresponding multiple levels of intents.

[0128] At step S202, a plurality of search results are recalled based on the target intent.

[0129] Optionally, recalling the plurality of search results based on the target intent comprises: determining, based on the target intent, a target recall manner corresponding to the target intent from a correspondence between intents and recall manners; and recalling the plurality of search results according to the target recall manner based on the target intent.

[0130] Recalling refers to a process of extracting data related to a user's search term from a database or index. Different intents may require different recall strategies. Therefore, the correspondence between intents and recall manners can be customized as needed. For example, the intent is a navigation query, and the corresponding recall manner is to recall a list of nearby destinations from geographic location data. The intent is an academic information query, and the corresponding recall manner is to recall relevant content from academic literature or popular science articles. The intent is a product information query, and the corresponding recall manner is to recall relevant product information from the commodity catalog of an e-commerce platform.

[0131] Embodiments of the present application select a matching recall manner according to different types of intents, which can ensure that the returned search results are not only sufficient in quantity but also highly relevant. And it can speed up the search process and reduce unnecessary consumption of computing resources.

[0132] Optionally, recalling the plurality of search results based on the target intent comprises: determining, based on the target intent, a target sorting manner corresponding to the target intent from a correspondence between intents and sorting manners; and sorting the recalled plurality of search results according to the target sorting manner.

[0133] The sorting manner refers to arranging the recalled plurality of search results according to certain standards, with the purpose of letting the user see the information that best meets their needs first. The choice of sorting manner also depends on the understanding of the user's intent. The correspondence between intents and sorting manners can be customized as needed. For example, the intent is a news query, and the sorting manner is time-sensitive sorting, that is, recently published or upcoming events should be displayed first. For example, the intent is an academic information query, and the sorting manner is based on the authority of the website, that is, the content of the website with higher authority is displayed first. For example, the intent is a product information query, and the sorting manner is based on relevance, that is, search results with higher relevance to the intent are displayed first.

[0134] Embodiments of the present application select a matching result sorting manner according to different types of intents, which can ensure that search results that better meet user needs are ranked first. This makes it easier for the subsequent second large language model to filter important information when generating answers based on the sorted plurality of search results, thereby generating answers that better meet user needs.

[0135] Based on the first embodiment of the present application, the fourth embodiment of the present application is proposed. The same or similar contents as the first embodiment can be referred to the foregoing description, and will not be described in detail hereinafter. Refer to Figure 5 In the third embodiment, the step S20 includes a step S203 and a step S204.

[0136] In the step S203, the question is taken as a search word, and at least one new search word is extended based on the search word.

[0137] Optionally, the implementation manner of the step includes at least one of the following:

[0138] Firstly, the search word is split to obtain a plurality of new search words after splitting. Splitting refers to decomposing a complex search word into a plurality of simple and independent search words. For example, the original search word is “A amusement park ticket price”, and the plurality of new search words after splitting include “A amusement park” and “ticket price”.

[0139] Secondly, the search word is rewritten to obtain at least one new search word after rewriting. Rewriting refers to replacing the original search word with a synonym or the like to obtain a search word with the same meaning but different expression. For example, the original search word is “Where is the best Chinese restaurant”, and the new search word after rewriting is “Nearby Chinese restaurant with high score”.

[0140] Thirdly, the search word is split, and at least one new search word is reconstructed based on the splitting result. Reconstruction refers to reorganizing the split search words to obtain a search word with the same meaning but different expression as the original search word. For example, the original search word is “Tomorrow's weather forecast in B area”, and the new search word reconstructed based on the splitting result is “Tomorrow's weather forecast in B area”.

[0141] Fourthly, the search word is corrected to obtain a new search word after correction. Correction refers to correcting spelling errors, grammatical errors and the like in the original search word to obtain a new search word. For example, the original search word is “Name tomorrow rain or not”, and the new search word after correction is “Will it rain tomorrow?”.

[0142] In the step S204, information search is performed based on each search word before and after extension respectively to obtain a plurality of search results.

[0143] In the embodiments of the present application, the new search word is extended by rewriting, splitting, reconstructing, correcting and the like of the search word, and information search is performed based on each search word before and after extension respectively to obtain a plurality of search results. On the one hand, the search system can better understand the user's intention and obtain more relevant search results. On the other hand, the number of recalled search results can be increased to provide more data support for the second large language model to generate answers.

[0144] It should be noted that the above examples are only for understanding the present application and do not constitute a limitation on the answer generation method of the present application. More forms of simple changes based on this technical concept are within the protection scope of the present application.

[0145] The present application also provides an answer generation device, please refer to Figure 6 The answer generation device comprises:

[0146] The question acquisition module 10 is configured to acquire a question corresponding to the dialogue instruction in response to the dialogue instruction.

[0147] The information recall module 20 is configured to recall a plurality of search results by taking the question as a search term.

[0148] The information generation module 30 is configured to generate answer guidance information by a first large language model based on the question, the answer guidance information indicating a strategy for generating an answer.

[0149] The answer generation module 40 is configured to generate an answer corresponding to the question by a second large language model based on the answer guidance information and the plurality of search results.

[0150] Optionally, the answer guidance information comprises a plurality of reasoning steps of the question.

[0151] The answer generation module 40 is configured to sequentially acquire a result of each reasoning step in order of the plurality of reasoning steps by the second large language model based on the plurality of search results until a result of a last reasoning step is acquired, and take the result of the last reasoning step as the answer corresponding to the question; wherein the result of each reasoning step other than the first reasoning step is obtained based on a result of a previous reasoning step.

[0152] Optionally, the answer guidance information further comprises result reference information of each reasoning step.

[0153] In the process of sequentially acquiring the result of each reasoning step by the second large language model, the result of each reasoning step is acquired based on corresponding result reference information.

[0154] Optionally, the answer generation module 40 is configured to, in the case that the result of the previous reasoning step is a plurality of results, obtain a plurality of results of the current reasoning step by the second large language model based on each result of the previous reasoning step; and in the case that the result of the last reasoning step is a plurality of results, determine a result with the highest matching degree to the question from the plurality of results as the answer corresponding to the question by the second large language model.

[0155] Optionally, the information recall module 20 comprises:

[0156] The intent recognition unit is configured to perform multi-level intent recognition on the search term, and determine a target intent of a highest level from the multiple levels of recognized intents, wherein the intent of the high level is a sub-intent of the intent of the low level.

[0157] The information retrieval unit is configured to retrieve a plurality of search results based on the target intent.

[0158] Optionally, the information retrieval unit is configured to determine, based on the target intent, a target retrieval manner corresponding to the target intent from a correspondence between intents and retrieval manners, and retrieve the plurality of search results based on the target retrieval manner.

[0159] Optionally, the information retrieval unit is configured to determine, based on the target intent, a target sorting manner corresponding to the target intent from a correspondence between intents and sorting manners, and sort the retrieved plurality of search results according to the target sorting manner.

[0160] Optionally, the intent recognition unit is configured to perform multi-level intent recognition on the search term by multiple intent recognition models respectively, determine a target intent of a highest level from multiple levels of intents recognized by each of the intent recognition models respectively, and select, from the multiple determined intents of the highest level, an intent with the highest frequency of occurrence as the target intent.

[0161] Optionally, the intent recognition unit is configured to, based on the search term, sequentially obtain an intent recognized by each of the multiple intent recognition models in an order of the multiple intent recognition models until an intent recognized by a last intent recognition model is obtained, and determine the intent recognized by the last intent recognition model as a target intent of a highest level; wherein, except for a first intent recognition model, each of the remaining intent recognition models performs intent recognition based on an intent recognized by a previous intent recognition model, and an obtained intent is a sub-intent of the intent recognized by the previous intent recognition model.

[0162] Optionally, the information retrieval module 20 comprises:

[0163] The search term expansion unit is configured to expand at least one new search term based on the search term.

[0164] The information search unit is configured to perform information search based on each of the search term before expansion and the search term after expansion respectively to obtain a plurality of search results.

[0165] Optionally, the search term expansion unit is configured to perform at least one of the following operations:

[0166] Split the search term to obtain a plurality of new search terms after splitting;

[0167] Rewrite the search term to obtain at least one new search term after rewriting;

[0168] split the search word, reconstruct at least one new search word based on the split result;

[0169] correct the search word, and obtain a new search word after correction.

[0170] The application provides a scheme, in response to a dialogue instruction, obtaining a question corresponding to the dialogue instruction; taking the question as a search word, recalling a plurality of search results. Then, through a first large language model, reasoning based on the question to generate answer guidance information. Since the answer guidance information indicates the strategy for generating an answer, it can clearly indicate how the second large language model generates the answer corresponding to the question, so that through the second large language model, based on the answer guidance information and the plurality of search results, the answer corresponding to the question is generated. The second large language model can more accurately distinguish which information in the search results is key based on the answer guidance information, thereby avoiding the interference of redundant and irrelevant information, to ensure that the generated answer is more accurate and improve the quality of the answer.

[0171] The answer generation device provided by the application adopts the answer generation method in the above embodiments, which can solve the technical problems of uneven quality and low accuracy of the generated answers in the related art. Compared with the prior art, the answer generation device provided by the application has the same beneficial effects as the answer generation method provided by the above embodiments, and other technical features in the answer generation device are the same as the features disclosed in the above embodiments, which will not be repeated here.

[0172] The application provides an answer generation device, which comprises at least one processor and a memory in communication connection with the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the answer generation method in Embodiment I.

[0173] Reference will now be made to the following description Figure 7 which shows a structural schematic diagram of an answer generation device suitable for implementing the embodiments of the application. The answer generation device in the embodiments of the application can include, but is not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (Personal Digital Assistant), PADs (Portable Application Description), PMPs (Portable Media Player), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), and the like, and fixed terminals such as digital TVs, desktop computers, and the like. Figure 7 The answer generation device shown is only an example, and should not impose any limitation on the functions and use range of the embodiments of the application.

[0174] As shown in Figure 7 The answer generation device can include a processing device 1001 (e.g., a central processing unit, a graphics processing unit, etc.) that can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 1002 or programs loaded from a storage device 1003 into a random access memory (RAM) 1004. Various programs and data required for the operation of the answer generation device are also stored in the RAM 1004. The processing device 1001, the ROM 1002, and the RAM 1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; the storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the answer generation device to communicate wirelessly or wired with other devices to exchange data. Although the answer generation device with various systems is shown in the figure, it should be understood that all the systems shown are not required to be implemented or possessed. More or less systems can be alternatively implemented or possessed.

[0175] In particular, according to the embodiments of the present disclosure, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, the embodiments of the present disclosure include a computer program product including a computer program carried on a computer readable medium, the computer program containing program codes for executing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network through the communication device, or installed from the storage device 1003, or installed from the ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the methods of the embodiments of the present disclosure are performed.

[0176] The answer generation device provided by the present disclosure adopts the answer generation method in the above embodiments, and can solve the technical problems of uneven quality and low accuracy of the generated answers in the related art. Compared with the prior art, the answer generation device provided by the present disclosure has the same beneficial effects as the answer generation method provided by the above embodiments, and other technical features in the answer generation device are the same as the features disclosed in the previous embodiment method, which will not be repeated here.

[0177] It should be understood that various aspects of the disclosure can be implemented in hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any appropriate manner in any one or more embodiments or examples.

[0178] The above description is merely illustrative of the application and is not intended to limit the scope of the application. Any variations and modifications that can be made by any person skilled in the art within the spirit and scope of the application are intended to be encompassed by the application. Therefore, the scope of the application should be determined by the appended claims.

[0179] The application provides a computer readable storage medium having stored thereon computer readable program instructions (i.e., a computer program) for performing the answer generation method in the above embodiments.

[0180] The computer readable storage medium provided by the application may, for example, be a U disk, but is not limited to an electric, magnetic, optical, electromagnetic, infrared, or semiconductor system, system, or device, or any combination thereof. More specific examples of the computer readable storage medium can include, but are not limited to, an electric connection with one or more conductive wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any appropriate combination thereof. In the present embodiment, the computer readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer readable storage medium can be transmitted by any appropriate medium, including but not limited to an electric wire, an optical cable, an RF (Radio Frequency), etc., or any appropriate combination thereof.

[0181] The above computer readable storage medium can be included in the answer generation device; or can exist separately and not be assembled into the answer generation device.

[0182] The computer readable storage medium described above carries one or more programs, when the one or more programs are executed by the answer generation device, cause the answer generation device to: in response to the dialogue instruction, acquire a question corresponding to the dialogue instruction; recall a plurality of search results by taking the question as a search word; generate answer guidance information by a first large language model based on the question, the answer guidance information indicating a strategy for generating an answer; and generate an answer corresponding to the question by a second large language model based on the answer guidance information and the plurality of search results.

[0183] Computer program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0184] The flow diagrams and the block diagrams in the drawings are illustrations of architectures, functionalities, and operations of possible implementations of systems, methods, and computer program products according to various embodiments of present application. In this regard, each block in the flow diagrams or block diagrams can represent a module, a procedure, or a part of code, which comprises one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur in a different order than that noted in the figures. For example, two blocks noted in succession can in fact be executed substantially concurrently or in the opposite order, depending on the functionality involved. It is also noted that each block in the block diagrams and / or flow diagrams, and combinations of blocks in the block diagrams and / or flow diagrams, can be implemented by dedicated hardware-based systems that perform the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0185] The modules involved in the embodiments of the present application can be implemented in a software manner or in a hardware manner. In some cases, the name of the module does not constitute a limitation on the module itself.

[0186] The readable storage medium provided in the application is a computer readable storage medium, and the computer readable storage medium stores computer readable program instructions (that is, a computer program) for executing the above-mentioned answer generation method, and can solve the technical problem that the quality of the generated answers is uneven and the accuracy of the answers is low in the related art. Compared with the prior art, the computer readable storage medium provided in the application has the same beneficial effects as the answer generation method provided in the above-mentioned embodiments, and will not be described here.

[0187] The application further provides a computer program product comprising a computer program, which, when executed by a processor, implements the steps of the answer generation method as described above.

[0188] The computer program product provided in the application can solve the technical problem that the quality of the generated answers is uneven and the accuracy of the answers is low in the related art. Compared with the prior art, the computer program product provided in the application has the same beneficial effects as the answer generation method provided in the above-mentioned embodiments, and will not be described here.

[0189] The above-mentioned is only part of the embodiments of the application, and does not limit the patent scope of the application, and any equivalent structural transformation, direct / indirect application in other related technical fields made by using the content of the application specification and drawings under the technical concept of the application are included in the patent protection scope of the application.

Claims

1. A method for generating an answer, characterized in that: The method comprises: In response to a dialogue instruction, obtaining a question corresponding to the dialogue instruction; Using the question as a search term, multiple search results are retrieved; generating answer guidance information by reasoning based on the question using the first language model, wherein the answer guidance information indicates a strategy for generating an answer; An answer corresponding to the question is generated based on the answer guidance information and the multiple search results through a second language model.

2. The method according to claim 1, wherein The answer guidance information includes multiple reasoning steps for the question; Generating an answer corresponding to the question based on the answer guidance information and the plurality of search results using the second language model includes: Using the second largest language model, based on the multiple search results, and in the order of the multiple reasoning steps, the results of each reasoning step are obtained in sequence until the result of the last reasoning step is obtained, and the result of the last reasoning step is used as the answer to the question; except for the first reasoning step, the results of the remaining reasoning steps are obtained based on the results of the previous reasoning step.

3. The method according to claim 2, wherein The answer guidance information also includes reference information of the results of each reasoning step; In the process of sequentially obtaining the result of each reasoning step through the second largest language model, the result of each reasoning step is obtained based on the corresponding result reference information.

4. The method according to claim 2, wherein The method further comprises: When there are multiple results in the previous reasoning step, the second largest language model is used to perform reasoning based on each result of the previous reasoning step to obtain multiple results of the current reasoning step; When there are multiple results in the last reasoning step, the second largest language model is used to determine the result with the highest matching degree with the question among the multiple results as the answer corresponding to the question.

5. The method according to claim 1, wherein The question is used as a search term to retrieve multiple search results, including: Performing multi-level intent recognition on the search term, and determining a highest-level target intent from the multiple levels of recognized intents, wherein the high-level intent is a sub-intent of the lower-level intents; Based on the target intent, a plurality of search results are retrieved.

6. The method according to claim 5, wherein Based on the target intent, multiple search results are recalled, including: Based on the target intention, determining a target recall method corresponding to the target intention from a correspondence between intention and recall method; Based on the target intent, multiple search results are recalled according to the target recall method.

7. An answer generation device, characterized in that: The device comprises: A question acquisition module, configured to acquire, in response to a dialogue instruction, a question corresponding to the dialogue instruction; An information recall module, configured to use the question as a search term and recall multiple search results; an information generation module, configured to generate answer guidance information by reasoning based on the question using the first language model, wherein the answer guidance information indicates a strategy for generating an answer; The answer generation module is used to generate an answer corresponding to the question based on the answer guidance information and the multiple search results through a second language model.

8. An answer generation device, characterized in that The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the answer generation method according to any one of claims 1 to 6.

9. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the answer generation method according to any one of claims 1 to 6 are implemented.

10. A computer program product, characterized in that The computer program product comprises a computer program, which implements the steps of the answer generation method according to any one of claims 1 to 6 when executed by a processor.

Images