Method, device and storage medium for processing user questions and answers

Abstract

The present application relates to the field of artificial intelligence technology, and in particular, to a method, device and storage medium for processing user questions and answers. The method includes: obtaining a user's query question and a domain interest table corresponding to the user; determining a target domain for the query question according to the domain interest table, and obtaining a prompt word template corresponding to the target domain; inputting the prompt word template into the first large language model to construct a question rewriting large language model corresponding to the target domain; inputting the query question into the question rewriting large language model to obtain a question cluster after the query question is rewritten; inputting each sub-question in the question cluster into the second large language model in order from low to high levels, to obtain the corresponding reply to each sub-question in the question cluster in turn; determining the final reply to the query question based on the reply to the highest-level sub-question in the question cluster, so that the large language model can better understand and process the query question, obtain a more accurate reply, and improve user satisfaction.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and specifically to a method, device and storage medium for processing user questions and answers. Background Art

[0002] Natural language processing technology is widely used in search engines, intelligent question-answering systems, machine translation and other fields. It mainly refers to the conversion or transformation of user query statements (also known as query rewriting) to improve the accuracy and efficiency of user queries or meet user-specific query requirements. With the development of natural language processing technology, intelligent question-answering systems have emerged, and intelligent question-answering related technologies have been applied to many industries and fields.

[0003] In existing question-answering systems, when users ask difficult-to-understand or highly complex questions, the questions are generally rewritten. Query rewriting methods generally include synonym replacement, query expansion, sentence adjustment, spelling correction, query simplification, etc., and then the intelligent question-answering system responds to the questions after the query rewrite. However, when there is a large gap between the user's question and the question after the query rewrite, the existing rewriting method cannot accurately obtain the question that the user really needs, or when the user's question is more complex, it cannot split and rewrite the user's essential needs, resulting in the intelligent question-answering system's response being inaccurate and unable to meet the user's needs well. It often gives irrelevant answers, affecting the user's experience. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a method, device and storage medium for processing user questions and answers, so as to solve the problem of inaccurate replies and reduced user experience in the prior art.

[0005] In order to achieve the above-mentioned purpose, the first aspect of the present application provides a method for processing user questions and answers, the method comprising:

[0006] Obtaining a user's query question and a corresponding domain interest table of the user, wherein the domain interest table includes the user's interest level in at least two knowledge fields;

[0007] Determine the target domain for the query question according to the domain interest table, and obtain a prompt word template corresponding to the target domain, where the prompt word template includes domain knowledge corresponding to the target domain;

[0008] Inputting the prompt word template into the first large language model to construct a question rewriting large language model corresponding to the target domain;

[0009] Inputting the query question into the question rewriting large language model to obtain a question cluster after the query question is rewritten, wherein the question cluster includes sub-questions at multiple levels, and the answer to a sub-question at any level depends on the answer to a sub-question at least one level lower than the any level;

[0010] Input each sub-question in the question cluster into the second largest language model in order from low to high level, so as to obtain the corresponding answer of each sub-question in the question cluster in turn, wherein when a sub-question other than the lowest level is input into the second largest language model, the answer of the lower-level sub-question on which it depends is carried;

[0011] The final response to the query question is determined based on the responses to the highest-level sub-questions in the question cluster.

[0012] In an embodiment of the present application, the method also includes: when the rewritten query question is a question of the first preset type, using the large language model to split the rewritten query question according to the prompt word template to obtain at least one sub-question of the rewritten query question; determining a first reply to the at least one sub-question; inputting the rewritten query question into the large language model to obtain a second reply to the rewritten query question; and determining a final reply to the rewritten query question based on the first reply and the second reply.

[0013] In an embodiment of the present application, obtaining a user's query question includes: obtaining an original query question input by the user; inputting the original query question into a question understanding large language model to determine through the question understanding large language model whether the completeness of the original query question is greater than a preset completeness, and / or whether the original query question is ambiguous; if the completeness of the original query question is less than the preset completeness and / or there is ambiguity, updating the original query question to obtain a query question, wherein the completeness of the query question is not less than the preset completeness and there is no ambiguity.

[0014] In an embodiment of the present application, determining the target domain for the query question according to the domain interest table includes: converting the domain interest table into a domain probability table, and determining the knowledge domain with the highest probability in the domain probability table as the target domain.

[0015] In an embodiment of the present application, determining the target domain for the query question based on the domain interest table includes: randomly sampling the highest score selection mode and the random selection mode to obtain a sampling result, wherein the sum of the hit probability of the highest score selection mode and the hit probability of the random selection mode is 1; when the sampling result is that the highest score selection mode is adopted, the knowledge domain with the highest interest level in the domain interest table is determined as the target domain; when the sampling result is that the random selection mode is adopted, the knowledge domain with the highest interest level is eliminated from the domain interest table, and a knowledge domain is randomly selected from the remaining knowledge domains in the domain interest table according to the level of interest as the target domain.

[0016] In an embodiment of the present application, the method also includes: obtaining feedback data from the user regarding the final reply, wherein the feedback data includes like behavior data or dislike behavior data; and updating the user's field interest table according to the feedback data.

[0017] In an embodiment of the present application, updating the user's domain interest table based on feedback data includes: when the feedback data is like behavior data, increasing the domain interest table to represent the user's interest in the target domain; when the feedback data is dislike behavior data, reducing the domain interest table to represent the user's interest in the target domain.

[0018] In the embodiment of the present application, the user's domain interest table is updated according to expression (1):

[0019] (1)

[0020] in, Q ’ is the user's interest in the target field in the updated field interest table, Q is the user's interest in the target field in the field interest table before updating, α is the learning rate and is a constant. When the feedback data is the like behavior data, r is a positive number. When the feedback data is click behavior data, r Is a negative number.

[0021] In the embodiment of the present application, the second largest language model is a large language model for answering questions in the target domain.

[0022] In an embodiment of the present application, the question rewriting large language model is used to: when the query question belongs to a first preset type, output a question cluster after the query question is rewritten; when the query question belongs to a second preset type, output a second preset type identifier; the method also includes: when the question rewriting large language model outputs the second preset type identifier, inputting the query question into the second large language model to obtain a final response to the query question.

[0023] A second aspect of the present application provides a device for processing user questions and answers, including:

[0024] a memory configured to store instructions;

[0025] The processor is configured to call instructions from the memory and implement the above-mentioned method for processing user questions and answers when executing the instructions.

[0026] A third aspect of the present application provides a machine-readable storage medium having instructions stored thereon, the instructions being used to enable a machine to execute the above-mentioned method for processing user questions and answers.

[0027] Through the above technical solution, the user's query question and the user's corresponding domain interest table can be obtained, and the target domain for the query question can be determined according to the domain interest table, and the prompt word template corresponding to the target domain can be obtained, and then the prompt word template is input into the first large language model to construct the question rewriting large language model corresponding to the target domain; then the query question is input into the question rewriting large language model to obtain a question cluster after the query question is rewritten, and each sub-question in the question cluster is input into the second large language model in order from low to high level, so as to obtain the corresponding reply of each sub-question in the question cluster in turn; finally, the final reply to the query question is determined based on the reply to the highest-level sub-question in the question cluster, so that the large language model can better understand and process the query question, obtain a more accurate reply, and improve user satisfaction.

[0028] Other features and advantages of the embodiments of the present application will be described in detail in the subsequent specific implementation section. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the following specific implementations, they are used to explain the embodiments of the present application, but do not constitute a limitation on the embodiments of the present application. In the accompanying drawings:

[0030] Figure 1 A schematic diagram of a process of processing user questions and answers according to an embodiment of the present application is shown;

[0031] Figure 2 Another flowchart of a method for processing user questions and answers according to an embodiment of the present application is schematically shown;

[0032] Figure 3 The internal structure diagram of a computer device according to an embodiment of the present application is schematically shown. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the specific implementation methods described herein are only used to illustrate and explain the embodiments of the present application, and are not used to limit the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0034] In addition, if there are descriptions involving "first", "second", etc. in the embodiments of the present application, the descriptions of "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or suggesting their relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0035] Figure 1 A schematic diagram of a process flow of a method for processing user questions and answers according to an embodiment of the present application is shown. Figure 1 As shown, an embodiment of the present application provides a method for processing user questions and answers, which may include the following steps.

[0036] Step 101: Obtain a user's query question and a corresponding domain interest table of the user, wherein the domain interest table includes the user's interest level in at least two knowledge domains.

[0037] The processor may obtain the query question of the user and the domain interest table corresponding to the user, wherein the domain interest table includes the user's interest level in at least two knowledge fields.

[0038] In an embodiment of the present application, obtaining a user's query question includes: obtaining an original query question input by the user; inputting the original query question into a question understanding large language model to determine through the question understanding large language model whether the completeness of the original query question is greater than a preset completeness, and / or whether the original query question is ambiguous; if the completeness of the original query question is less than the preset completeness and / or there is ambiguity, updating the original query question to obtain a query question, wherein the completeness of the query question is not less than the preset completeness and there is no ambiguity.

[0039] The processor may obtain a query question from a user. Specifically, the processor may obtain an original query question input by a user. After obtaining the original query question, the processor may input the original query question into a large language model for question understanding to determine whether the completeness of the original query question is greater than a preset completeness, and / or whether the original query question is ambiguous through the large language model for question understanding. The preset completeness may be determined based on actual conditions. In the event that the completeness of the original query question is less than a preset completeness and / or there is ambiguity, the processor may update the original query question through the large language model for question understanding to obtain a query question, wherein the completeness of the query question is not less than a preset completeness and there is no ambiguity, so that the query question is very clear and complete, and can accurately express the needs of the user.

[0040] Step 102: Determine the target domain for the query question according to the domain interest table, and obtain a prompt word template corresponding to the target domain, where the prompt word template includes domain knowledge corresponding to the target domain.

[0041] After obtaining the domain interest table corresponding to the user, the processor can determine the target domain for the query question according to the domain interest table. After determining the target domain for the query question, the processor can obtain a prompt word template corresponding to the target domain, which can include domain knowledge corresponding to the target domain.

[0042] In an embodiment of the present application, determining the target domain for the query question according to the domain interest table includes: converting the domain interest table into a domain probability table, and determining the knowledge domain with the highest probability in the domain probability table as the target domain.

[0043] The processor can determine the target domain for the query question based on the domain interest table. Specifically, the processor can convert the domain interest table into a domain probability table, and determine the knowledge domain with the highest probability in the domain probability table as the target domain, so as to improve the efficiency and accuracy of selecting the target domain.

[0044] In an embodiment of the present application, determining the target domain for the query question based on the domain interest table includes: randomly sampling the highest score selection mode and the random selection mode to obtain a sampling result, wherein the sum of the hit probability of the highest score selection mode and the hit probability of the random selection mode is 1; when the sampling result is that the highest score selection mode is adopted, the knowledge domain with the highest interest level in the domain interest table is determined as the target domain; when the sampling result is that the random selection mode is adopted, the knowledge domain with the highest interest level is eliminated from the domain interest table, and a knowledge domain is randomly selected from the remaining knowledge domains in the domain interest table according to the level of interest as the target domain.

[0045] The processor can determine the target domain for the query question based on the domain interest table. Specifically, the processor can set two modes for determining the target domain for the query question: a high-score selection mode and a random selection mode. The processor can randomly sample the high-score selection mode and the random selection mode to obtain a sampling result. Among them, the sum of the hit probability of the highest score selection mode and the hit probability of the random selection mode is 1. In the case where the sampling result is to adopt the high-score selection mode, the processor can determine the knowledge field with the highest degree of interest in the domain interest table as the target field. In the case where the sampling result is to adopt the random selection mode, the processor can eliminate the knowledge field with the highest degree of interest from the domain interest table to obtain the remaining knowledge fields except the knowledge field with the highest degree of interest. And randomly select a knowledge field from the remaining knowledge fields in the domain interest table according to the degree of interest as the target field.

[0046] Step 103: Input the prompt word template into the first large language model to construct a question rewriting large language model corresponding to the target domain.

[0047] Step 104: Input the query question into the question rewriting large language model to obtain a question cluster after the query question is rewritten, where the question cluster includes sub-questions at multiple levels, and the answer to a sub-question at any level depends on the answer to a sub-question at least one level lower than the level.

[0048] Step 105: Input each sub-question in the question cluster into the second largest language model in order from low to high levels, so as to obtain the corresponding responses to each sub-question in the question cluster in turn, wherein when a sub-question other than the lowest level is input into the second largest language model, the responses to the lower-level sub-questions on which it depends are carried along.

[0049] Step 106: Determine a final answer to the query question based on the answers to the highest-level sub-questions in the question cluster.

[0050] After obtaining the prompt word template corresponding to the target domain, the processor can input the prompt word template into the first language model to construct the problem rewriting large language model corresponding to the target domain. After obtaining the problem rewriting large language model, the processor can input the query question into the problem rewriting large language model to obtain the query question rewritten question cluster, the question cluster includes sub-questions at multiple levels, and the answer to the sub-question at any level depends on the reply of the sub-question at least one level lower than the any level. After obtaining the query question rewritten question cluster, the processor can input each sub-question in the question cluster into the second language model in order from low to high level, so as to obtain the reply corresponding to each sub-question in the question cluster in turn. Wherein, when inputting the sub-questions other than the lowest level into the second language model, the reply of the sub-question of the lower level on which it depends is carried. In an embodiment of the present application, the second language model can be a large language model for answering questions for the target domain. After obtaining the reply of each sub-question in the question cluster, the processor can determine the final reply of the query question based on the reply of the sub-question of the highest level in the question cluster.

[0051] In a specific embodiment, the target field may be the field of engineering machinery, and the prompt word template is as follows:

[0052] - Role: Expert in sub-task analysis in the field of construction machinery.

[0053] - Profile: You are an expert in problem grading and task splitting with good logical analysis, able to quickly identify task difficulty (processing steps) and decompose the various subtasks in user queries.

[0054] - Skills: You have advanced text analysis skills, can understand the deep meaning of user queries, and split them into specific subtasks. You are particularly good at handling query rewriting problems in the field of engineering machinery.

[0055] - Goals: Determine the difficulty level (processing steps) of the user's query and clearly and accurately break down and list each subtask in the user's query.

[0056] - OutputFormat: The response must include the problem classification (processing steps) and the subtasks after splitting, and is returned in json format. This json field contains three key values: Difficulty, sub_question, and sum_question. Difficulty is a bool value, which indicates whether it is necessary to wait for the results of the level 1 subquestions before processing the level 2 subquestions. sub_question and sum_question use lists to save the level 1 subquestions and level 2 subquestions respectively.

[0057] - Workflow:

[0058] 1. Carefully read and understand the entire content of the User's Query.

[0059] 2. Determine the difficulty level of the user's query and identify the individual subtasks in the query.

[0060] 3. Return the result in the format specified by OutputFormat.

[0061] - Examples:

[0062] - User query: Please compare the performance of xx pump truck and yy pump truck.

[0063] Subtask splitting:

[0064] {“Difficulty”:True,

[0065] “sub_question”: [“Parameters of xx pump truck”, “Parameters of yy pump truck”],

[0066] “sum_question”: [“Parameter comparison of xx pump truck and yy pump truck”]}.

[0067] In an embodiment of the present application, the method also includes: when the rewritten query question is a question of the first preset type, using the large language model to split the rewritten query question according to the prompt word template to obtain at least one sub-question of the rewritten query question; determining a first reply to the at least one sub-question; inputting the rewritten query question into the large language model to obtain a second reply to the rewritten query question; and determining a final reply to the rewritten query question based on the first reply and the second reply.

[0068] The processor may input the query question into the question rewriting large language model to obtain the rewritten query question. And determine the question type of the rewritten query question by the large language model. In the case where the rewritten query question is a question of the first preset type, the processor may use the large language model to split the rewritten query question according to the prompt word template to obtain at least one sub-question of the rewritten query question. The question type of the rewritten query question may include simple questions, primary questions, intermediate questions, and complex questions with increasing semantic complexity. The first preset type of question may be set as an intermediate question or a complex question based on actual conditions. After obtaining at least one sub-question of the rewritten query question, the processor may determine a first reply to the at least one sub-question. The processor may also input the rewritten query question into the large language model to obtain a second reply to the rewritten query question. After obtaining the first reply and the second reply, the processor may determine the final reply of the rewritten query question according to the first reply and the second reply to improve the accuracy of the final reply and better solve the user's problem.

[0069] In an embodiment of the present application, the question rewriting large language model is used to: when the query question belongs to a first preset type, output a question cluster after the query question is rewritten; when the query question belongs to a second preset type, output a second preset type identifier; the method also includes: when the question rewriting large language model outputs the second preset type identifier, inputting the query question into the second large language model to obtain a final response to the query question.

[0070] The question rewriting large language model can be used to output a question cluster after the query question is rewritten when the query question belongs to the first preset type. When the query question belongs to the second preset type, the second preset type identifier is output. The second preset type can be determined according to the actual situation. For example, the second preset type can be a simple question, that is, a question that does not require retrieval and can be directly answered. The second preset type identifier can be used to characterize the query question as a simple question. When the question rewriting large language model outputs the identifier of the second preset type, the processor can input the query question into the second large language model to obtain the final reply to the query question to improve the efficiency of the reply.

[0071] In an embodiment of the present application, the method also includes: obtaining feedback data from the user regarding the final reply, wherein the feedback data includes like behavior data or dislike behavior data; and updating the user's field interest table according to the feedback data.

[0072] The processor may obtain feedback data from the user regarding the final reply, wherein the feedback data includes like behavior data or dislike behavior data. The like behavior data indicates that the user approves the final reply, and the dislike behavior data indicates that the user is dissatisfied with the final reply. After obtaining the feedback data, the processor may update the user's domain interest table according to the feedback data.

[0073] In an embodiment of the present application, updating the user's domain interest table based on feedback data includes: when the feedback data is like behavior data, increasing the domain interest table to represent the user's interest in the target domain; when the feedback data is dislike behavior data, reducing the domain interest table to represent the user's interest in the target domain.

[0074] The processor can update the user's domain interest table based on the feedback data. Specifically, when the feedback data is like behavior data, the processor can increase the degree of interest of the user in the target domain in the domain interest table. When the feedback data is dislike behavior data, the processor can reduce the degree of interest of the user in the target domain in the domain interest table, so as to update the domain interest table in real time based on user feedback and improve the fit between the domain interest table and the user. In the embodiment of the present application, the user's domain interest table is updated according to expression (1):

[0075] (1)

[0076] in, Q ’ is the user's interest in the target field in the updated field interest table, Q is the user's interest in the target field in the field interest table before updating, α is the learning rate and is a constant. When the feedback data is the like behavior data, r is a positive number. When the feedback data is click behavior data, r Is a negative number.

[0077] For example, the domain interest table includes the user's interest in the general knowledge field, engineering machinery knowledge field, legal knowledge field, financial knowledge field, and artificial intelligence knowledge field, which are 0.1, 0.25, 0.08, 0, and 0.5 respectively. Assume that the target field is the artificial intelligence knowledge field, and the user's feedback on the final reply is a like, that is, r=1. α The value is 0.1. The processor can determine that the interest level of the artificial intelligence knowledge field in the field interest table is updated to 0.5+0.1*(1-0.5)=0.55. Assume that the target field is the legal knowledge field, and the user's feedback on the final reply is a thumbs-up, that is, r=-1. αThe value is 0.1. The processor can determine that the interest level of the artificial intelligence knowledge field in the field interest table is updated to 0.08+0.1*(-1-0.08)=-0.028.

[0078] In the embodiments of the present application, Figure 2 As shown, the processor can obtain the user's question (query question) and the user's corresponding domain interest table (Q-table), and the domain interest table includes the user's interest in at least two knowledge fields. The processor can input the question into the question understanding large language model to determine whether the question needs to be clarified through the question understanding large language model. That is, the question understanding large language model determines whether the question is complete and / or whether there is ambiguity. In the case of determining that the question needs to be clarified, the processor can input the question into the question rewriting large language model. In the case of incomplete and / or ambiguous questions, the question rewriting large language model will seek additional key information from the human (user) and output the rewritten question to the human. In the case that the question does not need to be clarified, the processor can determine the target domain for the question according to the domain interest table, and obtain the query rewriting template corresponding to the target domain. The processor can input the query rewriting template into the first large language model to construct a question rewriting large language model corresponding to the target domain, so as to classify and split the question through the question rewriting large language model. Specifically, the processor can input the question into the question rewriting large language model to grade the question and obtain level 1 sub-question 1...level 1 sub-question n and level 2 sub-questions. Among them, the level of the level 1 sub-question is lower than that of the level 2 sub-question. The grading determines whether the level 2 sub-question is a summary of the level 1 sub-question RAG (Retrieval-Augmented Generation) or a summary of the level 1 retrieval results. The processor can retrieve and summarize each level 1 sub-question in turn through the second largest language model, and input the retrieval and summary of all level 1 sub-questions and level 2 sub-questions into the second largest language model for retrieval and summary to obtain the final reply.

[0079] Through the above technical solutions, the large language model can better understand and process query questions, obtain more accurate responses, and improve user satisfaction.

[0080] Figure 1 and Figure 2 FIG. 1 is a flow chart of a method for processing user questions and answers in one embodiment. It should be understood that although Figure 1 and Figure 2The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1 and Figure 2 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0081] The present application also provides a device for processing user questions and answers, including:

[0082] a memory configured to store instructions;

[0083] The processor is configured to call instructions from the memory and implement the above-mentioned method for processing user questions and answers when executing the instructions.

[0084] An embodiment of the present application also provides a machine-readable storage medium, on which instructions are stored, and the instructions are used to enable a machine to execute the above-mentioned method for processing user questions and answers.

[0085] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 3 As shown. The computer device includes a processor A01, a network interface A02, a memory (not shown in the figure) and a database (not shown in the figure) connected through a system bus. Among them, the processor A01 of the computer device is used to provide computing and control capabilities. The memory of the computer device includes an internal memory A03 and a non-volatile storage medium A04. The non-volatile storage medium A04 stores an operating system B01, a computer program B02 and a database (not shown in the figure). The internal memory A03 provides an environment for the operation of the operating system B01 and the computer program B02 in the non-volatile storage medium A04. The database of the computer device is used to store data such as query questions, field interest tables, prompt word templates and final replies. The network interface A02 of the computer device is used to communicate with an external terminal through a network connection. When the computer program B02 is executed by the processor A01, a method for processing user questions and answers is implemented.

[0086] Those skilled in the art will understand that Figure 3The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0087] The embodiment of the present application provides a device, the device includes a processor, a memory, and a program stored in the memory and executable on the processor, and the processor implements the following steps when executing the program: obtaining a user's query question and a domain interest table corresponding to the user, wherein the domain interest table includes the user's interest level in at least two knowledge fields; determining a target domain for the query question according to the domain interest table, and obtaining a prompt word template corresponding to the target domain, wherein the prompt word template includes domain knowledge corresponding to the target domain; inputting the prompt word template into a first large language model to construct a question rewriting large language model corresponding to the target domain; inputting the query question into the question rewriting large language model to obtain a question cluster after the query question is rewritten, wherein the question cluster includes sub-questions at multiple levels, and the answer to a sub-question at any level depends on the answer to a sub-question at least one level lower than the any level; inputting each sub-question in the question cluster into the second large language model in order from low to high levels, to obtain the answer corresponding to each sub-question in the question cluster in order, wherein when a sub-question other than the lowest level is input into the second large language model, the answer to the lower level sub-question on which it depends is carried; and determining the final answer to the query question based on the answer to the highest level sub-question in the question cluster.

[0088] In one embodiment, the method also includes: when the rewritten query question is a question of the first preset type, using the large language model to split the rewritten query question according to the prompt word template to obtain at least one sub-question of the rewritten query question; determining a first reply to the at least one sub-question; inputting the rewritten query question into the large language model to obtain a second reply to the rewritten query question; and determining a final reply to the rewritten query question based on the first reply and the second reply.

[0089] In one embodiment, obtaining a user's query question includes: obtaining an original query question input by the user; inputting the original query question into a question understanding large language model to determine whether the completeness of the original query question is greater than a preset completeness, and / or whether the original query question is ambiguous through the question understanding large language model; if the completeness of the original query question is less than the preset completeness and / or there is ambiguity, updating the original query question to obtain a query question, wherein the completeness of the query question is not less than the preset completeness and there is no ambiguity.

[0090] In one embodiment, determining the target domain for the query question according to the domain interest table includes: converting the domain interest table into a domain probability table, and determining the knowledge domain with the highest probability in the domain probability table as the target domain.

[0091] In one embodiment, determining the target domain for the query question based on the domain interest table includes: randomly sampling the highest score selection mode and the random selection mode to obtain a sampling result, wherein the sum of the hit probability of the highest score selection mode and the hit probability of the random selection mode is 1; when the sampling result is that the highest score selection mode is adopted, the knowledge domain with the highest interest level in the domain interest table is determined as the target domain; when the sampling result is that the random selection mode is adopted, the knowledge domain with the highest interest level is eliminated from the domain interest table, and a knowledge domain is randomly selected from the remaining knowledge domains in the domain interest table according to the level of interest as the target domain.

[0092] In one embodiment, the method further includes: obtaining feedback data of the user regarding the final reply, wherein the feedback data includes like behavior data or dislike behavior data; and updating the user's field interest table according to the feedback data.

[0093] In one embodiment, updating the user's domain interest table according to feedback data includes: when the feedback data is like behavior data, increasing the domain interest table representing the user's interest in the target domain; when the feedback data is dislike behavior data, decreasing the domain interest table representing the user's interest in the target domain.

[0094] In one embodiment, the user's domain interest table is updated according to expression (1):

[0095] (1)

[0096] in, Q ’ is the user's interest in the target field in the updated field interest table, Q is the user's interest in the target field in the field interest table before updating, α is the learning rate and is a constant. When the feedback data is the like behavior data, r is a positive number. When the feedback data is click behavior data, r Is a negative number.

[0097] In one embodiment, the second large language model is a large language model for answering questions in the target domain.

[0098] In one embodiment, the question rewriting large language model is used to: when the query question belongs to a first preset type, output a question cluster after the query question is rewritten; when the query question belongs to a second preset type, output a second preset type identifier; the method also includes: when the question rewriting large language model outputs the second preset type identifier, inputting the query question into the second large language model to obtain a final response to the query question.

[0099] The present application also provides a computer program product, which, when executed on a data processing device, is suitable for executing the program of the method steps of initializing processing user questions and answers.

[0100] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0101] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0102] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0103] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0104] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0105] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0106] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0107] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0108] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A method for processing user questions and answers, characterized in that: The method comprises: Acquire a query question of a user and a field interest table corresponding to the user, wherein the field interest table includes the user's interest level in at least two knowledge fields; Determine a target domain for the query question according to the domain interest table, and obtain a prompt word template corresponding to the target domain, wherein the prompt word template includes domain knowledge corresponding to the target domain; Inputting the prompt word template into a first large language model to construct a question rewriting large language model corresponding to the target domain; Inputting the query question into the question rewriting large language model to obtain a question cluster after the query question is rewritten, wherein the question cluster includes sub-questions at multiple levels, and the answer to a sub-question at any level depends on the answer to a sub-question at least one level lower than the any level; Inputting each sub-question in the question cluster into the second largest language model in order from low to high levels, so as to obtain the corresponding answer to each sub-question in the question cluster in turn, wherein when a sub-question other than the lowest level is input into the second largest language model, the answer to the sub-question of the lower level on which it depends is carried; Determining a final answer to the query question based on answers to the highest-level sub-questions in the question cluster; Wherein, determining the target domain for the query question according to the domain interest table includes: Randomly sampling the highest score selection mode and the random selection mode to obtain a sampling result, wherein the sum of the hit probability of the highest score selection mode and the hit probability of the random selection mode is 1; In the case where the sampling result adopts the highest score selection mode, the knowledge field with the highest interest level in the field interest table is determined as the target field; When the sampling result is in a random selection mode, the knowledge field with the highest interest level is removed from the field interest table, and a knowledge field is randomly selected from the remaining knowledge fields in the field interest table according to the interest level as the target field.

2. The method for processing user questions and answers according to claim 1, characterized in that: The method further comprises: When the rewritten query question is a question of the first preset type, split the rewritten query question according to the prompt word template using a large language model to obtain at least one sub-question of the rewritten query question; determining a first response to the at least one sub-question; Inputting the rewritten query question into the large language model to obtain a second answer to the rewritten query question; A final answer to the rewritten query question is determined according to the first answer and the second answer.

3. The method for processing user questions and answers according to claim 1, characterized in that: The query questions of obtaining the user include: Get the original query question entered by the user; Inputting the original query question into a question understanding large language model, so as to determine whether the completeness of the original query question is greater than a preset completeness, and / or whether the original query question is ambiguous, through the question understanding large language model; In the case where the completeness of the original query question is less than a preset completeness and / or there is ambiguity, the original query question is updated to obtain the query question, and the completeness of the query question is not less than a preset completeness and there is no ambiguity.

4. The method for processing user questions and answers according to claim 1, characterized in that: Determining the target domain for the query question according to the domain interest table includes: The domain interest table is converted into a domain probability table, and the knowledge domain with the highest probability in the domain probability table is determined as the target domain.

5. The method for processing user questions and answers according to claim 1, characterized in that: The method further comprises: Acquiring feedback data of the user in response to the final reply, wherein the feedback data includes like behavior data or dislike behavior data; The user's domain interest table is updated according to the feedback data.

6. The method for processing user questions and answers according to claim 5, characterized in that: The updating of the user's domain interest table according to the feedback data comprises: In the case where the feedback data is like behavior data, adding the field interest table to represent the user's interest in the target field; In the case where the feedback data is click behavior data, the interest level of the user in the target domain represented in the domain interest table is reduced.

7. The method for processing user questions and answers according to claim 6, characterized in that: Update the user's domain interest table according to expression (1): （1） in, Q ’ is the user's interest in the target field as described in the updated field interest table, Q is the user's interest in the target field as described in the field interest table before updating, α is the learning rate and is a constant. When the feedback data is the like behavior data, r is a positive number. In the case where the feedback data is the tapping behavior data, r Is a negative number.

8. The method for processing user questions and answers according to claim 1, characterized in that: The second largest language model is a large language model for answering questions in the target domain.

9. The method for processing user questions and answers according to claim 1, characterized in that: The question rewrites the large language model to: When the query question belongs to a first preset type, outputting the question cluster after the query question is rewritten; When the query question belongs to a second preset type, outputting a second preset type identifier; The method further comprises: When the question rewriting large language model outputs the second preset type identifier, the query question is input into the second large language model to obtain a final answer to the query question.

10. A device for processing user questions and answers, characterized in that: include: a memory configured to store instructions; A processor is configured to call the instructions from the memory and implement the method for processing user questions and answers according to any one of claims 1 to 9 when executing the instructions.

11. A machine-readable storage medium, characterized in that: The machine-readable storage medium stores instructions for causing a machine to execute the method for processing user questions and answers according to any one of claims 1 to 9.

Images