Model processing method, speech interaction method and device thereof, equipment and storage medium

Abstract

The present disclosure provides a model processing method, a voice interaction method and device thereof, equipment and a storage medium, relates to the field of data processing, and particularly relates to the fields of artificial intelligence, big data, voice technology and the like. The specific implementation scheme is as follows: a candidate question set of each initial sample data in M initial sample data is obtained, wherein the initial sample data contains m rounds of question and answer between an object and an agent; the candidate question set of the initial sample data includes a next round question set corresponding to the mth round question in the m rounds of question and answer; based on the M initial sample data, the candidate question set of each initial sample data and the label data of each initial sample data, M training sample data is obtained, the label data of the initial sample data includes a target question required to be generated by the agent in the m+1th round; and the M training sample data is used to train a to-be-trained model, so as to obtain a target model capable of predicting a question required to be generated by the agent in the next round based on historical question and answer.

Description

Technical Field

[0001] This disclosure relates to the field of data processing technology, and in particular to the fields of artificial intelligence, big data, and voice technology. Background Technology

[0002] Intelligent voice assistants are intelligent voice agents capable of conducting multi-turn dialogues. These agents can communicate with others through understanding, proactive questioning, and clarification to achieve specific goals (such as information gathering or targeted research). With the development of AI technologies such as deep learning, the capabilities of intelligent voice agents are becoming increasingly powerful, enabling fully automated communication. They are currently widely used in numerous fields, such as intelligent customer service. Summary of the Invention

[0003] This disclosure provides a model processing method, a voice interaction method, and an apparatus, device, and storage medium thereof.

[0004] According to one aspect of this disclosure, a model processing method is provided, comprising:

[0005] Obtain a candidate question set for each of the M initial sample data sets, where each initial sample data set contains m rounds of question-and-answer interactions between the object and the agent; the candidate question set for the initial sample data set includes the set of questions for the next round corresponding to the question in the m-th round of question-and-answer interactions; M and m are both positive integers greater than or equal to 1;

[0006] Based on M initial sample data, the candidate question set of each initial sample data, and the label data of each initial sample data, M training sample data are obtained. The label data of the initial sample data includes the target question that the agent needs to generate in the (m+1)th round.

[0007] The model to be trained is trained using M training sample data to obtain a target model that can predict the questions that the agent needs to generate in the next round based on historical question answers.

[0008] According to another aspect of this disclosure, a voice interaction method is provided, comprising:

[0009] Obtain target text data, which is obtained based on the object's voice response to the question in the previous round between the object and the agent;

[0010] Obtain the next set of candidate questions for the question addressed by the target text data;

[0011] The target text data and the next set of candidate questions for the question addressed by the target text data are input into the target model to obtain the question text that the agent needs to generate.

[0012] According to another aspect of this disclosure, a model processing apparatus is provided, comprising:

[0013] A data processing unit is used to obtain a candidate question set for each of the M initial sample data sets, wherein the initial sample data sets contain m rounds of question-and-answer interactions between the object and the agent; the candidate question set for the initial sample data sets includes the set of questions for the next round corresponding to the question in the m-th round of question-and-answer interactions; M and m are both positive integers greater than or equal to 1; based on the M initial sample data sets, the candidate question sets for each initial sample data set, and the label data for each initial sample data set, M training sample data sets are obtained, wherein the label data for the initial sample data sets includes the target question that the agent needs to generate in the (m+1)-th round;

[0014] The model training unit is used to train the model to be trained using M training sample data to obtain a target model that can predict the questions that the agent needs to generate in the next round based on historical question answers.

[0015] According to another aspect of this disclosure, a voice interaction device is provided, comprising:

[0016] The acquisition unit is used to acquire target text data, which is obtained based on the object's answer voice in the previous round of questions between the object and the agent; and to acquire a set of candidate questions for the next round of questions that the target text data addresses.

[0017] The model prediction unit is used to input the target text data and the next set of candidate questions for the question addressed by the target text data into the target model to obtain the question text to be generated by the agent.

[0018] According to another aspect of this disclosure, an electronic device is provided, comprising:

[0019] At least one processor; and

[0020] The memory is communicatively connected to the at least one processor; wherein,

[0021] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform any of the methods described in the present disclosure.

[0022] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause the computer to perform any of the methods according to embodiments of this disclosure.

[0023] According to another aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements any of the methods according to embodiments of this disclosure.

[0024] In this way, the disclosed solution can obtain M training sample data based on M initial sample data, the candidate question set of each initial sample data, and the label data of each initial sample data, so as to train the model to be trained. This enables the model to be trained to perform reasoning within a specified range (such as the candidate question set), realizes dialogue management, and can also effectively avoid the illusion problem in the model generation process, thereby improving the inference speed of the model and effectively enhancing the user experience.

[0025] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0026] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0027] Figure 1 This is a schematic diagram of the structural framework of a modularly designed intelligent voice assistant.

[0028] Figure 2 This is an illustrative flow diagram of a model processing method according to an embodiment of this application. Figure 1 ;

[0029] Figure 3 This is a schematic diagram of a scenario for model training according to an embodiment of this application;

[0030] Figure 4(a) is a schematic flowchart of a model processing method according to an embodiment of this application. Figure 2 ;

[0031] Figures 4(b) to 4(d) This is an exemplary diagram of the finite state machine used in a model processing method according to an embodiment of this application;

[0032] Figure 5 This is an illustrative flow diagram of a model processing method according to an embodiment of this application. Figure 3 ;

[0033] Figure 6(a) is a schematic flowchart of a voice interaction method according to an embodiment of the present application;

[0034] Figure 6(b) is a schematic diagram of a scenario of a voice interaction method according to an embodiment of this application;

[0035] Figure 7 This is a schematic flowchart of a model processing method according to an embodiment of this application in a specific example;

[0036] Figure 8This is a schematic diagram of the structure of a model processing device according to an embodiment of this application;

[0037] Figure 9 This is a schematic diagram of the structure of a voice interaction device according to an embodiment of this application;

[0038] Figure 10 This is a block diagram of an electronic device used to implement the model processing method of the embodiments of this disclosure. Detailed Implementation

[0039] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0040] In this document, the term "and / or" merely describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. The term "at least one" in this document indicates any combination of at least two of a plurality of elements. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C. The terms "first" and "second" in this document refer to and distinguish between multiple similar technical terms, not to restrict the order or to limit there to only two. For example, "first feature" and "second feature" refer to two categories / two features; the first feature can be one or more, and the second feature can also be one or more.

[0041] Furthermore, to better illustrate this disclosure, numerous specific details are set forth in the following detailed description. Those skilled in the art will understand that this disclosure can still be practiced even without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art have not been described in detail in order to highlight the main points of this disclosure.

[0042] The following describes the related technologies of the embodiments of this disclosure. The following related technologies are optional solutions and can be combined with the technical solutions of the embodiments of this disclosure in any way, and they all fall within the protection scope of the embodiments of this disclosure.

[0043] Intelligent voice assistants are intelligent voice agents capable of conducting multi-turn dialogues. These agents can communicate with others through understanding, proactive questioning, and clarification to achieve specific goals (such as information gathering or targeted research). With the development of AI technologies such as deep learning, the capabilities of intelligent voice agents are becoming increasingly powerful, enabling fully automated communication. They are currently widely used in numerous fields, such as intelligent customer service.

[0044] Typically, smart voice assistants deployed in applications are primarily designed in a modular fashion, such as... Figure 1 As shown, it mainly includes the following independent modules:

[0045] (1) Natural Language Understanding (NLU) module, used to understand the content of the speech of the object and identify the object's intention, etc.

[0046] (2) Dialogue Management (DM) module: Based on the understanding of the content of the conversation, determine the topic of the next conversation.

[0047] (3) Natural Language Generation (NLG) module, used to generate the subsequent chat content.

[0048] In addition, to achieve the conversion between speech and text, it will also include:

[0049] (a) Automatic Speech Recognition (ASR) module, used to convert the speech data of an object into text content.

[0050] (b) Text to Speech (TTS) module, which is used to convert the text content output by the speech agent into speech data.

[0051] The modular design of intelligent voice assistants breaks down the entire dialogue management process into multiple modules, each responsible for a specific function. However, this modular design has certain limitations. For example, there is the problem of error accumulation, where errors from one module accumulate to the next, thus amplifying the errors. In addition, modular design also increases maintenance and service update costs.

[0052] Based on this, the disclosed solution provides an LLM-based intelligent voice assistant solution that leverages the powerful language understanding, scheduling, and generation capabilities of large models to achieve end-to-end dialogue control. For example, it replaces the multiple text processing modules (e.g., NLU, DM, NLG) in the intelligent voice assistant with LLM. This addresses the issues of error accumulation, high maintenance costs, and high service update costs.

[0053] In order to achieve end-to-end dialogue control, this disclosure provides a model processing scheme to enable the trained LLM to have end-to-end dialogue management capabilities. Moreover, this disclosure also effectively improves the accuracy of the output results, thereby effectively improving the user experience.

[0054] Specifically, Figure 2 This is an illustrative flow diagram of a model processing method according to an embodiment of this application. Figure 1 This method can be optionally applied to electronic devices, such as personal computers, servers, server clusters, and other electronic devices.

[0055] Furthermore, the method includes at least a portion of the following: For example... Figure 2 As shown, it includes:

[0056] Step S201: Obtain the candidate problem set for each of the M initial sample data.

[0057] Here, in this example, the initial sample data package contains m rounds of question-and-answer between the object and the agent (e.g., an intelligent voice agent); further, one round of question-and-answer between the object and the agent includes the question generated by the agent and the object's answer.

[0058] It should be noted that the number of dialogue turns can be the same or different for different initial sample data. In other words, the number of dialogue turns can be the same or different for different initial sample data.

[0059] For example, let Q be the question generated by the agent, A be the object's answer, and C be the question-and-answer (dialogue) between the agent and the object. Then, the m rounds of question-and-answer (i.e., m rounds of dialogue) between the object and the agent can be specifically represented as follows:

[0060] C [1,m] =(Q1,A1;Q2,A2;...;Q k A k ;…;Q m A m )

[0061] Here, (Q) k A kQ represents one round of question-and-answer communication in m rounds between the object and the agent, i.e., the k-th round of question-and-answer communication. k Let A represent the question generated by the agent in the k-th round of question answering. k This represents the object's response to Qk in the k-th round of question and answer.

[0062] Furthermore, in this example, the candidate question set of the initial sample data may specifically include the next round question set corresponding to the question in the m-th round of question-and-answer.

[0063] Here, M and m are both positive integers greater than or equal to 1; it is understandable that the values ​​of M and m are irrelevant.

[0064] For example, continuing with m rounds of question-and-answer C between the object and the agent. [1,m] For example, if the initial sample data consists of m rounds of question-and-answer sessions C [1,m] Then the candidate question set of the initial sample data is C for m rounds of question answering. [1,m] The m-th round problem Q m The corresponding set of questions for the next round {Q m+1,1 Q m+1,2 ,…}.

[0065] Step S202: Based on M initial sample data, the candidate question set of each initial sample data, and the label data of each initial sample data, obtain M training sample data.

[0066] Here, if the initial sample data includes m rounds of question-and-answer sessions, the label data of the initial sample data specifically includes the target question that the agent needs to generate in the (m+1)th round.

[0067] In other words, the label data of the initial sample data is the target question that the agent needs to generate in the next round of the most recent question-and-answer session contained in the initial sample data.

[0068] Step S203: Use M training sample data to train the model to be trained, so as to obtain a target model that can predict the questions that the agent needs to generate in the next round based on historical question answering.

[0069] In one example, the model to be trained described in this disclosure is a large language model. For instance, it is a large language model with adjustable parameters and a parameter count less than a preset value. This can effectively reduce inference time and lay the foundation for improving user experience.

[0070] In this way, the disclosed solution can obtain M training sample data based on M initial sample data, the candidate question set of each initial sample data, and the label data of each initial sample data, so as to train the model to be trained. This enables the model to be trained to perform reasoning within a specified range (such as the candidate question set), realizes dialogue management, and can also effectively avoid the illusion problem in the model generation process, thereby improving the inference speed of the model and effectively enhancing the user experience.

[0071] Furthermore, it should be noted that because this disclosed solution can use a large language model with stronger reasoning capabilities, it can effectively improve the accuracy of the target model's output, thereby further enhancing the user experience. Moreover, in the application scenario of intelligent voice assistants, this disclosed solution effectively avoids the problems of error accumulation, high maintenance costs, and high service update costs caused by modular solutions.

[0072] Furthermore, in a specific example, the training method can be used to train the model to be trained; specifically, the above-described method of training the model to be trained using M training sample data to obtain a target model that can predict the questions to be generated by the agent in the next round based on historical question answers (e.g., step S203 above) can specifically include:

[0073] Step S203-1: Input the m rounds of question-and-answer data contained in the training sample data, and the candidate question set corresponding to the m-th round question (i.e., the next round question set corresponding to the m-th round question) contained in the training sample data into the model to be trained to obtain the initial estimation result. Here, the initial estimation result represents the predicted question that the agent needs to generate in the (m+1)-th round.

[0074] In this example, the training sample data may specifically include initial sample data, a set of candidate questions for the initial sample data, and label data for the initial sample data. For instance, in one example, a single training sample data set may specifically include: m rounds of question-and-answer interaction between the object and the agent, the set of questions for the next round corresponding to the question in the m-th round of question-and-answer interaction, and the target question (label data) that the agent needs to generate in the (m+1)-th round. Here, the m corresponding to different training sample data sets can be the same or different. Thus, by training the model based on the multiple training sample data sets constructed above, the model can effectively achieve dialogue management. Simultaneously, by efficiently reasoning to obtain the question that the agent needs to generate, the related problems of the above modular design are solved.

[0075] Step S203-2: Based on the initial estimation results and the target problem that the agent needs to generate in the (m+1)th round contained in the label data of the training sample data, obtain the loss value of the loss function.

[0076] Here, the loss function can characterize the distance between the predicted problem and the target problem.

[0077] Step S203-3: Based on the loss value of the loss function, adjust at least some of the adjustable parameters in the model to be trained to train and obtain the target model.

[0078] That is to say, in one example, such as Figure 3 As shown, firstly, the m rounds of question-and-answer sessions contained in the training sample data, and the candidate question set corresponding to the m-th round question contained in the training sample data, are input into the model to be trained to obtain an initial estimation result; secondly, the loss function value between the obtained initial estimation result and the target question to be generated by the agent in the (m+1)-th round in the label data is calculated; thirdly, the adjustable parameters of the model to be trained are adjusted according to the obtained loss function value. This process is repeated until a preset number of iterations is reached, or the loss function value meets a preset requirement (e.g., converges to a specified value), thereby obtaining the target model.

[0079] Thus, this disclosed solution provides a specific scheme for model training that is simple and efficient. This allows the model to perform inference within a specified range (such as a set of candidate questions), achieving dialogue management. At the same time, it can effectively avoid the illusion problem in the model generation process, thereby improving the model's inference speed and effectively enhancing the user experience.

[0080] Moreover, since the model performs inference within a specified range without needing to regenerate new questions, the training method of this disclosure also effectively saves computational resources and further improves model training efficiency.

[0081] Furthermore, in a specific example, the model to be trained can be used in the following manner; specifically, the input of the m rounds of question-and-answer sessions contained in the training sample data, and the candidate question set corresponding to the m-th round of questions contained in the training sample data, into the model to be trained (for example, step S203-1 described above), specifically includes:

[0082] Step S203-1-1: Based on the m rounds of question-and-answer contained in the training sample data, and the candidate question set corresponding to the m-th round of questions contained in the training sample data, obtain the target prompt word question.

[0083] In this example, a prompt word template can be pre-set. After determining the data to be input into the model, the prompt word template can be used to obtain the target prompt word question, thereby further improving the model's processing efficiency.

[0084] It should be noted that the prompt template can be designed based on actual needs, and this public solution does not impose any restrictions on it.

[0085] Step S203-1-2: Input the target prompt word question into the model to be trained.

[0086] In this way, the proposed solution can input the m rounds of question-and-answer data contained in the training sample data, along with the candidate question set corresponding to the m-th round question (i.e., the next round question set corresponding to the m-th round question), into the model to be trained in the form of target prompt questions. This allows the model to obtain appropriate questions from the candidate question set as output results, thereby enabling the model to perform reasoning within a specified range (such as the candidate question set), thus improving the training efficiency of the model. Moreover, since the model performs reasoning within a specified range without regenerating new questions, the training method of this proposed solution can also effectively save computational resources. At the same time, it can also effectively avoid the illusion problem in the generation process of the target model, thereby improving the inference speed of the model and effectively enhancing the user experience.

[0087] Specifically, Figure 4(a) is a schematic diagram of a model processing method according to an embodiment of this application. This method can optionally be applied to electronic devices, such as personal computers, servers, server clusters, and other electronic devices. It is understood that the above... Figure 2 The methods shown can also be applied to this example, and the related content will not be elaborated further in this example.

[0088] Furthermore, the method includes at least a portion of the following, as shown in Figure 4(a):

[0089] Step S401: Based on the determined finite state machine, obtain M initial sample data that satisfy at least one of the following conditions:

[0090] Data condition 1: Among the M initial sample data, the distribution of the number of initial sample data of different lengths satisfies the first distribution requirement.

[0091] Here, the length of the initial sample data is determined based on the finite state machine and represents the number of question-and-answer rounds.

[0092] Here, in one example, the first distribution is required to be specifically a uniform distribution. Alternatively, it can be other distributions, such as those determined based on the training objective; this disclosure does not impose any specific restrictions on this.

[0093] Data condition 2: In the M initial sample data, the distribution of the answers from the previous question to the next question satisfies the second distribution requirement.

[0094] Here, in one example, the second distribution requirement can also be specifically a uniform distribution. Alternatively, it can be other distributions, such as those determined based on the training objective, and this disclosure does not impose any specific restrictions on this.

[0095] Furthermore, in one example, both the first and second distribution requirements can be uniform distributions. Alternatively, in another example, the M initial sample data satisfy both of the above conditions. This effectively addresses the long-tail problem of data, effectively preventing the model from overemphasizing frequently occurring answers while weakening its ability to process less common answers, thus laying the foundation for further improving the accuracy of the model's inference results.

[0096] Here, the finite state machine can represent a set of questions generated by the agent, and the transition conditions from the current question to the next question; the transition conditions are related to the agent's answer to the current question. In other words, the effective state machine provides the flow path between questions and answers. For example, the candidate question set corresponding to the m-th round question mentioned above can be specifically determined based on the given finite state machine. In this way, the model can further effectively manage the dialogue process and improve the model's inference efficiency.

[0097] Step S402: Obtain the candidate problem set for each of the M initial sample data.

[0098] Here, the initial sample data package contains m rounds of question-and-answer exchanges between the object and the agent; the candidate question set of the initial sample data includes the set of questions for the next round corresponding to the question in the m-th round of question-and-answer exchanges; M and m are both positive integers greater than or equal to 1. It should be noted that the explanation of the initial sample data can be found in the example above, and will not be repeated in detail here.

[0099] Step S403: Based on the M initial sample data, the candidate question set of each initial sample data, and the label data of each initial sample data, obtain the M training sample data.

[0100] Here, the label data of the initial sample data includes the target question that the agent needs to generate in the (m+1)th round.

[0101] Step S404: Use M training sample data to train the model to be trained, so as to obtain a target model that can predict the questions that the agent needs to generate in the next round based on historical question answering.

[0102] In this way, the disclosed solution can obtain initial sample data that meets the requirements based on a finite state machine, and then obtain M training sample data based on the obtained initial sample data to train the model to be trained. Thus, by using a finite state machine to constrain the reasoning process, the dialogue flow can be effectively managed. At the same time, the illusion problem in the model generation process can be effectively avoided, thereby improving the inference speed of the model and effectively enhancing the user experience.

[0103] Furthermore, in a specific example, the finite state machine described above can be determined in the following manner; specifically, before obtaining M initial sample data satisfying at least one of the following conditions based on the determined finite state machine (e.g., before step S401 described above), the method further includes:

[0104] Step S400: Determine the finite state machine based on N historical interaction data between the object and the agent.

[0105] Here, the historical interaction data in the N historical interaction data includes m rounds of question and answer between the object and the agent, and the questions that the agent needs to generate in the (m+1)th round; N is a positive integer greater than or equal to 1.

[0106] For example, Figure 4(b) shows a specific example of a finite state machine. As shown in Figure 4(b), this finite state machine includes four questions generated by the agent, namely Q1-Q5. Also, as indicated by the arrows in Figure 4(b), it includes the flow paths (also called transition paths) between questions. Furthermore, the transition condition from one question to the next is related to the content of the agent's answer to the current question. For example, for question Q1, when the answer to question Q1 is A... 11 If the answer to question Q1 is A, then proceed to question Q2. 12 If the answer is A, then proceed to question Q4. Similarly, for question Q2, if the answer to question Q2 is A... 21 If the answer to question Q2 is A, then proceed to question Q3. 22 If so, proceed to question Q5.

[0107] Here, the set of questions for the next round corresponding to question Q1 may include questions Q2 and Q4; similarly, the set of questions for the next round corresponding to question Q2 may include questions Q3 and Q5.

[0108] Furthermore, the data condition 1 described above will be further explained using the finite state machine shown in Figure 4(b). For the finite state machine shown in Figure 4(b), the number of question-and-answer rounds contained in the historical interaction data (i.e., historical dialogue data), i.e., m, can specifically take the values ​​1 and 2. For example, the historical interaction data is (Q1, A... 12 ;Q4), or, the historical interaction data is (Q1,A 11 Q2,A 22 (Q5), or, historical interaction data is (Q1, A) 11 Q2,A 21Q3), etc. At this time, in order to meet the requirements of data condition 1, in the process of selecting all initial sample data, the number of initial sample data of different m follows the first distribution requirement. For example, the number of initial sample data of m=1 and the number of initial sample data of m=2 need to meet the first distribution requirement.

[0109] Furthermore, the data condition 2 described above is further explained using the finite state machine shown in Figure 4(b). As shown in Figure 4(c), the answer content A for question Q1 transitions to question Q2. 11 This includes various scenarios, including content 1 to content 6. In this case, to avoid long-tail problems, the number of different answer contents that are transferred from question Q1 to question Q2 needs to meet the second distribution requirement. For example, for different answer contents that are transferred from question Q1 to question Q2, the number of content 1, content 2, content 3, content 4, content 5, and content 6 need to meet the second distribution requirement. For example, the number of the above 6 contents needs to be the same.

[0110] Additionally, it should be noted that, as shown in Figure 4(d), the answer content A for question Q1 is transferred to question Q4. 12 This includes various scenarios, such as including content 7 to content 9. In these cases, the number of answers transferred from the same question to different questions can also meet the requirements of the third distribution, for example, it can also meet the requirements of a uniform distribution, such as the number of answers transferred from question Q1 to question Q4, such as content A. 12 The quantity, and the content of the answer A transferred from question Q1 to question Q2. 11 If the number of elements is the same, the long-tail problem of data can be further solved.

[0111] In this way, the initial sample data selected through the above method can effectively solve the problem of long-tail data, thereby effectively avoiding the model from focusing too much on frequently occurring answers and weakening its ability to process some uncommon answers, thus providing effective support for further improving the accuracy of the model's inference results.

[0112] Furthermore, the finite state machine of this disclosed solution is constructed based on historical interaction data, thus providing support for selecting initial sample data that meets the conditions described above, thereby laying the foundation for effective management of the dialogue process, improving the inference speed of the model, and enhancing the user experience.

[0113] Specifically, Figure 5 This is an illustrative flow diagram of a model processing method according to an embodiment of this application. Figure 3 This method can be optionally applied to electronic devices, such as personal computers, servers, and server clusters. It is understood that the above... Figure 1The methods shown in Figure 4 can also be applied to this example, but the related content will not be elaborated further in this example.

[0114] Furthermore, the method includes at least a portion of the following: For example... Figure 5 As shown, it includes:

[0115] Step S501: Obtain the candidate problem set for each of the M initial sample data.

[0116] Here, in this example, the initial sample data packet contains m rounds of question-and-answer exchanges between the object and the agent.

[0117] Furthermore, in this example, the candidate question set of the initial sample data includes the next round question set corresponding to the question in the m-th round of question-and-answer.

[0118] Here, M and m are both positive integers greater than or equal to 1.

[0119] Step S502: Based on M initial sample data, the candidate question set of each initial sample data, and the label data of each initial sample data, obtain M training sample data.

[0120] Here, the label data of the initial sample data includes the target question that the agent needs to generate in the (m+1)th round.

[0121] Step S503: Use M training sample data to train the model to be trained, so as to obtain a target model that can predict the questions that the agent needs to generate in the next round based on historical question answering.

[0122] Step S504: After obtaining the target model, the target output result of the target model is evaluated using at least one evaluation model to obtain a target evaluation result that evaluates the accuracy of the target output result.

[0123] In other words, this disclosed solution, after obtaining the target model, can also use the evaluation model to evaluate the trained target model. This lays the foundation for further improving the inference accuracy of the target model, and at the same time, it also lays the foundation for further improving the user experience.

[0124] It should be noted that the evaluation model used to assess the accuracy of the target output is the model after training, such as the model after training a large language model. Only in this way can the rationality and accuracy of the evaluation results be ensured.

[0125] In a specific example, the evaluation model in the at least one evaluation model satisfies at least one of the following conditions:

[0126] Model condition 1: When the parameters of the evaluation model are adjustable (for example, when the evaluation model is a trainable large language model), the number of adjustable parameters of the evaluation model is greater than the number of adjustable parameters of the target model, and the evaluation model is a model obtained after training based on the M training sample data.

[0127] It should be noted that the training method of the evaluation model can be the same as that of the model to be trained. This allows the evaluation model to have a similar problem generation capability to the target model. Moreover, since the number of adjustable parameters of the evaluation model is greater than that of the model to be trained, the evaluation model has stronger understanding and higher accuracy.

[0128] In one example, both the model to be trained and the model to be evaluated are large language models, and the latter has a larger number of adjustable parameters than the former.

[0129] Furthermore, in a specific example, if the evaluation model is an evaluation model that satisfies model condition 1, then, after obtaining the target model, the target output result of the target model is evaluated using at least one evaluation model to obtain the target evaluation result of the accuracy of the target output result (i.e., step S504 mentioned above), specifically includes:

[0130] After obtaining the target model, and assuming the evaluation model is obtained after training on the M training sample data, the target output of the target model is evaluated based on the evaluation model (i.e., the evaluation model that satisfies model condition 1), to obtain the initial evaluation result corresponding to the evaluation model. Here, the initial evaluation result of the evaluation model is used to measure the accuracy of the target output of the target model.

[0131] Thus, this disclosure provides a specific scheme for evaluating the target output of the target model using an evaluation model that satisfies model condition 1. This scheme can effectively evaluate the target model, thereby laying the foundation for further improving the inference accuracy of the target model and also laying the foundation for further improving the user experience.

[0132] Here, in one example, the initial evaluation result corresponding to the evaluation model is represented by the following value: the likelihood of the target output result output by the target model of the trained evaluation model (e.g., the evaluation model trained based on the M training sample data mentioned above).

[0133] For example, in a specific instance, the likelihood of the target output of the trained evaluation model can be expressed by the following formula:

[0134] here, The evaluation model representing the completed training (denoted as ) The target output result Y of the target model. * The conditional probability. That is, after X is input into the trained evaluation model, the output result is Y. * The conditional probability of Y; here, Y * This represents the target output result after inputting X into the trained target model. For example, in one example, X represents the m rounds of question-and-answer sessions contained in the training sample data, and the set of candidate questions corresponding to the m-th round of questions contained in the training sample data.

[0135] Thus, this disclosure provides a specific scheme for evaluating the target output of the target model using an evaluation model that satisfies model condition 1. Moreover, this scheme is simple and highly interpretable, thereby laying the foundation for further improving the reasoning accuracy of the target model, and also laying the foundation for further improving the user experience.

[0136] Model condition 2: When the parameters of the evaluation model are adjustable, the number of adjustable parameters of the evaluation model is greater than the number of adjustable parameters of the target model, and the evaluation model is obtained after training based on the M training sample data and the pre-constructed evaluation dataset.

[0137] It's important to note that, to further improve the evaluation performance of the evaluation model, it's not enough to simply give it question-generating capabilities. For example, training the evaluation model with the same training sample data as the model to be trained (i.e., the M training sample data mentioned above) will result in a model capable of question generation. Furthermore, it's possible to give the evaluation model discriminative capabilities. For instance, training the evaluation model with a pre-constructed evaluation dataset will also give it discriminative abilities. This further improves the accuracy of the evaluation results, laying the foundation for further improving the inference accuracy of the target model, and simultaneously enhancing the user experience.

[0138] Here, in a specific example, the pre-constructed evaluation dataset contains multiple positive sample data and multiple negative sample data constructed based on the multiple positive sample data;

[0139] The positive sample data in the plurality of positive sample data is obtained based on the initial sample data in the M initial sample data. For example, the positive sample data may specifically include m rounds of question-and-answer sessions and the true question generated by the target model in the (m+1)th round. Further, the negative sample data constructed based on the positive sample data, for example, the negative sample data constructed based on the m rounds of question-and-answer sessions and the true question generated by the target model in the (m+1)th round, includes the m rounds of question-and-answer sessions and the constructed false question in the (m+1)th round. For example, in one example, the constructed evaluation dataset is as follows: For details, please refer to the following statement, which will not be repeated here.

[0140] Thus, this disclosed solution provides a specific method for constructing an evaluation dataset. This construction method is simple and efficient, and enables the evaluation model to not only have the ability to generate questions, but also to have the ability to make judgments. This further improves the performance of the evaluation model and lays the foundation for further improving the reasoning accuracy of the target model and further improving the user experience.

[0141] Furthermore, in a specific example, if the evaluation model is the evaluation model that satisfies the above model condition 2, then, after obtaining the target model, the above-described method of evaluating the target output result of the target model using at least one evaluation model to obtain the target evaluation result of the accuracy of the target output result (i.e., step S504 above) specifically includes:

[0142] After obtaining the target model, and given that the evaluation model is obtained by training based on the M training sample data and the pre-constructed evaluation dataset, the target output result of the target model is evaluated based on the evaluation model (i.e., the evaluation model that satisfies model condition 2), and the first initial result and the second initial result corresponding to the evaluation model are obtained.

[0143] Based on the first and second initial results, the target evaluation results are obtained to assess the accuracy of the target output results.

[0144] Here, the first initial result is used to measure the accuracy of the target model's target output; the second initial result is used to determine the probability that the target model's target output is an accurate value.

[0145] Thus, this disclosure provides a specific scheme for evaluating the target output of the target model using an evaluation model that satisfies model condition 2. This scheme can effectively evaluate the target model and has higher evaluation accuracy, thereby laying the foundation for further improving the inference accuracy of the target model and also laying the foundation for further improving the user experience.

[0146] Furthermore, in one example, the first initial result is represented by the following value: the likelihood of the target output result output by the trained evaluation model; for example, in one example, the likelihood of the target output result output by the trained evaluation model can be expressed by the following formula:

[0147] here, The evaluation model representing the completed training (denoted as ) The target output result Y of the target model. * The conditional probability. That is, after X is input into the trained evaluation model, the output result is Y. * The conditional probability of Y; here, Y * This represents the target output result after inputting X into the trained target model. For example, in one example, X represents the m rounds of question-and-answer sessions contained in the training sample data, and the set of candidate questions corresponding to the m-th round of questions contained in the training sample data.

[0148] And / or, in another example, the second initial result is represented by the following value: the target output of the trained evaluation model is the accurate likelihood of the target output of the target model.

[0149] For example, in a specific instance, the likelihood that the target output of the trained evaluation model is accurate can be expressed by the following formula:

[0150] here, This indicates that after inputting X into the trained evaluation model, the output Y is... * Y is the probability of the correct value. * This represents the target output result after inputting X into the trained target model. Here, in one example, X can represent the m rounds of question-and-answer sessions contained in the training sample data, and the set of candidate questions corresponding to the m-th round of questions contained in the training sample data.

[0151] Thus, this disclosure provides a specific scheme for evaluating the target output of the target model using an evaluation model that satisfies model condition 2. Moreover, the scheme has higher accuracy in evaluation results, and is simple and highly interpretable. This lays the foundation for further improving the inference accuracy of the target model, and also lays the foundation for further improving the user experience.

[0152] Furthermore, in a specific example, when the evaluation model possesses both problem generation and discrimination capabilities, i.e., when the evaluation model satisfies model condition 2 as described above, the target evaluation result can also be obtained in the following manner: that is, the target evaluation result based on the first and second initial results, which assesses the accuracy of the target output result, specifically includes:

[0153] The first and second initial results are weighted to obtain the target evaluation result that assesses the accuracy of the target output.

[0154] For example, the first initial result is The second initial result is P. In this case, the target evaluation result can be specifically represented as follows:

[0155]

[0156] Here, α can be an empirical value. In this case, the above formula can represent the confidence level of the trained evaluation model in the target output of the trained target model.

[0157] Thus, this disclosed solution provides a specific scheme for evaluating the target output of the target model using an evaluation model that satisfies model condition 2. Moreover, the scheme has higher accuracy in evaluation results, and is simple and highly interpretable. This lays the foundation for further improving the inference accuracy of the target model, and also lays the foundation for further improving the user experience.

[0158] Model condition 3: When the parameters of the evaluation model are not adjustable, the number of adjustable parameters of the evaluation model is greater than the number of adjustable parameters of the target model.

[0159] It should be noted that this disclosed solution can also select a Black-box LLM with greater generality and adjustable parameters (such as GPT-4) as the evaluation model. In this case, although the Black-box LLM is not trainable, it can still serve as a fair judge due to its strong understanding ability and rich knowledge. This can be used to evaluate the accuracy of the target model's target data results, thereby further improving the accuracy of the evaluation results and laying the foundation for further improving the inference accuracy of the target model. At the same time, it also lays the foundation for further improving the user experience.

[0160] In a specific example of this disclosed solution, when using two or more evaluation models for evaluation, such as using an evaluation model that satisfies model condition 2 and an evaluation model that satisfies model condition 3, a total evaluation result of the accuracy of the target output result can be obtained based on the target evaluation results of each evaluation model. This effectively compensates for the erroneous evaluation problem caused by the evaluation model having the same distribution of training sample data as the target model. Furthermore, the M training sample data can be corrected based on the total evaluation result to obtain corrected M training sample data. For example, if both the evaluation model that satisfies model condition 2 and the evaluation model that satisfies model condition 3 consider the target output result of the trained target model to be correct, the training sample data corresponding to the target output result can be retained. Otherwise, the training sample data with incorrect target output results can be corrected, thereby improving the accuracy of the training sample data and further increasing the amount of training sample data.

[0161] Furthermore, in a specific example, the target model can be fine-tuned based on the modified M training samples to obtain a modified target model, thereby further improving the inference accuracy of the target model and enhancing the user experience; and / or, the parameter-adjustable evaluation model can be fine-tuned based on the modified M training samples to obtain a modified evaluation model. For example, the evaluation model to be trained can be trained based on the modified M training samples and the modified evaluation dataset (based on the modified M training samples, modifying the pre-constructed evaluation dataset), thereby further improving its generation capabilities and discrimination skills, thus laying the foundation for further improving the accuracy of evaluation results, the inference accuracy of the target model, and the user experience.

[0162] Specifically, Figure 6(a) is a schematic flowchart of a voice interaction method according to an embodiment of this application. This method can be optionally applied to electronic devices, such as personal computers, servers, server clusters, and other electronic devices.

[0163] Furthermore, the method includes at least a portion of the following. As shown in Figure 6(a), it includes:

[0164] Step S601: Obtain the target text data.

[0165] Here, the target text data is obtained from the object's voice response to the question in the previous round between the object and the agent.

[0166] Here, in a specific example, the target text data can be obtained in the following way: obtain the audio response to the object in the previous round of questions; and convert the audio response to the object in the previous round of questions into target text data.

[0167] Thus, this disclosed solution provides a specific scheme for dialogue between an object and an intelligent agent. This scheme is simple, efficient, applicable to a wide range of scenarios, and compatible with existing schemes, thereby laying the foundation for further enriching and improving the user experience.

[0168] Step S602: Obtain the next round of candidate questions for the question addressed by the target text data.

[0169] Here, in a specific example, the set of candidate questions for the next round of questions addressed by the target text data is determined based on a finite state machine. This finite state machine can represent a set of questions generated by the agent, and the transition conditions from the current question to the next question; the transition conditions are related to the agent's answer to the current question. For details regarding finite state machines, please refer to the above description, which will not be repeated here. Thus, since this disclosure provides a set of candidate questions for the next round, the target model can reason within a given range. This effectively avoids the illusion problem in the target model's generation process, while also reducing the length of the generated questions, thereby improving the reasoning speed of the target model and enhancing the user experience.

[0170] Step S603: Input the target text data and the next set of candidate questions for the question addressed by the target text data into the target model to obtain the question text to be generated by the agent. That is, obtain the question text for the question to be generated by the agent.

[0171] Here, in one example, the target model is obtained by training with the M training sample data described above; further, it is obtained by training with the M training sample data described above and the training method described above.

[0172] In this way, the disclosed solution can utilize the target model to infer the question that the agent needs to generate in the next round of dialogue, given the most recent dialogue, for example, based on the understanding of the answers in the most recent dialogue, or based on the understanding of the most recent dialogue. In this way, the dialogue needs between the object and the agent are met, enriching the user experience and improving the user experience.

[0173] Moreover, since the proposed solution provides a set of candidate questions for the next round during the reasoning process, the target model is able to reason within a given range, thus achieving dialogue management. At the same time, it can effectively avoid the illusion problem in the target model during the generation process, reduce the length of generated questions, thereby improving the reasoning speed of the target model and further enhancing the user experience.

[0174] Furthermore, in a specific example, after obtaining the question text, it can be converted into spoken question and then output, thus enabling free dialogue between the object and the intelligent agent. Moreover, this process is simple, efficient, applicable to a wide range of scenarios, and compatible with existing solutions, thereby further enriching and enhancing the user experience.

[0175] For example, Figure 6(b) shows a schematic diagram of a reasoning scenario. As shown in Figure 6(b), the speech recognition module can obtain the speech of the target object's response to the question generated by the agent in the previous round. The speech recognition module can convert the speech of the question into target text data and input the set of candidate questions for the next round corresponding to the question in the previous round into the target model. Based on the understanding of the question and answer in the previous round, the target model generates the question text that the agent needs to generate in the next round. The speech synthesis model converts the question text generated by the target model into question speech to output the question speech. In this way, the intelligent dialogue process between the object and the agent is realized.

[0176] The following provides further explanation of this disclosure with specific examples; this disclosure proposes an intelligent questioning scheme for intelligent voice assistants based on a Large Language Model (LLM, hereinafter referred to as LLM or large model). It leverages the powerful language understanding, scheduling, and generation capabilities of the large model to achieve end-to-end dialogue control, compared to modular solutions in existing technologies (such as...). Figure 1 The modular solution shown in this disclosure replaces the processing of multiple modules (such as NLU, DM, NLG, etc.) in traditional voice assistants with single-module processing of LLM. This can effectively reduce maintenance costs and service update costs. At the same time, this disclosure solution also effectively improves the accuracy of output results, thereby effectively improving the user experience.

[0177] Before introducing this disclosed solution, the following concepts are defined for the purpose of simplification:

[0178] Let Q be the question generated by the agent; A be the answer from the object; and C be the interaction data (dialogue) between the agent and the object. Then, m rounds of question-and-answer (m rounds of dialogue) can be specifically represented as:

[0179] C [1,m]=(Q1,A1;Q2,A2;...;Q k A k ;…;Q m A m )

[0180] Here, Q k Let A represent the question generated by the agent in the k-th round of question-and-answer (or the k-th round of dialogue). k This represents the object's response in the k-th round of question and answer.

[0181] Furthermore, the objective of this disclosed solution is: given m rounds of historical dialogue between agents and objects, i.e., m rounds of question-and-answer C... [1,m] This enables LLM to predict the question that the agent needs to generate in the next round (i.e., the (m+1)th round) of dialogue, namely question Q. m+1 .

[0182] The following three aspects will be specifically explained: enhancement of LLM training data, model training and implementation of dialogue management, and continuous learning to improve the effectiveness of LLM.

[0183] I. Augmentation of LLM Training Data

[0184] (I) Preparation of Historical Dialogue Data

[0185] Training data is the foundation for building intelligent voice assistants. In one example, historical dialogue data (i.e., historical interaction data) can be prepared in the following ways:

[0186] (1) The dialogue data between the agent and the object is manually synthesized by humans.

[0187] (2) Collect the communication logs between the agent and the object to obtain dialogue data.

[0188] (II) Training Data Augmentation

[0189] In one example, historical dialogue data can be segmented according to the position of the object's response (e.g., to obtain the above m-round question-and-answer C). [1,m] ), and the agent's next question (e.g., Q) m+1 Using this as tag data, multiple historical dialogue data can be obtained through this segmentation method.

[0190] It should be noted here that if the historical dialogue data between the agent and the object is directly used to train the LLM, such as directly using m rounds of question-and-answer C... [1,m] As input to the model to be trained, the agent's (m+1)th problem Q is... m+1Using labeled data to train a model can cause the model to overemphasize frequently occurring dialogues while weakening its ability to handle less common dialogues because:

[0191] (1) The dialogue length in the training data follows a long-tail distribution, with shorter dialogues appearing more frequently and longer dialogues appearing less frequently.

[0192] (2) The responses (i.e., the answers) of the objects in the training data follow a long-tail distribution because some responses are very frequent while others are very infrequent.

[0193] Therefore, this disclosed solution focuses on how to improve the training data of LLM through data augmentation to solve the above problems. Specifically:

[0194] First, based on the collected historical dialogue data, the structure of the finite state machine is organized. For example, in one example, the specific form of the finite state machine is defined as follows:

[0195] FSM={S,Σ,δ,s0,F},

[0196] Here, S represents the set of states, specifically the set of questions generated by the agent in all historical dialogue data in this example, i.e., S = {Q}. k}; Σ represents the set of all responses from objects in the historical dialogue data, i.e., Σ={A k};δ:S×Σ→S, representing the transfer function, specifically used in this example to indicate that based on the questions generated by the agent in the previous round, the agent determines the questions to be generated in the next round according to the object's answer; for example, δ i Q m ×A m →Q m+1 ; s0 represents the initial state; F represents the set of final states, which is a subset of S. For an example of a specific state machine, please refer to Figure 4(b), which will not be elaborated upon here.

[0197] Furthermore, based on a finite state machine, data that meets the following distribution requirements is selected from the prepared set of historical dialogue data to serve as the initial sample data after augmentation:

[0198] Condition 1: In all the initial sample data, the distribution of object responses (i.e., answer content) during transitions between different states (i.e. different questions) conforms to a uniform distribution.

[0199] Condition 2: In all the initial sample data, the distribution of different dialogue rounds conforms to a uniform distribution.

[0200] In other words, all the selected initial sample data, such as the M initial sample data mentioned above, satisfy the two conditions described above.

[0201] Specifically, in one example, the j-th initial sample data among the selected M initial sample data can be represented by x. j This indicates that the j-th initial sample data x j Including the first m rounds of question and answer, its expression can be specifically as follows:

[0202]

[0203] x j ~p j (x),

[0204] Here, p j (x) represents the data distribution of different initial training samples.

[0205] II. Model Training and Dialogue Management

[0206] The goal of this section is to leverage large models for dialogue management, that is, given historical dialogues, to enable large models to accurately generate appropriate dialogue content.

[0207] Generally, the data augmented sample data (i.e., the initial sample data mentioned above) can be used directly for training. For example, using m rounds of question-answering C... [1,m] The problem asked by the agent in the (m+1)th round is used as label data for model training. However, this training scheme has the following problems:

[0208] To ensure that the responses from intelligent voice assistants meet latency requirements, a low-parameter LLM (Limited Linear Model) with fast inference speed is used. Furthermore, a self-training (SFT) method is employed to train the low-parameter LLM. This training process has some insecurity factors, such as instability and error-proneness when generating longer texts. In addition, all LLMs suffer from a flaw called the illusion problem, which makes the above training process prone to producing unsafe outputs, severely degrading the user experience.

[0209] In view of this, this disclosure proposes a selective generation method to solve the above problems. Specifically, it includes:

[0210] First, by combining the finite state machine, all possible problems for the agent in the next round are determined and formed into a candidate set (i.e., the candidate problem set mentioned above). For details regarding the candidate set, please refer to the description above; it will not be repeated here.

[0211] Secondly, LLM analyzes the input historical dialogues and selects the best option from the candidate set as the question to be generated in the next round. In this way, compared with LLM directly generating complete questions, it can effectively avoid the illusion problem in the LLM generation process, while also reducing the length of the generated questions and improving the reasoning speed of LLM.

[0212] Finally, this disclosed solution can also use Chain-of-Thought (CoT) technology to enhance the LLM's responses. Specifically, when constructing training sample data, this disclosed solution can also use the intent expressed by the question-and-answer content in the most recent round of dialogue as reasoning, and provide the agent with a suitable candidate set for the next round. In this way, it can solve the problem that the selective generation method is difficult to stably perform SFT due to the limited number of supervised signals.

[0213] In this example, the initial training set, which contains initial sample data and label data, can be represented as follows: denoted as the initial training set.

[0214]

[0215] Here, M represents the total number of all training sample data; x j C represents the m rounds of question-and-answer interaction between the object and the agent in the j-th initial sample data. [1,m] ;y j Let Q represent the problem that the agent needs to generate in the next round from the j-th initial sample data. m+1 During the training process, the y j It can be used as tag data.

[0216] Furthermore, for a given set of data The input to the model to be trained, such as the LLM to be trained, can be defined as:

[0217] X = prompt(x j (ReplyOptions)

[0218] Here, ReplyOptions represents the set of questions for the next round corresponding to the question in the m-th round from the j-th initial sample data, which is also the set of candidate questions corresponding to the question in the m-th round; prompt(·) represents the prompt word function, used to set x j Together with ReplyOptions (i.e., the candidate set mentioned above), they form a natural language representation so that the LLM to be trained can understand it.

[0219] Furthermore, the output of the model to be trained, such as the LLM to be trained (i.e., the initial estimation result mentioned above), can be defined as:

[0220]

[0221] Here, Θ represents the adjustable parameters of the LLM to be trained, specifically the vector composed of the adjustable parameters.

[0222] Furthermore, the label data (i.e., the real labels) corresponding to the input X of the LLM can be defined in the following form:

[0223] Y = {Chain of Thought, y j correspondingoption}

[0224] Here, Chain-of-Thought represents the user's intent in the previous round. For example, it represents the understanding of the object's answer in the previous round, such as the understanding of the answer in round m. Alternatively, it can represent the understanding of the question and answer in the previous round, such as the understanding of the question and answer in round m.

[0225] Based on this, the training sample data can be specifically represented as (X, Y).

[0226] Furthermore, the training objective can be specifically defined as minimizing the distance between the LLM output and the true label:

[0227]

[0228] Furthermore, during the inference phase, the trained LLM can infer the question that the agent needs to generate in the next round based on its understanding of the answers in the most recent round of dialogue, or its understanding of the most recent round of dialogue.

[0229] III. Model Reasoning

[0230] Here, it should be noted that during the inference phase: the answers from the most recent dialogue and the set of candidate questions corresponding to the questions in the most recent dialogue can be organized into prompt questions and then input into the large model after training. In this way, the large module can infer the questions that the agent needs to generate in the next round based on its understanding of the answers from the most recent dialogue or its understanding of the most recent dialogue.

[0231] IV. Continuous learning to improve LLM effectiveness

[0232] To further improve the performance of the trained LLM, this disclosure also proposes a collaborative iterative scheme.

[0233] First, it's important to clarify that this publicly disclosed approach categorizes large models into three types: LLM-S, LLM-L, and Black-box LLM. Here, both LLM-S and LLM-L represent finely tuned large models. LLM-S indicates a large model with a relatively small number of adjustable parameters (e.g., less than 2 billion parameters), while LLM-L indicates a large model with a relatively large number of adjustable parameters (e.g., more than 2 billion parameters). In one example, the model structures of LLM-S and LLM-L can be similar, differing only in the number of parameters. Understandably, due to the larger number of adjustable parameters in LLM-L, it performs better than LLM-S, but its inference speed is slower. Black-box LLM, on the other hand, represents closed-source models with extremely large parameter counts, such as GPT-4. These models are very powerful, possessing a wealth of world knowledge, but do not support parameter fine-tuning.

[0234] Secondly, it should be noted that in the intelligent voice assistant scenario, in order to meet the low latency requirement, the proposed solution deploys an LLM-S with a small number of parameters as the underlying model (that is, the model to be trained mentioned above is an LLM-S). Moreover, in order to further improve the performance of the trained LLM-S, the proposed solution will utilize LLM-L and Black-box LLM, along with a small amount of human feedback, and through continuous iteration, to identify and correct the shortcomings of the online model (the trained LLM-S, that is, the target model mentioned above), thereby assisting the online LLM-S (that is, the trained LLM-S) in improving its performance.

[0235] Specifically, the continuous learning process of this disclosed solution includes two core steps: a data enhancement step and an inference enhancement step. For example, through this continuous learning process, the training sample data mentioned above can be optimized to further improve the data quality of the training sample data, thereby achieving data enhancement. On the other hand, the improved training sample data can be used to fine-tune the trained model (e.g., the trained LLM-S, i.e., the target model mentioned above) to further improve the inference performance. Additionally, the improved training sample data can also be used to fine-tune the evaluation model to further improve the evaluation performance of the evaluation model.

[0236] Before introducing the continuous learning process, the following explanation is provided:

[0237] First, this continuous learning process optimizes the training sample data enhanced as described above, and the model obtained after training LLM-S using the training methods described above; here, the trained LLM-S can be denoted as...

[0238] Second, in this continuous learning process, the candidate dataset for the data enhancement stage is defined as follows. For example, in the data enhancement stage, the candidate dataset used in the t-th iteration is defined as follows:

[0239]

[0240] Here, x j For the j-th initial sample data mentioned above, it includes the first m rounds of question and answer; y j ′ indicates that x j The input is the output of the LLM-S after training, that is... The output results. In other words, the input data used in the data enhancement step includes the initial sample data mentioned above, while the label data is the output result of the trained model, i.e. The output results. Thus, an evaluator, such as an evaluation model, can be used to evaluate... The accuracy of the output results.

[0241] Understandably, the input data used in the data enhancement step can also include the set of questions for the next round corresponding to the most recent round of questions in the initial sample size. For example, for the initial sample data containing the first m rounds of question-and-answer C... [1,m] In this case, the input data used in the data enhancement step also includes the set of questions for the next round corresponding to the questions in the m-th round. That is to say, in this continuous learning phase, the original processing mode of the model is followed, and the model performance is improved without changing the original processing mode.

[0242] Third, before proceeding with the continuous learning process, such as before the data enhancement process, it is necessary to pre-construct an evaluator, such as an evaluation model, to evaluate the output of the LLM-S after training, for example, on the candidate datasets mentioned above. Accuracy assessment is performed. If problematic data is identified, it can be corrected manually, further refining the enhanced training sample data to achieve further data augmentation. Here, the set of corrected training sample data can be denoteed. for:

[0243]

[0244] here, This represents the corrected initial sample data x as described above, specifically the j-th initial sample data. j , This indicates the corrected y as described above. j That is, the modified Q m+1 .

[0245] It should be noted here that if the pre-constructed evaluation model is a model with adjustable parameters, then the evaluation model used in this data enhancement step is the model after training.

[0246] Fourth, in the strategy enhancement phase, this disclosed solution can also be based on the modified set of training sample data. This will further enhance the generation capability of trained LLM-S models and the evaluation capability of evaluation models.

[0247] In this way, this disclosed solution can continuously improve the online LLM-S capabilities by alternately executing data improvement steps and strategy improvement steps.

[0248] The following combination Figure 7 The data enhancement steps and inference enhancement steps are explained in detail:

[0249] (I) Data Enhancement Phase

[0250] In the data enhancement phase, it is crucial to develop an effective evaluator. Therefore, this disclosed solution selects an LLM-L with adjustable parameters and a large number of parameters as the evaluation model. At the same time, it also selects the general Black-box LLM as the evaluation model and combines the evaluation results of the two to optimize the training sample data in order to achieve data enhancement.

[0251] It should be noted that before using LLM-L to evaluate the trained LLM-S, LLM-L needs to be trained in advance. For example, LLM-L can be trained using the same training sample data as LLM-S (e.g., the M training sample data mentioned above) so that LLM-L learns the relevant domain knowledge and becomes a domain expert. At this time, since LLM-L has a larger number of parameters, it can better understand the specific business scenario.

[0252] Furthermore, considering that the training sample data used by LLM-L and LLM-S overlaps, errors in the training data will be fitted into the model. In this case, a general Black-box LLM can be introduced as an evaluation model to conduct a fair evaluation. Although the evaluation model Black-box LLM cannot be directly fine-tuned, it has strong general knowledge and can make some objective evaluations. In this way, the erroneous evaluation problem caused by the consistent distribution of training sample data between LLM-L and LLM-S can be effectively made up for.

[0253] (1) Evaluation Model - LLM-L

[0254] Understandably, given the same training sample data, LLM-L is expected to outperform LLM-S in performance. Therefore, by using the training sample data used to train LLM-S to fine-tune the parameters of LLM-L, LLM-L can also have good generative capabilities. In other words, after training LLM-L and LLM-S with the same training sample data, given the same input, the trained LLM-L is more likely to generate correct results than the trained LLM-S.

[0255] Here, the trained LLM-L can be denoted as... Θ′ represents the adjustable parameters of LLM-L, which can specifically represent the vector of adjustable parameters in the trained LLM-L.

[0256] Furthermore, the trained LLM-L (i.e., To evaluate the accuracy of the output of a trained LLM-S, for example, the likelihood of the trained LLM-L producing the output of a trained LLM-S can be used to measure the accuracy of the trained LLM-S output; the likelihood of the trained LLM-L producing the output of a trained LLM-S (which can be denoted as...) The first initial result described above can be expressed by the following formula:

[0257]

[0258] This indicates that the LLM-L training is complete (i.e., ... The conditional probability of the LLM-S output after training is completed is given by X = prompt(x). j The input (ReplyOptions) is fed into the trained LLM-L, and the output is Y. * The probability of Y; here, Y * This means that X = prompt(x) j The input (ReplyOptions) is fed into the output of the trained LLM-S.

[0259] Furthermore, Indicates the output result Y * The i-th token in; g(v|X) represents the probability that the output of the trained LLM-L is v (referring to a token), and its calculation formula is as follows:

[0260]

[0261] in, This represents the logical values ​​(logits) of the output v given an input X; V represents a predefined vocabulary. g(v|X) is equivalent to... Normalization was performed.

[0262] Furthermore, to further enhance the evaluation capability of the LLM-L evaluation model, it is also possible to train the LLM-L evaluation model from the perspective of decision-making, thereby further improving the evaluation capability of LLM-L.

[0263] Specifically, for each data It can be formally represented as a positive sample of (X,Y), while generating a different form than y′. j The problem is to construct negative samples. In this way, a new dataset (also known as an evaluation dataset) is obtained. The specific format is as follows:

[0264]

[0265] Furthermore, based on the (X,Y) and... contained in the evaluation dataset The evaluation prompt is obtained and input into the LLM-L, and the corresponding labeled data (e.g., the label data of positive samples is 1, and the label data of negative samples is 0) is used to train the LLM-L for the evaluation task, thus obtaining the trained LLM-L.

[0266] After fine-tuning the parameters of the LLM-L using the evaluation dataset and the training sample data of the LLM-S, the trained LLM-S can be obtained. At this point, the trained LLM-S not only has evaluation capabilities but also discriminative capabilities. For example, based on the trained LLM-S, the input X and output Y can be obtained. * The correct likelihood (can be denoted as) Corresponding to the second initial result mentioned above, its definition is as follows:

[0267]

[0268] This indicates that after inputting X into the trained LLM-S, the output Y is... * Y is the probability of the correct value. * This represents the output result after inputting X into the trained LLM-S.

[0269] Furthermore, X e =prompt e (X,Y * ) indicates that X and Y * The resulting prompt word problem; (l|X e This indicates that after inputting X into the trained LLM-L, the output Y is... * Logits are logical values ​​of 1 (0 or 1).

[0270] It should be noted that, in one example, the parameters of the LLM-L can be fine-tuned using the training sample data used to train the LLM-S. Then, the accuracy of the output results of the trained LLM-S can be evaluated using the trained LLM-L. For example, using the P mentioned above... This allows for the evaluation of the accuracy of the LLM-S output after training, which can then be used to optimize the training sample data.

[0271] Alternatively, in another example, the training sample data for training the LLM-S and the evaluation dataset constructed above can be used simultaneously. Fine-tuning the LLM-L can further improve the accuracy of the evaluation results.

[0272] Furthermore, in one example, the evaluation results from the above two perspectives can be combined to obtain the confidence level of the trained LLM-L over the output of the trained LLM-S:

[0273]

[0274] Here, α is an empirical value; or it is a hyperparameter whose specific value can be determined through iteration. For example, if α exceeds a preset threshold, the output of the trained LLM-S can be considered correct.

[0275] (2) Evaluation Model - Black-box LLM

[0276] Black-box LLMs can serve as impartial judges, unaffected by the current training data distribution. For example, they can be used to evaluate the output of a trained LLM-S through a designed prompt.

[0277] (3) Collaborative voting mechanism

[0278] for Each data point can be used for co-evaluation by integrating the evaluation results of the trained LLM-L and Black-box LLM. For example, if both the trained LLM-L and Black-box LLM consider the output of the trained LLM-S to be correct, then that data is retained; otherwise, it can be manually reviewed and errors corrected. This correction step yields a corrected dataset.

[0279] (II) Strategy Enhancement Phase

[0280] During this phase, the revised dataset can be used. This allows for further fine-tuning of the trained LLM-S and the trained evaluation model. For example, using the revised dataset. Fine-tuning the trained LLM-S yields the corrected LLM-S. As for the trained LLM-L,

[0281] The modified data set can be used To optimize the evaluation dataset to obtain the optimized evaluation dataset. And use the optimized dataset and the optimized evaluation dataset We can continue to fine-tune the trained LLM-L to further improve its generation and discrimination capabilities. Additionally, we can enhance the evaluation capabilities of the Black-box LLM by updating the prompt.

[0282] Based on this, the iterative approach of this disclosed solution can ensure that the performance of the entire online LLM-S is continuously improved, and the performance of the evaluation model used will also be continuously improved.

[0283] It should be noted that this disclosed solution can be applied to the field of intelligent voice assistants. Based on this solution, an LLM-based intelligent voice assistant can be built from scratch, or an LLM-based AI native transformation can be achieved on the basis of an existing modular voice assistant. Furthermore, it should be noted that this disclosed solution is not limited to intelligent voice assistant scenarios and can also be applied to other fields; that is, this disclosed solution has good versatility and can be adapted to other application scenarios.

[0284] The method proposed in this disclosure, when applied to projects involving map POI attribute acquisition, can increase the success rate of POI attribute recovery by 4 percentage points.

[0285] This disclosure also provides a model processing device, such as... Figure 8 As shown, it includes:

[0286] The data processing unit 801 is used to obtain a candidate question set for each of the M initial sample data sets, wherein the initial sample data sets contain m rounds of question-and-answer interactions between the object and the agent; the candidate question set for the initial sample data sets includes the set of questions for the next round corresponding to the question in the m-th round of question-and-answer interactions; M and m are both positive integers greater than or equal to 1; based on the M initial sample data sets, the candidate question sets for each initial sample data set, and the label data for each initial sample data set, M training sample data sets are obtained, wherein the label data for the initial sample data sets includes the target question that the agent needs to generate in the (m+1)-th round;

[0287] The model training unit 802 is used to train the model to be trained using M training sample data to obtain a target model that can predict the questions that the agent needs to generate in the next round based on historical question answers.

[0288] In a specific example of the disclosed solution, the model training unit is specifically used for:

[0289] The m rounds of question-and-answer data contained in the training sample data, and the candidate question set corresponding to the m-th round question contained in the training sample data, are input into the model to be trained to obtain the initial estimation result, where the initial estimation result represents the predicted question that the agent needs to generate in the (m+1)-th round.

[0290] Based on the initial estimation results and the target problem that the agent needs to generate in the (m+1)th round contained in the label data of the training sample data, the loss value of the loss function is obtained, whereby the loss function can characterize the distance between the predicted problem and the target problem;

[0291] Based on the loss value of the loss function, at least some of the adjustable parameters in the model to be trained are adjusted to train and obtain the target model.

[0292] In a specific example of the disclosed solution, the model training unit is specifically used for:

[0293] Based on the m rounds of question-and-answer sessions contained in the training sample data, and the set of candidate questions corresponding to the m-th round of questions contained in the training sample data, the target prompt word question is obtained;

[0294] Input the target prompt word question into the model to be trained.

[0295] In a specific example of the disclosed solution, the data processing unit is further configured to:

[0296] Based on the determined finite state machine, the M initial sample data satisfying at least one of the following conditions are obtained:

[0297] Among the M initial sample data, the distribution of the number of initial sample data of different lengths satisfies the first distribution requirement. The length of the initial sample data is determined based on the finite state machine and represents the number of question-and-answer rounds.

[0298] In the M initial sample data, the distribution of the answers from the previous question to the next question satisfies the second distribution requirement;

[0299] The finite state machine can represent a set of questions generated by the agent, and the transition conditions from the current question to the next question; the transition conditions are related to the content of the agent's answer to the current question.

[0300] In a specific example of the disclosed solution, the data processing unit is further configured to:

[0301] The finite state machine is determined based on N historical interaction data between the object and the agent; wherein, the historical interaction data in the N historical interaction data includes m rounds of question and answer between the object and the agent, and the question to be generated by the agent in the (m+1)th round; and N is a positive integer greater than or equal to 1.

[0302] Based on the determined finite state machine, select M initial sample data from the N historical interaction data that satisfy at least one of the following conditions.

[0303] In a specific example of the scheme disclosed herein, the model processing apparatus further includes the model evaluation unit, wherein the model evaluation unit is used for:

[0304] After obtaining the target model, the target output result of the target model is evaluated using at least one evaluation model to obtain the target evaluation result of the accuracy of the target output result.

[0305] Wherein, the evaluation model in the at least one evaluation model satisfies at least one of the following conditions:

[0306] When the parameters of the evaluation model are adjustable, the number of adjustable parameters of the evaluation model is greater than the number of adjustable parameters of the target model, and the evaluation model is a model obtained after training based on the M training sample data.

[0307] When the parameters of the evaluation model are adjustable, the number of adjustable parameters of the evaluation model is greater than the number of adjustable parameters of the target model, and the evaluation model is obtained after training based on the M training sample data and the pre-constructed evaluation dataset.

[0308] When the parameters of the evaluation model are not adjustable, the number of adjustable parameters of the evaluation model is greater than the number of adjustable parameters of the target model.

[0309] In a specific example of the scheme disclosed herein, the pre-constructed evaluation dataset includes multiple positive sample data and multiple negative sample data constructed based on the multiple positive sample data;

[0310] The positive sample data in the plurality of positive sample data is obtained based on the initial sample data in the M initial sample data, including m rounds of question and answer, and the true questions generated by the target model in the (m+1)th round;

[0311] The negative sample data constructed based on the positive sample data includes m rounds of question and answer, as well as a fake question constructed in the (m+1)th round.

[0312] In a specific example of the disclosed solution, the model evaluation unit is specifically used for:

[0313] After obtaining the target model, and given that the evaluation model is obtained after training based on the M training sample data, the target output result of the target model is evaluated based on the evaluation model to obtain the initial evaluation result corresponding to the evaluation model. The initial evaluation result of the evaluation model is used to measure the accuracy of the target output result of the target model.

[0314] In a specific example of the scheme disclosed herein, the initial evaluation result corresponding to the evaluation model is represented by the following value: the likelihood of the target output result output by the target model after the evaluation model has been trained.

[0315] In a specific example of the disclosed solution, the model evaluation unit is specifically used for:

[0316] After obtaining the target model, and given that the evaluation model is obtained by training based on the M training sample data and the pre-constructed evaluation dataset, the target output result of the target model is evaluated based on the evaluation model to obtain the first initial result and the second initial result corresponding to the evaluation model.

[0317] Based on the first and second initial results, the target evaluation result for assessing the accuracy of the target output is obtained;

[0318] The first initial result is used to measure the accuracy of the target model's target output; the second initial result is used to determine the probability that the target model's target output is an accurate value.

[0319] In a specific example of the scheme disclosed herein, the first initial result is represented by the following value: the likelihood of the target output result output by the target model after the evaluation model has been trained;

[0320] And / or, the second initial result is represented by the following value: the target output of the trained evaluation model is the accurate likelihood of the target output of the target model.

[0321] In a specific example of the disclosed solution, the model evaluation unit is specifically used for:

[0322] The first and second initial results are weighted to obtain the target evaluation result that assesses the accuracy of the target output.

[0323] In a specific example of the scheme disclosed herein,

[0324] The model evaluation unit is also used to obtain a total evaluation result of the accuracy of the evaluation target output result based on the target evaluation results of each evaluation model when two or more evaluation models are used for evaluation.

[0325] The data processing unit is further configured to correct the M training sample data based on the overall evaluation result to obtain corrected M training sample data.

[0326] In a specific example of the scheme disclosed herein, the model training unit is further configured to:

[0327] Based on the corrected M training sample data, the target model is fine-tuned to obtain the corrected target model;

[0328] And / or,

[0329] Based on the corrected M training sample data, the parameter-adjustable evaluation model is fine-tuned to obtain the corrected evaluation model.

[0330] For a description of the specific functions and examples of each unit of the model processing apparatus in this embodiment, please refer to the relevant descriptions of the corresponding steps in the above method embodiments, which will not be repeated here.

[0331] This disclosure also provides a voice interaction device, such as Figure 9 As shown, it includes:

[0332] The acquisition unit 901 is used to acquire target text data, wherein the target text data is obtained based on the object's answer voice in the previous round of questions between the object and the agent; and to acquire a set of candidate questions for the next round of questions that the target text data addresses.

[0333] The model prediction unit 902 is used to input the target text data and the next round of candidate questions for the question addressed by the target text data into the target model to obtain the question text to be generated by the agent.

[0334] In a specific example of the disclosed solution, the voice interaction device further includes: a voice synthesis unit; wherein, the voice synthesis unit is used for:

[0335] The obtained question text is converted into question speech, and the question speech is output.

[0336] In a specific example of the disclosed solution, the voice interaction device further includes: a voice recognition unit; wherein, the voice recognition unit is used for:

[0337] Obtain the audio of the response from the person in the previous round of questions;

[0338] Convert the speech of the object's answer to the question in the previous round into target text data.

[0339] In the voice interaction device, the next set of candidate questions for the question addressed by the target text data is determined based on a finite state machine. The finite state machine can represent a set of questions generated by the agent, as well as the transition conditions from the current question to the next question. The transition conditions are related to the content of the object's answer to the current question.

[0340] For a description of the specific functions and examples of each unit of the voice interaction device in this embodiment, please refer to the relevant descriptions of the corresponding steps in the above method embodiments, which will not be repeated here.

[0341] The acquisition, storage, and application of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0342] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0343] Figure 10 A schematic block diagram of an example electronic device 1000 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0344] like Figure 10As shown, device 1000 includes a computing unit 1001, which can perform various appropriate actions and processes according to a computer program stored in read-only memory (ROM) 1002 or a computer program loaded from storage unit 1008 into random access memory (RAM) 1003. The RAM 1003 may also store various programs and data required for the operation of device 1000. The computing unit 1001, ROM 1002, and RAM 1003 are interconnected via bus 1004. Input / output (I / O) interface 1005 is also connected to bus 1004.

[0345] Multiple components in device 1000 are connected to I / O interface 1005, including: input unit 1006, such as keyboard, mouse, etc.; output unit 1007, such as various types of monitors, speakers, etc.; storage unit 1008, such as disk, optical disk, etc.; and communication unit 1009, such as network card, modem, wireless transceiver, etc. Communication unit 1009 allows device 1000 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0346] The computing unit 1001 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 1001 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1001 performs the various methods and processes described above, such as model processing methods or voice interaction methods. For example, in some embodiments, the model processing method or voice interaction method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 1008. In some embodiments, part or all of the computer program may be loaded and / or installed on device 1000 via ROM 1002 and / or communication unit 1009. When the computer program is loaded into RAM 1003 and executed by the computing unit 1001, one or more steps of the model processing method or voice interaction method described above may be performed. Alternatively, in other embodiments, the computing unit 1001 may be configured to perform a model processing method or a voice interaction method by any other suitable means (e.g., by means of firmware).

[0347] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0348] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0349] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0350] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0351] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0352] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0353] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0354] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A model processing method, comprising: A candidate question set is obtained for each of the M initial sample data sets, wherein the initial sample data set contains m rounds of question-and-answer between the object and the agent; the candidate question set of the initial sample data set includes the set of next-round questions corresponding to the m-th round question in the m rounds of question-and-answer; M and m are both positive integers greater than or equal to 1; the M initial sample data sets satisfy the following: the distribution of the number of initial sample data sets of different lengths in the M initial sample data sets satisfies the uniform distribution requirement, and the length of the initial sample data set represents the number of rounds of question-and-answer; and the distribution of the answer content from the previous question to the next question in the M initial sample data sets satisfies the uniform distribution requirement. Based on M initial sample data, the candidate question set of each initial sample data, and the label data of each initial sample data, M training sample data are obtained. The label data of the initial sample data includes the target question that the agent needs to generate in the (m+1)th round. The model to be trained is trained using M training sample data to obtain a target model that can predict the questions that the agent needs to generate in the next round based on historical question answers; the target model is a large language model. The step of training the model to be trained using M training sample data to obtain a target model capable of predicting the questions to be generated by the agent in the next round based on historical question-answering includes: The training sample data contains m rounds of question-and-answer sessions, and the candidate question set corresponding to the m-th round of questions in the training sample data is input into the model to be trained, so that the model to be trained can perform reasoning within the candidate question set to avoid the illusion problem in the reasoning process. The initial estimation result is obtained, where the initial estimation result represents the predicted problem that the agent needs to generate in the (m+1)th round; Based on the initial estimation results and the target problem that the agent needs to generate in the (m+1)th round contained in the label data of the training sample data, the loss value of the loss function is obtained, whereby the loss function can characterize the distance between the predicted problem and the target problem; Based on the loss value of the loss function, at least some of the adjustable parameters in the model to be trained are adjusted to train and obtain the target model.

2. The method according to claim 1, wherein, The step of inputting the m rounds of question-and-answer data contained in the training sample data, and the candidate question set corresponding to the m-th round question contained in the training sample data, into the model to be trained includes: Based on the m rounds of question-and-answer sessions contained in the training sample data, and the set of candidate questions corresponding to the m-th round of questions contained in the training sample data, the target prompt word question is obtained; Input the target prompt word question into the model to be trained.

3. The method according to claim 1, further comprising: Based on the determined finite state machine, the M initial sample data satisfying the following conditions are obtained: In the M initial sample data, the distribution of the number of initial sample data of different lengths satisfies the requirement of uniform distribution. The length of the initial sample data is determined based on the finite state machine and represents the number of rounds of question and answer. In the M initial sample data, the distribution of the answers from the previous question to the next question satisfies the requirement of uniform distribution; The finite state machine can represent a set of questions generated by the agent, and the transition conditions from the current question to the next question; the transition conditions are related to the content of the agent's answer to the current question.

4. The method according to claim 3, further comprising: The finite state machine is determined based on N historical interaction data between the object and the agent; wherein, the historical interaction data in the N historical interaction data includes m rounds of question and answer between the object and the agent, and the question to be generated by the agent in the (m+1)th round; and N is a positive integer greater than or equal to 1. The M initial sample data obtained based on the determined finite state machine, satisfying the following conditions, include: Based on the determined finite state machine, select M initial sample data that satisfy the following conditions from the N historical interaction data.

5. The method according to claim 1, further comprising: After obtaining the target model, the target output result of the target model is evaluated using at least one evaluation model to obtain the target evaluation result of the accuracy of the target output result. Wherein, the evaluation model in the at least one evaluation model satisfies at least one of the following conditions: When the parameters of the evaluation model are adjustable, the number of adjustable parameters of the evaluation model is greater than the number of adjustable parameters of the target model, and the evaluation model is a model obtained after training based on the M training sample data. When the parameters of the evaluation model are adjustable, the number of adjustable parameters of the evaluation model is greater than the number of adjustable parameters of the target model, and the evaluation model is obtained after training based on the M training sample data and the pre-constructed evaluation dataset. When the parameters of the evaluation model are not adjustable, the number of adjustable parameters of the evaluation model is greater than the number of adjustable parameters of the target model.

6. The method according to claim 5, wherein, The pre-constructed evaluation dataset contains multiple positive sample data and multiple negative sample data constructed based on the multiple positive sample data; The positive sample data in the plurality of positive sample data is obtained based on the initial sample data in the M initial sample data, including m rounds of question and answer, and the true questions generated by the target model in the (m+1)th round; The negative sample data constructed based on the positive sample data includes m rounds of question and answer, as well as a fake question constructed in the (m+1)th round.

7. The method according to claim 5, wherein, After obtaining the target model, the step of evaluating the target output result of the target model using at least one evaluation model to obtain a target evaluation result that assesses the accuracy of the target output result includes: After obtaining the target model, and given that the evaluation model is obtained after training based on the M training sample data, the target output result of the target model is evaluated based on the evaluation model to obtain the initial evaluation result corresponding to the evaluation model. The initial evaluation result of the evaluation model is used to measure the accuracy of the target output result of the target model.

8. The method according to claim 7, wherein, The initial evaluation result corresponding to the evaluation model is represented by the following value: the likelihood of the target output result output by the trained evaluation model.

9. The method according to claim 6, wherein, After obtaining the target model, the step of evaluating the target output result of the target model using at least one evaluation model to obtain a target evaluation result that assesses the accuracy of the target output result includes: After obtaining the target model, and given that the evaluation model is obtained by training based on the M training sample data and the pre-constructed evaluation dataset, the target output result of the target model is evaluated based on the evaluation model to obtain the first initial result and the second initial result corresponding to the evaluation model. Based on the first and second initial results, the target evaluation result for assessing the accuracy of the target output is obtained; The first initial result is used to measure the accuracy of the target model's target output; the second initial result is used to determine the probability that the target model's target output is an accurate value.

10. The method according to claim 9, wherein, The first initial result is represented by the following value: the likelihood of the target output of the trained evaluation model to the target model; And / or, the second initial result is represented by the following value: the target output of the trained evaluation model is the accurate likelihood of the target output of the target model.

11. The method according to claim 9, wherein, The target evaluation result, which assesses the accuracy of the target output result based on the first and second initial results, includes: The first and second initial results are weighted to obtain the target evaluation result that assesses the accuracy of the target output.

12. The method according to any one of claims 5-11, further comprising: When using two or more evaluation models for evaluation, the overall evaluation result of the accuracy of the evaluation target output result is obtained based on the target evaluation results of each evaluation model. Based on the overall evaluation results, the M training sample data are corrected to obtain the corrected M training sample data.

13. The method of claim 12, further comprising: Based on the corrected M training sample data, the target model is fine-tuned to obtain the corrected target model; And / or, Based on the corrected M training sample data, the parameter-adjustable evaluation model is fine-tuned to obtain the corrected evaluation model.

14. A voice interaction method, comprising: Obtain target text data, which is obtained based on the object's voice response to the question in the previous round between the object and the agent; Obtain the next set of candidate questions for the question addressed by the target text data; The target text data and the next set of candidate questions for the question addressed by the target text data are input into the target model to obtain the question text to be generated by the agent; wherein, the target model is obtained by training based on any one of the model processing methods in claims 1-13.

15. The method of claim 14, further comprising: The obtained question text is converted into question speech, and the question speech is output.

16. The method of claim 14, further comprising: Obtain the audio of the response from the person in the previous round of questions; Convert the speech of the object's answer to the question in the previous round into target text data.

17. The method according to any one of claims 14 to 16, wherein, The next set of candidate questions for the problem addressed by the target text data is determined based on a finite state machine, which can represent a set of questions generated by the agent and the transition conditions from the current question to the next question; The transition condition is related to the content of the object's answer to the current question.

18. A model processing apparatus, comprising: A data processing unit is used to obtain a candidate question set for each of the M initial sample data sets, wherein the initial sample data sets contain m rounds of question-and-answer interactions between the object and the agent; the candidate question set for the initial sample data sets includes the set of questions for the next round corresponding to the question in the m-th round of question-and-answer interactions; M and m are both positive integers greater than or equal to 1; the M initial sample data sets satisfy the following conditions: the distribution of the number of initial sample data sets of different lengths satisfies the uniform distribution requirement, and the length of the initial sample data sets represents the number of rounds of question-and-answer interactions; and the distribution of the answer content from the previous question to the next question satisfies the uniform distribution requirement; based on the M initial sample data sets, the candidate question set for each initial sample data set, and the label data for each initial sample data set, M training sample data sets are obtained, wherein the label data for the initial sample data sets includes the target question to be generated by the agent in the (m+1)-th round; The model training unit is used to train the model to be trained using M training sample data to obtain a target model that can predict the questions that the agent needs to generate in the next round based on historical question answers; the target model is a large language model. Specifically, the model training unit is used for: The training sample data contains m rounds of question-and-answer sessions, and the candidate question set corresponding to the m-th round question in the training sample data is input into the model to be trained, so that the model to be trained can perform reasoning within the candidate question set to avoid the illusion problem in the reasoning process; the initial estimation result is obtained, where the initial estimation result represents the predicted question that the agent needs to generate in the (m+1)-th round; Based on the initial estimation results and the target problem that the agent needs to generate in the (m+1)th round contained in the label data of the training sample data, the loss value of the loss function is obtained, whereby the loss function can characterize the distance between the predicted problem and the target problem; Based on the loss value of the loss function, at least some of the adjustable parameters in the model to be trained are adjusted to train and obtain the target model.

19. The apparatus according to claim 18, wherein, The model training unit is specifically used for: Based on the m rounds of question-and-answer sessions contained in the training sample data, and the set of candidate questions corresponding to the m-th round of questions contained in the training sample data, the target prompt word question is obtained; Input the target prompt word question into the model to be trained.

20. The apparatus according to claim 18, wherein, The data processing unit is further configured to: Based on the determined finite state machine, the M initial sample data satisfying the following conditions are obtained: In the M initial sample data, the distribution of the number of initial sample data of different lengths satisfies the requirement of uniform distribution. The length of the initial sample data is determined based on the finite state machine and represents the number of rounds of question and answer. In the M initial sample data, the distribution of the answers from the previous question to the next question satisfies the requirement of uniform distribution; The finite state machine can represent a set of questions generated by the agent, and the transition conditions from the current question to the next question; the transition conditions are related to the content of the agent's answer to the current question.

21. The apparatus according to claim 20, wherein, The data processing unit is further configured to: The finite state machine is determined based on N historical interaction data between the object and the agent; wherein, the historical interaction data in the N historical interaction data includes m rounds of question and answer between the object and the agent, and the question to be generated by the agent in the (m+1)th round; and N is a positive integer greater than or equal to 1. Based on the determined finite state machine, select M initial sample data that satisfy the following conditions from the N historical interaction data.

22. The apparatus of claim 18, further comprising: Model evaluation unit, wherein the model evaluation unit is used for: After obtaining the target model, the target output result of the target model is evaluated using at least one evaluation model to obtain the target evaluation result of the accuracy of the target output result. Wherein, the evaluation model in the at least one evaluation model satisfies at least one of the following conditions: When the parameters of the evaluation model are adjustable, the number of adjustable parameters of the evaluation model is greater than the number of adjustable parameters of the target model, and the evaluation model is a model obtained after training based on the M training sample data. When the parameters of the evaluation model are adjustable, the number of adjustable parameters of the evaluation model is greater than the number of adjustable parameters of the target model, and the evaluation model is obtained after training based on the M training sample data and the pre-constructed evaluation dataset. When the parameters of the evaluation model are not adjustable, the number of adjustable parameters of the evaluation model is greater than the number of adjustable parameters of the target model.

23. The apparatus according to claim 22, wherein, The pre-constructed evaluation dataset contains multiple positive sample data and multiple negative sample data constructed based on the multiple positive sample data; The positive sample data in the plurality of positive sample data is obtained based on the initial sample data in the M initial sample data, including m rounds of question and answer, and the true questions generated by the target model in the (m+1)th round; The negative sample data constructed based on the positive sample data includes m rounds of question and answer, as well as a fake question constructed in the (m+1)th round.

24. The apparatus according to claim 22, wherein, The model evaluation unit is specifically used for: After obtaining the target model, and given that the evaluation model is obtained after training based on the M training sample data, the target output result of the target model is evaluated based on the evaluation model to obtain the initial evaluation result corresponding to the evaluation model. The initial evaluation result of the evaluation model is used to measure the accuracy of the target output result of the target model.

25. The apparatus according to claim 24, wherein, The initial evaluation result corresponding to the evaluation model is represented by the following value: the likelihood of the target output result output by the trained evaluation model.

26. The apparatus according to claim 23, wherein, The model evaluation unit is specifically used for: After obtaining the target model, and given that the evaluation model is obtained by training based on the M training sample data and the pre-constructed evaluation dataset, the target output result of the target model is evaluated based on the evaluation model to obtain the first initial result and the second initial result corresponding to the evaluation model. Based on the first and second initial results, the target evaluation result for assessing the accuracy of the target output is obtained; The first initial result is used to measure the accuracy of the target output of the target model; The second initial result is used to determine the probability that the target output of the target model is an accurate value.

27. The apparatus according to claim 26, wherein, The first initial result is represented by the following value: the likelihood of the target output of the trained evaluation model to the target model; And / or, the second initial result is represented by the following value: the target output of the trained evaluation model is the accurate likelihood of the target output of the target model.

28. The apparatus according to claim 26, wherein, The model evaluation unit is specifically used for: The first and second initial results are weighted to obtain the target evaluation result that assesses the accuracy of the target output.

29. The apparatus according to any one of claims 22-28, wherein, The model evaluation unit is also used to obtain a total evaluation result of the accuracy of the evaluation target output result based on the target evaluation results of each evaluation model when two or more evaluation models are used for evaluation. The data processing unit is further configured to correct the M training sample data based on the overall evaluation result to obtain corrected M training sample data.

30. The apparatus according to claim 29, wherein, The model training unit is also used for: Based on the corrected M training sample data, the target model is fine-tuned to obtain the corrected target model; And / or, Based on the corrected M training sample data, the parameter-adjustable evaluation model is fine-tuned to obtain the corrected evaluation model.

31. A voice interaction device, comprising: The acquisition unit is used to acquire target text data, which is obtained based on the object's answer voice in the previous round of questions between the object and the agent; and to acquire a set of candidate questions for the next round of questions that the target text data addresses. The model prediction unit is used to input the target text data and the next round of candidate questions for the question addressed by the target text data into the target model to obtain the question text to be generated by the agent; wherein, the target model is obtained after training based on any one of the model processing methods in claims 1-13.

32. The apparatus of claim 31, further comprising: A speech synthesis unit; wherein the speech synthesis unit is used for: The obtained question text is converted into question speech, and the question speech is output.

33. The apparatus of claim 31, further comprising: A speech recognition unit; wherein the speech recognition unit is used for: Obtain the audio of the response from the person in the previous round of questions; Convert the speech of the object's answer to the question in the previous round into target text data.

34. The apparatus according to any one of claims 31 to 33, wherein, The next set of candidate questions for the problem addressed by the target text data is determined based on a finite state machine, which can represent a set of questions generated by the agent and the transition conditions from the current question to the next question; The transition condition is related to the content of the object's answer to the current question.

35. An electronic device comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-17.

36. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-17.

37. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-17.

Images