Training method of user intention model

Abstract

This application provides a method for training a user intent model. The method includes introducing a structured output format that includes a reasoning process, user intent, stop flag, evaluation process, and evaluation score. Combined with a hybrid training mechanism of supervised fine-tuning and reinforcement learning, this significantly improves the performance of a first model in complex hierarchical intent recognition tasks, enhances the first model's ability to learn deep intent features, and ultimately determines the trained first model as the user intent model. This allows for user intent recognition during practical use, improving the accuracy of user intent recognition.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method for training a user intent model. Background Technology

[0002] In intelligent customer service systems, virtual assistants, and multi-turn dialogue platforms, hierarchical recognition of user intent is a core technical aspect for achieving accurate service routing, automated work order generation, and intelligent responses. Taking a banking customer service scenario as an example, user requests typically have a natural hierarchical structure, usually categorized into service type, business domain, and specific operation levels.

[0003] In related technologies, hierarchical user intents are flattened into single labels (such as account inquiry-savings-balance), and then a pre-trained language model is used to extract features and classify and predict user questions, directly outputting the corresponding single-label user intent.

[0004] However, the label space expands exponentially with the depth of the hierarchy (for example, when the intent hierarchy expands from 3 layers to 5 layers, the number of labels will surge from hundreds to tens of thousands or even hundreds of thousands). During language model training, it is easy to encounter problems such as sparse training samples and class imbalance. The language model cannot stably learn effective features, resulting in inaccurate user intent recognition. Summary of the Invention

[0005] This application provides a method for training a user intent model to improve the accuracy of user intent recognition.

[0006] In a first aspect, embodiments of this application provide a method for training a user intent model, comprising:

[0007] Obtain a preset intent hierarchy tree and multiple sample data. The intent hierarchy tree indicates the hierarchy of user intents, and the sample data includes user questions and standard user intents.

[0008] Based on the intent hierarchy tree, a standard output format for the first model is constructed. The standard output format indicates that the output sequence generated by the first model includes, in sequence: the reasoning process for the user question, the user intent based on the reasoning process, the stop flag, the evaluation process for the user intent, and the evaluation score for the evaluation process.

[0009] Based on the standard output format and the first model, forward reasoning is performed on multiple user questions to generate multiple output sequences for each user question.

[0010] Based on multiple output sequences of each user question and the corresponding standard user intent, the first model is trained using a hybrid method that includes supervised fine-tuning and reinforcement learning to obtain the user intent model.

[0011] In some embodiments, based on a standard output format and a first model, forward reasoning is performed on multiple user questions to generate multiple output sequences for each user question, including:

[0012] Use the standard output format as the prompt input for the first model;

[0013] For any given user question, any output sequence;

[0014] Based on the first model, the user's question is processed to generate a reasoning process.

[0015] Based on the first model, intent parsing is performed on each reasoning process to obtain the user intent, and the user intent is then concatenated after the reasoning process.

[0016] A stop marker is appended after the user's intent;

[0017] Based on the first model, the user intent is evaluated and reasoned to obtain the evaluation process, and the evaluation process is concatenated after the stop flag.

[0018] Based on the first model, score prediction processing is performed on each evaluation process to obtain the evaluation score. After the evaluation process, the evaluation scores are concatenated to obtain the output sequence.

[0019] In some embodiments, based on multiple output sequences of each user question and the corresponding standard user intent, the first model is subjected to hybrid training including supervised fine-tuning and reinforcement learning to obtain a user intent model, including:

[0020] For any given user problem; based on the user problem and its multiple output sequences, determine the target loss value for each output sequence; and determine the output sequence corresponding to the minimum target loss value among the multiple target loss values ​​as the target output sequence.

[0021] Based on multiple user questions and the target output sequence corresponding to each user question, the first model is trained using a hybrid method that includes supervised fine-tuning and reinforcement learning to obtain the user intent model.

[0022] In some embodiments, based on the user question and multiple output sequences of the user question, determining the target loss value for each output sequence includes:

[0023] Any output sequence for the user's question;

[0024] Based on the user intent in the output sequence and the standard user intent corresponding to the user question, a first evaluation value is determined. The first evaluation value is used to indicate the accuracy of the user intent recognition.

[0025] Based on the output sequence, a second evaluation value is determined, which is used to indicate whether the output sequence is complete.

[0026] Based on the evaluation scores and user questions in the output sequence, a third evaluation value is determined, which is used to indicate the reliability of the self-evaluation results of the first model;

[0027] Based on the output sequence, the first evaluation value, the second evaluation value, and the third evaluation value, the target loss value of the output sequence is determined.

[0028] In some embodiments, a first evaluation value is determined based on the user intent in the output sequence and the standard user intent corresponding to the user question, including:

[0029] Determine whether the user intent in the output sequence is the same as the standard user intent corresponding to the user question;

[0030] When the user intent in the output sequence is the same as the standard user intent corresponding to the user question, the first evaluation value is determined to be the first preset value. The first evaluation value indicates that the user intent is accurately identified.

[0031] When the user intent in the output sequence is different from the standard user intent corresponding to the user question, the first evaluation value is determined to be the second preset value. The first evaluation value being the second preset value indicates that the user intent recognition is inaccurate.

[0032] In some embodiments, determining a second evaluation value based on the output sequence includes:

[0033] Determine whether the output sequence includes reasoning, user intent, evaluation process, and evaluation score;

[0034] When the output sequence includes the reasoning process, user intent, evaluation process, and evaluation score, the second evaluation value is determined to be the first preset value, and the second evaluation value to be the first preset value indicates that the output sequence is complete;

[0035] If the output sequence does not include any one or more of the following: reasoning process, user intent, evaluation process, or evaluation score, the second evaluation value is determined to be the second preset value. The second evaluation value indicates that the output sequence is incomplete.

[0036] In some embodiments, a third evaluation value is determined based on the evaluation scores and user questions in the output sequence, including:

[0037] Based on the evaluation scores, determine the predicted evaluation scores of user intent in the output sequence;

[0038] Based on the preset first script, the user's question is analyzed for intent and the score is quantified to obtain the true evaluation score;

[0039] The absolute value of the difference between the predicted assessment score and the actual assessment score is determined as the assessment deviation value;

[0040] The difference between the third preset value and the evaluation deviation value is determined as the third evaluation value.

[0041] In some embodiments, determining the target loss value of the output sequence based on the output sequence, a first evaluation value, a second evaluation value, and a third evaluation value includes:

[0042] Based on the user question, reasoning process, and user intent, a first loss value is determined, which is used for first-supervised fine-tuning training.

[0043] Based on the user question, reasoning process, user intent, evaluation process, and evaluation score, a second loss value is determined, which is used for second-supervised fine-tuning training.

[0044] Based on the user question, the output sequence, the first evaluation value, the second evaluation value, and the third evaluation value, a third loss value is determined, which is used for reinforcement learning training.

[0045] The target loss value is obtained by adding the first loss value, the second loss value, and the third loss value.

[0046] In some embodiments, a third loss value is determined based on the user question, the output sequence, a first evaluation value, a second evaluation value, and a third evaluation value, including:

[0047] The sum of the first evaluation value, the second evaluation value, and the third evaluation value is determined as the comprehensive evaluation value;

[0048] The third loss value is determined based on the user question, the output sequence, and the comprehensive evaluation value.

[0049] In some embodiments, based on multiple user questions and the target output sequence corresponding to each user question, a first model is trained using a hybrid training method including supervised fine-tuning and reinforcement learning to obtain a user intent model, including:

[0050] The average of the target loss values ​​for multiple user questions is used to obtain the epoch loss value for the current epoch of the mixed training.

[0051] Based on the round loss value of the current round and the round loss values ​​of the first number of rounds prior to the current round, determine whether the loss curve converges;

[0052] When the loss curve converges, the first model in the current round is determined as the user intent model;

[0053] When the loss curve has not converged, the parameters of the first model are updated based on multiple user questions and the target output sequence corresponding to each user question, resulting in an updated first model. The next round of forward inference processing and hybrid training is then performed based on the updated first model until the loss curve converges.

[0054] Secondly, embodiments of this application provide a training apparatus for a user intent model, comprising:

[0055] The first processing module is used to obtain a preset intent hierarchy tree and multiple sample data. The intent hierarchy tree indicates the hierarchy of user intents, and the sample data includes user questions and standard user intents.

[0056] The second processing module is used to construct the standard output format of the first model based on the intent hierarchy tree. The standard output format indicates that the output sequence generated by the first model includes, in sequence: the reasoning process for the user question, the user intent based on the reasoning process, the stop flag, the evaluation process for the user intent, and the evaluation score for the evaluation process.

[0057] The third processing module is used to perform forward reasoning processing on multiple user questions based on the standard output format and the first model, and generate multiple output sequences for each user question.

[0058] The fourth processing module is used to perform hybrid training of the first model, including supervised fine-tuning and reinforcement learning, based on multiple output sequences of each user question and the corresponding standard user intent, to obtain the user intent model.

[0059] In some embodiments, the third processing module is specifically used for:

[0060] Use the standard output format as the prompt input for the first model;

[0061] For any given user question, any output sequence;

[0062] Based on the first model, the user's question is processed to generate a reasoning process.

[0063] Based on the first model, intent parsing is performed on each reasoning process to obtain the user intent, and the user intent is then concatenated after the reasoning process.

[0064] A stop marker is appended after the user's intent;

[0065] Based on the first model, the user intent is evaluated and reasoned to obtain the evaluation process, and the evaluation process is concatenated after the stop flag.

[0066] Based on the first model, score prediction processing is performed on each evaluation process to obtain the evaluation score. After the evaluation process, the evaluation scores are concatenated to obtain the output sequence.

[0067] In some embodiments, the fourth processing module is specifically used for:

[0068] For any given user problem; based on the user problem and its multiple output sequences, determine the target loss value for each output sequence; and determine the output sequence corresponding to the minimum target loss value among the multiple target loss values ​​as the target output sequence.

[0069] Based on multiple user questions and the target output sequence corresponding to each user question, the first model is trained using a hybrid method that includes supervised fine-tuning and reinforcement learning to obtain the user intent model.

[0070] In some embodiments, the fourth processing module is specifically used for:

[0071] Any output sequence for the user's question;

[0072] Based on the user intent in the output sequence and the standard user intent corresponding to the user question, a first evaluation value is determined. The first evaluation value is used to indicate the accuracy of the user intent recognition.

[0073] Based on the output sequence, a second evaluation value is determined, which is used to indicate whether the output sequence is complete.

[0074] Based on the evaluation scores and user questions in the output sequence, a third evaluation value is determined, which is used to indicate the reliability of the self-evaluation results of the first model;

[0075] Based on the output sequence, the first evaluation value, the second evaluation value, and the third evaluation value, the target loss value of the output sequence is determined.

[0076] In some embodiments, the fourth processing module is specifically used for:

[0077] Determine whether the user intent in the output sequence is the same as the standard user intent corresponding to the user question;

[0078] When the user intent in the output sequence is the same as the standard user intent corresponding to the user question, the first evaluation value is determined to be the first preset value. The first evaluation value indicates that the user intent is accurately identified.

[0079] When the user intent in the output sequence is different from the standard user intent corresponding to the user question, the first evaluation value is determined to be the second preset value. The first evaluation value being the second preset value indicates that the user intent recognition is inaccurate.

[0080] In some embodiments, the fourth processing module is specifically used for:

[0081] Determine whether the output sequence includes reasoning, user intent, evaluation process, and evaluation score;

[0082] When the output sequence includes the reasoning process, user intent, evaluation process, and evaluation score, the second evaluation value is determined to be the first preset value, and the second evaluation value to be the first preset value indicates that the output sequence is complete;

[0083] If the output sequence does not include any one or more of the following: reasoning process, user intent, evaluation process, or evaluation score, the second evaluation value is determined to be the second preset value. The second evaluation value indicates that the output sequence is incomplete.

[0084] In some embodiments, the fourth processing module is specifically used for:

[0085] Based on the evaluation scores, determine the predicted evaluation scores of user intent in the output sequence;

[0086] Based on the preset first script, the user's question is analyzed for intent and the score is quantified to obtain the true evaluation score;

[0087] The absolute value of the difference between the predicted assessment score and the actual assessment score is determined as the assessment deviation value;

[0088] The difference between the third preset value and the evaluation deviation value is determined as the third evaluation value.

[0089] In some embodiments, the fourth processing module is specifically used for:

[0090] Based on the user question, reasoning process, and user intent, a first loss value is determined, which is used for first-supervised fine-tuning training.

[0091] Based on the user question, reasoning process, user intent, evaluation process, and evaluation score, a second loss value is determined, which is used for second-supervised fine-tuning training.

[0092] Based on the user question, the output sequence, the first evaluation value, the second evaluation value, and the third evaluation value, a third loss value is determined, which is used for reinforcement learning training.

[0093] The target loss value is obtained by adding the first loss value, the second loss value, and the third loss value.

[0094] In some embodiments, the fourth processing module is specifically used for:

[0095] The sum of the first evaluation value, the second evaluation value, and the third evaluation value is determined as the comprehensive evaluation value;

[0096] The third loss value is determined based on the user question, the output sequence, and the comprehensive evaluation value.

[0097] In some embodiments, the fourth processing module is specifically used for:

[0098] The average of the target loss values ​​for multiple user questions is used to obtain the epoch loss value for the current epoch of the mixed training.

[0099] Based on the round loss value of the current round and the round loss values ​​of the first number of rounds prior to the current round, determine whether the loss curve converges;

[0100] When the loss curve converges, the first model in the current round is determined as the user intent model;

[0101] When the loss curve has not converged, the parameters of the first model are updated based on multiple user questions and the target output sequence corresponding to each user question, resulting in an updated first model. The next round of forward inference processing and hybrid training is then performed based on the updated first model until the loss curve converges.

[0102] Thirdly, embodiments of this application provide an electronic device, including: a memory and a processor;

[0103] The memory stores instructions that the computer executes;

[0104] The processor executes computer execution instructions stored in memory, causing the processor to perform the first aspect and / or various possible implementations of the first aspect as described above.

[0105] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the first aspect and / or various possible implementations of the first aspect.

[0106] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the first aspect and / or various possible implementations of the first aspect.

[0107] This application provides a method for training a user intent model. The method includes: acquiring a preset intent hierarchy tree and multiple sample data, wherein the intent hierarchy tree indicates the hierarchy of user intents, and the sample data includes user questions and standard user intents; constructing a standard output format for a first model based on the intent hierarchy tree, wherein the standard output format indicates that the output sequence generated by the first model sequentially includes: a reasoning process for the user question, a user intent based on the reasoning process, a stop flag, an evaluation process for the user intent, and an evaluation score for the evaluation process; performing forward reasoning processing on multiple user questions based on the standard output format and the first model to generate multiple output sequences for each user question; and performing hybrid training on the first model, including supervised fine-tuning and reinforcement learning, based on the multiple output sequences of each user question and the corresponding standard user intent, to obtain a user intent model. In the above method, by introducing a structured output format that includes reasoning process, user intent, stop flag, evaluation process, and evaluation score, and combining a hybrid training mechanism of supervised fine-tuning and reinforcement learning, the performance of the first model in complex hierarchical intent recognition tasks is significantly improved, the learning ability of the first model to deep intent features is enhanced, and the first model after training is finally determined as the user intent model. In the actual use stage, the user intent model can be used to identify user intent, improve the accuracy of user intent recognition, and thus achieve more accurate service routing and automated response in scenarios such as intelligent customer service. Attached Figure Description

[0108] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0109] Figure 1 A flowchart illustrating a user intent model training method provided in an embodiment of this application;

[0110] Figure 2 An example of an output sequence provided in an embodiment of this application;

[0111] Figure 3 A flowchart illustrating a method for determining a target loss value provided in an embodiment of this application. Figure 1 ;

[0112] Figure 4 A flowchart illustrating a method for determining a target loss value provided in an embodiment of this application. Figure 2 ;

[0113] Figure 5 A flowchart illustrating a method for determining a user intent model provided in an embodiment of this application;

[0114] Figure 6A schematic diagram of the structure of a training device for a user intent model provided in an embodiment of this application;

[0115] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0116] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0117] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0118] In intelligent customer service systems, virtual assistants, and multi-turn dialogue platforms, hierarchical recognition of user intent is a core technical aspect for achieving accurate service routing, automated work order generation, and intelligent responses. Taking a banking customer service scenario as an example, user requests typically have a natural hierarchical structure, usually categorized into service type, business domain, and specific operation levels.

[0119] In related technologies, hierarchical user intents are flattened into single labels (such as account inquiry-savings-balance), and then a pre-trained language model is used to extract features and classify and predict user questions, directly outputting the corresponding single-label user intent.

[0120] However, the label space (the set of all intent labels) expands exponentially with the depth of the hierarchy (for example, when the intent hierarchy expands from 3 to 5 layers, the number of labels will increase from hundreds to tens of thousands or even hundreds of thousands). During language model training, the problem of sparse training samples and class imbalance is very likely to occur, and the language model cannot stably learn effective features, resulting in inaccurate intent recognition.

[0121] In view of this, this application provides a training method for a user intent model. The method includes: acquiring a preset intent hierarchy tree and multiple sample data, wherein the intent hierarchy tree indicates the hierarchy of user intent, and the sample data includes user questions and standard user intents; constructing a standard output format for a first model based on the intent hierarchy tree, wherein the standard output format indicates that the output sequence generated by the first model sequentially includes: a reasoning process for the user question, a user intent based on the reasoning process, a stop flag, an evaluation process for the user intent, and an evaluation score for the evaluation process; performing forward reasoning processing on multiple user questions based on the standard output format and the first model to generate multiple output sequences for each user question; and performing hybrid training on the first model, including supervised fine-tuning and reinforcement learning, based on the multiple output sequences of each user question and the corresponding standard user intent, to obtain a user intent model. In the above method, by introducing a structured output format that includes reasoning process, user intent, stop flag, evaluation process, and evaluation score, and combining a hybrid training mechanism of supervised fine-tuning and reinforcement learning, the performance of the first model in complex hierarchical intent recognition tasks is significantly improved, the learning ability of the first model to deep intent features is enhanced, and the first model after training is finally determined as the user intent model. In the actual use stage, the user intent model can be used to identify user intent, improve the accuracy of user intent recognition, and thus achieve more accurate service routing and automated response in scenarios such as intelligent customer service.

[0122] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0123] Figure 1 A flowchart illustrating a user intent model training method provided in this application embodiment is shown below. Figure 1 As shown, the method includes:

[0124] S101. Obtain a preset intent hierarchy tree and multiple sample data, wherein the intent hierarchy tree indicates the hierarchy of user intents, and the sample data includes user questions and standard user intents.

[0125] An intent hierarchy tree is a predefined, categorized knowledge framework that organizes business intents in a tree structure, defining the hierarchy and attribution logic between intents. This tree structure can be designed and developed by business experts in different target business scenarios or vertical domains.

[0126] For example, in the field of financial customer service dialogue, the intent hierarchy tree can be designed as a three-level structure of "service type → business domain → specific operation". Specifically, this intent hierarchy tree indicates the hierarchy of user intent as follows: the root node is the service type, the child nodes of the root node are the business domains, and the leaf nodes of the business domains are the specific operations.

[0127] Each sample data contains a user question and its corresponding standard user intent.

[0128] For example, a sample data could be: a user's question is "I want to see how much credit I have left on my credit card.", and the corresponding standard user intent is "Service type: Account inquiry > Business area: Credit card > Specific operation: Inquire about credit limit".

[0129] S102. Based on the intent hierarchy tree, construct the standard output format of the first model, wherein the standard output format indicates that the output sequence generated by the first model includes, in sequence: the reasoning process for the user question, the user intent based on the reasoning process, the stop flag, the evaluation process for the user intent, and the evaluation score for the evaluation process.

[0130] In some embodiments, the stop identifier is, for example, post-completion.

[0131] The stop identifier is a predefined output sequence separator used to distinguish user intent content (including reasoning process and user intent) from subsequent self-assessment content (including assessment process and assessment score).

[0132] The following example illustrates the standard output format of the first model when the intent hierarchy tree is "Service Type → Business Domain → Specific Operation":

[0133] The reasoning process is based on the intent hierarchy tree, combined with the semantic features of the user's question, to reason about the service type and business domain to which the user's question belongs.

[0134] The user intent is: "Service Type: A > Business Domain: B > Specific Operation: C". Here, A, B, and C are exemplary placeholders for business category nodes in the intent hierarchy tree, used only for illustrative purposes.

[0135] The evaluation process includes: 1) determining whether “A” is a valid root node; 2) determining whether “B” is a valid child node of “A”; 3) determining whether “C” is a valid leaf node of “B”; 4) ensuring that the entire user intent path is complete and without skipping levels.

[0136] The evaluation scores include the root node score (path_valid), the child node score (parent_child_consistent), and the leaf node score (leaf_node_correct).

[0137] The first model can be a large language model. A large language model is a large neural network model trained using massive amounts of text data and containing a large number of parameters (usually billions or even trillions).

[0138] S103. Based on the standard output format and the first model, perform forward reasoning processing on multiple user questions to generate multiple output sequences for each user question.

[0139] The forward inference process receives the standard output format of the input and the user question from the first model, and then gradually generates the forward inference calculation process, user intent, stop flag, evaluation process and evaluation score.

[0140] In some embodiments, based on a standard output format and a first model, forward reasoning is performed on multiple user questions to generate multiple output sequences for each user question, including:

[0141] Use the standard output format as the prompt input for the first model;

[0142] For any output sequence of any user question; based on the first model, perform reasoning generation processing on the user question to obtain the reasoning process; based on the first model, perform intent parsing processing on each reasoning process to obtain the user intent, and concatenate the user intent after the reasoning process; concatenate a stop flag after the user intent; based on the first model, perform evaluation reasoning processing on the user intent to obtain the evaluation process, and concatenate the evaluation process after the stop flag; based on the first model, perform score prediction processing on each evaluation process to obtain the evaluation score, and concatenate the evaluation score after the evaluation process to obtain the output sequence.

[0143] Understandably, after the standard output format is used as a prompt input to the first model, the subsequent inference generation processing, intent parsing processing, evaluation inference processing, and score prediction processing performed based on the first model are all forward inference processes under the constraints of this prompt.

[0144] Based on the first model, the user question is processed by reasoning to obtain the reasoning process, which can be understood as follows: Under the constraints of the standard output format, the first model performs semantic parsing on the user question and derives the reasoning logic description of its service type, business domain and specific operation.

[0145] Based on the first model, intent parsing is performed on each reasoning process to obtain the user intent. This can be understood as follows: the first model extracts the semantic content of the reasoning process in a structured manner, matches the node affiliation of the intent hierarchy tree, and obtains the user intent of the user question.

[0146] Based on the first model, the user intent is evaluated and reasoned to obtain the evaluation process, which can be understood as follows: the first model verifies the legality and hierarchical integrity of the parsed user intent according to the hierarchical rules of the intent hierarchy tree, and generates a processing action that describes the self-evaluation logic.

[0147] The following is an example of how to predict scores for each evaluation process based on the first model to obtain evaluation scores:

[0148] For example, the user intent is: "Service type: A > Business domain: B > Specific operation: C".

[0149] The root node score is determined based on whether "A" is a valid root node. For example, if "A" is a valid root node, path_valid is set to 1; if "A" is an invalid root node, path_valid is set to 0.

[0150] The score of a child node is determined based on whether it is valid for "B" to be a child node of "A". For example, if it is valid for "B" to be a child node of "A", the parent_child_consistent value is set to 1; if it is invalid for "B" to be a child node of "A", the parent_child_consistent value is set to 0.

[0151] The leaf node score is determined based on whether the leaf node where "C" is "B" is valid. For example, when the leaf node where "C" is "B" is valid, leaf_node_correct is set to 1; when the leaf node where "C" is "B" is invalid, leaf_node_correct is set to 0.

[0152] Figure 2 This is an example of an output sequence provided in an embodiment of the present application. The output sequence is generated by the first model after performing forward reasoning on the user question, based on the standard output format constraints as in S102. It fully includes the reasoning process, user intent, stop flag, evaluation process, and evaluation score.

[0153] S104. Based on multiple output sequences of each user question and the corresponding standard user intent, perform hybrid training on the first model, including supervised fine-tuning and reinforcement learning, to obtain the user intent model.

[0154] In some embodiments, based on multiple output sequences of each user question and the corresponding standard user intent, the first model is subjected to hybrid training including supervised fine-tuning and reinforcement learning to obtain a user intent model, including:

[0155] For any given user problem; based on the user problem and its multiple output sequences, determine the target loss value for each output sequence; and determine the output sequence corresponding to the minimum target loss value among the multiple target loss values ​​as the target output sequence.

[0156] Based on multiple user questions and the target output sequence corresponding to each user question, the first model is trained using a hybrid method that includes supervised fine-tuning and reinforcement learning to obtain the user intent model.

[0157] Furthermore, in the actual use phase of the user intent model, the stop marker is used as the stop word of the user intent model. This means that once the user intent model has generated the user intent, it will stop running once it is ready to generate the stop marker, and will no longer generate subsequent evaluation processes and evaluation scores.

[0158] In this embodiment, a preset intent hierarchy tree and multiple sample data are obtained. The intent hierarchy tree indicates the hierarchy of user intents, and the sample data includes user questions and standard user intents. Based on the intent hierarchy tree, a standard output format for a first model is constructed. The standard output format indicates that the output sequence generated by the first model sequentially includes: a reasoning process for the user question, a user intent based on the reasoning process, a stop flag, an evaluation process for the user intent, and an evaluation score for the evaluation process. Based on the standard output format and the first model, forward reasoning processing is performed on multiple user questions to generate multiple output sequences for each user question. Based on the multiple output sequences for each user question and the corresponding standard user intent, the first model is subjected to hybrid training including supervised fine-tuning and reinforcement learning to obtain a user intent model. In the above method, by introducing a structured output format that includes reasoning process, user intent, stop flag, evaluation process, and evaluation score, and combining a hybrid training mechanism of supervised fine-tuning and reinforcement learning, the performance of the first model in complex hierarchical intent recognition tasks is significantly improved, the learning ability of the first model to deep intent features is enhanced, and the first model after training is finally determined as the user intent model. In the actual use stage, the user intent model can be used to identify user intent, improve the accuracy of user intent recognition, and thus achieve more accurate service routing and automated response in scenarios such as intelligent customer service.

[0159] Furthermore, in the actual use phase of the user intent model, the stop marker is used as the stop word of the user intent model. This means that once the user intent model has generated the user intent, it will stop running once it is ready to generate the stop marker, and will no longer generate subsequent evaluation processes and evaluation scores. Therefore, the user intent model of this application will not introduce additional response latency, computational overhead or system resource consumption in the actual trial phase, thus ensuring the efficiency and performance of the actual application.

[0160] Hereinafter, based on any of the above embodiments, by... Figure 3 For any output sequence of the user problem, the document further explains "determining the target loss value of each output sequence based on the user problem and multiple output sequences of the user problem".

[0161] Figure 3 A flowchart illustrating a method for determining a target loss value provided in an embodiment of this application. Figure 1 ,like Figure 3 As shown, the method includes:

[0162] S301. Based on the user intent in the output sequence and the standard user intent corresponding to the user question, determine a first evaluation value, wherein the first evaluation value is used to indicate the accuracy of the user intent recognition.

[0163] In some embodiments, a first evaluation value is determined based on the user intent in the output sequence and the standard user intent corresponding to the user question, including:

[0164] Determine whether the user intent in the output sequence is the same as the standard user intent corresponding to the user question;

[0165] When the user intent in the output sequence is the same as the standard user intent corresponding to the user question, the first evaluation value is determined to be the first preset value, wherein the first evaluation value is the first preset value indicating that the user intent is accurately identified;

[0166] When the user intent in the output sequence is different from the standard user intent corresponding to the user question, the first evaluation value is determined to be the second preset value, wherein the first evaluation value is the second preset value, indicating that the user intent is not accurately identified.

[0167] The first preset value is, for example, 1.

[0168] The second preset value is, for example, 0.

[0169] The first evaluation value can be the accuracy reward (Ra).

[0170] S302. Based on the output sequence, determine a second evaluation value, wherein the second evaluation value is used to indicate whether the output sequence is complete.

[0171] In some embodiments, determining a second evaluation value based on the output sequence includes:

[0172] Determine whether the output sequence includes reasoning, user intent, evaluation process, and evaluation score;

[0173] When the output sequence includes the reasoning process, user intent, evaluation process, and evaluation score, the second evaluation value is determined to be the first preset value, wherein the second evaluation value is the first preset value, indicating that the output sequence is complete;

[0174] When the output sequence does not include any one or more of the following: reasoning process, user intent, evaluation process, or evaluation score, the second evaluation value is determined to be the second preset value, wherein the second evaluation value is the second preset value indicating that the output sequence is incomplete.

[0175] The second evaluation value can be the format reward (Rf).

[0176] S303. Based on the evaluation scores and user questions in the output sequence, determine a third evaluation value, wherein the third evaluation value is used to indicate the reliability of the self-evaluation results of the first model.

[0177] In some embodiments, a third evaluation value is determined based on the evaluation scores and user questions in the output sequence, including:

[0178] Based on the evaluation scores, determine the predicted evaluation scores of user intent in the output sequence;

[0179] Based on the preset first script, the user's question is analyzed for intent and the score is quantified to obtain the true evaluation score;

[0180] The absolute value of the difference between the predicted assessment score and the actual assessment score is determined as the assessment deviation value;

[0181] The difference between the third preset value and the evaluation deviation value is determined as the third evaluation value.

[0182] The third preset value is, for example, 1.

[0183] The third evaluation value can be the Consistency Reward (Rc).

[0184] In some embodiments, when the evaluation score includes root node score, child node score, and leaf node score, determining a predicted evaluation score for the user intent in the output sequence based on the evaluation score may include:

[0185] The average of the root node score, child node score, and leaf node score is calculated to obtain the average evaluation score.

[0186] The average of the evaluation scores is used as the predicted evaluation score of the user intent in the output sequence.

[0187] Based on a pre-defined first script, intent analysis and score quantification are performed on user questions to obtain a true evaluation score. This can be understood as follows: the first script is a pre-written programming language script with built-in real rules. It first parses the user question to obtain the corresponding real user intent, then performs legality verification on the real user intent and completes score quantification to obtain a true evaluation score.

[0188] The true rules can be understood as: judgment rules used to verify whether a user's intent is legitimate and whether it conforms to the hierarchical relationship defined in the intent hierarchy tree and the constraints of the business ontology.

[0189] Understandably, validating the legitimacy of the real user intent and quantifying the score to obtain the real evaluation score can be understood as follows: using the programming language in the first script, the evaluation process of determining the real user intent and the score prediction processing of the evaluation process are implemented to obtain the real evaluation score, and then the average value of the real evaluation score (i.e., the average value of the root node score, child node score and leaf node score in the real evaluation score) is determined as the real evaluation score.

[0190] In some embodiments, when the third preset value is 1, the predicted evaluation score, the actual evaluation score, and the third evaluation value satisfy the following formula 1:

[0191] Formula 1;

[0192] in, This indicates the third assessment value. Indicates the predicted assessment score. Indicates the actual assessment score. This indicates the evaluation deviation value.

[0193] S304. Based on the output sequence, the first evaluation value, the second evaluation value, and the third evaluation value, determine the target loss value of the output sequence.

[0194] Specifically, a first loss value is determined by combining the user question, reasoning process, and user intent. This first loss value is used to supervise and fine-tune the accuracy of the reasoning process and the generation of user intent. A second loss value is determined by combining the evaluation process and evaluation score to optimize the generation quality of the evaluation process and evaluation score. Subsequently, the first, second, and third evaluation values ​​are fused to obtain a comprehensive evaluation value, which is used to calculate the third loss value for reinforcement learning. Finally, the first, second, and third loss values ​​are added together to obtain the target loss value of the output sequence, thereby achieving multi-objective joint optimization training of the first model.

[0195] In this embodiment, for any output sequence of a user question: a first evaluation value is determined based on the user intent in the output sequence and the standard user intent corresponding to the user question, the first evaluation value being used to indicate the accuracy of user intent recognition; a second evaluation value is determined based on the output sequence, the second evaluation value being used to indicate whether the output sequence is complete; a third evaluation value is determined based on the evaluation score in the output sequence and the user question, the third evaluation value being used to indicate the reliability of the self-evaluation result of the first model; and a target loss value for the output sequence is determined based on the output sequence, the first evaluation value, the second evaluation value, and the third evaluation value. In the above method, the first model is guided to generate correct user intent through an accuracy reward (first evaluation value), and the output sequence is ensured to follow the preset structured output specifications through a format reward (second evaluation value). Furthermore, the consistency reward (third evaluation value) promotes the alignment of the first model's internal evaluation logic with external objective standards. Moreover, white-box reinforcement learning is achieved through the explicit reasoning and evaluation processes in the structured output (i.e., the model's decision-making basis, reasoning link, and self-evaluation logic are all presented in a parsable form). This makes the intent recognition logic of the first model traceable and interpretable, avoiding the decision-making blindness of black-box reinforcement learning (i.e., relying solely on input-output mapping relationships for optimization, the model's internal logic and reasoning process are invisible and uninterpretable, and the optimization direction lacks process support). Thus, more stable and reliable optimization is achieved during the training process, effectively improving the accuracy of the user intent model in recognizing user intent.

[0196] Hereinafter, based on any of the above embodiments, by... Figure 4 The document further explains the process of "determining the target loss value of the output sequence based on the output sequence, the first evaluation value, the second evaluation value, and the third evaluation value".

[0197] Figure 4 A flowchart illustrating a method for determining a target loss value provided in an embodiment of this application. Figure 2 ,like Figure 4 As shown, the method includes:

[0198] S401. Based on the user's question, reasoning process, and user intent, determine the first loss value, wherein the first loss value is used for first supervised fine-tuning training.

[0199] In some embodiments, the user question, reasoning process, user intent, and first loss value satisfy the following formula 2:

[0200] Formula 2;

[0201] in, This represents the first loss value. This represents the i-th word element generated by the first model during the reasoning process and user intent generation. This indicates a user issue. This indicates a user issue. This indicates that, given the context and problem, the first model predicts the generated... The probability, The logarithm represents the probability. This represents the sum of the logarithmic probabilities of all lexical units in the first model during the generation of inference and user intent. , where n represents the total number of all lexical units in the first model during the generation of reasoning processes and user intents.

[0202] The first supervised fine-tuning training is used to make the user intent generated by the first model more accurate in the next round.

[0203] S402. Based on the user question, reasoning process, user intent, evaluation process, and evaluation score, determine a second loss value, wherein the second loss value is used for second supervised fine-tuning training.

[0204] In some embodiments, the user question, reasoning process, user intent, evaluation process, evaluation score, and first loss value satisfy the following formula 3:

[0205] Formula 3;

[0206] in, This represents the second loss value. This represents the i-th word element generated by the first model during the evaluation process and evaluation score generation. This represents the reasoning process and user intent. This indicates a user issue. This indicates that, given the context and problem, the first model predicts the generated... The probability, The logarithm represents the probability. This represents the sum of the logarithmic probabilities of all words in the first model during the generation and evaluation process and the evaluation score. , m represents the total number of all lexical units in the first model during the generation of the evaluation process and evaluation scores.

[0207] The second supervised fine-tuning training is used to make the evaluation process and evaluation scores generated by the first model more accurate in the next round.

[0208] S403. Based on the user question, the output sequence, the first evaluation value, the second evaluation value, and the third evaluation value, determine the third loss value, which is used for reinforcement learning training.

[0209] In some embodiments, a third loss value is determined based on the user question, the output sequence, a first evaluation value, a second evaluation value, and a third evaluation value, including:

[0210] The sum of the first evaluation value, the second evaluation value, and the third evaluation value is determined as the comprehensive evaluation value;

[0211] The third loss value is determined based on the user question, the output sequence, and the comprehensive evaluation value.

[0212] In some embodiments, the first evaluation value, the second evaluation value, the third evaluation value, and the comprehensive evaluation value satisfy the following formula 4:

[0213] Formula 4;

[0214] in, This represents the overall evaluation value. This indicates the first evaluation value. This indicates the second evaluation value. This indicates the third evaluation value.

[0215] In some embodiments, the user question, output sequence, comprehensive evaluation value, and third loss value satisfy the following formula 5:

[0216] Formula 5;

[0217] in, This represents the third loss value. This represents the output sequence generated by the first model. This indicates a user issue. This represents the strategy of the first model. This represents the logarithmic probability that the policy network generates the action given the current state. R_total represents the expected value of the product of the overall evaluation value and the log probability of the action under policy πθ.

[0218] The strategy of the first model can be understood as the probability distribution of each word generated by the first model.

[0219] S404. Add the first loss value, the second loss value, and the third loss value to obtain the target loss value.

[0220] In some embodiments, the first loss value, the second loss value, the third loss value, and the target loss value satisfy the following formula 6:

[0221] = + + Formula 6;

[0222] in, This represents the target loss value.

[0223] In this embodiment, a first loss value is determined based on the user question, reasoning process, and user intent, and the first loss value is used for first supervised fine-tuning training; a second loss value is determined based on the user question, reasoning process, user intent, evaluation process, and evaluation score, and the second loss value is used for second supervised fine-tuning training; a third loss value is determined based on the user question, output sequence, first evaluation value, second evaluation value, and third evaluation value, and the third loss value is used for reinforcement learning training; the first loss value, second loss value, and third loss value are added together to obtain the target loss value. In the above method, a multi-objective joint optimization training loss calculation mechanism is introduced to effectively coordinate supervised fine-tuning and reinforcement learning. The first loss value accurately optimizes the reasoning and intent generation capabilities of the first model. Then, the second loss value is used to specifically optimize the model's evaluation process and evaluation score. Finally, the reinforcement learning loss incorporates high-level semantic reward signals such as intent accuracy, format integrity, and evaluation consistency into the optimization process. This hierarchical loss design ensures that the first model can self-correct and improve based on multi-dimensional feedback while following the structured output specifications. Ultimately, it achieves better convergence, accuracy, and robustness in the user intent recognition task.

[0224] Hereinafter, based on any of the above embodiments, by... Figure 5 The document further explains the process of "training the first model using a hybrid approach of supervised fine-tuning and reinforcement learning based on multiple user questions and the target output sequence corresponding to each user question, to obtain a user intent model".

[0225] Figure 5 A flowchart illustrating a method for determining a user intent model provided in an embodiment of this application is shown below. Figure 5 As shown, the method includes:

[0226] S501. Average the target loss values ​​of multiple user questions to obtain the current round loss value for the mixed training.

[0227] Understandably, the first model needs to undergo forward inference processing and hybrid training in each round.

[0228] Understandably, the round loss value is used to characterize the overall loss level of the first model on all user questions in the current round. By averaging the target loss values ​​of all samples, the loss fluctuations caused by a single sample can be smoothed out, and the overall training effect of the first model can be reflected more stably.

[0229] S502. Based on the round loss value of the current round and the round loss values ​​of the first number of rounds prior to the current round, determine whether the loss curve has converged.

[0230] If so, execute S503;

[0231] Otherwise, execute S504.

[0232] Understandably, when the current round is the first to the first number of rounds of hybrid training, since a sufficient number of historical round loss values ​​have not yet been accumulated, a stable trend of loss change cannot be formed. Therefore, the convergence judgment is not performed for the time being, and the forward inference processing and hybrid training of the next round are continued.

[0233] The first quantity is, for example, 5.

[0234] The following example illustrates how to determine whether the loss curve converges based on the round loss value of the current round and the round loss values ​​of the first number of rounds prior to the current round:

[0235] For example, if the first quantity is 3 and the current round is the 5th round, then the first number of rounds before the current round includes the 2nd round, the 3rd round, and the 4th round;

[0236] The first difference between the round loss values ​​of the second and third rounds, the second difference between the round loss values ​​of the third and fourth rounds, and the third difference between the round loss values ​​of the fourth and fifth rounds are determined sequentially.

[0237] If the first difference is less than the preset convergence threshold, the second difference is less than the preset convergence threshold, and the third difference is less than the preset convergence threshold, then the loss curve is determined to have converged.

[0238] Otherwise, it is determined that the loss curve has not converged.

[0239] It should be noted that the above is only an illustrative example. In actual training, the first quantity and convergence threshold can be flexibly set according to the convergence speed of the first model, the scale of training data and business needs. This application does not limit this.

[0240] S503. Determine the first model of the current round as the user intent model.

[0241] Specifically, the convergence of the loss curve means that the first model, under the hybrid training of supervised fine-tuning and reinforcement learning, has a stable and accurate user intent recognition capability, which meets the usage requirements. Therefore, the first model trained in the current round is taken as the final user intent model.

[0242] S504. Based on multiple user questions and the target output sequence corresponding to each user question, update the parameters of the first model to obtain the updated first model, and perform the next round of forward inference processing and hybrid training based on the updated first model until the loss curve converges.

[0243] Specifically, the loss curve did not converge, indicating that the reliability of the first model's user intent recognition still has room for improvement and requires further iterative training.

[0244] Based on multiple user questions and the target output sequence corresponding to each user question, the parameters of the first model are updated. This can be understood as follows: using the target output sequence corresponding to each user question as a supervision signal, combined with the supervision fine-tuning loss and the reinforcement learning reward signal, the network parameters of the first model are backpropagated and optimized, so that the first model is more inclined to generate output sequences with high accuracy, complete format and reliable self-evaluation in subsequent inference.

[0245] In this embodiment, the target loss values ​​of multiple user questions are averaged to obtain the current round loss value of the hybrid training. Based on the current round loss value and the round loss values ​​of the first number of rounds prior to the current round, it is determined whether the loss curve has converged. If the loss curve converges, the first model under the current round is determined as the user intent model. If the loss curve has not converged, the parameters of the first model are updated based on multiple user questions and the target output sequence corresponding to each user question to obtain the updated first model. The next round of forward inference processing and hybrid training are then performed based on the updated first model until the loss curve converges. In the above method, the average loss is calculated after each training round, and the convergence status of the model is intelligently judged based on the recent trend of the loss curve. When the loss curve is determined to be converged, the training is automatically terminated and the final user intent model is output, ensuring the stability of the user intent model performance and training efficiency. When the loss curve has not converged, the iterative update of the first model and the training of the next round are triggered, forming a closed loop of continuous optimization, which effectively avoids overfitting and underfitting, and can stop in time when the first model reaches the optimal performance, thereby obtaining a user intent model with strong generalization ability and accurate and reliable intent recognition.

[0246] In summary, this application is the first to introduce the Post-Completion Learning (PCL) paradigm into the field of hierarchical intent recognition, proposing a method for decoupling training and inference. Specifically, PCL involves guiding the first model to continue outputting a structured evaluation process and evaluation scores after generating hierarchical user intents during the training phase. Furthermore, a reinforcement learning mechanism is used to internalize the ability to judge intent hierarchy and compliance within the first model, thereby improving the accuracy of recognizing complex levels of user intents.

[0247] Figure 6 This is a schematic diagram of the structure of a training device for a user intent model provided in an embodiment of this application, as shown below. Figure 6 As shown, the user intent model training device 600 provided in this embodiment includes:

[0248] The first processing module 601 is used to obtain a preset intent hierarchy tree and multiple sample data. The intent hierarchy tree indicates the hierarchy of user intents, and the sample data includes user questions and standard user intents.

[0249] The second processing module 602 is used to construct a standard output format of the first model based on the intent hierarchy tree. The standard output format indicates that the output sequence generated by the first model includes, in sequence: the reasoning process for the user question, the user intent based on the reasoning process, the stop flag, the evaluation process for the user intent, and the evaluation score for the evaluation process.

[0250] The third processing module 603 is used to perform forward reasoning processing on multiple user questions based on the standard output format and the first model, and generate multiple output sequences for each user question.

[0251] The fourth processing module 604 is used to perform hybrid training of the first model, including supervised fine-tuning and reinforcement learning, based on multiple output sequences of each user question and the corresponding standard user intent, to obtain the user intent model.

[0252] The user intent model training device 600 provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0253] In some embodiments, the third processing module 603 is specifically used for:

[0254] Use the standard output format as the prompt input for the first model;

[0255] For any given user question, any output sequence;

[0256] Based on the first model, the user's question is processed to generate a reasoning process.

[0257] Based on the first model, intent parsing is performed on each reasoning process to obtain the user intent, and the user intent is then concatenated after the reasoning process.

[0258] A stop marker is appended after the user's intent;

[0259] Based on the first model, the user intent is evaluated and reasoned to obtain the evaluation process, and the evaluation process is concatenated after the stop flag.

[0260] Based on the first model, score prediction processing is performed on each evaluation process to obtain the evaluation score. After the evaluation process, the evaluation scores are concatenated to obtain the output sequence.

[0261] In some embodiments, the fourth processing module 604 is specifically used for:

[0262] For any given user problem; based on the user problem and its multiple output sequences, determine the target loss value for each output sequence; and determine the output sequence corresponding to the minimum target loss value among the multiple target loss values ​​as the target output sequence.

[0263] Based on multiple user questions and the target output sequence corresponding to each user question, the first model is trained using a hybrid method that includes supervised fine-tuning and reinforcement learning to obtain the user intent model.

[0264] In some embodiments, the fourth processing module 604 is specifically used for:

[0265] Any output sequence for the user's question;

[0266] Based on the user intent in the output sequence and the standard user intent corresponding to the user question, a first evaluation value is determined. The first evaluation value is used to indicate the accuracy of the user intent recognition.

[0267] Based on the output sequence, a second evaluation value is determined, which is used to indicate whether the output sequence is complete.

[0268] Based on the evaluation scores and user questions in the output sequence, a third evaluation value is determined, which is used to indicate the reliability of the self-evaluation results of the first model;

[0269] Based on the output sequence, the first evaluation value, the second evaluation value, and the third evaluation value, the target loss value of the output sequence is determined.

[0270] In some embodiments, the fourth processing module 604 is specifically used for:

[0271] Determine whether the user intent in the output sequence is the same as the standard user intent corresponding to the user question;

[0272] When the user intent in the output sequence is the same as the standard user intent corresponding to the user question, the first evaluation value is determined to be the first preset value. The first evaluation value indicates that the user intent is accurately identified.

[0273] When the user intent in the output sequence is different from the standard user intent corresponding to the user question, the first evaluation value is determined to be the second preset value. The first evaluation value being the second preset value indicates that the user intent recognition is inaccurate.

[0274] In some embodiments, the fourth processing module 604 is specifically used for:

[0275] Determine whether the output sequence includes reasoning, user intent, evaluation process, and evaluation score;

[0276] When the output sequence includes the reasoning process, user intent, evaluation process, and evaluation score, the second evaluation value is determined to be the first preset value, and the second evaluation value to be the first preset value indicates that the output sequence is complete;

[0277] If the output sequence does not include any one or more of the following: reasoning process, user intent, evaluation process, or evaluation score, the second evaluation value is determined to be the second preset value. The second evaluation value indicates that the output sequence is incomplete.

[0278] In some embodiments, the fourth processing module 604 is specifically used for:

[0279] Based on the evaluation scores, determine the predicted evaluation scores of user intent in the output sequence;

[0280] Based on the preset first script, the user's question is analyzed for intent and the score is quantified to obtain the true evaluation score;

[0281] The absolute value of the difference between the predicted assessment score and the actual assessment score is determined as the assessment deviation value;

[0282] The difference between the third preset value and the evaluation deviation value is determined as the third evaluation value.

[0283] In some embodiments, the fourth processing module 604 is specifically used for:

[0284] Based on the user question, reasoning process, and user intent, a first loss value is determined, which is used for first-supervised fine-tuning training.

[0285] Based on the user question, reasoning process, user intent, evaluation process, and evaluation score, a second loss value is determined, which is used for second-supervised fine-tuning training.

[0286] Based on the user question, the output sequence, the first evaluation value, the second evaluation value, and the third evaluation value, a third loss value is determined, which is used for reinforcement learning training.

[0287] The target loss value is obtained by adding the first loss value, the second loss value, and the third loss value.

[0288] In some embodiments, the fourth processing module 604 is specifically used for:

[0289] The sum of the first evaluation value, the second evaluation value, and the third evaluation value is determined as the comprehensive evaluation value;

[0290] The third loss value is determined based on the user question, the output sequence, and the comprehensive evaluation value.

[0291] In some embodiments, the fourth processing module 604 is specifically used for:

[0292] The average of the target loss values ​​for multiple user questions is used to obtain the epoch loss value for the current epoch of the mixed training.

[0293] Based on the round loss value of the current round and the round loss values ​​of the first number of rounds prior to the current round, determine whether the loss curve converges;

[0294] When the loss curve converges, the first model in the current round is determined as the user intent model;

[0295] When the loss curve has not converged, the parameters of the first model are updated based on multiple user questions and the target output sequence corresponding to each user question, resulting in an updated first model. The next round of forward inference processing and hybrid training is then performed based on the updated first model until the loss curve converges.

[0296] The user intent model training device 600 provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0297] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 7 As shown, the electronic device 700 includes a processor 701 and a memory 702. The processor 701 is communicatively connected to the memory 702, which stores computer execution instructions. The processor 701 is configured to execute the technical solutions in any of the aforementioned method embodiments by executing the computer execution instructions stored in the memory 702.

[0298] Optionally, the memory 702 can be either independent or integrated with the processor 701. Optionally, when the memory 702 is a device independent of the processor 701, the electronic device 700 may further include a bus 703 for connecting the aforementioned devices.

[0299] The electronic device is used to execute the technical solutions in any of the foregoing method embodiments. Its implementation principle and technical effect are similar, and will not be described again here.

[0300] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0301] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0302] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0303] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0304] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described method.

[0305] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0306] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0307] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0308] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0309] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0310] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0311] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0312] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A method for training a user intent model, characterized in that, include: Obtain a preset intent hierarchy tree and multiple sample data, wherein the intent hierarchy tree indicates the hierarchy of user intents and the sample data includes user questions and standard user intents; Based on the intent hierarchy tree, a standard output format for the first model is constructed. The standard output format indicates that the output sequence generated by the first model includes, in sequence: the reasoning process for the user question, the user intent based on the reasoning process, a stop flag, the evaluation process for the user intent, and the evaluation score for the evaluation process. Based on the standard output format and the first model, forward reasoning is performed on multiple user questions to generate multiple output sequences for each user question. Based on the multiple output sequences of each user question and the corresponding standard user intent, the first model is trained using a hybrid method that includes supervised fine-tuning and reinforcement learning to obtain a user intent model.

2. The method according to claim 1, characterized in that, Based on the standard output format and the first model, forward reasoning is performed on multiple user questions to generate multiple output sequences for each user question, including: Input the standard output format as a prompt into the first model; For any given user question, any output sequence; Based on the first model, the user question is subjected to reasoning generation processing to obtain the reasoning process; Based on the first model, intent parsing is performed on each reasoning process to obtain the user intent, and the user intent is concatenated after the reasoning process. The stop identifier is appended after the user intent; Based on the first model, the user intent is evaluated and reasoned to obtain the evaluation process, and the evaluation process is concatenated after the stop flag; Based on the first model, score prediction processing is performed on each evaluation process to obtain the evaluation score, and the evaluation score is concatenated after the evaluation process to obtain the output sequence.

3. The method according to claim 1, characterized in that, Based on the multiple output sequences of each user question and the corresponding standard user intent, the first model is subjected to hybrid training including supervised fine-tuning and reinforcement learning to obtain a user intent model, including: For any given user problem; based on the user problem and its multiple output sequences, determine the target loss value for each output sequence; and determine the output sequence corresponding to the smallest target loss value among the multiple target loss values ​​as the target output sequence. Based on the multiple user questions and the target output sequence corresponding to each user question, the first model is subjected to hybrid training including supervised fine-tuning and reinforcement learning to obtain the user intent model.

4. The method according to claim 3, characterized in that, The step of determining the target loss value for each output sequence based on the user question and multiple output sequences of the user question includes: For any one of the output sequences for the user question; Based on the user intent in the output sequence and the standard user intent corresponding to the user question, a first evaluation value is determined, which is used to indicate the accuracy of the recognition of the user intent; Based on the output sequence, a second evaluation value is determined, which indicates whether the output sequence is complete. Based on the evaluation scores in the output sequence and the user question, a third evaluation value is determined, which is used to indicate the reliability of the self-evaluation results of the first model; Based on the output sequence, the first evaluation value, the second evaluation value, and the third evaluation value, the target loss value of the output sequence is determined.

5. The method according to claim 4, characterized in that, The step of determining the first evaluation value based on the user intent in the output sequence and the standard user intent corresponding to the user question includes: Determine whether the user intent in the output sequence is the same as the standard user intent corresponding to the user question; When the user intent in the output sequence is the same as the standard user intent corresponding to the user question, the first evaluation value is determined to be a first preset value, and the first evaluation value indicates that the user intent is accurately identified. When the user intent in the output sequence is different from the standard user intent corresponding to the user question, the first evaluation value is determined to be a second preset value. The first evaluation value being a second preset value indicates that the user intent is not accurately identified.

6. The method according to claim 4, characterized in that, Determining the second evaluation value based on the output sequence includes: Determine whether the output sequence includes the reasoning process, the user intent, the evaluation process, and the evaluation score; When the output sequence includes the reasoning process, the user intent, the evaluation process, and the evaluation score, the second evaluation value is determined to be a first preset value, and the second evaluation value to be the first preset value indicates that the output sequence is complete; If the output sequence does not include any one or more of the reasoning process, the user intent, the evaluation process, or the evaluation score, the second evaluation value is determined to be a second preset value, and the second evaluation value indicates that the output sequence is incomplete.

7. The method according to claim 4, characterized in that, The process of determining a third evaluation value based on the evaluation score in the output sequence and the user question includes: Based on the evaluation score, a predicted evaluation score for the user intent in the output sequence is determined; Based on a preset first script, the user's question is subjected to intent analysis and score quantification to obtain a true evaluation score; The absolute value of the difference between the predicted evaluation score and the actual evaluation score is determined as the evaluation deviation value; The difference between the third preset value and the evaluation deviation value is determined as the third evaluation value.

8. The method according to claim 4, characterized in that, Determining the target loss value of the output sequence based on the output sequence, the first evaluation value, the second evaluation value, and the third evaluation value includes: Based on the user question, the reasoning process, and the user intent, a first loss value is determined, and the first loss value is used for first supervised fine-tuning training. Based on the user question, the reasoning process, the user intent, the evaluation process, and the evaluation score, a second loss value is determined, and the second loss value is used for second supervised fine-tuning training. Based on the user question, the output sequence, the first evaluation value, the second evaluation value, and the third evaluation value, a third loss value is determined, which is used for reinforcement learning training. The first loss value, the second loss value, and the third loss value are added together to obtain the target loss value.

9. The method according to claim 8, characterized in that, The step of determining the third loss value based on the user question, the output sequence, the first evaluation value, the second evaluation value, and the third evaluation value includes: The sum of the first evaluation value, the second evaluation value, and the third evaluation value is determined as the comprehensive evaluation value; A third loss value is determined based on the user question, the output sequence, and the comprehensive evaluation value.

10. The method according to claim 3, characterized in that, The user intent model is obtained by performing hybrid training, including supervised fine-tuning and reinforcement learning, on the first model based on the multiple user questions and the target output sequence corresponding to each user question, including: The average of the target loss values ​​for the multiple user questions is used to obtain the round loss value for the current round of the mixed training. Based on the round loss value of the current round and the round loss values ​​of the first number of rounds prior to the current round, determine whether the loss curve converges; When the loss curve converges, the first model in the current round is determined as the user intent model; When the loss curve fails to converge, the parameters of the first model are updated based on the multiple user questions and the target output sequence corresponding to each user question to obtain the updated first model. Then, the next round of forward inference processing and hybrid training is performed based on the updated first model until the loss curve converges.

Images