Model training methods, task processing methods, devices, equipment, media, and products

Abstract

This disclosure provides a model training method, task processing method, apparatus, device, medium, and product. The model training method includes: determining the reward value corresponding to the predicted text set in multiple preset optimization dimensions based on the predicted text set obtained by processing training data by the text generation model in the current training round and the sample label set corresponding to the training data; determining the advantage value of each preset optimization dimension based on the reward value corresponding to each preset optimization dimension; determining the dimensional loss value of each preset optimization dimension based on the preset dimensional weight and the advantage value of each preset optimization dimension; and training the text generation model based on the dimensional loss values ​​of multiple preset optimization dimensions to obtain the text generation model for the next training round. Embodiments of this disclosure can improve the training accuracy of the text generation model.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, and in particular to a model training method, a task processing method and apparatus, an electronic device, a computer-readable storage medium, and a program product. Background Technology

[0002] In the development of training methods for Reinforcement Learning (RL), policy optimization has always been a core direction for improving the model's decision-making ability. In recent years, Group Relative Policy Optimization (GRPO) has attracted attention as an emerging collaborative training paradigm.

[0003] Currently, model training methods based on Group Relative Policy Optimization (GRPO) typically adjust the training gradient during model training based on the dominance values ​​of different groups. However, in this process, the allocation of dominance values ​​is too coarse, making it impossible to distinguish the contributions of different fields or segments in the model's output text. Overly coarse-grained dominance values ​​make it difficult to accurately locate training errors, thus reducing the accuracy of model training. Summary of the Invention

[0004] This disclosure provides a model training method, a task processing method, and apparatus, equipment, medium, and product.

[0005] In a first aspect, this disclosure provides a model training method, comprising: determining reward values ​​corresponding to the predicted text set in multiple preset optimization dimensions based on the predicted text set obtained by processing training data by the text generation model in the current training round and the sample label set corresponding to the training data, wherein the predicted text set includes multiple predicted texts and the sample label set includes multiple sample labels corresponding to the training data; determining an advantage value for each preset optimization dimension based on the reward value corresponding to each preset optimization dimension, wherein the advantage value for each preset optimization dimension is used to characterize the relative advantage of each predicted text in each preset optimization dimension; determining a dimensional loss value for each preset optimization dimension based on the preset dimensional weights of each preset optimization dimension and the advantage value of each preset optimization dimension; and training the text generation model based on the dimensional loss values ​​of the multiple preset optimization dimensions to obtain the text generation model for the next training round.

[0006] Secondly, this disclosure provides a task processing method, comprising: upon receiving a task processing instruction, inputting the task processing instruction into a text generation model to obtain task parsing text; and executing task processing operations in the task parsing text, wherein the text generation model is trained based on the aforementioned model training method.

[0007] Thirdly, this disclosure provides a model training apparatus, comprising: a reward determination module configured to determine reward values ​​corresponding to multiple preset optimization dimensions for the predicted text set obtained by processing training data by the text generation model in the current training round and the sample label set corresponding to the training data, wherein the predicted text set includes multiple predicted texts and the sample label set includes multiple sample labels corresponding to the training data; an advantage determination module configured to determine an advantage value for each preset optimization dimension based on the reward value corresponding to each preset optimization dimension, wherein the advantage value for each preset optimization dimension is used to characterize the relative advantage of each predicted text in each preset optimization dimension; a loss determination module configured to determine a dimensional loss value for each preset optimization dimension based on the preset dimensional weights of each preset optimization dimension and the advantage value of each preset optimization dimension; and a training module configured to train the text generation model based on the dimensional loss values ​​of the multiple preset optimization dimensions to obtain a text generation model for the next training round.

[0008] Fourthly, this disclosure provides a task processing apparatus, comprising: an input module configured to input a task processing instruction into a text generation model upon receiving a task processing instruction to obtain task parsing text; and an execution module configured to execute task processing operations in the task parsing text, wherein the text generation model is trained based on the aforementioned model training apparatus.

[0009] Fifthly, this disclosure provides an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores one or more computer programs executable by the at least one processor, the one or more computer programs being executed by the at least one processor to enable the at least one processor to perform the model training method or task processing method described above.

[0010] Sixthly, this disclosure provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the above-described model training method or task processing method.

[0011] In a seventh aspect, this disclosure provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is run in a processor of an electronic device, the processor in the electronic device executes the model training method or task processing method described above.

[0012] The embodiments provided in this disclosure can, in each training epoch of training a text generation model, determine the reward values ​​corresponding to multiple predicted texts in multiple preset optimization dimensions based on the predicted text set obtained by the text generation model from processing the training data and the sample label set corresponding to the training data. This allows for further determination of advantage values ​​based on the reward values ​​of different preset optimization dimensions, thus achieving the determination of advantage values ​​for multiple preset optimization dimensions. In subsequent model training, based on the advantage values ​​of multiple preset optimization dimensions and the preset dimensional weights of each preset optimization dimension, a loss value for updating the model parameters of the text generation model is determined, refining the loss value to each preset optimization dimension. Therefore, during the training of the text generation model based on the dimensional loss values ​​of multiple preset optimization dimensions, the accuracy of updating model parameters is improved, further enhancing the training precision of the text generation model and improving its performance in practical applications.

[0013] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0014] The accompanying drawings are provided to further illustrate the present disclosure and form part of the specification. They are used together with the embodiments of the present disclosure to explain the disclosure and do not constitute a limitation thereof. The above and other features and advantages will become more apparent to those skilled in the art from the detailed description of exemplary embodiments with reference to the accompanying drawings, in which:

[0015] Figure 1 This is a schematic diagram illustrating an application scenario of a model training method and a task processing method provided in an embodiment of the present disclosure;

[0016] Figure 2 A flowchart of a model training method provided in this embodiment of the disclosure;

[0017] Figure 3 A flowchart illustrating a model training method provided in this embodiment of the disclosure;

[0018] Figure 4 A flowchart of a task processing method provided in an embodiment of this disclosure;

[0019] Figure 5 A block diagram of a model training apparatus provided in an embodiment of this disclosure;

[0020] Figure 6 A block diagram of a task processing apparatus provided in an embodiment of this disclosure;

[0021] Figure 7 This is a block diagram of an electronic device provided in an embodiment of the present disclosure. Detailed Implementation

[0022] To enable those skilled in the art to better understand the technical solutions of this disclosure, exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments of this disclosure to aid understanding. These should be considered merely exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0023] Where there is no conflict, the various embodiments of this disclosure and the features thereof in the embodiments may be combined with each other.

[0024] As used herein, the term “and / or” includes any and all combinations of one or more related enumerated entries.

[0025] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. As used herein, the singular forms “a” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that when the terms “comprising” and / or “made of” are used in this specification, the presence of the stated feature, integral, step, operation, element, and / or component is specified, but the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof is not excluded. Words such as “connected” or “linked” are not limited to physical or mechanical connections but can include electrical connections, whether direct or indirect.

[0026] Unless otherwise specified, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and this disclosure, and will not be interpreted as having an idealized or overly formal meaning, unless expressly so defined herein.

[0027] GRPO aims to generate policy gradient signals more efficiently by leveraging collaborative sharing within groups or competitive comparisons between groups, thereby guiding policies to evolve towards greater robustness or higher relative performance. Traditional GRPO uses a single dominance value for each text sequence, adjusting the gradient of the log-likelihood of the entire token segment, failing to differentiate the contributions of different fields or segments within the sequence. Furthermore, the single dominance value is shared by all tokens in the text sequence, making field-level differentiation impossible.

[0028] Based on this, the present disclosure provides a model training method and apparatus, a task processing method and apparatus, an electronic device, a computer-readable storage medium, and a computer program product, which are described in detail in the following embodiments.

[0029] The model training method or task processing method according to embodiments of this disclosure can be executed by electronic devices such as terminal devices or servers. The terminal device can be an in-vehicle device, user equipment (UE), mobile device, user terminal, terminal, cellular phone, cordless phone, personal digital assistant (PDA), handheld device, computing device, in-vehicle device, wearable device, etc. The method can be implemented by a processor calling computer-readable program instructions stored in memory. Alternatively, the method can be executed by a server.

[0030] See Figure 1 , Figure 1 This is a schematic diagram illustrating an application scenario of a model training method and a task processing method provided in an embodiment of this disclosure, such as... Figure 1In the application scenario shown, users can interact with the intelligent interactive interface on their user terminal. For example, a user can trigger a command such as "Open XX application, play XX music" on the intelligent interactive interface. This command can be a voice command or a text command, and the form of the triggering command can be determined according to the actual application situation. This disclosure does not limit the form of the triggering command here. When the user terminal receives the command processed by the user, it inputs the command (if it is a voice command, it can be converted into a text command) into a text generation model. The text generation model processes the input command and outputs the corresponding task parsing text. The task parsing text includes instructions for executing the processing operation corresponding to the command, as well as relevant information related to the processing operation, so that the user terminal can execute the command according to the task parsing text. For example, the processing operation included in the task parsing text is "Open XX application", "Find XX music in XX application", "Play XX music", etc. The relevant information of the processing operation can include the attribute information of XX application, the position information of the music playback control, etc. The user terminal can execute the command triggered by the user according to the above task parsing text to complete the music playback.

[0031] In the aforementioned application scenarios, the accuracy of the text generation model's processing of trigger commands and the output of the parsed task text is crucial. Therefore, this disclosure provides a model training method to improve the training accuracy of the text generation model.

[0032] It should be noted that, Figure 1 The application scenarios shown are merely illustrative examples of the model training and task processing methods provided in this disclosure. The applications of the model training and task processing methods provided in this disclosure are not limited to those described above. Figure 1 The application scenarios shown are as follows.

[0033] Figure 2 A flowchart illustrating a model training method provided in an embodiment of this disclosure. See also... Figure 2 The method specifically includes the following steps:

[0034] Step 202: Based on the predicted text set obtained by the text generation model in the current training round processing the training data and the sample label set corresponding to the training data, determine the reward value corresponding to the predicted text set in multiple preset optimization dimensions.

[0035] In this context, a text generation model refers to the model that needs to be trained using the model training method. The trained text generation model is used to generate task parsing text for task processing instructions. The task parsing text is used to indicate the task processing operations required to execute the task processing instructions. In practical applications, text generation models can be Large Language Models (LLM), Multimodal Large Language Models (MLLM), etc.

[0036] Training data refers to the sample data used to train the text generation model. Training data includes historical task processing instructions triggered by the user and processed by the text generation model. These instructions can be either voice or text. The predicted text set refers to the set of text output by the text generation model after processing the training data. The predicted text set includes multiple predicted texts. The sample label set refers to the set of standard texts corresponding to the training data, including multiple sample labels. Preset optimization dimensions refer to the dimensions that need to be optimized for the text generation model, including format, inference chain, actions, coordinates, and values. Format refers to the text format output by the text generation model; inference chain refers to the inference chain of the text output by the text generation model; actions refer to the actions included in the text output by the text generation model, such as clicking, inputting, or searching; coordinates refer to the position coordinates of any object included in the text output by the text generation model; and values ​​refer to the user-inputted text or pre-constructed prompts included in the text output by the text generation model. The content of the values ​​can be determined according to the actual application.

[0037] For example, sample labels may include the following: <action> CLICK< / action> , <bboxes> …< / bboxes> , <value> …< / value> wait.

[0038] Specifically, in the current training round of training the text generation model, after inputting the training data into the text generation model, multiple predicted texts output by the text generation model can be obtained. Based on the multiple predicted texts of the current training round and the multiple sample labels corresponding to the training data, the reward value corresponding to each predicted text in multiple preset optimization dimensions is determined.

[0039] As described above, in this embodiment of the disclosure, multiple preset optimization dimensions are pre-set. To improve the accuracy of determining the reward value of each preset optimization dimension, this embodiment of the disclosure sets a reward model for each preset optimization dimension, and determines the reward value of each preset optimization dimension through the reward model corresponding to each preset optimization dimension. The specific implementation method is as follows:

[0040] In one specific embodiment provided in this disclosure, the reward value corresponding to the predicted text set in multiple preset optimization dimensions is determined based on the predicted text set obtained by the text generation model in the current training round processing the training data and the sample label set corresponding to the training data. This includes: inputting the multiple predicted texts and the multiple sample labels into a target reward model to obtain the target reward value of the multiple predicted texts corresponding to the target optimization dimension output by the target reward model; wherein, the target optimization dimension is any one of the multiple preset optimization dimensions, and the target reward model is used to determine the reward value of the predicted text set in the target optimization dimension.

[0041] Specifically, for any one of the multiple preset optimization dimensions (i.e., the target optimization dimension), multiple predicted texts and multiple sample labels are input into the reward model corresponding to that preset optimization dimension to obtain the reward value for each predicted text in that preset optimization dimension. In practical applications, the reward model can be an Outcome Reward Model (ORM) or an Offline Reward Model (ORM).

[0042] In this embodiment, a reward model is set for each preset optimization dimension, so that when determining the reward value of the predicted text in different preset optimization dimensions, there is a corresponding reward model that can be used, thereby improving the accuracy of determining the reward value of each preset optimization dimension.

[0043] Step 204: Determine the advantage value of each preset optimization dimension based on the reward value corresponding to each preset optimization dimension.

[0044] After determining the reward value for the predicted text in each preset optimization dimension, the advantage value for each preset optimization dimension can be determined based on the reward value. The advantage value characterizes the relative advantage of the predicted text in each preset optimization dimension, that is, the degree of deviation of the predicted text from the baseline within that dimension. Therefore, before determining the relative advantage of the predicted text in each preset optimization dimension, a baseline value for measuring the relative advantage needs to be determined first, and then the advantage value for each preset optimization dimension is determined based on the reward value and the baseline value.

[0045] In one specific embodiment provided in this disclosure, determining the advantage value of each preset optimization dimension based on the reward value corresponding to each preset optimization dimension includes: determining a baseline reward value based on a plurality of target reward values; and determining the target advantage value of the target optimization dimension based on the plurality of target reward values ​​and the baseline reward value.

[0046] The baseline reward value measures the relative advantage of the predicted text in the preset optimization dimensions, with each preset optimization dimension corresponding to a baseline reward value. The target advantage value refers to the advantage of the predicted text in the target optimization dimension.

[0047] Specifically, for any one of the multiple preset optimization dimensions (i.e., the target optimization dimension), obtain the reward values ​​of multiple predicted texts under the target optimization dimension. Calculate the mean among the reward values ​​of the multiple predicted texts, and determine this mean as the baseline reward value of the target optimization dimension; or determine the median among the reward values ​​of the multiple predicted texts, and determine this median as the baseline reward value of the target optimization dimension; or, after determining the mean or median, determine the standard deviation among the reward values ​​of the multiple predicted texts, divide the mean or median by the standard deviation, and determine the result after dividing by the standard deviation as the baseline reward value of the target optimization dimension.

[0048] Furthermore, the difference between each reward value and the baseline reward value is determined, and this difference is used to determine the advantage value of each predicted text in the target optimization dimension.

[0049] In this embodiment, the reward value of each predicted text is grouped according to a preset optimization dimension, and a baseline reward value is determined for each group (i.e., each preset optimization dimension). Based on the baseline reward value and the reward value of each predicted text, the advantage value of each predicted text under each preset optimization dimension is determined to clarify the relative advantage of the predicted text in each preset optimization dimension, which is beneficial for locating the training error that occurs during the training process of the text generation model.

[0050] Step 206: Determine the dimension loss value of each preset optimization dimension based on the preset dimension weight and the advantage value of each preset optimization dimension.

[0051] Since the model training method provided in this embodiment can train the text generation model in multiple preset optimization dimensions, the bias in the training of each preset optimization dimension is different in practical applications. Therefore, in order to ensure the training accuracy of each preset optimization dimension, a dimension weight can be set for each preset optimization dimension to reflect the training bias of each preset optimization dimension.

[0052] The preset dimension weights refer to the pre-defined weights for each preset optimization dimension. For example, the preset dimension weights for each preset optimization dimension could be: Action > Coordinate / Value > Inference > Format. These preset dimension weights are not directly used to train the text generation model; instead, they are adaptively adjusted based on the predicted text output by the text generation model in each training epoch. The target dimension weights are the weights adjusted based on the predicted text generated by the text generation model in the current training epoch. The target dimension weights can be used to train the text generation model.

[0053] Therefore, after obtaining multiple predicted texts output by the text generation model in the current training round, the preset dimension weights of each preset optimization dimension are adjusted according to the multiple predicted texts to determine the target dimension weights of each preset optimization dimension. Then, the dimension loss value of each preset optimization dimension is determined by combining the advantage value of each preset optimization dimension.

[0054] In one specific embodiment provided in this disclosure, determining the dimension loss value of each preset optimization dimension based on the preset dimension weight and the advantage value of each preset optimization dimension includes: determining the target dimension weight of each preset optimization dimension based on the preset dimension weight and the plurality of predicted texts; and determining the dimension loss value of each preset optimization dimension based on the advantage value and the target dimension weight of each preset optimization dimension.

[0055] Among them, the dimensional loss value refers to the loss value of the predicted text in each preset optimization dimension.

[0056] Specifically, based on the multiple predicted texts obtained by the text generation model, the preset dimension weights of each preset optimization dimension are adjusted to obtain the target dimension weights of each preset optimization dimension (the process of determining the target dimension weights will be explained in detail below). After determining the advantage value and target dimension weights of the predicted text in the preset optimization dimension, the dimension loss value of the predicted text in the preset optimization dimension can be determined based on the advantage value and target dimension weights.

[0057] In this embodiment, the target dimension weight of each preset optimization dimension is obtained by optimizing the preset dimension weights. The dimension loss value of each preset optimization dimension is determined by combining the target dimension weight and the advantage value of each preset optimization dimension. In the subsequent process, the text generation model is trained based on the dimension loss value, thereby refining the dimension loss value to each preset optimization dimension, improving the accuracy of updating model parameters, and improving the training accuracy of the text generation model.

[0058] The following explains the specific implementation process for determining the target dimension weights.

[0059] In one specific embodiment provided in this disclosure, determining the target dimension weight of each preset optimization dimension based on the preset dimension weight of each preset optimization dimension and the plurality of predicted texts includes: determining the gating coefficient of the target predicted text in each preset optimization dimension, wherein the target predicted text is any one of the plurality of predicted texts; determining the intermediate dimension weight of the target predicted text in each preset optimization dimension based on the preset dimension weight of the target predicted text in each preset optimization dimension and the gating coefficient of the target predicted text in each preset optimization dimension; and normalizing each intermediate dimension weight to obtain the target dimension weight of the target predicted text in each preset optimization dimension.

[0060] In practical applications, multiple preset optimization dimensions are determined based on the preset optimization dimensions contained in each sample label. However, during the training of the text generation model, the predicted text output by the text generation model may not necessarily contain the preset optimization dimensions in the sample labels. Therefore, it is necessary to determine the target dimension weights based on the preset optimization dimensions actually contained in the predicted text.

[0061] The gating coefficient is used to characterize whether the predicted text contains a preset optimized dimension, and each preset optimized dimension corresponds to a gating coefficient. The intermediate dimension weight refers to the dimension weight of the preset optimized dimension directly determined based on the preset dimension weight and the gating coefficient; the target dimension weight refers to the dimension weight after normalizing the intermediate dimension weight.

[0062] Specifically, for any one of the multiple predicted texts (i.e. the target predicted text), the gating coefficient of the target predicted text in each preset optimization dimension is determined. Based on the preset dimension weights and gating coefficients of the target predicted text in each preset optimization dimension, the intermediate dimension weights of the target predicted text in each preset optimization dimension are determined. After normalizing each intermediate dimension weight, the target dimension weights of the target predicted text in each preset optimization dimension are obtained, such that the sum of the target dimension weights of each preset optimization dimension contained in each predicted text is 1 or any constant.

[0063] Furthermore, the process of normalizing the intermediate dimension weights to determine the target dimension weights can be found in the following formula 1:

[0064] Formula 1

[0065] in, For predicting text In the preset optimization dimensions The target dimension weights For predicting text In the preset optimization dimensions The weight of the intermediate dimension, To preset the number of optimization dimensions, To calculate the factor, avoid dividing by 0 when determining the target dimension weight.

[0066] In this embodiment of the disclosure, by setting a preset dimension weight for each preset optimization dimension and adjusting the preset dimension weights adaptively according to the content of the predicted text based on the actual preset optimization dimensions contained in the predicted text, the target dimension weights are obtained, thereby improving the accuracy of the target dimension weights.

[0067] Furthermore, in a specific embodiment provided in this disclosure, determining the gating coefficient of the target predicted text in each of the preset optimization dimensions includes: identifying a first optimization dimension contained in the target predicted text; determining a second optimization dimension not contained in the target predicted text based on the plurality of preset optimization dimensions and the first optimization dimension; determining the gating coefficient of the first optimization dimension as a valid value, and determining the gating coefficient of the second optimization dimension as an invalid value.

[0068] Here, the first optimization dimension refers to the preset optimization dimension included in the predicted text, and the second optimization dimension refers to the preset optimization dimension not included in the predicted text. Valid values ​​indicate that the predicted text contains the preset optimization dimension, such as 1, while invalid values ​​indicate that the predicted text does not contain the preset optimization dimension, such as 0.

[0069] Specifically, the target predicted text is identified by pre-defined optimization dimensions to determine the first optimization dimension contained in the target predicted text. Based on multiple pre-defined optimization dimensions contained in the sample labels and the identified first optimization dimension, a second optimization dimension not contained in the target predicted text is determined. The gating coefficient of the target predicted text for the first optimization dimension is set to a valid value to indicate that the target predicted text contains the first optimization dimension, while the gating coefficient of the target predicted text for the second optimization dimension is set to an invalid value to indicate that the target predicted text does not contain the second optimization dimension.

[0070] For example, if the predicted text b does not contain "pos" (corresponding to the preset optimization dimension coordinate), then the gate coefficient of the predicted text b in the preset optimization dimension coordinate is set to an invalid value, that is, gate_coordinate(b) = 0; if the predicted text b does not contain "value" (corresponding to the preset optimization dimension value), then the gate coefficient of the predicted text b in the preset optimization dimension value is set to an invalid value, that is, gate_value(b) = 0.

[0071] In this embodiment of the disclosure, by determining the preset optimization dimensions included in the predicted text and the preset optimization dimensions not included, corresponding gating coefficients are set for the predicted text in different preset optimization dimensions. This allows the target dimension weights of the predicted text in different preset optimization dimensions, determined based on the gating coefficients, to be adaptively adjusted based on the predicted text, thereby increasing the controllability of the target dimension weights.

[0072] Further, in a specific embodiment provided in this disclosure, determining the intermediate dimension weight of the target predicted text in each preset optimization dimension based on the preset dimension weight of the target predicted text in each preset optimization dimension and the gating coefficient of the target predicted text in each preset optimization dimension includes: determining the product between the preset dimension weight of the target predicted text in the target optimization dimension and the gating coefficient of the target optimization dimension, wherein the target optimization dimension is any one of the plurality of preset optimization dimensions; and determining the product as the intermediate dimension weight of the target predicted text in the target optimization dimension.

[0073] In practical applications, the product of the preset dimension weights and the gating coefficients can be determined as the intermediate dimension weights of the predicted text in the target optimization dimension, as shown in Formula 2 below:

[0074] Formula 2

[0075] in, Preset optimization dimensions Preset dimension weights, For predicting text In the preset optimization dimensions The gating coefficient.

[0076] In this embodiment of the disclosure, the intermediate dimension weight of the predicted text in each preset optimization dimension is determined based on the gating coefficient of the predicted text in the preset optimization dimension and the preset dimension weight of the preset optimization dimension. The intermediate dimension weight of the preset optimization dimension that is not included in the predicted text can be filtered out to avoid affecting the subsequent training process.

[0077] In practical applications, predicted text often contains a large number of characters. If the predicted text is used directly to determine the dimensional loss value for each preset optimization dimension, the accuracy of the determined dimensional loss value will be low. Therefore, to improve the accuracy of the dimensional loss value, word segmentation is usually performed on the characters in the predicted text. Current word segmentation methods mostly use character length estimation followed by approximation to tokens. However, this can cause a mismatch between word segmentation boundaries and character boundaries, leading to the handling of irrelevant tokens, thus reducing the accuracy of the determined dimensional loss value and affecting the training of the text generation model.

[0078] To avoid the aforementioned problems during the training of the text generation model, this embodiment of the present disclosure, in determining the dimensional loss value based on the advantage value and target dimension weight of the predicted text in each preset optimization dimension, further incorporates the word segmentation of the predicted text to determine the dimensional loss value. Therefore, in a specific embodiment provided in this disclosure, determining the dimensional loss value of each preset optimization dimension based on the advantage value and target dimension weight of each preset optimization dimension includes: determining the dimensional loss value of each predicted text in each preset optimization dimension based on the word segmentation tag value, the advantage value, and the target dimension weight of each predicted text in each preset optimization dimension.

[0079] The token value is used to characterize whether the tokenized elements and character segments effectively overlap. Tokenized elements are obtained by segmenting the predicted text; character segments refer to continuous segments composed of tokenized elements. Specifically, after determining the advantage value and target dimension weight of the predicted text in each preset optimization dimension, the dimensional loss value of the predicted text in each preset optimization dimension is determined by combining the token value of the predicted text in each preset optimization dimension. See Formula 3 below for details:

[0080] Formula 3

[0081] For predicting text In the preset optimization dimensions dimensional loss value, For predicting text In the preset optimization dimensions Selected effective word segmentation elements The length of the word segmentation element, The total number of valid word segmentation elements. For predicting text The log probability of each word segment element in the text. For predicting text In the preset optimization dimensions The advantage value.

[0082] Specifically, when outputting predicted text, the text generation model also outputs the logits of each segmented element in the predicted text. These logits are the raw output values ​​of the linear layer of the text generation model. The softmax function is used to process the logits of each segmented element to obtain the log probability of each segmented element. Valid segmented elements can be determined based on segmentation tag values ​​and text tag values. The text tag value characterizes whether a segmented element in the predicted text is the output text obtained by the text generation model. The output text refers to the text content corresponding to each preset optimization dimension in the predicted text. For example, the predicted text includes "action": CLICK, where "action" is the preset optimization dimension (i.e., action), and CLICK is the text content of the action in the preset optimization dimension (i.e., "click"), which is the output text. In practical applications, valid segmented elements can be obtained by performing a logical AND operation on the segmentation tag values ​​and text tag values. After obtaining the various formula elements contained in Formula 3 above, the dimensional loss value of the predicted text in each preset optimization dimension can be calculated based on Formula 3.

[0083] In this embodiment, based on the advantage value and target dimension weight of the predicted text in each preset optimization dimension, the word segmentation tag value and text tag value of the predicted text are further combined to determine the dimension loss value of the predicted text in each preset optimization dimension, which greatly improves the accuracy of determining the dimension loss value and improves the accuracy of subsequent training of the text generation model.

[0084] The following explains the process of determining the word segmentation tag value and the text tag value. In a specific embodiment provided in this disclosure, the method further includes: segmenting the plurality of predicted texts into words respectively to obtain a word segmentation sequence for each predicted text and a word segmentation offset for each word segmentation element in the word segmentation sequence; identifying the output text corresponding to each preset optimization dimension in each word segmentation sequence and tagging each output text to obtain a text tag value for each output text; and determining the word segmentation tag value for each predicted text in each preset optimization dimension based on the word segmentation offset and text tag value of each predicted text.

[0085] The word segmentation sequence refers to the sequence formed by the word segmentation elements in the predicted text after word segmentation. The word segmentation offset refers to the character range corresponding to the word segmentation element.

[0086] Specifically, multiple predicted texts are segmented into words to obtain a segmentation sequence for each predicted text and a segmentation offset for each segmented element in the segmentation sequence. The output text corresponding to each preset optimization dimension is identified in the segmentation sequence of each predicted text, and the identified output text is marked to obtain a text tag value for each output text. For example, the "CLICK" in "action":CLICK is identified and marked, obtaining a text tag value for "CLICK", for example, set to 1. Alternatively, if no output text in the preset optimization dimension is identified in the segmentation sequence (the text content is empty, or the text content is user input, constructed prompts, etc.), the output text is also marked, obtaining a text tag value for the output text, for example, set to 0. Further, based on the segmentation offset and text tag value of the predicted text, the segmentation tag value of the predicted text in the preset optimization dimension is determined.

[0087] In this embodiment, the output text is identified for each preset optimization dimension in the word segmentation sequence, and the corresponding text tag value is obtained by tagging the output text. This ensures that during the training of the text generation model, only the output text is affected, without affecting other content in the word segmentation sequence. Combining the word segmentation offset and the text tag value to determine the word segmentation tag value for each preset optimization dimension is beneficial for the subsequent screening of effective word segmentation elements.

[0088] In one specific embodiment provided in this disclosure, the plurality of predicted texts are segmented into words to obtain a segmentation sequence for each predicted text, including: inputting the plurality of predicted texts into a word segmenter to obtain a segmentation sequence output by the word segmenter; or segmenting the plurality of predicted texts based on a preset word segmentation window to obtain a segmentation sequence for the plurality of predicted texts.

[0089] The tokenizer is used to segment the predicted text, such as the fast tokenizer. The preset tokenization window is a custom window created in advance for segmenting the predicted text. The size of the preset tokenization window can be customized according to training requirements.

[0090] Specifically, multiple predicted texts can be input into a word segmenter, which performs word segmentation on the predicted texts, resulting in a word segmentation sequence output by the word segmenter. In addition, the word segmenter can generate the word segmentation offset for each word element in the word segmentation sequence. Alternatively, a preset word segmentation window can be created in advance and used to segment the predicted text, obtaining a word segmentation sequence. This embodiment preferably uses a word segmenter to process the predicted text. In practical applications, if the training environment for training the text generation model is limited and a word segmenter cannot be obtained and used, a preset word segmentation window can be used for word segmentation.

[0091] In this embodiment of the disclosure, before determining the dimensional loss value of the predicted text in each preset optimization dimension, the predicted text can be segmented into words. In subsequent processes, training can be performed based on the segmented sequence. Compared with directly determining the dimensional loss value based on the entire predicted text, using the segmented sequence to determine the dimensional loss value is more accurate, and the accuracy of training the text generation model is also higher.

[0092] As mentioned above, the segmentation tag value is used to characterize whether the segmented element and the character fragment effectively overlap. Before determining the segmentation tag value, it is necessary to judge whether the segmented element and the character fragment overlap.

[0093] Based on this, in a specific embodiment provided in this disclosure, determining the word segmentation tag value of each predicted text in each preset optimization dimension according to the word segmentation offset and text tag value of each predicted text includes: obtaining a character fragment sequence of the target predicted text in the target optimization dimension, wherein the target predicted text is any one of the plurality of predicted texts, and the target optimization dimension is any one of the plurality of preset optimization dimensions; determining whether the character fragment sequence overlaps with the target word segmentation offset based on the character fragment sequence and the target word segmentation offset of the target word segmentation element; and determining the target word segmentation tag value of the target predicted text in the target optimization dimension as a valid value if the character fragment sequence overlaps with the target word segmentation offset and the text tag value of the target predicted text is valid.

[0094] In practical applications, the reward model corresponding to each preset optimization dimension outputs the reward value of the predicted text in that preset optimization dimension, as well as the character fragment sequence of the predicted text. Thus, after the predicted text is segmented, it is possible to determine whether the segmented elements in the segmented sequence overlap with their corresponding character fragments based on the segmented sequence and the character fragment sequence of the predicted text.

[0095] Here, the target segmentation element is any segmentation element in the segmentation sequence of the target predicted text, and the target segmentation offset is the segmentation offset of the target segmentation element. Specifically, for any predicted text among multiple predicted texts (i.e., the target predicted text), the character fragment sequence of the target predicted text in the target optimization dimension is obtained, and based on the character fragment sequence and the target segmentation offset of the target segmentation element, it is determined whether the character fragment sequence overlaps with the target segmentation offset of the target segmentation element. If the character fragment sequence overlaps with the target segmentation offset, and the text token value of the target predicted text is valid, then the segmentation token value of the target predicted text in the target optimization dimension can be determined as a valid value, i.e., token_masks is set. i,b,t =1.

[0096] In this embodiment, by determining whether the word segmentation offset of the word segmentation element overlaps with the character fragment sequence, the character fragment sequence is mapped to the word segmentation element respectively, thereby improving the matching between the word segmentation boundary and the character boundary. Furthermore, the word segmentation tag value is only set to a valid value if the text tag value of the predicted text is valid, so that the subsequent model training process only applies to the output text of the text generation model, thereby improving the accuracy of training the text generation model.

[0097] Furthermore, the following describes the specific implementation method for determining whether the character segment sequence overlaps with the word segmentation offset.

[0098] In one specific embodiment provided in this disclosure, determining whether the character fragment sequence and the target word segmentation offset overlap based on the character fragment sequence and the target word segmentation offset of the target word segmentation element includes: obtaining the character start value and character end value in the character fragment sequence, and the offset start value and offset end value in the target word segmentation offset; determining the maximum value among the character start value and the offset start value, and determining the minimum value among the character end value and the offset end value; and determining that the character fragment sequence overlaps with the target word segmentation offset if the maximum value is less than the minimum value.

[0099] The character start value and character end value are used to form the character range of the character segment sequence, i.e., [span_start, span_end), where span_start is the character start value and span_end is the character end value; the offset start value and offset end value are used to form the character range of the token offset, i.e., [token_start, token_end), where token_start is the offset start value and token_end is the offset end value.

[0100] Specifically, the start and end values ​​of characters in the character segment sequence, and the start and end values ​​of the target word offset are obtained. If max(span_start, token_start) < min(span_end, token_end), it is determined that the target word element overlaps with the character segment sequence. That is, the maximum value is determined among the start and end values, and the minimum value is determined among the end and end values. If the maximum value among the start and end values ​​is less than the minimum value among the end and end values, it is determined that the target word offset overlaps with the character segment sequence.

[0101] In this embodiment of the disclosure, the word segmentation offset of the word segmentation element is compared with the character fragment sequence to determine whether the word segmentation offset and the character fragment sequence actually overlap, thereby improving the accuracy of determining the overlap between the word segmentation offset and the character fragment sequence.

[0102] The embodiments disclosed herein achieve the following: by combining the word segmentation tag value, advantage value, and target dimension weight of the predicted text in each preset optimization dimension, the dimensional loss value of the predicted text in each preset optimization dimension is determined, thereby improving the accuracy of determining each preset optimization dimension.

[0103] Step 208: Train the text generation model based on the dimensional loss values ​​of the multiple preset optimization dimensions to obtain the text generation model for the next training round.

[0104] After determining the dimensional loss values ​​of the predicted text in multiple preset optimization dimensions, the text generation model can be trained based on the dimensional loss values ​​of the multiple preset optimization dimensions to obtain the text generation model for the next training round. The text generation model can then be trained again in the next training round based on the same model training method.

[0105] In one specific embodiment provided in this disclosure, training the text generation model based on the dimensional loss values ​​of the plurality of preset optimization dimensions includes: summing the dimensional loss values ​​of each of the preset optimization dimensions to obtain the total loss value of the plurality of preset optimization dimensions; and training the text generation model based on the total loss value.

[0106] Specifically, after obtaining the dimensional loss value of the predicted text in each preset optimization dimension, the dimensional loss values ​​of each preset optimization dimension are summed to obtain the total loss value of multiple preset optimization dimensions, and the model parameters of the text generation model are adjusted and trained according to the total loss value.

[0107] In practical applications, a well-trained text generation model is obtained when the training of the text generation model reaches a training stopping condition. The training stopping condition may include the number of training epochs or a preset loss threshold. For example, training of the text generation model can be stopped when the preset number of training epochs is reached; or, training can be stopped when the total loss value of the text generation model reaches a preset loss threshold; or, training can be stopped when either the number of training epochs or the preset loss threshold is reached. The specific training stopping condition for the text generation model can be set according to the actual application situation, and this embodiment does not limit it.

[0108] This embodiment of the disclosure achieves the following: In each training epoch of training a text generation model, based on the predicted text set obtained by the text generation model from processing the training data and the sample label set corresponding to the training data, the reward values ​​corresponding to multiple predicted texts in the predicted text set for multiple preset optimization dimensions are determined. This allows for the further determination of advantage values ​​based on the reward values ​​of different preset optimization dimensions, thus achieving the determination of advantage values ​​for multiple preset optimization dimensions. Furthermore, in subsequent model training, the loss values ​​used to update the model parameters of the text generation model are determined based on the advantage values ​​of multiple preset optimization dimensions, thereby improving the accuracy of training the text generation model. In addition, in setting a preset... Based on the established dimensional weights, and considering the different predicted texts output by the text generation model, the target dimensional weights for each preset optimization dimension are further determined, making the target dimensional weights of each preset optimization dimension controllable. The advantage value and the target dimensional weight of each preset optimization dimension are combined to determine the dimensional loss value of each preset optimization dimension, thus refining the loss value to each preset optimization dimension. In this way, during the training of the text generation model based on the dimensional loss values ​​of multiple preset optimization dimensions, the accuracy of updating model parameters is improved, further enhancing the training precision of the text generation model and improving its performance in practical applications.

[0109] The following is in conjunction with the appendix Figure 3 The model training method provided in the embodiments of this disclosure will be further explained and illustrated. Figure 3 A flowchart of a model training method provided in this disclosure embodiment is shown below. Figure 3 As shown, the example illustrates the interaction scenario between a user and a GUI (Graphical User Interface) intelligent agent.

[0110] Step 302: Input the predicted text and sample labels obtained by the text generation model of the current training round from the training data into the reward model corresponding to each preset optimization dimension.

[0111] Specifically, in the GUI click task scenario, the predicted text includes "Next Action":"CLICK", "pos":"x,y"; the sample labels include... <action> CLICK< / action> and <bboxes> [x0,y0,x1,y1]< / bboxes> Input the predicted text and sample labels into the action reward model to obtain the reward value for the action optimization dimension and the character span for the "NextAction" field; input the predicted text and sample labels into the coordinate reward model to obtain the reward value for the coordinate optimization dimension and the character span for the "pos" field; input the predicted text and sample labels into the format reward model and the inference link reward model respectively, and the format reward model and the inference link reward model will output the corresponding reward value and the span of the corresponding field according to the preset rules.

[0112] In a GUI text input task scenario, the predicted text includes "value":"..."; the sample labels include... <value> …< / value> Input the predicted text and sample labels into the value reward model to obtain the reward value for the value optimization dimension and the character span for the "value" field; input the predicted text and sample labels into the format reward model and the inference link reward model respectively, and the format reward model and the inference link reward model will output the corresponding reward value and the span of the corresponding field according to the preset rules.

[0113] Step 304: Determine the advantage value of each preset optimization dimension based on the reward value corresponding to each preset optimization dimension.

[0114] Specifically, in the GUI click task scenario, baseline reward values ​​are determined for the action optimization dimension, coordinate optimization dimension, format optimization dimension, and inference link optimization dimension, respectively, and the advantage value of each optimization dimension is determined based on the baseline reward values. The advantage value of each optimization dimension is determined using the same method in the GUI text input task scenario.

[0115] Step 306: Map character fragments to the word segmentation element level based on the word segmentation offset of the predicted text.

[0116] Specifically, the predicted text is input into the tokenizer to obtain the tokenized sequence output by the tokenizer and the token offset of each token element in the sequence. The (span_start, span_end) and token offsets [token_start, token_end] of the span are obtained. If max(span_start, token_start) < min(span_end, token_end), then it is determined that the token element overlaps with the span, and the token marker value token_masks[i,b,t] = 1. It should be noted that if the text marker value completion_mask[b, t] = 0, then overlap is ignored regardless of whether it exists.

[0117] Step 308: Determine the target dimension weight of the predicted text in each preset optimization dimension based on the preset dimension weight and gating coefficient of the predicted text in each preset optimization dimension.

[0118] Specifically, the gating coefficient of each optimization dimension is determined based on the optimization dimensions contained in the predicted text. The product of the gating coefficient and the preset dimension weight is determined as the intermediate dimension weight. The intermediate dimension weight is then normalized to obtain the target dimension weight.

[0119] Step 310: Determine the dimensional loss value of the predicted text in each preset optimization dimension based on the word segmentation tag value, advantage value, and target dimension weight of the predicted text in each preset optimization dimension.

[0120] Specifically, the softmax function is used to process the logits output by the text generation model to obtain the log probability of each token. Based on the log probability, token values, advantage values, and target dimension weights, the dimensional loss value for each optimization dimension is determined.

[0121] Step 312: Sum the dimensional loss values ​​of each preset optimization dimension to obtain the total loss value of multiple preset optimization dimensions, and train the text generation model based on the total loss value.

[0122] This embodiment of the disclosure achieves the following: During the training of a text generation model, based on the predicted text and sample labels, the reward value corresponding to the predicted text in multiple preset optimization dimensions is determined, thereby further determining the advantage value based on the reward value of different preset optimization dimensions, realizing the determination of the advantage value of multiple preset optimization dimensions; Based on the preset dimension weights, the target dimension weight of each preset optimization dimension is determined according to the different predicted texts output by the text generation model, making the target dimension weight of each preset optimization dimension controllable; The advantage value of each preset optimization dimension and the target dimension weight of each preset optimization dimension are combined to determine the dimension loss value of each preset optimization dimension, thereby refining the loss value to each preset optimization dimension; The dimension loss values ​​of multiple preset optimization dimensions are summed to obtain the total loss value, and the text generation model is trained based on the total loss value, improving the accuracy of model parameter updates and the training precision of the text generation model, and improving the performance of the text generation model in practical applications.

[0123] Figure 4 A flowchart illustrating a task processing method provided in an embodiment of this disclosure. See also... Figure 4 The method specifically includes the following steps:

[0124] Step 402: Upon receiving a task processing instruction, input the task processing instruction into the text generation model to obtain the task parsing text.

[0125] Step 404: Perform the task processing operation in the task parsing text, wherein the text generation model is trained based on the above model training method.

[0126] In practical applications, after obtaining a trained text generation model based on the above model training method, the text generation model can be applied. The execution entity of the task processing method and the execution entity of the above model training method can be the same entity or different entities. For example, after the text generation model is trained, it can be directly deployed on the terminal device or cloud-based device used to train the text generation model for subsequent applications, or it can be deployed on other terminal devices for subsequent applications. This disclosure does not limit the execution entity of the task processing method.

[0127] Task processing instructions refer to user-triggered commands that interact with terminal devices or intelligent agents, such as GUI agents. These instructions can be image processing commands, music playback commands, ticket ordering commands, etc. Specifically, after receiving a task processing instruction, a terminal device equipped with a text generation model inputs the instruction into the model for processing. If the instruction is a voice command, it can be converted into text before being input into the model. The text generation model processes the instruction and outputs a parsed task text. The terminal device can then execute the task processing operations recorded in the parsed text, thus completing the execution of the task processing instruction.

[0128] This embodiment of the disclosure implements a text generation model obtained through the above-described model training method, which processes task processing instructions to achieve higher accuracy in generating task parsing text, thereby further improving the accuracy of task processing instruction execution and enhancing the user experience.

[0129] It is understood that the various method embodiments mentioned above in this disclosure can be combined with each other to form combined embodiments without violating the principle and logic. Due to space limitations, this disclosure will not elaborate further. Those skilled in the art will understand that in the above methods of specific implementation, the specific execution order of each step should be determined by its function and possible internal logic.

[0130] In addition, this disclosure also provides a model training device, a task processing device, an electronic device, a computer-readable storage medium, and a computer program product, all of which can be used to implement the model training method or task processing method provided in this disclosure. The corresponding technical solutions and descriptions are described in the corresponding section of the method and will not be repeated here.

[0131] Figure 5 This is a block diagram of a model training device provided in an embodiment of the present disclosure.

[0132] See Figure 5 This disclosure provides a model training apparatus, which includes:

[0133] The reward determination module 502 is configured to determine the reward value corresponding to the predicted text set in multiple preset optimization dimensions based on the predicted text set obtained by the text generation model in the current training round and the sample label set corresponding to the training data. The predicted text set includes multiple predicted texts, and the sample label set includes multiple sample labels corresponding to the training data.

[0134] The advantage determination module 504 is configured to determine the advantage value of each preset optimization dimension based on the reward value corresponding to each preset optimization dimension, wherein the advantage value of each preset optimization dimension is used to characterize the relative advantage of each predicted text in each preset optimization dimension;

[0135] The loss determination module 506 is configured to determine the dimension loss value of each preset optimization dimension based on the preset dimension weight of each preset optimization dimension and the advantage value of each preset optimization dimension.

[0136] Training module 508 is configured to train the text generation model based on the dimensional loss values ​​of the multiple preset optimization dimensions to obtain the text generation model for the next training round.

[0137] Optionally, the reward determination module 502 is further configured to:

[0138] Input the multiple predicted texts and the multiple sample labels into the target reward model to obtain the target reward values ​​of the multiple predicted texts in the target optimization dimension, respectively, output by the target reward model.

[0139] Wherein, the target optimization dimension is any one of the plurality of preset optimization dimensions, and the target reward model is used to determine the reward value of the predicted text set in the target optimization dimension.

[0140] Optionally, the advantage determination module 504 is further configured to:

[0141] A baseline reward value is determined based on the multiple target reward values;

[0142] The target advantage value of the target optimization dimension is determined based on the multiple target reward values ​​and the baseline reward value.

[0143] Optionally, the loss determination module 506 is further configured to:

[0144] Based on the preset dimension weights of each preset optimization dimension and the plurality of predicted texts, determine the target dimension weights of each preset optimization dimension;

[0145] The dimension loss value of each preset optimization dimension is determined based on the advantage value of each preset optimization dimension and the target dimension weight of each preset optimization dimension.

[0146] Optionally, the loss determination module 506 is further configured to:

[0147] Determine the gating coefficient of the target predicted text for each of the preset optimization dimensions, wherein the target predicted text is any one of the plurality of predicted texts, and the gating coefficient is used to characterize whether the target predicted text contains the preset optimization dimension;

[0148] Based on the preset dimension weights of the target predicted text in each preset optimization dimension and the gating coefficients of the target predicted text in each preset optimization dimension, the intermediate dimension weights of the target predicted text in each preset optimization dimension are determined.

[0149] The weights of each intermediate dimension are normalized to obtain the target dimension weights of the target predicted text in each of the preset optimization dimensions.

[0150] Optionally, the loss determination module 506 is further configured to:

[0151] Identify the first optimization dimension contained in the target predicted text;

[0152] Based on the multiple preset optimization dimensions and the first optimization dimension, a second optimization dimension that is not included in the target predicted text is determined;

[0153] The gating coefficient of the first optimization dimension is determined as a valid value, and the gating coefficient of the second optimization dimension is determined as an invalid value.

[0154] Optionally, the loss determination module 506 is further configured to:

[0155] Determine the product between the preset dimension weight of the target predicted text in the target optimization dimension and the gate coefficient of the target optimization dimension, wherein the target optimization dimension is any one of the plurality of preset optimization dimensions;

[0156] The product is determined as the weight of the intermediate dimension of the target predicted text in the target optimization dimension.

[0157] Optionally, the device further includes a word segmentation module configured to:

[0158] The multiple predicted texts are segmented into words respectively to obtain the word segmentation sequence of each predicted text and the word segmentation offset of each word segmentation element in the word segmentation sequence, wherein the word segmentation offset is the character range corresponding to each word segmentation element;

[0159] In each of the word segmentation sequences, the output text corresponding to each preset optimization dimension is identified, and each of the output texts is marked to obtain a text mark value for each output text. The text mark value is used to characterize whether each of the word segmentation elements is text obtained by the text generation model.

[0160] Based on the word segmentation offset and text tag value of each predicted text, the word segmentation tag value of each predicted text in each preset optimization dimension is determined, wherein the word segmentation tag value is used to characterize whether the word segmentation elements in each predicted text effectively overlap with the character segments in the character segment sequence of each predicted text in each preset optimization dimension;

[0161] The loss determination module 506 is further configured as follows:

[0162] Based on the word segmentation tag value, the advantage value, and the target dimension weight of each predicted text in each preset optimization dimension, determine the dimension loss value of each predicted text in each preset optimization dimension.

[0163] Optionally, the word segmentation module is further configured as follows:

[0164] Input the multiple predicted texts into a word segmenter to obtain the word segmentation sequence output by the word segmenter; or

[0165] The multiple predicted texts are segmented based on a preset segmentation window to obtain a segmentation sequence of the multiple predicted texts.

[0166] Optionally, the word segmentation module is further configured as follows:

[0167] Obtain the character fragment sequence of the target predicted text in the target optimization dimension, wherein the target predicted text is any one of the plurality of predicted texts, and the target optimization dimension is any one of the plurality of preset optimization dimensions;

[0168] Based on the character fragment sequence and the target word segmentation offset of the target word segmentation element, determine whether the character fragment sequence overlaps with the target word segmentation offset;

[0169] If the character fragment sequence overlaps with the target word segmentation offset and the text tag value of the target predicted text is a valid value, the target word segmentation tag value of the target predicted text in the target optimization dimension is determined to be a valid value. The valid value of the text tag value of the target predicted text is used to characterize that each word segmentation element in the target predicted text is text obtained by the text generation model.

[0170] Optionally, the word segmentation module is further configured as follows:

[0171] Obtain the character start value and character end value in the character segment sequence, and the offset start value and offset end value in the target word segmentation offset;

[0172] Determine the maximum value between the character start value and the offset start value, and determine the minimum value between the character end value and the offset end value;

[0173] If the maximum value is less than the minimum value, it is determined that the character segment sequence overlaps with the target word offset.

[0174] Optionally, the training module 508 is further configured as follows:

[0175] The dimensional loss values ​​of each of the preset optimization dimensions are summed to obtain the total loss value of the multiple preset optimization dimensions;

[0176] The text generation model is trained based on the total loss value.

[0177] This embodiment of the disclosure achieves the following: In each training epoch of training a text generation model, based on the predicted text set obtained by the text generation model from processing the training data and the sample label set corresponding to the training data, the reward values ​​corresponding to multiple predicted texts in the predicted text set for multiple preset optimization dimensions are determined. This allows for the further determination of advantage values ​​based on the reward values ​​of different preset optimization dimensions, thus achieving the determination of advantage values ​​for multiple preset optimization dimensions. Furthermore, in subsequent model training, the loss values ​​used to update the model parameters of the text generation model are determined based on the advantage values ​​of multiple preset optimization dimensions, thereby improving the accuracy of training the text generation model. In addition, in setting a preset... Based on the established dimensional weights, and considering the different predicted texts output by the text generation model, the target dimensional weights for each preset optimization dimension are further determined, making the target dimensional weights of each preset optimization dimension controllable. The advantage value and the target dimensional weight of each preset optimization dimension are combined to determine the dimensional loss value of each preset optimization dimension, thus refining the loss value to each preset optimization dimension. In this way, during the training of the text generation model based on the dimensional loss values ​​of multiple preset optimization dimensions, the accuracy of updating model parameters is improved, further enhancing the training precision of the text generation model and improving its performance in practical applications.

[0178] Figure 6 This is a block diagram of a task processing apparatus provided in an embodiment of the present disclosure.

[0179] See Figure 6 This disclosure provides a task processing apparatus, which includes:

[0180] The input module 602 is configured to input the task processing instruction into the text generation model upon receiving the task processing instruction to obtain the task parsing text.

[0181] The execution module 604 is configured to perform task processing operations in the task parsing text, wherein the text generation model is trained based on the aforementioned model training device.

[0182] This embodiment of the disclosure implements a text generation model obtained through the above-described model training method, which processes task processing instructions to achieve higher accuracy in generating task parsing text, thereby further improving the accuracy of task processing instruction execution and enhancing the user experience.

[0183] Figure 7 This is a block diagram of an electronic device provided in an embodiment of the present disclosure.

[0184] See Figure 7 This disclosure provides an electronic device 700, which includes: at least one processor 701; at least one memory 702; and one or more I / O interfaces 703 connected between the processor 701 and the memory 702; wherein the memory 702 stores one or more computer programs that can be executed by the at least one processor 701, and the one or more computer programs are executed by the at least one processor 701 to enable the at least one processor 701 to perform the above-described model training method or task processing method.

[0185] This disclosure also provides a computer-readable storage medium storing a computer program thereon, wherein the computer program, when executed by a processor, implements the model training method or task processing method described above. The computer-readable storage medium may be volatile or non-volatile.

[0186] This disclosure also provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code. When the computer-readable code is run in the processor of an electronic device, the processor in the electronic device executes the model training method or task processing method described above.

[0187] Those skilled in the art will understand that all or some of the steps, systems, and apparatuses disclosed above, and their functional modules / units, can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware implementations, the division between functional modules / units mentioned above does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit (ASIC). Such software can be distributed on a computer-readable storage medium, which may include computer storage media (or non-transitory media) and communication media (or transient media).

[0188] As is known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information, such as computer-readable program instructions, data structures, program modules, or other data. Computer storage media includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), static random access memory (SRAM), flash memory or other memory technologies, portable compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, it is known to those skilled in the art that communication media typically contain computer-readable program instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0189] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0190] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0191] The computer program product described herein can be implemented specifically through hardware, software, or a combination thereof. In one alternative embodiment, the computer program product is specifically embodied in a computer storage medium; in another alternative embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0192] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0193] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0194] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0195] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0196] Example embodiments have been disclosed herein, and while specific terminology has been used, it is for illustrative purposes only and should be construed as such, and is not intended to be limiting. In some instances, it will be apparent to those skilled in the art that features, characteristics, and / or elements described in connection with particular embodiments may be used alone, or in combination with features, characteristics, and / or elements described in connection with other embodiments, unless otherwise expressly indicated. Therefore, those skilled in the art will understand that various changes in form and detail may be made without departing from the scope of this disclosure as set forth by the appended claims.

Claims

1. A model training method, characterized in that, include: Based on the predicted text set obtained by the text generation model in the current training round processing the training data and the sample label set corresponding to the training data, the reward value corresponding to the predicted text set in multiple preset optimization dimensions is determined. The predicted text set includes multiple predicted texts, the sample label set includes multiple sample labels corresponding to the training data, and the preset optimization dimensions are dimensions used to optimize the text generation model. The reward value corresponding to each preset optimization dimension is determined based on the reward model corresponding to each preset optimization dimension. Based on the reward value corresponding to each preset optimization dimension, the advantage value of each preset optimization dimension is determined, wherein the advantage value of each preset optimization dimension is used to characterize the relative advantage of each predicted text in each preset optimization dimension; Based on the preset dimension weights and the advantage values ​​of each preset optimization dimension, determine the dimension loss value of each preset optimization dimension; The text generation model is trained based on the dimensional loss values ​​of the multiple preset optimization dimensions to obtain the text generation model for the next training round.

2. The method as described in claim 1, characterized in that, Based on the predicted text set obtained by the text generation model in the current training round through processing the training data and the sample label set corresponding to the training data, the reward values ​​corresponding to the predicted text set in multiple preset optimization dimensions are determined, including: Input the multiple predicted texts and the multiple sample labels into the target reward model to obtain the target reward values ​​of the multiple predicted texts in the target optimization dimension, respectively, output by the target reward model. Wherein, the target optimization dimension is any one of the plurality of preset optimization dimensions, and the target reward model is used to determine the reward value of the predicted text set in the target optimization dimension.

3. The method as described in claim 2, characterized in that, Based on the reward value corresponding to each preset optimization dimension, determine the advantage value of each preset optimization dimension, including: A baseline reward value is determined based on the multiple target reward values; The target advantage value of the target optimization dimension is determined based on the multiple target reward values ​​and the baseline reward value.

4. The method as described in claim 1, characterized in that, Based on the preset dimension weights and advantage values ​​of each preset optimization dimension, the dimension loss value of each preset optimization dimension is determined, including: Based on the preset dimension weights of each preset optimization dimension and the plurality of predicted texts, determine the target dimension weights of each preset optimization dimension; The dimension loss value of each preset optimization dimension is determined based on the advantage value of each preset optimization dimension and the target dimension weight of each preset optimization dimension.

5. The method as described in claim 4, characterized in that, Based on the preset dimension weights of each preset optimization dimension and the plurality of predicted texts, the target dimension weights of each preset optimization dimension are determined, including: Determine the gating coefficient of the target predicted text for each of the preset optimization dimensions, wherein the target predicted text is any one of the plurality of predicted texts, and the gating coefficient is used to characterize whether the target predicted text contains the preset optimization dimension; Based on the preset dimension weights of the target predicted text in each preset optimization dimension and the gating coefficients of the target predicted text in each preset optimization dimension, the intermediate dimension weights of the target predicted text in each preset optimization dimension are determined. The weights of each intermediate dimension are normalized to obtain the target dimension weights of the target predicted text in each of the preset optimization dimensions.

6. The method as described in claim 5, characterized in that, Determine the gating coefficients for the target predicted text in each of the preset optimization dimensions, including: Identify the first optimization dimension contained in the target predicted text; Based on the multiple preset optimization dimensions and the first optimization dimension, a second optimization dimension that is not included in the target predicted text is determined; The gating coefficient of the first optimization dimension is determined as a valid value, and the gating coefficient of the second optimization dimension is determined as an invalid value.

7. The method as described in claim 5, characterized in that, Based on the preset dimension weights of the target predicted text in each preset optimization dimension and the gating coefficients of the target predicted text in each preset optimization dimension, the intermediate dimension weights of the target predicted text in each preset optimization dimension are determined, including: Determine the product between the preset dimension weight of the target predicted text in the target optimization dimension and the gate coefficient of the target optimization dimension, wherein the target optimization dimension is any one of the plurality of preset optimization dimensions; The product is determined as the weight of the intermediate dimension of the target predicted text in the target optimization dimension.

8. The method as described in claim 4, characterized in that, The method further includes: The multiple predicted texts are segmented into words respectively to obtain the word segmentation sequence of each predicted text and the word segmentation offset of each word segmentation element in the word segmentation sequence, wherein the word segmentation offset is the character range corresponding to each word segmentation element; In each of the word segmentation sequences, the output text corresponding to each preset optimization dimension is identified, and each of the output texts is marked to obtain a text mark value for each output text. The text mark value is used to characterize whether each of the word segmentation elements is text obtained by the text generation model. Based on the word segmentation offset and text tag value of each predicted text, the word segmentation tag value of each predicted text in each preset optimization dimension is determined, wherein the word segmentation tag value is used to characterize whether the word segmentation elements in each predicted text effectively overlap with the character segments in the character segment sequence of each predicted text in each preset optimization dimension; Based on the advantage value of each preset optimization dimension and the target dimension weight of each preset optimization dimension, determine the dimension loss value of each preset optimization dimension, including: Based on the word segmentation tag value, the advantage value, and the target dimension weight of each predicted text in each preset optimization dimension, determine the dimension loss value of each predicted text in each preset optimization dimension.

9. The method as described in claim 8, characterized in that, The multiple predicted texts are segmented into words to obtain a segmentation sequence for each predicted text, including: Input the multiple predicted texts into a word segmenter to obtain the word segmentation sequence output by the word segmenter; or The multiple predicted texts are segmented based on a preset segmentation window to obtain a segmentation sequence of the multiple predicted texts.

10. The method as described in claim 8, characterized in that, Based on the word segmentation offset and text tag value of each predicted text, determine the word segmentation tag value of each predicted text in each preset optimization dimension, including: Obtain the character fragment sequence of the target predicted text in the target optimization dimension, wherein the target predicted text is any one of the plurality of predicted texts, and the target optimization dimension is any one of the plurality of preset optimization dimensions; Based on the character fragment sequence and the target word segmentation offset of the target word segmentation element, determine whether the character fragment sequence overlaps with the target word segmentation offset; If the character fragment sequence overlaps with the target word segmentation offset and the text tag value of the target predicted text is a valid value, the target word segmentation tag value of the target predicted text in the target optimization dimension is determined to be a valid value. The valid value of the text tag value of the target predicted text is used to characterize that each word segmentation element in the target predicted text is text obtained by the text generation model.

11. The method as described in claim 10, characterized in that, Based on the character fragment sequence and the target word segmentation offset of the target word segmentation element, determine whether the character fragment sequence overlaps with the target word segmentation offset, including: Obtain the character start value and character end value in the character segment sequence, and the offset start value and offset end value in the target word segmentation offset; Determine the maximum value between the character start value and the offset start value, and determine the minimum value between the character end value and the offset end value; If the maximum value is less than the minimum value, it is determined that the character segment sequence overlaps with the target word offset.

12. The method as described in claim 1, characterized in that, Training the text generation model based on the dimensional loss values ​​of the multiple preset optimization dimensions includes: The dimensional loss values ​​of each of the preset optimization dimensions are summed to obtain the total loss value of the multiple preset optimization dimensions; The text generation model is trained based on the total loss value.

13. A task processing method, characterized in that, include: Upon receiving a task processing instruction, the instruction is input into a text generation model to obtain the task parsing text. The task processing operation in the task parsing text is performed, wherein the text generation model is trained based on the model training method described in any one of claims 1-12.

14. A model training device, characterized in that, include: The reward determination module is configured to determine the reward value corresponding to the predicted text set in multiple preset optimization dimensions based on the predicted text set obtained by the text generation model in the current training round and the sample label set corresponding to the training data. The predicted text set includes multiple predicted texts, the sample label set includes multiple sample labels corresponding to the training data, and the preset optimization dimensions are dimensions used to optimize the text generation model. The reward value corresponding to each preset optimization dimension is determined based on the reward model corresponding to each preset optimization dimension. The advantage determination module is configured to determine the advantage value of each preset optimization dimension based on the reward value corresponding to each preset optimization dimension, wherein the advantage value of each preset optimization dimension is used to characterize the relative advantage of each predicted text in each preset optimization dimension; The loss determination module is configured to determine the dimension loss value of each preset optimization dimension based on the preset dimension weight of each preset optimization dimension and the advantage value of each preset optimization dimension. The training module is configured to train the text generation model based on the dimensional loss values ​​of the multiple preset optimization dimensions to obtain the text generation model for the next training round.

15. A task processing device, characterized in that, include: The input module is configured to input the task processing instruction into the text generation model upon receiving the task processing instruction, thereby obtaining the task parsing text; An execution module is configured to perform task processing operations in the task parsing text, wherein the text generation model is trained based on the model training device described in claim 14.

16. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores one or more computer programs that can be executed by the at least one processor, the one or more computer programs being executed by the at least one processor to enable the at least one processor to perform the method as described in any one of claims 1-13.

17. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-13.

18. A computer program product, characterized in that, Includes computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is executed in a processor of an electronic device, the processor in the electronic device performs the method as described in any one of claims 1-13.

Images