Multi-task model training and prediction method, device, equipment and medium

Abstract

The present disclosure provides a training and prediction method, device, equipment and medium for a multi-task model, which relates to the fields of deep learning and natural language processing. The specific implementation scheme is: obtaining sample text sets corresponding to multiple tasks, wherein each sample text set is used to perform adaptation training on the multi-task model for the corresponding task; obtaining the reference quantization range corresponding to each sample text set; determining the target quantization range according to the reference quantization range corresponding to each sample text set; and training the multi-task model according to the target quantization range. Thus, the reference quantization ranges of the sample text sets corresponding to multiple tasks are combined to determine the target quantization range adopted by the multi-task model, so that the multi-task model quantizes the text according to the target quantization range, which can avoid the problem that the multi-task model uses the quantization range corresponding to a single sample text set to quantize the text corresponding to different tasks, resulting in poor quantization effect, thereby improving the quantization effect of the multi-task model.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence, specifically to technical fields such as deep learning and natural language processing, and in particular to training and prediction methods, devices, equipment and media for multi-task models. Background Art

[0002] Neural network model quantization refers to the process of converting the model's weights, activation values, etc. from high precision to low precision, such as converting the floating point type float32 to the integer type int8, in order to reduce the model's computational intensity, parameter size, and memory consumption.

[0003] At present, most natural language processing models are based on the Transformer architecture, such as BERT (Bidirectional Encoder Representations from Transformers, based on bidirectional encoding representations for machine translation), ERNIE (Enhanced Language Representation with Informative Entities, based on knowledge enhanced semantic representation), etc. Natural language processing models based on the Transformer structure often have multi-task characteristics. For example, different tasks (such as text classification tasks, scoring tasks, sequence labeling tasks, sorting tasks, etc.) are trained based on part or all of the output results of the last network layer of the model.

[0004] Since the distribution of data sets, training sets or sample sets used to train different tasks is different, if a multi-task-based natural language processing model (referred to as a multi-task model in this disclosure) uses a set quantization range scale to quantize the text corresponding to different tasks, it may lead to a loss of quantization effect, thereby reducing the prediction accuracy of the multi-task model. Summary of the invention

[0005] The present invention provides a method, device, equipment and medium for training and predicting a multi-task model.

[0006] According to one aspect of the present disclosure, a method for training a multi-task model is provided, comprising:

[0007] Acquire sample text sets corresponding to multiple tasks, wherein each of the sample text sets is used to perform adaptation training on the multi-task model for the corresponding task;

[0008] Obtaining a reference quantization range corresponding to each of the sample text sets;

[0009] Determining a target quantization range according to a reference quantization range corresponding to each of the sample text sets;

[0010] The multi-task model is trained according to the target quantization range.

[0011] According to another aspect of the present disclosure, there is provided a prediction method, comprising:

[0012] Get the text to be recognized;

[0013] The multi-task model is used to quantize the text to be recognized according to the target quantization range to obtain quantization data corresponding to the text to be recognized, and the quantization data is predicted to obtain a prediction result corresponding to the text to be recognized.

[0014] According to another aspect of the present disclosure, a multi-task model training device is provided, comprising:

[0015] A first acquisition module is used to acquire sample text sets corresponding to multiple tasks, wherein each sample text set is used to perform adaptation training of a multi-task model for a corresponding task;

[0016] A second acquisition module is used to acquire a reference quantization range corresponding to each of the sample text sets;

[0017] A first determination module, configured to determine a target quantization range according to a reference quantization range corresponding to each of the sample text sets;

[0018] A training module is used to train the multi-task model according to the target quantization range.

[0019] According to another aspect of the present disclosure, there is provided a prediction device, comprising:

[0020] An acquisition module is used to acquire the text to be recognized;

[0021] The prediction module is used to quantize the text to be recognized according to the target quantization range using a multi-task model to obtain quantization data corresponding to the text to be recognized, and predict the quantization data to obtain a prediction result corresponding to the text to be recognized.

[0022] According to another aspect of the present disclosure, there is provided an electronic device, including:

[0023] at least one processor; and

[0024] a memory communicatively connected to the at least one processor; wherein,

[0025] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the training method of the multi-task model proposed in the above first aspect of the present disclosure, or execute the prediction method proposed in the above second aspect of the present disclosure.

[0026] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium of computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the training method of the multi-task model proposed in the above-mentioned first aspect of the present disclosure, or to execute the prediction method proposed in the above-mentioned other aspect of the present disclosure.

[0027] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, which, when executed by a processor, implements the training method of the multi-task model proposed in the above-mentioned first aspect of the present disclosure, or implements the prediction method proposed in the above-mentioned other aspect of the present disclosure.

[0028] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0030] Figure 1 A flowchart of a multi-task model training method provided in the first embodiment of the present disclosure;

[0031] Figure 2 A flowchart of a multi-task model training method provided in the second embodiment of the present disclosure;

[0032] Figure 3 A flowchart of a multi-task model training method provided in Embodiment 3 of the present disclosure;

[0033] Figure 4 A flowchart of a multi-task model training method provided in Embodiment 4 of the present disclosure;

[0034] Figure 5 A flowchart of a multi-task model training method provided in Embodiment 5 of the present disclosure;

[0035] Figure 6 A flowchart of a multi-task model training method provided in Embodiment 6 of the present disclosure;

[0036] Figure 7 This is a flowchart of the prediction method provided in Embodiment 7 of the present disclosure;

[0037] Figure 8 A flowchart of the prediction method provided in Embodiment 8 of the present disclosure;

[0038] Fig. 9 A schematic diagram of the structure of a training device for a multi-task model provided in Embodiment 9 of the present disclosure;

[0039] Fig.10 This is a schematic diagram of the structure of the prediction device provided in the tenth embodiment of the present disclosure;

[0040] Fig.11 A schematic block diagram of an example electronic device that can be used to implement embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0041] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0042] At present, in the quantization process of each network layer of the multi-task model, the target quantization range used by the multi-task model is determined according to the maximum and minimum values ​​of the sampled data. And after the quantization process is completed, F1 (F1 value or F1 score can be regarded as the weighted harmonic mean of the model precision and recall rate, with a maximum value of 1 and a minimum value of 0) or PNR (Positive Negative Rate, positive-negative ratio, equal to positive number / negative number) is used as an evaluation indicator to evaluate the effect of the quantized multi-task model.

[0043] However, most mainstream natural language processing models are based on the Transformer architecture, such as BERT, ERNIE, etc. Natural language processing models based on the Transformer structure often have multi-task characteristics. For example, different tasks (such as text classification tasks, scoring tasks, sequence labeling tasks, sorting tasks, etc.) are trained based on part or all of the output results of the last network layer of the model.

[0044] Since the distribution of data sets (also called training sets or sample sets) used for training different tasks is different, the target quantization range selected for quantization of each network layer generally depends on the distribution of the data sets. If the data sets corresponding to all tasks are mixed into a single data set, it will often lead to a loss in effect.

[0045] In addition, existing natural language processing models generally have only a single evaluation metric, whether before or after quantization. For example, for classification models, F1 is generally used, and for ranking models, PNR is generally used. For multi-task natural language processing models after quantization (i.e., multi-task models), using a single evaluation metric often cannot comprehensively evaluate the quantization effect of multi-task models.

[0046] In view of the above problems, the present disclosure proposes a multi-task model training and prediction method, device, equipment and medium.

[0047] The following describes the training and prediction method, device, equipment and medium of the multi-task model of the embodiments of the present disclosure with reference to the accompanying drawings.

[0048] Figure 1 A flowchart of a multi-task model training method provided in Embodiment 1 of the present disclosure.

[0049] The embodiment of the present disclosure takes the training method of the multi-task model being configured in a training device of the multi-task model as an example. The training device of the multi-task model can be applied to any electronic device so that the electronic device can perform the training function of the multi-task model.

[0050] Among them, the electronic device can be any device with computing capabilities, such as a personal computer, a mobile terminal, a server, etc. The mobile terminal can be, for example, a mobile phone, a tablet computer, a personal digital assistant, a wearable device, and other hardware devices with various operating systems, touch screens and / or display screens.

[0051] like Figure 1 As shown, the training method of the multi-task model may include the following steps:

[0052] Step 101, obtaining sample text sets corresponding to multiple tasks, wherein each sample text set is used to perform adaptation training for the corresponding task on the multi-task model.

[0053] In the embodiments of the present disclosure, the multiple tasks may include at least two tasks, such as a text classification task, a scoring task, a sequence labeling task, a sorting task, and the like.

[0054] In the embodiments of the present disclosure, each task may have a corresponding sample text set, wherein the sample text set is used to perform adaptation training for the corresponding task on the multi-task model.

[0055] In the embodiments of the present disclosure, there is no restriction on the method for obtaining the sample text sets corresponding to the multiple tasks. For example, the sample text set corresponding to each task can be obtained from an existing training set; or, the sample text set corresponding to each task can also be collected online, for example, multiple sample texts corresponding to each task can be collected online through web crawler technology, and the sample text set is composed of the multiple sample texts; or, the sample text set corresponding to each task can also be collected offline, for example, for any of the multiple tasks, an image of the paper document corresponding to the any task can be collected, and then each character in the image can be identified through OCR (Optical Character Recognition) technology to obtain the sample text corresponding to the any task, and the above method can be repeated to obtain multiple sample texts corresponding to the any task, so that the sample text set corresponding to the any task can be composed of the multiple sample texts, and so on. The present disclosure does not impose any restriction on this.

[0056] Step 102: Obtain a reference quantization range corresponding to each sample text set.

[0057] In an embodiment of the present disclosure, the reference quantization range corresponding to each sample text set can be set based on experience. For example, for symmetric quantization, the lower limit of the reference quantization range can be 0, and the upper limit can be determined based on an empirical value; alternatively, the reference quantization range corresponding to each sample text set can also be determined based on the distribution of the corresponding sample text set, and the present disclosure does not impose any restrictions on this.

[0058] As an example, for each sample text set, each sample text in the sample text set can be input into the multi-task model, and the reference quantization range of the sample text set can be determined according to the output of each network layer in the multi-task model.

[0059] Step 103: determining a target quantization range according to the reference quantization range corresponding to each sample text set.

[0060] In the disclosed embodiment, the reference quantization ranges corresponding to each sample text set may be combined to determine the target quantization range used by the multi-task model. For example, the target quantization range may be generated by weighted summing the reference quantization ranges of each sample text set.

[0061] Step 104: training the multi-task model according to the target quantization range.

[0062] In the disclosed embodiment, the multi-task model may be trained according to the target quantization range.

[0063] The training method of the multi-task model of the disclosed embodiment is as follows: by obtaining sample text sets corresponding to multiple tasks, wherein each sample text set is used to perform adaptation training on the multi-task model for the corresponding task; obtaining the reference quantization range corresponding to each sample text set; determining the target quantization range according to the reference quantization range corresponding to each sample text set; and training the multi-task model according to the target quantization range. Thus, the reference quantization ranges of the sample text sets corresponding to multiple tasks are combined to determine the target quantization range adopted by the multi-task model, so that the multi-task model quantizes the text according to the target quantization range, which can avoid the problem that the multi-task model uses the quantization range corresponding to a single sample text set to quantize the text corresponding to different tasks, resulting in poor quantization effect, that is, the quantization effect of the multi-task model can be improved, thereby improving the accuracy and reliability of the model prediction results.

[0064] It should be noted that in the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information are all carried out with the user's consent, and are in compliance with the relevant laws and regulations and do not violate public order and good morals.

[0065] In order to clearly explain how the multi-task model is trained according to the target quantization range in the above embodiments of the present disclosure, the present disclosure also proposes a training method for the multi-task model.

[0066] Figure 2 A flowchart of a multi-task model training method provided in Embodiment 2 of the present disclosure.

[0067] like Figure 2 As shown, the training method of the multi-task model may include the following steps:

[0068] Step 201, obtaining sample text sets corresponding to multiple tasks, wherein each sample text set is used to perform adaptation training for the corresponding task on the multi-task model.

[0069] Step 202: Obtain a reference quantization range corresponding to each sample text set.

[0070] Step 203: determine a target quantization range according to the reference quantization range corresponding to each sample text set.

[0071] The explanation of steps 201 to 203 can be found in the relevant description of any embodiment of the present disclosure, and will not be repeated here.

[0072] Step 204 , using a multi-task model to quantize at least one sample text in each sample text set according to a target quantization range, so as to obtain a quantization result corresponding to the at least one sample text.

[0073] In the embodiment of the present disclosure, a multi-task model may be used to quantize at least one sample text in each sample text set according to a target quantization range to obtain a quantization result corresponding to the at least one sample text.

[0074] In a possible implementation of the embodiment of the present disclosure, the multi-task model may include multiple layers of network layers, each layer of the network layer may have a corresponding target quantization range, wherein the target quantization ranges corresponding to different network layers may be different. For example, taking the multi-task model as an example of a model based on the Transformer architecture, the multi-task model may include 12 layers of network layers, and the target quantization ranges corresponding to the 12 layers of network layers may be different. For any sample text in the at least one sample text mentioned above, the first layer of the network layer in the multiple layers of network layers may be used to quantize the floating-point input data according to the target quantization range of the first layer of the network layer to obtain the quantized input data, wherein the input data is determined according to the feature vector of any training sample, and the quantized input data is within the target quantization range. In addition, the first layer of the network layer may be used to perform a matrix-vector operation matching the target task corresponding to any sample text on the quantized input data to obtain a first operation result, and the first operation result may be dequantized to obtain the quantization result of the first layer of the network layer.

[0075] For non-first network layers in multiple network layers (such as the second network layer to the last network layer), the non-first network layer can be used to quantize the quantization result of the previous network layer of the non-first network layer according to the target quantization range of the non-first network layer, and the above-mentioned matrix-vector operation is performed on the quantized quantization result to obtain a second operation result, and the second operation result is dequantized to obtain the quantization result of the non-first network layer.

[0076] That is to say, for the second network layer, the second network layer can be used to quantize the quantization result of the first network layer according to the target quantization range of the second network layer, and the above-mentioned matrix-vector operation is performed on the quantized quantization result to obtain the second operation result, and the second operation result is inverse quantized to obtain the quantization result of the second network layer.

[0077] For the third network layer, the third network layer can be used to quantize the quantization result of the second network layer according to the target quantization range of the third network layer, and the above-mentioned matrix-vector operation is performed on the quantized quantization result to obtain a second operation result, and the second operation result is inversely quantized to obtain the quantization result of the third network layer.

[0078] By analogy, the quantization result of the last network layer can be determined.

[0079] Therefore, each network layer needs to perform quantization, matrix-vector operations and inverse quantization operations on the input data to obtain the quantization result of the corresponding network layer. Moreover, the quantization result of the previous network layer can be used as the input of the next network layer. Through layer-by-layer quantization, the quantization result corresponding to the sample text can be effectively calculated.

[0080] Step 205: training the multi-task model according to the quantization result corresponding to at least one sample text.

[0081] In the embodiment of the present disclosure, the multi-task model may be trained according to the quantization result corresponding to at least one sample text.

[0082] It should be noted that when the multi-task model includes multiple network layers, each network layer needs to quantize, perform matrix-vector operations and dequantize the input data to obtain the quantization result of the corresponding network layer. Accordingly, when training the multi-task model, the model parameters of the corresponding network layer can be adjusted according to the quantization result corresponding to each network layer to improve the quantization effect of each network layer.

[0083] The training method of the multi-task model of the embodiment of the present disclosure adopts the multi-task model to quantize at least one sample text in each sample text set according to the target quantization range to obtain the quantization result corresponding to the at least one sample text; and trains the multi-task model according to the quantization result corresponding to the at least one sample text. Thus, by quantizing the sample texts through the multi-task model, the prediction efficiency of the multi-task model can be improved, and by training the multi-task model according to the quantization result of the sample text, the quantization effect of the multi-task model can be improved.

[0084] In order to clearly illustrate how to train a multi-task model according to a target quantization range in any embodiment of the present disclosure, the present disclosure also proposes a training method for a multi-task model.

[0085] Figure 3 A flowchart of a multi-task model training method provided in Embodiment 3 of the present disclosure.

[0086] like Figure 3 As shown, the training method of the multi-task model may include the following steps:

[0087] Step 301, obtaining sample text sets corresponding to multiple tasks, wherein each sample text set is used to perform adaptation training for the multi-task model for the corresponding task.

[0088] Step 302: Obtain a reference quantization range corresponding to each sample text set.

[0089] Step 303: determine a target quantization range according to the reference quantization range corresponding to each sample text set.

[0090] Step 304: for any sample text in at least one sample text, use the first network layer in the multi-layer network layer to quantize the floating-point input data according to the target quantization range corresponding to the first network layer to obtain the quantization result of the first network layer.

[0091] The input data is determined based on the feature vector of any training sample.

[0092] Step 305, for a non-first network layer in the multi-layer network layer, the non-first network layer is used to quantize the quantization result of the previous network layer of the non-first network layer according to the target quantization range corresponding to the non-first network layer, so as to obtain the quantization result of the non-first network layer.

[0093] For explanations of steps 301 to 305 , reference may be made to the relevant descriptions in any embodiment of the present disclosure, and will not be repeated here.

[0094] Step 306, using the first network layer to perform matrix-vector operations on the feature vector of any sample text to obtain the output result of the first network layer.

[0095] It can be understood that the purpose of quantization is to reduce the computational intensity, parameter size and memory consumption of the multi-task model, but the premise of quantization is that the prediction accuracy of the model should not be reduced too much. Therefore, in the present disclosure, the unquantized output results of each network layer can be calculated, so that the model parameters in the corresponding network layer can be adjusted according to the difference between the unquantized output results and the quantized results corresponding to each network layer.

[0096] That is, in the present disclosure, for the first network layer in the multi-layer network, the first network layer can be used to perform matrix-vector operations matching the target task corresponding to any of the sample texts on the feature vectors of the above-mentioned any of the sample texts to obtain the output result of the first network layer.

[0097] Step 307: Adjust the model parameters in the first network layer according to the first difference between the quantization result corresponding to the first network layer and the output result.

[0098] In the disclosed embodiment, the model parameters in the first network layer can be adjusted according to the first difference between the quantization result and the output result corresponding to the first network layer. Thus, the model parameters of each network layer can be adjusted separately to improve the quantization effect of each network layer.

[0099] As an example, a first loss function can be generated based on the first difference, wherein the value of the first loss function is positively correlated with the above-mentioned first difference (i.e., positively correlated), that is, the smaller the first difference, the smaller the value of the first loss function, and conversely, the larger the first difference, the larger the value of the first loss function. Therefore, in the present disclosure, the model parameters in the first network layer can be adjusted according to the value of the first loss function to minimize the value of the first loss function.

[0100] It should be noted that the above only takes the training termination condition of the first network layer as the minimization of the value of the first loss function as an example. In actual application, other termination conditions can also be set. For example, the termination condition can also be that the number of training times reaches a set number threshold, the training time is greater than a set time threshold, etc. The present disclosure does not impose any restrictions on this.

[0101] In a possible implementation of the embodiment of the present disclosure, a first weight corresponding to a sample text set to which any of the above-mentioned training samples belongs can be obtained, a first value can be generated based on the first weight and the first difference, and the variance of each sample text set can be determined, a second value can be generated based on the variance corresponding to each sample text set, and a first loss function corresponding to the first network layer can be generated based on the first value and the second value, so as to adjust the model parameters in the first network layer according to the first loss function corresponding to the first network layer. Thus, by training the first network layer according to the loss function, the training effect of the first network layer can be improved, and then the quantization effect of the first network layer can be improved.

[0102] As an example, the first loss function loss can be as follows:

[0103]

[0104] in, σ i represents the variance corresponding to the i-th sample text set, σ j represents the variance corresponding to the j-th sample text set, Represents the quantized result of the network layer output corresponding to the sample text k, represents the unquantized output of the network layer corresponding to the sample text k, α k Indicates the weight corresponding to the sample text set to which the sample text k belongs.

[0105] Step 308: Use the non-first network layer to perform matrix-vector operations on the output result of the previous network layer to obtain the output result of the non-first network layer.

[0106] In an embodiment of the present disclosure, for a non-first network layer in a multi-layer network layer, the non-first network layer can be used to perform a matrix-vector operation matching the target task corresponding to any sample text on the output result of the previous network layer to obtain the output result of the non-first network layer.

[0107] That is to say, for the second network layer, the second network layer can be used to perform the above matrix-vector operation on the output result of the first network layer to obtain the output result of the second network layer.

[0108] For the third network layer, the third network layer can be used to perform the above matrix-vector operation on the output result of the second network layer to obtain the output result of the third network layer.

[0109] By analogy, the output result of the last network layer can be determined.

[0110] Therefore, each network layer does not need to perform quantization and dequantization operations, but only needs to perform matrix-vector operations on the input data to obtain the output result of the corresponding network layer. Moreover, the output result of the previous network layer can be used as the input of the next network layer. The output results of each network layer can be effectively calculated through layer-by-layer calculation. Therefore, according to the output results of each network layer, the model parameters in the corresponding network layer can be adjusted to improve the quantization effect of each network layer in the multi-task model, thereby improving the prediction accuracy of the model.

[0111] Step 309: Adjust the model parameters in the non-first network layer according to the second difference between the quantization result and the output result corresponding to the non-first network layer.

[0112] In the disclosed embodiment, the model parameters in the non-first network layer can be adjusted according to the second difference between the quantization result and the output result corresponding to the non-first network layer. Thus, the model parameters of each network layer can be adjusted separately to improve the quantization effect of each network layer.

[0113] As an example, a second loss function can be generated based on the second difference, wherein the value of the second loss function is positively correlated with the above-mentioned second difference (i.e., positively correlated), that is, the smaller the second difference, the smaller the value of the second loss function, and conversely, the larger the second difference, the larger the value of the second loss function. Therefore, in the present disclosure, the model parameters in the non-first-layer network layer can be adjusted according to the value of the second loss function to minimize the value of the second loss function.

[0114] It should be noted that the above only takes the training termination condition of the non-first network layer as the minimization of the value of the second loss function as an example. In actual application, other termination conditions can also be set. For example, the termination condition can also be that the number of training times reaches a set number threshold, the training time is greater than a set time threshold, etc. The present disclosure does not impose any restrictions on this.

[0115] In a possible implementation of the embodiment of the present disclosure, the first weight corresponding to the sample text set to which any of the above training samples belongs can be obtained, and a third value can be generated according to the first weight and the second difference, and a second value can be generated according to the variance corresponding to each sample text set, and a second loss function corresponding to the non-first network layer can be generated according to the third value and the second value, so as to adjust the model parameters in the non-first network layer according to the second loss function corresponding to the non-first network layer. For example, formula (1) can also be used to determine the second loss function.

[0116] The training method of the multi-task model of the embodiment of the present disclosure can improve the quantization effect of each network layer by adjusting the model parameters in the corresponding network layer according to the difference between the unquantized output result and the quantized result corresponding to each network layer, thereby improving the quantization effect of the multi-task model.

[0117] In order to clearly explain how to determine the reference quantization range corresponding to each sample text set in any embodiment of the present disclosure, the present disclosure also proposes a training method for a multi-task model.

[0118] Figure 4 A flowchart of a multi-task model training method provided in Embodiment 4 of the present disclosure.

[0119] like Figure 4 As shown, the training method of the multi-task model may include the following steps:

[0120] Step 401, obtaining sample text sets corresponding to multiple tasks, wherein each sample text set is used to perform adaptation training for the corresponding task on the multi-task model.

[0121] For explanation of step 401, reference may be made to the relevant description in any embodiment of the present disclosure, and no further details will be given here.

[0122] Step 402: for any sample text set in each sample text set, determine a plurality of sample texts from any sample text set.

[0123] In an embodiment of the present disclosure, for any sample text set among multiple sample text sets, multiple sample texts can be determined from the any sample text set, wherein there is no limitation on the method for determining the multiple sample texts. For example, multiple sample texts can be randomly selected from the any sample text set, or multiple sample texts can be selected in sequence from the any sample text set, and so on.

[0124] Step 403, inputting the multiple sample texts layer by layer into the multi-layer network layer in the multi-task model to obtain an output matrix corresponding to the multiple sample texts output by each layer of the network layer.

[0125] In the embodiment of the present disclosure, a plurality of sample texts may be input into a multi-layer network layer by layer to obtain an output matrix corresponding to the plurality of sample texts output by each layer of the network layer.

[0126] In a possible implementation of the embodiments of the present disclosure, for any text among the above-mentioned multiple sample texts, the first network layer among the multi-layer network layers can be used to perform a matrix-vector operation matching the task corresponding to the any text on the feature vector of the any text, so as to obtain an output matrix corresponding to the any text output by the first network layer, and a non-first network layer among the multi-layer network layers can be used to perform a matrix-vector operation matching the task corresponding to the any text on the output matrix of the previous network layer of the non-first network layer, so as to obtain an output matrix corresponding to the any text output by the non-first network layer.

[0127] That is to say, the output matrix of any text output by the first network layer can be used as the input of the second network layer, and the second network layer can perform matrix-vector operations on the output matrix of the first network layer to obtain the output matrix corresponding to the any text output by the second network layer.

[0128] The output matrix of any text output by the second network layer can be used as the input of the third network layer. The third network layer can perform matrix-vector operations on the output matrix of the second network layer to obtain the output matrix corresponding to the any text output by the third network layer.

[0129] By analogy, we can get the output matrix corresponding to any text output by the last network layer.

[0130] Therefore, the output matrix of the previous network layer is used as the input of the next network layer, and the output matrix of each network layer is effectively calculated through layer-by-layer calculation. Therefore, the reference quantization range of each network layer can be effectively calculated based on the output matrix of each network layer.

[0131] Step 404, determining a reference quantization range of each network layer corresponding to any sample text set according to an output matrix corresponding to a plurality of sample texts output by each network layer.

[0132] In the embodiment of the present disclosure, for any network layer in the multi-layer network layer, the reference quantization range of any network layer corresponding to any of the above sample text sets can be determined according to the output matrix corresponding to multiple sample texts output by the any network layer.

[0133] In a possible implementation of the embodiments of the present disclosure, for any network layer in the multi-layer network layers, the maximum element value in the output matrix corresponding to the multiple sample texts output by the any network layer can be determined, so that the reference quantization range of the any network layer corresponding to the above-mentioned any sample text set can be determined based on the maximum element value in the output matrix corresponding to the multiple sample texts.

[0134] For example, assuming that the reference quantization range of each network layer corresponding to the sample text set A is determined based on 100 sample texts in the sample text set A, for each network layer, the maximum element value in each of the 100 output matrices output by the network layer can be determined, so that the reference quantization range of the network layer corresponding to the sample text set A can be determined based on the 100 maximum element values.

[0135] For example, taking the multi-task model to convert floating-point data into int8 as an example, each maximum element value can be divided by 127 to obtain the quantization scale, so that a reference quantization range can be generated based on the minimum quantization scale and the maximum quantization scale.

[0136] Step 405 , determining a target quantization range of each network layer according to a reference quantization range of each network layer corresponding to each sample text set.

[0137] In the embodiment of the present disclosure, the target quantization range of each network layer may be determined according to the reference quantization range of each network layer corresponding to the plurality of sample text sets.

[0138] As an example, the reference quantization ranges of the same network layer corresponding to multiple sample text sets may be fused to obtain the target quantization range of the same network layer.

[0139] Step 406, training the multi-task model according to the target quantization range of each network layer.

[0140] For explanations of steps 405 to 406, reference may be made to the relevant descriptions in any embodiment of the present disclosure, and will not be repeated here.

[0141] The training method of the multi-task model of the embodiment of the present disclosure is to determine multiple sample texts from any sample text set in each sample text set; input the multiple sample texts layer by layer into the multi-layer network layer in the multi-task model to obtain the output matrix corresponding to the multiple sample texts output by each network layer; and determine the reference quantization range of each network layer corresponding to any sample text set according to the output matrix corresponding to the multiple sample texts output by each network layer. Thus, the reference quantization range of each network layer can be effectively determined according to the output of each network layer in the multi-task model.

[0142] In order to clearly explain how to determine the target quantization range used by the multi-task model in any embodiment of the present disclosure, the present disclosure also proposes a training method for the multi-task model.

[0143] Figure 5 A flowchart of the multi-task model training method provided in Embodiment 5 of the present disclosure.

[0144] like Figure 5 As shown, the training method of the multi-task model may include the following steps:

[0145] Step 501, obtaining sample text sets corresponding to multiple tasks, wherein each sample text set is used to perform adaptation training for the multi-task model for the corresponding task.

[0146] Step 502: for any sample text set in each sample text set, determine a plurality of sample texts from any sample text set.

[0147] Step 503, inputting the multiple sample texts layer by layer into the multi-layer network layer in the multi-task model to obtain an output matrix corresponding to the multiple sample texts output by each layer of the network layer.

[0148] Step 504: Determine the reference quantization range of each network layer corresponding to any sample text set according to the output matrix corresponding to the multiple sample texts output by each network layer.

[0149] For explanations of steps 501 to 504 , reference may be made to the relevant descriptions in any embodiment of the present disclosure and will not be repeated here.

[0150] Step 505: Obtain the weight corresponding to each sample text set.

[0151] In the embodiments of the present disclosure, the weights corresponding to the sample text sets may be obtained, wherein the weights corresponding to the sample text sets may be set according to empirical values, or may be calculated according to set rules, and the present disclosure does not impose any limitation on this.

[0152] As a possible implementation method, the total number of sample texts contained in each sample text set can be determined. For example, the number of sample texts contained in the i-th sample text set is marked as A. i , then the total quantity is For any sample text set in each sample text set, a first quantity of sample texts included in the sample text set is determined, so that the weight of the sample text set can be determined according to the ratio of the first quantity to the total quantity.

[0153] For example, the weight of labeling the i-th sample text set is α i , then:

[0154]

[0155] It is understandable that the higher the number of sample texts contained in a sample text set, the greater the weight corresponding to the sample text set, so that the model is more inclined to train the task corresponding to the sample text set to improve the training effect of the task.

[0156] Step 506 , based on the weights corresponding to the sample text sets, weighted sum is performed on the reference quantization upper limits of the reference quantization range of the same network layer corresponding to the sample text sets to obtain the target quantization upper limit of the target quantization range of the same network layer.

[0157] In the embodiment of the present disclosure, the reference quantization upper limits of the reference quantization range of the same network layer corresponding to each sample text set may be weighted summed according to the weight corresponding to each sample text set to obtain the target quantization upper limit of the target quantization range of the same network layer.

[0158] For example, the target quantization upper limit of the target quantization range of a certain network layer is scale 0 , the reference quantization upper limit of the reference quantization range of the network layer corresponding to the i-th sample text set is scale 0i , then the target quantization upper limit of the target quantization range of the network layer is:

[0159]

[0160] For example, suppose there are two tasks, namely text classification and sorting. For a certain network layer in the multi-task model, suppose the reference quantization range of the network layer corresponding to the sample text set corresponding to the text classification task is [a, b], and the reference quantization range of the network layer corresponding to the sample text set corresponding to the sorting task is [c, d], and the weight corresponding to the text classification task is 0.3, and the weight corresponding to the sorting task is 0.7. Then the target quantization upper limit scale of the target quantization range of the network layer is 0 It is b*0.3+d*0.7.

[0161] Step 507 , according to the weight corresponding to each sample text set, weighted sum is performed on the reference quantization lower limits of the reference quantization range of the same network layer corresponding to each sample text set to obtain the target quantization lower limit of the target quantization range of the same network layer.

[0162] In the disclosed embodiment, the reference quantization lower limits of the reference quantization range of the same network layer corresponding to each sample text set may be weighted summed according to the weight corresponding to each sample text set to obtain the target quantization lower limit of the target quantization range of the same network layer.

[0163] For example, the target quantization upper limit of the target quantization range of a certain network layer is scale 1 , the reference quantization upper limit of the reference quantization range of the network layer corresponding to the i-th sample text set is scale 1i , then the target quantization upper limit of the target quantization range of the network layer is:

[0164]

[0165] Still taking the above example as an example, the target quantization upper limit scale of the target quantization range of the above network layer is 1 It is a*0.3+c*0.7.

[0166] Therefore, in the present disclosure, the target quantization range of each network layer can be determined according to the target quantization upper limit and target quantization lower limit corresponding to each network layer. 0 ,scale 1 ].

[0167] Step 508: Train the multi-task model according to the target quantization range of each network layer.

[0168] For explanation of step 508, reference may be made to the relevant description in any embodiment of the present disclosure, and will not be repeated here.

[0169] The training method of the multi-task model of the embodiment of the present disclosure determines the target quantization range adopted by the multi-task model by weighted summing up the reference quantization ranges of the sample text sets corresponding to multiple tasks, so that the multi-task model quantizes the text according to the target quantization range, which can improve the quantization effect of the multi-task model.

[0170] In a possible implementation of the embodiment of the present disclosure, the number of trained multi-task models can be multiple. For example, each sample text set can be divided into multiple parts, and one of the parts in each sample text set is used to train a multi-task model, so that multiple multi-task models can be obtained. Alternatively, in the process of training the multi-task model, models trained at multiple time points can be obtained, or multiple multi-task models with different network parameters can be trained to obtain multiple multi-task models, etc. In the case of multiple multi-task models, it is very important to select the model with the best comprehensive evaluation from the multiple multi-task models, so that the text to be recognized can be predicted based on the model with the best comprehensive evaluation to improve the accuracy and reliability of the prediction results.

[0171] To address the above issues, we can Figure 6 In the illustrated embodiment, a target multi-task model with the best comprehensive evaluation is selected from a plurality of multi-task models.

[0172] Figure 6 A flowchart of the training method of the multi-task model provided in Embodiment 6 of the present disclosure.

[0173] like Figure 6 As shown, the training method of the multi-task model may include the following steps:

[0174] Step 601, obtaining sample text sets corresponding to multiple tasks, wherein each sample text set is used to perform adaptation training for the corresponding task on the multi-task model.

[0175] Step 602: Obtain a reference quantization range corresponding to each sample text set.

[0176] Step 603: Determine a target quantization range according to the reference quantization range corresponding to each sample text set.

[0177] Step 604: training the multi-task model according to the target quantization range, wherein the trained multi-task models are multiple.

[0178] For explanations of steps 601 to 604 , reference may be made to the relevant descriptions in any embodiment of the present disclosure and will not be repeated here.

[0179] Step 605 , for any multi-task model among the multiple multi-task models, determine a plurality of evaluation indicators of any multi-task model.

[0180] In the embodiments of the present disclosure, the multiple evaluation indicators may include accuracy, recall rate, F1 value, PNR value, etc., and the present disclosure does not impose any limitation on this.

[0181] In the embodiment of the present disclosure, for any multi-task model among the multiple multi-task models, multiple evaluation indicators of the multi-task model can be determined according to the output of the any multi-task model.

[0182] Step 606: for any evaluation indicator among the multiple evaluation indicators, determine the correlation coefficient between the any evaluation indicator and the remaining evaluation indicators among the multiple evaluation indicators except the any evaluation indicator.

[0183] In the embodiment of the present disclosure, for any evaluation indicator among the multiple evaluation indicators, a correlation coefficient between the any evaluation indicator and the remaining evaluation indicators among the multiple evaluation indicators except the any evaluation indicator may be determined.

[0184] As an example, suppose any multi-task model includes x 1 、x 2 , …, x n These n evaluation indicators, then the evaluation indicator x in a multi-task model i and x j The mutual correlation coefficient It can be calculated by the following formula:

[0185]

[0186] in, It refers to the same evaluation index x in multiple multi-task models i The corresponding mean, It refers to the same evaluation index x in multiple multi-task models j The corresponding mean.

[0187] Step 607, determining a target evaluation index corresponding to any multi-task model according to the correlation coefficient between any evaluation index and the remaining evaluation indexes.

[0188] In the disclosed embodiment, the target evaluation index corresponding to any of the above-mentioned multi-task models can be determined according to the correlation coefficient between any of the above-mentioned evaluation indexes and the remaining evaluation indexes.

[0189] In the embodiment of the present disclosure, the target correlation coefficient corresponding to any evaluation indicator can be determined according to the sum of the correlation coefficients between any evaluation indicator and the other evaluation indicators. For example, the evaluation indicator x is marked as i The corresponding target correlation coefficient is Then we have:

[0190]

[0191] In the present disclosure, the weight corresponding to any evaluation indicator can be determined according to the target correlation coefficient corresponding to any evaluation indicator; wherein the weight is negatively correlated with the target correlation coefficient, that is, the larger the target correlation coefficient, the smaller the weight, and vice versa, the smaller the target correlation coefficient, the larger the weight. Thus, according to the weights corresponding to multiple evaluation indicators, multiple evaluation indicators can be weighted and summed to obtain the target evaluation indicator corresponding to any of the above multi-task models.

[0192] For example, if the target evaluation index is x, then:

[0193]

[0194] Therefore, by using the mutual correlation coefficient to characterize the relationship between multiple evaluation indicators and fusing or weighted summing multiple evaluation indicators to determine the target evaluation indicator of the multi-task model, the accuracy and reliability of the target evaluation indicator determination results used to evaluate the quantitative effect of the model can be improved, so that the quantized target multi-task model with the best comprehensive evaluation can be selected based on reliable target evaluation indicators.

[0195] Step 608 , determining a target multi-task model from among the multi-task models according to the target evaluation index corresponding to each multi-task model.

[0196] In the disclosed embodiment, a target multi-task model may be determined from among the multi-task models according to target evaluation indicators corresponding to the multi-task models.

[0197] As a possible implementation method, the multi-task model with the largest target evaluation index can be used as the target multi-task model.

[0198] As another possible implementation manner, a multi-task model whose target evaluation index exceeds a set threshold may be used as a target multi-task model.

[0199] The training method of the multi-task model of the embodiment of the present disclosure comprehensively evaluates the quantization effect of the multi-task model according to multiple evaluation indicators, so that the target multi-task model after quantization with the best comprehensive evaluation can be selected according to the target evaluation indicators, and then the text to be recognized is predicted according to the target multi-task model, which can improve the accuracy and reliability of the prediction results.

[0200] In summary, in the process of multi-task quantization, assuming that the sample text sets corresponding to the multi-tasks obey the multivariate Gaussian distribution, the quantization ranges of each sample data set can be weighted summed to determine the target quantization range adopted by the multi-task model, which can improve the quantization effect of the multi-task model.

[0201] In addition, the negative correlation coefficient is used to characterize the relationship between multiple evaluation indicators, and the target evaluation indicator is determined by fusing multiple evaluation indicators, so that the quantized multi-task model with the best comprehensive evaluation can be selected according to the target evaluation indicator.

[0202] The above are various embodiments corresponding to the training method of the multi-task model. The present disclosure also proposes an application method of the multi-task model, namely, a prediction method.

[0203] Figure 7 This is a flow chart of the prediction method provided in Embodiment 7 of the present disclosure.

[0204] like Figure 7 As shown, the prediction method may include the following steps:

[0205] Step 701, obtaining text to be recognized.

[0206] In the embodiments of the present disclosure, the text to be recognized can be obtained from an existing test set, or the text to be recognized can be collected online, for example, by using web crawler technology, or the text to be recognized can be collected offline, or the text to be recognized can be text input by a user, etc. The embodiments of the present disclosure do not limit this.

[0207] Step 702: quantize the text to be recognized according to the target quantization range using a multi-task model to obtain quantized data corresponding to the text to be recognized, and predict the quantized data to obtain a prediction result corresponding to the text to be recognized.

[0208] Among them, the multi-task model can be adopted Figures 1 to 6 Any method embodiment is trained.

[0209] In the embodiments of the present disclosure, the prediction result is related to the target task corresponding to the text to be recognized. For example, when the target task corresponding to the text to be recognized is a text classification task, the prediction result is a classification label; when the target task corresponding to the text to be recognized is a scoring task (such as text quality scoring), the prediction result can be a text quality score, and so on. The present disclosure does not list them one by one.

[0210] In the embodiments of the present disclosure, the explanation of the target quantization range can refer to the relevant description of any of the above embodiments of the present disclosure, and will not be repeated here.

[0211] In an embodiment of the present disclosure, the text to be recognized can be input into a trained multi-task model, which quantizes the text to be recognized according to a target quantization range to obtain quantized data corresponding to the text to be recognized, and predicts the quantized data to obtain a prediction result corresponding to the text to be recognized.

[0212] The prediction method of the disclosed embodiment obtains the text to be recognized, uses a multi-task model to quantize the text to be recognized according to the target quantization range, so as to obtain the quantized data corresponding to the text to be recognized, and predicts the quantized data to obtain the prediction result corresponding to the text to be recognized. Therefore, based on the deep learning technology, the text to be recognized is predicted (such as classification, scoring, sorting, labeling, etc.), which can improve the accuracy and reliability of the prediction result.

[0213] In order to clearly illustrate how the multi-task model in the above embodiment of the present disclosure predicts the text to be recognized, the present disclosure also proposes a prediction method.

[0214] Figure 8 This is a flow chart of the prediction method provided in Embodiment 8 of the present disclosure.

[0215] like Figure 8 As shown, the prediction method may include the following steps:

[0216] Step 801, obtaining text to be recognized.

[0217] For the explanation of step 801 , reference may be made to the relevant description in the above embodiment, which will not be elaborated here.

[0218] Step 802, using the first network layer in the multi-layer network layer in the multi-task model to quantize the floating-point target data according to the target quantization range corresponding to the first network layer, so as to obtain the quantized data of the first network layer; wherein the target data is determined according to the feature vector of the text to be recognized.

[0219] In an embodiment of the present disclosure, a multi-task model may include multiple network layers, each network layer may have a corresponding target quantization range, wherein the target quantization ranges corresponding to different network layers may be different. For example, taking the multi-task model as an example of a model based on the Transformer architecture, the multi-task model may include 12 network layers, and the target quantization ranges corresponding to the 12 network layers may be different. For the text to be recognized, the first network layer in the multiple network layers may be used to quantize the floating-point target data according to the target quantization range of the first network layer to obtain the quantized target data, wherein the target data is determined according to the feature vector of the text to be recognized, and the quantized target data is within the target quantization range. In addition, the first network layer may be used to perform a matrix-vector operation matching the target task corresponding to the text to be recognized on the quantized target data to obtain a candidate operation result, and the candidate operation result is dequantized to obtain the quantized data of the first network layer.

[0220] Step 803, for a non-first network layer in the multi-layer network layer, the non-first network layer is used to quantize the quantized data of the previous network layer of the non-first network layer according to the target quantization range corresponding to the non-first network layer, so as to obtain the quantized data of the non-first network layer.

[0221] In the embodiments of the present disclosure, for a non-first network layer in a multi-layer network layer (such as the second network layer to the last network layer), the non-first network layer can be used to quantize the quantized data of the previous network layer of the non-first network layer according to the target quantization range of the non-first network layer, and the above-mentioned matrix-vector operation is performed on the quantized quantized data to obtain the target operation result, and the target operation result is dequantized to obtain the quantized data of the non-first network layer.

[0222] That is to say, for the second network layer, the second network layer can be used to quantize the quantized data of the first network layer according to the target quantization range of the second network layer, and the above-mentioned matrix-vector operation is performed on the quantized data to obtain the target operation result, and the target operation result is dequantized to obtain the quantized data of the second network layer.

[0223] For the third network layer, the third network layer can be used to quantize the quantized data of the second network layer according to the target quantization range of the third network layer, and the above-mentioned matrix-vector operation is performed on the quantized data to obtain the target operation result, and the target operation result is dequantized to obtain the quantized data of the third network layer.

[0224] By analogy, the quantitative data of the last network layer can be determined.

[0225] Step 804, using the prediction layer in the multi-task model to predict the quantized data of the last network layer to obtain a prediction result corresponding to the text to be recognized.

[0226] In the disclosed embodiment, the prediction layer in the multi-task model can be used to predict (such as classification, scoring, sorting, labeling, etc.) the quantitative data of the last network layer to obtain the prediction result corresponding to the text to be recognized.

[0227] For example, when the target task corresponding to the text to be recognized is a text classification task, the prediction layer can be used to perform text classification on the quantized data of the last network layer to obtain the classification label corresponding to the text to be recognized.

[0228] For another example, when the target task corresponding to the text to be recognized is a scoring task (such as text quality scoring), the prediction layer can be used to predict the quality of the quantized data of the last network layer to obtain the text quality score.

[0229] The prediction method of the embodiment of the present disclosure can improve the prediction efficiency of the model by adopting a multi-task model to quantify the text to be recognized to obtain quantified data and predict the quantified data.

[0230] With the above Figures 1 to 6 Corresponding to the training method of the multi-task model provided in the embodiment, the present disclosure also provides a training device for a multi-task model. Figures 1 to 6 The training method of the multi-task model provided in the embodiment corresponds to the embodiment, so the implementation method of the multi-task model training method is also applicable to the training device of the multi-task model provided in the embodiment of the present disclosure, and will not be described in detail in the embodiment of the present disclosure.

[0231] Fig. 9 A schematic diagram of the structure of a training device for a multi-task model provided in Embodiment 9 of the present disclosure.

[0232] like Fig. 9 As shown, the multi-task model training device 900 may include: a first acquisition module 901, a second acquisition module 902, a first determination module 903 and a training module 904.

[0233] The first acquisition module 901 is used to acquire sample text sets corresponding to multiple tasks, wherein each sample text set is used to perform adaptation training for the multi-task model for the corresponding task.

[0234] The second acquisition module 902 is used to acquire the reference quantization range corresponding to each sample text set.

[0235] The first determination module 903 is used to determine a target quantization range according to a reference quantization range corresponding to each sample text set.

[0236] The training module 904 is used to train the multi-task model according to the target quantization range.

[0237] In a possible implementation of the embodiment of the present disclosure, the training module 904 may include:

[0238] The quantization unit is used to quantize at least one sample text in each sample text set according to a target quantization range by using a multi-task model to obtain a quantization result corresponding to the at least one sample text.

[0239] The training unit is used to train the multi-task model according to the quantization result corresponding to at least one sample text.

[0240] In a possible implementation of the embodiment of the present disclosure, the multi-task model includes multiple network layers, each of which has a corresponding target quantization range; a quantization unit is specifically used to: for any sample text in at least one sample text, use the first network layer in the multiple network layers to quantize floating-point input data according to the target quantization range corresponding to the first network layer, so as to obtain a quantization result of the first network layer; wherein the input data is determined according to a feature vector of any training sample; for a non-first network layer in the multiple network layers, use the non-first network layer to quantize the quantization result of a previous network layer of the non-first network layer according to the target quantization range corresponding to the non-first network layer, so as to obtain a quantization result of the non-first network layer.

[0241] In a possible implementation of the embodiment of the present disclosure, the training unit is specifically used to: use the first network layer to perform matrix-vector operations on the feature vector of any sample text to obtain the output result of the first network layer; according to the first difference between the quantization result corresponding to the first network layer and the output result, adjust the model parameters in the first network layer.

[0242] In a possible implementation of the embodiment of the present disclosure, the training unit is specifically used to: obtain a first weight corresponding to a sample text set to which any training sample belongs; generate a first value based on the first weight and the first difference; determine the variance of each sample text set; generate a second value based on the variance corresponding to each sample text set; generate a loss function corresponding to the first network layer based on the first value and the second value; and adjust the model parameters in the first network layer based on the loss function corresponding to the first network layer.

[0243] In a possible implementation of the embodiment of the present disclosure, the training unit is further used to: use a non-first network layer to perform matrix-vector operations on the output result of the previous network layer to obtain the output result of the non-first network layer; and adjust the model parameters in the non-first network layer according to the second difference between the quantization result corresponding to the non-first network layer and the output result.

[0244] In a possible implementation of the embodiment of the present disclosure, the second acquisition module 902 may include:

[0245] The first determining unit is used to determine a plurality of sample texts from any sample text set in each sample text set.

[0246] The input unit is used to input multiple sample texts layer by layer into the multi-layer network layer in the multi-task model to obtain an output matrix corresponding to the multiple sample texts output by each layer of the network layer.

[0247] The second determination unit is used to determine the reference quantization range of each network layer corresponding to any sample text set according to the output matrix corresponding to multiple sample texts output by each network layer.

[0248] In a possible implementation of the embodiment of the present disclosure, the input unit is specifically used to: for any text among multiple sample texts, use the first network layer among the multi-layer network layers to perform matrix-vector operations on the feature vector of any text to obtain an output matrix corresponding to any text output by the first network layer; use a non-first network layer among the multi-layer network layers to perform matrix-vector operations on the output matrix of the previous network layer of the non-first network layer to obtain an output matrix corresponding to any text output by the non-first network layer.

[0249] In a possible implementation manner of the embodiments of the present disclosure, the second determination unit is specifically used to: determine, for any network layer in the multi-layer network layers, the maximum element value in the output matrix corresponding to multiple sample texts output by any network layer; and determine, based on the maximum element value in the output matrix corresponding to the multiple sample texts, a reference quantization range of any network layer corresponding to any sample text set.

[0250] In a possible implementation manner of the embodiment of the present disclosure, the first determination module 903 is specifically used to: obtain the weight corresponding to each sample text set; according to the weight corresponding to each sample text set, perform weighted summation of the reference quantization upper limits of the reference quantization range of the same network layer corresponding to each sample text set to obtain the target quantization upper limit of the target quantization range of the same network layer; according to the weight corresponding to each sample text set, perform weighted summation of the reference quantization lower limits of the reference quantization range of the same network layer corresponding to each sample text set to obtain the target quantization lower limit of the target quantization range of the same network layer.

[0251] In a possible implementation of the embodiment of the present disclosure, the first determination module 903 is specifically used to: determine the total number of sample texts included in each sample text set; for any sample text set in each sample text set, determine the first number of sample texts included in any sample text set; and determine the weight of any sample text set based on the ratio of the first number to the total number.

[0252] In a possible implementation of the embodiment of the present disclosure, there are multiple multi-task models obtained through training, and the network parameters of the multiple multi-task models are different; the multi-task model training device 800 may further include:

[0253] The second determination module is used to determine multiple evaluation indicators of any multi-task model among the multiple multi-task models.

[0254] The third determination module is used to determine, for any evaluation indicator among the multiple evaluation indicators, a correlation coefficient between the any evaluation indicator and the remaining evaluation indicators among the multiple evaluation indicators except the any evaluation indicator.

[0255] The fourth determination module is used to determine the target evaluation index corresponding to any multi-task model according to the correlation coefficient between any evaluation index and the remaining evaluation indexes.

[0256] The fifth determination module is used to determine the target multi-task model from the multi-task models according to the target evaluation indicators corresponding to the multi-task models.

[0257] In a possible implementation of the embodiment of the present disclosure, the fourth determination module is specifically used to: determine the target correlation coefficient based on the sum of the correlation coefficients between any evaluation indicator and the remaining evaluation indicators; determine the weight corresponding to any evaluation indicator based on the target correlation coefficient; wherein the weight is negatively correlated with the target correlation coefficient; and perform weighted summation of multiple evaluation indicators based on the weights corresponding to multiple evaluation indicators to obtain the target evaluation indicator corresponding to any multi-task model.

[0258] The training device of the multi-task model of the disclosed embodiment obtains sample text sets corresponding to multiple tasks, wherein each sample text set is used to perform adaptation training on the multi-task model for the corresponding task; obtains the reference quantization range corresponding to each sample text set; determines the target quantization range according to the reference quantization range corresponding to each sample text set; and trains the multi-task model according to the target quantization range. Thus, the target quantization range adopted by the multi-task model is determined by combining the reference quantization ranges of the sample text sets corresponding to multiple tasks, so that the multi-task model quantizes the text according to the target quantization range, which can avoid the problem that the multi-task model uses the quantization range corresponding to a single sample text set to quantize the text corresponding to different tasks, resulting in poor quantization effect, that is, the quantization effect of the multi-task model can be improved, thereby improving the accuracy and reliability of the model prediction results.

[0259] With the above Figures 7 and 8 Corresponding to the prediction method provided in the embodiment, the present disclosure also provides a prediction device. Figures 7 and 8 The prediction method provided in the embodiment corresponds to the above, so the implementation of the prediction method is also applicable to the prediction device provided in the embodiment of the present disclosure, and will not be described in detail in the embodiment of the present disclosure.

[0260] Fig.10 This is a schematic diagram of the structure of the prediction device provided in the tenth embodiment of the present disclosure.

[0261] like Fig.10As shown, the prediction device 1000 may include: an acquisition module 1001 and a prediction module 1002 .

[0262] The acquisition module 1001 is used to acquire the text to be recognized.

[0263] The prediction module 1002 is used to quantize the text to be recognized according to the target quantization range using a multi-task model to obtain quantized data corresponding to the text to be recognized, and predict the quantized data to obtain a prediction result corresponding to the text to be recognized.

[0264] In a possible implementation of the embodiment of the present disclosure, the multi-task model includes a multi-layer network layer and a prediction layer, and the prediction module 1002 is specifically used to: use the first network layer in the multi-layer network layer to quantize the floating-point target data according to the target quantization range corresponding to the first network layer, so as to obtain the quantized data of the first network layer; wherein the target data is determined according to the feature vector of the text to be recognized; for the non-first network layer in the multi-layer network layer, use the non-first network layer to quantize the quantized data of the previous network layer of the non-first network layer according to the target quantization range corresponding to the non-first network layer, so as to obtain the quantized data of the non-first network layer; use the prediction layer to predict the quantized data of the last network layer, so as to obtain the prediction result corresponding to the text to be recognized.

[0265] The prediction device of the disclosed embodiment obtains the text to be recognized, uses a multi-task model to quantize the text to be recognized according to the target quantization range to obtain the quantized data corresponding to the text to be recognized, and predicts the quantized data to obtain the prediction result corresponding to the text to be recognized. Therefore, based on the deep learning technology, the text to be recognized is predicted (such as classification, scoring, sorting, labeling, etc.), which can improve the accuracy and reliability of the prediction result.

[0266] In order to implement the above embodiments, the present disclosure also provides an electronic device, which may include at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the training method or prediction method of the multi-task model proposed in any of the above embodiments of the present disclosure.

[0267] In order to implement the above embodiments, the present disclosure also provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute the training method or prediction method of the multi-task model proposed in any of the above embodiments of the present disclosure.

[0268] In order to implement the above embodiments, the present disclosure also provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the training method or prediction method of the multi-task model proposed in any of the above embodiments of the present disclosure.

[0269] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0270] Fig.11 A schematic block diagram of an example electronic device that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0271] like Fig.11 As shown, the electronic device 1100 includes a computing unit 1101, which can perform various appropriate actions and processes according to a computer program stored in a ROM (Read-Only Memory) 1102 or a computer program loaded from a storage unit 1108 to a RAM (Random Access Memory) 1103. In the RAM 1103, various programs and data required for the operation of the electronic device 1100 can also be stored. The computing unit 1101, the ROM 1102, and the RAM 1103 are connected to each other via a bus 1104. An I / O (Input / Output) interface 1105 is also connected to the bus 1104.

[0272] Multiple components in the electronic device 1100 are connected to the I / O interface 1105, including: an input unit 1106, such as a keyboard, a mouse, etc.; an output unit 1107, such as various types of displays, speakers, etc.; a storage unit 1108, such as a disk, an optical disk, etc.; and a communication unit 1109, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 1109 allows the electronic device 1100 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0273] The computing unit 1101 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 1101 include, but are not limited to, CPU (Central Processing Unit), GPU (Graphic Processing Units), various dedicated AI (Artificial Intelligence) computing chips, various computing units running machine learning model algorithms, DSP (Digital Signal Processor), and any appropriate processor, controller, microcontroller, etc. The computing unit 1101 performs the various methods and processes described above, such as the training method or prediction method of the multi-task model described above. For example, in some embodiments, the training method or prediction method of the multi-task model described above may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 1108. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 1100 via ROM 1102 and / or communication unit 1109. When the computer program is loaded into RAM 1103 and executed by the computing unit 1101, one or more steps of the training method or prediction method of the multi-task model described above may be performed. Alternatively, in other embodiments, the computing unit 1101 may be configured to perform the training method or prediction method of the multi-task model described above in any other appropriate manner (e.g., by means of firmware).

[0274] Various embodiments of the systems and techniques described above herein may be implemented in digital electronic circuit systems, integrated circuit systems, FPGAs (Field Programmable Gate Arrays), ASICs (Application-Specific Integrated Circuits), ASSPs (Application Specific Standard Products), SOCs (System On Chips), CPLDs (Complex Programmable Logic Devices), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: being implemented in one or more computer programs that may be executed and / or interpreted on a programmable system including at least one programmable processor that may be a dedicated or general-purpose programmable processor that may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0275] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0276] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include electrical connections based on one or more lines, portable computer disks, hard disks, RAM, ROM, EPROM (Electrically Programmable Read-Only-Memory) or flash memory, optical fiber, CD-ROM (Compact Disc Read-Only Memory), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0277] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (Cathode-Ray Tube) or LCD (Liquid Crystal Display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0278] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: LAN (Local Area Network), WAN (Wide Area Network), the Internet, and blockchain networks.

[0279] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services (Virtual Private Server). The server may also be a server of a distributed system, or a server combined with a blockchain.

[0280] It should be noted that artificial intelligence is a discipline that studies how computers can simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, planning, etc.), and includes both hardware-level and software-level technologies. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, and big data processing; artificial intelligence software technologies mainly include computer vision technology, speech recognition technology, natural language processing technology, as well as machine learning / deep learning, big data processing technology, knowledge graph technology, and other major directions.

[0281] According to the technical solution of the embodiment of the present disclosure, by obtaining sample text sets corresponding to multiple tasks, wherein each sample text set is used to perform adaptation training on the multi-task model for the corresponding task; obtaining the reference quantization range corresponding to each sample text set; determining the target quantization range according to the reference quantization range corresponding to each sample text set; and training the multi-task model according to the target quantization range. Thus, the reference quantization ranges of the sample text sets corresponding to multiple tasks are combined to determine the target quantization range adopted by the multi-task model, so that the multi-task model quantizes the text according to the target quantization range, which can avoid the problem that the multi-task model uses the quantization range corresponding to a single sample text set to quantize the text corresponding to different tasks, resulting in poor quantization effect, that is, the quantization effect of the multi-task model can be improved, thereby improving the accuracy and reliability of the model prediction results.

[0282] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions proposed in this disclosure can be achieved, and this document does not limit this.

[0283] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A training method for a multi-task model, the method comprising: Acquire sample text sets corresponding to multiple tasks, wherein each of the sample text sets is used to perform adaptation training on the multi-task model for the corresponding task; Obtaining a reference quantization range corresponding to each of the sample text sets; Determining a target quantization range according to a reference quantization range corresponding to each of the sample text sets; Training the multi-task model according to the target quantization range; The obtaining of the reference quantization range corresponding to each of the sample text sets includes: For any sample text set in each of the sample text sets, determining a plurality of sample texts from the sample text set; Inputting the plurality of sample texts layer by layer into the multi-layer network layer in the multi-task model to obtain output matrices corresponding to the plurality of sample texts output by the network layer at each layer; Determine the reference quantization range of each layer of the network layer corresponding to any one of the sample text sets according to the output matrix corresponding to the multiple sample texts output by the network layer of each layer; The step of determining the target quantization range according to the reference quantization range corresponding to each of the sample text sets includes: Obtaining weights corresponding to each of the sample text sets; According to the weights corresponding to the sample text sets, weighted summation is performed on the reference quantization upper limits of the reference quantization range of the same network layer corresponding to the sample text sets, so as to obtain the target quantization upper limit of the target quantization range of the same network layer; According to the weights corresponding to the sample text sets, the reference quantization lower limits of the reference quantization ranges of the same network layer corresponding to the sample text sets are weighted summed to obtain the target quantization lower limit of the target quantization range of the same network layer.

2. The method according to claim 1, wherein: The training of the multi-task model according to the target quantization range includes: quantizing at least one sample text in each of the sample text sets according to the target quantization range using the multi-task model to obtain a quantization result corresponding to the at least one sample text; The multi-task model is trained according to the quantization result corresponding to the at least one sample text.

3. The method according to claim 2, wherein: The multi-task model includes multiple network layers, each of which has a corresponding target quantization range; The adopting the multi-task model to quantize at least one sample text in the plurality of sample text sets according to the target quantization range to obtain a quantization result corresponding to the at least one sample text includes: For any sample text in the at least one sample text, the first network layer in the multi-layer network layer is used to quantize the floating-point input data according to the target quantization range corresponding to the first network layer, so as to obtain the quantization result of the first network layer; wherein the input data is determined according to the feature vector of any training sample; For a non-first network layer in the multi-layer network layers, the non-first network layer is used to quantize a quantization result of a previous network layer of the non-first network layer according to a target quantization range corresponding to the non-first network layer, so as to obtain a quantization result of the non-first network layer.

4. The method according to claim 3, wherein: The training of the multi-task model according to the quantization result corresponding to the at least one sample text includes: Using the first network layer to perform matrix-vector operations on the feature vector of any sample text to obtain an output result of the first network layer; According to a first difference between a quantization result and an output result corresponding to the first network layer, a model parameter in the first network layer is adjusted.

5. The method according to claim 4, wherein: The adjusting the model parameters in the first network layer according to the first difference between the quantization result and the output result corresponding to the first network layer includes: Obtaining a first weight corresponding to a sample text set to which any training sample belongs; Generate a first value according to the first weight and the first difference; Determining the variance of each of the sample text sets; Generating a second value according to the variance corresponding to each of the sample text sets; Generate a loss function corresponding to the first network layer according to the first value and the second value; According to the loss function corresponding to the first network layer, the model parameters in the first network layer are adjusted.

6. The method according to claim 4, wherein: The training of the multi-task model according to the quantization result corresponding to the at least one sample text further includes: Using the non-first network layer to perform the matrix-vector operation on the output result of the previous network layer to obtain the output result of the non-first network layer; According to a second difference between the quantization result and the output result corresponding to the non-first network layer, the model parameters in the non-first network layer are adjusted.

7. The method according to claim 1, wherein: The step of inputting the plurality of sample texts layer by layer into the multi-layer network layers in the multi-task model to obtain output matrices corresponding to the plurality of sample texts output by the network layers at each layer includes: For any text among the multiple sample texts, use the first network layer among the multiple network layers to perform matrix-vector operations on the feature vector of the any text to obtain an output matrix corresponding to the any text output by the first network layer; A non-first network layer in the multi-layer network layers is used to perform matrix-vector operations on the output matrix of a previous network layer of the non-first network layer to obtain an output matrix corresponding to any text output by the non-first network layer.

8. The method according to claim 7, wherein: The step of determining the reference quantization range of each network layer corresponding to any one of the sample text sets according to the output matrices corresponding to the multiple sample texts output by the network layers of each layer comprises: For any network layer in the multi-layer network layers, determining the maximum element value in an output matrix corresponding to the plurality of sample texts output by the any network layer; A reference quantization range of any network layer corresponding to any sample text set is determined according to a maximum element value in an output matrix corresponding to the multiple sample texts.

9. The method according to claim 1, wherein: The obtaining of the weights corresponding to the sample text sets includes: Determining the total number of sample texts contained in each of the sample text sets; For any sample text set in each of the sample text sets, determining a first number of sample texts included in the sample text set; The weight of any one of the sample text sets is determined according to a ratio of the first number to the total number.

10. The method according to any one of claims 1 to 6, wherein: There are multiple multi-task models obtained through training, and the network parameters of the multiple multi-task models are different; The method further comprises: For any multi-task model among the plurality of multi-task models, determining a plurality of evaluation indicators of the any multi-task model; For any evaluation indicator among the multiple evaluation indicators, determining a correlation coefficient between the any evaluation indicator and the remaining evaluation indicators among the multiple evaluation indicators except the any evaluation indicator; Determining a target evaluation index corresponding to any one of the multi-task models according to a correlation coefficient between any one of the evaluation indexes and the remaining evaluation indexes; According to the target evaluation index corresponding to each of the multi-task models, a target multi-task model is determined from among the multi-task models.

11. The method according to claim 10, wherein: Determining the target evaluation index corresponding to any one of the multi-task models according to the correlation coefficient between any one of the evaluation indexes and the remaining evaluation indexes includes: Determining a target correlation coefficient according to the sum of the correlation coefficients between any one of the evaluation indicators and the remaining evaluation indicators; Determine the weight corresponding to any evaluation indicator according to the target correlation coefficient; wherein the weight is negatively correlated with the target correlation coefficient; According to the weights corresponding to the multiple evaluation indicators, the multiple evaluation indicators are weighted and summed to obtain the target evaluation indicator corresponding to any multi-task model.

12. A prediction method, comprising: Get the text to be recognized; The multi-task model trained by the multi-task model training method as described in any one of claims 1 to 11 quantizes the text to be recognized according to the target quantization range to obtain quantized data corresponding to the text to be recognized, and predicts the quantized data to obtain a prediction result corresponding to the text to be recognized.

13. The method according to claim 12, wherein: The multi-task model includes a multi-layer network layer and a prediction layer. The multi-task model is used to quantize the text to be recognized according to the target quantization range to obtain quantized data corresponding to the text to be recognized, and the quantized data is predicted to obtain a prediction result corresponding to the text to be recognized, including: The first network layer in the multi-layer network layer is used to quantize the floating point target data according to the target quantization range corresponding to the first network layer, so as to obtain the quantized data of the first network layer; wherein the target data is determined according to the feature vector of the text to be recognized; For a non-first network layer in the multiple network layers, quantizing the quantized data of a previous network layer of the non-first network layer using the non-first network layer according to a target quantization range corresponding to the non-first network layer, so as to obtain the quantized data of the non-first network layer; The prediction layer is used to predict the quantized data of the last network layer to obtain a prediction result corresponding to the text to be recognized.

14. A training device for a multi-task model, the device comprising: A first acquisition module is used to acquire sample text sets corresponding to multiple tasks, wherein each sample text set is used to perform adaptation training for a multi-task model for a corresponding task; A second acquisition module is used to acquire a reference quantization range corresponding to each of the sample text sets; A first determination module, configured to determine a target quantization range according to a reference quantization range corresponding to each of the sample text sets; A training module, used for training the multi-task model according to the target quantization range; Wherein, the second acquisition module includes: A first determining unit is used to determine a plurality of sample texts from any sample text set in each of the sample text sets; An input unit, used for inputting the plurality of sample texts layer by layer into the multi-layer network layer in the multi-task model, so as to obtain an output matrix corresponding to the plurality of sample texts output by each layer of the network layer; A second determining unit is used to determine the reference quantization range of each layer of the network layer corresponding to any one of the sample text sets according to the output matrix corresponding to the multiple sample texts output by the network layer of each layer; Wherein, the first determining module is specifically used to: Obtaining weights corresponding to each of the sample text sets; According to the weights corresponding to the sample text sets, weighted summation is performed on the reference quantization upper limits of the reference quantization range of the same network layer corresponding to the sample text sets, so as to obtain the target quantization upper limit of the target quantization range of the same network layer; According to the weights corresponding to the sample text sets, the reference quantization lower limits of the reference quantization ranges of the same network layer corresponding to the sample text sets are weighted summed to obtain the target quantization lower limit of the target quantization range of the same network layer.

15. The device according to claim 14, wherein: The training module comprises: A quantization unit, configured to quantize at least one sample text in each of the sample text sets according to the target quantization range using the multi-task model to obtain a quantization result corresponding to the at least one sample text; A training unit is used to train the multi-task model according to the quantization result corresponding to the at least one sample text.

16. The device according to claim 15, wherein: The multi-task model includes multiple network layers, each of which has a corresponding target quantization range; The quantization unit is specifically used for: For any sample text in the at least one sample text, the first network layer in the multi-layer network layer is used to quantize the floating-point input data according to the target quantization range corresponding to the first network layer, so as to obtain the quantization result of the first network layer; wherein the input data is determined according to the feature vector of any training sample; For a non-first network layer in the multi-layer network layers, the non-first network layer is used to quantize a quantization result of a previous network layer of the non-first network layer according to a target quantization range corresponding to the non-first network layer, so as to obtain a quantization result of the non-first network layer.

17. The device according to claim 16, wherein: The training unit is specifically used for: Using the first network layer to perform matrix-vector operations on the feature vector of any sample text to obtain an output result of the first network layer; According to a first difference between a quantization result and an output result corresponding to the first network layer, a model parameter in the first network layer is adjusted.

18. The device according to claim 17, wherein: The training unit is specifically used for: Obtaining a first weight corresponding to a sample text set to which any training sample belongs; Generate a first value according to the first weight and the first difference; Determining the variance of each of the sample text sets; Generating a second value according to the variance corresponding to each of the sample text sets; Generate a loss function corresponding to the first network layer according to the first value and the second value; According to the loss function corresponding to the first network layer, the model parameters in the first network layer are adjusted.

19. The device according to claim 17, wherein: The training unit is further used for: Using the non-first network layer to perform the matrix-vector operation on the output result of the previous network layer to obtain the output result of the non-first network layer; According to a second difference between the quantization result and the output result corresponding to the non-first network layer, the model parameters in the non-first network layer are adjusted.

20. The device according to claim 14, wherein: The input unit is specifically used for: For any text among the multiple sample texts, use the first network layer among the multiple network layers to perform matrix-vector operations on the feature vector of the any text to obtain an output matrix corresponding to the any text output by the first network layer; A non-first network layer in the multi-layer network layers is used to perform matrix-vector operations on the output matrix of a previous network layer of the non-first network layer to obtain an output matrix corresponding to any text output by the non-first network layer.

21. The device according to claim 20, wherein: The second determining unit is specifically configured to: For any network layer in the multi-layer network layers, determining the maximum element value in an output matrix corresponding to the plurality of sample texts output by the any network layer; A reference quantization range of any network layer corresponding to any sample text set is determined according to a maximum element value in an output matrix corresponding to the multiple sample texts.

22. The device according to claim 14, wherein: The first determining module is specifically configured to: Determining the total number of sample texts contained in each of the sample text sets; For any sample text set in each of the sample text sets, determining a first number of sample texts included in the sample text set; The weight of any one of the sample text sets is determined according to a ratio of the first number to the total number.

23. The device according to any one of claims 14 to 19, wherein: There are multiple multi-task models obtained through training, and the network parameters of the multiple multi-task models are different; The device further comprises: A second determination module is used to determine a plurality of evaluation indicators of any multi-task model among the plurality of multi-task models; A third determination module is used to determine, for any evaluation indicator among the multiple evaluation indicators, a correlation coefficient between the any evaluation indicator and the remaining evaluation indicators among the multiple evaluation indicators except the any evaluation indicator; A fourth determination module, used to determine the target evaluation index corresponding to any one of the multi-task models according to the correlation coefficient between any one of the evaluation indexes and the remaining evaluation indexes; The fifth determination module is used to determine a target multi-task model from each of the multi-task models according to the target evaluation index corresponding to each of the multi-task models.

24. The device according to claim 23, wherein: The fourth determining module is specifically used for: Determining a target correlation coefficient according to the sum of the correlation coefficients between any one of the evaluation indicators and the remaining evaluation indicators; Determine the weight corresponding to any evaluation indicator according to the target correlation coefficient; wherein the weight is negatively correlated with the target correlation coefficient; According to the weights corresponding to the multiple evaluation indicators, the multiple evaluation indicators are weighted and summed to obtain the target evaluation indicator corresponding to any multi-task model.

25. A prediction device, comprising: An acquisition module is used to acquire the text to be recognized; A prediction module, used to quantize the text to be recognized according to a target quantization range using a multi-task model trained using the multi-task model training method as described in any one of claims 1 to 11 to obtain quantized data corresponding to the text to be recognized, and to predict the quantized data to obtain a prediction result corresponding to the text to be recognized.

26. The device according to claim 25, wherein The multi-task model includes multiple network layers and prediction layers, and the prediction module is specifically used for: The first network layer in the multi-layer network layer is used to quantize the floating point target data according to the target quantization range corresponding to the first network layer, so as to obtain the quantized data of the first network layer; wherein the target data is determined according to the feature vector of the text to be recognized; For a non-first network layer in the multiple network layers, quantizing the quantized data of a previous network layer of the non-first network layer using the non-first network layer according to a target quantization range corresponding to the non-first network layer, so as to obtain the quantized data of the non-first network layer; The prediction layer is used to predict the quantized data of the last network layer to obtain a prediction result corresponding to the text to be recognized.

27. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method of any one of claims 1-11, or, alternatively, to execute the method of claim 12 or 13.

28. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 11, or, alternatively, to execute the method according to claim 12 or 13.

29. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the steps of the method according to any one of claims 1 to 11, or implements the steps of the method according to claim 12 or 13.

Images