Model miniaturization method, device, electronic device and computer-readable storage medium

Abstract

The present application provides a model miniaturization method, device, electronic device and computer-readable storage medium, wherein the method comprises: obtaining online application data generated by the teacher model to be miniaturized in the process of performing a variety of different segmentation tasks; wherein the online application data includes the input data and output data of the teacher model; determining the online application data generated by the teacher model in the process of performing the same segmentation task as the data set corresponding to the segmentation task; for each segmentation task, using the online application data included in the data set corresponding to the segmentation task to train the initial model, and obtaining a student model used only for performing the segmentation task. This method is conducive to ensuring the reasoning accuracy of the miniaturized student model.

Description

Technical Field

[0001] The present application relates to the technical field of model miniaturization, and in particular to a model miniaturization method, device, electronic device and computer-readable storage medium. Background Art

[0002] With the rapid development of deep learning and artificial intelligence technologies, large models (such as large cognitive models) have been widely used in various fields. Among them, large models refer to models with a large number of model parameters and large model files. However, such large models require more computer resources when running, resulting in high model operation costs and slow inference speed.

[0003] In order to solve the above problems, model miniaturization technology has emerged. Model miniaturization technology refers to reducing the size and computational complexity of large models by various means so that they can be deployed and run in an environment with limited computer resources. Model miniaturization technology usually includes pruning and quantization. Among them, pruning refers to reducing the size of large models by removing some unimportant connections or neurons in large models. These unimportant parts can be weights or neurons that contribute less to the final output. However, this model miniaturization technology will affect the accuracy of model reasoning after pruning because it removes some connections or neurons in large models. Quantization generally reduces the storage requirements and computational complexity of large models by reducing the number of bits of weights and activations in large models. Common quantization includes converting floating-point parameters into fixed-point numbers or low-bit-width floating-point numbers. This model miniaturization method will also affect the accuracy of model reasoning. Summary of the invention

[0004] In view of this, the purpose of the present application is to provide a model miniaturization method, device, electronic device and computer-readable storage medium, so as to ensure the reasoning accuracy of the miniaturized model after miniaturization of a large model.

[0005] In a first aspect, an embodiment of the present application provides a model miniaturization method, comprising:

[0006] Acquire online application data generated by the teacher model to be miniaturized during the execution of a variety of different segmentation tasks; wherein the online application data includes input data and output data of the teacher model;

[0007] Determine the online application data generated by the teacher model in the process of executing the same subdivision task as the data set corresponding to the subdivision task;

[0008] For each of the subdivided tasks, the initial model is trained using the online application data contained in the data set corresponding to the subdivided task to obtain a student model that is only used to perform the subdivided task.

[0009] In combination with the first aspect, the embodiment of the present application provides a first possible implementation of the first aspect, wherein the online application data generated by the teacher model in the process of executing the same subdivision task is determined as a data set corresponding to the subdivision task, including:

[0010] Classifying the online application data according to the output type to which the output data in each of the online application data belongs, so as to classify the online application data corresponding to the output data having the same output type into one category; wherein the output type includes a single-label output type, a multi-label output type and a text output type;

[0011] For the output data whose output type belongs to the single-label output type, determine an output result label set carried in each output data, so as to divide the online application data corresponding to the output data having the same output result label set into the same data set;

[0012] For the output data whose output type belongs to the multi-label output type, determining the information extraction items contained in each output data, and dividing the online application data corresponding to the output data having the same information extraction items into the same data set;

[0013] For each of the online application data corresponding to the output data whose output type belongs to the text output type, the online application data having the same task indication information is divided into the same data set according to the task indication information carried in the online application data.

[0014] In combination with the first aspect, the embodiment of the present application provides a second possible implementation of the first aspect, wherein, for each of the subdivided tasks, the initial model is trained using the online application data contained in the data set corresponding to the subdivided task to obtain a student model for performing the subdivided task, including:

[0015] For each data set corresponding to the subdivided task, determining whether the amount of online application data included in the data set is greater than a preset amount;

[0016] When the amount of online application data contained in the data set is greater than a preset amount, the data set corresponding to the subdivision task is divided according to a preset ratio to obtain a training set and a validation set;

[0017] Performing model training on the initial model using the online application data included in the training set to obtain a candidate student model for performing the segmentation task;

[0018] Using the validation set to perform model validation on the candidate student model to calculate a model evaluation index of the candidate student model;

[0019] When the model evaluation index meets the preset requirements, the candidate student model is determined as the student model of the subdivided task.

[0020] In combination with the second possible implementation of the first aspect, the embodiment of the present application provides a third possible implementation of the first aspect, wherein the using the online application data included in the training set to perform model training on the initial model includes:

[0021] For each of the online application data in the training set, use the input data in the online application data as the input sample of the initial model, and use the output data in the online application data as the sample label of the input sample;

[0022] Inputting the input sample into the initial model so that the initial model outputs an output sample corresponding to the input sample;

[0023] The loss of the initial model is calculated according to the output sample and the sample label corresponding to the input sample, so as to perform model training on the initial model using the loss.

[0024] In combination with the second possible implementation of the first aspect, the embodiment of the present application provides a fourth possible implementation of the first aspect, wherein the using the verification set to perform model verification on the candidate student model to calculate the model evaluation index of the candidate student model includes:

[0025] Determine the model evaluation index of each candidate student model for the task category to which the subdivided task performed by each candidate student model belongs; wherein the task category includes classification tasks, information extraction tasks and text tasks;

[0026] When the task category to which the subdivision task performed by the candidate student model belongs is any one of the classification task and the information extraction task, calculating any one or more of the following model evaluation indicators: accuracy, precision, recall, and F1 score;

[0027] When the task category to which the subdivision task performed by the candidate student model belongs is the text task, any one or more of the following model evaluation indicators are calculated: BLEU, ROUGE, and perplexity.

[0028] In combination with the first aspect, the embodiment of the present application provides a fifth possible implementation of the first aspect, wherein, after generating the student model, the method further includes:

[0029] Deploy the student model online;

[0030] When receiving a user request, based on the task tag carried in the user request, it is determined whether there is a target student model in the currently online student model that is only used to perform the subdivided task corresponding to the task tag;

[0031] When the target student model exists, the user request is routed to the target student model so that the target student model executes the user request.

[0032] In combination with the first aspect, the embodiment of the present application provides a sixth possible implementation of the first aspect, wherein the method further includes:

[0033] When the target student model does not exist, the user request is routed to the teacher model so that the teacher model executes the user request.

[0034] In a second aspect, the present application also provides a model miniaturization device, including:

[0035] An acquisition module, used to acquire online application data generated by the teacher model to be miniaturized in the process of executing a variety of different segmentation tasks; wherein the online application data includes input data and output data of the teacher model;

[0036] A determination module, used to determine the online application data generated by the teacher model in the process of executing the same subdivision task as a data set corresponding to the subdivision task;

[0037] The training module is used to train the initial model for each of the subdivided tasks using the online application data contained in the data set corresponding to the subdivided task, so as to obtain a student model that is only used to perform the subdivided task.

[0038] In combination with the second aspect, an embodiment of the present application provides a first possible implementation of the second aspect, wherein the determination module, when used to determine the online application data generated by the teacher model in the process of executing the same segmentation task as the data set corresponding to the segmentation task, is specifically used to:

[0039] Classifying the online application data according to the output type to which the output data in each of the online application data belongs, so as to classify the online application data corresponding to the output data having the same output type into one category; wherein the output type includes a single-label output type, a multi-label output type and a text output type;

[0040] For the output data whose output type belongs to the single-label output type, determine an output result label set carried in each output data, so as to divide the online application data corresponding to the output data having the same output result label set into the same data set;

[0041] For the output data whose output type belongs to the multi-label output type, determining the information extraction items contained in each output data, and dividing the online application data corresponding to the output data having the same information extraction items into the same data set;

[0042] For each of the online application data corresponding to the output data whose output type belongs to the text output type, the online application data having the same task indication information is divided into the same data set according to the task indication information carried in the online application data.

[0043] In combination with the second aspect, an embodiment of the present application provides a second possible implementation of the second aspect, wherein the training module, when used for each of the segmented tasks, uses the online application data contained in the data set corresponding to the segmented task to perform model training on the initial model to obtain a student model only for performing the segmented task, is specifically used to:

[0044] For each data set corresponding to the subdivided task, determining whether the amount of online application data included in the data set is greater than a preset amount;

[0045] When the amount of online application data contained in the data set is greater than a preset amount, the data set corresponding to the subdivision task is divided according to a preset ratio to obtain a training set and a validation set;

[0046] Performing model training on the initial model using the online application data included in the training set to obtain a candidate student model for performing the segmentation task;

[0047] Using the validation set to perform model validation on the candidate student model to calculate a model evaluation index of the candidate student model;

[0048] When the model evaluation index meets the preset requirements, the candidate student model is determined as the student model of the subdivided task.

[0049] In combination with the second possible implementation of the second aspect, the embodiment of the present application provides a third possible implementation of the second aspect, wherein the training module, when used to perform model training on the initial model using the online application data included in the training set, is specifically used to:

[0050] For each of the online application data in the training set, use the input data in the online application data as the input sample of the initial model, and use the output data in the online application data as the sample label of the input sample;

[0051] Inputting the input sample into the initial model so that the initial model outputs an output sample corresponding to the input sample;

[0052] The loss of the initial model is calculated according to the output sample and the sample label corresponding to the input sample, so as to perform model training on the initial model using the loss.

[0053] In combination with the second possible implementation of the second aspect, the embodiment of the present application provides a fourth possible implementation of the second aspect, wherein the training module, when used to perform model verification on the candidate student model using the verification set to calculate the model evaluation index of the candidate student model, is specifically used to:

[0054] Determine the model evaluation index of each candidate student model for the task category to which the subdivided task performed by each candidate student model belongs; wherein the task category includes classification tasks, information extraction tasks and text tasks;

[0055] When the task category of the subdivision task performed by the candidate student model is any one of the classification task and the information extraction task, calculating any one or more of the following model evaluation indicators: accuracy, precision, recall, and F1 score;

[0056] When the task category of the subdivided task performed by the candidate student model is the text task, any one or more of the following model evaluation indicators are calculated: BLEU, ROUGE, and perplexity.

[0057] In combination with the second possible implementation of the second aspect, the embodiment of the present application provides a fifth possible implementation of the second aspect, wherein the device further includes:

[0058] A deployment module, used to deploy the student model online;

[0059] A judgment module is used to, when receiving a user request, determine, based on the task tag carried in the user request, whether there is a target student model in the currently online student model that is only used to execute the subdivided task corresponding to the task tag;

[0060] The first routing module is used to route the user request to the target student model when the target student model exists, so that the target student model executes the user request.

[0061] In combination with the second possible implementation of the second aspect, the embodiment of the present application provides a sixth possible implementation of the second aspect, wherein the device further includes:

[0062] The second routing module is used to route the user request to the teacher model when the target student model does not exist, so that the teacher model executes the user request.

[0063] In a third aspect, an embodiment of the present application further provides an electronic device, comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate through the bus, and when the machine-readable instructions are executed by the processor, the steps in any possible implementation of the first aspect above are performed.

[0064] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps in any possible implementation of the first aspect described above are executed.

[0065] The embodiment of the present application provides a model miniaturization method, device, electronic device and computer-readable storage medium, wherein, considering that the model of the teacher model capable of performing a variety of different segmentation tasks is large and requires more computer resources during operation, in this embodiment, by obtaining the online application data generated by the teacher model in the process of performing a variety of different segmentation tasks, the online application data generated by the teacher model in the process of performing the same segmentation task is determined as the data set corresponding to the segmentation task; then, for each segmentation task, the initial model is trained using the online application data contained in the data set corresponding to the segmentation task to obtain a student model that is only used to perform the segmentation task. That is, in this embodiment, by automatically distilling the teacher model, a student model that is only used to perform one of the segmentation tasks is obtained, so the generated student model is smaller in size, faster in reasoning speed, and fewer computer resources are occupied during operation. At the same time, since the student model is trained using the online application data of the teacher model, it is conducive to ensuring the reasoning accuracy of the student model.

[0066] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0068] Figure 1 A flow chart of a model miniaturization method provided in an embodiment of the present application is shown;

[0069] Figure 2A flow chart showing another model miniaturization method provided in an embodiment of the present application is shown;

[0070] Figure 3 A schematic structural diagram of a model miniaturization device provided in an embodiment of the present application is shown;

[0071] Figure 4 A schematic structural diagram of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0072] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0073] In addition to pruning and quantization, model miniaturization technology also includes model distillation. Model distillation refers to approximating a trained large model (large model parameters and large model file) by training a smaller model. This large model is usually called a teacher model, while the small model (fewer model parameters and smaller model file) is called a student model. However, in the prior art, the large model that is subjected to model distillation usually refers to a model that is only used to perform one segmentation task. For example, the large model is only used to perform classification tasks, or only used to perform information extraction tasks. Therefore, there is an urgent need in the prior art for a method to distill a large model that can perform multiple segmentation tasks.

[0074] In view of the above problems, based on this, the embodiments of the present application provide a model miniaturization method, device, electronic device and computer-readable storage medium, so as to flexibly and automatically distill the large model according to the various different subdivided tasks performed by the large model, so as to optimize the performance of the small model obtained after distillation, which is described below through embodiments.

[0075] To facilitate understanding of this embodiment, a model miniaturization method disclosed in an embodiment of the present application is first introduced in detail. Figure 1 A flow chart of a model miniaturization method provided in an embodiment of the present application is shown. Figure 1 As shown, the following steps S101-S103 are included:

[0076] S101: Obtain online application data generated by the teacher model to be miniaturized during the execution of a variety of different segmentation tasks; wherein the online application data includes input data and output data of the teacher model.

[0077] In this embodiment, the teacher model is an AI model used to perform a variety of different segmentation tasks. For example, the teacher model can perform tasks such as news information classification, personal information extraction, and text translation. Since the teacher model can perform a variety of different segmentation tasks, the teacher model contains a large number of model parameters and a large model file.

[0078] The teacher model refers to a model that has been trained, deployed, and started running. When the teacher model is running, it takes up more computer resources, resulting in higher model running costs. At the same time, because the model file of the teacher model is large, the inference speed is slow when the teacher model is running.

[0079] Online application data refers to data generated in the process of using the teacher model. For example, the online application data includes input data input into the teacher model and output data output by the teacher model based on the input data.

[0080] In this embodiment, the online application data obtained is the data generated by the teacher model in the historical time period. The longer the teacher model runs (historical time period), the richer the online application data. The amount of online application data has an important impact on the effect of model distillation. A larger amount of data helps the student model learn more comprehensive knowledge, which is conducive to improving the performance of the learning model.

[0081] S102: Determine the online application data generated by the teacher model in the process of executing the same subdivision task as the data set corresponding to the subdivision task.

[0082] In this embodiment, since the teacher model can perform a variety of different segmentation tasks, the acquired online application data includes data generated when the teacher model performs various different segmentation tasks.

[0083] For each subdivision task performed by the teacher model, the online application data generated by the teacher model when performing the subdivision task is determined as the data set corresponding to the subdivision task. Each subdivision task corresponds to a data set.

[0084] S103: For each subdivision task, the initial model is trained using the online application data contained in the data set corresponding to the subdivision task to obtain a student model that is only used to perform the subdivision task.

[0085] In this embodiment, input data in the online application data is used as input samples for model training, and output data in the online application data is used as sample labels for model training, so as to perform model training on the initial model.

[0086] The initial model can be any of the following: DistillBERT, TinyBERT, MobileBERT. DistillBERT is a lightweight version of BERT, specifically designed for model distillation. TinyBERT is designed to perform BERT tasks in resource-constrained environments. MobileBERT, a version of BERT suitable for mobile devices, has fewer parameters. Lightweight structure refers to choosing a smaller model structure, such as reducing the number of layers and the number of hidden units in each layer.

[0087] In this embodiment, in order to facilitate full automation, the manual labeling process is omitted in this embodiment, and the output of the teacher model is used as the sample label, which constitutes a large number of data pairs, including input samples and sample labels.

[0088] In a possible implementation manner, when executing step S102, the following steps S1021-S1024 may be specifically performed:

[0089] S1021: Classify the online application data according to the output type to which the output data in each online application data belongs, so as to classify the online application data corresponding to the output data with the same output type into one category; wherein the output type includes a single-label output type, a multi-label output type and a text output type.

[0090] In this embodiment, all subdivided tasks that the teacher model can perform can be divided into three task categories, namely: classification tasks, information extraction tasks and text generation tasks. The classification task is characterized by text input and single-label output. The information extraction task is characterized by text input and multi-label output. The text generation task is characterized by text input and text output.

[0091] That is to say, when the segmentation task performed by the teacher model belongs to a classification task, the output type of the output data in the online application data generated by the teacher model in the process of performing the segmentation task is a single-label output type. When the segmentation task performed by the teacher model belongs to an information extraction task, the output type of the output data in the online application data generated by the teacher model in the process of performing the segmentation task is a multi-label output type. Similarly, when the segmentation task performed by the teacher model belongs to a text generation task, the output type of the output data in the online application data generated by the teacher model in the process of performing the segmentation task is a text output type.

[0092] Therefore, the online application data can be classified according to the output type of the output data in each online application data, so that each task category (classification task, information extraction task and text generation task) corresponds to a set containing multiple online application data. Since there are three task categories, all online application data are divided into three categories and three sets are obtained.

[0093] Among them, for the set corresponding to the same task category (for example, the set corresponding to the classification task), the output types of the output data in all online application data contained in the set are the same, that is, they are all output types corresponding to the task category (for example, they are all single-label output types).

[0094] S1022: For output data of a single-label output type, determine an output result label set carried in each output data, so as to classify online application data corresponding to the output data having the same output result label set into the same data set.

[0095] In this embodiment, the output data of the single-label output type refers to the output data output by the teacher model when the segmentation task performed by the teacher model belongs to the classification task. The classification task refers to classifying the input data into different categories, which can be binary classification (classifying the input data into one of two categories) or multi-class classification (classifying the input data into one of multiple categories).

[0096] Exemplarily, a news information classification task (a type of subdivision task) belongs to a classification task. The news information classification task specifically classifies the news information in news apps to determine which category the news information belongs to, such as current affairs, sports, education, entertainment, etc. The obtained category name (for example, one of the multiple categories such as current affairs, sports, education, entertainment, etc.) is used as the output data corresponding to the news information.

[0097] For example, the input data contained in online application data A1 is "xxx takes up xx position", and the output data is "current affairs". The input data contained in online application data A2 is "xxx athlete announces retirement", and the output data is "sports". The input data contained in online application data A3 is "xx Ministry issued a document stating that the number of people taking xx exam this year will exceed 10 million", and the output data is "education". The input data contained in online application data A4 is "xxx actor won xx award", and the output data is "entertainment".

[0098] In this embodiment, when the segmentation task performed by the teacher model is a classification task, the output data output by the teacher model carries an output result label set of the segmentation task; the output result label set is used to characterize all classification categories of the segmentation task.

[0099] Exemplarily, taking the news information classification task as an example, the output result label set of the news information classification task is "current affairs, sports, education, entertainment, finance, technology, and health".

[0100] For output data whose output type belongs to the single-label output type, for example, for the output data in the online application data A1 (i.e., for "current affairs"), determine the output result label set "current affairs, sports, education, entertainment, finance, science and technology, health" carried in "current affairs". Similarly, for the output data in the online application data A2 (i.e., for "sports"), determine the output result label set "current affairs, sports, education, entertainment, finance, science and technology, health" carried in "sports". For the output data contained in the online application data A3 (i.e., "education"), determine the output result label set "current affairs, sports, education, entertainment, finance, science and technology, health" carried in "education". And for the output data contained in the online application data A4 (i.e., "entertainment"), determine the output result label set "current affairs, sports, education, entertainment, finance, science and technology, health" carried in "entertainment".

[0101] After obtaining the output result label set carried in each output data, the online application data corresponding to the output data with the same output result label set are divided into the same data set. In this example, since the output data in the online application data A1, the output data in the online application data A2, the output data in the online application data A3, and the output data in the online application data A4 carry the same output result label set, the online application data A1, A2, A3, and A4 are divided into the same data set.

[0102] S1023: For output data of a multi-label output type, determine the information extraction items contained in each output data, and divide the online application data corresponding to the output data having the same information extraction items into the same data set.

[0103] In this embodiment, the output data of the multi-label output type refers to the output data output by the teacher model when the segmentation task performed by the teacher model belongs to the information extraction task. The information extraction task refers to extracting information corresponding to a preset information extraction item from the input data.

[0104] Exemplarily, a personal information extraction task belongs to the information extraction task, and the personal information extraction task specifically refers to extracting pre-set information extraction items (for example, name, date of birth) from input data.

[0105] For example, the input data of online application data B1 is "Zhang San, born in 1996, graduated from xx University", and the output data is "Name: Zhang San", "Date of Birth: 1996". The input data of online application data B2 is "Li Si was born in xx city in 2021", and the output data is "Name: Li Si", "Date of Birth: 2021".

[0106] In this embodiment, for output data of a multi-label output type, for example, for output data in online application data B1 ("Name: Zhang San", "Date of Birth: 1996"), the information extraction item "Name, Date of Birth" contained in the output data is determined. Similarly, for output data in online application data B2 ("Name: Li Si", "Date of Birth: 2021"), the information extraction item "Name, Date of Birth" contained in the output data is determined.

[0107] After obtaining the information extraction items contained in each output data of the multi-label output type, the output data with the same information extraction items are divided into the same data set. In this example, since the information extraction items contained in the output data in the online application data B1 and the output data in the online application data B2 are the same, the online application data B1 and B2 are divided into the same data set.

[0108] S1024: For the online application data corresponding to the output data whose output type belongs to the text output type, the online application data having the same task indication information is divided into the same data set according to the task indication information carried in the online application data.

[0109] In this embodiment, the output data of the text output type refers to the output data output by the teacher model when the subdivision task performed by the teacher model belongs to the text generation task. The text generation task includes any one or more of the following tasks: machine translation, question answering, reading comprehension, and writing.

[0110] Each online application data corresponding to the output data of the text output type carries task indication information. The task indication information is used to indicate the subdivided tasks that the teacher model needs to perform. For example, if the input data in the online application data C1 is "The weather is sunny today", and the task indication information it carries is "Translate into English", then the output data output by the teacher model is "The weather is sunny today".

[0111] In this embodiment, the subdivided task corresponding to the task indication information belongs to one of machine translation, question answering, reading comprehension, and writing.

[0112] In a possible implementation manner, when executing step S103, the following steps S1031-S1035 may be specifically performed:

[0113] S1031: For each data set corresponding to each subdivided task, determine whether the amount of online application data included in the data set is greater than a preset amount.

[0114] S1032: When the amount of online application data included in the data set is greater than a preset amount, the data set corresponding to the subdivided task is divided according to a preset ratio to obtain a training set and a validation set.

[0115] In order to ensure the performance of the trained student model, in this embodiment, when the amount of online application data included in the data set is greater than a preset amount, the data set is used to train the initial model. If the amount of online application data included in the data set is not greater than the preset amount, the data set is not used to train the initial model.

[0116] S1033: Use the online application data included in the training set to perform model training on the initial model to obtain a candidate student model for performing the segmentation task.

[0117] In a possible implementation manner, when executing step S1033, specifically:

[0118] For each online application data in the training set, the input data in the online application data is used as the input sample of the initial model, and the output data in the online application data is used as the sample label of the input sample;

[0119] Input the input sample into the initial model so that the initial model outputs the output sample corresponding to the input sample;

[0120] According to the output sample and the sample label corresponding to the input sample, the loss of the initial model is calculated to train the initial model using the loss.

[0121] In this embodiment, when the number of training times reaches a certain number threshold, the training is stopped, and the initial model obtained by the current training is used as the candidate student model. Alternatively, when the loss of the initial model is less than a certain loss threshold, the training is stopped, and the initial model obtained by the current training is used as the candidate student model.

[0122] S1034: Use the validation set to perform model validation on the candidate student model to calculate model evaluation indicators of the candidate student model.

[0123] In a possible implementation manner, when executing step S1034, specifically:

[0124] S10341: Determine the model evaluation index of each candidate student model according to the task category to which the subdivided task performed by the candidate student model belongs; wherein the task category includes classification tasks, information extraction tasks and text tasks.

[0125] S10342: When the task category of the subdivision task performed by the candidate student model is any one of the classification tasks and information extraction tasks, calculate any one or more of the following model evaluation indicators: accuracy, precision, recall rate, and F1 score.

[0126] In this embodiment, when the task category of the subdivided task performed by the candidate student model is any one of the classification tasks and information extraction tasks, the verification set of the candidate student model is determined according to the subdivided task of the candidate student model. For each online application data in the verification set, the input data in the online application data is used as the input verification sample of the candidate student model, and the output data in the online application data is used as the verification sample label of the input verification sample; the input verification sample is input into the candidate student model so that the candidate student model outputs the output verification sample corresponding to the input verification sample; according to the output verification sample and the verification sample label corresponding to the input verification sample, any one or more of the following model evaluation indicators are calculated: accuracy, precision, recall rate, F1 score.

[0127] S10343: When the task category of the segmented task performed by the candidate student model is a text task, calculate any one or more of the following model evaluation indicators: BLEU, ROUGE, and perplexity.

[0128] In this embodiment, when the task category of the subdivided task performed by the candidate student model is a text task, the verification set of the candidate student model is determined according to the subdivided task of the candidate student model. For each online application data in the verification set, the input data in the online application data is used as the input verification sample of the candidate student model, and the output data in the online application data is used as the verification sample label of the input verification sample; the input verification sample is input into the candidate student model so that the candidate student model outputs the output verification sample corresponding to the input verification sample; according to the output verification sample and the verification sample label corresponding to the input verification sample, any one or more of the following model evaluation indicators are calculated: BLEU (Bilingual Evaluation Understudy), ROUGE (Recall-Oriented Understudy for Gisting Evaluation), and perplexity.

[0129] S1035: When the model evaluation index meets the preset requirements, the candidate student model is determined as the student model of the segmented task.

[0130] In one possible implementation, Figure 2 A flow chart of another model miniaturization method provided in an embodiment of the present application is shown. Figure 2 As shown, after executing step S103, the following steps may also be performed:

[0131] S1041: Deploy the student model online.

[0132] S1042: When a user request is received, based on the task tag carried in the user request, it is determined whether there is a target student model in the currently online student model that is only used to execute the segmented task corresponding to the task tag.

[0133] In this embodiment, each sub-task corresponds to its own task label. If the task label carried in the user request is "translate into English", then the sub-task corresponding to the task label is the translate into English task.

[0134] S1043: When a target student model exists, routing the user request to the target student model so that the target student model executes the user request.

[0135] S1044: When the target student model does not exist, the user request is routed to the teacher model so that the teacher model executes the user request.

[0136] In this embodiment, since the student model is only used to perform one segmentation task, the number of parameters in the student model is small and the model file is small. The computer resources occupied by the student model during operation are also small. Therefore, using the student model instead of the teacher model to perform the corresponding segmentation task is conducive to reducing the occupation of computer resources, thereby reducing the operating cost.

[0137] Based on the same technical concept, the present application also provides a model miniaturization device, Figure 3 A schematic diagram of the structure of a model miniaturization device provided in an embodiment of the present application is shown. Figure 3 As shown, the device comprises:

[0138] An acquisition module 301 is used to acquire online application data generated by the teacher model to be miniaturized in the process of executing a variety of different segmentation tasks; wherein the online application data includes input data and output data of the teacher model;

[0139] A determination module 302 is used to determine the online application data generated by the teacher model in the process of executing the same subdivision task as a data set corresponding to the subdivision task;

[0140] The training module 303 is used to train the initial model for each of the subdivided tasks using the online application data contained in the data set corresponding to the subdivided task, so as to obtain a student model that is only used to perform the subdivided task.

[0141] Optionally, when the determination module 302 is used to determine the online application data generated by the teacher model in the process of executing the same subdivision task as the data set corresponding to the subdivision task, it is specifically used to:

[0142] Classifying the online application data according to the output type to which the output data in each of the online application data belongs, so as to classify the online application data corresponding to the output data having the same output type into one category; wherein the output type includes a single-label output type, a multi-label output type and a text output type;

[0143] For the output data whose output type belongs to the single-label output type, determine an output result label set carried in each output data, so as to divide the online application data corresponding to the output data having the same output result label set into the same data set;

[0144] For the output data whose output type belongs to the multi-label output type, determining the information extraction items contained in each output data, and dividing the online application data corresponding to the output data having the same information extraction items into the same data set;

[0145] For each of the online application data corresponding to the output data whose output type belongs to the text output type, the online application data having the same task indication information is divided into the same data set according to the task indication information carried in the online application data.

[0146] Optionally, when the training module 303 is used to perform model training on the initial model for each of the segmented tasks using the online application data contained in the data set corresponding to the segmented task to obtain a student model used only for performing the segmented task, it is specifically used to:

[0147] For each data set corresponding to the subdivided task, determining whether the amount of online application data included in the data set is greater than a preset amount;

[0148] When the amount of online application data contained in the data set is greater than a preset amount, the data set corresponding to the subdivision task is divided according to a preset ratio to obtain a training set and a validation set;

[0149] Performing model training on the initial model using the online application data included in the training set to obtain a candidate student model for performing the segmentation task;

[0150] Using the validation set to perform model validation on the candidate student model to calculate a model evaluation index of the candidate student model;

[0151] When the model evaluation index meets the preset requirements, the candidate student model is determined as the student model of the subdivided task.

[0152] Optionally, when the training module 303 is used to perform model training on the initial model using the online application data included in the training set, it is specifically used to:

[0153] For each of the online application data in the training set, use the input data in the online application data as the input sample of the initial model, and use the output data in the online application data as the sample label of the input sample;

[0154] Inputting the input sample into the initial model so that the initial model outputs an output sample corresponding to the input sample;

[0155] The loss of the initial model is calculated according to the output sample and the sample label corresponding to the input sample, so as to perform model training on the initial model using the loss.

[0156] Optionally, when the training module is used to perform model verification on the candidate student model using the verification set to calculate a model evaluation index of the candidate student model, it is specifically used to:

[0157] Determine the model evaluation index of each candidate student model for the task category to which the subdivided task performed by each candidate student model belongs; wherein the task category includes classification tasks, information extraction tasks and text tasks;

[0158] When the task category of the subdivision task performed by the candidate student model is any one of the classification task and the information extraction task, calculating any one or more of the following model evaluation indicators: accuracy, precision, recall, and F1 score;

[0159] When the task category of the subdivided task performed by the candidate student model is the text task, any one or more of the following model evaluation indicators are calculated: BLEU, ROUGE, and perplexity.

[0160] Optionally, the device further comprises:

[0161] A deployment module, used to deploy the student model online;

[0162] A judgment module is used to, when receiving a user request, determine, based on the task tag carried in the user request, whether there is a target student model in the currently online student model that is only used to execute the subdivided task corresponding to the task tag;

[0163] The first routing module is used to route the user request to the target student model when the target student model exists, so that the target student model executes the user request.

[0164] Optionally, the device further comprises:

[0165] The second routing module is used to route the user request to the teacher model when the target student model does not exist, so that the teacher model executes the user request.

[0166] Figure 4 A structural diagram of an electronic device provided for an embodiment of the present application includes: a processor 401, a memory 402 and a bus 403, wherein the memory 402 stores machine-readable instructions executable by the processor 401. When the electronic device runs the above-mentioned information processing method, the processor 401 communicates with the memory 402 through the bus 403, and the processor 401 executes the machine-readable instructions to perform the method steps described in Example 1.

[0167] Embodiment 4 of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method steps described in Embodiment 1 are executed.

[0168] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices, electronic devices, and computer-readable storage media can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0169] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation. For example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some communication interface, device or unit, which can be electrical, mechanical or other forms.

[0170] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0171] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0172] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application can essentially be embodied in the form of a software product, or in other words, the part that contributes to the prior art or the part of the technical solution. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0173] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solutions of the present application, rather than to limit them. The protection scope of the present application is not limited thereto. Although the present application is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed in the present application, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A model miniaturization method, characterized in that: include: Acquire online application data generated by the teacher model to be miniaturized during the execution of a variety of different segmentation tasks; wherein the online application data includes input data and output data of the teacher model; Determine the online application data generated by the teacher model in the process of executing the same subdivision task as the data set corresponding to the subdivision task; For each of the subdivided tasks, the initial model is trained using the online application data contained in the data set corresponding to the subdivided task to obtain a student model that is only used to perform the subdivided task; The using the online application data contained in the data set corresponding to the segmentation task to perform model training on the initial model includes: The input data in the online application data is used as the input sample of the model training, and the output data in the online application data is used as the sample label of the model training to train the initial model; The step of determining the online application data generated by the teacher model in the process of executing the same subdivision task as a data set corresponding to the subdivision task includes: Classifying the online application data according to the output type to which the output data in each of the online application data belongs, so as to classify the online application data corresponding to the output data having the same output type into one category; wherein the output type includes a single-label output type, a multi-label output type and a text output type; For the output data whose output type belongs to the single-label output type, determine an output result label set carried in each output data, so as to divide the online application data corresponding to the output data having the same output result label set into the same data set; For the output data whose output type belongs to the multi-label output type, determining the information extraction items contained in each output data, and dividing the online application data corresponding to the output data having the same information extraction items into the same data set; For each of the online application data corresponding to the output data whose output type belongs to the text output type, the online application data having the same task indication information is divided into the same data set according to the task indication information carried in the online application data.

2. The method according to claim 1, characterized in that: For each of the subdivision tasks, the initial model is trained using the online application data contained in the data set corresponding to the subdivision task to obtain a student model for performing the subdivision task, including: For each data set corresponding to the subdivided task, determining whether the amount of online application data included in the data set is greater than a preset amount; When the amount of online application data contained in the data set is greater than a preset amount, the data set corresponding to the subdivision task is divided according to a preset ratio to obtain a training set and a validation set; Performing model training on the initial model using the online application data included in the training set to obtain a candidate student model for performing the segmentation task; Using the validation set to perform model validation on the candidate student model to calculate a model evaluation index of the candidate student model; When the model evaluation index meets the preset requirements, the candidate student model is determined as the student model of the subdivided task.

3. The method according to claim 2, characterized in that: The using the online application data included in the training set to perform model training on the initial model includes: For each of the online application data in the training set, use the input data in the online application data as the input sample of the initial model, and use the output data in the online application data as the sample label of the input sample; Inputting the input sample into the initial model so that the initial model outputs an output sample corresponding to the input sample; The loss of the initial model is calculated according to the output sample and the sample label corresponding to the input sample, so as to perform model training on the initial model using the loss.

4. The method according to claim 2, characterized in that: The using the validation set to perform model validation on the candidate student model to calculate a model evaluation index of the candidate student model includes: Determine the model evaluation index of each candidate student model for the task category to which the subdivided task performed by each candidate student model belongs; wherein the task category includes classification tasks, information extraction tasks and text tasks; When the task category of the subdivision task performed by the candidate student model is any one of the classification task and the information extraction task, calculating any one or more of the following model evaluation indicators: accuracy, precision, recall, and F1 score; When the task category of the subdivided task performed by the candidate student model is the text task, any one or more of the following model evaluation indicators are calculated: BLEU, ROUGE, and perplexity.

5. The method according to claim 1, characterized in that: After generating the student model, the method further comprises: Deploy the student model online; When receiving a user request, based on the task tag carried in the user request, it is determined whether there is a target student model in the currently online student model that is only used to perform the subdivided task corresponding to the task tag; When the target student model exists, the user request is routed to the target student model so that the target student model executes the user request.

6. The method according to claim 5, characterized in that: The method further comprises: When the target student model does not exist, the user request is routed to the teacher model so that the teacher model executes the user request.

7. A model miniaturization device, characterized in that: include: An acquisition module, used to acquire online application data generated by the teacher model to be miniaturized in the process of executing a variety of different segmentation tasks; wherein the online application data includes input data and output data of the teacher model; A determination module, used to determine the online application data generated by the teacher model in the process of executing the same subdivision task as a data set corresponding to the subdivision task; A training module is used to train the initial model for each of the subdivided tasks using the online application data contained in the data set corresponding to the subdivided task, so as to obtain a student model used only for executing the subdivided task; When the training module is used to perform model training on the initial model using the online application data included in the data set corresponding to the segmentation task, it is specifically used to: The input data in the online application data is used as the input sample of the model training, and the output data in the online application data is used as the sample label of the model training to train the initial model; When the determination module is used to determine the online application data generated by the teacher model in the process of executing the same subdivision task as the data set corresponding to the subdivision task, it is specifically used to: Classifying the online application data according to the output type to which the output data in each of the online application data belongs, so as to classify the online application data corresponding to the output data having the same output type into one category; wherein the output type includes a single-label output type, a multi-label output type and a text output type; For the output data whose output type belongs to the single-label output type, determine an output result label set carried in each output data, so as to divide the online application data corresponding to the output data having the same output result label set into the same data set; For the output data whose output type belongs to the multi-label output type, determining the information extraction items contained in each output data, and dividing the online application data corresponding to the output data having the same information extraction items into the same data set; For each of the online application data corresponding to the output data whose output type belongs to the text output type, the online application data having the same task indication information is divided into the same data set according to the task indication information carried in the online application data.

8. An electronic device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, the steps of the method as described in any one of claims 1 to 6 are performed.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are executed.

Images