Model processing method, data classification method, device, medium, equipment and program product

Abstract

The present disclosure relates to the field of data processing technology, and provides a model processing method, a data classification method, an apparatus, a medium, a device and a program product, including: using a first sample set to train a first model to obtain a second model; the first sample set includes a first sample subset and a second sample subset, the first sample subset includes samples of a first category, the second sample subset includes samples of a second category, and the number of samples in the first sample subset is less than the number of samples in the second sample subset; using the second sample set to train a second model to obtain a third model; the second sample set includes a third sample subset and a fourth sample subset, the category of samples included in the third sample subset is the same as that of the first category, the category of samples included in the fourth sample subset is the same as that of the second category, and the number of samples in the third sample subset is the same as that of the fourth sample subset. The method can mitigate the long tail effect, so that the third model has a higher accuracy rate.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular, to a model processing method, a data classification method, an apparatus, a medium, a device, and a program product. Background Art

[0002] With the rapid development of artificial intelligence technology, machine learning and deep learning have achieved remarkable results in many artificial intelligence fields. In traditional classification and recognition tasks, naturally collected training sample sets often show a long-tail distribution, that is, most of the samples are concentrated in a few categories, while the remaining small part of the samples are distributed in a large number of other categories.

[0003] In the related art, if a training classification model with long-tail distribution characteristics is directly used, the classification model will tend to be more biased towards a few categories with a larger number and perform poorly for categories with a smaller number, which will lead to low overall accuracy of the classification model. Summary of the invention

[0004] The purpose of the present disclosure is to provide a model processing method, data classification method, device, medium, equipment and program product.

[0005] In a first aspect, an embodiment of the present disclosure provides a model processing method, the model processing method comprising: Performing first model training on a first model using a first sample set to obtain a second model; the first sample set includes a first sample subset and a second sample subset, the first sample subset includes samples of a first category, the second sample subset includes samples of a second category, and the number of samples in the first sample subset is less than the number of samples in the second sample subset; The second model is trained on the second model using the second sample set to obtain a third model; the second sample set includes a third sample subset and a fourth sample subset, the category of the samples included in the third sample subset is the same as the first category, the category of the samples included in the fourth sample subset is the same as the second category, and the number of samples in the third sample subset is the same as the number of samples in the fourth sample subset.

[0006] In a second aspect, an embodiment of the present disclosure provides a data classification method, the data classification method comprising: Acquire conversation data between the first user and the second user; The conversation data is input into a classification model to obtain an intent category output by the classification model, wherein the classification model is obtained according to the model processing method provided in any one of the first aspects above.

[0007] In a third aspect, an embodiment of the present disclosure provides a model processing device, the model processing device comprising: A first processing module is configured to perform first model training on a first model using a first sample set to obtain a second model; the first sample set includes a first sample subset and a second sample subset, the first sample subset includes samples of a first category, the second sample subset includes samples of a second category, and the number of samples in the first sample subset is less than the number of samples in the second sample subset; The second processing module is configured to use the second sample set to perform a second model training on the second model to obtain a third model; the second sample set includes a third sample subset and a fourth sample subset, the category of the samples included in the third sample subset is the same as the first category, the category of the samples included in the fourth sample subset is the same as the second category, and the number of samples in the third sample subset is the same as the number of samples in the fourth sample subset.

[0008] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any model processing method provided in the first aspect.

[0009] In a fifth aspect, an embodiment of the present disclosure provides an electronic device, including: a memory having a computer program stored thereon; A processor is used to execute the computer program in the memory to implement the steps of the model processing method provided in any one of the first aspects.

[0010] In a sixth aspect, an embodiment of the present disclosure provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of any model processing method provided in the first aspect.

[0011] It is worth noting that the model processing method performs the first model training and the second model training in stages. In the first model training, the basic features of the first category are learned based on the samples of the first sample subset, and the basic features of the second category are learned based on the samples of the second sample subset. Then, in the second model training, the training is performed based on the same number of samples of the third sample subset (the same as the first category) and the fourth sample subset (the same as the second category). That is, to increase the proportion of the number of samples of the first category in the total number of training samples in the entire model training, and to reduce the model's preference for categories with a larger number of samples (head categories, such as the second category) during the training process, which helps the model to better learn the features of categories with a smaller number of samples (tail categories, such as the first category), thereby improving the recognition ability of the tail categories, and then mitigating the long-tail effect, so that the third model obtained has a higher accuracy rate.

[0012] Other features and advantages of the present disclosure will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The accompanying drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the following specific embodiments, they are used to explain the present disclosure but do not constitute a limitation of the present disclosure. In the accompanying drawings: Figure 1 The figure is a flowchart of a model processing method according to an exemplary embodiment.

[0014] Figure 2 It is a schematic diagram of a model processing method according to an exemplary embodiment.

[0015] Figure 3 It is a block diagram of a model processing device according to an exemplary embodiment.

[0016] Figure 4 It is a block diagram of an electronic device according to an exemplary embodiment.

[0017] Figure 5 is a block diagram of another electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0018] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0019] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0020] The term "including" and its variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0021] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0022] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0023] With the rapid development of artificial intelligence technology, machine learning and deep learning have achieved remarkable results in many artificial intelligence fields. In traditional classification and recognition tasks, naturally collected training sample sets often show a long-tail distribution, that is, most of the samples are concentrated in a few categories, while the remaining small part of the samples are distributed in a large number of other categories.

[0024] Long-tail problems are very common in the real world. For example, in natural language processing, common words appear much more frequently than uncommon words; in image recognition, the number of pictures of common objects is also much greater than that of rare objects.

[0025] In the related art, if a training classification model with long-tail distribution characteristics is directly used, the classification model will tend to be more biased towards a few categories with a larger number and perform poorly for categories with a smaller number, which will lead to low overall accuracy of the classification model.

[0026] In customer service dialogue scenarios based on intent recognition, it is necessary to formulate a response strategy based on user intent. The inventors found that the bidirectional transformer model (Bidirectional Encoder Representations from Transformers, Bert) achieved good results on public academic datasets. However, it is very difficult to obtain high-quality intent recognition training datasets in actual scenarios. On the one hand, randomly sampled intent data are usually uneven long-tail distribution intents. On the other hand, the tail category sample data obtained based on keyword sampling lacks diversity. Based on the sampling scheme, you can choose to undersample the head category or resample the tail category. The undersampling scheme does not make full use of manually annotated data, and the resampling scheme model is prone to overfitting when repeatedly learning the same sample. There are usually many similar intents in actual scenarios, and sample-level data augmentation may change the intent category of the sample, thereby introducing label noise.

[0027] In order to solve the above problems, the model processing method is divided into stages of first model training and second model training. In the first model training, the basic features of the first category are learned based on the samples of the first sample subset, and the basic features of the second category are learned based on the samples of the second sample subset. Then, in the second model training, the same number of samples of the third sample subset (the same as the first category) and the fourth sample subset (the same as the second category) are used for training. That is, in order to increase the proportion of the number of samples of the first category in the total number of training samples in the entire model training, and reduce the model's preference for categories with a larger number of samples (head categories, such as the second category) during the training process, it helps the model to better learn the features of categories with a smaller number of samples (tail categories, such as the first category), thereby improving the recognition ability of the tail categories, and then mitigating the long tail effect, so that the third model has a higher accuracy rate.

[0028] Figure 1 is a flowchart of a model processing method according to an exemplary embodiment. Figure 2 is a schematic diagram of a model processing method according to an exemplary embodiment. The model processing method can be applied to electronic devices. Figure 1 and Figure 2 , the model processing method may include step S1 and step S2.

[0029] Step S1, using a first sample set to perform a first model training on a first model to obtain a second model.

[0030] The first sample set includes a first sample subset and a second sample subset. The first sample subset includes samples of a first category, the second sample subset includes samples of a second category, and the number of samples in the first sample subset is smaller than the number of samples in the second sample subset.

[0031] The samples in each sample subset correspond to a category. The samples in the first sample subset all correspond to the first category, and the samples in the second sample subset all correspond to the second category. The categories corresponding to the samples in different sample subsets are different.

[0032] In other embodiments, the first sample set may further include more sample subsets, for example, a third sample subset, a fourth sample subset, a fifth sample subset, a sixth sample subset, ... the sample subsets correspond to the third category, the fourth category, the fifth category, the sixth category, ... respectively. The first sample subset is extracted from the samples of the first category of the total sample set according to a preset extraction ratio, and the second sample subset is extracted from the samples of the second category of the total sample set according to a preset extraction ratio. The total sample set includes training samples of multiple categories (e.g., the first category, the second category, etc.) of different numbers.

[0033] For example, the total set of samples based on intent recognition can be as shown in Table 1.

[0034]

[0035] Table 1 The total sample set includes 7 intent categories, namely, “apply for refund”, “apply for loan”, “check repayment amount”, “consult members”, “check interest”, “check repayment period”, and “want to pay in installments”, and their corresponding quantities are 5000, 5000, 4000, 3000, 1000, 1000, and 500 respectively.

[0036] The first sample set includes training samples of multiple categories with the same extraction ratio. Samples are extracted from each category in the total sample set according to a preset extraction ratio to obtain a sample subset corresponding to each category. That is, the first sample subset is extracted from the samples of the first category of the total sample set according to the preset extraction ratio, and the second sample subset is extracted from the samples of the second category of the total sample set according to the preset extraction ratio.

[0037] For example, if the preset extraction ratio is 50%, the categories corresponding to the sample subsets in the first sample set are: "Apply for a refund", "Apply for a loan", "Check the repayment amount", "Consult a member", "Check interest", "Check the number of repayment periods", and "Want to pay in installments", and the corresponding quantities are 2500, 2500, 2000, 1500, 500, 500, and 250 respectively.

[0038] For example, if the preset extraction ratio is 100%, then the categories corresponding to the sample subsets in the first sample set are: "Apply for refund", "Apply for loan", "Check repayment amount", "Consult members", "Check interest", "Check repayment period", and "Want to pay in installments", and the corresponding quantities are 5000, 5000, 4000, 3000, 1000, 1000, and 500 respectively.

[0039] It should be understood that the larger the preset extraction ratio is set, the higher the accuracy of the second model obtained through training.

[0040] Using the first sample set to train the first model to obtain the second model can be understood as training the first model for multiple rounds using the first sample set until the first training stop condition is met to obtain the second model.

[0041] For different rounds of training, the same first sample set may be used, or samples may be re-extracted from the total sample set based on the same extraction rule (as the first sample set) to form a new first sample set, which is not limited in this embodiment.

[0042] The first training stop condition can be set according to actual conditions. For example, the first training stop condition can be that the number of iterations of the first model reaches the first set number of times, the first training stop condition can also be that the loss function of the first model converges or is less than the first set value, and the first training stop condition can also be that the training time of the first model reaches the first set time, which is not limited in this embodiment.

[0043] Through the sampling method of sample balance (ie, the same preset extraction ratio), the second model obtained by training the first sample set obtained based on sample balanced sampling has a stronger feature extraction capability.

[0044] Step S2: Use the second sample set to train the second model to obtain a third model.

[0045] The second sample set includes a third sample subset and a fourth sample subset. The category of samples included in the third sample subset is the same as the first category. The category of samples included in the fourth sample subset is the same as the second category. The number of samples in the third sample subset is the same as the number of samples in the fourth sample subset.

[0046] The samples in the third sample subset all correspond to the first category, and the samples in the fourth sample subset all correspond to the second category.

[0047] The second sample set includes training samples of multiple categories with the same number of samples. Samples are extracted from each category in the total sample set according to a preset number of samples to obtain a sample subset corresponding to each category. That is, the third sample subset is extracted from the samples of the first category of the total sample set according to the preset number of samples, and the fourth sample subset is extracted from the samples of the second category of the total sample set according to the preset number of samples. That is, the number of samples in the third sample subset is the same as the number of samples in the fourth sample subset, and both are preset numbers of samples.

[0048] For different rounds of training, the same second sample set may be used, or samples may be re-extracted from the total sample set based on the same extraction rule (as the second sample set) to form a new second sample set, which is not limited in this embodiment.

[0049] It should be understood that when the third sample subset and the fourth sample subset in the second sample set are both extracted from the total sample set, the upper limit of the preset extraction quantity is the number of samples corresponding to the first category (tail category, i.e., the category with a smaller number) in the total sample set.

[0050] Please continue to refer to Table 1. The category with the least number of samples in the total sample set is "want to pay in installments", and its corresponding number is 500, which means that the upper limit of the preset number of draws is 500. The preset number of draws can be set to 400, that is, the categories corresponding to the sample subsets included in the second sample set are: "apply for refund", "apply for loan", "check repayment amount", "consult members", "check interest", "check repayment period", and "want to pay in installments", and the corresponding numbers are 400, 400, 400, 400, 400, 400, 400, 400, and 400 respectively.

[0051] In other embodiments, the third sample subset may be extracted from samples of the first category of other sample sets according to a preset number of extractions, and the fourth sample subset may be extracted from samples of the second category of other sample sets according to a preset number of extractions.

[0052] It should be understood that other total sample sets also include samples of the first category and samples of the second category, but the other total sample sets are sample sets different from the total sample set.

[0053] Using the second sample set to train the second model to obtain the third model can be understood as training the second model for multiple rounds using the second sample set until the second training stop condition is met to obtain the third model.

[0054] The second training stop condition can be set according to actual conditions. For example, the second training stop condition can be that the number of iterations of the second model reaches the second set number, the second training stop condition can also be that the loss function of the second model converges or is less than the second set value, and the second training stop condition can also be that the training time of the second model reaches the second set time, which is not limited in this embodiment.

[0055] Through the sampling method of class balance (ie, the same preset number of extractions), the third model trained on the second sample set obtained by class balanced sampling has a stronger classification generalization ability.

[0056] The third model obtained after training can be used to process classification tasks of input data. The classification tasks can be, but are not limited to, image classification, text classification, speech recognition, intent recognition, etc.

[0057] It should be understood that by dividing the model processing method into stages of first model training and second model training, in the first model training, the basic features of the first category are learned based on the samples of the first sample subset, and the basic features of the second category are learned based on the samples of the second sample subset. Then, in the second model training, training is performed based on the same number of samples of the third sample subset (the same as the first category) and the fourth sample subset (the same as the second category). Whether the third sample set and the fourth sample set are extracted from the total sample set or other total sample sets, they can increase the proportion of the number of samples of the first category in the total number of training samples in the entire model training, reduce the model's preference for categories with a larger number of samples (head categories, such as the second category) during the training process, and help the model better learn the features of categories with a smaller number of samples (tail categories, such as the first category), thereby improving the recognition ability of the tail categories, and then mitigating the long tail effect, so that the third model obtained has a higher accuracy rate.

[0058] In one embodiment, the first model training may include: extracting features of samples included in the first sample set to obtain first sample features; performing classification prediction based on the first sample features to obtain a first prediction category; determining a first loss value of the first model based on the first prediction category; and adjusting parameters of the first model based on the first loss value.

[0059] The first model includes a representation layer and a classification layer, wherein the representation layer is used to extract features of input data, and the classification layer is used to make a final classification decision based on the features extracted by the representation layer. The representation layer may include an embedding layer and a multi-layer transformer layer. In other embodiments, the representation layer may also be a convolutional neural network (CNN), a long short-term memory network (LSTM), an enhanced representation model through knowledge integration (Ernie), a robustly optimized BERT pretraining approach (RoBERTa), etc. The classification layer may include a linear layer and a softmax activation function. In other embodiments, the classification layer may also be a multi-layer perceptron model, etc.

[0060] The first sample feature may include features of samples in the first sample subset or features of samples in the second sample subset.

[0061] The first predicted category may be the category to which the sample most likely belongs, output by the first model according to the first sample feature.

[0062] According to the difference between the first predicted category and the true category corresponding to the sample, a first loss value of the first model is obtained.

[0063] Parameters of a representation layer and parameters of a classification layer in the first model are adjusted according to the first loss value.

[0064] The first model is trained by using the first sample set obtained based on sample balanced sampling, so that the trained second model has a stronger feature extraction capability.

[0065] In one embodiment, the second model training may include: extracting features of samples included in the second sample set to obtain second sample features; performing classification prediction based on the second sample features to obtain a second prediction category; determining a second loss value of the second model based on the second prediction category; and adjusting parameters of the second model based on the second loss value.

[0066] The second model includes a representation layer and a classification layer, wherein the representation layer is used to extract features of the input data, and the classification layer is used to make a final classification decision based on the features extracted by the representation layer. The representation layer may include an embedding layer and a multi-layer transformer layer. In other embodiments, the representation layer may also be CNN, LSTM, Ernie, RoBERTa, etc. The classification layer may include a Linear layer and a softmax activation function. In other embodiments, the classification layer may also be a multi-layer perceptron model, etc.

[0067] The second sample feature may include features of samples in the third sample subset or features of samples in the fourth sample subset.

[0068] The second predicted category may be the category to which the sample most likely belongs, output by the second model according to the second sample feature.

[0069] According to the difference between the second predicted category and the true category corresponding to the sample, a second loss value of the second model is obtained.

[0070] Parameters of the representation layer and parameters of the classification layer in the second model are adjusted according to the second loss value.

[0071] The second model is trained by using the second sample set obtained based on category balanced sampling, so that the trained third model has stronger classification generalization ability.

[0072] Through the staged first model training and second model training, different characteristics of the category are learned during the first model training, and the number of samples is balanced during the second model training to further optimize the model performance and reduce the negative impact of the long tail effect of the number of samples.

[0073] In one embodiment, the second model includes a representation layer and a classification layer, the parameters of the representation layer include a first learning rate parameter and a first weight parameter, and the parameters of the classification layer include a second learning rate parameter and a second weight parameter.

[0074] Adjusting parameters of the second model based on the second loss value may include: Adjusting a first weight parameter of a representation layer of the second model based on the second loss value and the first learning rate parameter; Based on the second loss value and the second learning rate parameter, adjust the second weight parameter of the classification layer of the second model; wherein the second learning rate parameter is greater than the first learning rate parameter.

[0075] The learning rate is a hyperparameter used to adjust model parameters in deep learning. The learning rate determines the magnitude of parameter (e.g., weight parameter) updates in each iteration (or gradient descent step). A higher learning rate means a larger step size for parameter updates, while a lower learning rate means a smaller step size.

[0076] Based on the second loss value and the first learning rate parameter, the first weight parameter of the representation layer is adjusted. It can be understood that during the training process, the model will update the weight of the representation layer according to the second loss value and the first learning rate. Since the representation layer is closer to the input and the representation layer of the model has been trained during the first model training process, more detailed adjustments (i.e., smaller learning rates) are usually required to avoid overfitting and ensure the stability of feature extraction.

[0077] Based on the second loss value and the second learning rate parameter, the second weight parameter of the classification layer is adjusted. It can be understood that during the training process, the model will update the weight of the classification layer according to the second loss value and the second learning rate. Since this layer is closer to the output, it usually requires a larger adjustment (i.e., a larger learning rate) so that the model can learn how to map features to the correct category faster.

[0078] The classification layer is directly related to the final prediction result. By setting the second learning rate of the classification layer of the second model to be greater than the first learning rate of the representation layer of the second model, the second model can learn how to correctly classify more quickly and accelerate the convergence speed of the second model. At the same time, the feature extraction capability of the trained second model can also be retained.

[0079] Optionally, the second learning rate parameter is set to K times the first learning rate parameter, where K is a ratio of the number of samples in the second sample subset to the number of samples in the first sample subset.

[0080]

[0081] is the number of samples in the second sample subset, is the number of samples in the first sample subset.

[0082] In other embodiments, if the first sample set also includes more sample subsets, that is, corresponding to more categories, the total sample set also includes sample subsets corresponding to multiple different categories, and the sample subset with the least number of samples is determined from the multiple sample subsets, and the sample number of the sample subset is used as the first number, and the sample subset with the largest number of samples is determined from the multiple sample subsets, and the sample number of the sample subset is used as the second number. K is the ratio of the second number to the first number.

[0083]

[0084] is the second quantity, is the first quantity.

[0085] For example, the second learning rate of the classification layer of the second model is set to the scaling factor K*5e -4 , the first learning rate of the representation layer of the second model is set to the scaling factor 5e -4 .

[0086] Please continue to refer to Table 1. In Table 1, the category with the largest number of samples in the total sample set is "apply for repayment", which has 5000 samples. The category with the least number of samples in the total sample set is "want to pay in installments", which has 500 samples. K=5000 / 500=10.

[0087] By accurately setting the first learning rate parameter and the second learning rate parameter, it can be ensured that the classification layer and the representation layer of the second model can converge almost synchronously in a fast and reasonable time.

[0088] In another embodiment, after adjusting the second weight parameter of the classification layer of the second model based on the second loss value and the second learning rate parameter, the model processing method may further include: Regularization is performed on the second weight parameter of the classification layer of the second model to obtain a third model.

[0089] The regularization process may be, but is not limited to, L1 regularization, L2 regularization, Bayesian regularization, etc.

[0090] For example, if the second weight parameter of the classification layer of the second model is subjected to L2 regularization. The second weight parameter matrix in the classification layer of the second model is , H is the hyperparameter in the second model, specifically the size of the matrix in the second model. Here H is usually 768. The calculation formula for the second weight parameter of each class of L2 regularization is:

[0091] in, is the second weight parameter of the i-th category after normalization of the classification layer, is the second weight parameter of the i-th category before normalization of the classification layer, , That is, corresponding to the 7 different categories in Table 1, in other embodiments, the value of i can be determined according to the specific number of categories. It is L2-normalization, that is, L2 regularization.

[0092] By regularizing the second weight parameter of the classification layer of the second model after parameter adjustment, the weight vector can be scaled, thereby further eliminating the deviation of the head category parameter weight.

[0093] In one embodiment, the second model is trained on the second model using the second sample set to obtain a third model, including: Initializing the second weight parameter of the classification layer of the second model by taking random numbers to obtain an initialized second model; The second sample set is used to perform second model training on the initialized second model to obtain a third model.

[0094] It should be understood that the parameters of the initial representation layer and the initial classification layer of the second model are obtained after training the first model, that is, they are not arbitrary values. The parameters of the initial representation layer of the second model remain unchanged, that is, the feature extraction capability of the second model is retained, and the overfitting of the model to the training data at the classification layer is reduced by initializing the second weight parameter of the initial classification layer of the second model, that is, resetting the parameters of the initial classification layer of the second model, which will help the performance of the model on the second training sample, and the introduction of randomness can help the second model jump out of the local optimal solution and increase the generalization ability of the third model obtained by training.

[0095] Based on the same inventive concept, the present disclosure also provides a data classification method, which may include: Acquire conversation data between the first user and the second user; The conversation data is input into the classification model to obtain the intent category output by the classification model. The classification model is obtained according to the above-mentioned model processing method.

[0096] The conversation data between the first user and the second user can be a real-time chat record or historical data extracted from the chat record data. The conversation data is then input into a classification model, and the classification model outputs the intent category to understand the real intent of the users in the conversation.

[0097] In customer service, by analyzing the intent of the user conversation, customers can be automatically directed to the correct service process or provided with targeted assistance. In intelligent assistants, understanding user intent can help the assistant perform specific tasks, such as setting reminders, searching for information, etc.

[0098] Based on the same inventive concept, the present disclosure also provides a model processing device, which can be applied to electronic devices. Figure 3 , the model processing device 600 may include: The first processing module 601 is configured to perform first model training on the first model using the first sample set to obtain a second model; the first sample set includes a first sample subset and a second sample subset, the first sample subset includes samples of the first category, the second sample subset includes samples of the second category, and the number of samples in the first sample subset is less than the number of samples in the second sample subset; The second processing module 602 is configured to use the second sample set to perform a second model training on the second model to obtain a third model; the second sample set includes a third sample subset and a fourth sample subset, the category of the samples included in the third sample subset is the same as the first category, the category of the samples included in the fourth sample subset is the same as the second category, and the number of samples in the third sample subset is the same as the number of samples in the fourth sample subset.

[0099] Optionally, the first processing module 601 includes: A first sub-processing module is configured to extract features of samples included in the first sample set to obtain first sample features; A second sub-processing module is configured to perform classification prediction according to the first sample feature to obtain a first prediction category; A third sub-processing module is configured to determine a first loss value of the first model based on the first prediction category; a fourth sub-processing module, configured to adjust parameters of the first model based on the first loss value; The second processing module 602 includes: a fifth sub-processing module, configured to extract features of samples included in the second sample set to obtain second sample features; a sixth sub-processing module, configured to perform classification prediction according to the second sample feature to obtain a second prediction category; a seventh sub-processing module, configured to determine a second loss value of a second model based on a second predicted category; An eighth sub-processing module is configured to adjust parameters of the second model based on the second loss value.

[0100] Optionally, the second model includes a representation layer and a classification layer, the parameters of the representation layer include a first learning rate parameter and a first weight parameter, and the parameters of the classification layer include a second learning rate parameter and a second weight parameter; The eighth sub-processing module is specifically configured as follows: Adjusting a first weight parameter of a representation layer of the second model based on the second loss value and the first learning rate parameter; Based on the second loss value and the second learning rate parameter, adjust the second weight parameter of the classification layer of the second model; wherein the second learning rate parameter is greater than the first learning rate parameter.

[0101] Optionally, the model processing device 600 may further include: The third processing module is configured to set the second learning rate parameter to K times the first learning rate parameter, where K is the ratio of the number of samples in the second sample subset to the number of samples in the first sample subset.

[0102] Optionally, the model processing device 600 may further include: The fourth processing module is configured to perform regularization processing on the second weight parameter of the classification layer of the second model to obtain the third model.

[0103] Optionally, the second processing module 602 may include: A ninth sub-processing module is configured to initialize the second weight parameter of the classification layer of the second model by taking a random number value, so as to obtain an initialized second model; The tenth sub-processing module is configured to perform second model training on the initialized second model using the second sample set to obtain a third model.

[0104] Regarding the model processing device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the model processing method, and will not be elaborated here.

[0105] Figure 4 1 is a block diagram of an electronic device 1900 according to an exemplary embodiment. For example, the electronic device 1900 may be provided as a server. Figure 4 The electronic device 1900 includes a first processor 1922, which may be one or more, and a first memory 1932 for storing a computer program executable by the first processor 1922. The computer program stored in the first memory 1932 may include one or more modules each corresponding to a set of instructions. In addition, the first processor 1922 may be configured to execute the computer program to perform the above-mentioned model processing method.

[0106] In addition, the electronic device 1900 may further include a power supply component 1926 and a first communication component 1950, the power supply component 1926 may be configured to perform power management of the electronic device 1900, and the first communication component 1950 may be configured to implement communication of the electronic device 1900, for example, wired or wireless communication. In addition, the electronic device 1900 may further include an input / output (I / O) interface 1958. The electronic device 1900 may operate based on an operating system stored in the first memory 1932, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, etc.

[0107] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, and when the program instructions are executed by a processor, the steps of the above-mentioned model processing method are implemented. For example, the computer-readable storage medium can be the above-mentioned first memory 1932 including program instructions, and the above-mentioned program instructions can be executed by the first processor 1922 of the electronic device 1900 to complete the above-mentioned model processing method.

[0108] Figure 5 FIG. 7 is a block diagram of another electronic device 700 according to an exemplary embodiment. Figure 5 As shown, the electronic device 700 may include: a second processor 701 , a second memory 702 . The electronic device 700 may also include one or more of a multimedia component 703 , an input / output (I / O) interface 704 , and a second communication component 705 .

[0109] The second processor 701 is used to control the overall operation of the electronic device 700 to complete all or part of the steps in the above-mentioned model processing method. The second memory 702 is used to store various types of data to support the operation of the electronic device 700. These data may include instructions for any application or method used to operate on the electronic device 700, and application-related data, such as contact data, messages sent and received, pictures, audio, video, etc. The second memory 702 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, referred to as SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, referred to as EEPROM), erasable programmable read-only memory (Erasable Programmable Read-Only Memory, referred to as EPROM), programmable read-only memory (Programmable Read-Only Memory, referred to as PROM), read-only memory (Read-Only Memory, referred to as ROM), magnetic memory, flash memory, magnetic disk or optical disk. The multimedia component 703 may include a screen and an audio component. The screen may be, for example, a touch screen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone, which is used to receive external audio signals. The received audio signal may be further stored in the second memory 702 or sent through the second communication component 705. The audio component also includes at least one speaker for outputting audio signals. The I / O interface 704 provides an interface between the second processor 701 and other interface modules, and the above-mentioned other interface modules may be keyboards, mice, buttons, etc. These buttons may be virtual buttons or physical buttons. The second communication component 705 is used for wired or wireless communication between the electronic device 700 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G or 4G, or a combination of one or more of them, so the corresponding second communication component 705 may include: Wi-Fi module, Bluetooth module, NFC module.

[0110] In an exemplary embodiment, the electronic device 700 can be implemented by one or more application specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPD), programmable logic devices (PLD), field programmable gate arrays (FPGA), controllers, microcontrollers, microprocessors or other electronic components to execute the above-mentioned model processing method.

[0111] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, and when the program instructions are executed by a processor, the steps of the above-mentioned model processing method are implemented. For example, the computer-readable storage medium can be the above-mentioned second memory 702 including program instructions, and the above-mentioned program instructions can be executed by the second processor 701 of the electronic device 700 to complete the above-mentioned model processing method.

[0112] Based on the same concept, an embodiment of the present disclosure also provides a computer program product, including a computer program, which implements the steps of the above-mentioned model processing method when executed by a processor.

[0113] It should be understood that the above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) to form a technical solution.

[0114] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0115] Although the subject matter has been described in language specific to structural features and / or method logic actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims. Regarding the device in the above embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.

Claims

1. A model processing method, characterized in that: The method comprises: Performing first model training on a first model using a first sample set to obtain a second model; the first sample set includes a first sample subset and a second sample subset, the first sample subset includes samples of a first category, the second sample subset includes samples of a second category, and the number of samples in the first sample subset is less than the number of samples in the second sample subset; The second model is trained on the second model using the second sample set to obtain a third model; the second sample set includes a third sample subset and a fourth sample subset, the category of the samples included in the third sample subset is the same as the first category, the category of the samples included in the fourth sample subset is the same as the second category, and the number of samples in the third sample subset is the same as the number of samples in the fourth sample subset.

2. The method according to claim 1, characterized in that The first model training comprises: Extracting features of samples included in the first sample set to obtain first sample features; performing classification prediction based on the first sample features to obtain a first prediction category; determining a first loss value of the first model based on the first prediction category; and adjusting parameters of the first model based on the first loss value.

3. The method according to claim 1, characterized in that The second model training comprises: Extract features of samples included in the second sample set to obtain second sample features; perform classification prediction based on the second sample features to obtain a second prediction category; determine a second loss value of the second model based on the second prediction category; and adjust parameters of the second model based on the second loss value.

4. The method according to claim 3, characterized in that The second model includes a representation layer and a classification layer, the parameters of the representation layer include a first learning rate parameter and a first weight parameter, and the parameters of the classification layer include a second learning rate parameter and a second weight parameter; The adjusting the parameters of the second model based on the second loss value comprises: Adjusting a first weight parameter of a representation layer of the second model based on the second loss value and the first learning rate parameter; Based on the second loss value and the second learning rate parameter, adjust the second weight parameter of the classification layer of the second model; wherein the second learning rate parameter is greater than the first learning rate parameter.

5. The method according to claim 4, characterized in that The method further comprises: The second learning rate parameter is set to K times the first learning rate parameter, where K is a ratio of the number of samples in the second sample subset to the number of samples in the first sample subset.

6. The method according to claim 4, characterized in that After adjusting the second weight parameter of the classification layer of the second model based on the second loss value and the second learning rate parameter, the method further includes: Regularization is performed on the second weight parameter of the classification layer of the second model to obtain a third model.

7. The method according to claim 4, characterized in that The step of performing second model training on the second model using the second sample set to obtain a third model includes: Initializing the second weight parameter of the classification layer of the second model by random number selection to obtain an initialized second model; The second sample set is used to perform second model training on the initialized second model to obtain a third model.

8. The method according to claim 1, characterized in that The first sample subset is extracted from samples of the first category of the total sample set according to a preset extraction ratio; The second sample subset is extracted from the samples of the second category of the total sample set according to the preset extraction ratio; The third sample subset is extracted from the samples of the first category of the total sample set according to a preset extraction quantity; The fourth sample subset is extracted from the samples of the second category of the total sample set according to the preset extraction quantity.

9. A data classification method, characterized in that: The data classification method comprises: Acquire conversation data between the first user and the second user; The conversation data is input into a classification model to obtain an intent category output by the classification model, wherein the classification model is obtained according to the model processing method according to any one of claims 1-8.

10. A model processing device, characterized in that: include: A first processing module is configured to perform first model training on a first model using a first sample set to obtain a second model; the first sample set includes a first sample subset and a second sample subset, the first sample subset includes samples of a first category, the second sample subset includes samples of a second category, and the number of samples in the first sample subset is less than the number of samples in the second sample subset; The second processing module is configured to use the second sample set to perform a second model training on the second model to obtain a third model; the second sample set includes a third sample subset and a fourth sample subset, the category of the samples included in the third sample subset is the same as the first category, the category of the samples included in the fourth sample subset is the same as the second category, and the number of samples in the third sample subset is the same as the number of samples in the fourth sample subset.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the model processing method described in any one of claims 1 to 8 are implemented.

12. An electronic device, characterized in that: include: a memory having a computer program stored thereon; A processor is used to execute the computer program in the memory to implement the steps of the model processing method according to any one of claims 1 to 8.

13. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the model processing method described in any one of claims 1 to 8 are implemented.

Images