A method and apparatus for training a classification model, classifying text

Abstract

The application provides a method and device for training a classification model and classifying text. The method comprises obtaining a to-be-processed text, wherein the to-be-processed text at least comprises online classification text, an online question, a plurality of online classification labels, and a prediction probability filling position corresponding to each online classification label; the online question is used to prompt the prediction probability of an answer of the online classification text to the online question belonging to each online classification label; and the prediction probability of each online classification label corresponding to the prediction probability filling position filled in the to-be-processed text is predicted based on a classification model, wherein the prediction probability comprises the probability of the answer of the online classification text to the online question belonging to each online classification label. The application has higher classification accuracy in a zero-shot scene, can reduce the labeling workload of a service platform, reduce the labor cost, reduce the network parameters in the classification model, reduce the system resources consumed for loading the classification model, improve the operation speed, and improve the classification efficiency.

Description

Technical Field

[0001] This application relates to the field of computer science, and in particular to a method and apparatus for training a classification model and classifying text. Background Technology

[0002] Currently, an increasing number of service platforms have emerged online. These platforms provide services to users; for example, shopping platforms offer shopping services to shoppers, gaming platforms offer gaming services to gamers, and chat platforms offer chat services to chatters. To better serve users and enhance the competitiveness of these platforms, user preferences can be collected and categorized. Services can then be tailored to these preferences to meet users' core needs as much as possible, thereby increasing user stickiness to the platform.

[0003] Currently, models can be used to classify user preferences. However, the accuracy of user preference classification obtained using models is currently low. Furthermore, the model needs to be trained beforehand, and data labeling is required before training, which is a large workload and labor cost. Secondly, the efficiency of using models to classify user preferences is currently low. Summary of the Invention

[0004] This application discloses a training classification model, a method for classifying text, and an apparatus.

[0005] Firstly, a method for training a classification model is shown, comprising: obtaining a pre-trained language model; obtaining a first training text, the first training text including at least a first sample classification text, a first sample question, multiple first sample classification labels, and prediction probability fill-in positions corresponding to each first sample classification label; the first sample question being used to prompt the prediction probability that the answer of the first sample classification text to the first sample question belongs to each first sample classification label; obtaining a first labeled classification label of the first sample classification text, the first labeled classification label being one of multiple first sample classification labels; using at least the first training text and the first labeled classification label, optimizing the pre-trained language model for a probability prediction task for each prediction probability fill-in position corresponding to each first sample classification label, thereby obtaining a classification model, wherein the probability prediction task includes the task of predicting the prediction probability that the answer of the first sample classification text to the first sample question belongs to each first sample classification label.

[0006] Secondly, a method for text classification is shown, comprising: obtaining a first text to be processed, the first text to be processed including at least a first online classification text, a first online question, multiple first online classification labels, and prediction probability fields corresponding to each first online classification label; the first online question is used to prompt the predicted probability that the answer of the first online classification text to the first online question belongs to each first online classification label; based on a trained classification model, predicting the predicted probability to be filled in the prediction probability fields corresponding to each first online classification label in the first text to be processed, the predicted probability including the probability that the answer of the first online classification text to the first online question belongs to each first online classification label; wherein, the trained classification model is at least... The language model is optimized using the first training text and the first labeled classification label. The probability prediction task is to predict the probability of the answer of the first sample classification text to the first sample question belonging to each of the first sample classification labels. The first training text includes at least the first sample classification text, the first sample question, multiple first sample classification labels, and the prediction probability filling position corresponding to each first sample classification label. The first sample question is used to prompt the prediction probability of the answer of the first sample classification text to the first sample question belonging to each of the first sample classification labels. The first labeled classification label is one of the multiple first sample classification labels.

[0007] Thirdly, an apparatus for training a classification model is shown, comprising: a first acquisition module for acquiring a pre-trained language model; a second acquisition module for acquiring a first training text, the first training text including at least a first sample classification text, a first sample question, multiple first sample classification labels, and prediction probability fill-in positions corresponding to each first sample classification label; the first sample question is used to prompt the prediction probability that the answer of the first sample classification text to the first sample question belongs to each first sample classification label; a third acquisition module for acquiring a first labeled classification label of the first sample classification text, the first labeled classification label being one of multiple first sample classification labels; and an optimization learning module for optimizing the pre-trained language model using at least the first training text and the first labeled classification label to perform probability prediction tasks for the prediction probability fill-in positions corresponding to each first sample classification label, thereby obtaining a classification model, wherein the probability prediction task includes the task of predicting the prediction probability that the answer of the first sample classification text to the first sample question belongs to each first sample classification label.

[0008] Fourthly, an apparatus for text classification is shown, comprising: a fourth acquisition module for acquiring a first text to be processed, the first text to be processed including at least a first online classification text, a first online question, multiple first online classification labels, and prediction probability fields corresponding to each first online classification label; the first online question is used to prompt the predicted probability that the answer of the first online classification text to the first online question belongs to each first online classification label; a prediction module for predicting, based on a trained classification model, the predicted probability to be filled in the prediction probability fields corresponding to each first online classification label in the first text to be processed, the predicted probability including the probability that the answer of the first online classification text to the first online question belongs to each first online classification label; wherein, the trained... The classification model is obtained by optimizing the pre-trained language model using at least the first training text and the first labeled classification label, and performing a probability prediction task for the prediction probability fill-in position corresponding to each first sample classification label. The probability prediction task includes the task of predicting the predicted probability that the answer of the first sample classification text to the first sample question belongs to each first sample classification label. The first training text includes at least the first sample classification text, the first sample question, multiple first sample classification labels, and the prediction probability fill-in position corresponding to each first sample classification label. The first sample question is used to prompt the predicted probability that the answer of the first sample classification text to the first sample question belongs to each first sample classification label. The first labeled classification label is one of multiple first sample classification labels.

[0009] Fifthly, an electronic device is shown, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to perform the methods shown in any of the preceding aspects.

[0010] In a sixth aspect, a non-transitory computer-readable storage medium is shown, which, when the instructions in the storage medium are executed by a processor of an electronic device, enables the electronic device to perform the methods shown in any of the foregoing aspects.

[0011] In a seventh aspect, a computer program product is shown that, when the instructions in the computer program product are executed by a processor of an electronic device, enables the electronic device to perform the methods shown in any of the preceding aspects.

[0012] In this application, sample classification text, sample questions, multiple sample classification labels, and prediction probability fill-in positions corresponding to each sample classification label can be combined to obtain training text. The labeled classification labels of the sample classification text are used to supervise the training of "the predicted probabilities filled in at the prediction probability fill-in positions corresponding to each sample classification label according to the sample questions". This enables the pre-trained language model to be optimized for the probability prediction task of the prediction probability fill-in positions corresponding to each sample classification label, thereby obtaining a classification model. This results in better performance (e.g., higher prediction accuracy) of the classification model on the task of predicting the prediction probability that the answer to the question belongs to each classification label.

[0013] Furthermore, the text to be processed input into the classification model of this application includes at least online classification text, online questions, multiple online classification labels, and prediction probability fields corresponding to each online classification label. That is, the classification model will know which classification labels to classify for the online classification text to be classified this time, and the classification model can know the classification range of the online classification text this time. For example, it is classified within the range of multiple online classification labels in the text to be processed, rather than within the range of other classification labels. Moreover, the classification model has the ability to "predict the prediction probability of the answer of the online classification text to the online question belonging to each online classification label". Even if the classification model has not previously perceived the multiple online classification labels in the text to be processed, the presence of multiple online classification labels in the text to be processed and the prediction probability fields corresponding to each online classification label can guide the classification model to predict the prediction probability of the answer of the online classification text to the online question belonging to each online classification label, rather than guiding the classification model to generate classification labels. This can overcome the problem of low classification accuracy in zero-shot scenarios, thus making the classification model of this application more accurate in zero-shot scenarios.

[0014] Secondly, the classification model of this application can be shared by multiple service platforms. Each service platform no longer needs to train the classification model according to its own specific needs, and each service platform does not need to master the training technology of the classification model, thus lowering the technical threshold. For example, even if the classification requirements of each service platform are different, and even if the classification labels required for text classification are different, each service platform can directly deploy the classification model of this application. When it needs to classify online classified text, the text to be processed input to the classification model includes "online classified text, online question, multiple online classification labels, and the prediction probability fill-in position corresponding to each online classification label". This allows the classification model to predict the prediction probability of the answer of the online classified text to the online question belonging to each online classification label, enabling cold start use of the classification model for each service platform. Since each service platform no longer needs to train the classification model, it does not need to obtain sample classified text, nor does it need to label the sample classified text, reducing the labeling workload of each service platform and reducing labor costs.

[0015] Furthermore, the classification model of this application predicts the probability that the answer of the online classified text to the online question belongs to each of the known online classification labels, without generating classification labels (that is, without performing the task of generating text). Therefore, the classification model of this application may not have a classification layer, which simplifies the structure of the classification model. For example, it reduces the network parameters in the classification model, which makes the system resources required to load and run the classification model less, and can improve the computing speed, thereby improving the classification efficiency. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating the steps of a method for training a classification model according to this application.

[0017] Figure 2 This is a flowchart illustrating the steps of a method for training a classification model according to this application.

[0018] Figure 3 This is a flowchart illustrating the steps of a method for training a classification model according to this application.

[0019] Figure 4 This is a flowchart illustrating the steps of a text classification method according to this application.

[0020] Figure 5 This is a flowchart illustrating the steps of a text classification method according to this application.

[0021] Figure 6 This is a flowchart illustrating the steps of a text classification method according to this application.

[0022] Figure 7 This is a structural block diagram of a device for training a classification model according to this application.

[0023] Figure 8 This is a structural block diagram of a text classification device according to this application.

[0024] Figure 9 This is a structural block diagram of a device according to this application. Detailed Implementation

[0025] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0026] The inventors discovered that in many cases, when users encounter problems, they communicate with service platform staff via text to resolve their issues (meet their needs). Therefore, the inventors collected historical communication texts between a large number of users and service platform staff, and conducted extensive statistical analysis on these texts. They found that the content of these historical communication texts often reveals user preference categories (what types of services users like and dislike, etc.).

[0027] In light of this, to obtain user preference classifications, the inventors conceived of using historical communication texts between users and service platform staff. Specifically, they devised a method where a model can be used to categorize user preferences based on these historical communication texts. For example, a classification model can be used to categorize historical communication texts to obtain classification labels (e.g., sentiment classification of historical communication texts to obtain sentiment labels for the user's feelings towards the services provided by the service platform, which can include positive and negative labels). These labels can reflect what types of services the user likes or dislikes. Thus, user preference classifications can be derived from these labels, achieving the goal of obtaining user preference classifications based on classification labels from historical communication texts.

[0028] In order to classify historical communication texts and obtain classification labels for them, a classification model can be trained in advance. Then, when it is necessary to classify texts, the text (such as historical communication texts between users and staff of the service platform) can be input into the classification model so that the classification model can generate classification labels for the texts based on their content and output the classification labels for the texts.

[0029] Furthermore, with the advent of pre-trained language models, NLP (Natural Language Processing) has entered a new era. Each service platform can train its own classification model based on the pre-trained language model. For example, any service platform can collect a large amount of training data, including sample classified text and its labeled classification tags. A classification layer can be added to the end of the pre-trained language model, and the number of classifications in the layer can be set (e.g., 2-class, 3-class, or more). Then, the pre-trained language model with the classification layer can be optimized using the training data to obtain a suitable classification model. The classification layer in this model has the ability to generate classification tags, and the tags generated by the model are often the labeled classification tags used in the training data, not those not used in the training data. The same applies to any other service platform.

[0030] However, the inventors discovered that the above method has the following problems:

[0031] On the one hand, the classification model obtained in this way has low accuracy in zero-shot scenarios. For example, suppose the classification label of a certain text is not the labeled classification label used in the training data, but a previously unknown classification label. However, after processing the text with the classification model, the generated classification label of the text will only be the labeled classification label used in the training data, and will not be a previously unknown classification label, resulting in low classification accuracy.

[0032] On the other hand, the classification requirements of different service platforms are often different. As a result, the classification labels required for text classification by each service platform are also different. Consequently, the classification labels used by each service platform to train its own classification model are also different. The classification models trained by each service platform cannot be used by other service platforms. Thus, each service platform needs to obtain the annotation labels of its own sample text, and the annotation data cannot be shared, resulting in a large workload and high labor costs for each platform.

[0033] On the other hand, the classification models trained by various service platforms need to generate classification labels (that is, they need to perform the task of generating text). Therefore, the classification models need to have classification layers, which makes the structure of the classification models complex. For example, the classification models have more network parameters, which leads to more system resources required to load and run the classification models.

[0034] On the other hand, classification models need to generate classification labels and predict the probability of each classification label, and then output the classification label with the highest probability. This method is slow and results in low efficiency for text classification.

[0035] To address at least one of the issues mentioned above, see [link to relevant documentation]. Figure 1 This paper demonstrates a method for training a classification model, applicable to electronic devices. These devices can include cloud-based or user-controlled devices. The cloud can exist as a server, and user-controlled devices can include tablets, laptops, desktop computers, mobile phones, and PDAs (Personal Digital Assistants). The method includes:

[0036] In step S101, a pre-trained language model is obtained. A first training text is obtained, which includes at least a first sample classification text, a first sample question, multiple first sample classification labels, and prediction probability fields corresponding to each first sample classification label. The first sample question is used to prompt the predicted probability that the answer of the first sample classification text to the first sample question belongs to each first sample classification label. A first labeled classification label is obtained from the first sample classification text, which is one of the multiple first sample classification labels.

[0037] Pre-trained language models include at least: pre-trained BERT (Bidirectional Encoder Representation from Transformers), GPT (Gererate Pre-Training Model), PLM (Pre-trained Language Model), RoBERTa (Robustly optimized BERT approach), DeBERTa (Decoding-enhanced BERT with disentangled attention), and ALBERT, etc. Of course, pre-trained language models can also include other types of pre-trained language models, which are not limited in this application. A pre-trained language model can be a neural network model that learns semantic information from a large-scale corpus in an unsupervised or supervised manner. It can be a complex learning model with multi-layered neural networks. Pre-trained language models can more accurately capture semantic information in text (possessing a knowledge space), improving the accuracy of the model in performing downstream tasks.

[0038] The training texts used in this application for optimizing the pre-trained language model can be multiple, and the number of sample classification labels in different training texts can be different, so that the trained classification model is adaptable to various numbers of classification labels, that is, the trained classification model can be adapted to binary classification, ternary classification, and so on, as well as N-class classification scenarios. The sample classification texts in different training texts can be different. These sample classification texts can be collected from the internet, etc. The labeled classification labels of the sample classification texts can be manually labeled or collected from the internet, etc. The question is used to guide and mine the pre-trained language model's embedded knowledge to more accurately predict the predicted probability of the answer to the question belonging to each classification label, based on the probability prediction task and requirements corresponding to the prediction probability of each classification label. The question can include natural language text. The question can be manually set in advance according to the application scenario, etc., and the questions applicable to different application scenarios can be different, so that the trained classification model is adaptable to various application scenarios.

[0039] The multiple first-sample classification labels can be preset, and the number of first-sample classification labels and the specific classification labels of each first-sample label can be determined according to the actual situation. For example, in a binary classification scenario of emotions, the binary emotions include positive emotions and negative emotions. The classification labels corresponding to positive emotions can include "like" or "good", etc., and the classification labels corresponding to negative emotions can include "dislike" or "bad", etc.

[0040] The first training text of this application is illustrated with an example, but this is not intended to limit the scope of protection of this application. For example, the first training text may include "[] positive[] negative; How is this restaurant; The dishes are all the same and taste terrible." Here, the first sample classification text includes "The dishes are all the same and taste terrible." The first sample question includes "How is this restaurant?" The first sample classification labels include "positive" and "negative," where "positive" and "negative" represent sentiment classification. "[]" is used to mark the prediction probability fields. The prediction probability fields may include placeholders, etc. The first sample question "How is this restaurant?" is used to prompt the prediction of the probability that the answer to the first sample classification text "The dishes are all the same and taste terrible" belongs to the first sample classification label "positive" and the first sample classification label "negative" for the first sample question "How is this restaurant?". In this example, it is a prediction of sentiment classification, that is, a prediction of the sentiment classification of the first sample classification text.

[0041] The languages ​​of the texts involved in this application may include English, Russian, German, French, Spanish, Japanese, and Chinese, etc. Of course, other languages ​​may also be included depending on the actual situation, which will not be detailed here.

[0042] In step S102, at least the first training text and the first labeled classification label are used to optimize the pre-trained language model for the probability prediction task of the prediction probability filling position corresponding to each first sample classification label, so as to obtain a classification model. The probability prediction task includes the task of predicting the prediction probability of the answer of the first sample classification text to the first sample question belonging to each first sample classification label.

[0043] In this application, each prediction probability field corresponding to a first sample classification label corresponds one-to-one with each first sample classification label. This is used to guide the pre-trained language model to predict the probability that the answer of the first sample classification text to the first sample question belongs to each first sample classification label during the optimization learning process. The input data of the pre-trained language model may include the first training text, etc. The output data of the pre-trained language model may include the prediction probabilities that the answer of the first sample classification text to the first sample question belongs to each first sample classification label, which are filled in at the prediction probability fields corresponding to each first sample classification label. Using the labeled classification labels of the first sample classification text, supervised training is performed on the predicted probabilities that the answer of the first sample classification text to the first sample question belongs to each first sample classification label. This achieves optimized learning of the pre-trained language model for the probability prediction task corresponding to each first sample classification label. For example, it enables fine-tuning of the parameters of the pre-trained language model to optimize the learning of the parameters of the pre-trained language model until the parameters in the pre-trained language model converge, thereby obtaining a classification model.

[0044] For example, suppose the first sample classification text is "The dishes are all the same, and they're awful." We need to perform sentiment classification on this text to determine whether it expresses positive or negative sentiment. The first sample question is: "How is this restaurant?" Multiple first sample classification labels include "positive" and "negative." The prediction probability fields are "[]", etc. We can combine the first sample classification text "The dishes are all the same, and they're awful," the first sample question "How is this restaurant?", the multiple first sample classification labels "positive" and "negative," and the prediction probability fields "[]" for each label to obtain the first training text "[]positive[]negative; How is this restaurant?; The dishes are all the same, and they're awful." Then, the first training text "[] positive[] negative; How is this restaurant; The dishes are all the same, so unpalatable" is input into the pre-trained language model. This allows the pre-trained language model to predict the predicted probabilities of the first sample classification text "The dishes are all the same, so unpalatable" belonging to the first sample classification label "How is this restaurant?" in the prediction probability fill-in positions "[]" corresponding to the first sample classification labels "positive" and "negative", respectively. Based on the predicted predicted probabilities of the first sample classification text "The dishes are all the same, so unpalatable" belonging to the first sample classification label "How is this restaurant?" in the first sample question "How is this restaurant?", and the first labeled classification label of the first sample classification text "The dishes are all the same, so unpalatable", the pre-trained language model is optimized for the probability prediction task corresponding to each first sample classification label, so as to optimize the network parameters in the pre-trained language model.

[0045] In this application, sample classification text, sample questions, multiple sample classification labels, and prediction probability fill-in positions corresponding to each sample classification label can be combined to obtain training text. The labeled classification labels of the sample classification text are used to supervise the training of "the predicted probabilities filled in at the prediction probability fill-in positions corresponding to each sample classification label according to the sample questions". This enables the pre-trained language model to be optimized for the probability prediction task of the prediction probability fill-in positions corresponding to each sample classification label, thereby obtaining a classification model. This results in better performance (e.g., higher prediction accuracy) of the classification model on the task of predicting the prediction probability that the answer to the question belongs to each classification label.

[0046] Furthermore, the text to be processed input into the classification model of this application includes at least online classification text, online questions, multiple online classification labels, and prediction probability fields corresponding to each online classification label. That is, the classification model will know which classification labels to classify for the online classification text to be classified this time, and the classification model can know the classification range of the online classification text this time. For example, it is classified within the range of multiple online classification labels in the text to be processed, rather than within the range of other classification labels. Moreover, the classification model has the ability to "predict the prediction probability of the answer of the online classification text to the online question belonging to each online classification label". Even if the classification model has not previously perceived the multiple online classification labels in the text to be processed, the presence of multiple online classification labels in the text to be processed and the prediction probability fields corresponding to each online classification label can guide the classification model to predict the prediction probability of the answer of the online classification text to the online question belonging to each online classification label, rather than guiding the classification model to generate classification labels. This can overcome the problem of low classification accuracy in zero-shot scenarios, thus making the classification model of this application more accurate in zero-shot scenarios.

[0047] Secondly, the classification model of this application can be shared by multiple service platforms. Each service platform no longer needs to train the classification model according to its own specific needs, and each service platform does not need to master the training technology of the classification model, thus lowering the technical threshold. For example, even if the classification requirements of each service platform are different, and even if the classification labels required for text classification are different, each service platform can directly deploy the classification model of this application. When it needs to classify online classified text, the text to be processed input to the classification model includes "online classified text, online question, multiple online classification labels, and the prediction probability fill-in position corresponding to each online classification label". This allows the classification model to predict the prediction probability of the answer of the online classified text to the online question belonging to each online classification label, enabling cold start use of the classification model for each service platform. Since each service platform no longer needs to train the classification model, it does not need to obtain sample classified text, nor does it need to label the sample classified text, reducing the labeling workload of each service platform and reducing labor costs.

[0048] Furthermore, the classification model of this application predicts the probability that the answer of the online classified text to the online question belongs to each of the known online classification labels, without generating classification labels (that is, without performing the task of generating text). Therefore, the classification model of this application may not have a classification layer, which simplifies the structure of the classification model. For example, it reduces the network parameters in the classification model, which makes the system resources required to load and run the classification model less, and can improve the computing speed, thereby improving the classification efficiency.

[0049] In one embodiment of this application, at least a first training text and a first labeled classification label can be used to perform at least one round of optimization learning on the pre-trained language model for the probability prediction task of the prediction probability fill-in position corresponding to each first sample classification label, thereby obtaining a classification model. Any round of optimization learning may include:

[0050] 11) Obtain the cross-entropy loss value of the pre-trained language model based on the first training text and the first labeled classification label.

[0051] In this application, a pre-trained language model can be used to process the first training text, and then the cross-entropy loss value of the pre-trained language model is obtained based on the processing result of the first training text and the first labeled classification label. For example, the first training text is input into a pre-trained language model (e.g., a pre-trained language model obtained after the previous round of optimization learning), so that the pre-trained language model processes the first training text to obtain the predicted probabilities of the first sample classification text's answer to the first sample question belonging to each of the first sample classification labels. Then, the cross-entropy loss value of the pre-trained language model can be calculated based on the predicted probabilities of the first sample classification text's answer to the first sample question belonging to each of the first sample classification labels, the first labeled classification label, and the cross-entropy loss function.

[0052] 12) At least based on the cross-entropy loss value, optimize the pre-trained language model for the probability prediction task of the prediction probability filling position corresponding to each first sample classification label.

[0053] For example, the network parameters in the pre-trained language model can be adjusted based on the cross-entropy loss value for the probability prediction task corresponding to the prediction probability fill-in positions for each first sample classification label. After at least one round of optimization learning on the pre-trained language model for the probability prediction task corresponding to the prediction probability fill-in positions for each first sample classification label, and with the convergence of the network parameters in the pre-trained language model, the latest pre-trained language model is used as the classification model, and then the classification model can be deployed online.

[0054] In another embodiment of this application, when optimizing the pre-trained language model to perform probability prediction tasks for the prediction probability fill-in positions corresponding to each first sample classification label using at least the first training text and the first labeled classification label, and obtaining the classification model, the following process can be used:

[0055] 21) Mask at least one character in the first sample classification text of the first training text to obtain the masked training text.

[0056] For example, at least one character in the first sample classification text of the first training text can be masked, and then the masked first training text can be used as the masked training text. The mask may include replacing at least one character in the first sample classification text with a preset character (e.g., a mask token). The preset character can be set according to actual conditions, such as [XXX], (YYY), or "ZZZ," etc., and this application does not limit this. Alternatively, the mask may include deleting at least one character in the first sample classification text. The masked training text contains at least the unmasked content of the first sample classification text (masked classification text), a first sample question, multiple first sample classification labels, and prediction probability fields corresponding to each first sample classification label. The first sample question is used to indicate the predicted probability that the answer to the first sample question in the predicted masked classification text belongs to each first sample classification label.

[0057] The first label of the first sample classified text can be regarded as the label of the unmasked content (masked classified text) in the first sample classified text.

[0058] 22) At least use the masked training text and the first labeled classification label to optimize the pre-trained language model for the probability prediction task of the prediction probability filling position corresponding to each first sample classification label, and obtain the classification model.

[0059] This step can be referred to in the process of 11) to 12) above, and will not be described in detail here.

[0060] In a scenario where a pre-trained language model is optimized for a probability prediction task targeting the prediction probability fill-in positions corresponding to each first sample classification label, there is a masked training text. The masked training text contains at least the unmasked content of the first sample classification text (masked classification text), the first sample question, multiple first sample classification labels, and prediction probability fill-in positions corresponding to each first sample classification label.

[0061] This approach allows for optimization learning of the pre-trained language model based on the masked training text and the labeled classification tags of the masked classification text (the first labeled classification tags of the first sample classification text). This enables the pre-trained language model to obtain training texts with fewer characters (e.g., fewer characters in the classification text, fewer characters in the masked classification text than in the first sample classification text). In other words, it allows the pre-trained language model to obtain training texts with fewer characters (e.g., classification texts with fewer characters). Consequently, the pre-trained language model can focus more on learning relevant information from texts with fewer characters, resulting in a stronger ability to extract relevant information from texts with fewer characters. This also improves the accuracy of the trained classification model in classifying texts with fewer characters, thereby enhancing the robustness and effectiveness of the trained classification model and ultimately improving its accuracy in text classification.

[0062] In another embodiment of this application, see Figure 2 The process of obtaining the first training text in step S101 includes:

[0063] In step S201, the first sample classification text is obtained, multiple first sample classification labels are obtained, and the first sample question is obtained.

[0064] The first sample classification text can be manually submitted or entered, thus allowing us to obtain the manually submitted or entered first sample classification text. Multiple first sample classification labels can be manually submitted or entered along with the first sample classification text, thus allowing us to obtain the manually submitted or entered multiple first sample classification labels. The first sample question can be manually submitted or entered along with the first sample classification text, thus allowing us to obtain the manually submitted or entered first sample question. Furthermore, the methods for obtaining the first sample question and multiple first sample classification labels can also be found in the embodiments shown later, and will not be detailed here.

[0065] In step S202, a first training text is generated based on at least the first sample classification text, multiple first sample classification labels, and the first sample question. The first training text also includes prediction probability fields corresponding to each first sample classification label.

[0066] The positional relationships between the sample classification text, multiple sample classification labels, and sample questions can be pre-defined, and this application does not limit the specific positional relationships. The prediction probability fields for each sample classification label are different. In one example, based on the positional relationships between the sample classification text, multiple sample classification labels, and sample questions, the first sample classification text, multiple first sample classification labels, and the first sample question can be combined, and corresponding prediction probability fields can be added to the left or right of each first sample classification label to obtain the first training text.

[0067] In some cases, multiple first sample classification labels are not manually submitted. Since there are many training texts used in optimizing the pre-trained language model, and multiple sample classification labels are needed in each training text, manually expanding the multiple sample classification labels in each training text would result in high labor costs and difficulty. Therefore, to reduce labor costs and difficulty, in another embodiment of this application, when the electronic device receives the first labeled classification label of the manually submitted or input first sample classification text, the first sample classification label can be automatically expanded based on the first labeled classification label of the first sample classification text. For example, the manually submitted or input first labeled classification label is determined as one first sample classification label, and at least one first sample classification label is expanded. Then, multiple first sample classification labels can be obtained based on this one first sample classification label and the automatically expanded at least one first sample classification label.

[0068] Specifically, in one embodiment, regarding the acquisition of multiple first sample classification labels in step S201, see [link to relevant documentation]. Figure 3 This can be achieved through the following process, including:

[0069] In step S301, given that the first annotation classification label of the first sample classification text has been obtained, at least one classification label is obtained based on the first annotation classification label. The at least one classification label is different from the first annotation classification label.

[0070] In one embodiment of this application, the first annotation classification label can be considered as one of multiple first sample classification labels. Multiple sample classification labels in a training text all belong to the same domain. Thus, the domain to which the first annotation classification label belongs can be determined. Domains include time domains, location domains, sentiment domains, personal name domains, and place name domains. Of course, it is understood that other possible domains may also be included, which this application does not limit and will not list in detail. For example, existing domain identification methods can be used to identify the domain to which the first annotation classification label belongs; this application does not limit the specific domain identification method. Then, at least one classification label for that domain, different from the first annotation classification label, can be generated. For example, multiple classification label sets can be pre-set for different domains. A classification label set corresponding to a domain includes multiple classification labels within that domain. Thus, at least one classification label different from the first annotation classification label can be obtained from the classification label set corresponding to the domain to which the first annotation classification label belongs. Alternatively, use an existing generation method to generate at least one category label in the domain that is different from the first labeled category label. For example, input the first labeled category label as input data into an existing generation model to obtain at least one category label generated by the existing generation model based on the input first labeled category label.

[0071] In another embodiment of this application, multiple classification label sets can be pre-set, each set including multiple classification labels. The classification labels in different sets are applicable to different scenarios. For example, one set might include "positive" and "negative" for sentiment classification; another set might include "positive" and "negative" for sentiment classification; yet another set might include "good-looking" and "not good-looking" for sentiment classification; and yet another set might include "Beijing," "Guangzhou," and "Shanghai" for location classification. Thus, a target classification label set can be searched among the pre-set multiple sets. Each set includes multiple classification labels, and the target set includes a first labeled classification label, or it may include similar classification labels whose semantic similarity to the first labeled classification label is greater than a first preset similarity. The first preset similarity can be set according to actual conditions, and this application does not limit it; it can be in the form of a percentage, etc. The method for calculating the semantic similarity between classification labels can refer to existing methods for calculating semantic similarity, and this application does not limit it. Then, at least one category label can be obtained from the category labels in the target category label set other than the first labeled category label or similar category labels. For example, the category labels in the target category label set other than the first labeled category label or similar category labels can be determined as at least one category label.

[0072] In another embodiment of this application, the optimization learning of a pre-trained language model often requires a large amount of training text. Before obtaining the first training text, at least one second training text may have already been obtained, for example, multiple second training texts may have been obtained (the multiple second training texts and the first training text are both used for optimization learning of the pre-trained language model). The second training text includes at least a second sample classification text, a second sample question, multiple second sample classification labels, and prediction probability fields corresponding to each second sample classification label. The second sample question is used to prompt the predicted probability that the answer of the second sample classification text to the second sample question belongs to each second sample classification label. The second labeled classification label of the second sample classification text has also been obtained, and the second labeled classification label is one of the multiple second sample classification labels. Thus, a target second training text can be determined from the obtained second training text. The target second training text includes at least a second sample classification text, a second sample question, multiple second sample classification labels, and prediction probability fields corresponding to each second sample classification label. The target second training text includes multiple second sample classification labels, including the first labeled classification label; or, the target second training text includes multiple second sample classification labels, including similar classification labels whose semantic similarity to the first labeled classification label is greater than a first preset similarity. The first preset similarity can be set according to actual conditions, and this application does not limit it; it can be in the form of a percentage, etc. The method for calculating the semantic similarity between classification labels can refer to existing methods for calculating semantic similarity, and this application does not limit it.

[0073] Then, at least one classification label can be obtained from the multiple second sample classification labels other than the first label classification label or the similarity classification label. For example, the classification label other than the first label classification label or the similarity classification label from the multiple second sample classification labels can be determined as at least one classification label.

[0074] In step S302, multiple first sample classification labels are obtained based on the first labeled classification label and at least one obtained classification label.

[0075] For example, the first annotation classification label and the at least one obtained classification label are a total of multiple classification labels. Thus, the first annotation classification label can be determined as a first sample classification label, and the at least one obtained classification label can be determined as a first sample classification label, thereby obtaining multiple first sample classification labels.

[0076] This application enables the optimization learning of a pre-trained language model in the scenario of predicting the probability of each first sample classification label for the corresponding prediction probability filling position. It supports the automatic expansion of multiple sample classification labels for a single training text without the need for manual submission or input, thereby reducing manual costs and difficulty.

[0077] Alternatively, to reduce labor costs and simplify the process, in another embodiment of this application, when the electronic device obtains the first sample classification text and its first sample question submitted or input manually, multiple first sample classification labels can be automatically expanded based on the first sample question. Specifically, in one embodiment, for obtaining multiple first sample classification labels, if the first sample question has already been obtained, at least multiple classification labels can be obtained based on the first sample question. For example, when the first sample classification text is obtained, the first sample question may already be obtained. The first sample question may be manually input or submitted along with the first sample classification, or the first sample question may be preset.

[0078] Furthermore, various questions that may be used in subsequent classification scenarios can be statistically analyzed in advance. For any given question, multiple category labels applicable to that question can be statistically analyzed, and these applicable category labels can be grouped into a category label set. Then, the question and the grouped category label set are combined into a corresponding table entry and stored in the first correspondence between questions and category label sets. Thus, given a first sample question, questions with a similarity greater than a second preset similarity can be found in the first correspondence between questions and category label sets. Each category label set in the first correspondence includes multiple category labels. The second preset similarity can be set according to actual conditions, and this application does not limit it; it can be in the form of a percentage, etc. The method for calculating the semantic similarity between questions can refer to existing methods for calculating semantic similarity, and this application does not limit it. Then, in the first correspondence, the category label set corresponding to the found question is searched. Multiple first sample category labels are obtained based on the category labels in the found category label set; for example, all category labels in the found category label set are determined as multiple first sample category labels.

[0079] In some cases, the optimization learning of a pre-trained language model requires a large number of training texts. It's not always possible to manually submit sample questions for each training text. Manually supplementing the sample questions for each training text would be costly and difficult. Therefore, to reduce labor costs and difficulty, in another embodiment of this application, when the electronic device receives the manually submitted or input first sample classification text and at least one first sample classification label (e.g., the first labeled classification text can be considered as a first sample classification text), the first sample question can be automatically supplemented.

[0080] Specifically, the problem of obtaining the first sample in step S201 can be implemented through the following embodiments.

[0081] For example, various questions that may be used in subsequent classification scenarios can be counted in advance. For any question counted, multiple classification labels applicable to the question can be counted, and the classification labels applicable to the question can be combined into a classification label set. Then, the question and the appropriate classification label set can be combined into a corresponding table entry and stored in the first correspondence between the question and the classification label set.

[0082] Thus, in one embodiment of this application, when multiple first sample category labels have been obtained, a target category label set can be searched in a first correspondence between questions and category label sets, where each category label set in the first correspondence includes multiple category labels. The label similarity between the target category label set and the multiple first sample category labels is greater than a third preset similarity. The third preset similarity can be set according to actual conditions, and this application does not limit it; it can be in the form of a percentage, etc. The method for calculating the label similarity between the category label set and the multiple category labels can refer to existing methods for calculating label similarity, and this application does not limit it. For example, the number of identical category labels between the category label set and the multiple category labels can be counted, and then the ratio between this number and the minimum / maximum value of "the number of category labels in the category label set and the number of category labels in the multiple category labels" can be calculated. Then, a question corresponding to the target category label set can be searched in the first correspondence. A first sample question is obtained based on the found question; for example, the found question is determined as the first sample question.

[0083] Alternatively, given the first labeled category label, the target category label set can be searched within the first correspondence between questions and category label sets. Each category label set in the first correspondence includes multiple category labels. The target category label set includes the first labeled category label, or it includes similar category labels whose semantic similarity to the first labeled category label is greater than a first preset similarity. The first preset similarity can be set according to actual circumstances; this application does not limit it, and it can be in the form of a percentage, etc. The method for calculating the semantic similarity between category labels can refer to existing methods for calculating semantic similarity; this application does not limit it. Then, questions corresponding to the target category label set can be searched within the first correspondence. A first sample question is obtained based on the found question; for example, the found question is determined as the first sample question.

[0084] In another embodiment of this application, the optimization learning of a pre-trained language model often requires a large amount of training text. Before obtaining the first training text, at least one third training text may have already been obtained. For example, multiple third training texts may have been obtained (both the multiple third training texts and the first training text are used for optimization learning of the pre-trained language model). The third training text includes at least a third sample classification text, a third sample question, multiple third sample classification labels, and prediction probability fields corresponding to each third sample classification label. The third sample question is used to prompt the predicted probability that the answer of the third sample classification text to the third sample question belongs to each third sample classification label. Furthermore, the third labeled classification label of the third sample classification text has already been obtained, and the third labeled classification label is one of the multiple third sample classification labels.

[0085] Thus, a target third training text can be determined from the obtained third training text. The target third training text includes at least a third sample classification text, a third sample question, multiple third sample classification labels, and prediction probability fields corresponding to each third sample classification label. The multiple third sample classification labels in the target third training text include a first labeled classification label, or, the multiple third sample classification labels in the target third training text include similar classification labels whose semantic similarity to the first labeled classification label is greater than a first preset similarity. The first preset similarity can be set according to actual conditions; this application does not limit it, and it can be in the form of a percentage, etc. The method for calculating the semantic similarity between classification labels can refer to existing methods for calculating semantic similarity; this application does not limit it. Then, the first sample question can be obtained based on the third sample question; for example, the third sample question can be determined as the first sample question.

[0086] In another embodiment of this application, when a first label classification tag has been obtained, the field to which the first label classification tag belongs can be determined (the first label classification tag can be regarded as one of the multiple first sample classification tags). Alternatively, when multiple first sample classification tags have been obtained (the multiple first sample classification tags all belong to the same field), the field to which the multiple first sample classification tags belong can be determined.

[0087] The domains include time domains, location domains, emotion domains, personal name domains, and place name domains. Of course, it is understood that other possible domains may also be included, but this application does not limit them and will not provide examples of them.

[0088] For example, existing domain identification methods can be used to identify the domain to which the first labeled classification tag belongs, or existing domain identification methods can be used to identify the domain to which each first sample classification tag belongs (the domains to which each first sample classification tag belongs are often the same), and this application does not limit this. Then, a first sample question for the determined domain can be generated. For example, multiple questions can be pre-set for different domains, with one question corresponding to one domain. In this way, the question corresponding to the determined domain can be obtained and used as the first sample question. Alternatively, existing generation methods can be used to generate the first sample question for the determined domain. For example, the determined domain can be input as input data into an existing generation model to obtain the question generated based on the determined domain, which is then used as the first sample question.

[0089] Furthermore, after using at least the first training text and the first labeled classification label, the pre-trained language model is optimized to perform probability prediction tasks for the prediction probability fill-in positions corresponding to each first sample classification label. After obtaining the classification model, the classification model can be put online for use, such as for online text classification.

[0090] Specifically, see Figure 4 This paper demonstrates a method for text classification applied to electronic devices, which can include cloud-based devices or devices controlled by a wide range of users. The cloud can exist in the form of servers, and devices controlled by a wide range of users can include tablets, laptops, desktop computers, mobile phones, and PDAs. The method includes:

[0091] In step S401, a first text to be processed is obtained. The first text to be processed includes at least a first online classification text, a first online question, multiple first online classification labels, and prediction probability fields corresponding to each first online classification label. The first online question is used to indicate the predicted probability that the answer of the first online classification text to the first online question belongs to each first online classification label.

[0092] In one embodiment, the first online categorized text can be text entered or submitted by the user in real time. The first online categorized text is the text that actually needs to be categorized. The first online categorized text can include user-inputted comment text and search text, etc. For example, in one scenario, if a user inputs comment text about a product, the electronic device can categorize the comment text and then recommend other products to the user based on the categorization (e.g., positive sentiment categorization or negative sentiment categorization). Thus, the comment text obtained by the electronic device can be the first online categorized text.

[0093] When the electronic device receives the first online category text, it can also obtain the first online question and multiple first online category tags. The first online question can be entered or submitted by the user in real time along with the first online category text, or it can be preset. The multiple first online category tags can also be entered or submitted by the user in real time along with the first online category text, or they can be preset. Then, the first text to be processed is obtained based on the fact that the first text to be processed includes at least the first online category text, the first online question, and multiple first online category tags. The specific process can be found in the embodiments shown later, and will not be described in detail here.

[0094] In step S402, based on the trained classification model, the predicted probabilities are predicted for filling in the prediction probability fields corresponding to each first line classification label in the first text to be processed. The predicted probabilities include the probability that the answer of the first line classification text to the first line question belongs to each first line classification label.

[0095] The trained classification model is obtained by optimizing the pre-trained language model using at least the first training text and the first labeled classification tags, performing a probability prediction task on the prediction probability fill-in positions corresponding to each first sample classification tag. The probability prediction task includes predicting the probability that the answer of the first sample classification text to the first sample question belongs to each of the first sample classification tags. The first training text includes at least the first sample classification text, the first sample question, multiple first sample classification tags, and prediction probability fill-in positions corresponding to each first sample classification tag. The first sample question is used to prompt the prediction probability that the answer of the first sample classification text to the first sample question belongs to each of the first sample classification tags. The first labeled classification tag is one of the multiple first sample classification tags. The specific optimization learning process can be found in the description of the foregoing embodiments, and will not be detailed here.

[0096] The scenarios for text classification in this application may include sentiment classification of text, classification of relationships between entities in text, and classification of user intent reflected in text, etc. Of course, other scenarios may also be included, and this application does not limit them.

[0097] The first text to be processed can be input into a trained classification model, enabling the model to perform a probability prediction task for each first online classification label. This probability prediction task includes predicting the probability that the answer to the first online question belongs to each of the first online classification labels, thus obtaining the predicted probabilities of the answer to the first online question belonging to each of the first online classification labels. Then, the electronic device can output at least the predicted probabilities that the answer to the first online question belongs to at least some of the first online classification labels. For example, it can output at least the largest predicted probability among the predicted probabilities of the answer to the first online question belonging to each of the first online classification labels. Furthermore, along with the output predicted probabilities, the corresponding online classification labels can also be output, allowing the user to know which predicted probabilities correspond to which online classification labels.

[0098] For example, given the probability that the answer to a first-line category question belongs to each of the first-line category tags, the probability that the answer to the first-line category question belongs to at least some of the first-line category tags can be displayed, allowing users to see the probability that the answer to the first-line category question belongs to at least some of the first-line category tags. Furthermore, along with the output predicted probabilities, the corresponding online category tags can also be displayed, allowing users to see which predicted probabilities correspond to which online category tags.

[0099] The text to be processed input into the classification model of this application includes at least online classification text, online questions, multiple online classification labels, and prediction probability fields corresponding to each online classification label. That is, the classification model will know which classification labels to classify for the online classification text to be classified, and the classification model can know the classification range of the online classification text, for example, classification within the range of multiple online classification labels in the text to be processed, rather than classification within the range of other classification labels. Moreover, the classification model has the ability to "predict the predicted probability that the answer of the online classification text to the online question belongs to each online classification label". Even if the classification model has not previously perceived the multiple online classification labels in the text to be processed, the presence of multiple online classification labels in the text to be processed, and the prediction probability fields corresponding to each online classification label, can guide the classification model to predict the predicted probability that the answer of the online classification text to the online question belongs to each online classification label, rather than guiding the classification model to generate classification labels. This can overcome the problem of low classification accuracy in zero-shot scenarios, thus making the classification model of this application more accurate in zero-shot scenarios.

[0100] Secondly, the classification model of this application can be shared by multiple service platforms. Each service platform no longer needs to train the classification model according to its own specific needs, and each service platform does not need to master the training technology of the classification model, thus lowering the technical threshold. For example, even if the classification requirements of each service platform are different, and even if the classification labels required for text classification are different, each service platform can directly deploy the classification model of this application. When it needs to classify online classified text, the text to be processed input to the classification model includes "online classified text, online question, multiple online classification labels, and the prediction probability fill-in position corresponding to each online classification label". This allows the classification model to predict the prediction probability of the answer of the online classified text to the online question belonging to each online classification label, enabling cold start use of the classification model for each service platform. Since each service platform no longer needs to train the classification model, it does not need to obtain sample classified text, nor does it need to label the sample classified text, reducing the labeling workload of each service platform and reducing labor costs.

[0101] Furthermore, the classification model of this application predicts the probability that the answer of the online classified text to the online question belongs to each of the known online classification labels, without generating classification labels (that is, without performing the task of generating text). Therefore, the classification model of this application may not have a classification layer, which simplifies the structure of the classification model. For example, it reduces the network parameters in the classification model, which makes the system resources required to load and run the classification model less, and can improve the computing speed, thereby improving the classification efficiency.

[0102] In one embodiment of this application, when multiple first online classification labels are all user input (e.g., input by the user in real time, such as inputting the first online classification labels along with the first text to be classified), the probability that the answer of the first online classification text to the first online question belongs to each first online classification label can be displayed at least. In addition, along with the output prediction probability, the online classification label corresponding to each prediction probability can also be displayed so that the user can know which prediction probabilities correspond to which online classification labels.

[0103] Alternatively, in another embodiment, if some of the first online classification labels among the multiple first online classification labels are user input (e.g., input by the user in real time, such as inputting the first online classification label along with the first text to be classified), then the remaining first online classification labels among the multiple first online classification labels, excluding those partial first online classification labels, are often automatically expanded by the electronic device rather than input by the user. That is, the remaining first online classification labels are often used to assist the electronic device in using a trained classification model to predict the probability that the answer of the first online classification text to the first online question belongs to that partial first online classification label.

[0104] For example, it enables the use of a trained classification model to predict the probability that the answer of the first-line classified text to the first-line question belongs to each first-line classification label, thereby obtaining the probability that the answer of the first-line classified text to the first-line question belongs to that part of the first-line classification labels.

[0105] However, users often don't need to see the probability that the answer to the first-line category text for the first-line question belongs to the remaining first-line category tags. Therefore, to avoid showing users unnecessary content and to prevent interfering with their viewing of the probability that the answer to the first-line category text for the first-line question belongs to that particular first-line category tag, it's possible to display the probability that the answer to that particular first-line category text for the first-line question belongs to at least some of the first-line category tags, instead of displaying the probability that the answer to the first-line category text for the first-line question belongs to each of the remaining first-line category tags individually.

[0106] In one embodiment of this application, when predicting the predicted probabilities to be filled in the prediction probability fields corresponding to the classification labels on each first line in the first text to be processed, based on a trained classification model, at least one character in the classification text on the first line in the first text to be processed can also be masked to obtain masked processed text. The predicted probabilities to be filled in the prediction probability fields corresponding to the classification labels on each first line in the masked processed text are then predicted based on the trained classification model.

[0107] For example, at least one character in the first online classification text of the first text to be processed can be masked, and then the masked first text to be processed can be used as the masked text to be processed. The mask may include replacing at least one character in the first online classification text with a preset character (e.g., a mask token). The preset character can be set according to the actual situation, such as including [XXX], (YYY), or "ZZZ", etc., which is not limited in this application. Alternatively, the mask may include deleting at least one character in the first online classification text. Wherein, the masked text to be processed contains at least the unmasked content of the first online classification text (masked classification text), the first online question, multiple first online classification labels, and prediction probability fields corresponding to each first online classification label. The first online question is used to indicate the predicted probability that the answer of the masked classification text to the first online question belongs to each first online classification label.

[0108] In one embodiment of this application, see [link to embodiment]. Figure 5 The process of obtaining the first text to be processed in step S401 includes:

[0109] In step S501, the first online classification text is obtained, multiple first online classification labels are obtained, and the first online questions are obtained.

[0110] The first online category text can be submitted or entered by the user to the electronic device in real time, thus the first online category text submitted or entered by the user to the electronic device in real time can be obtained. Multiple first online category tags can be submitted or entered by the user to the electronic device in real time along with the first online category text, thus the multiple first online category tags submitted or entered by the user to the electronic device in real time can be obtained. The first online question can be manually submitted or entered by the user to the electronic device in real time along with the first online category text, thus the first online question submitted or entered by the user to the electronic device in real time can be obtained. Furthermore, the methods for obtaining the first online question and multiple first online category tags can also be found in the embodiments shown later, and will not be detailed here.

[0111] In step S502, a first text to be processed is generated based on at least the first online classification text, multiple first online classification labels, and the first online question. The first text to be processed also includes prediction probability fields corresponding to each first online classification label.

[0112] The positional relationships between online classification text, multiple online classification labels, and online questions can be pre-defined, and this application does not limit the specific positional relationships. The prediction probability fields for each online classification label are different. In one example, based on the positional relationships between online classification text, multiple online classification labels, and online questions, the first online classification text, multiple first online classification labels, and the first online question can be combined, and corresponding prediction probability fields can be added to the left or right of each first online classification label to obtain the first text to be processed.

[0113] In one embodiment of this application, the first online category text, multiple first online category tags, and first online questions can be input by the user in real time. For example, the electronic device can display an input page, which includes a category text input box, a category tag input box, and a question input box. For example, when the electronic device receives a category input operation from the user, the input page can be displayed on the screen of the electronic device. In this way, it can support users to submit first online category text to the electronic device through the category text input box, support users to submit multiple first online category tags to the electronic device through the category tag input box, and support users to submit first online questions to the electronic device through the question input box. For example, the electronic device can obtain the content entered in the category text input box and then obtain the first online category text based on the content entered in the category text input box. As another example, the electronic device can obtain the content entered in the category tag input box and then obtain multiple first online category tags based on the content entered in the category tag input box. As yet another example, the electronic device can obtain the content entered in the question input box and then obtain the first online question based on the content entered in the question input box. The content entered in the category text input box can be regarded as a single text, i.e., the first online category text, so the content entered in the category text input box can be determined as the first online category text.

[0114] Secondly, the content entered in the question input box can be considered as a single text, i.e., the first-line question. Therefore, the content entered in the question input box can be identified as the first-line question. Furthermore, since multiple first-line category tags need to be obtained, meaning each first-line category tag is treated as different text, in order for the electronic device to recognize which content entered in the category tag input box belongs to one category tag and which belongs to another, the user can manually separate the different category tags using a preset separator. For example, the preset separator can include commas, semicolons, periods, etc., or it can include other symbols, such as percent signs, double quotes, and single quotes, etc. This application does not limit this. Thus, in this case, there are multiple contents entered in the category tag input box, and these contents are separated by preset separators. Therefore, when obtaining multiple first-line category tags based on the content entered in the category tag input box, each content separated by the preset separator can be identified as a first-line category tag, resulting in multiple different first-line category tags. In addition, to prevent users from being unaware that they need to manually separate the different category tags by using a preset separator, and thus to prevent electronic devices from being unable to identify which content in the category tag input box is one category tag and which content is another category tag, in another embodiment of this application, a prompt message can also be displayed on the input page. The prompt message is used to prompt users to manually separate the different category tags by using a preset separator, so as to prompt users to manually separate the different category tags by using a preset separator.

[0115] In some cases, the first online category label that a user submits or enters to an electronic device in real time is not necessarily multiple first online category labels, but rather a subset of multiple first online category labels.

[0116] Since multiple online classification labels are needed in the process of classifying online classified text using a trained classification model, in order to classify the first online classification label using the trained classification model, the first online classification label can be automatically expanded based on the obtained part of the online classification label. For example, at least one more first online classification label can be expanded. Then, multiple first online classification labels can be obtained based on the part of the first online classification label submitted or input by the user to the electronic device in real time and the automatically expanded at least one first online classification label.

[0117] Specifically, in one embodiment, regarding the acquisition of multiple first online classification labels in step S501, see [link to relevant documentation]. Figure 6 This can be achieved through the following process, including:

[0118] In step S601, if a portion of the multiple first online classification labels have been obtained, at least one classification label is obtained based on that portion of the first online classification labels. The at least one classification label is different from that portion of the first online classification labels.

[0119] In one embodiment of this application, the domain to which the first online classification label belongs can be determined. The domain includes time domain, location domain, emotion domain, personal name domain, and place name domain. Of course, it is understood that other possible domains may also be included, which this application does not limit and will not list in detail. For example, existing domain identification methods can be used to identify the domain to which the first online classification label belongs (if there are multiple first online classification labels, the domains to which they belong are often the same). This application does not limit the specific domain identification method. Then, at least one classification label for that domain, different from the first online classification label, can be generated. For example, multiple classification label sets can be pre-set for different domains. A classification label set corresponding to one domain includes multiple classification labels for that domain. Thus, at least one classification label different from the first online classification label can be obtained from the classification label set corresponding to the domain to which the first online classification label belongs. Alternatively, at least one classification label in the domain that is different from the first online classification label of that part can be generated using an existing generation method. For example, the first online classification label of that part can be input into an existing generation model to obtain at least one classification label generated by the existing generation model based on the input first online classification label of that part.

[0120] In another embodiment of this application, multiple classification tag sets can be pre-set, each set including multiple classification tags. The classification tags in different sets are applicable to different scenarios. For example, one set might include "positive" and "negative" for sentiment classification; another set might include "positive" and "negative" for sentiment classification; yet another set might include "good-looking" and "not good-looking" for sentiment classification; and yet another set might include "Beijing," "Guangzhou," and "Shanghai" for location classification. Thus, a target classification tag set can be searched among the pre-set multiple sets. Each set includes multiple classification tags, and the target set includes either the first set of online classification tags or similar classification tags whose semantic similarity to the first set of online classification tags is greater than a third preset similarity. The third preset similarity can be set according to actual conditions; this application does not limit it, and it can be in the form of a percentage, etc. The method for calculating the label similarity between the set of category labels and the category labels on the first line of the part can refer to existing methods for calculating label similarity, and this application does not limit it. For example, the number of identical category labels between the set of category labels and the category labels on the first line of the part can be counted, and then the ratio between this number and the minimum / maximum value of "the number of category labels in the set of category labels and the number of category labels on the first line of the part ...

[0121] In another embodiment of this application, in scenarios where a trained classification model is used to classify online classified text, the trained classification model is often used to classify a large number of online classified texts. Before obtaining the first text to be processed, at least one second text to be processed may have already been obtained. For example, multiple second texts to be processed may have been obtained and processed using the trained classification model. The second text to be processed includes at least a second online classified text, a second online question, multiple second online classification labels, and prediction probability fields corresponding to each second online classification label. The second online question is used to indicate the predicted probability that the answer of the second online classified text to the second online question belongs to each second online classification label. In this way, the target second text to be processed can be determined from the obtained second texts to be processed. The target second text to be processed includes at least a second online classified text, a second online question, multiple second online classification labels, and prediction probability fields corresponding to each second online classification label. The target second text to be processed includes multiple second online category tags, which include the first online category tags; or, the target second text to be processed includes multiple second online category tags, which include similar category tags whose semantic similarity to the first online category tags is greater than a third preset similarity. The third preset similarity can be set according to actual conditions, and this application does not limit it; it can be in the form of a percentage, etc. The method for calculating the tag similarity between multiple second online category tags and the first online category tags can refer to existing methods for calculating tag similarity, and this application does not limit it. For example, the number of identical category tags between multiple second online category tags and the first online category tags can be counted, and then the ratio between this number and the minimum / maximum value of "the number of category tags in multiple second online category tags and the number of category tags in the first online category tags" can be calculated.

[0122] Then, at least one category label is obtained from the multiple second online category labels other than the first online category labels or similar category labels. For example, the category labels other than the first online category labels or similar category labels from the multiple second online category labels are determined as at least one category label.

[0123] In step S602, multiple first online classification labels are obtained based on the first online classification label of this part and at least one obtained classification label.

[0124] For example, the first online category label and the at least one category label obtained in this part constitute a total of multiple category labels. Thus, the at least one category label obtained can be determined as the first online category label, and by combining the first online category labels in this part, multiple first online category labels can be obtained.

[0125] This embodiment enables users to categorize their input first-line classification text without manually entering multiple first-line classification tags in real time. Instead, users can enter only some first-line tags, and the electronic device can automatically supplement the classification tags to obtain multiple first-line classification tags. The obtained multiple first-line classification tags are compatible with the first-line classification text, which can meet the user's classification needs for the first-line classification text, reduce the user's operational complexity, and improve the user experience.

[0126] In some cases, users may not submit or input the first online classification label to the electronic device in real time. Since multiple online classification labels are needed when classifying online classified text using a trained classification model, a third online classification label can be automatically added to enable classification using the trained model. For example, in another embodiment of this application, upon obtaining the first online classified text, a first online question may also be obtained. This first online question could be something the user inputs or submits to the electronic device in real time along with the first online classification, or it could be a pre-set first online question.

[0127] Furthermore, various questions that may be used in subsequent classification scenarios can be statistically analyzed in advance. For any question analyzed, multiple category labels applicable to that question can be statistically analyzed, and these applicable category labels can be combined into a category label set. Then, the question and the combined category label set are combined into a corresponding table entry and stored in the first correspondence between questions and category label sets. Thus, given the first online questions, questions with a similarity greater than a second preset similarity can be found in the first correspondence between questions and category label sets. Each category label set in the first correspondence includes multiple category labels. The second preset similarity can be set according to actual conditions, and this application does not limit it; it can be in the form of a percentage, etc. The method for calculating the semantic similarity between questions can refer to the existing methods for calculating semantic similarity, and this application does not limit it. Then, the category label set corresponding to the found question can be found in the first correspondence. At least one category label is obtained based on the category labels in the found category label set; for example, all category labels in the found category label set are determined as multiple first online category labels.

[0128] In some cases, users may not submit or input a first online question to the electronic device in real time. Since multiple online classification labels are needed in the process of classifying online text using a trained classification model, the first online question can be automatically supplemented in order to enable the classification of the first online classification label using the trained classification model. Specifically, for obtaining the first online question in step S501, it can be implemented through the following embodiments.

[0129] For example, various questions that may be used in subsequent classification scenarios can be counted in advance. For any question counted, multiple classification labels applicable to the question can be counted, and the classification labels applicable to the question can be combined into a classification label set. Then, the question and the appropriate classification label set can be combined into a corresponding table entry and stored in the first correspondence between the question and the classification label set.

[0130] Thus, in one embodiment of this application, when multiple first online category labels have been obtained, a target category label set can be searched in a first correspondence between questions and category label sets, where each category label set in the first correspondence includes multiple category labels. The label similarity between the target category label set and the multiple first online category labels is greater than a third preset similarity. The third preset similarity can be set according to actual conditions, and this application does not limit it; it can be in the form of a percentage, etc. The method for calculating the label similarity between the category label set and the multiple category labels can refer to existing methods for calculating label similarity, and this application does not limit it. For example, the number of identical category labels between the category label set and the multiple category labels can be counted, and then the ratio between this number and the minimum / maximum value of "the number of category labels in the category label set and the number of category labels in the multiple category labels" can be calculated. Then, questions corresponding to the target category label set can be searched in the first correspondence. First online questions are obtained based on the found questions; for example, the found questions are determined as first online questions.

[0131] Alternatively, if some of the first online category tags from multiple online category tags have been obtained, the target category tag set can be searched in the first correspondence between the question and the category tag set. Each category tag set in the first correspondence includes multiple category tags. The target category tag set includes the portion of the first online category tags, or the target category tag set includes similar category tags whose semantic similarity to the portion of the first online category tags is greater than a third preset similarity. The third preset similarity can be set according to actual conditions, and this application does not limit it; it can be in the form of a percentage, etc. The method for calculating the tag similarity between the category tag set and multiple category tags can refer to existing methods for calculating tag similarity, and this application does not limit it. For example, the number of identical category tags between the category tag set and multiple category tags can be counted, and then the ratio between this number and the minimum / maximum value of "the number of category tags in the category tag set and the number of category tags in multiple category tags" can be calculated. Then, the question corresponding to the target category tag set can be searched in the first correspondence. The first online question is obtained based on the found question; for example, the found question is determined as the first online question.

[0132] In another embodiment of this application, in scenarios where a trained classification model is used to classify online classified text, the trained classification model is often used to classify a large number of online classified texts separately. Before obtaining the first text to be processed, at least one second text to be processed may have already been obtained. For example, multiple second texts to be processed may have been obtained and processed using the trained classification model. The second text to be processed includes at least a second online classified text, a second online question, multiple second online classification labels, and prediction probability fields corresponding to each second online classification label. The second online question is used to prompt the predicted probability that the answer of the second online classified text to the second online question belongs to each second online classification label.

[0133] Thus, the target third text to be processed can be determined from the obtained third text to be processed. The target third text to be processed includes at least the third online classification text, the third online question, multiple third online classification labels, and the prediction probability input field corresponding to each third online classification label. The multiple third online classification labels in the target third text to be processed include the first online classification labels, or the multiple third online classification labels in the target third text to be processed include similar classification labels with a semantic similarity greater than a third preset similarity with the first online classification labels. The third preset similarity can be set according to the actual situation, and this application does not limit it. It can be in the form of a percentage, etc. The method for calculating the label similarity between multiple second online classification labels and the first online classification labels can refer to the currently existing methods for calculating label similarity, and this application does not limit it. For example, the number of identical classification labels between multiple second online classification labels and the first online classification labels can be counted, and then the ratio between this number and the minimum / maximum value of "the number of classification labels in multiple second online classification labels and the number of classification labels in the first online classification labels" can be calculated, etc.

[0134] Then, the first online question can be obtained based on the third online question; for example, the third online question can be identified as the first online question.

[0135] In another embodiment of this application, when at least some of the first online classification labels among a plurality of first online classification labels have been obtained, the field to which the portion of the first online classification labels belong can be determined; or, when a plurality of first online classification labels have been obtained, the field to which the plurality of first online classification labels belong can be determined.

[0136] The domains include time domains, location domains, emotion domains, personal name domains, and place name domains. Of course, it is understood that other possible domains may also be included, but this application does not limit them and will not provide examples of them.

[0137] For example, existing domain identification methods can be used to identify the domain to which the first online classification label belongs, or existing domain identification methods can be used to identify the domain to which multiple first online classification labels belong (the domains to which the first online classification labels belong are often the same), and this application does not limit this.

[0138] Then, first-line questions for the determined domain can be generated. For example, multiple questions can be pre-set for different domains, with one question corresponding to each domain. In this way, the questions corresponding to the determined domains can be obtained and used as the first-line questions. Alternatively, existing generation methods can be used to generate first-line questions for the determined domains. For example, the determined domains can be input into an existing generation model to obtain questions generated based on the determined domains, which can then be used as the first-line questions.

[0139] In some cases, different users may have different needs for text categorization. For example, different users may have different categorization requirements for the text they want to categorize. Some users may need text categorized by sentiment, including positive and negative sentiment. Some users may need text categorized by location, including location A and location B. Some users may need text categorized by time, such as 9 o'clock and 12 o'clock. Statistical analysis of a large number of users reveals that each user's specific text categorization requirements tend to be relatively fixed and change little. Each user's specific text categorization requirements involve questions and multiple category tags. Thus, the questions and multiple category tags involved in each user's specific text categorization requirements often differ.

[0140] Therefore, for any given user, online questions relevant to that user can be pre-calculated. These questions can be submitted by the user themselves via electronic devices, or they can be derived from contextual analysis of the user's historical text classification. This analysis can be automated by electronic devices or conducted manually by staff. Then, the user's identity information can be obtained, and a corresponding table entry can be created by mapping the user's identity information to their online questions, storing this information in a mapping relationship between user identity information and online questions. The same process applies to every other user.

[0141] Thus, in step S501, when obtaining the first online question, the identity information of the user who input the first online category text can be obtained. For example, the electronic device can obtain the account of a user already logged in on the electronic device and use it as the user's identity information. Then, based on the user's identity information, the online question suitable for the user is obtained. For example, in the mapping relationship between user identity information and suitable online questions, the online question corresponding to the user's identity information is found and used as the suitable online question. Then, the first online question can be obtained based on the suitable online question; for example, the suitable online question is determined as the first online question.

[0142] This embodiment enables users to automatically obtain online questions when they need to categorize the first online text they input, without having to manually input the questions in real time. The obtained questions are suitable for use and can meet the user's categorization needs, reducing the complexity of user operations and improving the user experience.

[0143] For any given user, multiple online category tags suitable for that user can be pre-collected. These tags can be submitted by the user voluntarily to their electronic device, or obtained through analysis of the user's historical text classification scenarios. This analysis can be automated by the electronic device or conducted manually by staff. Then, the user's identity information can be obtained, and a corresponding table entry can be created by mapping the user's identity information to the multiple online category tags, storing this information in a mapping relationship between user identity information and these online category tags. The same process applies to every other user.

[0144] Thus, when obtaining multiple first online category tags in step S501, the identity information of the user who inputs the first online category text can be obtained. For example, the electronic device can obtain the account of a user already logged into the electronic device and use it as the user's identity information. Then, based on the user's identity information, multiple online category tags suitable for the user are obtained. For example, in the mapping relationship between the user's identity information and the multiple online category tags suitable for the user, multiple online category tags corresponding to the user's identity information are found and used as the multiple online category tags suitable for the user. Then, multiple first online category tags can be obtained based on the multiple online category tags suitable for the user. For example, the multiple online category tags suitable for the user are determined as multiple first online category tags.

[0145] This embodiment enables users to automatically acquire multiple online classification tags instead of manually inputting them in real time when they need to categorize the first online classification text they input. The acquired online classification tags are suitable for use and can meet the user's classification needs, reducing the user's operational complexity and improving the user experience.

[0146] It should be noted that, for the sake of simplicity, the method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, because according to this application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also understand that the embodiments described in the specification are all optional embodiments, and the actions involved are not necessarily required by this application.

[0147] Reference Figure 7 The diagram illustrates a structural block diagram of an apparatus for training a classification model, comprising: a first acquisition module 11 for acquiring a pre-trained language model; a second acquisition module 12 for acquiring a first training text, the first training text including at least a first sample classification text, a first sample question, multiple first sample classification labels, and prediction probability fill-in positions corresponding to each first sample classification label; the first sample question is used to prompt the prediction probability that the answer of the first sample classification text to the first sample question belongs to each first sample classification label; a third acquisition module 13 for acquiring a first labeled classification label of the first sample classification text, the first labeled classification label being one of multiple first sample classification labels; and an optimization learning module 14 for optimizing the pre-trained language model using at least the first training text and the first labeled classification label to perform probability prediction tasks for the prediction probability fill-in positions corresponding to each first sample classification label, thereby obtaining a classification model, wherein the probability prediction task includes the task of predicting the prediction probability that the answer of the first sample classification text to the first sample question belongs to each first sample classification label.

[0148] The optimization learning module includes: a masking unit, used to mask at least one character in the first sample classification text in the first training text to obtain a masked training text; and an optimization learning unit, used to optimize the pre-trained language model by using at least the masked training text and the first labeled classification label for the probability prediction task of the prediction probability fill-in position corresponding to each first sample classification label, to obtain a classification model.

[0149] The second acquisition module includes: a first acquisition unit for acquiring first sample classification text; a second acquisition unit for acquiring multiple first sample classification labels; a third acquisition unit for acquiring first sample questions; and a first generation unit for generating first training text based on at least the first sample classification text, multiple first sample classification labels, and the first sample questions. The first training text also includes prediction probability fields corresponding to each first sample classification label.

[0150] The second acquisition unit includes: a first acquisition subunit, used to acquire at least one classification label based on the first annotation classification label when the first annotation classification label of the first sample classification text has been obtained; the at least one classification label is different from the first annotation classification label; and a second acquisition subunit, used to acquire multiple first sample classification labels based on the first annotation classification label and the acquired at least one classification label.

[0151] The first acquisition subunit is specifically used for: determining the domain to which the first labeled classification label belongs; generating at least one classification label for the domain that is different from the first labeled classification label. Alternatively, the first acquisition subunit is specifically used for: searching for a target classification label set among a plurality of pre-set classification label sets; each classification label set includes a plurality of classification labels, and the target classification label set includes the first labeled classification label, or the target classification label set includes similar classification labels whose semantic similarity to the first labeled classification label is greater than a first preset similarity; acquiring at least one classification label based on the classification labels in the target classification label set other than the first labeled classification label or similar classification labels. Alternatively, the first acquisition subunit is specifically used to: determine a target second training text from the obtained second training text, wherein the target second training text includes at least a second sample classification text, a second sample question, multiple second sample classification labels, and prediction probability fill-in positions corresponding to each second sample classification label; the multiple second sample classification labels in the target second training text include a first labeled classification label, or the multiple second sample classification labels in the target second training text include similar classification labels whose semantic similarity to the first labeled classification label is greater than a first preset similarity; and acquire at least one classification label based on the second sample classification labels other than the first labeled classification label or similar classification labels among the multiple second sample classification labels.

[0152] The second acquisition unit includes: a first search subunit, used to find, in the first correspondence between questions and classification label sets, a question whose similarity to the first sample question is greater than a second preset similarity, given that the first sample question has been obtained; each classification label set in the first correspondence includes multiple classification labels; a second search subunit, used to find the classification label set corresponding to the found question in the first correspondence; and a third acquisition subunit, used to acquire multiple first sample classification labels based on the classification labels in the found classification label sets.

[0153] The third acquisition unit includes: a third search subunit, used to search for a target category label set in the first correspondence between questions and category label sets, given that multiple first sample category labels have been obtained; each category label set in the first correspondence includes multiple category labels; the label similarity between the target category label set and the multiple first sample category labels is greater than a third preset similarity; a fourth search subunit, used to search for a question corresponding to the target category label set in the first correspondence; and a fourth acquisition subunit, used to acquire a first sample question based on the found question. Alternatively, the third acquisition unit includes: a fifth search subunit, used to search for a target category label set in the first correspondence between questions and category label sets, given that first labeled category labels have been obtained; each category label set in the first correspondence includes multiple category labels; the target category label set includes the first labeled category label, or the target category label set includes similar category labels whose semantic similarity to the first labeled category label is greater than a first preset similarity; a sixth search subunit, used to search for a question corresponding to the target category label set in the first correspondence; and a fifth acquisition subunit, used to acquire a first sample question based on the found question. Alternatively, the third acquisition unit includes: a first determining subunit, used to determine a target third training text from the obtained third training text, wherein the target third training text includes at least a third sample classification text, a third sample question, multiple third sample classification labels, and prediction probability fields corresponding to each third sample classification label; the multiple third sample classification labels in the target third training text include a first labeled classification label, or the multiple third sample classification labels in the target third training text include similar classification labels whose semantic similarity to the first labeled classification label is greater than a first preset similarity; and a sixth acquisition subunit, used to obtain a first sample question based on the third sample question. Alternatively, the third acquisition unit includes: a second determining subunit, used to determine the domain to which the first labeled classification label belongs when the first labeled classification label has been obtained, or, when multiple first sample classification labels have been obtained, to determine the domain to which multiple first sample classification labels belong; and a first generating subunit, used to generate a first sample question for the determined domain.

[0154] In this application, sample classification text, sample questions, multiple sample classification labels, and prediction probability fill-in positions corresponding to each sample classification label can be combined to obtain training text. The labeled classification labels of the sample classification text are used to supervise the training of "the predicted probabilities filled in at the prediction probability fill-in positions corresponding to each sample classification label according to the sample questions". This enables the pre-trained language model to be optimized for the probability prediction task of the prediction probability fill-in positions corresponding to each sample classification label, thereby obtaining a classification model. This results in better performance (e.g., higher prediction accuracy) of the classification model on the task of predicting the prediction probability that the answer to the question belongs to each classification label. Furthermore, the text to be processed input into the classification model of this application includes at least online classification text, online questions, multiple online classification labels, and prediction probability fields corresponding to each online classification label. That is, the classification model will know which classification labels to classify for the online classification text to be classified this time, and the classification model can know the classification range of the online classification text this time. For example, it is classified within the range of multiple online classification labels in the text to be processed, rather than within the range of other classification labels. Moreover, the classification model has the ability to "predict the prediction probability of the answer of the online classification text to the online question belonging to each online classification label". Even if the classification model has not previously perceived the multiple online classification labels in the text to be processed, the presence of multiple online classification labels in the text to be processed and the prediction probability fields corresponding to each online classification label can guide the classification model to predict the prediction probability of the answer of the online classification text to the online question belonging to each online classification label, rather than guiding the classification model to generate classification labels. This can overcome the problem of low classification accuracy in zero-shot scenarios, thus making the classification model of this application more accurate in zero-shot scenarios.

[0155] Secondly, the classification model of this application can be shared by multiple service platforms. Each service platform no longer needs to train the classification model according to its own specific needs, and each service platform does not need to master the training technology of the classification model, thus lowering the technical threshold. For example, even if the classification requirements of each service platform are different, and even if the classification labels required for text classification are different, each service platform can directly deploy the classification model of this application. When it needs to classify online classified text, the text to be processed input to the classification model includes "online classified text, online question, multiple online classification labels, and the prediction probability fill-in position corresponding to each online classification label". This allows the classification model to predict the prediction probability of the answer of the online classified text to the online question belonging to each online classification label, enabling cold start use of the classification model for each service platform. Since each service platform no longer needs to train the classification model, it does not need to obtain sample classified text, nor does it need to label the sample classified text, reducing the labeling workload of each service platform and reducing labor costs. Furthermore, the classification model of this application predicts the probability that the answer of the online classified text to the online question belongs to each of the known online classification labels, without generating classification labels (that is, without performing the task of generating text). Therefore, the classification model of this application may not have a classification layer, which simplifies the structure of the classification model. For example, it reduces the network parameters in the classification model, which makes the system resources required to load and run the classification model less, and can improve the computing speed, thereby improving the classification efficiency.

[0156] Reference Figure 8The diagram illustrates a structural block diagram of a text classification device, comprising: a fourth acquisition module 21, used to acquire a first text to be processed, the first text to be processed including at least a first online classification text, a first online question, multiple first online classification labels, and prediction probability fields corresponding to each first online classification label; the first online question is used to prompt the predicted probability that the answer of the first online classification text to the first online question belongs to each first online classification label; a prediction module 22, used to predict the predicted probability to be filled in the prediction probability fields corresponding to each first online classification label in the first text to be processed based on a trained classification model, the predicted probability including the probability that the answer of the first online classification text to the first online question belongs to each first online classification label; wherein, the trained... The trained classification model is obtained by optimizing the pre-trained language model using at least the first training text and the first labeled classification label, and performing a probability prediction task for the prediction probability fill-in position corresponding to each first sample classification label. The probability prediction task includes the task of predicting the predicted probability that the answer of the first sample classification text to the first sample question belongs to each first sample classification label. The first training text includes at least the first sample classification text, the first sample question, multiple first sample classification labels, and the prediction probability fill-in position corresponding to each first sample classification label. The first sample question is used to prompt the predicted probability that the answer of the first sample classification text to the first sample question belongs to each first sample classification label. The first labeled classification label is one of multiple first sample classification labels.

[0157] It also includes: a display module, used to display at least the predicted probability that the answer of the first online categorized text to the first online question belongs to at least some of the first online categorized labels. The display module includes: a first display unit, used to display the probability that the answer of the first online categorized text to the first online question belongs to each of the first online categorized labels when all of the multiple first online categorized labels are user input; or, a second display unit, used to display the probability that the answer of the first online categorized text to the first online question belongs to at least some of the first online categorized labels when some of the multiple first online categorized labels are user input.

[0158] The fourth acquisition module includes: a fourth acquisition unit for acquiring first online classification text, a fifth acquisition unit for acquiring multiple first online classification labels, and a sixth acquisition unit for acquiring first online questions; a second generation unit for generating first text to be processed based on at least the first online classification text, multiple first online classification labels, and first online questions; the first text to be processed also includes prediction probability fields corresponding to each first online classification label.

[0159] The fourth, fifth, and sixth acquisition units include: a first display subunit for displaying an input page, which includes a category text input box, a category label input box, and a question input box; a seventh acquisition subunit for acquiring the content entered in the category text input box and acquiring the first online category text based on the content entered in the category text input box; an eighth acquisition subunit for acquiring the content entered in the category label input box and acquiring multiple first online category labels based on the content entered in the category label input box; and a ninth acquisition subunit for acquiring the content entered in the question input box and acquiring the first online question based on the content entered in the question input box.

[0160] The content entered in the category label input box can be multiple, and the multiple contents are separated by a preset separator; the eighth acquisition sub-unit is specifically used to: determine each of the contents entered in the category label input box separated by the preset separator as the first online category label.

[0161] The fourth acquisition unit, the fifth acquisition unit, and the sixth acquisition unit further include: a second display subunit, used to display prompt information on the input page, the prompt information being used to suggest separating the different category labels entered by a preset separator.

[0162] The fifth acquisition unit includes: a tenth acquisition subunit, used to acquire at least one classification label based on a portion of the multiple first online classification labels, wherein the at least one classification label is different from the portion of the first online classification labels; and an eleventh acquisition subunit, used to acquire multiple first online classification labels based on the portion of the first online classification labels and the acquired at least one classification label.

[0163] The tenth acquisition subunit is specifically used to: determine the domain to which the first online classification label belongs; and generate at least one classification label for the domain that is different from the first online classification label. Alternatively, the tenth acquisition subunit is specifically used to: search for a target classification label set among a plurality of pre-set classification label sets; each classification label set includes a plurality of classification labels, and the target classification label set includes the first online classification label, or the target classification label set includes similar classification labels whose semantic similarity to the first online classification label is greater than a third preset similarity; and acquire at least one classification label based on the classification labels in the target classification label set other than the first online classification label or similar labels. Alternatively, the tenth acquisition subunit is specifically used for: determining the target second text to be processed in the obtained second text to be processed, wherein the target second text to be processed includes at least a second online classification text, a second online question, multiple second online classification labels, and prediction probability filling positions corresponding to each second online classification label; the multiple second online classification labels in the target second text to be processed include the first online classification label, or the multiple second online classification labels in the target second text to be processed include similar classification labels with a semantic similarity greater than a third preset similarity with the first online classification label; and acquiring at least one classification label based on the second online classification labels other than the first online classification label or similar classification labels among the multiple second online classification labels.

[0164] The fifth acquisition unit includes: a seventh search subunit, used to, given the first online questions, search for questions with a similarity greater than a second preset similarity in the first correspondence between questions and category tag sets; each category tag set in the first correspondence includes multiple category tags; an eighth search subunit, used to search for the category tag set corresponding to the found questions in the first correspondence; and a twelfth acquisition subunit, used to acquire multiple first online category tags based on the category tags in the found category tag sets. Alternatively, the fifth acquisition unit includes: a nineteenth acquisition subunit, used to acquire the identity information of the user who inputs the first online category text; a twentieth acquisition subunit, used to acquire multiple online category tags suitable for the user based on the user's identity information; and a twenty-first acquisition subunit, used to acquire multiple first online category tags based on the multiple first online category tags suitable for the user.

[0165] The sixth acquisition unit includes: a ninth search subunit, used to search for a target category label set in the first correspondence between questions and category label sets when multiple first online category labels have been obtained, wherein each category label set in the first correspondence includes multiple category labels; the label similarity between the target category label set and the multiple first online category labels is greater than a third preset similarity; a tenth search subunit, used to search for questions corresponding to the target category label set in the first correspondence; and a thirteenth acquisition subunit, used to acquire first online questions based on the found questions. Alternatively, the sixth acquisition unit includes: an eleventh search subunit, used to search for a target category tag set in a first correspondence between questions and category tag sets, given that a portion of the first online category tags from multiple online category tags have been obtained; each category tag set in the first correspondence includes multiple category tags; the target category tag set includes the portion of the first online category tags, or the target category tag set includes similar category tags whose semantic similarity to the portion of the first online category tags is greater than a third preset similarity; a second search subunit, used to search for questions corresponding to the target category tag set in the first correspondence; and a fourteenth acquisition subunit, used to acquire first online questions based on the found questions. Alternatively, the sixth acquisition unit includes: a third determining subunit, used to determine a target third text to be processed from the obtained third text to be processed, wherein the target third text to be processed includes at least a third online classification text, a third online question, multiple third online classification labels, and prediction probability fields corresponding to each third online classification label; the multiple third online classification labels in the target third text to be processed include the first online classification labels, or the multiple third online classification labels in the target third text to be processed include similar classification labels whose semantic similarity to the first online classification labels is greater than a third preset similarity; a fifteenth acquisition subunit, used to obtain a first online question based on the third online question. Alternatively, the sixth acquisition unit includes: a fourth determining subunit, used to determine the domain to which at least some of the first online classification labels belong when at least some of the first online classification labels have been obtained; a second generating subunit, used to generate a first sample question for the determined domain. Alternatively, the sixth acquisition unit may include: a sixteenth acquisition subunit, used to acquire the identity information of the user who inputs the first online classification text; a seventeenth acquisition subunit, used to acquire the online question suitable for the user based on the user's identity information; and an eighteenth acquisition subunit, used to acquire the first online question based on the online question suitable for the user.

[0166] The text to be processed input into the classification model of this application includes at least online classification text, online questions, multiple online classification labels, and prediction probability fields corresponding to each online classification label. That is, the classification model will know which classification labels to classify for the online classification text to be classified, and the classification model can know the classification range of the online classification text, for example, classification within the range of multiple online classification labels in the text to be processed, rather than classification within the range of other classification labels. Moreover, the classification model has the ability to "predict the predicted probability that the answer of the online classification text to the online question belongs to each online classification label". Even if the classification model has not previously perceived the multiple online classification labels in the text to be processed, the presence of multiple online classification labels in the text to be processed, and the prediction probability fields corresponding to each online classification label, can guide the classification model to predict the predicted probability that the answer of the online classification text to the online question belongs to each online classification label, rather than guiding the classification model to generate classification labels. This can overcome the problem of low classification accuracy in zero-shot scenarios, thus making the classification model of this application more accurate in zero-shot scenarios.

[0167] Secondly, the classification model of this application can be shared by multiple service platforms. Each service platform no longer needs to train the classification model according to its own specific needs, and each service platform does not need to master the training technology of the classification model, thus lowering the technical threshold. For example, even if the classification requirements of each service platform are different, and even if the classification labels required for text classification are different, each service platform can directly deploy the classification model of this application. When it needs to classify online classified text, the text to be processed input to the classification model includes "online classified text, online question, multiple online classification labels, and the prediction probability fill-in position corresponding to each online classification label". This allows the classification model to predict the prediction probability of the answer of the online classified text to the online question belonging to each online classification label, enabling cold start use of the classification model for each service platform. Since each service platform no longer needs to train the classification model, it does not need to obtain sample classified text, nor does it need to label the sample classified text, reducing the labeling workload of each service platform and reducing labor costs. Furthermore, the classification model of this application predicts the probability that the answer of the online classified text to the online question belongs to each of the known online classification labels, without generating classification labels (that is, without performing the task of generating text). Therefore, the classification model of this application may not have a classification layer, which simplifies the structure of the classification model. For example, it reduces the network parameters in the classification model, which makes the system resources required to load and run the classification model less, and can improve the computing speed, thereby improving the classification efficiency.

[0168] This application also provides a non-volatile readable storage medium storing one or more modules (programs). When these modules are applied to a device, they enable the device to execute the instructions for the method steps in this application.

[0169] This application provides one or more machine-readable media storing instructions that, when executed by one or more processors, cause an electronic device to perform one or more methods as described in the above embodiments. In this application, the electronic device includes a server, a gateway, sub-devices, etc., and the sub-devices are devices such as Internet of Things (IoT) devices.

[0170] Embodiments of this disclosure can be implemented as an apparatus with any suitable hardware, firmware, software, or any combination thereof, configured as desired. This apparatus may include electronic devices such as servers (clusters) and terminal devices such as IoT devices.

[0171] Figure 9 An exemplary apparatus 1300 that can be used to implement the various embodiments of this application is schematically illustrated. For one embodiment, Figure 9 An exemplary device 1300 is illustrated, comprising one or more processors 1302, a control module (chipset) 1304 coupled to at least one of the processors 1302, a memory 1306 coupled to the control module 1304, a non-volatile memory (NVM) / storage device 1308 coupled to the control module 1304, one or more input / output devices 1310 coupled to the control module 1304, and a network interface 1312 coupled to the control module 1304. The processors 1302 may include one or more single-core or multi-core processors, and may include any combination of general-purpose processors or special-purpose processors (e.g., graphics processors, application processors, baseband processors, etc.). In some embodiments, the device 1300 can function as a server device such as a gateway in the embodiments of this application. In some embodiments, apparatus 1300 may include one or more computer-readable media (e.g., memory 1306 or NVM / storage device 1308) having instructions 1314 and one or more processors 1302 configured to execute the instructions 1314 in conjunction with the one or more computer-readable media to implement a module thereby performing the actions of this disclosure. In one embodiment, control module 1304 may include any suitable interface controller to provide any suitable interface to at least one of the processors 1302 and / or any suitable device or component communicating with control module 1304. Control module 1304 may include a memory controller module to provide an interface to memory 1306. The memory controller module may be a hardware module, a software module, and / or a firmware module.

[0172] Memory 1306 may be used, for example, to load and store data and / or instructions 1314 for device 1300. In one embodiment, memory 1306 may include any suitable volatile memory, such as suitable DRAM. In some embodiments, memory 1306 may include double data rate quad synchronous dynamic random access memory (DDR4 SDRAM). In one embodiment, control module 1304 may include one or more input / output controllers to provide an interface to NVM / storage device 1308 and (one or more) input / output devices 1310. For example, NVM / storage device 1308 may be used to store data and / or instructions 1314. NVM / storage device 1308 may include any suitable non-volatile memory (e.g., flash memory) and / or may include any suitable (one or more) non-volatile storage devices (e.g., one or more hard disk drive (HDD), one or more optical disc (CD) drives, and / or one or more digital universal optical disc (DVD) drives). NVM / storage device 1308 may include storage resources that are physically part of a device on which device 1300 is mounted, or that are accessible to the device but do not necessarily need to be part of the device. For example, NVM / storage device 1308 may be accessed via a network via one or more input / output devices 1310. The one or more input / output devices 1310 may provide an interface for device 1300 to communicate with any other suitable device; the input / output devices 1310 may include communication components, input components, sensor components, etc. Network interface 1312 may provide an interface for device 1300 to communicate over one or more networks. Device 1300 may wirelessly communicate with one or more components of a wireless network according to any of one or more wireless network standards and / or protocols, such as accessing a wireless network based on a communication standard, such as WiFi, 2G, 3G, 4G, 5G, etc., or a combination thereof. In one embodiment, at least one of the one or more processors 1302 may be logically encapsulated with one or more controllers (e.g., memory controller modules) of control module 1304. In one embodiment, at least one of the processors 1302 may be packaged together with the logic of one or more controllers of the control module 1304 to form a system-in-package (SiP). In one embodiment, at least one of the processors 1302 may be integrated with the logic of one or more controllers of the control module 1304 on the same die. In one embodiment, at least one of the processors 1302 may be integrated with the logic of one or more controllers of the control module 1304 on the same die to form a system-on-a-chip (SoC).

[0173] In various embodiments, device 1300 may be, but is not limited to, a server, desktop computing device, or mobile computing device (e.g., laptop computing device, handheld computing device, tablet computer, netbook, etc.). In various embodiments, device 1300 may have more or fewer components and / or different architectures. For example, in some embodiments, device 1300 includes one or more cameras, a keyboard, a liquid crystal display (LCD) screen (including a touchscreen display), a non-volatile memory port, multiple antennas, a graphics chip, an application-specific integrated circuit (ASIC), and a speaker.

[0174] This application provides an electronic device, including: one or more processors; and one or more machine-readable media having instructions stored thereon, which, when executed by the one or more processors, cause the electronic device to perform one or more methods as described in this application.

[0175] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0176] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. The embodiments of this application are described with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable information processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable information processing terminal device, generate instructions for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable information processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The functions specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable information processing terminal equipment to cause a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the functions specified in one or more boxes. Although preferred embodiments of the embodiments of this application have been described, those skilled in the art, once they learn the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the embodiments of this application. Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes the element.

[0177] The foregoing has provided a detailed description of the training classification model, text classification method, and apparatus provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method of training a classification model, the method comprising: The method comprises: obtaining a pre-trained language model; obtaining a first training text, the first training text comprising at least a first sample classification text, a first sample question, a plurality of first sample classification labels, and a prediction probability filling position corresponding to each first sample classification label; the first sample question is used to prompt the prediction probability that the answer of the first sample classification text to the first sample question belongs to each first sample classification label; obtaining a first labeled classification label of the first sample classification text, the first labeled classification label being one of the plurality of first sample classification labels; wherein the prediction probability filling position corresponding to each first sample classification label is used to guide the pre-trained language model to predict the prediction probability that the answer of the first sample classification text to the first sample question belongs to each first sample classification label in the process of optimizing and learning the pre-trained language model. At least one character mask in the first sample classification text in the first training text is obtained to obtain a masked training text. At least using the masked training text and the first labeled classification label, the pre-trained language model is subjected to optimization learning of a probability prediction task of the prediction probability filling position corresponding to each first sample classification label, to obtain a classification model, and the probability prediction task comprises a task of predicting the prediction probability that the answer of the first sample classification text to the first sample question belongs to each first sample classification label.

2. A method of classifying text, characterized by, The method comprises: obtaining a first to-be-processed text, the first to-be-processed text comprising at least a first online classification text, a first online question, a plurality of first online classification labels, and a prediction probability filling position corresponding to each first online classification label; the first online question is used to prompt the prediction probability that the answer of the first online classification text to the first online question belongs to each first online classification label; based on the trained classification model, predicting the prediction probability filled in the prediction probability filling position corresponding to each first online classification label in the first to-be-processed text, the prediction probability comprising the probability that the answer of the first online classification text to the first online question belongs to each first online classification label; The trained classification model is obtained by at least using the mask training text and the first labeled classification label to perform optimization learning on the pre-trained language model for a probability prediction task corresponding to each first sample classification label, and the probability prediction task includes a task of predicting the prediction probability of the answer of the first sample classification text to the first sample question belonging to each first sample classification label; the first training text at least includes the first sample classification text, the first sample question, the plurality of first sample classification labels, and the prediction probability filling position corresponding to each first sample classification label; the first sample question is used to prompt the prediction probability of the answer of the first sample classification text to the first sample question belonging to each first sample classification label; the first labeled classification label is one of the plurality of first sample classification labels; wherein the mask training sample is obtained by masking at least one character in the first sample classification text in the first training text, and the prediction probability filling position corresponding to each first sample classification label is used to guide the pre-trained language model to predict the prediction probability of the answer of the first sample classification text to the first sample question belonging to each first sample classification label during the optimization learning of the pre-trained language model.

3. The method of claim 2, wherein, The method further comprises: At least display the prediction probability of the answer of the first online classification text to the first online question belonging to at least part of the first online classification label.

4. The method of claim 3, wherein, The at least display the prediction probability of the answer of the first online classification text to the first online question belonging to at least part of the first online classification label comprises: In a case where the plurality of first online classification labels are all input by the user, display the probability of the answer of the first online classification text to the first online question belonging to each first online classification label; Or, In a case where part of the plurality of first online classification labels are input by the user, display the probability of the answer of the part of the first online classification text to the first online question belonging to at least part of the first online classification label.

5. The method of claim 3, wherein, The obtaining of the first to-be-processed text comprises: Obtaining the first online classification text, obtaining the plurality of first online classification labels, and obtaining the first online question; Generating the first to-be-processed text according to at least the first online classification text, the plurality of first online classification labels, and the first online question; the first to-be-processed text further includes the prediction probability filling position corresponding to each first online classification label.

6. The method of claim 5, wherein, The obtaining of the first online classification text, the plurality of first online classification labels, and the first online question comprises: Displaying an input page, the input page including a classification text input box, a classification label input box, and a question input box; Obtaining the content input in the classification text input box, and obtaining the first online classification text according to the content input in the classification text input box; obtaining the content input in the classification label input box, and obtaining the plurality of first online classification labels according to the content input in the classification label input box; obtaining the content input in the question input box, and obtaining the first online question according to the content input in the question input box.

7. The method of claim 6, wherein, The content input in the classification label input box is multiple, and the multiple contents are separated by a preset separator; The multiple first online classification labels are obtained according to the content input in the classification label input box, including: Each content separated by the preset separator input in the classification label input box is determined as a first online classification label.

8. The method of claim 7, wherein, The method further includes: Displaying prompt information in the input page, the prompt information being used to prompt that different classification labels input are separated by a preset separator.

9. The method of claim 5, wherein, The multiple first online classification labels are obtained, including: In a case where part of the first online classification labels have been obtained, at least one classification label different from the part of the first online classification labels is obtained according to the part of the first online classification labels, and the multiple first online classification labels are obtained according to the part of the first online classification labels and the obtained at least one classification label.

10. The method of claim 9, wherein, The at least one classification label is obtained according to the part of the first online classification labels, including: The target second to-be-processed text is determined in the obtained second to-be-processed text, the target second to-be-processed text at least including a second online classification text, a second online question, multiple second online classification labels and a predicted probability filling position corresponding to each second online classification label; the multiple second online classification labels in the target second to-be-processed text include the part of the first online classification labels, or the multiple second online classification labels in the target second to-be-processed text include similar classification labels with a semantic similarity greater than a third preset similarity to the part of the first online classification labels; at least one classification label is obtained according to second online classification labels in the multiple second online classification labels except the part of the first online classification labels or the similar classification labels.

11. The method of claim 5, wherein, The multiple first online questions are obtained, including: In a case where the multiple first online classification labels have been obtained, a target classification label set is found in a first correspondence relationship between questions and classification label sets, each classification label set in the first correspondence relationship respectively including multiple classification labels; a label similarity between the target classification label set and the multiple first online classification labels is greater than a third preset similarity; In the first correspondence relationship, a question corresponding to the target classification label set is found; The first online question is obtained according to the found question.

12. The method of claim 5, wherein, The multiple first online questions are obtained, including: The target third to-be-processed text is determined in the obtained third to-be-processed text, the target third to-be-processed text at least including a third online classification text, a third online question, multiple third online classification labels and a predicted probability filling position corresponding to each third online classification label; the multiple third online classification labels in the target third to-be-processed text include the part of the first online classification labels, or the multiple third online classification labels in the target third to-be-processed text include similar classification labels with a semantic similarity greater than a third preset similarity to the part of the first online classification labels; The first online question is obtained according to the third online question.

13. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor, when executing the program, implements the steps of the method of any one of claims 1 to 12.

Images