Multi-label text classification method and device, equipment, medium and program product

Abstract

The invention provides a multi-label text classification method, and relates to the field of artificial intelligence. The method comprises the steps of obtaining a to-be-classified text and a first label set, wherein the first label set comprises a plurality of labels for text classification; selecting N candidate tags of the to-be-classified text from the first tag set, wherein N is an integer greater than or equal to 2; the to-be-classified texts and the N candidate tags are input into a multi-tag text classification model, and the multi-tag text classification model is obtained by training a large language model based on text samples and corresponding tag samples in advance; and obtaining M target tags predicted from the N candidate tags by the multi-tag text classification model, wherein M is an integer greater than or equal to 1.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence, and more specifically, to a multi-label text classification method, apparatus, device, medium and program product. Background Art

[0002] Multi-label text classification is the task of assigning multiple labels to text at the same time. It has a wide range of applications in multiple scenarios such as recommendation and retrieval.

[0003] With the accumulation of a large amount of data on the Internet, the data labeling system has become more and more complex, and the number of labels involved has also increased. Extreme multi-label text classification tasks have been widely studied. Specifically, in extreme multi-label text classification tasks, the scale of labels is huge, and a large number of labels involve little training data. The overall distribution of labels is long-tailed, which will lead to classification errors due to uneven sample distribution. This makes extreme multi-label text classification more difficult than multi-label text classification tasks, and there is an urgent need to design relevant methods to solve such problems. Summary of the invention

[0004] In view of the above problems, the present disclosure provides a multi-label text classification method, apparatus, device, medium and program product.

[0005] According to a first aspect of the present disclosure, a multi-label text classification method is provided, characterized in that it includes: obtaining a text to be classified and a first label set, wherein the first label set includes multiple labels for text classification; selecting N candidate labels for the text to be classified from the first label set, where N is an integer greater than or equal to 2; inputting the text to be classified and the N candidate labels into a multi-label text classification model, wherein the multi-label text classification model is pre-trained on a large language model based on text samples and corresponding label samples; and obtaining M target labels predicted by the multi-label text classification model from the N candidate labels, where M is an integer greater than or equal to 1.

[0006] According to an embodiment of the present disclosure, after obtaining the M target labels predicted by the multi-label text classification model from the N candidate labels, the method also includes: inputting the text to be classified and the M target labels into an evaluation model, the evaluation model being pre-trained by a machine learning model based on text samples, their corresponding label samples and matching samples; obtaining the label matching degree output by the evaluation model, the label matching degree characterizing the degree of matching between the text to be classified and the M target labels; and determining the multi-label classification result of the text to be classified based on the label matching degree.

[0007] According to an embodiment of the present disclosure, when the label matching degree is less than a preset value, determining the multi-label classification result of the text to be classified based on the label matching degree includes: updating the N candidate labels of the text to be classified from the first label set; inputting the text to be classified and the updated N candidate labels into the multi-label text classification model; and obtaining K target labels predicted by the multi-label text classification model from the updated N candidate labels, where K is an integer greater than or equal to 1.

[0008] According to an embodiment of the present disclosure, selecting N candidate tags for the text to be classified from the first tag set includes: retrieving S1 candidate texts from a vector knowledge base based on the text to be classified, wherein the similarity between each candidate text and the text to be classified satisfies a first preset condition, the vector knowledge base stores vectorized candidate texts and corresponding tag sets, and S1 is an integer greater than or equal to 1; obtaining the N candidate tags based on the tag sets of each of the S1 candidate texts.

[0009] According to an embodiment of the present disclosure, updating the N candidate tags of the text to be classified from the first tag set includes: retrieving S2 candidate texts from a vector knowledge base based on the text to be classified, wherein the similarity between each candidate text and the text to be classified satisfies a second preset condition, the similarity requirement within the second preset condition is lower than the similarity requirement within the first preset condition, and S2 is an integer greater than S1; and obtaining the updated N candidate tags based on the respective tag sets of the S2 candidate texts.

[0010] According to an embodiment of the present disclosure, the S1 candidate texts are retrieved based on a first similarity algorithm, and updating the N candidate tags of the text to be classified from the first tag set includes: based on the text to be classified, using a second similarity algorithm to retrieve S3 candidate texts from the vector knowledge base, wherein the similarity between each candidate text and the text to be classified satisfies the first preset condition, and the first similarity algorithm is different from the second similarity algorithm.

[0011] According to an embodiment of the present disclosure, selecting N candidate labels for the text to be classified from the first label set includes: inputting the text to be classified into a first pre-classification model, the first pre-classification model is pre-trained based on a machine learning model based on text samples and corresponding label samples, and the first label set includes the text samples and corresponding label samples; obtaining the N candidate labels output by the first pre-classification model.

[0012] According to an embodiment of the present disclosure, updating the N candidate labels of the text to be classified from the first label set includes: inputting the text to be classified into a second pre-classification model, the second pre-classification model is pre-trained based on a machine learning model based on text samples and corresponding label samples, and the second pre-classification model is different from the first pre-classification model; obtaining the updated N candidate labels output by the second pre-classification model.

[0013] According to an embodiment of the present disclosure, a first prompt word is pre-configured, and inputting the text to be classified and the N candidate tags into a multi-label text classification model includes: inputting the text to be classified, the N candidate tags and the first prompt word into the multi-label text classification model, wherein the first prompt word is used to guide the multi-label text classification model to generate a multi-label classification result for the text to be classified.

[0014] According to an embodiment of the present disclosure, a second prompt word is pre-configured, and the second prompt word is different from the first prompt word. When the label matching degree is less than a preset value, determining the multi-label classification result of the text to be classified based on the label matching degree includes: inputting the text to be classified, the N candidate labels and the second prompt word into the multi-label text classification model, wherein the second prompt word is used to guide the multi-label text classification model to generate a multi-label classification result of the text to be classified.

[0015] According to an embodiment of the present disclosure, when the label matching degree is greater than or equal to a preset value, determining the multi-label classification result of the text to be classified based on the label matching degree includes: taking the M target labels as the multi-label classification result of the text to be classified.

[0016] Another aspect of an embodiment of the present disclosure provides a multi-label text classification device, characterized in that it includes: a data acquisition module, used to acquire a text to be classified and a first label set, wherein the first label set includes multiple labels for text classification; a candidate label module, used to select N candidate labels for the text to be classified from the first label set, where N is an integer greater than or equal to 2; a data input module, used to input the text to be classified and the N candidate labels into a multi-label text classification model, wherein the multi-label text classification model is pre-trained on a large language model based on text samples and corresponding label samples; a target label module, used to obtain M target labels predicted by the multi-label text classification model from the N candidate labels, where M is an integer greater than or equal to 1.

[0017] Another aspect of an embodiment of the present disclosure provides an electronic device, including: one or more processors; and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors execute the method described above.

[0018] Another aspect of an embodiment of the present disclosure provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the processor executes the method as described above.

[0019] Another aspect of an embodiment of the present disclosure provides a computer program product, including a computer program, which implements the above method when executed by a processor.

[0020] One or more of the above embodiments have the following beneficial effects: a multi-label text classification model is obtained by training with a large language model. Since the number of input tokens of the large language model is limited, and the set of candidate labels in extreme multi-label text classification is large, N candidate labels are first screened out to narrow the set of label candidates, and then input into the M target labels predicted by the multi-label text classification model. This can effectively solve the problem of the limited number of input tokens of the large language model, and can also use the generalization ability of the large language model to solve the learning problem of the long-tail distribution of samples in extreme multi-label text classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The above contents and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0022] Figure 1 The application scenario diagram of multi-label text classification according to an embodiment of the present disclosure is schematically shown;

[0023] Figure 2 A flowchart of a multi-label text classification method according to an embodiment of the present disclosure is schematically shown;

[0024] Figure 3 A flowchart of selecting candidate tags according to an embodiment of the present disclosure is schematically shown;

[0025] Figure 4 Schematically shows a flow chart of selecting candidate tags according to another embodiment of the present disclosure;

[0026] Figure 5 A flowchart of a multi-label text classification method according to another embodiment of the present disclosure is schematically shown;

[0027] Figure 6 A flowchart of regenerating multiple target tags according to an embodiment of the present disclosure is schematically shown;

[0028] Figure 7 Schematically shows a flow chart of updating candidate tags according to an embodiment of the present disclosure;

[0029] Figure 8 The structure block diagram of the multi-label text classification device according to an embodiment of the present disclosure is schematically shown;

[0030] Fig. 9 A block diagram of an electronic device suitable for implementing a multi-label text classification method according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0031] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0032] In the technical solution of the present invention, the user information (including but not limited to user personal information, user image information, user device information, such as location information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved are all information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data comply with relevant laws, regulations and standards, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0033] In the scenario of using personal information for automated decision-making, the methods, devices, and systems provided by the embodiments of the present invention provide users with corresponding operation portals for users to choose to agree or reject the automated decision-making results; if the user chooses to reject, the expert decision-making process will be entered. The expression "automated decision-making" here refers to the activity of automatically analyzing and evaluating an individual's behavioral habits, interests and hobbies, or economic, health, credit status, etc. through computer programs, and making decisions. The expression "expert decision-making" here refers to the activity of making decisions by people who specialize in a certain field, have specialized experience, knowledge and skills, and have reached a certain level of professionalism.

[0034] Taking financial technology as an example, under the user voice function of the user experience management platform, not only are there traditional labels for various products and product functions, but users will also elaborate on different aspects of the function when expressing their opinions. For example, under the login label of mobile banking, there will be operational label problems such as "I used the U shield to log in to the corporate mobile banking according to the rules and completed the monthly account management query, but did not participate in the lottery", U shield label problems such as "The new U shield is often not recognized and needs to be logged in again", and login label problems such as "It crashes after logging in, and it always crashes when operating with Apple 12pro max". This type of label is the real situation of customer feedback under various customer products, with many labels and a small number of samples under the labels. Classifying different multi-label problems is the best way to accurately locate user pain points and transfer them to the corresponding handlers, avoiding the problem of multiple transfers during the transfer process due to the mismatch between the professional field of the handler and the problem label.

[0035] So for the extreme multi-label text situation mentioned above with a large number of labels and few label samples, the use of traditional multi-label text classification methods will lead to classification errors due to uneven sample distribution. The traditional manual labeling method can certainly achieve accurate multi-label classification, but the manpower and time costs consumed are very expensive.

[0036] The number of tokens that a large language model can accept is limited. However, the labels of extreme multi-label text classification problems are large and cannot be input into the large language model at one time. Especially considering the computational cost of the large language model, inputting a large number of irrelevant labels into the large language model each time is a waste of computing resources.

[0037] In some embodiments of the present disclosure, a multi-label text classification method is provided, in which a multi-label text classification model is obtained by training a large language model. Since the number of input tokens of the large language model is limited, and the size of the candidate label set in extreme multi-label text classification is large, N candidate labels are first screened out to narrow the label candidate set, and then input into the M target labels predicted by the multi-label text classification model. This can effectively solve the problem of the limited number of input tokens of the large language model, and can also use the generalization ability of the large language model to solve the learning problem of the long-tail distribution of samples in extreme multi-label text classification.

[0038] Figure 1 The following schematically shows an application scenario diagram of multi-label text classification according to an embodiment of the present disclosure. It should be noted that: Figure 1 What is shown are merely examples to which the embodiments of the present disclosure can be applied, so as to help those skilled in the art understand the technical content of the present disclosure, but it does not mean that the embodiments of the present disclosure cannot be used in other devices, systems, environments or scenarios.

[0039] like Figure 1 As shown, the application scenario 100 according to this embodiment may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for a communication link between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0040] Users can use terminal devices 101, 102, 103 to interact with server 105 through network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only examples).

[0041] The terminal devices 101 , 102 , and 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers.

[0042] The server 105 may be a server that provides various services, such as a background management server that provides support for websites browsed by users using the terminal devices 101, 102, and 103 (for example only). The background management server may analyze and process the received data such as user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device. For example, the server 105 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud computing, network services, and middleware services.

[0043] In applications such as client applications and web applications (abbreviated as APP in English), the client (i.e., front-end) and the server (i.e., back-end) can communicate data through network messages. For example, in the APP client, the parameters that the server needs to obtain from the client (such as the text to be classified and the multi-label text classification instructions) are assembled, and the network request method is called to send them to the server.

[0044] It should be noted that the multi-label text classification method provided in the embodiment of the present disclosure can generally be executed by at least one of a terminal device or a server. Accordingly, the multi-label text classification device provided in the embodiment of the present disclosure can generally be set in at least one of a terminal device or a server.

[0045] It should be understood that Figure 1The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0046] The following will be based on Figure 1 The scene described by Figure 2~Figure 7 The multi-label text classification method of the embodiment of the present disclosure is described in detail.

[0047] Figure 2 The flowchart of the multi-label text classification method according to an embodiment of the present disclosure is schematically shown.

[0048] like Figure 2 As shown, this embodiment includes:

[0049] In operation S210, the text to be classified and a first label set are obtained, wherein the first label set includes a plurality of labels for text classification; and raw data and all possible classification labels are prepared for subsequent processing.

[0050] Extreme multi-label text classification is a task in Natural Language Processing (NLP) that involves associating a piece of text with multiple labels. Unlike traditional multi-label text classification, the challenge of extreme multi-label text classification is that the number of labels can be very large, sometimes even reaching thousands.

[0051] The text to be classified refers to the text to be classified into extreme multi-label categories, such as user complaints, user feedback, user comments, or news information. The first label set contains the set of all possible labels. For example, in the FinTech scenario, the first label set may include "login label", "operation label", "improvement suggestion label", "design feedback label", and "transaction experience, etc."

[0052] In operation S220, N candidate tags for the text to be classified are selected from the first tag set, where N is an integer greater than or equal to 2; the candidate tags that are most likely to be related to the text are selected, which can reduce the number of tags that the classification model needs to process, thereby improving processing efficiency and accuracy.

[0053] The candidate tags include tags that are most likely to be related to the text to be classified and are selected from the first tag set. For example, the tags may be selected using preset rules, such as selecting multiple candidate tags based on keywords in the text to be classified; multiple candidate tags that are most related to the text may be determined using TF-IDF or other feature selection algorithms; multiple candidate tags may be selected from a semantic perspective using natural language processing technology, etc.

[0054] In operation S230, the text to be classified and N candidate labels are input into a multi-label text classification model, wherein the multi-label text classification model is pre-trained on a large language model based on text samples and corresponding label samples;

[0055] The multi-label text classification model includes a pre-trained large language model for performing classification tasks. The text to be classified and the candidate labels are passed through an embedding layer such as a neural network or a multi-label text classification model encoder to encode the text into a vector.

[0056] A large language model refers to a deep learning model trained with a large amount of text data, which can generate natural language text or understand the meaning of language text. In this embodiment, the initial large language model refers to a large language model obtained by pre-training. Through the adjustment of parameters at the billion level or even the hundred billion level, a general large language model is trained. After training with a large-scale data set, it can be applied to specific application scenarios. For example, the large language model of this embodiment can be a GPT (Generative Pre-trained Transformer) model, an LLaMA series model, a LaMDA model, etc.

[0057] Pre-training a large language model based on text samples and corresponding label samples may include:

[0058] First, choose a pre-trained model. Under legal conditions, choose a model that has been pre-trained on a large amount of pre-training data from models such as BERT, GPT, Llama, etc. as a starting point.

[0059] Second, prepare task-specific data. Collect datasets directly related to the target task, which will be used to fine-tune the model. For example, collect extremely multi-label text samples related to the fintech field. You can also apply a large language model to clean and preprocess the collected data, including removing noise, filtering duplicate content, word segmentation, etc. The text sample can include historical text content or generated text content, and each label sample includes multiple labels corresponding to the text sample.

[0060] Next, fine-tune the training for the multi-label text classification task. Train the pre-trained model on task-specific data and adjust the model parameters to adapt to the multi-label text classification task.

[0061] It can be understood that fine-tuning means adjusting small parameters of data for specific tasks to adapt them to different natural language processing tasks. Compared with the tens of billions or hundreds of billions of parameters in the pre-training of large language models, the parameters involved in fine-tuning are of a smaller order of magnitude, the training efficiency is higher, and it is more suitable for specific application scenarios.

[0062] The performance of the fine-tuned model can then be evaluated on the validation set to ensure that the model generalizes well to new data.

[0063] Finally, the model with verified performance is deployed to practical applications.

[0064] In some embodiments, a first prompt word is pre-configured, and inputting the text to be classified and N candidate tags into a multi-label text classification model includes: inputting the text to be classified, N candidate tags and the first prompt word into the multi-label text classification model, wherein the first prompt word is used to guide the multi-label text classification model to generate a multi-label classification result for the text to be classified.

[0065] Exemplarily, the prompt includes keywords or text provided when training or using a large language model, which is used to guide the output content of the model. For example, the first prompt may be "Please output the M tags that are most relevant to the text to be processed based on the input content."

[0066] In some embodiments, in addition to adjusting the model parameters, the fine-tuning stage also includes adjusting the prompt words so as to determine the optimal prompt word content, for example, adjusting the persona information and interface range information in the prompt words and eliminating redundant information.

[0067] In operation S240 , M target labels predicted by the multi-label text classification model from the N candidate labels are obtained, where M is an integer greater than or equal to 1.

[0068] The target label is the label predicted by the multi-label text classification model, which represents the label that the model thinks is most relevant to the text to be classified. For example, when processing a review about the mobile phone transfer function, the target labels may include "complaint" and "operation".

[0069] In some embodiments, the multi-label text classification model can also predict the relevance to sort the M target labels, and obtain the multi-label classification result according to the sorting result.

[0070] For example, a user commented: "I am very satisfied with the transfer function of mobile banking. The interface is intuitive and the operation is easy. However, I encountered a transfer failure. Although it was finally resolved, I hope the bank can improve the stability of the system." First, the text content to be classified is extracted from the user comments. Then, by analyzing the keywords in the comments, such as "transfer function", "intuitive interface", "easy operation", "transfer failure", and "system stability", the most relevant candidate tags are selected from the preset tag set. Then, the extracted text and candidate tags are input into a multi-label text classification model based on a large language model. Finally, the model analyzes the text and predicts one or more target tags that best match the content, such as "positive sentiment", "problem report", and "improvement suggestion". Subsequently, based on the predicted target tags, the comments can be automatically classified and the user feedback can be processed accordingly. For example, based on these tags, user feedback can be classified and responded to more effectively. Comments with positive sentiment can be used for marketing or user satisfaction surveys, and problem reports and improvement suggestions can be conveyed to the technical team to improve the product. In this way, the multi-label classification system can help banks better understand and respond to user needs.

[0071] According to the embodiments of the present disclosure, a multi-label text classification model is obtained by training with a large language model. Since the number of input tokens of the large language model is limited, and the set of candidate labels in extreme multi-label text classification is large, N candidate labels are first screened out to narrow the set of label candidates, and then input into the M target labels predicted by the multi-label text classification model. This can effectively solve the problem of the limited number of input tokens of the large language model, and can also use the generalization ability of the large language model to solve the learning problem of the long-tail distribution of samples in extreme multi-label text classification.

[0072] Figure 3 The flowchart of selecting candidate tags according to an embodiment of the present disclosure is schematically shown.

[0073] like Figure 3 As shown, this embodiment is one of the embodiments of operation S220, including:

[0074] In operation S310, S1 candidate texts are retrieved from a vector knowledge base based on the text to be classified, wherein the similarity between each candidate text and the text to be classified satisfies a first preset condition, the vectorized candidate texts and corresponding label sets are stored in the vector knowledge base, and S1 is an integer greater than or equal to 1;

[0075] A vector knowledge base refers to a database that stores information, where data is represented in vector form, which is convenient for similarity retrieval and machine learning tasks. A vector knowledge base can be constructed through steps such as preprocessing, word segmentation, normalization, and weighting algorithms. Preprocessing includes removing stop words, symbols, etc., word segmentation is performed using specific algorithms such as N-gram, and normalization usually uses the TF-IDF method. After that, all texts are vectorized to form a vector knowledge base. The knowledge base can be maintained and updated through continuous machine learning and specified update mechanisms (such as regular updates, updates when new data appears, etc.).

[0076] For example, a vectorized database containing different texts represented in vector form and corresponding tag sets. Candidate texts include content retrieved from the vector knowledge base that may be related to the text to be processed. The tag set may include multiple tags corresponding to the candidate text, and the candidate text and the tag set are stored in key-value form.

[0077] For example, a cosine similarity algorithm may be used to calculate similarity between the vector of the text to be processed and the vector of the candidate text in the vector knowledge base, and retrieve the most relevant S1 candidate texts. The first preset condition is a set standard used to determine what degree of similarity is acceptable, such as a cosine similarity greater than 0.8.

[0078] In operation S320 , N candidate tags are obtained based on the tag sets of the S1 candidate texts.

[0079] For example, based on the tag sets of the retrieved S1 texts, N tags with the highest occurrence frequency are summarized and selected as candidate tags; or, the tag sets of the retrieved S1 texts are unioned to obtain N tags.

[0080] According to an embodiment of the present disclosure, relevant candidate texts are selected before processing the large language model to obtain a tag set, thereby reducing the number of tokens input into the large language model.

[0081] Figure 4 The flowchart of selecting candidate tags according to another embodiment of the present disclosure is schematically shown.

[0082] like Figure 4 As shown, this embodiment is one of the embodiments of operation S220, including:

[0083] In operation S410, the text to be classified is input into a first pre-classification model, the first pre-classification model is pre-trained by a machine learning model based on a text sample and a corresponding label sample, and the first label set includes the text sample and the corresponding label sample;

[0084] In operation S420, N candidate labels output by the first pre-classification model are obtained.

[0085] The first pre-classification model is a model trained by machine learning techniques that can predict the classification labels that the text data may belong to. For example, these models can be trained using algorithms such as random forests, neural networks, or support vector machines.

[0086] Because extremely multi-label text samples have a long-tail distribution problem, the first pre-classification model is used for coarse screening to narrow the range of candidates without pursuing accurate classification results. The large language model is then used for accurate prediction.

[0087] According to an embodiment of the present disclosure, the first pre-classification model is used to preliminarily screen out N candidate tags that are most likely to be related to the text to be classified, thereby narrowing the tag range processed by the large language model.

[0088] Figure 5 The following schematically shows a flowchart of a multi-label text classification method according to another embodiment of the present disclosure.

[0089] like Figure 5 As shown, in addition to operations S210 to S240, after obtaining M target labels predicted by the multi-label text classification model from N candidate labels, this embodiment further includes:

[0090] In operation S510, the text to be classified and M target labels are input into an evaluation model, and the evaluation model is pre-trained by a machine learning model based on text samples, their corresponding label samples and matching degree samples;

[0091] An evaluation model is a model trained by machine learning that can evaluate the degree of match between text and labels. For example, a neural network-based evaluation model can estimate the correlation between them by learning historical data of text and label pairings. A matching sample is a matching value between a text sample and its corresponding label sample.

[0092] In operation S520, a label matching degree output by the evaluation model is obtained, where the label matching degree represents the matching degree between the to-be-classified text and the M target labels;

[0093] Label matching refers to a metric output by the evaluation model, which characterizes the degree of correspondence between the text content and the M target labels. For example, a score from 0 to 1 can be used to represent the label matching, where a higher score indicates a higher correlation. For example, the evaluation model outputs the matching degree between each target label and the text to be classified, and the matching degrees corresponding to the M target labels are summed to obtain the label matching degree. The preset value can be the label matching degree multiplied by 0.8, which is only an example.

[0094] In operation S530 , a multi-label classification result of the text to be classified is determined based on the label matching degree.

[0095] The multi-label classification result includes the final set of labels, which includes all labels that are considered to be a high match to the text.

[0096] In some embodiments, supervised learning methods, such as support vector machines (SVM), deep learning models (such as convolutional neural networks (CNN)), or large language models, can be used to train the evaluation model using text samples, label samples, and matching samples as training data. The output layer of the evaluation model can be obtained, for example, using a Softmax function to give each label a probability value between 0 and 1 to reflect its matching degree.

[0097] In some embodiments, an adversarial learning task can be set up during the training phase, that is, the multi-label text classification model and the evaluation model are trained together, so that the two models compete or confront each other to improve the robustness and accuracy of the multi-label text classification task. In other embodiments, in addition to the multi-label text classification model and the evaluation model, a pre-classification model can also be introduced during the training phase to participate in the learning.

[0098] According to an embodiment of the present disclosure, an evaluation model is introduced to improve the accuracy of classification based on the matching degree between text content and labels.

[0099] In some embodiments, when the tag matching degree is greater than or equal to a preset value, determining the multi-tag classification result of the text to be classified based on the tag matching degree includes: taking M target tags as the multi-tag classification result of the text to be classified.

[0100] When the label matching degree is less than the preset value, multiple target labels are regenerated, so as to use the evaluation model to guide the large language model to generate a multi-label classification result with good matching degree.

[0101] Figure 6 The flowchart of regenerating multiple target tags according to an embodiment of the present disclosure is schematically shown.

[0102] When the tag matching degree is less than the preset value, such as Figure 6 As shown, the multi-label classification results of the text to be classified based on the label matching degree include:

[0103] In operation S610, N candidate tags of the text to be classified are updated from the first tag set;

[0104] When the label matching degree given by the evaluation model is lower than the preset value, it means that the original candidate label set may be inaccurate or incomplete. The purpose of updating the candidate labels is to find new candidate labels that better match the text to be classified to improve the classification accuracy.

[0105] In operation S620, the text to be classified and the updated N candidate labels are input into a multi-label text classification model;

[0106] In operation S630, K target labels predicted by the multi-label text classification model from the updated N candidate labels are obtained, where K is an integer greater than or equal to 1. K and M may be equal or different, which is not limited in the present disclosure. Operation S630 is equivalent to re-prediction after operation S240.

[0107] According to the embodiments of the present disclosure, by introducing the process of updating candidate labels and re-predicting, the classification result can be self-corrected when the initial prediction is inaccurate, thereby improving the accuracy and flexibility of the multi-label classification system.

[0108] It can be understood that, if necessary, operations S510 to S530 can be re-executed based on the K target tags until the tag matching degree is greater than or equal to a preset value.

[0109] Figure 7 The flowchart of updating candidate tags according to an embodiment of the present disclosure is schematically shown.

[0110] like Figure 7 As shown, this embodiment is one of the embodiments of operation S610, including:

[0111] In operation S710, S2 candidate texts are retrieved from the vector knowledge base based on the text to be classified, wherein the similarity between each candidate text and the text to be classified satisfies a second preset condition, the similarity requirement in the second preset condition is lower than the similarity requirement in the first preset condition, and S2 is an integer greater than S1;

[0112] In operation S720, updated N candidate tags are obtained based on the tag sets of the S2 candidate texts.

[0113] The similarity requirements of the first preset condition and the second preset condition correspond to different threshold levels. For example, the first preset condition may require the similarity to be greater than 0.8, while the second preset condition is relaxed to 0.6.

[0114] According to an embodiment of the present disclosure, the scope of retrieval is expanded, and a lower similarity threshold (second preset condition) is used to allow more potentially relevant texts to be selected as candidate texts. On the basis of maintaining the relevance of existing candidate tags, further exploration and discovery of related tags that may be overlooked. By considering a wider set of tags to obtain a wider range of candidate texts, the original N candidate tags can be expanded, which helps to improve the comprehensiveness of tag coverage and classification. It should be noted that N is a variable, and the updated N candidate tags may have a different number and content of candidate tags compared to operation S220.

[0115] In some embodiments, S1 candidate texts are retrieved based on a first similarity algorithm, and N candidate tags of the text to be classified are updated from the first tag set, including: based on the text to be classified, S3 candidate texts are retrieved from a vector knowledge base using a second similarity algorithm, wherein the similarity between each candidate text and the text to be classified satisfies a first preset condition, the first similarity algorithm is different from the second similarity algorithm, and S3 is an integer greater than or equal to 1.

[0116] For example, the first similarity algorithm can calculate the Euclidean distance between texts by extracting keywords, such as using the TF-IDF algorithm to calculate the similarity between the text to be classified and the text in the first tag set. The second similarity algorithm can extract the semantic vector of the text and find candidate texts with similar semantics from the vector knowledge base, such as using the cosine similarity algorithm to calculate the similarity between the semantic vector of the text to be classified and the semantic vector of the candidate text in the knowledge base.

[0117] Considering that the similarity algorithm may not be compatible with the text to be classified, resulting in inaccurate candidate texts, making the N candidate tags inaccurate. According to the embodiments of the present disclosure, by switching the similarity algorithm, different similarity retrieval results can be provided, thereby providing different inputs to the multi-label text classification model. The accuracy and robustness of label classification are improved.

[0118] In some embodiments, a first pre-classification model is used to output N candidate labels, and updating the N candidate labels of the text to be classified from the first label set includes: inputting the text to be classified into a second pre-classification model, the second pre-classification model is pre-trained based on a machine learning model based on text samples and corresponding label samples, and the second pre-classification model is different from the first pre-classification model; and obtaining the updated N candidate labels output by the second pre-classification model.

[0119] Exemplarily, the first pre-classification model can be a deep learning model, such as a convolutional neural network (CNN), which is pre-trained based on a large number of text samples and corresponding label samples. The second pre-classification model can use a different machine learning algorithm, such as a long short-term memory network (LSTM), to capture long-term dependencies in the text. The second pre-classification model is also trained based on text samples and corresponding label samples, but is intended to capture different features or patterns from the first pre-classification model.

[0120] According to an embodiment of the present disclosure, two different pre-classification models are used to obtain prediction results based on the text to be classified, thereby reducing the deviation that may be caused by relying on a single model, and the accuracy of prediction by the multi-label text classification model is improved based on the updated N candidate labels.

[0121] In some embodiments, a second prompt word is pre-configured, and the second prompt word is different from the first prompt word. When the label matching degree is less than a preset value, determining the multi-label classification result of the text to be classified based on the label matching degree includes: inputting the text to be classified, N candidate labels and the second prompt word into the multi-label text classification model, wherein the second prompt word is used to guide the multi-label text classification model to generate a multi-label classification result of the text to be classified.

[0122] For example, in the FinTech scenario, different second prompt words can be configured for different financial businesses. The first prompt word can be a general prompt word, such as "The text to be processed is related to FinTech, please perform multi-label text classification." When the M target labels obtained based on the first prompt word have a low match, the second prompt word can be selected based on the content and semantics of the text to be processed, such as "The text to be processed is related to cross-border payments, please consider labels related to cross-border payments for multi-label text classification."

[0123] According to the embodiments of the present disclosure, by switching prompt words, different context information can be given to the multi-label text classification model, and different guiding conditions can be prompted to it, thereby further improving the label matching degree of the predicted multi-label classification results.

[0124] It should be noted that N candidate tags can be obtained by simultaneously using operations S310 to S320 and operations S410 to S420. Accordingly, when the tag matching degree is less than a preset value, one or more means of switching similarity preset conditions, similarity algorithms, pre-classification models, and prompt words can be used to achieve re-prediction of multi-tag classification results.

[0125] In combination with the above embodiment, a multi-label text classification method according to another embodiment of the present disclosure is described again.

[0126] First, train a multi-label text classification model. As follows:

[0127] 1) Dataset and problem definition: Given labeled training data With unlabeled test data , Is the training data The purpose of extreme multi-label text classification is to The knowledge learned in predicting test samples The corresponding tag set .

[0128] 2) Similar example retrieval: Use the encoder of the pre-trained large language model to generate sentence representations and process user queries. For example, Step 1: Encode the training set data; Step 2: Given a test example , get sentence representation through encoder , as the query for retrieval; Step 3: Based on the two parts of input from Step 1 and Step 2, 10 results are returned. This result is the training set and the test sample The 10 most similar examples; Step 4: Take the union of the label sets corresponding to the 10 retrieved examples to form a sample The set of candidate labels C.

[0129] 3) Candidate label ranking: given a test example With the generated candidate tag set C, the following in-context-learning prompt word template is designed:

[0130] Task: Given an input sentence, select the 10 most relevant tags based on the candidate tag set and sort the tags from most relevant to least relevant. (i.e. the first prompt word)

[0131] enter:

[0132] Candidate label set:

[0133] Format: list of labels, example: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

[0134] Input this into the multi-label text classification model to obtain the final target label list .

[0135] Secondly, the feedback opinions that have been tagged and classified in the past are converted into a knowledge vector library through the retriever, and the multi-label text classification model is deployed to participate in the multi-label text classification task in the actual production scenario. For example, when the user inputs feedback opinions, the encoder of the multi-label text classification model converts the input into input 1 by word embedding including word segmentation, syntactic analysis and semantic understanding; the input 1 is searched in the knowledge vector library, and the 10 closest knowledge vector documents are retrieved according to the vector similarity, and the labels corresponding to the 10 knowledge vectors are used as candidate labels C. The feedback opinions, candidate labels C and the first prompt word are input into the multi-label text classification model according to the prompt word template to obtain M target labels.

[0136] Then, the M target labels can be input into the evaluation model to determine the multi-label classification result of the text to be classified based on the label matching degree.

[0137] According to the embodiments of the present disclosure, since the number of input tokens of the large language model is limited and the size of the candidate label set in extreme multi-label text classification is large, the large language model cannot directly solve the problem of extreme multi-label text classification. A search-first-then-rank architecture is designed to effectively solve this problem. Benefiting from the powerful generalization and understanding capabilities of the large language model itself in low-resource scenarios, the zero-sample learning problem of extreme multi-label text classification is effectively solved.

[0138] Based on the above multi-label text classification method, the present disclosure also provides a multi-label text classification device. Figure 8 The device is described in detail.

[0139] Figure 8 The structural block diagram of a multi-label text classification device according to an embodiment of the present disclosure is schematically shown.

[0140] like Figure 8 As shown, the multi-label text classification device 800 of this embodiment includes a data acquisition module 810 , a candidate label module 820 , a data input module 830 and a target label module 840 .

[0141] The data acquisition module 810 may perform operation S210 for acquiring the text to be classified and a first tag set, wherein the first tag set includes a plurality of tags for text classification;

[0142] The candidate tag module 820 may perform operation S220 for selecting N candidate tags for the text to be classified from the first tag set, where N is an integer greater than or equal to 2;

[0143] The data input module 830 may perform operation S230, for inputting the to-be-classified text and the N candidate labels into a multi-label text classification model, wherein the multi-label text classification model is pre-trained on a large language model based on text samples and corresponding label samples;

[0144] The target label module 840 may perform operation S240 to obtain M target labels predicted by the multi-label text classification model from N candidate labels, where M is an integer greater than or equal to 1.

[0145] For the parts not mentioned in the device part, they can be understood with reference to the various embodiments of the above method. That is, the device part includes modules for executing the various steps of any one of the method embodiments described above. In addition, the implementation methods, technical problems solved, functions implemented, and technical effects achieved of each module / unit / subunit, etc. in the device part embodiment are respectively the same or similar to the implementation methods, technical problems solved, functions implemented, and technical effects achieved of each corresponding step in the method part embodiment, and will not be repeated here.

[0146] According to an embodiment of the present disclosure, any multiple modules among the data acquisition module 810, the candidate label module 820, the data input module 830 and the target label module 840 can be combined into one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module.

[0147] According to an embodiment of the present disclosure, at least one of the data acquisition module 810, the candidate label module 820, the data input module 830 and the target label module 840 may be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or may be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or by any one of the three implementation methods of software, hardware and firmware or by an appropriate combination of any of them. Alternatively, at least one of the data acquisition module 810, the candidate label module 820, the data input module 830 and the target label module 840 may be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding function may be performed.

[0148] Fig. 9 A block diagram of an electronic device suitable for implementing a multi-label text classification method according to an embodiment of the present disclosure is schematically shown.

[0149] like Fig. 9 As shown, the electronic device 900 according to an embodiment of the present disclosure includes a processor 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage part 908 to a random access memory (RAM) 903. The processor 901 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a dedicated microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 901 may also include an onboard memory for caching purposes. The processor 901 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0150] In RAM 903, various programs and data required for the operation of electronic device 900 are stored. Processor 901, ROM 902 and RAM 903 are connected to each other via bus 904. Processor 901 performs various operations of the method flow according to the embodiment of the present disclosure by executing the program in ROM 902 and / or RAM 903. It should be noted that the program can also be stored in one or more memories other than ROM 902 and RAM 903. Processor 901 can also perform various operations of the method flow according to the embodiment of the present disclosure by executing the program stored in one or more memories.

[0151] According to an embodiment of the present disclosure, the electronic device 900 may further include an input / output (I / O) interface 905, which is also connected to the bus 904. The electronic device 900 may further include one or more of the following components connected to the I / O interface 905: an input portion 906 including a keyboard, a mouse, etc.; an output portion 907 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 908 including a hard disk, etc.; and a communication portion 909 including a network interface card such as a LAN card, a modem, etc. The communication portion 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the I / O interface 905 as needed. A removable medium 911, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 910 as needed, so that a computer program read therefrom is installed into the storage portion 908 as needed.

[0152] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present disclosure is implemented.

[0153] According to an embodiment of the present disclosure, a computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present disclosure, a computer-readable storage medium may include the ROM 902 and / or RAM 903 described above and / or one or more memories other than ROM 902 and RAM 903.

[0154] The embodiment of the present disclosure also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the method provided by the embodiment of the present disclosure.

[0155] The above functions defined in the system / device of the embodiment of the present disclosure are performed when the computer program is executed by the processor 901. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0156] In one embodiment, the computer program may be based on a tangible storage medium such as an optical storage device, a magnetic storage device, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of a signal on a network medium, and downloaded and installed through the communication part 909, and / or installed from a removable medium 911. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0157] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 909, and / or installed from the removable medium 911. When the computer program is executed by the processor 901, the above functions defined in the system of the embodiment of the present disclosure are performed. According to the embodiment of the present disclosure, the system, device, means, module, unit, etc. described above can be implemented by a computer program module.

[0158] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect through the Internet).

[0159] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0160] It will be appreciated by those skilled in the art that the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways, even if such combinations and / or combinations are not explicitly described in the present disclosure. In particular, the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways without departing from the spirit and teachings of the present disclosure. All of these combinations and / or combinations fall within the scope of the present disclosure.

[0161] The embodiments of the present disclosure are described above. However, these embodiments are only for illustrative purposes and are not intended to limit the scope of the present disclosure. Although the embodiments are described above separately, this does not mean that the measures in the various embodiments cannot be used in combination to advantage. The scope of the present disclosure is defined by the attached claims and their equivalents. Without departing from the scope of the present disclosure, those skilled in the art may make a variety of substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. A multi-label text classification method, characterized in that: include: Acquire a text to be classified and a first label set, wherein the first label set includes a plurality of labels for text classification; Selecting N candidate labels for the text to be classified from the first label set, where N is an integer greater than or equal to 2; Inputting the text to be classified and the N candidate labels into a multi-label text classification model, wherein the multi-label text classification model is pre-trained on a large language model based on text samples and corresponding label samples; Obtain M target labels predicted by the multi-label text classification model from the N candidate labels, where M is an integer greater than or equal to 1.

2. The method according to claim 1, characterized in that After obtaining the M target labels predicted by the multi-label text classification model from the N candidate labels, the method further includes: Inputting the to-be-classified text and the M target labels into an evaluation model, wherein the evaluation model is pre-trained by a machine learning model based on text samples, their corresponding label samples, and matching degree samples; Obtaining a label matching degree output by the evaluation model, wherein the label matching degree represents a matching degree between the text to be classified and the M target labels; A multi-label classification result of the text to be classified is determined based on the label matching degree.

3. The method according to claim 2, characterized in that When the label matching degree is less than a preset value, determining the multi-label classification result of the to-be-classified text based on the label matching degree includes: Update N candidate labels of the text to be classified from the first label set; Inputting the text to be classified and the updated N candidate labels into the multi-label text classification model; Obtain K target labels predicted by the multi-label text classification model from the updated N candidate labels, where K is an integer greater than or equal to 1.

4. The method according to claim 3, characterized in that Selecting N candidate tags for the text to be classified from the first tag set includes: Retrieving S1 candidate texts from a vector knowledge base based on the text to be classified, wherein the similarity between each candidate text and the text to be classified satisfies a first preset condition, the vectorized candidate texts and corresponding label sets are stored in the vector knowledge base, and S1 is an integer greater than or equal to 1; The N candidate tags are obtained based on the tag sets of the S1 candidate texts.

5. The method according to claim 4, characterized in that Updating the N candidate labels of the to-be-classified text from the first label set includes: Retrieving S2 candidate texts from a vector knowledge base based on the text to be classified, wherein the similarity between each candidate text and the text to be classified satisfies a second preset condition, the similarity requirement in the second preset condition is lower than the similarity requirement in the first preset condition, and S2 is an integer greater than S1; The updated N candidate labels are obtained based on the label sets of the S2 candidate texts.

6. The method according to claim 4, characterized in that The S1 candidate texts are retrieved based on a first similarity algorithm, and updating the N candidate tags of the to-be-classified text from the first tag set includes: Based on the text to be classified, S3 candidate texts are retrieved from the vector knowledge base using a second similarity algorithm, wherein the similarity between each candidate text and the text to be classified satisfies the first preset condition, the first similarity algorithm is different from the second similarity algorithm, and S3 is an integer greater than or equal to 1.

7. The method according to claim 3, characterized in that Selecting N candidate tags for the text to be classified from the first tag set includes: Inputting the to-be-classified text into a first pre-classification model, wherein the first pre-classification model is pre-trained by a machine learning model based on text samples and corresponding label samples, and the first label set includes the text samples and corresponding label samples; Obtain the N candidate labels output by the first pre-classification model.

8. The method according to claim 7, characterized in that Updating the N candidate labels of the to-be-classified text from the first label set includes: Inputting the to-be-classified text into a second pre-classification model, where the second pre-classification model is pre-trained by a machine learning model based on text samples and corresponding label samples, and the second pre-classification model is different from the first pre-classification model; Obtain the updated N candidate labels output by the second pre-classification model.

9. The method according to claim 2, characterized in that: A first prompt word is pre-configured, and the text to be classified and the N candidate labels are input into a multi-label text classification model, comprising: The text to be classified, the N candidate labels and the first prompt word are input into the multi-label text classification model, wherein the first prompt word is used to guide the multi-label text classification model to generate a multi-label classification result of the text to be classified.

10. The method according to claim 9, characterized in that A second prompt word is pre-configured, and the second prompt word is different from the first prompt word. When the label matching degree is less than a preset value, determining the multi-label classification result of the text to be classified based on the label matching degree includes: The text to be classified, the N candidate labels and the second prompt word are input into the multi-label text classification model, wherein the second prompt word is used to guide the multi-label text classification model to generate a multi-label classification result of the text to be classified.

11. The method according to claim 2, characterized in that When the label matching degree is greater than or equal to a preset value, determining the multi-label classification result of the to-be-classified text based on the label matching degree includes: The M target labels are used as the multi-label classification results of the text to be classified.

12. A multi-label text classification device, characterized in that: include: A data acquisition module, used to acquire the text to be classified and a first label set, wherein the first label set includes a plurality of labels for text classification; A candidate label module, used to select N candidate labels for the text to be classified from the first label set, where N is an integer greater than or equal to 2; A data input module, used for inputting the text to be classified and the N candidate labels into a multi-label text classification model, wherein the multi-label text classification model is pre-trained by a large language model based on text samples and corresponding label samples; The target label module is used to obtain the M target labels predicted by the multi-label text classification model from the N candidate labels, where M is an integer greater than or equal to 1.

13. An electronic device, comprising: one or more processors; a memory for storing one or more computer programs, It is characterized in that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 11.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 11 are implemented.

15. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 11 are implemented.

Images