Multi-label text classification method, device and storage medium

Abstract

The present disclosure relates to a multi-label text classification method, device and storage medium. The multi-label text classification method includes: determining text data to be labeled; extracting text features of the text data based on the ALBERT language model; using the text features as the input of the encoding part of the sequence-to-sequence model to obtain the encoding output and the encoding hidden state, and using the encoding output and the encoding hidden state as the input of the decoding part of the sequence-to-sequence model to obtain one or more words of the decoded output; mapping the one or more words of the decoded output to one or more labels, and obtaining the label of the text data based on the one or more labels. The present disclosure can improve the accuracy and recall rate of multi-label text classification.

Description

Technical Field

[0001] The present disclosure relates to the field of natural language processing, and in particular to a multi-label text classification method, device and storage medium. Background Art

[0002] Text classification in natural language processing can automatically determine the category associated with a text based on the text content under a given classification system. Multi-label text classification is a type of text classification that can extract multiple labels from a text and distinguish the determined text category through multiple labels.

[0003] In related technologies, multi-label text classification is usually performed by extracting text classification labels based on language models and downstream tasks, such as language models such as word2vec and BERT, and combining downstream tasks such as CNN, RNN, or Full-Connection for label extraction.

[0004] However, the current multi-label text classification method has low accuracy and cannot obtain the order of label appearance, the number of times the label appears, etc. Summary of the invention

[0005] In order to overcome the problems existing in the related art, the present disclosure provides a multi-label text classification method, device and storage medium.

[0006] According to a first aspect of an embodiment of the present disclosure, a multi-label text classification method is provided, comprising:

[0007] Determine text data to be labeled; extract text features of the text data based on the ALBERT language model; use the text features as input to the encoding part of the sequence-to-sequence model to obtain an encoding output and an encoding latent state, and use the encoding output and the encoding latent state as input to the decoding part of the sequence-to-sequence model to obtain one or more words of the decoded output; map the one or more words of the decoded output to one or more labels, and obtain a label for the text data based on the one or more labels.

[0008] In one implementation, the extracting text features of the text data based on the ALBERT language model includes:

[0009] A text feature vector of the text data is extracted based on the ALBERT language model, wherein the text feature vector includes the number of samples fed each time the model is updated, the maximum length of the text, and the hidden layer size of the ALBERT language model word vector.

[0010] In one implementation, the encoded output and the encoded hidden state are used as inputs of a decoding part to obtain one or more words of the decoded output, including:

[0011] The encoded output is used as the input of each unidirectional long short-term memory network LSTM layer in the decoding part to obtain the output of each unidirectional LSTM layer; the local attention weight of the output of each unidirectional LSTM layer is determined based on the encoded hidden state, and the weighted sum of the output of each unidirectional LSTM layer is performed based on the local attention weight to obtain a semantic vector; based on the semantic vector, one or more words corresponding to the decoded output and the decoded hidden state in each unidirectional LSTM layer and used to be mapped as a label are determined.

[0012] In one implementation, based on the words corresponding to the decoded outputs and decoded hidden states in each unidirectional LSTM layer, one or more words for mapping as labels are determined, including: performing a beam search based on the words corresponding to the decoded outputs and decoded hidden states in each unidirectional LSTM layer, and using the one or more words obtained after the beam search as the one or more words for mapping as labels.

[0013] In one implementation, obtaining a label for the text data based on the one or more labels includes:

[0014] The one or more tags are sequentially combined in the order of appearance to serve as tags for the text data.

[0015] In one implementation, determining the text data to be tagged includes: using comments of users on the e-commerce platform as the text data to be tagged.

[0016] According to a second aspect of an embodiment of the present disclosure, a multi-label text classification device is provided, comprising:

[0017] A determination unit is used to determine text data to be labeled; a feature extraction unit is used to extract text features of the text data based on the ALBERT language model; an encoding unit is used to use the text features as input to the encoding part of the sequence-to-sequence model to obtain an encoded output and an encoded latent state; a decoding unit is used to use the encoded output and the encoded latent state as input to the decoding part of the sequence-to-sequence model to obtain one or more words of the decoded output; a label mapping unit is used to map the one or more words of the decoded output to one or more labels, and obtain a label of the text data based on the one or more labels.

[0018] In one implementation, the feature extraction unit extracts text features of the text data based on the ALBERT language model in the following manner:

[0019] A text feature vector of the text data is extracted based on the ALBERT language model, wherein the text feature vector includes the number of samples fed each time the model is updated, the maximum length of the text, and the hidden layer size of the ALBERT language model word vector.

[0020] In one embodiment, the decoding unit is used to use the encoded output and the encoded hidden state as inputs to the decoding part to obtain one or more words of the decoded output in the following manner:

[0021] The encoded output is used as the input of each unidirectional long short-term memory network LSTM layer in the decoding part to obtain the output of each unidirectional LSTM layer; the local attention weight of the output of each unidirectional LSTM layer is determined based on the encoded hidden state, and the weighted sum of the output of each unidirectional LSTM layer is performed based on the local attention weight to obtain a semantic vector; based on the semantic vector, one or more words corresponding to the decoded output and the decoded hidden state in each unidirectional LSTM layer and used to be mapped as a label are determined.

[0022] In one embodiment, the decoding unit is used to determine one or more words used to be mapped as labels based on the words corresponding to the decoded outputs and decoded hidden states in each unidirectional LSTM layer in the following manner: a beam search is performed based on the words corresponding to the decoded outputs and decoded hidden states in each unidirectional LSTM layer, and one or more words obtained after the beam search are used as one or more words used to be mapped as labels.

[0023] In one implementation, the label mapping unit obtains the label of the text data based on the one or more labels in the following manner: sequentially combining the one or more labels to obtain the label of the text data.

[0024] In one implementation, the determining unit determines the text data to be labeled in the following manner: using comments of users in the e-commerce platform as the text data to be labeled.

[0025] According to a third aspect of an embodiment of the present disclosure, a multi-label text classification device is provided, comprising:

[0026] a processor; a memory for storing instructions executable by the processor;

[0027] The processor is configured to: execute the multi-label text classification method described in the first aspect or any one of the implementations of the first aspect.

[0028] According to a fourth aspect of an embodiment of the present disclosure, a non-temporary computer-readable storage medium is provided. When instructions in the storage medium are executed by a processor of a terminal, the terminal is enabled to execute the multi-label text classification method intelligently described in the first aspect or any one of the implementations of the first aspect.

[0029] The technical solution provided by the embodiments of the present disclosure may include the following beneficial effects: extracting text features of text data based on the ALBERT language model, and using the text features as the input of the encoding part of the sequence-to-sequence model, mapping the output of the decoding part of the sequence-to-sequence model to one or more labels, and implementing labeling of text data based on the fusion of the ALBERT language model and the sequence-to-sequence model, which can improve the accuracy of labeling, and can determine the order of label appearance and the number of label appearances based on the number of words in the decoded output, thereby improving the accuracy and recall rate of multi-label text classification.

[0030] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0032] Figure 1 The figure is a flowchart of a multi-label text classification method according to an exemplary embodiment.

[0033] Figure 2 It is a block diagram of a multi-label text classification implementation process shown in an exemplary embodiment of the present disclosure.

[0034] Figure 3 It is a flowchart of a method for extracting text features of text data based on the ALBERT language model according to an exemplary embodiment of the present disclosure.

[0035] Figure 4 It is a flowchart of a method for determining mapping labels based on a local attention mechanism according to an exemplary embodiment of the present disclosure.

[0036] Figure 5 A process schematic diagram of a multi-label text classification task shown in an exemplary embodiment of the present disclosure is shown.

[0037] Figure 6 A schematic diagram of performing multi-label text classification in an exemplary embodiment of the present disclosure is shown.

[0038] Figure 7 The figure is a block diagram of a multi-label text classification device according to an exemplary embodiment.

[0039] Figure 8 The invention is a block diagram of a device for multi-label text classification according to an exemplary embodiment. DETAILED DESCRIPTION

[0040] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0041] The multi-label text classification method provided by the embodiment of the present disclosure integrates the ALBERT language model and the sequence-to-sequence (Seq2seq) model, and performs a multi-label text classification task based on the integrated ALBERT language model and sequence-to-sequence model.

[0042] In one implementation, the disclosed embodiment extracts text features of text data based on the ALBERT language model. The text features of the text data extracted by the ALBERT language model are used as inputs to the encoding part of the sequence-to-sequence model to obtain an encoding output and an encoding hidden state. The encoding output and the encoding hidden state are used as inputs to the decoding part to obtain one or more words of the decoding output, and the one or more words of the decoding output are mapped to one or more labels, and a label of the text data is obtained based on the one or more labels.

[0043] The multi-label text classification method provided by the embodiments of the present disclosure can be applied to scenarios of label extraction, for example, label extraction in the e-commerce field, such as extracting labels from user comments in the e-commerce field to implement labeling of the comments.

[0044] In the embodiment of the present disclosure, the multi-label text classification task based on the integrated ALBERT language model and sequence-to-sequence model may be a hardware device with two graphics cards, such as a terminal.

[0045] Figure 1 FIG. 1 is a flowchart of a multi-label text classification method according to an exemplary embodiment. Figure 1 As shown, the multi-label text classification method is used in a terminal and includes the following steps.

[0046] In step S11, text data to be labeled is determined.

[0047] In the embodiment of the present disclosure, the text data to be tagged can be determined according to the needs. The text data to be tagged can be a sentence or an article. For example, the text data to be tagged can be a user's comment on an e-commerce platform.

[0048] In step S12, text features of the text data are extracted based on the ALBERT language model.

[0049] In the embodiment of the present disclosure, the ALBERT language model is used to read the text data to be labeled, and the text features of the text data to be labeled can be obtained. The text features can also be understood as text feature vectors.

[0050] In one embodiment, the text vector obtained by extracting text features from text data to be labeled using the ALBERT language model in the disclosed embodiment may include the number of samples batch_size fed into the model each time the model is updated, the maximum length of the text sequence_length, and the hidden layer size hidden_size of the ALBERT language model word vector. Based on the maximum length of the text sequence_length, zero padding can be performed on texts shorter than the length of sequence_length, and segmentation can be performed on texts longer than the length of sequence_length, that is, only the text of the first length of sequence_length is taken.

[0051] In step S13, the text features extracted by the ALBERT language model are used as the input of the encoding part of the sequence-to-sequence model to obtain the encoded output and encoded hidden state.

[0052] In the disclosed embodiment, the encoding part of the sequence-to-sequence model includes a multi-layer bidirectional long short-term memory network (BiLong Short-Term Memory, Bi LSTM). Each layer in the multi-layer bidirectional Bi LSTM will correspond to the encoding output and encoding hidden state of the bidirectional BiLSTM of this layer. For example, the encoding output may include the number of samples batch_size fed each time the model is updated, the maximum length of the text sequence_length, and the encoding hidden layer size 2*encoder_hidden_size. The encoding hidden state may include the number of samples batch_size fed each time the model is updated and the encoding hidden layer size encoder_hidden_size.

[0053] In step S14, the encoded output and the encoded hidden state are used as input to the decoding part of the sequence-to-sequence model to obtain one or more words of the decoded output.

[0054] The decoding part of the sequence-to-sequence model in the disclosed embodiment includes a multi-layer unidirectional LSTM. Each layer in the multi-layer unidirectional LSTM corresponds to the decoding output and decoding hidden state of the LSTM of that layer. For example, the decoding output may include the number of samples batch_size fed each time the model is updated, the maximum length of the text sequence_length, and the decoding hidden layer size decoder_hidden_size. The decoding hidden state may include the number of samples batch_size fed each time the model is updated and the decoding hidden layer size decoder_hidden_size.

[0055] In one implementation, the disclosed embodiment can use the encoded output and encoded hidden state of the last layer of the multi-layer bidirectional BiLSTM of the encoding part of the sequence-to-sequence model as the input of the decoding part to obtain the decoded output and the decoded hidden state. In the disclosed embodiment, the decoded output of the sequence-to-sequence model can be understood as each output corresponding to a word (token).

[0056] In step S15, one or more characters output by decoding are mapped to one or more labels, and a label of the text data is obtained based on the one or more labels.

[0057] In the disclosed embodiment, one or more characters corresponding to the decoding output of the sequence-to-sequence model are respectively mapped to one or more labels.

[0058] In the disclosed embodiment, the output of LSTM can be mapped to the label identifier (label_id) under the action of full connection (FC) to obtain the prediction result of the mapped label, and the sequence-to-sequence model can be optimized through the difference between the loss function and the true value.

[0059] The multi-label text classification method provided by the embodiment of the present disclosure extracts text features of text data based on the ALBERT language model, and uses the text features as the input of the encoding part of the sequence-to-sequence model, and maps the output of the decoding part of the sequence-to-sequence model to one or more labels, thereby realizing labeling of text data based on the fusion of the ALBERT language model and the sequence-to-sequence model, which can improve the accuracy of labeling, and can determine the order of label appearance and the number of label appearances based on the number of words in the decoded output, thereby improving the accuracy and recall rate of multi-label text classification.

[0060] The embodiments of the present disclosure will illustrate the multi-label text classification method involved above in combination with practical applications.

[0061] Figure 2 is a block diagram of a multi-label text classification implementation process shown in an exemplary embodiment of the present disclosure. Figure 2As shown, the text data to be labeled is used as input to extract text features. When extracting text features, the text is divided into a string sequence after data processing. For example, tokenization is performed, and the text is divided into a string sequence according to specific requirements, wherein the elements in the string sequence are generally called characters (tokens) or words. The string sequence after data processing is input into the ALBERT language model, and the text feature vector is extracted based on the ALBERT language model. The extracted text feature vector is input into the encoding part of the sequence-to-sequence model, and the encoding output and encoding hidden state of the last layer of bidirectional BiLSTM are obtained through the encoding part of the multi-layer bidirectional BiLSTM. The encoding output and encoding hidden state of the last layer of bidirectional BiLSTM are used as the input of the decoding part of the sequence-to-sequence model. The decoding part includes multi-layer unidirectional LSTM. Among them, each layer of unidirectional LSTM includes a local attention mechanism (Local Attention). The output of each layer of unidirectional LSTM has an encoding output output (batch_size, sequence_length, decoder_hidden_size) and an encoding hidden state state (batch_size, decoder_hidden_size), and then under the action of full connection (FC), the output of LSTM is mapped to label id (label id) to obtain the prediction result. Among them, the decoding part of the sequence-to-sequence model of the embodiment of the present disclosure outputs the corresponding token, which can be understood as the label obtained after classifying the text data to be annotated.

[0062] In one example, the embodiment of the present disclosure determines one or more words mapped as labels based on the local attention mechanism. For example, the encoded output is used as the input of each unidirectional LSTM layer in the decoding part to obtain the output of each unidirectional LSTM layer; the local attention weight of each unidirectional LSTM layer output is determined based on the encoded hidden state, and the weighted sum of each unidirectional LSTM layer output is performed based on the local attention weight to obtain a semantic vector. Based on the semantic vector, one or more words corresponding to the decoded output and the decoded hidden state in each unidirectional LSTM layer and used for mapping as labels are determined.

[0063] In the disclosed embodiment, in order to ensure the speed of training and inference, the number of bidirectional BiLSTM layers in the encoding part and the number of unidirectional LSTM layers in the decoding part in the sequence-to-sequence model can be set based on the required training and inference speed.

[0064] Furthermore, in the embodiments of the present disclosure, when extracting text features of text data based on the ALBERT language model, the dimension of the text feature vector of the text data extracted by the ALBERT language model can be determined based on the number of layers of the bidirectional LSTM in the sequence-to-sequence model encoding part.

[0065] Figure 3 FIG. 1 is a flowchart of a method for extracting text features of text data based on an ALBERT language model according to an exemplary embodiment of the present disclosure. Figure 3 As shown, the method for extracting text features of text data based on the ALBERT language model includes the following steps:

[0066] In step S21, the number of layers of the bidirectional long short-term memory network LSTM in the encoding part of the sequence-to-sequence model is determined.

[0067] In step S22, a text feature vector of the text data is extracted based on the ALBERT language model, and the dimension of the text feature vector is the same as the number of layers of the bidirectional LSTM.

[0068] Among them, the text feature vector in the embodiment of the present disclosure includes the number of samples fed each time the model is updated, the maximum length of the text, and the hidden layer size of the ALBERT language model word vector.

[0069] In one example, the number of bidirectional LSTM layers in the encoding part of the sequence-to-sequence model is 3, and the ALBERT language model extracts a three-dimensional text feature vector (batch_size, sequence_length, hidden_size) of text data. Among them, batch_size is the number of samples fed into the model each time it is updated. Sequence_length is the maximum length of the text. Texts shorter than sequence_length are padded with zeros, and texts longer than sequence_length are segmented, that is, only the texts of the previous sequence_length are taken. Hidden_size is the hidden layer size of ALBERT's word vector.

[0070] It can be understood that in the embodiments of the present disclosure, the number of bidirectional LSTM layers is not limited to the same as the dimension of the text feature vector extracted by the LBERT language model. The number of bidirectional LSTM layers can also be smaller than the dimension of the text feature vector extracted by the LBERT language model, or the number of bidirectional LSTM layers must be greater than the dimension of the text feature vector extracted by the LBERT language model.

[0071] In the disclosed embodiment, the decoding part includes multiple layers of unidirectional LSTM. Each layer of unidirectional LSTM includes a local attention mechanism (Local Attention). The output of each layer of unidirectional LSTM has an encoded output output(batch_size, sequence_length, decoder_hidden_size) and an encoded hidden state state(batch_size, decoder_hidden_size). In one implementation, the disclosed embodiment uses the encoded output and the encoded hidden state as the input of the decoding part. When one or more words of the decoded output are obtained, one or more words mapped as labels can be determined based on the local attention mechanism of each unidirectional long short-term memory network LSTM layer in the decoding part.

[0072] Figure 4 is a flowchart of a method for determining a mapping label based on a local attention mechanism according to an exemplary embodiment of the present disclosure. Figure 4 As shown, the method for determining mapping labels based on the local attention mechanism includes the following steps:

[0073] In step S31, the encoded output is used as the input of each unidirectional LSTM layer in the decoding part to obtain the output of each unidirectional LSTM layer.

[0074] In the disclosed embodiment, after the encoded output passes through the corresponding unidirectional LSTM layers, the LSTM layer output after the unidirectional LSTMs are spliced ​​can be realized.

[0075] In step S32, the local attention weights of the outputs of each unidirectional LSTM layer are determined based on the encoded hidden state, and the weighted summation of the outputs of each unidirectional LSTM layer is performed based on the local attention weights to obtain a semantic vector.

[0076] In the disclosed embodiment, the local attention weight of each unidirectional LSTM layer output can be determined based on the encoded hidden state. Each unidirectional LSTM output is multiplied by its own local attention weight and then weighted summed to obtain a semantic vector.

[0077] In step S33, based on the semantic vector, one or more words corresponding to the decoded output and the decoded hidden state in each unidirectional LSTM layer and used for mapping as a label are determined.

[0078] In the disclosed embodiment, based on the word mapped as the label at the previous moment, the word mapped as the label at the current moment, and the semantic vector, the corresponding matrices can be multiplied respectively to obtain one or more words corresponding to the decoding output and decoding hidden state in each unidirectional LSTM layer and used to map as labels.

[0079] Among them, when determining one or more words used to be mapped as labels based on the words corresponding to the decoded output and the decoded hidden state in each unidirectional LSTM layer, a beam search can be performed based on the words corresponding to the decoded output and the decoded hidden state in each unidirectional LSTM layer, and the one or more words obtained after the beam search are used as the one or more words used to be mapped as labels.

[0080] Furthermore, in the embodiment of the present disclosure, label mapping is performed based on the local attention mechanism of the unidirectional LSTM, and the number of label occurrences and the order of the text data to be labeled can be obtained.

[0081] Furthermore, in the disclosed embodiments, a label can be understood as a word, and multiple labels are combined into a query as the final label of the text data to be labeled, which can better determine the number of label occurrences and order of the text data to be labeled. In one embodiment, when obtaining a label for text data based on one or more labels, one or more labels can be combined in order according to the order of appearance as the label of the text data.

[0082] The multi-label text classification method provided by the embodiment of the present disclosure can be applied to the scenario of classifying and labeling user comments on an e-commerce platform. In one example, when determining the text data to be labeled, the comments of users on the e-commerce platform can be used as the text data to be labeled.

[0083] The following embodiment of the present disclosure exemplifies the process of labeling user comments on an e-commerce platform.

[0084] In one example, we first prepare the data to be labeled. The original data to be labeled is the comments of users on the e-commerce platform. For example, we can obtain the comments of users on the e-commerce platform, and then establish a label system. We get a total of 318 label categories. We need to pay attention to the independence and interpretability of the labels. Finally, we label the data. We labeled a total of 200,000 data. During the labeling process, we need to label the order in which the labels appear. In addition, if a label appears multiple times in a sentence, it also needs to be labeled multiple times in the order in which it appears.

[0085] The user comments obtained from the e-commerce platform are input into the integrated ALBERT language model and sequence-to-sequence model for multi-label text classification tasks, and descriptions (labels) of some features of the product can be obtained. For example, if a user's comment on the e-commerce platform is "the phone is not only good-looking but also cheap, and the screen and sound effects are also very good", then the multi-label text classification method provided by the embodiment of the present disclosure is applied, and the multi-label text classification task is input into the integrated ALBERT language model and sequence-to-sequence model for the multi-label text classification task. The schematic diagram is as follows: Figure 5 shown. Figure 5FIG. 1 shows a process diagram of a multi-label text classification task according to an exemplary embodiment of the present disclosure. Figure 5 As shown in the figure, "The phone is not only good-looking but also cheap, and the screen and sound effects are also very good." After data processing and text feature vector extraction by the ALBERT language model, the text feature vector (x1, x2...xn-1, xn-2) can be obtained, that is, the phone...is very good. The text feature vector (x1, x2...xn-1, xn-2) is used as the encoding part input of the sequence-to-sequence model, and the encoding output and encoding hidden state are obtained accordingly. The local attention mechanism (corresponding to the encoding of the three-layer bidirectional LSTM and the decoding of the three-layer unidirectional LSTM) Figure 5 c1, c2 and c3 in the figure), the corresponding decoded outputs Y1, Y2, Y3 and Y4 can be obtained. The decoded outputs Y1, Y2, Y3 and Y4 can be respectively mapped to labels including good-looking appearance, reasonable price, screen display and sound effects. Through the above method, the disclosed embodiment can pass the user's comment "the mobile phone is not only good-looking but also cheap, and the screen and sound effects are also very good" to obtain the four features (labels) described by the user for the product: "good-looking appearance", "reasonable price", "good screen display" and "good sound effects". By obtaining more user comments and opinions, it is possible to understand the user's concerns about the product, thereby providing support for solving problems existing in current products and launching new products.

[0086] Figure 6 A schematic diagram of text multi-label classification in an exemplary embodiment of the present disclosure is shown. Input the sentence "The mobile phone is not only good-looking but also cheap, and the screen and sound effects are also very good", then the corresponding labels "good appearance", "reasonable price", "good screen display" and "good sound effects" can be obtained. Combining these four labels can finally get a label corresponding to the sentence (which can be understood as a query, a sentence). Among them, Figure 6 In the example, "good appearance" is mapped to label a (Labet: a), "reasonable price" is mapped to label b (Labet: b), "good screen display" is mapped to label c (Labet: c), and "good sound effect" is mapped to label d (Labet: d). Finally, the label "abcd" is output as the label of "the mobile phone is not only good-looking and cheap, but also has good screen and sound effects".

[0087] Based on the same concept, the embodiment of the present disclosure also provides a multi-label text classification device.

[0088] It is understandable that in order to realize the above functions, the multi-label text classification device provided by the embodiment of the present disclosure includes hardware structures and / or software modules corresponding to the execution of each function. In combination with the units and algorithm steps of each example disclosed in the embodiment of the present disclosure, the embodiment of the present disclosure can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the technical solution of the embodiment of the present disclosure.

[0089] Figure 7 FIG. 1 is a block diagram of a multi-label text classification device according to an exemplary embodiment. Figure 7 The multi-label text classification device 100 comprises:

[0090] The determination unit 101 is used to determine the text data to be labeled. The feature extraction unit 102 is used to extract the text features of the text data based on the ALBERT language model. The encoding unit 103 is used to use the text features as the input of the encoding part of the sequence-to-sequence model to obtain the encoding output and the encoding hidden state. The decoding unit 104 is used to use the encoding output and the encoding hidden state as the input of the decoding part of the sequence-to-sequence model to obtain one or more words of the decoded output. The label mapping unit 105 is used to map the one or more words of the decoded output to one or more labels, and obtain the label of the text data based on the one or more labels.

[0091] In one implementation, the feature extraction unit 102 extracts text features of text data based on the ALBERT language model in the following manner:

[0092] The text feature vector of the text data is extracted based on the ALBERT language model. The text feature vector includes the number of samples fed into the model each time, the maximum length of the text, and the hidden layer size of the ALBERT language model word vector.

[0093] In one implementation, the decoding unit 104 is configured to use the encoded output and the encoded hidden state as inputs to the decoding part to obtain one or more words of the decoded output in the following manner:

[0094] The encoded output is used as the input of each unidirectional long short-term memory network LSTM layer in the decoding part to obtain the output of each unidirectional LSTM layer; the local attention weight of each unidirectional LSTM layer output is determined based on the encoded hidden state, and the weighted sum of each unidirectional LSTM layer output is performed based on the local attention weight to obtain a semantic vector; based on the semantic vector, one or more words corresponding to the decoded output and the decoded hidden state in each unidirectional LSTM layer and used to map as a label are determined.

[0095] In one implementation, the decoding unit 104 is used to determine one or more words for mapping as labels based on the words corresponding to the decoded outputs and decoded hidden states in each unidirectional LSTM layer in the following manner: a beam search is performed based on the words corresponding to the decoded outputs and decoded hidden states in each unidirectional LSTM layer, and one or more words obtained after the beam search are used as one or more words for mapping as labels.

[0096] In one implementation, the label mapping unit 105 obtains the label of the text data based on one or more labels in the following manner: one or more labels are sequentially combined to serve as the label of the text data.

[0097] In one implementation, the determining unit 101 determines the text data to be labeled in the following manner: using comments of users in the e-commerce platform as the text data to be labeled.

[0098] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0099] Figure 8 1 is a block diagram of an apparatus 200 for multi-label text classification according to an exemplary embodiment. For example, the apparatus 200 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0100] Reference Figure 8 , the device 200 may include one or more of the following components: a processing component 202 , a memory 204 , a power component 206 , a multimedia component 208 , an audio component 210 , an input / output (I / O) interface 212 , a sensor component 214 , and a communication component 216 .

[0101] The processing component 202 generally controls the overall operation of the device 200, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 202 may include one or more processors 220 to execute instructions to perform all or part of the steps of the above-described method. In addition, the processing component 202 may include one or more modules to facilitate interaction between the processing component 202 and other components. For example, the processing component 202 may include a multimedia module to facilitate interaction between the multimedia component 208 and the processing component 202.

[0102] The memory 204 is configured to store various types of data to support operations on the device 200. Examples of such data include instructions for any application or method operating on the device 200, contact data, phone book data, messages, pictures, videos, etc. The memory 204 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0103] The power component 206 provides power to the various components of the device 200. The power component 206 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 200.

[0104] The multimedia component 208 includes a screen that provides an output interface between the device 200 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 208 includes a front camera and / or a rear camera. When the device 200 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0105] The audio component 210 is configured to output and / or input audio signals. For example, the audio component 210 includes a microphone (MIC), and when the device 200 is in an operation mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 204 or sent via the communication component 216. In some embodiments, the audio component 210 also includes a speaker for outputting audio signals.

[0106] I / O interface 212 provides an interface between processing component 202 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0107] The sensor assembly 214 includes one or more sensors for providing various aspects of the status assessment of the device 200. For example, the sensor assembly 214 can detect the open / closed state of the device 200, the relative positioning of components, such as the display and keypad of the device 200, the sensor assembly 214 can also detect the position change of the device 200 or a component of the device 200, the presence or absence of user contact with the device 200, the orientation or acceleration / deceleration of the device 200 and the temperature change of the device 200. The sensor assembly 214 can include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 214 can also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 214 can also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor or a temperature sensor.

[0108] The communication component 216 is configured to facilitate wired or wireless communication between the device 200 and other devices. The device 200 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 216 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 216 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0109] In an exemplary embodiment, the apparatus 200 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above method.

[0110] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 204 including instructions, and the instructions can be executed by the processor 220 of the device 200 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0111] It is to be understood that in the present disclosure, "plurality" refers to two or more than two, and other quantifiers are similar. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects before and after are in an "or" relationship. The singular forms "a", "the" and "the" are also intended to include plural forms, unless the context clearly indicates other meanings.

[0112] It is further understood that the terms "first", "second", etc. are used to describe various information, but such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other, and do not indicate a specific order or degree of importance. In fact, the expressions "first", "second", etc. can be used interchangeably. For example, without departing from the scope of the present disclosure, the first information can also be referred to as the second information, and similarly, the second information can also be referred to as the first information.

[0113] It can be further understood that, unless otherwise specified, “connection” includes a direct connection without other components between the two, and also includes an indirect connection with other components between the two.

[0114] It is further understood that, although the operations are described in a specific order in the drawings in the embodiments of the present disclosure, it should not be understood as requiring the operations to be performed in the specific order shown or in a serial order, or requiring the execution of all the operations shown to obtain the desired results. In certain environments, multitasking and parallel processing may be advantageous.

[0115] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0116] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A multi-label text classification method, characterized in that: include: Determine the text data to be labeled; Extracting text features of the text data based on the ALBERT language model; Using the text features as input to an encoding part of a sequence-to-sequence model to obtain an encoding output and an encoding hidden state, and using the encoding output and the encoding hidden state as input to a decoding part of the sequence-to-sequence model to obtain one or more words of a decoded output; Mapping the one or more characters output by the decoding to one or more labels, and obtaining a label of the text data based on the one or more labels; The extracting of text features of the text data based on the ALBERT language model includes: Extracting a text feature vector of the text data based on the ALBERT language model, wherein the text feature vector includes the number of samples fed each time the model is updated, the maximum length of the text, and the hidden layer size of the ALBERT language model word vector; The step of using the encoded output and the encoded hidden state as inputs to a decoding part of a sequence-to-sequence model to obtain one or more words of the decoded output comprises: Using the encoded output as input to each unidirectional long short-term memory network LSTM layer in the decoding part, determining one or more words corresponding to the decoded output and the decoded hidden state in each unidirectional LSTM layer and used for mapping as a label; The determining one or more words corresponding to the decoding output and the decoding hidden state in each unidirectional LSTM layer and used for mapping as a label includes: A beam search is performed based on the words corresponding to the decoded outputs and the decoded hidden states in each unidirectional LSTM layer, and one or more words obtained after the beam search are used as one or more words for mapping to labels.

2. The multi-label text classification method according to claim 1, characterized in that: The step of using the encoded output as the input of each unidirectional long short-term memory network LSTM layer in the decoding part, and determining one or more words corresponding to the decoded output and the decoded hidden state in each unidirectional LSTM layer and used for mapping as a label, comprises: Using the encoded output as the input of each unidirectional long short-term memory network LSTM layer in the decoding part to obtain the output of each unidirectional LSTM layer; Determine the local attention weights of the outputs of each unidirectional LSTM layer based on the encoded hidden state, and sum the weights of the outputs of each unidirectional LSTM layer based on the local attention weights to obtain a semantic vector; Based on the semantic vector, one or more words corresponding to the decoding output and the decoding hidden state in each unidirectional LSTM layer and used for mapping as a label are determined.

3. The multi-label text classification method according to claim 1, characterized in that: The obtaining a label of the text data based on the one or more labels includes: The one or more tags are sequentially combined in the order of appearance to serve as tags for the text data.

4. The multi-label text classification method according to claim 1, characterized in that: The step of determining the text data to be labeled includes: The comments of users on the e-commerce platform are used as text data to be labeled.

5. A multi-label text classification device, characterized in that: include: A determination unit, used to determine text data to be labeled; A feature extraction unit, used for extracting text features of the text data based on an ALBERT language model; An encoding unit, configured to use the text features as input to an encoding part of a sequence-to-sequence model to obtain an encoding output and an encoding latent state; A decoding unit, configured to use the encoded output and the encoded hidden state as input to a decoding part of a sequence-to-sequence model to obtain one or more words of a decoded output; A label mapping unit, used for mapping one or more characters output by the decoding to one or more labels, and obtaining a label of the text data based on the one or more labels; The feature extraction unit extracts text features of the text data based on the ALBERT language model in the following manner: Extracting a text feature vector of the text data based on the ALBERT language model, wherein the text feature vector includes the number of samples fed each time the model is updated, the maximum length of the text, and the hidden layer size of the ALBERT language model word vector; The decoding unit is used to use the encoded output and the encoded hidden state as inputs of the decoding part to obtain one or more words of the decoded output in the following manner: Using the encoded output as input to each unidirectional long short-term memory network LSTM layer in the decoding part, determining one or more words corresponding to the decoded output and the decoded hidden state in each unidirectional LSTM layer and used for mapping as a label; The decoding unit determines one or more words corresponding to the decoding output and the decoding hidden state in each unidirectional LSTM layer and used for mapping as a label in the following manner, including: A beam search is performed based on the words corresponding to the decoded outputs and the decoded hidden states in each unidirectional LSTM layer, and one or more words obtained after the beam search are used as one or more words for mapping to labels.

6. The multi-label text classification device according to claim 5, characterized in that: The decoding unit is used to use the encoding output as the input of each unidirectional long short-term memory network LSTM layer in the decoding part in the following manner, and determine one or more words corresponding to the decoding output and the decoding hidden state in each unidirectional LSTM layer and used for mapping as a label: Using the encoded output as the input of each unidirectional long short-term memory network LSTM layer in the decoding part to obtain the output of each unidirectional LSTM layer; Determine the local attention weights of the outputs of each unidirectional LSTM layer based on the encoded hidden state, and sum the weights of the outputs of each unidirectional LSTM layer based on the local attention weights to obtain a semantic vector; Based on the semantic vector, one or more words corresponding to the decoding output and the decoding hidden state in each unidirectional LSTM layer and used for mapping as a label are determined.

7. The multi-label text classification device according to claim 5, characterized in that: The label mapping unit obtains the label of the text data based on the one or more labels in the following manner: The one or more tags are sequentially combined to serve as tags for the text data.

8. The multi-label text classification device according to claim 5, characterized in that: The determining unit determines the text data to be labeled in the following manner: The comments of users on the e-commerce platform are used as text data to be labeled.

9. A multi-label text classification device, characterized in that: include: processor; a memory for storing processor-executable instructions; Wherein, the processor is configured to: execute the multi-label text classification method described in any one of claims 1 to 4.

10. A non-transitory computer-readable storage medium, when the instructions in the storage medium are executed by a processor of a terminal, enables the terminal to execute the multi-label text classification method described in any one of claims 1 to 4.

Images