Multi-label classification method, device and readable storage medium

Abstract

The present application discloses a multi-label classification method, device and readable storage medium, which relates to the field of data processing technology, in order to improve the accuracy of the order of predicted labels. The method comprises: obtaining a text to be processed; using the text to be processed as the input of a multi-label classification model, running the multi-label classification model, and obtaining labels included in the text to be processed, wherein the labels include sequence information; wherein the multi-label classification model is trained based on the sequence information of the labels included in the text. The embodiment of the present application can improve the accuracy of the order of predicted labels.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a multi-label classification method, device and readable storage medium. Background Art

[0002] Usually, a piece of data may have one or more labels. The purpose of multi-label classification methods is to identify these labels. At present, multi-label classification methods can be divided into three categories. One method is to decompose multiple labels into multiple independent binary classification problems; the second method is to process based on the pairwise association between labels; the third method is to process based on the association between multiple labels, such as considering the influence of all other labels on each label.

[0003] However, in multi-label classification tasks, the order of labels is of great significance. Therefore, how to use the order information of labels for label classification to improve the accuracy of the predicted label order is an issue that needs to be considered. Summary of the invention

[0004] Embodiments of the present application provide a multi-label classification method, device, and readable storage medium to improve the accuracy of the predicted label order.

[0005] In a first aspect, an embodiment of the present application provides a multi-label classification method, including:

[0006] Get the text to be processed;

[0007] Using the text to be processed as an input of a multi-label classification model, running the multi-label classification model, and obtaining labels included in the text to be processed, wherein the labels include sequence information;

[0008] The multi-label classification model is trained based on the sequential information of labels included in the text.

[0009] Optionally, before obtaining the text to be processed, the method further includes:

[0010] The multi-label classification model is trained.

[0011] Optionally, the training of the multi-label classification model includes:

[0012] Preprocessing the training text, including data format conversion, data cleaning, and label sorting;

[0013] Get the word vectors in the preprocessed training text;

[0014] According to the word vector, a hidden state representation of each word in the training text is obtained;

[0015] Generate a label vector of the training text, wherein the label vector includes sequence information of the labels of the training text;

[0016] The label probability of the training text is obtained according to the hidden state representation and the label vector.

[0017] Optionally, after obtaining the hidden state representation of each word in the training text according to the word vector, the method further includes:

[0018] Obtaining a context vector of the training text according to the hidden state representation;

[0019] The step of obtaining the label probability of the training text according to the hidden state representation and the label vector includes:

[0020] The label probability of the training text is obtained according to the context vector and the label vector.

[0021] Optionally, generating a label vector for the training text includes:

[0022] By means of linear regression, a reverse mapping function is constructed, wherein the reverse mapping function is a mapping function from a vector space to a label sorting space;

[0023] The reverse mapping function is used to obtain label vectors of various labels of the training text.

[0024] Optionally, generating a label vector for the training text includes:

[0025] The label vectors of each label of the training text are calculated using the following formula:

[0026]

[0027] Among them, ranking(y (i,2j) ) represents the label vector, i represents the order index, j represents the dimension index, d model Represents the preset model dimension, j is an even number;

[0028]

[0029] Among them, ranking(y (i,2j+1) ) represents the label vector, i represents the order index, j represents the dimension index, d model Represents the preset model dimension, j is an odd number.

[0030] Optionally, obtaining the label probability of the training text according to the context vector and the label vector includes:

[0031] The label probability of the training text is obtained using the following formula:

[0032] y t =softmax(O t +I t )

[0033] Among them, y t represents the probability of time step t in the label space L; O t represents the transformation vector, I t represents the mask vector;

[0034] O t =W o f(W d S t +V d C t ), W o , W d , V d are greater than 0, representing weight parameters, S t represents the hidden state at time step t, C t represents the context vector at time step t; f represents the nonlinear activation function;

[0035] S t =LSTM(S t-1 [rangking(y t-1 ); g(y t-1 );C t ]), S t-1 represents the hidden state at time step t-1, g(y t-1 ) is in the probability distribution y t-1 The label vector of the label with the highest probability; y t-1 is the probability distribution of time step t-1 in the label space L; [ranging(y t-1 ); g(y t-1 );C t ] represents the vector (y t-1 )、g(y t-1 ), C t connection;

[0036] Among them, a ti Greater than 0, indicating the weight of the i-th word in the training text, h i represents the hidden state representation of the i-th word.

[0037] In a second aspect, an embodiment of the present application further provides a multi-label classification device, including:

[0038] A first acquisition module, used to acquire the text to be processed;

[0039] A first processing module, used for taking the text to be processed as an input of a multi-label classification model, running the multi-label classification model, and obtaining labels included in the text to be processed, wherein the labels include sequence information;

[0040] The multi-label classification model is trained based on the sequential information of labels included in the text.

[0041] Optionally, the device further comprises:

[0042] A training module is used to train the multi-label classification model.

[0043] Optionally, the training module includes:

[0044] A preprocessing submodule, used for preprocessing the training text, wherein the preprocessing includes label sorting;

[0045] The first acquisition submodule is used to obtain word vectors in the preprocessed training text;

[0046] A second acquisition submodule is used to obtain a hidden state representation of each word in the training text according to the word vector;

[0047] A generating submodule, used to generate a label vector of the training text, wherein the label vector includes sequence information of the labels of the training text;

[0048] The third acquisition submodule is used to obtain the label probability of the training text according to the hidden state representation and the label vector.

[0049] Optionally, the training module also includes:

[0050] A fourth acquisition submodule, used to obtain a context vector of the training text according to the hidden state representation;

[0051] The third acquisition submodule is used to obtain the label probability of the training text according to the context vector and the label vector.

[0052] Optionally, the generating submodule includes:

[0053] A first construction unit is used to construct a reverse mapping function by a linear regression method, wherein the reverse mapping function is a mapping function from a vector space to a label sorting space;

[0054] The first acquisition unit is used to obtain the label vector of each label of the training text by using the reverse mapping function.

[0055] Optionally, the generating submodule is used to calculate the label vector of each label of the training text using the following formula:

[0056]

[0057] Among them, ranking(y (i,2j) ) represents the label vector, i represents the order index, j represents the dimension index, d model Represents the preset model dimension, j is an even number;

[0058]

[0059] Among them, ranking(y (i,2j+1) ) represents the label vector, i represents the order index, j represents the dimension index, d model Represents the preset model dimension, j is an odd number.

[0060] Optionally, the fourth acquisition submodule is used to:

[0061] The label probability of the training text is obtained using the following formula:

[0062] y t =softmax(O t +I t )

[0063] Among them, y t represents the probability of time step t in the label space L; O t represents the transformation vector, I t represents the mask vector;

[0064] O t =W o f(W d S t +V d C t ), W o , W d , V d are greater than 0, representing weight parameters, S t represents the hidden state at time step t, C t represents the context vector at time step t; f represents the nonlinear activation function;

[0065] S t =LSTM(S t-1 [rangking(y t-1 ); g(y t-1 );C t ]), S t-1 represents the hidden state at time step t-1, g(yt-1 ) is in the probability distribution y t-1 The label vector of the label with the highest probability; y t-1 is the probability distribution of time step t-1 in the label space L; [ranging(y t-1 ); g(y t-1 );C t ] represents the vector (y t-1 )、g(y t-1 ), C t connection;

[0066] Among them, a ti Greater than 0, indicating the weight of the i-th word in the training text, h i represents the hidden state representation of the i-th word.

[0067] In a third aspect, an embodiment of the present application further provides a multi-label classification device, including:

[0068] Processor; and

[0069] A memory, wherein program instructions are stored in the memory, wherein when the program instructions are executed by the processor, the processor is caused to perform the following steps:

[0070] Get the text to be processed;

[0071] Using the text to be processed as an input of a multi-label classification model, running the multi-label classification model, and obtaining labels included in the text to be processed, wherein the labels include sequence information;

[0072] The multi-label classification model is trained based on the sequential information of labels included in the text.

[0073] In a fourth aspect, an embodiment of the present application further provides a readable storage medium, on which a program is stored, and when the program is executed by a processor, the steps in the multi-label classification method as described above are implemented.

[0074] In an embodiment of the present application, when multi-label classification is performed on a text to be processed, the text to be processed is used as the input of a multi-label classification model, and the labels included in the text to be processed are obtained by running the multi-label classification model; the labels include sequence information; and the multi-label classification model is trained based on the sequence information of the labels included in the text. Since the sequence information of the labels of the text is taken into account when training the multi-label classification model, when performing multi-label classification, the labels of the text to be processed with a certain sequence can be accurately obtained, thereby improving the accuracy of the predicted sequence of the labels. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] Figure 1 This is one of the flow charts of the multi-label classification method provided in the embodiment of the present application;

[0076] Figure 2 This is the second flowchart of the multi-label classification method provided in the embodiment of the present application;

[0077] Figure 3 This is one of the flowcharts of training a multi-label classification model provided in an embodiment of the present application;

[0078] Figure 4 This is the second flowchart of training a multi-label classification model provided in an embodiment of the present application;

[0079] Figure 5 is a schematic diagram of a multi-label classification system provided in an embodiment of the present application;

[0080] Figure 6 This is one of the structural diagrams of the multi-label classification device provided in the embodiment of the present application;

[0081] Figure 7 This is the second structural diagram of the multi-label classification device provided in the embodiment of the present application;

[0082] Figure 8 It is one of the structural diagrams of the training module provided in the embodiment of the present application;

[0083] Fig. 9 This is the second structural diagram of the training module provided in the embodiment of the present application;

[0084] Fig.10 is a structural diagram of a generation submodule provided in an embodiment of the present application;

[0085] Fig.11 This is the third structural diagram of the multi-label classification device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0086] In the embodiments of the present application, the term "and / or" describes the association relationship of the associated objects, indicating that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects before and after are in an "or" relationship.

[0087] In the embodiments of the present application, the term "plurality" refers to two or more than two, and other quantifiers are similar.

[0088] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0089] See also Figure 1 , Figure 1 is a flowchart of the multi-label classification method provided in the embodiment of the present application. Figure 1 As shown, the following steps are included:

[0090] Step 101: Obtain the text to be processed.

[0091] The text to be processed may be any text, for example, a diagnosis report, a rectification report, a statistical report, etc.

[0092] Step 102: using the text to be processed as an input of a multi-label classification model, running the multi-label classification model, and obtaining labels included in the text to be processed.

[0093] The label includes sequence information. The sequence information refers to information that labels are sorted according to the size of the probability, such as sorting information in order from large to small probability. The multi-label classification model is trained based on the sequence information of the labels included in the text. For example, for a diagnostic report, the probability of the label may refer to the probability of matching between the label and the disease represented by the label.

[0094] The label can be understood as a feature of the text to be processed. Taking a diagnosis report as an example, the label can be, for example, information about the disease included in the diagnosis report, such as the name of the disease, etc. Taking a rectification report as an example, the label can be, for example, information about the rectification project included in the rectification report, such as the name of the project, etc.

[0095] In an embodiment of the present application, since the order information of the text labels is taken into account when training the multi-label classification model, when performing multi-label classification, labels with a certain order of the text to be processed can be accurately obtained, thereby improving the accuracy of the predicted label order.

[0096] See also Figure 2 , Figure 2 is a flowchart of the multi-label classification method provided in the embodiment of the present application. Figure 2 As shown, the following steps are included:

[0097] Step 201: Train a multi-label classification model.

[0098] See also Figure 3 , Figure 3 is a schematic diagram of a specific process of training a multi-label classification model in an embodiment of the present application, which may include:

[0099] Step 301: preprocess the training text, wherein the preprocessing includes label sorting.

[0100] Optionally, in order to improve data processing efficiency, the preprocessing may also include data format conversion and data cleaning.

[0101] Among them, data format conversion refers to converting the data format of the training text into a data format that meets the requirements. Data cleaning refers to removing noise. Label sorting refers to sorting labels using expert experience. Through this sorting, the order of labels of the training text can be directly used in the subsequent training model process.

[0102] The training text may be a text in any field, such as the medical field, the financial field, etc.

[0103] Step 302: Obtain word vectors in the preprocessed training text.

[0104] In this step, the main thing is to map the words or phrases included in the training text to real vectors. In practical applications, the word vectors can be generated using neural networks, dimensionality reduction of word co-occurrence matrices, probability models, interpretable knowledge base methods, and explicit representation of the context in which the words appear.

[0105] Step 303: Obtain a hidden state representation of each word in the training text according to the word vector.

[0106] In this step, a variety of methods can be used to obtain the hidden state representation of each word. For example, if a bidirectional LSTM (Long Short-Term Memory) is used, the word vectors of the training text can be read from two directions and the hidden state representation of each word can be calculated. The hidden state representations in the two directions are:

[0107]

[0108]

[0109] Therefore, in this step, the final hidden representation of the i-th word is obtained by utilizing the concatenated hidden states in both directions This reflects the sequence information centered on the i-th word.

[0110] Step 304: Generate a label vector for the training text, wherein the label vector includes sequence information of the labels of the training text.

[0111] In the embodiment of the present application, there are two methods to generate the label vector of the training text:

[0112] The first method is to construct a reverse mapping function by linear regression, wherein the reverse mapping function is a mapping function from a vector space to a label sorting space, and then obtain a label vector of each label of the training text by using the reverse mapping function.

[0113] In the embodiment of the present application, it is assumed that the sorted label space R and the vector space Y are known, and the goal of the vector function is to learn the mapping f: R→Y. Here, by constructing a reverse mapping function g: Y→R, the reverse mapping function is used to obtain the label vectors of each label of the training text.

[0114] Specifically, in the process of constructing the reverse mapping function, a label vector is randomly initialized for different label orders by training the vector for learning. The initialization process is implemented using a linear regression function. The trained vector and the obtained word vector are input into a training model, such as BERT (Bidirectional Encoder Representation from Transformers), for training, and the final label vector is obtained after the training is completed.

[0115] The second method is to directly construct the label vectors of each label of the training text through formulas such as trigonometric functions.

[0116] Specifically, the label vector of each label of the training text is calculated using the following formula:

[0117]

[0118] Among them, ranking(y (i,2j) ) represents the label vector, i represents the order index, j represents the dimension index, d model Represents the preset model dimension, j is an even number;

[0119]

[0120] Among them, ranking(y (i,2j+1) ) represents the label vector, i represents the order index, j represents the dimension index, d model Represents the preset model dimension, j is an odd number.

[0121] Step 305: Obtain the label probability of the training text according to the hidden state representation and the label vector.

[0122] In this step 305, the label probability of the training text is obtained using the following formula:

[0123] y t =softmax(O t +I t )

[0124] Among them, y t represents the probability of time step t in the label space L; O t represents the transformation vector, I t represents the mask vector;

[0125] O t =W o f(W d S t +V d h i ), W o , W d , V d are greater than 0, representing weight parameters, S t represents the hidden state at time step t, C t represents the context vector at time step t; f represents the nonlinear activation function;

[0126] S t =LSTM(S t-1 [rangking(y t-1 ); g(y t-1 );C t ]), S t-1 represents the hidden state at time step t-1, g(y t-1 ) is in the probability distribution y t-1 The label vector of the label with the highest probability; y t-1 is the probability distribution of time step t-1 in the label space L; [ranging(y t-1 ); g(y t-1 );C t ] represents the vector (y t-1 )、g(y t-1 ), C t connection;

[0127] in:

[0128]

[0129] In the embodiments of the present application,

[0130] Among them, H is the transformation gate that controls the scale of the weighted average vector,

[0131] W1,W2∈R L×L is the weight matrix. is the weighted average embedding at time t, which is calculated as follows:

[0132]

[0133] in, Yes t-1 The i-th element of i is the label vector of the i-th label. e represents the probability distribution y t-1 The label vector with the highest probability label.

[0134] Among them, g(y t-1 ) is to optimize the combination of the original label vector and the weighted average label vector by using the transformation gate H, so that the combination factor in each dimension can be automatically determined. t-1 Contains information about all possible labels, so by considering the probability of each label, the obtained model can reduce the loss caused by incorrect predictions in previous time steps, allowing the model to predict the label sequence more accurately.

[0135] Optional, such as Figure 4 As shown, after step 303, step 201 may further include:

[0136] Step 306: Obtain a context vector of the training text according to the hidden state representation.

[0137] In the embodiment of the present application, the attention mechanism is used to focus on different parts of the text sequence and aggregate the hidden representations of these information words to generate a context vector. The attention mechanism adds weight α to the time step t. ti Assigned to the i-th word as follows:

[0138]

[0139]

[0140] Where W a ,U a , is the weight parameter, S t is the current hidden state at time step t.

[0141] Then, the context vector C t It is expressed as: Among them, a ti Greater than 0, indicating the weight of the i-th word in the training text, h i represents the hidden state representation of the i-th word.

[0142] Then, at this time, step 305 specifically includes: obtaining the label probability of the training text according to the context vector and the label vector.

[0143] exist Figure 4 In the illustrated embodiment, there is no strict order relationship between the process of generating the label vector (including step 304) and the process of generating the context vector (including steps 302, 303 and 306). Figure 4 The order shown is only an example.

[0144] Specifically, in step 305, the label probability of the training text is obtained using the following formula:

[0145] y t =softmax(O t +I t )

[0146] Among them, y t represents the probability of time step t in the label space L; O t represents the transformation vector, I t represents the mask vector;

[0147] O t =W o f(W d S t +V d C t ), W o , W d , V d are greater than 0, representing weight parameters, S t represents the hidden state at time step t, C t represents the context vector at time step t; f represents the nonlinear activation function;

[0148] S t =LSTM(S t-1 [rangking(y t-1 ); g(y t-1 );C t ]), S t-1 represents the hidden state at time step t-1, g(y t-1 ) is in the probability distribution y t-1 The label vector of the label with the highest probability; y t-1 is the probability distribution of time step t-1 in the label space L; [ranging(y t-1 ); g(y t-1 );C t ] represents the vector (y t-1 )、g(yt-1 ), C t connection;

[0149] Among them, a ti Greater than 0, indicating the weight of the i-th word in the training text, h i represents the hidden state representation of the i-th word.

[0150] in:

[0151]

[0152] In the embodiments of the present application,

[0153] Among them, H is the transformation gate that controls the scale of the weighted average vector,

[0154] W1,W2∈R L×L is the weight matrix. is the weighted average embedding at time t, which is calculated as follows:

[0155]

[0156] in, Yes t-1 The i-th element of i is the label vector of the i-th label. e represents the probability distribution y t-1 The label vector with the highest probability label.

[0157] Among them, g(y t-1 ) is to optimize the combination of the original label vector and the weighted average label vector by using the transformation gate H, so that the combination factor in each dimension can be automatically determined. t-1 Contains information about all possible labels, so by considering the probability of each label, the obtained model can reduce the loss caused by incorrect predictions in previous time steps, allowing the model to predict the label sequence more accurately.

[0158] Step 202: Obtain the text to be processed.

[0159] The text to be processed may be any text, for example, a diagnosis report, a rectification report, a statistical report, etc.

[0160] Step 203: Use the text to be processed as an input of a multi-label classification model, run the multi-label classification model, and obtain labels included in the text to be processed.

[0161] The description of step 202 and step 203 may refer to the description of step 101 and step 102 mentioned above.

[0162] In an embodiment of the present application, since the order information of the text labels is taken into account when training the multi-label classification model, when performing multi-label classification, labels with a certain order of the text to be processed can be accurately obtained, thereby improving the accuracy of the predicted label order.

[0163] like Figure 5 As shown, it is a schematic diagram of a system for multi-label classification in an embodiment of the present application. It may include an encoder 501, a processor 502 and a decoder 503. Among them, the encoder 501 is used to map words or phrases in the training text to real vectors, and then encode the real vectors to obtain the hidden state representation of each word in the training text. The encoder can be understood as a stack of multiple units, each of which accepts a single element of the input sequence, collects the information of the element and propagates it forward. The encoder vector is the final hidden state generated from the encoder part of the model. It serves as the initial hidden state of the decoder part of the model. The processor 502 is used to construct the label vector of each label in the training text. The decoder 503 is used to decode the label vector to obtain an output sequence, which represents the probability distribution of each label.

[0164] The present application also provides a multi-label classification device. Figure 6 , Figure 6 is a structural diagram of a multi-label classification device provided by an embodiment of the present invention. Figure 6 As shown, the multi-label classification device 600 includes: a first acquisition module 601, used to obtain the text to be processed; a first processing module 602, used to use the text to be processed as the input of the multi-label classification model, run the multi-label classification model, and obtain the labels included in the text to be processed, and the labels include sequence information; wherein the multi-label classification model is trained based on the sequence information of the labels included in the text.

[0165] Optional, such as Figure 7 As shown, the device also includes: a training module 603, which is used to train the multi-label classification model.

[0166] Optional, such as Figure 8 As shown, the training module 603 includes:

[0167] A preprocessing submodule 6031 is used to preprocess the training text, wherein the preprocessing includes label sorting; a first acquisition submodule 6032 is used to acquire word vectors in the preprocessed training text; a second acquisition submodule 6033 is used to obtain a hidden state representation of each word in the training text based on the word vectors; a generation submodule 6034 is used to generate a label vector of the training text, wherein the label vector includes sequence information of the labels of the training text; and a third acquisition submodule 6035 is used to obtain a label probability rate of the training text based on the hidden state representation and the label vector.

[0168] Optional, such as Fig. 9 As shown, the training module 603 also includes:

[0169] The fourth acquisition submodule 6036 is used to obtain the context vector of the training text according to the hidden state representation; the third acquisition submodule 6035 is used to obtain the label probability of the training text according to the context vector and the label vector.

[0170] Optional, such as Fig.10 As shown, the generating submodule 6034 includes:

[0171] The first construction unit 60341 is used to construct a reverse mapping function by a linear regression method, wherein the reverse mapping function is a mapping function from a vector space to a label sorting space; the first acquisition unit 60342 is used to obtain a label vector of each label of the training text using the reverse mapping function.

[0172] The generating submodule 6034 is used to calculate the label vector of each label of the training text using the following formula:

[0173]

[0174] Among them, ranking(y (i,2j) ) represents the label vector, i represents the order index, j represents the dimension index, d model Represents the preset model dimension, j is an even number;

[0175]

[0176] Among them, ranking(y (i,2j+1) ) represents the label vector, i represents the order index, j represents the dimension index, d model Represents the preset model dimension, j is an odd number.

[0177] Wherein, the fourth acquisition submodule 6036 is used to:

[0178] The label probability of the training text is obtained using the following formula:

[0179] y t =softmax(O t +I t )

[0180] Among them, y t represents the probability of time step t in the label space L; O t represents the transformation vector, I t represents the mask vector;

[0181] O t =W o f(W d S t +V d C t ), W o , W d , V d are greater than 0, representing weight parameters, S t represents the hidden state at time step t, C t represents the context vector at time step t; f represents the nonlinear activation function;

[0182] S t =LSTM(S t-1 [rangking(y t-1 ); g(y t-1 );C t ]), S t-1 represents the hidden state at time step t-1, g(y t-1 ) is in the probability distribution y t-1 The label vector of the label with the highest probability; y t-1 is the probability distribution of time step t-1 in the label space L; [ranging(y t-1 ); g(y t-1 );C t ] represents the vector (y t-1 )、g(y t-1 ), C t connection;

[0183] Among them, a ti Greater than 0, indicating the weight of the i-th word in the training text, h i represents the hidden state representation of the i-th word.

[0184] The device provided in the embodiment of the present invention can execute the above method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not be repeated here.

[0185] See also Fig.11 The embodiment of the present invention also provides a hardware structure of a multi-label classification device. Fig.11 As shown, the multi-label classification device 1100 includes:

[0186] Processor 1102; and

[0187] The memory 1104 stores program instructions, wherein when the program instructions are executed by the processor, the processor 1102 is caused to perform the following steps:

[0188] Get the text to be processed;

[0189] Using the text to be processed as an input of a multi-label classification model, running the multi-label classification model, and obtaining labels included in the text to be processed, wherein the labels include sequence information;

[0190] The multi-label classification model is trained based on the sequential information of labels included in the text.

[0191] Furthermore, if Fig.11 As shown, the electronic device 1100 may also include a network interface 1101 , an input device 1103 , a hard disk 1105 , and a display device 1106 .

[0192] The above-mentioned interfaces and devices can be interconnected through a bus architecture. The bus architecture can be a bus and bridge including any number of interconnected buses. Specifically, various circuits of one or more central processing units (CPUs) represented by processor 1102 and one or more memories represented by memory 1104 are connected together. The bus architecture can also connect various other circuits such as peripheral devices, voltage regulators, and power management circuits together. It can be understood that the bus architecture is used to achieve connection and communication between these components. In addition to the data bus, the bus architecture also includes a power bus, a control bus, and a status signal bus, which are all well known in the art, so they will not be described in detail herein.

[0193] The network interface 1101 can be connected to a network (such as the Internet, a local area network, etc.), receive data from the network, and save the received data in the hard disk 1105.

[0194] The input device 1103 can receive various instructions input by the operator and send them to the processor 1102 for execution. The input device 1103 can include a keyboard or a pointing device (eg, a mouse, a trackball, a touch pad or a touch screen, etc.).

[0195] The display device 1106 can display the result obtained by the processor 1102 executing the instructions.

[0196] The memory 1104 is used to store programs and data necessary for the operation of the operating system, as well as data such as intermediate results during the calculation process of the processor 1102.

[0197] It is understood that the memory 1104 in the embodiment of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. The memory 1104 of the apparatus and method described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0198] In some embodiments, the memory 1104 stores the following elements, executable modules or data structures, or a subset thereof, or an extended set thereof: an operating system 11041 and an application program 11042 .

[0199] The operating system 11041 includes various system programs, such as a framework layer, a core library layer, a driver layer, etc., which are used to implement various basic services and process hardware-based tasks. The application 11042 includes various application programs, such as a browser, etc., which are used to implement various application services. The program for implementing the method of the embodiment of the present invention can be included in the application 11042.

[0200] The multi-label classification processing method disclosed in the above embodiment of the present invention can be applied to the processor 1102, or implemented by the processor 1102. The processor 1102 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above multi-label classification method can be completed by the hardware integrated logic circuit or software instructions in the processor 1102. The above-mentioned processor 1102 can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, and can implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of the present invention. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in conjunction with the embodiment of the present invention can be directly embodied as a hardware decoding processor to execute, or can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 1104, and the processor 1102 reads the information in the memory 1104 and completes the steps of the above method in combination with its hardware.

[0201] It is understood that the embodiments described herein can be implemented in hardware, software, firmware, middleware, microcode or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), general purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described in the present application or a combination thereof.

[0202] For software implementation, the techniques described herein can be implemented by modules (e.g., procedures, functions, etc.) that perform the functions described herein. The software code can be stored in a memory and executed by a processor. The memory can be implemented in the processor or outside the processor.

[0203] Specifically, when the program is executed by the processor 1102, the following steps may also be implemented:

[0204] The multi-label classification model is trained.

[0205] Specifically, when the program is executed by the processor 1102, the following steps may also be implemented:

[0206] Preprocessing the training text, wherein the preprocessing includes label sorting;

[0207] Get the word vectors in the preprocessed training text;

[0208] According to the word vector, a hidden state representation of each word in the training text is obtained;

[0209] Generate a label vector of the training text, wherein the label vector includes sequence information of the labels of the training text;

[0210] The label probability of the training text is obtained according to the hidden state representation and the label vector.

[0211] Specifically, when the program is executed by the processor 1102, the following steps may also be implemented:

[0212] Obtaining a context vector of the training text according to the hidden state representation;

[0213] The step of obtaining the label probability of the training text according to the hidden state representation and the label vector includes:

[0214] The label probability of the training text is obtained according to the context vector and the label vector.

[0215] Specifically, when the program is executed by the processor 1102, the following steps may also be implemented:

[0216] By means of linear regression, a reverse mapping function is constructed, wherein the reverse mapping function is a mapping function from a vector space to a label sorting space;

[0217] The reverse mapping function is used to obtain label vectors of various labels of the training text.

[0218] Specifically, when the program is executed by the processor 1102, the following steps may also be implemented:

[0219] The label vectors of each label of the training text are calculated using the following formula:

[0220]

[0221] Among them, ranking(y (i,2j) ) represents the label vector, i represents the order index, j represents the dimension index, d model Represents the preset model dimension, j is an even number;

[0222]

[0223] Among them, ranking(y (i,2j+1) ) represents the label vector, i represents the order index, j represents the dimension index, d model Represents the preset model dimension, j is an odd number.

[0224] Specifically, when the program is executed by the processor 1102, the following steps may also be implemented:

[0225] The label probability of the training text is obtained using the following formula:

[0226] y t =softmax(O t +I t )

[0227] Among them, y t represents the probability of time step t in the label space L; O t represents the transformation vector, I t represents the mask vector;

[0228] O t =W o f(W d S t +V d C t ), W o , W d , V d are greater than 0, representing weight parameters, S t represents the hidden state at time step t, C t represents the context vector at time step t; f represents the nonlinear activation function;

[0229] S t =LSTM(S t-1 [rangking(y t-1 ); g(y t-1 );C t ]), S t-1 represents the hidden state at time step t-1, g(y t-1 ) is in the probability distribution y t-1 The label vector of the label with the highest probability; y t-1 is the probability distribution of time step t-1 in the label space L; [ranging(y t-1 ); g(y t-1 );C t ] represents the vector (y t-1 )、g(y t-1 ), C t connection;

[0230] Among them, a ti Greater than 0, indicating the weight of the i-th word in the training text, h i represents the hidden state representation of the i-th word.

[0231] The multi-label classification device provided in the embodiment of the present invention can execute the above method embodiment, and its implementation principle and technical effect are similar, which will not be repeated in this embodiment.

[0232] The embodiment of the present invention also provides a readable storage medium, on which a program is stored. When the program is executed by the processor, each process of the above-mentioned multi-label classification method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it is not repeated here. Among them, the readable storage medium can be any available medium or data storage device that can be accessed by the processor, including but not limited to magnetic storage (such as floppy disk, hard disk, magnetic tape, magneto-optical disk (MO), etc.), optical storage (such as CD, DVD, BD, HVD, etc.), and semiconductor storage (such as ROM, EPROM, EEPROM, non-volatile memory (NAND FLASH), solid-state drive (SSD)), etc.

[0233] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0234] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, disk, CD), and includes a number of instructions for a terminal (which can be a mobile phone, computer, server, air conditioner, or network equipment, etc.) to execute the methods described in each embodiment of the present invention.

[0235] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation modes, which are merely illustrative rather than restrictive. Under the guidance of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the present invention and the claims, all of which are within the protection of the present invention.

Claims

1. A multi-label classification method, characterized in that: include: Get the text to be processed; Using the text to be processed as an input of a multi-label classification model, running the multi-label classification model, and obtaining labels included in the text to be processed, wherein the labels include sequence information; Wherein, the multi-label classification model is trained based on the sequential information of the labels included in the text; Wherein, training the multi-label classification model includes: Preprocessing the training text, wherein the preprocessing includes label sorting; Get the word vectors in the preprocessed training text; According to the word vector, a hidden state representation of each word in the training text is obtained; Obtaining a context vector of the training text according to the hidden state representation; Generate a label vector of the training text, wherein the label vector includes sequence information of the labels of the training text; Obtaining a label probability of the training text according to the context vector and the label vector; Wherein, obtaining the label probability of the training text according to the context vector and the label vector includes: The label probability of the training text is obtained using the following formula: and t =softmax(O t +I t ) Among them, y t represents the probability of time step t in the label space L; O t represents the transformation vector, I t represents the mask vector; O t =W o f(W d S t +V d C t ), W o , W d , V d are greater than 0, representing weight parameters, S t represents the hidden state at time step t, C t represents the context vector at time step t; f represents the nonlinear activation function; S t =LSTM(S t-1 [rangking(y t-1 ); g(y t-1 );C t ]), S t-1 represents the hidden state at time step t-1, g(y t-1 ) is in the probability distribution y t-1 The label vector of the label with the highest probability; y t-1 is the probability distribution of time step t-1 in the label space L; [ranging(y t-1 ); g(y t-1 );C t ] represents the vector (y t-1 )、g(y t-1 ), C t connection; Among them, a ti Greater than 0, indicating the weight of the i-th word in the training text, h i represents the hidden state representation of the i-th word.

2. The method according to claim 1, characterized in that The step of generating a label vector for the training text comprises: By means of linear regression, a reverse mapping function is constructed, wherein the reverse mapping function is a mapping function from a vector space to a label sorting space; The reverse mapping function is used to obtain label vectors of various labels of the training text.

3. The method according to claim 1, characterized in that The step of generating a label vector for the training text comprises: The label vectors of each label of the training text are calculated using the following formula: Among them, ranking(y (i，2j) ) represents the label vector, i represents the order index, j represents the dimension index, d model Represents the preset model dimension, 2j is an even number; Among them, ranking(y (i，2j+1) ) represents the label vector, i represents the order index, j represents the dimension index, d model Indicates the preset model dimension, 2j+1 is an odd number.

4. A multi-label classification device, characterized in that: include: A first acquisition module, used to acquire the text to be processed; A first processing module is used to use the text to be processed as an input of a multi-label classification model, run the multi-label classification model, and obtain labels included in the text to be processed, wherein the labels include sequence information; wherein the multi-label classification model is trained based on the sequence information of the labels included in the text; Wherein, the device further comprises: A training module, used to train the multi-label classification model, comprising: A preprocessing submodule, used for preprocessing the training text, wherein the preprocessing includes label sorting; The first acquisition submodule is used to obtain word vectors in the preprocessed training text; A second acquisition submodule is used to obtain a hidden state representation of each word in the training text according to the word vector; A fourth acquisition submodule, used to obtain a context vector of the training text according to the hidden state representation; A generating submodule, used to generate a label vector of the training text, wherein the label vector includes sequence information of the labels of the training text; A third acquisition submodule is used to obtain the label probability of the training text according to the context vector and the label vector; Wherein, the third acquisition submodule is used to: The label probability of the training text is obtained using the following formula: and t =softmax(O t +I t ) Among them, y t represents the probability of time step t in the label space L; O t represents the transformation vector, I t represents the mask vector; O t =W o f(W d S t +V d C t ), W o , W d , V d are greater than 0, representing weight parameters, S t represents the hidden state at time step t, C t represents the context vector at time step t; f represents the nonlinear activation function; S t =LSTM(S t-1 [rangking(y t-1 ); g(y t-1 );C t ]), S t-1 represents the hidden state at time step t-1, g(y t-1 ) is in the probability distribution y t-1 The label vector of the label with the highest probability; y t-1 is the probability distribution of time step t-1 in the label space L; [ranging(y t-1 ); g(y t-1 );C t ] represents the vector (y t-1 )、g(y t-1 ), C t connection; Among them, a ti Greater than 0, indicating the weight of the i-th word in the training text, h i represents the hidden state representation of the i-th word.

5. The device according to claim 4, characterized in that The generating submodule comprises: A first construction unit is used to construct a reverse mapping function by a linear regression method, wherein the reverse mapping function is a mapping function from a vector space to a label sorting space; The first acquisition unit is used to obtain the label vector of each label of the training text by using the reverse mapping function.

6. The device according to claim 4, characterized in that The generation submodule is used to calculate the label vector of each label of the training text using the following formula: Among them, ranking(y (i，2j) ) represents the label vector, i represents the order index, j represents the dimension index, d model Represents the preset model dimension, 2j is an even number; Among them, ranking(y (i,2j+1) ) represents the label vector, i represents the order index, j represents the dimension index, d model Indicates the preset model dimension, 2j+1 is an odd number.

7. A multi-label classification device, characterized in that: include: processor; as well as A memory, wherein program instructions are stored in the memory, wherein when the program instructions are executed by the processor, the processor is caused to perform the following steps: Get the text to be processed; Using the text to be processed as an input of a multi-label classification model, running the multi-label classification model, and obtaining labels included in the text to be processed, wherein the labels include sequence information; Wherein, the multi-label classification model is trained based on the sequential information of the labels included in the text; Wherein, training the multi-label classification model includes: Preprocessing the training text, wherein the preprocessing includes label sorting; Get the word vectors in the preprocessed training text; According to the word vector, a hidden state representation of each word in the training text is obtained; Obtaining a context vector of the training text according to the hidden state representation; Generate a label vector of the training text, wherein the label vector includes sequence information of the labels of the training text; Obtaining a label probability of the training text according to the context vector and the label vector; Wherein, obtaining the label probability of the training text according to the context vector and the label vector includes: The label probability of the training text is obtained using the following formula: and t =softmax(O t +I t ) Among them, y t represents the probability of time step t in the label space L; O t represents the transformation vector, I t represents the mask vector; O t =W o f(W d S t +V d C t ), W o , W d , V d are greater than 0, representing weight parameters, S t represents the hidden state at time step t, C t represents the context vector at time step t; f represents the nonlinear activation function; S t =LSTM(S t-1 [rangking(y t-1 ); g(y t-1 );C t ]), S t-1 represents the hidden state at time step t-1, g(y t-1 ) is in the probability distribution y t-1 The label vector of the label with the highest probability; y t-1 is the probability distribution of time step t-1 in the label space L; [ranging(y t-1 ); g(y t-1 );C t ] represents the vector (y t-1 )、g(y t-1 ), C t connection; Among them, a ti Greater than 0, indicating the weight of the i-th word in the training text, h i represents the hidden state representation of the i-th word.

8. A readable storage medium for storing a program, characterized in that: When the program is executed by a processor, the steps of the multi-label classification method according to any one of claims 1 to 3 are implemented.

Images