Internet text-oriented multi-label classification method and system

Abstract

The invention discloses an internet text-oriented multi-label classification method and system, and the method comprises the steps: obtaining a to-be-recognized text, and carrying out the preprocessing of the to-be-recognized text; inputting the preprocessed text, the tag set and the external knowledge into the trained multi-tag classification model to obtain a classified multi-tag result; wherein in the trained multi-label classification model, feature extraction is carried out on the preprocessed text, and text feature representation is obtained; smoothing the text feature representation to obtain a smoothed text feature representation; meanwhile, constructing a tag graph based on the original tag set and external knowledge, and extracting tag feature representation from the tag graph; the smoothed text feature representation, the smoothed tag feature representation and the smoothed original tag set are subjected to dual-granularity learning, classification features are obtained, and the classification features comprise information related to classification in a text-tag and information related to classification in a tag-tag; and performing classification by using the classification features to obtain a classification result.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a multi-label classification method and system for Internet texts. Background Art

[0002] In the era of self-media, the development of Internet technology has greatly changed people's lifestyles. Cyberspace is full of text data of various types and topics, including news reports, social media comments and other content. More and more Internet users publish and browse information through mobile network platforms such as WeChat and Weibo, resulting in an explosive growth of data on the Internet. Therefore, there is an urgent need for corresponding methods and systems to classify the massive text data on the Internet and extract valuable information from it.

[0003] Existing multi-label text classification methods can be mainly divided into the following three categories:

[0004] (1) Traditional text classification methods: including question conversion methods and algorithm adaptation methods, such as naive Bayes, decision tree, support vector machine methods, etc. These methods mainly rely on manually annotated features, which not only ignore the correlation between words, but also have low efficiency.

[0005] (2) Methods based on deep learning: especially methods based on convolutional neural networks (CNN) and recurrent neural networks (RNN), such as DCNN, TextRNN, and TextRCNN. These methods can automatically obtain classification features, but they mainly focus on the locality of words and lack the processing of long-distance and non-continuous word interactions.

[0006] (3) Methods based on graph neural networks: such as TextGCN, HyperGAT, and TextING. These methods can directly process complex structured data and prioritize mining global features, but they do not fully consider fine-grained label information and text information related to the labels.

[0007] At present, most studies focus on exploring the semantic information inside the text, while ignoring the semantic information inside the tag. Although tags are usually simple words or phrases, they contain important semantic information and can help the model learn text representation better. Tags not only have rich semantic information, but also have strong correlation with documents. In order to use tag information to improve classification performance, the deep extreme multi-label learning method (DXML) captures tag dependencies by exploring tag structures, and can efficiently handle multi-label classification problems with large-scale tags. The sequence generation model (SGM) regards the multi-label classification task as a sequence generation problem, captures the correlation between tags through an innovative decoder structure, and automatically selects the most informative words when predicting different tags. Although the above methods learn tag information from different perspectives to obtain tag relations, they ignore the role of tag co-occurrence information in capturing tag relations, so it is difficult to reasonably describe tag relations to improve model classification effects. The label embedding attention model (LEAM) obtains the embedded representation of text by combining words and tags, and the explicit interaction model (EXAM) uses class representation to obtain the interactive information between words and tags. These methods consider label information in the form of label embedding, but when the label space is vectorized, the label vector lacks the transmission of feature information between labels, resulting in the label representation failing to cover the complete semantics of the label space. Label Specific Attention Network (LSAN) uses label semantic information to determine the semantic connection between labels and documents, thereby constructing a label-specific document representation. At the same time, LSAN uses a self-attention mechanism to identify document representations of specific labels from document content information. However, this type of method assumes that all labels are independent of each other and does not fully consider label semantics and overall label correlation. Summary of the invention

[0008] In order to solve the shortcomings of the prior art, the present invention provides a multi-label classification method and system for Internet text; the method performs smoothing on text features and comprehensively considers the relative importance of each word. At the same time, it can capture the original semantics of the label and the association information between labels to generate a complete label embedding. Finally, by jointly learning the dual-granularity information of text-label and label-label, more information related to classification is captured, thereby improving the accuracy of Internet text classification.

[0009] On the one hand, a multi-label classification method for Internet text is provided, including:

[0010] Obtain the text to be recognized and preprocess the text to be recognized;

[0011] The preprocessed text, label set, and external knowledge are input into the trained multi-label classification model to obtain the classified multi-label results;

[0012] Among them, in the trained multi-label classification model, firstly, the preprocessed text is subjected to feature extraction to obtain text feature representation; secondly, the text feature representation is subjected to smoothing to obtain smoothed text feature representation; at the same time, a label graph is constructed based on the original label set and external knowledge, and the label feature representation is extracted from the label graph;

[0013] The smoothed text feature representation, label feature representation and original label set are subjected to dual-granularity learning to obtain classification features, wherein the classification features include: information related to classification in text-label and information related to classification in label-label; classification is performed using the classification features to obtain classification results.

[0014] On the other hand, a multi-label classification system for Internet text is provided, comprising:

[0015] An acquisition module is configured to: acquire the text to be recognized and pre-process the text to be recognized;

[0016] A classification module is configured to: input the preprocessed text, label set, and external knowledge into the trained multi-label classification model to obtain a classified multi-label result;

[0017] Among them, in the trained multi-label classification model, firstly, the preprocessed text is subjected to feature extraction to obtain text feature representation; secondly, the text feature representation is subjected to smoothing to obtain smoothed text feature representation; at the same time, a label graph is constructed based on the original label set and external knowledge, and the label feature representation is extracted from the label graph;

[0018] The smoothed text feature representation, label feature representation and original label set are subjected to dual-granularity learning to obtain classification features, wherein the classification features include: information related to classification in text-label and information related to classification in label-label; classification is performed using the classification features to obtain classification results.

[0019] On the other hand, there is also provided an electronic device, comprising:

[0020] a memory for non-transitory storage of computer-readable instructions; and

[0021] a processor for executing the computer readable instructions,

[0022] When the computer-readable instructions are executed by the processor, the method described in the first aspect is executed.

[0023] On the other hand, a storage medium is provided, which non-temporarily stores computer-readable instructions, wherein when the non-temporary computer-readable instructions are executed by a computer, the instructions of the method described in the first aspect are executed.

[0024] On the other hand, a computer program product is provided, comprising a computer program, wherein the computer program is used to implement the method described in the first aspect when running on one or more processors.

[0025] The above technical solution has the following advantages or beneficial effects:

[0026] The present invention provides a multi-label classification method for Internet text, which provides a text learning model with a smoothing strategy. The model extracts original text features from preprocessed text through an adaptive BERT model with 12 layers and 768 hidden dimensions in each layer, and then inputs the preprocessed text into a connection layer for splicing. The Dropout technology is applied to randomly discard some neurons to prevent the model from overfitting. After discarding the layer, the ReLU activation function and the multi-layer perceptron are used to refine the features to obtain the final text feature representation. Then, the text feature representation is smoothed, and the word probability distribution is introduced to generate a smoothed text feature representation, and the relative importance of each word is comprehensively considered.

[0027] The present invention provides a multi-label classification method for Internet text, which provides a label learning model that integrates external knowledge. The model uses the original label set to construct a label graph, and uses the label graph structure and label association matrix to transfer label semantic information and label association relationships between label nodes, thereby updating label nodes and obtaining a more enhanced label embedding representation. External knowledge is introduced to enrich label semantic information and obtain the final label feature representation.

[0028] The present invention provides a multi-label classification method for Internet text, which provides a text-label dual-granularity learning model, aggregates the smoothed text feature representation with the label feature representation, and respectively learns the classification-related information in the text-label and label-label. The text-label granularity learning part aims to discover the effective potential space between the text and label semantics, and the label-label granularity learning part is used to establish a direct mapping relationship between the label features and the real label. The model can extract more semantic information between the label and the text, and improve the classification effect of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0030] Figure 1 This is a flow chart of a multi-label classification method for Internet texts according to the first embodiment of the present invention;

[0031] Figure 2 is a text feature learning model diagram of the first embodiment of the present invention;

[0032] Figure 3 This is a label feature learning model diagram of integrated external knowledge in Example 1 of the present invention. DETAILED DESCRIPTION

[0033] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0034] Embodiment 1

[0035] This embodiment provides a multi-label classification method for Internet text;

[0036] like Figure 1 As shown, a multi-label classification method for Internet text includes:

[0037] S101: Obtain the text to be recognized and pre-process the text to be recognized;

[0038] S102: inputting the preprocessed text, label set and external knowledge into the trained multi-label classification model to obtain a classified multi-label result;

[0039] Among them, in the trained multi-label classification model, firstly, the preprocessed text is subjected to feature extraction to obtain text feature representation; secondly, the text feature representation is subjected to smoothing to obtain smoothed text feature representation; at the same time, a label graph is constructed based on the original label set and external knowledge, and the label feature representation is extracted from the label graph;

[0040] The smoothed text feature representation, label feature representation and original label set are subjected to dual-granularity learning to obtain classification features, wherein the classification features include: information related to classification in text-label and information related to classification in label-label; classification is performed using the classification features to obtain classification results.

[0041] Furthermore, the S101: obtains the text to be recognized, and preprocesses the text to be recognized, wherein the preprocessing includes: cleaning non-text data, removing stop words, removing low-frequency words, removing high-frequency words, and restoring word forms.

[0042] Furthermore, the cleaning of non-text data includes: deleting HTML tags, URL addresses, etc. Because these contents are of no help to classification and may even have a negative impact.

[0043] Furthermore, the removal of stop words includes: removing auxiliary verbs, function words, punctuation marks, etc., which exist in large quantities in all articles but cannot reflect the meaning of the text. Generally, removing these stop words in the text can make the model better fit the semantic features, thereby improving the generalization ability of the model.

[0044] Furthermore, the removal of low-frequency words and high-frequency words includes: the importance of a word is directly proportional to the number of times it appears in the text and inversely proportional to the frequency of its appearance in the corpus. Therefore, high-frequency words and low-frequency words in the corpus are of no help for classification and can be removed (this step is not required in short texts).

[0045] Furthermore, the word form restoration includes: words like is, are, and am are completely different in content to computers, but they have the same meaning. Word form restoration unifies them to facilitate subsequent processing and analysis. Word form restoration refers to unifying letters into lowercase letters.

[0046] Furthermore, the label set refers to: the label set part in the data set, including the set of all possible labels in the corresponding multi-label classification task. Each label represents a category, and the text can belong to one or more of these categories.

[0047] Furthermore, the external knowledge refers to: given an input tag, using Wikipedia-API to retrieve the most relevant Wikipedia documents, and then extracting the first five sentences describing the tag as the external knowledge of the corresponding tag.

[0048] Furthermore, the S102: inputting the preprocessed text, label set and external knowledge into the trained multi-label classification model to obtain a classified multi-label result, wherein the trained multi-label classification model includes:

[0049] Text learning module, label learning module, dual-granularity learning module and classifier;

[0050] The text learning module performs feature extraction on the preprocessed text to obtain a text feature representation; performs smoothing on the text feature representation to obtain a smoothed text feature representation;

[0051] The label learning module constructs a label graph based on the label set and external knowledge, and extracts label feature representation from the label graph;

[0052] The dual-granularity learning module performs dual-granularity learning on the smoothed text feature representation and the label feature representation to obtain classification features;

[0053] The classifier performs classification using the classification features to obtain a classification result.

[0054] Furthermore, if Figure 2 As shown, the text learning module includes: a text feature extractor and a smoothing processing unit connected in sequence; the text feature extractor includes:

[0055] The BERT model, connection layer, dropout layer, activation function layer and multi-layer perceptron connected in sequence.

[0056] The BERT model converts text into high-dimensional feature representations. Specifically, each word or subword of the input is converted into a fixed-length vector through an embedding layer and positional encoding, which is processed by a multi-layer Transformer encoder to generate rich contextual features.

[0057] The connection layer is used to concatenate the features output by the BERT model.

[0058] The discard layer is a Dropout layer, which refers to a regularization technique used to prevent overfitting of a neural network. The basic idea of ​​the Dropout layer is to randomly discard (i.e. set the value to zero) a portion of neurons with a certain probability during the training process, thereby reducing the mutual dependence between neurons and improving the generalization ability of the model.

[0059] Furthermore, the text feature extractor defines the length of the text sequence x as m, where x = {t1, t2, ... t m}, t i Represents the i-th word in the text sequence. For each word t in the text sequence i Encode and get the corresponding code It is expressed as:

[0060]

[0061] Encode all words After splicing, we get the features:

[0062]

[0063] The concatenated feature f is input into the discard layer D, and then refined by the ReLU activation function σ and the multi-layer perceptron (MLP) to obtain the final feature E:

[0064] E=W e σ(D(f))+b e (3)

[0065] Among them, W e and b e represents the parameters of MLP1, E={e1,e2,…e m}∈Rm*k .

[0066] It should be understood that BERT is a pre-trained language model based on the Transformer architecture. Its core feature is to capture contextual information through bidirectional encoder representation. The following is the principle and working process of BERT:

[0067] (11) Pre-training and fine-tuning: Pre-training stage: Pre-training is performed on a large amount of unlabeled text data. The main tasks include: Masked Language Model (MLM): randomly masking some words in the input text, and then predicting the original words of these masks through the model; Next Sentence Prediction (NSP): determining whether two given sentences appear consecutively.

[0068] Fine-tuning stage: Fine-tune the model parameters on a task-specific dataset to adapt to specific tasks, such as text classification, question-answering systems, etc.

[0069] (12) Transformer architecture: BERT is based on the encoder part of Transformer, which consists of multiple stacked self-attention layers and feed-forward neural network layers.

[0070] Self-attention mechanism: It can focus on the correlation of different positions in the input sequence, thereby capturing the global information of the text.

[0071] Positional encoding: Because the Transformer architecture does not have built-in sequential information, positional encoding is used to add positional information to the input, allowing the model to perceive the order of the sequence.

[0072] (13) Input representation: The input consists of three parts: word embedding, position embedding, and segment embedding, which represent the information of vocabulary, position, and sentence segment respectively. The input format starts with the [CLS] tag, and sentences are separated by the [SEP] tag.

[0073] (14) Bidirectional contextual representation: BERT processes text from left to right and from right to left simultaneously, thereby capturing bidirectional contextual information.

[0074] Furthermore, the text feature extractor is used to: extract the vector of the [CLS] tag from the preprocessed text through the BERT model as the text feature representation, input it into the connection layer for splicing, apply the Dropout layer as the discard layer, and randomly discard some neurons to prevent the model from overfitting. After the discard layer, the ReLU activation function and the multi-layer perceptron are used to refine the features to obtain the final text feature representation. The [CLS] tag is a tag added at the front of the input sequence. When processing the input text, the BERT model generates a vector representation for each tag (including the [CLS] tag). After passing through the model's multi-layer Transformer encoder, finally, the vector of the [CLS] tag is used as the feature representation of the entire input text.

[0075] Exemplarily, the text feature extractor is used to: for a text to be recognized or a text in a training set, convert it into a word fragment ID, an attention mask and a token type ID through the word segmenter of the BERT model, and then forward propagate through the BERT model to extract the vector of the [CLS] tag; then input the vector of the [CLS] tag into the connection layer for splicing, and apply the Dropout technology to randomly discard some neurons to prevent the model from overfitting. After the discard layer, the ReLU activation function and the multi-layer perceptron are used to refine the features to obtain the final text feature representation.

[0076] Furthermore, the smoothing processing unit includes: introducing word probability distribution to generate smooth text features, and comprehensively considering the relative importance of each word.

[0077] Furthermore, the smoothing processing unit includes:

[0078] Step (21): Smooth the text feature E to obtain a smoothed text feature. The specific operation is to input the above text feature E and word embedding matrix W into the smoothing processing module:

[0079] MLM(E) = softmax(EW T ) (4)

[0080] Among them, MLM(E) is a smooth representation, EW T Each row of represents the similarity between each word in the vocabulary and each word feature in the text feature E.

[0081] Through the softmax function, these similarities are converted into probability distributions, where each row represents the probability distribution of a word over the vocabulary, indicating a vocabulary choice that is compatible with the context.

[0082] Step (22): Finally, interpolation operation is performed to obtain the final smooth feature G = {g1, g2, ..., g m}for:

[0083] g i =λE i +(1-λ)MLM(E) i (5)

[0084] Among them, E i is the feature representation of the word, MLM(E) i is the smoothed representation, and λ is a balancing hyperparameter that controls the strength of the interpolation.

[0085] Furthermore, the label learning module constructs a label graph based on the training set (each text in the training set has a corresponding label set, that is, the set of categories to which the text belongs), and uses the label graph structure and label association matrix to transfer label semantic information and label association relationships between label nodes, thereby updating label nodes and obtaining a more enhanced label embedding representation. External knowledge is introduced to initialize node features to enhance the word embedding of labels, enrich label semantic information, and obtain the final label feature representation. That is, the category (political, military, economic, etc.).

[0086] Furthermore, if Figure 3 As shown, the label learning module includes:

[0087] Step (31): Construct a graph structure G = (V, E), where V represents the vertex set of the graph and E represents the edge set of the graph; G is composed of the association matrix A∈R c×c (c=|V|) indicates;

[0088] Step (32): Use the occurrence probability and co-occurrence probability of the tags to construct a tag association matrix:

[0089] Each element A of the label association matrix ij The definition is as follows:

[0090]

[0091] Among them, p(i) represents the number of times label i appears, p(j) represents the number of times label j appears, and p(i,j) represents the number of times labels i and j appear at the same time; (i,j) Indicates the number of documents with both labels i and j, #L (i) and #L (j) denotes the number of documents with label i and label j respectively, and #D denotes the total number of documents in the dataset. ij The larger the value, the higher the relevance of its vertices;

[0092] Step (33): After each node aggregates the features of all its neighbors and its own features, the final label association matrix calculation formula is as follows:

[0093]

[0094] Step (34): At the same time, use external knowledge to initialize the feature nodes of the input label and enrich the word embedding. Given an input label p i , first use Wikipedia-API to retrieve the most relevant Wikipedia document B(p i ). Then, extract the first five sentences {s1,s2,s3,s4,s5} describing the tag in the retrieved document. Finally, feed the first five sentences {s1,s2,s3,s4,s5} to Sentence-BERT to generate the embedding vector {s1′,s2′,s3′,s4′,s5′} of each sentence. The embedding vectors of the first five sentences {s1,s2,s3,s4,s5} are averaged as the node feature representation:

[0095]

[0096] Among them, X i is the averaged embedding vector, and is the label p i Node features of are used to initialize input features.

[0097] It should be understood that Wikipedia-API is a Python library for accessing and obtaining Wikipedia content programmatically. The working mechanism is to obtain Wikipedia content through HTTP requests, which return responses in JSON format, which can help retrieve and obtain information on specific topics or pages.

[0098] The results returned by the Wikipedia-API search contain multiple documents. The document that comes first (i.e. the first document) is the most relevant document. This relies on its working mechanism to sort documents according to the internal relevance scoring mechanism.

[0099] Wikipedia documents are open and widely used information resources. They refer to specific pages or entries created and maintained on Wikipedia. Each document typically contains detailed information and content about a specific topic.

[0100] Sentence-BERT is a model based on BERT, which is specially designed to handle sentence-level semantic representation tasks and is used to generate more semantically informative label embedding representations.

[0101] Furthermore, constructing a label graph based on the training set includes: for all labels in the training set, counting the number of co-occurrences of labels in a sliding window with a set window size and a set window step, and constructing a label graph with each label as a vertex and the number of label co-occurrences as an edge.

[0102] In this embodiment, the sliding window reads a fixed-length area from an article from left to right each time. The fixed length is the size of the sliding window, and the distance moved each time is the step length of the sliding window.

[0103] In this embodiment, the tag co-occurrence information is the number of times the tags co-occur in a sliding window of a set size.

[0104] Furthermore, the dual-granularity learning module includes:

[0105] First, the smoothed text feature representation and the label feature representation are learned to learn the association information between them, and the most representative label features are learned from the text features, thereby discovering the effective latent space between the text and label semantics. Specifically, the smoothed text features and label features pass through the fully connected layer. However, before the connection, the smoothed text features need to pass through the shape layer to match the shape of the label features.

[0106] Secondly, the label feature representation and the original label set are learned, and a direct mapping relationship is established between the label feature and the true label to connect the label and the label feature. Specifically, the label feature is aggregated with the original label set through a fully connected layer to learn its associated information.

[0107] Finally, the text-label aggregation information and label-label association information are aggregated to obtain the final complete classification features.

[0108] The dual-granularity learning module aggregates the smoothed text feature representation with the label feature representation, and learns the classification-related information in text-label and label-label respectively. The text-label granularity learning part aims to discover the effective latent space between text and label semantics, and the label-label granularity learning part is used to establish a direct mapping relationship between label features and true labels. This model can extract more semantic information between labels and texts, and improve the classification effect of the model.

[0109] Furthermore, the dual-granularity learning module includes:

[0110] Step (41): Connect the smoothed text features and label features through the first fully connected (FC) layer. Before connecting, the smoothed text features pass through the shape layer to match the shape of the label features. This part aims to discover the effective latent space between text and label semantics. Connect the label features and the label set through the second fully connected (FC) layer to establish a direct mapping relationship between the label features and the true labels, which helps to relieve the pressure of the subsequent classifier.

[0111] It should be understood that the shape layer is used to adjust the shape of the input data to make it consistent with the shape of subsequent layers or other tensors. In this patent, the smooth text feature needs to adjust its shape through the shape layer to match the shape of the label feature before it can be further processed through the first fully connected (FC) layer.

[0112] Step (42): Use E text and E label To represent smooth text features and label features, two loss functions L are used feature and L label , they play a solid bridge role in the text-label dual-granularity learning module. feature Mixed text space and label space, L label Connect the true label and label feature. The two loss functions are calculated as follows:

[0113]

[0114] Equation (10) represents the smooth text feature E text and label feature E label Calculate L feature , L feature is the mean square error loss (MSE). Equation (11) represents the Calculate L label , L label is the binary cross entropy loss (BCE). Among them, It is not generated by text features, but by label features.

[0115] The total loss function L guide For L feature and L label The sum of is described as follows:

[0116] L guide =L feature +L label (12)

[0117] By minimizing L guide Processing, the feature extractor and classifier are decoupled from the text-label dual-granularity learning module, which means that there is no dependency on label information during the testing phase.

[0118] During training, the text-label dual-granularity learning module guides the feature extractor and classifier, allowing them to independently find the path from text to the correct label. The key to this framework is to allow the model to learn to extract features and classify independently during training, without relying on specific label information.

[0119] Furthermore, the classifier includes: outputting the final classification result through a sigmoid layer.

[0120] The classification features are input into a multi-layer perceptron (MLP), and the obtained text representation is processed using the sigmoid function to obtain the final classification result.

[0121] In this embodiment, the multilayer perceptron (MLP) is a feed-forward neural network comprising an input layer, one or more hidden layers, and an output layer.

[0122] Furthermore, the S102: inputting the preprocessed text, label set and external knowledge into the trained multi-label classification model to obtain a classified multi-label result, wherein the trained multi-label classification model includes:

[0123] Constructing a training set, wherein the training set is preprocessed text, a label set, and external knowledge of known classification multi-label results;

[0124] The training set is input into the multi-label classification model to train the model. When the total loss function value of the model no longer decreases, the training is stopped to obtain the trained multi-label classification model.

[0125] In this embodiment, the tag association relationship refers to a relationship of mutual dependence or mutual exclusion between tags.

[0126] The present invention provides a multi-label classification method and system for Internet text, including: obtaining a text to be identified and preprocessing it; extracting text feature representation from the preprocessed text through a feature extractor; smoothing the obtained text feature representation to obtain a smoothed text feature representation; constructing a label graph based on a label set and external knowledge to extract label feature representation; performing dual-granularity learning based on the smoothed text feature representation and label feature representation to learn information related to classification in text-label and label-label to obtain classification features; based on the classification features, using a classifier to classify the text to be identified, matching corresponding labels, and obtaining classification results. The method for obtaining the text feature representation is as follows: extracting a vector of [CLS] tags from the preprocessed text through BERT as a text feature representation, inputting it into a connection layer for splicing, applying the Dropout technology, and randomly discarding some neurons to prevent the model from overfitting. After the discard layer, the ReLU activation function and the multi-layer perceptron are used to refine the features to obtain the final text feature representation. The method for obtaining label feature representation is as follows: construct a label graph based on the label set, use the label graph structure and label association matrix to transfer label semantic information and label association relationships between label nodes, thereby updating label nodes and obtaining a more enhanced label embedding representation. External knowledge is introduced to initialize node features to enhance the word embedding of labels, enrich label semantic information, and obtain the final label feature representation.

[0127] Embodiment 2

[0128] This embodiment provides a multi-label classification system for Internet text, including:

[0129] An acquisition module is configured to: acquire the text to be recognized and pre-process the text to be recognized;

[0130] A classification module is configured to: input the preprocessed text, label set, and external knowledge into the trained multi-label classification model to obtain a classified multi-label result;

[0131] Among them, in the trained multi-label classification model, firstly, the preprocessed text is subjected to feature extraction to obtain text feature representation; secondly, the text feature representation is subjected to smoothing to obtain smoothed text feature representation; at the same time, a label graph is constructed based on the original label set and external knowledge, and the label feature representation is extracted from the label graph;

[0132] The smoothed text feature representation, label feature representation and original label set are subjected to dual-granularity learning to obtain classification features, wherein the classification features include: information related to classification in text-label and information related to classification in label-label; classification is performed using the classification features to obtain classification results.

[0133] It should be noted that the acquisition module and the classification module correspond to steps S101 to S102 in Embodiment 1, and the examples and application scenarios implemented by the modules and the corresponding steps are the same, but are not limited to the contents disclosed in Embodiment 1. It should be noted that the modules as part of the system can be executed in a computer system such as a set of computer executable instructions.

[0134] The description of each embodiment in the above embodiments has different emphases. For parts not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0135] The proposed system can be implemented in other ways. For example, the system embodiment described above is only illustrative, and the division of the modules is only a logical function division. In actual implementation, there may be other division methods, such as multiple modules can be combined or integrated into another system, or some features can be ignored or not executed.

[0136] Embodiment 3

[0137] This embodiment also provides an electronic device, including: one or more processors, one or more memories, and one or more computer programs; wherein the processor is connected to the memory, and the one or more computer programs are stored in the memory. When the electronic device is running, the processor executes the one or more computer programs stored in the memory so that the electronic device executes the method described in the above embodiment one.

[0138] It should be understood that in this embodiment, the processor may be a central processing unit CPU, and the processor may also be other general-purpose processors, digital signal processors DSP, application-specific integrated circuits ASIC, off-the-shelf programmable gate arrays FPGA or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0139] The memory may include a read-only memory and a random access memory, and provide instructions and data to the processor. A portion of the memory may also include a non-volatile random access memory. For example, the memory may also store information about the device type.

[0140] In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in a processor or an instruction in the form of software.

[0141] The method in the first embodiment can be directly embodied as a hardware processor, or a combination of hardware and software modules in the processor. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware. To avoid repetition, it will not be described in detail here.

[0142] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with this embodiment can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0143] Embodiment 4

[0144] This embodiment further provides a computer-readable storage medium for storing computer instructions. When the computer instructions are executed by a processor, the method described in the first embodiment is completed.

[0145] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A multi-label classification method for Internet text, characterized in that: include: Obtain the text to be recognized and preprocess the text to be recognized; The preprocessed text, label set, and external knowledge are input into the trained multi-label classification model to obtain the classified multi-label results; Among them, in the trained multi-label classification model, firstly, the preprocessed text is subjected to feature extraction to obtain text feature representation; secondly, the text feature representation is subjected to smoothing to obtain smoothed text feature representation; at the same time, a label graph is constructed based on the original label set and external knowledge, and the label feature representation is extracted from the label graph; The smoothed text feature representation, label feature representation and original label set are subjected to dual-granularity learning to obtain classification features, wherein the classification features include: information related to classification in text-label and information related to classification in label-label; classification is performed using the classification features to obtain classification results.

2. A multi-label classification method for Internet text as claimed in claim 1, characterized in that: The preprocessed text, label set and external knowledge are input into the trained multi-label classification model to obtain a classified multi-label result. The trained multi-label classification model includes: Text learning module, label learning module, dual-granularity learning module and classifier; The text learning module performs feature extraction on the preprocessed text to obtain a text feature representation; performs smoothing on the text feature representation to obtain a smoothed text feature representation; The label learning module constructs a label graph based on the label set and external knowledge, and extracts label feature representation from the label graph; The dual-granularity learning module performs dual-granularity learning on the smoothed text feature representation and the label feature representation to obtain classification features; The classifier performs classification using the classification features to obtain a classification result.

3. A multi-label classification method for Internet text as claimed in claim 2, characterized in that: The text learning module comprises: a text feature extractor and a smoothing processing unit connected in sequence; the text feature extractor comprises: a BERT model, a connection layer, a discard layer, an activation function layer and a multi-layer perceptron connected in sequence; The text feature extractor defines the length of the text sequence x as m, where x = {t1, t2, ... t m }, t i Represents the i-th word in the text sequence; for each word t in the text sequence i Encode and get the corresponding code It is expressed as: Encode all words Perform splicing to obtain the features: The concatenated feature f is input into the discard layer D, and then refined by the ReLU activation function σ and the multi-layer perceptron to obtain the final feature E: E=W e σ(D(f))+b e (3) Among them, W e and b e represents the parameters of MLP1, E={e1,e2,…e m }∈R m*k .

4. A multi-label classification method for Internet text as claimed in claim 3, characterized in that: The smoothing processing unit comprises: Step (21): Smoothing the text feature E to obtain a smoothed text feature. The specific operation is to input the text feature E and the word embedding matrix W into the smoothing processing module: MLM(E)=softmax(EW T ) (4) Among them, MLM(E) is a smooth representation, EW T Each row of represents the similarity between each word in the vocabulary and each word feature in the text feature E; Through the softmax function, these similarities are converted into probability distributions, where each row represents the probability distribution of a word in the vocabulary, indicating a vocabulary choice that is compatible with the context; Step (22): Finally, interpolation operation is performed to obtain the final smooth feature G = {g1, g2, ..., g m }for: g i =λE i +(1-λ)MLM(E) i (5) Among them, E i is the feature representation of the word, MLM(E) i is the smoothed representation, and λ is a balancing hyperparameter that controls the strength of the interpolation.

5. A multi-label classification method for Internet text as claimed in claim 2, characterized in that: The label learning module includes: Step (31): Construct a graph structure G = (V, E), where V represents the vertex set of the graph and E represents the edge set of the graph; G is composed of the association matrix A∈R c×c (c=|V) indicates; Step (32): Use the occurrence probability and co-occurrence probability of the tags to construct a tag association matrix: Each element A of the label association matrix ij The definition is as follows: Among them, p(i) represents the number of times label i appears, p(j) represents the number of times label j appears, and p(i,j) represents the number of times labels i and j appear at the same time; (i,j) Indicates the number of documents with both labels i and j, #L (i) and #L (j) denotes the number of documents with label i and label j respectively, and #D denotes the total number of documents in the dataset; therefore, when the A of an edge ij The larger the value, the higher the relevance of its vertices; Step (33): After each node aggregates the features of all its neighbors and its own features, the final label association matrix calculation formula is as follows: Step (34): At the same time, use external knowledge to initialize the feature nodes of the input label and enrich the word embedding. Given an input label p i , first use Wikipedia-API to retrieve the most relevant Wikipedia document B(p i ); Then, extract the first five sentences {s1, s2, s3, s4, s5} describing the tag in the retrieved document; Finally, feed the first five sentences {s1, s2, s3, s4, s5} to Sentence-BERT to generate the embedding vector {s1′, s2′, s3′, s4′, s5′} of each sentence; The embedding vectors of the first five sentences {s1, s2, s3, s4, s5} are averaged as the node feature representation: Among them, X i is the averaged embedding vector, and is the label p i Node features of are used to initialize input features.

6. A multi-label classification method for Internet text as claimed in claim 2, characterized in that: The dual-granularity learning module includes: First, the correlation information between the smoothed text feature representation and the label feature representation is learned, and the most representative label features are learned from the text features, thereby discovering the effective latent space between the text and label semantics. Secondly, the label feature representation and the original label set are learned, and a direct mapping relationship is established between the label features and the true labels to connect the labels and label features; Finally, the text-label aggregation information and label-label association information are aggregated to obtain the final complete classification features.

7. A multi-label classification method for Internet text as claimed in claim 2, characterized in that: The dual-granularity learning module includes: The smoothed text features and label features are connected through the first fully connected layer; before the connection, the smoothed text features pass through the shape layer to match the shape of the label features; the label features and the label set are connected through the second fully connected layer to establish a direct mapping relationship between the label features and the true labels; Use E text and E label To represent smooth text features and label features, two loss functions L are used feature and L label , they play a solid bridge role in the text-label dual-granularity learning module; L feature Mixed text space and label space, L label Connect the true label and the label feature; The two loss functions are calculated as follows: Equation (10) represents the smooth text feature E text and label feature E label Calculate L feature , L feature is the mean square error loss; Equation (11) represents the loss based on the true label y and the predicted label Calculate L label , L label is the binary cross entropy loss; where It is not generated by text features, but by label features; The total loss function L guide For L feature and L label The sum of is described as follows: L guide =L feature +L label (12) 8. A multi-label classification system for Internet text, characterized by: include: An acquisition module is configured to: acquire the text to be recognized and pre-process the text to be recognized; A classification module is configured to: input the preprocessed text, label set, and external knowledge into the trained multi-label classification model to obtain a classified multi-label result; Among them, in the trained multi-label classification model, firstly, the preprocessed text is subjected to feature extraction to obtain text feature representation; secondly, the text feature representation is subjected to smoothing to obtain smoothed text feature representation; at the same time, a label graph is constructed based on the original label set and external knowledge, and the label feature representation is extracted from the label graph; The smoothed text feature representation, label feature representation and original label set are subjected to dual-granularity learning to obtain classification features, wherein the classification features include: information related to classification in text-label and information related to classification in label-label; classification is performed using the classification features to obtain classification results.

9. An electronic device, comprising: a memory for non-transitory storage of computer readable instructions; as well as a processor for executing the computer readable instructions, When the computer-readable instructions are executed by the processor, the method described in any one of claims 1 to 7 is executed.

10. A storage medium that non-transitorily stores computer-readable instructions, wherein: When the non-transitory computer-readable instructions are executed by a computer, the instructions of the method according to any one of claims 1 to 7 are executed.

Images