Fine-grained classification method and system for social media topic comments based on self-attention

Abstract

The application discloses a kind of based on self-attention's social media theme comment's fine-grained classification method, comprising: step a, social media theme comment data is preprocessed and fine-grained classification annotation, obtain text corpus dataset, vectorization obtains vectorized text corpus dataset;Step b, construct classification model based on multi-head input structure layer, LSTM layer, self-attention layer, regularization layer, classification output layer, after training, obtain the classification model after training;Step c, the social media theme comment data to be classified is preprocessed, vectorized, and then input into the classification model after training, and the fine-grained classification of the social media theme comment data is output to obtain.The application has the beneficial effect of fine-grained classification of social media theme comments for sentiment orientation.Provide a kind of based on self-attention's social media theme comment's fine-grained classification system, with the beneficial effect of less resource occupation, more easily deployed in local.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of fine-grained classification of social media topic comments. More specifically, the present application relates to a self-attention-based fine-grained classification method and system for social media topic comments. BACKGROUND

[0002] More and more offline activities are held online, the number of netizens in China has reached 1 billion, and social media users are more active. The amount of Internet information has also grown rapidly, resulting in a large amount of user subjective text data. These data usually imply the subjective emotions of users. By analyzing the sentiment tendency in these text data, we can understand the user's preferences, values, and other information about a topic or thing, which is of great significance to enterprises, institutions, or academic research groups.

[0003] The current public opinion analysis algorithm only supports coarse-grained classification of user sentiment tendency. If further analysis of the user's sentiment tendency towards a certain target attribute is needed, manual assistance is still required. As an important research direction of natural language processing, public opinion analysis, also known as sentiment analysis, is an important application of artificial intelligence technology in public opinion analysis. By further classifying the sentiment tendency of users, we can further determine the user's preferences, values, values, political views, and other implicit information, providing important data support for network public opinion analysis in the big data era.

[0004] To obtain a multi-label classification (fine-grained classification) of the sentiment tendency of the target text corpus, the following difficulties need to be addressed: 1. Establish a multi-label classification method; 2. Vectorize the target text corpus for model use; 3. Build an algorithm model that meets the multi-label sentiment classification; 4. Adjust the model to improve the algorithm's discriminant accuracy in the case of asymmetric data volume of each label. SUMMARY

[0005] An object of the present application is to solve at least the above problems and to provide at least the advantages to be described later.

[0006] To achieve these objects and other advantages in accordance with the present application, a self-attention-based fine-grained classification method for social media topic comments is provided, comprising the following steps:

[0007] Step a, pre-processing and fine-grained classification labeling of original social media topic comment data, obtaining text corpus data set, and vectorizing to obtain corresponding vectorized text corpus data set;

[0008] Step b, a classification model based on a multi-head input structure layer, an LSTM layer, a self-attention layer, a regularization layer, and a classification output layer is constructed, and then the vectorized text corpus data and the corresponding classification labels are input into the classification model for training, and the predicted classification results are output, and the error between the predicted classification results and the corresponding classification labels is calculated through a cross-entropy loss function, and then the error is back propagated through an AdamWarmup algorithm, and after multiple training, a trained classification model is obtained;

[0009] Step c, the social media topic review data to be classified is preprocessed according to step a and vectorized, and then the vectorized data is input into the trained classification model in step b, and the predicted classification results of the social media topic review data are output, that is, the fine-grained classification of the social media topic review data.

[0010] Preferably, the preprocessing of step a includes data cleaning and tokenization, the data cleaning is to delete URLs, user mentions, and expression characters, and the tokenization is to remove special characters and stop words.

[0011] Preferably, the skip-gram model is used for vectorization in step a.

[0012] Preferably, in step b, the vectorized text corpus data is copied multiple times and input into the multi-head input structure layer as multiple input quantities at the same time, the multi-head input structure layer encodes the positions of the input quantities, and then the position encoding is spliced and fused into each input quantity and input into the LSTM layer.

[0013] Preferably, the splicing and fusion formula of the position encoding is shown in formula (1):

[0014] (1)

[0015] wherein, represents the input vector of the LSTM layer; represents the vectorized text corpus data; represents the position vector of the model input quantity, which is obtained by position calculation formula; represents a fusion operation; the position encoding formula is shown in formula (2):

[0016] formula (2)

[0017] wherein, PE (2 pos , 2 i ) represents the position encoding of the 2 i dimension, PE (2 pos , 2 i+1) represents the 2 i +1-dimensional position encoding, pos represents the position of each subword in the sentence in the sentence, i represents a certain dimension of the input quantity, dmodel represents the dimension of the input quantity;

[0018] According to the trigonometric function and difference formula, as shown in formula (3):

[0019] Formula (3)

[0020] In the formula, a , b respectively represent two different variables;

[0021] The above position encoding formula is transformed into formula (4) as shown:

[0022] Formula (4).

[0023] Preferably, the LSTM layer extracts features of each input quantity to obtain a feature extraction vector data of each input quantity.

[0024] Preferably, the feature extraction vector data processed by the LSTM layer is taken as an input quantity and input to the self-attention layer, and the given feature extraction vector data is assigned as X, and three matrices , , are randomly initialized, respectively multiplied by X and the above three matrices to obtain three variables Q, K, V , and the attention score is calculated based on formula (5);

[0025] Equation (5)

[0026] wherein, Q represents the vector of the word to be Query in the sentence, K represents the vector of all words in the same sentence, V represents the evaluation or weight of the relationship between Q and K , d represents a constant, which is the square root of the first dimension of the matrix or is set to other values, i represents the position of the currently calculated word in the vector;

[0027] The attention score is converted by using the softmax function, and the normalized similarity of the two words is calculated under the given Q and K , and then compared with VThe multiplication result is an attention calculation result.

[0028] Preferably, the regularization layer is used to perform 'layer regularization' processing on the feature vector output by the self-attention layer.

[0029] The classification output layer is used to classify the attention calculation result and output the classification result, which includes the text corpus data and the corresponding fine-grained classification label.

[0030] Preferably, one LSTM layer and one self-attention layer form one feature extraction layer, and the classification model includes multiple feature extraction layers.

[0031] A fine-grained classification system for social media topic comments based on self-attention is provided, which is used to perform the above classification method.

[0032] The present application at least includes the following beneficial effects: the present application overcomes the defects that the amount of single comment text data is small and the information is difficult to accurately capture through the multi-head input structure, and performs multi-layer synchronous input. Secondly, the LSTM layer respectively performs parallel feature extraction on multiple (three) input quantities without interference, which can more comprehensively deepen the memory of important words in the sentence (comment text data) and weaken the memory of unimportant words. Feature extraction is performed according to this principle. Finally, the self-attention layer can effectively capture the dependency relationship and dependency degree between different features in the same group of information, and for the context self-dependent task scenario of comment text corpus sentiment analysis, it can more accurately identify the sentiment tendency and improve the classification precision and accuracy.

[0033] Other advantages, objects and features of the present application will be apparent from the following description, and will be understood by those skilled in the art through the study and practice of the present application. BRIEF DESCRIPTION OF DRAWINGS

[0034] Figure 1 The technical framework diagram of the classification method of one of the technical solutions of the present application;

[0035] Figure 2 The structure diagram of one layer of the classification model of one of the technical solutions of the present application;

[0036] Figure 3 The accuracy result diagram of the MH3-SA-LSTM (1 layer) model of the present application;

[0037] Figure 4 The loss rate result diagram of the MH3-SA-LSTM (1 layer) model of the present application;

[0038] Figure 5 The accuracy result diagram of the Transformer-1L (1 layer) model of the present application;

[0039] Figure 6 This is a schematic diagram of the loss rate results of the Transformer-1L (1-layer) model of the present invention;

[0040] Figure 7 This is a schematic diagram illustrating the accuracy results of the SA-LSTM (1-layer) model of the present invention;

[0041] Figure 8 This is a schematic diagram of the loss rate results of the SA-LSTM (1-layer) model of the present invention;

[0042] Figure 9 This is a schematic diagram showing the accuracy results of the LSTM (1-layer) model of the present invention;

[0043] Figure 10 This is a schematic diagram of the loss rate results of the LSTM (1-layer) model of the present invention;

[0044] Figure 11 This is a schematic diagram illustrating the accuracy results of the MH3-SA-LSTM (2-layer) model of the present invention;

[0045] Figure 12 This is a schematic diagram of the loss rate results of the MH3-SA-LSTM (2-layer) model of the present invention;

[0046] Figure 13 This is a schematic diagram illustrating the accuracy results of the Transformer-1L (2-layer) model of the present invention;

[0047] Figure 14 This is a schematic diagram of the loss rate results of the Transformer-1L (2-layer) model of the present invention;

[0048] Figure 15 This is a schematic diagram illustrating the accuracy results of the SA-LSTM (2-layer) model of the present invention;

[0049] Figure 16 This is a schematic diagram of the loss rate results of the SA-LSTM (2-layer) model of the present invention;

[0050] Figure 17 This is a schematic diagram illustrating the accuracy results of the LSTM (2-layer) model of the present invention;

[0051] Figure 18 This is a schematic diagram of the loss rate results of the LSTM (2-layer) model of the present invention. Detailed Implementation

[0052] The present invention will now be described in further detail with reference to the accompanying drawings, so that those skilled in the art can implement it based on the description.

[0053] It should be noted that the experimental methods in the following embodiments are conventional methods, and the reagents and materials are commercially available unless otherwise specified. In the description of the present application, the terms indicating the orientation or positional relationship are based on the orientation or positional relationship shown in the drawings, which is only for the convenience of describing the present application and simplifying the description, and does not indicate or imply that the indicated device or element must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation of the present application.

[0054] As shown in Figures 1-18 The present application provides a fine-grained classification method for social media topic comments based on self-attention, comprising the following steps:

[0055] Step a, preprocessing and fine-grained classification labeling of original social media topic comment data to obtain text corpus data set, and vectorizing to obtain corresponding vectorized text corpus data set; each social media topic comment data is generated as a text corpus data, and a large amount of text corpus data forms a text corpus data set.

[0056] Step b, constructing a classification model based on a multi-head input structure layer, an LSTM layer, a self-attention layer, a regularization layer, and a classification output layer, then inputting the vectorized text corpus data and the corresponding classification label into the classification model for training, outputting the predicted classification result, calculating the error between the predicted classification result and the corresponding classification label through the cross-entropy loss function, then driving the error back propagation through the AdamWarmup algorithm, and obtaining the trained classification model after multiple training; the multi-head input structure layer can enhance the understanding of the classification model to the input text corpus through multi-layer stacking. The LSTM layer has a gating unit, each gating unit realizes different functions, and each gating unit can adjust the parameters through the size of the threshold in the working process, and the gating units can also be trained through the back propagation algorithm. Since the number of words in the comment text corpus is mostly limited, the LSTM model is used to build the entire model in combination with the attention mechanism. Compared with the large pre-training model such as BERT, this model occupies less resources and is easier to deploy locally. The self-attention layer can effectively capture the dependency and dependency degree between different features in the same group of information. This feature extraction method can effectively play its own advantages for the context-dependent task scenario of comment text corpus sentiment analysis. After training with a large amount of text corpus data, a classification model with high prediction accuracy can be obtained, and the model is especially suitable for fine-grained classification requirements, which makes up for the defect that current classification can only be coarse-grained classification.

[0057] Step c, the social media topic comment data to be classified is pre-processed according to step a and vectorized, and then the vectorized data is input into the classification model trained in step b, and the predicted classification result of the social media topic comment data is output, that is, the fine-grained classification of the social media topic comment data.

[0058] In the above technical solution, the neural network based on the Multi-head (multi-head input) structure and the self-attention mechanism performs multi-label sentiment classification (determines the user's preference, the pro-con opinion, the value, the political view and other implicit information, that is, fine-grained classification), builds a fine-grained classification model of social media topic comments based on self-attention, and learns a certain amount of text corpus data through supervised learning to train a multi-label classification (fine-grained classification) model capable of classifying the sentiment of target text corpus.

[0059] The classification model overcomes the defects of small amount of single comment text data and difficulty in accurately capturing information through multi-head input structure, performs multi-layer synchronous input, such as three-layer simultaneous input, that is, a comment text data is input into the entire model with 3 times the amount. Secondly, the LSTM layer respectively extracts features in parallel without interference for the three input amounts, which can more comprehensively deepen the memory of important words in the sentence (comment text data) and weaken the memory of non-important words. Feature extraction is performed according to this principle. Finally, the self-attention layer can effectively capture the dependency relationship and dependency degree between different features in the same group of information, and for the context-dependent task scenario of comment text corpus sentiment analysis, it can more accurately identify the sentiment and improve the classification accuracy and precision.

[0060] In another technical solution, the pre-processing of step a includes data cleaning and tokenization, data cleaning is to delete URL and user mentions and emoji characters, and tokenization is to remove special characters and stop words.

[0061] In the above technical solution, since the original comment text data contains a lot of redundant and useless information, if these information is not deleted, directly input into the classification model will not only affect the processing speed of the classification model, but also affect the classification accuracy and precision.

[0062] The following illustrates what data cleaning is, as shown in Table 1, the original comment text data without cleaning, removing URL and user mentions (@username) and emoji characters (emoji), to facilitate data annotation personnel to read and understand.

[0063] Table 1 Comparison before and after data cleaning

[0064]

[0065]

[0066] The tokenization of data is described below. As shown in Table 2, the data is further cleaned for model training by removing special characters such as "#", ".", ’, and "?" from the comments, and removing meaningless words (stop words) such as "of", "with", and "am".

[0067] Table 2 Before and after data tokenization

[0068]

[0069]

[0070] An example of manual annotation is as follows: multi-label sentiment analysis is used, and when performing sentiment annotation, the theme of the comment and the influence of the comment keywords on the determination of the sentiment tendency of the comment need to be considered. In addition to the sentiment tendency label, the country and region label and the topic label are also included to complete the fine-grained annotation of the comment. As shown in Table 3, it is one of the sentiment fine-grained classification tables, and the specific classification method can be further optimized based on this method.

[0071] Table 3 Multi-label classification table

[0072]

[0073] Since computers use binary calculation, this annotation reserves a corresponding number of binary bits for each label. There is a certain increase or decrease space on individual labels, and the categories under a certain label can be added or deleted according to needs. Therefore, according to the actual number of each category label, the theoretical number of labels is:

[0074]

[0075] That is: 100. However, according to the actual annotation situation after the experimental data collected in the following examples, the actual number of labels is 92.

[0076] In another technical solution, the skip-gram model is used for vectorization in step a.

[0077] After the previous preprocessing, the text corpus data needs to be vectorized so that it can be input into the model for training. In the field of NLP, the commonly used text corpus vectorization method is word vector. The skip-gram model in the word2vec method will be used in this technical solution.

[0078] In another technical solution, the vectorized text corpus data in step b is copied multiple times as multiple inputs to the multi-head input structure layer at the same time, the multi-head input structure layer encodes the positions of each input, and the position encoding is spliced and fused into each input, and then input to the LSTM layer.

[0079] The splicing and fusion formula of the position encoding is shown in formula (1):

[0080] Formula (1)

[0081] wherein, represents the input vector of the LSTM layer; represents the vectorized text corpus data; represents the position vector of the model input, which is obtained by position calculation formula; represents the fusion operation; the position encoding formula is shown in formula (2):

[0082] Formula (2)

[0083] wherein, PE pos represents the position encoding of the 2th dimension, i represents the position encoding of the 2th dimension, i represents the position of each token in the sentence, PE represents a dimension of the input, pos model represents the dimension of the input; i i According to the trigonometric function and difference formula, as shown in formula (3): pos i d Formula (3)

[0084] wherein, ,

[0085] respectively represent two different variables; The above position encoding formula is transformed into formula (4):

[0086] a b Formula (4)

[0087]

[0088] Formula (4)

[0089] ​​​​​In the technical solution, the input layer of the classification model is the aforementioned preprocessed vectorized text corpus. In order to enable the classification model to fully learn the feature information in the text corpus, different input quantities are spliced and fused, so that the LSTM layer and the self-attention layer of the classification model can differentially extract the feature of the input quantity.

[0090] Considering that the vectorization coding in word units may lose the order information of the words, that is, the same words may contain different information expressions according to different sequential combinations of the words in the context position, in order to make up for the loss of information, the model adds a position coding. Here, an encoding vector is introduced, and the length of the vector is the length of the input word vector (vector dimension). In this way, the position information and the word vector can be superimposed. The position vector at the position pos + k The position vector at the position pos and the position vector at the position k can be linearly combined, which means that this position coding form not only indicates the absolute position of the token, but also contains its relative position information.

[0091] In another technical solution, the LSTM layer extracts features of each input quantity to obtain feature extraction vector data of each input quantity. The LSTM layer can solve the gradient disappearance and gradient explosion problems existing in the back propagation algorithm of the RNN model. The LSTM layer introduces a gating unit, each gating unit realizes different functions, and each gating unit can adjust the parameters by the size of the threshold value in the working process, and the gating units can also be trained by the back propagation algorithm. Since the number of words in the comment text corpus is mostly limited, the LSTM layer is used in combination with the self-attention layer to construct the entire classification model. Compared with the large pre-training model such as BERT, the classification model occupies less resources and is easier to deploy locally.

[0092] In another technical solution, the feature extraction vector data processed by the LSTM layer is input into the self-attention layer as an input quantity. The given feature extraction vector data is valued as X, and three matrices 、 、 are randomly initialized, and three variables Q, K, V are obtained by multiplying X with the above three matrices, respectively, and attention scoring is performed based on formula (5);

[0093] Formula (5)

[0094] wherein, Q represents the vector of the word to be queried in the sentence, K represents the vector of all words in the same sentence, V represents the value of the attention score of the word to be queried in the sentence.Q with K the evaluation or weight of the relationship, d denotes a constant, is the square root of the first dimension of the matrix or is set to other values, i denotes the position of the currently calculated word in the vector;

[0095] The attention score is numerically converted by using a softmax function, and the normalized similarity of the two words is calculated under the condition that Q and K , and the attention calculation result is obtained by multiplying V .

[0096] The technical solution adopts a deep learning model constructed based on a self-attention mechanism. The mechanism refers to the processing mechanism of the human brain neural network. When the neural network receives a large amount of external input information, it selects some key information for processing, thereby improving work efficiency. The self-attention layer can effectively capture the dependency relationship and dependency degree between different features in the same group of information. For the task scenario of comment text corpus sentiment analysis which is self-dependent on context, it can more accurately identify the sentiment tendency, that is, it can accurately score each comment data, thereby improving the classification precision and accuracy.

[0097] In another technical solution, the regularization layer is used to perform “layer regularization” processing on the feature vector output by the self-attention layer.

[0098] The function of the regularization layer: the self-attention layer is equivalent to scoring each word combination in each sentence (comment) and outputting a scoring result. The classification output layer is equivalent to classifying each sentence according to the scoring result, for example, good, medium, and poor (this classification result is relative. For example, after the first training, all scoring results are distributed between 0.0001 and 0.001, and the classification needs to be classified in this interval. After the second training, the results are distributed between 0.0000000001 and 0.00001, and this time the classification will also be performed in this interval). Similar to this; “layer regularization” is equivalent to converting the scoring result into a percentage system, which converts the relative result into an absolute result, thereby improving the classification efficiency.

[0099] In another technical solution, the classification output layer is used to classify the attention calculation result, and then output the classification result. The classification result includes text corpus data and corresponding fine-grained classification labels.

[0100] a classification output layer (a sentiment classification layer) that classifies the feature information after calculation and coding to achieve the goal of sentiment tendency judgment. For example, the text corpus contains three types of labels, and each label has four or five classification results. In actual application, the last layer can be fine-tuned to increase or decrease the sentiment classification categories to meet the needs of different sentiment analysis tasks such as coarse-grained and fine-grained.

[0101] One LSTM layer and one self-attention layer form a feature extraction layer, and the classification model includes multiple feature extraction layers. One LSTM layer and one self-attention layer are stacked to form a feature extraction layer,

[0102] A system for fine-grained classification of social media topic comments is provided, which is used to perform the classification method described above.

[0103] <EMBODIMENT>

[0104] The classification model is as shown in Figure 2 The classification model includes one layer of feature extraction layer, and the two-layer classification model includes two layers of stacked feature extraction layers. In this embodiment, one-layer and two-layer classification models are used for prediction respectively, and the classification results are compared with other classification models.

[0105] The training parameters are set as follows:

[0106] Batch size: 128; (Batch size represents the batch size)

[0107] Self-attention layer activation function: Gelu (Gaussian error linear unit)

[0108] Dropout: 0.2

[0109] Optimization algorithm: AdamWarmup (Adam algorithm with warm-up mechanism);

[0110] Loss function: binary_crossentropy (cross-entropy loss function);

[0111] Training set / validation set: 4 / 1; (the number ratio, for example, if there are 10,000 test data, 80% are used as training data and 20% are used as test data)

[0112] Epoch = 20. (An epoch represents the process of training all training samples once,

[0113] All other models listed below have the same basic parameters as the above parameter settings.

[0114] The transformer model, the LSTM model based on attention mechanism, the LSTM model, and the classification model of the application are compared. The F1-score is used to judge the advantages and disadvantages of each model. The comparison of the single round training results of each model is shown in the following table:

[0115] Table 4 Single round training results

[0116]

[0117] According to the above results, when the number of model layers is one, the classification model of the application and the transformer model have advantages and disadvantages, and the classification model of the application is better than the attention LSTM model (SA-LSTM) and the LSTM model; when the number of model layers is two, the attention LSTM model (SA-LSTM) and the LSTM model have certain overfitting, and the training results of both have different degrees of decline; while the classification model of the application and the transformer model are both improved, and it can be seen that the training results of the classification model of the application are relatively good under data imbalance.

[0118] MH3-SA-LSTM represents the 1-layer classification model of the application, Transformer represents the Transformer model, SA-LSTM represents the LSTM model with self-attention mechanism, and LSTM represents the long short-term memory model.

[0119] Figures 1-10 The meanings of each English letter in the above table are as follows:

[0120] Input : input, w ( t ): center word, Hidden-Layer : hidden layer (or intermediate layer), h ( t ): hidden layer variable, Output : output, w ( t-2 ): word t-2 time units before the center word, w ( t-1 ): word t-1 time units before the center word, w ( t+1 ): word t+1 time units after the center word, w ( t+2): the word at the t+2 time unit after the center word, accuracy: accuracy, epochs: training epochs (indicates the training result at this time is the training result of which generation), Trainning accuracy: training set accuracy, Vaildation accuracy: validation set accuracy, Trainningloss: training set loss rate, Vaildationloss: validation set loss rate, loss: loss rate.

[0121] While embodiments of the application have been disclosed in connection with the above specification, it will be apparent to those skilled in the art that numerous modifications can be made thereto without departing from the overall concept of the application. Accordingly, it is intended that all such modifications be included within the scope of the claims and their equivalents.

Claims

1. A method for fine-grained classification of social media topic comments based on self-attention, characterized in that, The method comprises the following steps: Step a, preprocessing and fine-grained classification labeling of original social media topic comment data to obtain a text corpus dataset, and vectorizing to obtain a corresponding vectorized text corpus dataset; Step b, constructing a classification model based on a multi-head input structure layer, an LSTM layer, a self-attention layer, a regularization layer, and a classification output layer, then inputting the vectorized text corpus data and the corresponding classification labels into the classification model for training, outputting the predicted classification results, calculating the error between the predicted classification results and the corresponding classification labels through a cross-entropy loss function, then driving the error back propagation through an AdamWarmup algorithm, and obtaining the trained classification model after multiple training; Step c, preprocessing and vectorizing the social media topic comment data to be classified according to step a, then inputting the vectorized data into the trained classification model in step b to output the predicted classification results of the social media topic comment data, i.e., the fine-grained classification of the social media topic comment data; The classification model comprises multiple feature extraction layers, each of which consists of an LSTM layer and a self-attention layer; The vectorized text corpus data in step b is copied multiple times and simultaneously input into the multi-head input structure layer as multiple input quantities, the multi-head input structure layer encodes the positions of the input quantities, and the position encoding is spliced and fused into each input quantity, then input into the LSTM layer; The splicing and fusion formula of the position encoding is shown in formula (1): （1） wherein, represents an input vector of the LSTM layer; represents vectorized text corpus data; represents a position vector of the model input quantity, which is obtained by a position calculation formula; represents a fusion operation; the position encoding formula is shown in formula (2): Equation (2) wherein, PE ( pos , 2 i ) denotes the position encoding of the 2 i th dimension, PE ( pos , 2 i + 1) denotes the position encoding of the 2 i + 1th dimension, pos denotes the position of each subword in the sentence, i denotes a dimension of the input quantity, d model denotes a dimension of the input quantity; According to the trigonometric function and difference formula, as shown in formula (3): Equation (3) wherein a , b are represented as two different variables; The above position encoding formula is transformed into formula (4): Equation (4).

2. The method for fine-grained classification of social media topic comments based on self-attention as claimed in claim 1, wherein, The preprocessing of step a includes data cleaning and tokenization, data cleaning is to delete URL and user mentions and expression characters, and tokenization is to remove special characters and stop words.

3. The method for fine-grained classification of social media topic comments based on self-attention as claimed in claim 1, wherein, The skip-gram model is used for vectorization in step a.

4. The method for fine-grained classification of social media topic comments based on self-attention as claimed in claim 1, wherein, The LSTM layer extracts features from each input quantity to obtain feature extraction vector data of each input quantity.

5. The method for fine-grained classification of social media topic comments based on self-attention as claimed in claim 1, wherein, The feature extraction vector data processed by the LSTM layer is input into the self-attention layer as an input quantity, and the given feature extraction vector data is assigned as X, three matrices are randomly initialized , , , respectively multiplied by X and the above three matrices to obtain three variables Q, K, and V , and attention scoring is performed based on formula (5); Equation (5) wherein, Q represents the vector of the word to be queried in the sentence, K represents the vector of all words in the same sentence, V represents the evaluation or weight of the relationship Q to K , d represents a constant, which is the square root of the first dimension of the matrix or is set to other values, i represents the position of the currently calculated word in the vector; The attention score is converted into a numerical value by using a softmax function, and the normalized similarity of the two words is calculated given Q and K , and the attention calculation result is obtained by multiplying V .

6. The method for fine-grained classification of social media topic comments based on self-attention as claimed in claim 1, wherein, The regularization layer is used to perform "layer regularization" processing on the feature vectors output by the self-attention layer; The classification output layer is used to classify the attention calculation results, and then output the classification results, which include the text corpus data and the corresponding fine-grained classification labels.

7. A system for fine-grained classification of social media topic reviews based on self-attention, characterized in that, A device for performing the classification method of any one of claims 1-6.

Images