Adaptive Weighted Short Text Sentiment Classification Method, Electronic Device and Storage Medium

Abstract

The present invention discloses an adaptive weighted short text sentiment classification method, an electronic device, and a storage medium. The steps of the short text sentiment classification method include: 1) obtaining a short text collection and performing preprocessing, and using the DeepMoji model to extract feature vectors; 2) pre-training a classification model using the short text sentiment labels and the short text feature vectors extracted in step 1; 3) adaptively assigning weights to samples of different age groups, and updating the classification model parameters based on the weighted loss function; 4) repeating step 3 until the sample weights and the model parameters converge, and obtaining an updated classification model; 4) using the updated classification model to perform sentiment classification on short texts. The present invention can remove the bias of the sentiment classification model towards age, so as to complete more accurate and fair sentiment classification.

Description

Technical Field

[0001] The present invention belongs to the fields of text classification and fair representation, and specifically relates to an adaptive weighted short text sentiment classification method, an electronic device, and a storage medium. Background Art

[0002] In recent years, the progress of e-commerce technology has advanced by leaps and bounds, and the number of e-commerce users has also exploded geometrically. This has generated a huge amount of user comments on goods. Therefore, the traditional method of manually reading comments is no longer suitable. Distinguishing the emotions of commenters through natural language processing technology is the future development direction. However, due to the complexity of human language, an evaluation expressing liking in one age group may mean irony in another age group, resulting in differences in the accuracy of the sentiment classification model for different groups, which is disadvantageous to some groups. Therefore, improving the fairness of the model is an important issue in natural language processing tasks. Summary of the Invention

[0003] The present invention is to solve the above-mentioned deficiencies existing in the prior art, and proposes an adaptive weighted short text sentiment classification method, an electronic device, and a storage medium, in order to improve the fairness of the model while ensuring the accuracy of the sentiment classification model.

[0004] To achieve the above object, the present invention adopts the following technical solutions:

[0005] The adaptive weighted short text sentiment classification method of the present invention is characterized by including the following steps:

[0006] S1. Obtain short texts with emoticons in the comment data as a corpus, and perform preprocessing operations of classification and data cleaning according to the age of the commenter of any short text and the types of emoticons it contains, to obtain a corresponding set of labeled short texts, denoted as wherein, represents the i-th short text of the commenter in the k-th age stage, represents the i-th short text of the commenter in the k-th age stage of the sentiment label, taking values in {0, 1}; i = 1, 2,..., N k N k represents the total number of short texts of the commenter in the k-th age group; k = 1, 2,..., K, and K represents the total number of age stages;

[0007] Use the trained feature extraction model DeepMoji to extract text features from the set of short texts to obtain a set of short text features wherein, represents the feature vector of the i-th short text of the commenter in the k-th age group;

[0008] S2. Construct a classification model consisting of a fully connected network, and input the short text feature set into the classification model for processing, and output the sentiment prediction value of the i-th short text feature of reviewers in the k-th age group

[0009] S3. Use Equation (1) to construct the cross-entropy loss L of the classification model:

[0010]

[0011] S4. Use the gradient descent method to pre-train the classification model, and calculate the cross-entropy loss L to update the model parameters until the cross-entropy loss L converges, so as to obtain the pre-trained classification model;

[0012] S5. Update the pre-trained classification model:

[0013] S5.1. Define the current iteration number as t, and initialize t = 1; use the pre-trained classification model as the classification model in the (t - 1)-th iteration;

[0014] S5.2. Use Equation (2) to calculate the weight of the i-th short text of reviewers in the k-th age group in the t-th iteration of

[0015]

[0016] In Equation (2), α is a parameter with a value range between [0, 100], η is a parameter with a value range between [0, 3], exp is the natural base, represents the weight of the i-th short text of reviewers in the k-th age group in the (t - 1)-th iteration. When t = 1, let represents the sentiment label predicted by the classification model for the i-th short text of reviewers in the k-th age group in the t-th iteration, and the value range is [0, 1]. When t = 1, let the sentiment label predicted by the classification model for the i-th short text in the (t - 1)-th iteration be the sentiment label predicted by the classification model for the i-th short text in the (t - 2)-th iteration

[0017] Use Equation (3) to construct the weighted loss function L in the t-th iteration t :

[0018]

[0019] S5.2. Train the classification model for the t-th iteration using the gradient descent method and minimize the weighted loss function L t to update the parameters of the classification model for the t-th iteration, thereby obtaining the classification model for the (t + 1)-th iteration;

[0020] S5.3. After assigning t + 1 to t, return to step S5.1 and execute sequentially until t reaches the maximum number of iterations t max to obtain the updated classification model;

[0021] S6. Input any short text feature vector into the updated classification model and output the predicted sentiment category of the corresponding short text.

[0022] An electronic device according to the present invention, comprising a memory and a processor, is characterized in that the memory is used to store a program supporting the processor to execute the short text classification method described in claim 1, and the processor is configured to execute the program stored in the memory.

[0023] A computer-readable storage medium according to the present invention, on which a computer program is stored, is characterized in that the computer program executes the steps of the short text classification method described in claim 1 when run by a processor.

[0024] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0025] 1. The present invention can adaptively calculate the optimal weight of each sample as the model parameters are updated, and the weights obtained are more accurate than those obtained by the direct weighting method, thereby greatly improving the fairness of the sentiment classification model for different groups.

[0026] 2. The present invention performs fine-grained weighting on the samples near the decision boundary at the sample level, assigns higher weights to the samples near the decision boundary, and can effectively solve the distribution shift problem of the test set, improving the generalization of the sentiment classification model.

[0027] 3. The present invention uses the DeepMoji model to extract text features, more truly and comprehensively reflecting the sentiment preferences expressed by users in the text, and improving the classification accuracy of the sentiment classification model for short text sentiment.

[0028] 4. The present invention is based on a deep fully-connected neural network model, calculates the weight for each short text according to the age stage of the reviewer of the short text and the distance from the output value of the model for the short text to the decision boundary, designs a weighted loss function, and continuously optimizes the model parameters using stochastic gradient descent, thereby improving the fairness of the sentiment classification model. Description of the Drawings

[0029] Figure 1This is the flowchart of the short text sentiment method with adaptive weighting of the present invention. Detailed implementation manners

[0030] In this embodiment, a short text sentiment classification method with adaptive weighting is as Figure 1 shown, and includes the following steps: 1) Obtain a short text set and perform preprocessing, and use the feature extraction model DeepMoji model to extract feature vectors; 2) Use the short text sentiment labels and the short text feature vectors extracted in step 1 to pre-train a classification model; 3) Adaptively assign weights to samples of different age stages, and update the classification model parameters based on the weighted loss function; 4) Repeat step 3 until the sample weights and model parameters converge, and obtain an updated classification model; 4) Use the updated classification model to perform sentiment classification on short texts. Specifically, the short text sentiment classification method with adaptive weighting includes the following steps:

[0031] S1. Obtain short texts with emoticons in the comment data as a corpus. Since the number of comments is huge and the cost of manual annotation is too high, and since emoticons can more accurately reflect the emotions of commenters, this method uses texts with emoticons in the comments as training data, and uses the types of emoticons in the text as the sentiment labels of the text. Perform preprocessing operations of classification and data cleaning according to the age of the commenter of any short text and the types of emoticons it contains, and obtain a corresponding labeled short text set, denoted as where, represents the i-th short text of the commenter in the k-th age stage, represents the i-th short text of the commenter in the k-th age stage 's sentiment label, taking values in {0, 1}; i = 1, 2,..., N k , N k represents the total number of short texts of the commenter in the k-th age group; k = 1, 2,..., K, K represents the total number of age stages, taking the value of 5, which means dividing the user group into 5 age stages, namely under 30 years old, 30 - 39 years old, 40 - 49 years old, 50 - 59 years old, and 60 years old and above;

[0032] Use the trained feature extraction model DeepMoji to extract text features from the short text set to obtain a short text feature set where, represents the feature vector of the i-th short text of the commenter in the k-th age group, and the feature dimension is 108 dimensions;

[0033] S2. Construct a classification model composed of fully connected networks, and use the short text feature set Input it into the classification model for processing, and output the sentiment prediction value of the i-th short text feature of reviewers in the k-th age group

[0034] S3. Construct the cross-entropy loss L of the classification model using Equation (1):

[0035]

[0036] S4. Pre-train the classification model using the gradient descent method, and calculate the cross-entropy loss L to update the model parameters until the cross-entropy loss L converges, so as to obtain the pre-trained classification model;

[0037] S5. Update the pre-trained classification model:

[0038] S5.1. Define the current iteration number as t, and initialize t = 1; use the pre-trained classification model as the classification model of the (t - 1)-th iteration;

[0039] S5.2. Calculate the weight of the i-th short text of reviewers in the k-th age group in the t-th iteration using Equation (2) of

[0040]

[0041] In Equation (2), α is a parameter with a value range between [0, 100], η is a parameter with a value range between [0, 3], exp is the natural logarithm base, represents the weight of the i-th short text of reviewers in the k-th age group in the (t - 1)-th iteration. When t = 1, let represents the sentiment label predicted by the classification model for the i-th short text of reviewers in the k-th age group in the t-th iteration, with a value range of [0, 1]. When t = 1, let the sentiment label predicted by the classification model for the i-th short text in the (t - 1)-th iteration the sentiment label predicted by the classification model for the i-th short text in the (t - 2)-th iteration This update formula has two main innovations: First, the termination condition for weight iteration is that the output value of the classification model is statistically independent of the reviewer's age. This termination condition meets the fairness index of demographic parity and can ensure the fairness of the classification model on the training set; Second, higher weights are assigned to samples whose predicted values are near the classification boundary, which can enhance the generalization of the classification model and obtain better classification accuracy and fairness on the test set.

[0042] Construct the weighted loss function L of the t-th iteration using Equation (3) t :[[]]

[0043]

[0044] S5.2. Use the gradient descent method to train the classification model for the t-th iteration and minimize the weighted loss function L t to update the parameters of the classification model for the t-th iteration, thereby obtaining the classification model for the (t + 1)-th iteration;

[0045] S5.3. After assigning t + 1 to t, return to step S5.1 and execute sequentially until t reaches the maximum number of iterations t max up to, where t max takes a value of 100, thereby obtaining the updated classification model;

[0046] S6. Input any short text feature vector into the updated classification model and output the predicted sentiment category of the corresponding short text.

[0047] In this embodiment, an electronic device includes a memory and a processor. The memory is used to store a program that supports the processor to execute the above short text sentiment classification method, and the processor is configured to execute the program stored in the memory.

[0048] In this embodiment, a computer-readable storage medium stores a computer program on the computer-readable storage medium. When the computer program is run by a processor, it executes the steps of the above short text sentiment classification method.

[0049] Table 1 shows the sentiment classification accuracy and fairness of different methods on the Twitter dataset. The TPR GAP is used as the fairness metric. The higher the accuracy metric, the better, and the lower the fairness metric, the better. In the experiment, ERM is used as the baseline model. In addition to this method, the INLP algorithm with spatial projection debiasing, the ADV algorithm with adversarial training, and the DADV algorithm combining adversarial training and ensemble learning are also compared. This method achieves the best accuracy and fairness.

[0050] Table 1: Sentiment classification accuracy and fairness of different methods on the Twitter dataset

[0051] model accuracy fairness ERM 71.6 31.2 INLP 68.5 23.8 ADV 74.3 22.2 DADV 74.5 18.5 this method 75.3 8.5

[0052] In summary, the present invention adaptively weights the text, removes the bias of the sentiment classification model against age, and thus completes more accurate and fair sentiment classification.

Claims

1. An adaptive weighted short text sentiment classification method, characterized in that Including the following steps: S1. Obtain short texts with emoticons in the comment data as the corpus, and perform preprocessing operations of classification and data cleaning according to the age of the commentator of any short text and the types of emoticons it contains, to obtain the corresponding set of short texts with labels, denoted as where represents the i-th short text of the commentator in the k-th age group, represents the i-th short text of the commentator in the k-th age group and the sentiment label of, taking values in {0, 1}; i = 1, 2, …, N k , N k represents the total number of short texts of the commentator in the k-th age group; k = 1, 2, …, K, and K represents the total number of age groups; Use the trained feature extraction model DeepMoji for the short text set Extract text features to obtain a short text feature set Among them, represents the feature vector of the i-th short text of reviewers in the k-th age group; S2. Construct a classification model composed of a fully connected network, and input the short text feature set into the classification model for processing, and output the sentiment prediction value of the i-th short text feature of reviewers in the k-th age group S3. Construct the cross-entropy loss L of the classification model using Equation (1): S4. Pre-train the classification model using the gradient descent method, and calculate the cross-entropy loss L to update the model parameters until the cross-entropy loss L converges, thereby obtaining the pre-trained classification model; S5. Update the pre-trained classification model: S5.

1. Define the current iteration number as t, and initialize t = 1; use the pre-trained classification model as the classification model for the (t - 1)-th iteration; S5.

2. Calculate the weight of the \(i\)-th short text of reviewers in the \(k\)-th age group in the \(t\)-th iteration using Equation (2). of In formula (2), α is a parameter with a value range between [0, 100], η is a parameter with a value range between [0, 3], exp is the natural base, represents the weight of the i-th short text of the reviewer in the k-th age group in the (t - 1)-th iteration. When t = 1, let represents the sentiment label predicted by the classification model for the i-th short text of the reviewer in the k-th age group in the t-th iteration, and the value range is [0, 1]. When t = 1, let the sentiment label predicted by the classification model for the t-th short text in the (t - 1)-th iteration be the sentiment label predicted by the classification model for the i-th short text in the (t - 2)-th iteration Construct the weighted loss function \(L\) for the \(t\)-th iteration using Equation (3). t : S5.

2. Train the classification model for the t-th iteration using the gradient descent method and minimize the weighted loss function L t to update the parameters of the classification model for the t-th iteration, thereby obtaining the classification model for the (t + 1)-th iteration; S5.

3. After assigning t + 1 to t, return to step S5.1 and execute sequentially until t reaches the maximum number of iterations t max to obtain the updated classification model S6. Input any short text feature vector into the updated classification model and output the corresponding predicted sentiment category of the short text.

2. An electronic device, comprising a memory and a processor, characterized in that, The memory is used to store a program for supporting the processor to execute the short text classification method described in claim 1, and the processor is configured to execute the program stored in the memory.

3. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is run by the processor, it executes the steps of the short text classification method described in claim 1.

Images