An unsupervised reader book review sentiment analysis method based on variational autoencoder

Abstract

The present invention belongs to the field of artificial intelligence technology, and specifically relates to an unsupervised reader review sentiment analysis method based on a variational autoencoder. The method first collects an unlabeled training corpus of reader review text, performs word segmentation, word embedding, and concatenation on each reader review in the training corpus to obtain a text matrix; these text matrices form a training dataset, and the text matrices in the obtained training dataset are used to train a variational autoencoder; the text to be analyzed is subjected to word segmentation, word embedding, and concatenation to obtain a text matrix, which is then input into the trained variational autoencoder to obtain a sentiment score. This method is suitable for situations where labeled training data is unavailable. It can use unlabeled training data to train a sentiment analysis model, helping to reduce the cost of training data annotation.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence technology, and in particular relates to an unsupervised reader book review sentiment analysis method based on a variational autoencoder. Background Art

[0002] Sentiment analysis based on online book reviews is a common method for studying the reception of translated works. Its most fundamental and important function is to analyze the sentiment of the textual content of readers' book reviews. Unsupervised deep learning methods are widely used in natural language processing due to their applicability to large-scale reviews and the time-saving and labor-saving nature of manual annotation. They offer valuable insights into translation research in the digital humanities era, facilitating research on text information mining and sentiment analysis of translated works and their reviews. Initial sentiment analysis algorithms were rule-based, requiring humans to manually define a set of sentiment-laden words, such as "good" and "boring," and then determine the sentiment of the text based on these rules. This approach, for example, was the DASA algorithm proposed by Qiu Guang et al. in 2010. However, the high degree of freedom and flexibility of natural language makes it difficult for manually summarized rules to accurately cover all situations. With the development of machine learning, machine learning-based methods have gradually become mainstream, such as the multi-class support vector machine-based algorithm proposed by Chen Chen Chien et al. in 2011. Due to its excellent fitting and generalization capabilities, deep learning has become the mainstream method in machine learning in recent years, such as the DPCNN model proposed by Rie Johnson et al. in 2017 and the XLNet model proposed by Zhilin Yang et al. in 2019.

[0003] However, all of the above methods belong to supervised learning, which usually requires a large amount of labeled training data to train deep neural networks. Labeling the sentiment tendencies of a large amount of reader book review corpus is a very time-consuming task. The text matrix dimension of the book review corpus is usually very high, and the distances between data points are relatively far. Therefore, traditional unsupervised clustering methods such as K-means, Gaussian mixture model and non-negative matrix factorization are prone to dimensionality curse. To overcome this difficulty, the present invention proposes an unsupervised reader book review sentiment analysis method based on variational autoencoder, which combines Gaussian mixture model and variational autoencoder to achieve more accurate unsupervised reader book review sentiment analysis. Summary of the Invention

[0004] The technical problem solved by the present invention is to overcome the deficiency in the prior art that training deep learning models requires a large amount of labeled data, and provide an unsupervised reader book review sentiment analysis method based on variational autoencoders, which can use a training corpus without labeled information to train a deep learning model for reader book review text sentiment analysis.

[0005] The present invention provides an unsupervised reader review sentiment analysis method based on a variational autoencoder. First, an unlabeled reader review text training corpus is collected. Each reader review in the training corpus is segmented, word embedded, and concatenated to obtain a text matrix. These text matrices constitute a training data set, and the text matrices in the obtained training data set are used to train a variational autoencoder. For the text to be analyzed, the text matrix is segmented, word embedded, and concatenated to obtain the text matrix, which is then input into the trained variational autoencoder to obtain a sentiment score. The specific steps are as follows:

[0006] Step S1: Collect a reader review dataset based on the actual task, or use a large-scale public reader review corpus, such as Amazon Review;

[0007] Step S2: Assuming that the training corpus contains L review corpora, for the i-th training corpus in the training corpus, perform word segmentation, word embedding and splicing operations to obtain the text matrix X i , the text matrix of all training corpora in the training corpus constitutes the training data set:

[0008] Step S3: Introduce a latent variable z∈{0,1} that obeys a discrete distribution to represent two different emotion categories. The probabilities of z taking 0 and 1 are π0 and π1 respectively. The vector π is denoted as: π=[π0 π1] T , initialize π to π = [0.5 0.5] T ;

[0009] For category z, we introduce the rule whose mean is μ y，z , the variance is The continuous latent variable y of the Gaussian distribution is μ y，z Initialized to μ y，z =0, Initialized to

[0010] Step S4: For any training text matrix X, two deep convolutional neural networks are used to fit the mean and variance of the distribution p(y|X) as encoders, denoted as μ and μ respectively. y (X) and For any latent variable y, two deep deconvolutional neural networks are used to fit the mean and variance of the distribution p(X|y) as decoders, denoted as μ and μ respectively. x (y) and

[0011] Step S5: Train the model by performing the encoder, decoder, and clustering training in turn.

[0012] Step S6: According to the clustering results of the training set, select several corpora from a certain cluster and manually judge their emotional tendencies. The one with the largest proportion is taken as the emotional tendency of the cluster, and the opposite emotional tendency is taken as the emotional tendency of the other cluster. The discrete latent variable value corresponding to the positive emotional tendency is recorded as c 正 ∈{0, 1};

[0013] Step S7: For any text matrix X to be analyzed, first start with the mean μ y The variance of (X) is The latent variable y′ is sampled from the Gaussian distribution of i Calculate the probability that the sentiment tendency is positive. If the probability is greater than 0.5, the sentiment tendency is judged to be positive. Otherwise, it is judged to be negative. The probability value of the sentiment being positive is used as the sentiment score. The larger the value, the more positive the evaluation.

[0014] Furthermore, the training encoder and decoder in step S5 are specifically as follows: i Represents the i-th training text matrix, and uses the stochastic gradient descent algorithm to minimize the following loss function:

[0015]

[0016] Among them, 1 is a column vector of all ones, ||·||2 is the vector l2 norm, ||·|| F is the matrix Frobenius norm, . / is the matrix bitwise division, y i Indicates that the mean The variance is A vector sampled from a Gaussian distribution; z i is the inferred X i The category is calculated as follows:

[0017]

[0018] ρ c，i For X i The probability that the category is c is calculated as:

[0019]

[0020] Among them, y′ i Indicates that the mean is μ y (X i ) variance is A vector sampled from a Gaussian distribution, represents the probability density function of the multivariate Gaussian distribution with mean μ and covariance matrix ∑ at x.

[0021] Furthermore, the cluster training in step S5 is specifically performed by alternately performing the following iterative processes:

[0022]

[0023] Furthermore, in step S7, for the latent variable y′, the method for calculating the probability that the corresponding category is positive is:

[0024]

[0025] Compared with the prior art, the advantages of the present invention are:

[0026] The unsupervised reader book review sentiment analysis method based on variational autoencoder of the present invention realizes the unsupervised reader book review sentiment analysis function. Compared with the traditional supervised deep learning reader book review sentiment analysis method, it does not require the pre-labeling of a large amount of training corpus, which can effectively save the labeling cost; compared with the traditional unsupervised clustering method, deep learning is introduced to fit the conditional probability distribution, thereby achieving better performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 Flowchart of the method of the present invention.

[0028] Figure 2 Two deep convolutional neural network structures introduced as encoders in this invention.

[0029] Figure 3 Two inverse deep convolutional neural network structures are introduced as decoders in this invention. DETAILED DESCRIPTION

[0030] The present invention will be further described below with reference to the embodiments and accompanying drawings.

[0031] like Figure 1 As shown, the unsupervised reader book review sentiment analysis method based on variational autoencoder provided by the present invention includes the following steps:

[0032] Step S1: collecting a training corpus of unlabeled reader reviews of the same text type according to a specific task type. For example, for a sentiment analysis task of English-translated classic book reviews, English-translated classic book reviews on an online book review platform are collected as a training corpus.

[0033] Step S2: Assuming that the training corpus contains L review corpora, for the i-th training corpus in the training corpus, perform word segmentation, word embedding and concatenation operations to obtain the text matrix X i , the text matrix of all training corpora in the training corpus constitutes the training dataset

[0034] Step S3, initialize vector π to π = [0.5 0.5] T , initialization vector μ y，z μ y，z =0, initialization vector for

[0035] In step S4, two independent deep convolutional neural networks are introduced as encoders, such as Figure 2 As shown in the figure, the two deep convolutional neural networks each contain a convolutional layer, a convolutional pooling layer and a convolutional adaptive pooling layer, which perform convolution, convolution and pooling, and convolution and adaptive pooling operations on the input comment text matrix in sequence, and predict the mean and variance of the conditional probability distribution p(y|X) respectively; these two deep convolutional neural networks are connected by the function μ y (X) and express;

[0036] Two independent deep deconvolutional neural networks are introduced as decoders, such as Figure 3 As shown in the figure, the two deep convolutional neural networks each contain three deconvolution zero-padding layers, which perform three deconvolution and zero-padding operations on the latent variables in sequence to predict the mean and variance of the conditional probability distribution p(X|y) respectively; these two deep deconvolutional neural networks are used to calculate the mean and variance of the conditional probability distribution p(X|y). x (y) and express;

[0037] Step S5, training the encoder-decoder and clustering in turn to train the model;

[0038] Step S5 specifically includes:

[0039] Step S501: Let X i Represents the i-th training text matrix, and uses the stochastic gradient descent algorithm to minimize the following loss function:

[0040]

[0041] Among them, 1 is a column vector of all ones, ||·||2 is the vector l2 norm, ||·|| F is the matrix Frobenius norm, . / is the matrix bitwise division, y i Indicates that the mean The variance is A vector sampled from a Gaussian distribution, and z i is the inferred X i The category is calculated as follows:

[0042]

[0043] ρ c，i For Xi The probability that the category is c is calculated as

[0044]

[0045] Among them, y′ i Indicates that the mean is μ y (X i ) variance is A vector sampled from a Gaussian distribution, represents the probability density function value of the multivariate Gaussian distribution with mean μ and covariance matrix ∑ at x;

[0046] Step S502: Alternately perform the following iterative process to perform clustering

[0047]

[0048]

[0049]

[0050]

[0051]

[0052] Step S6: According to the clustering results of the training set, select a number of corpora from a certain cluster, manually judge their emotional tendencies, take the one with the largest proportion as the emotional tendency of the cluster, and take the opposite emotional tendency as the emotional tendency of the other cluster. The discrete latent variable value corresponding to the positive emotional tendency is recorded as c 正 ∈{0, 1};

[0053] Step S7: For any text matrix X to be analyzed, firstly, y The variance of (X) is The latent variable y′ is sampled from the Gaussian distribution of i The probability that its sentiment tendency is positive is calculated as follows

[0054]

[0055] If the probability is greater than 0.5, the sentiment tendency is judged to be positive, otherwise it is judged to be negative.

[0056] The above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. An unsupervised reader book review sentiment analysis method based on variational autoencoder, characterized by: First, we collect an unlabeled training corpus of reader review texts. We perform word segmentation, word embedding, and concatenation on each reader review in the training corpus to obtain a text matrix. These text matrices form a training dataset, and we use the text matrices in the training dataset to train a variational autoencoder. For the text to be analyzed, we perform word segmentation, word embedding, and concatenation on it to obtain a text matrix, which is then input into the trained variational autoencoder to obtain a sentiment score. The specific steps are as follows: Step S1: Collect a reader book review dataset based on the actual task, or use a public large-scale reader book review corpus; Step S2: Assuming that the training corpus contains L review corpora, for the i-th training corpus in the training corpus, perform word segmentation, word embedding and concatenation operations to obtain the text matrix X i , the text matrix of all training corpora in the training corpus constitutes the training dataset Step S3: Introduce a latent variable z∈{0,1} that obeys a discrete distribution to represent two different emotion categories. The probabilities of z taking 0 and 1 are π0 and π1 respectively. The vector π is π=[π0 π1] T , initialize π to π = [0.5 0.5] T ; For the two categories z, we introduce the mean μ y，z The variance is The continuous latent variable y of the Gaussian distribution is μ y，z Initialized to μ y,z =0, Initialized to Step S4: For any training text matrix X, two deep convolutional neural networks are used to fit the mean and variance of the distribution p(y|X) as encoders, denoted as μy(X) and μy(X). For any latent variable y, two deep deconvolutional neural networks are used to fit the mean and variance of the distribution p(X|y) as decoders, denoted as μx(y) and μx(y) respectively. Step S5: Train the model by performing encoder-decoder training and clustering in turn; Step S6: According to the clustering results of the training set, select several corpora from a certain cluster and manually judge their emotional tendencies. The one with the largest proportion is taken as the emotional tendency of the cluster, and the opposite emotional tendency is taken as the emotional tendency of the other cluster. The discrete latent variable value corresponding to the positive emotional tendency is recorded as c 正 ∈{0, 1}; Step S7: For any text matrix X to be analyzed, first start with the mean μ y The variance of (X) is The latent variable y′ is sampled from the Gaussian distribution of i Calculate the probability that the sentiment tendency is positive. If the probability is greater than 0.5, the sentiment tendency is judged to be positive, otherwise it is judged to be negative. The probability value of the sentiment being positive is used as the sentiment score, and the larger the value, the more positive the evaluation; The specific method of training the encoder and decoder in step S5 is as follows: i Represents the i-th training text matrix, and uses the stochastic gradient descent algorithm to minimize the following loss function: Among them, 1 is a column vector of all ones, ||·||2 is the vector l2 norm, ||·|| F is the matrix Frobenius norm, . / is the matrix bitwise division, y i Indicates that the mean The variance is A vector sampled from a Gaussian distribution; z i is the inferred X i The category is calculated as follows: ρ c，i For X i The probability that the category is c is calculated as: Among them, y′ i Indicates that the mean is μ y (X i ) variance is A vector sampled from a Gaussian distribution, represents the probability density function of the multivariate Gaussian distribution with mean μ and covariance matrix ∑ at x.

2. The unsupervised reader book review sentiment analysis method according to claim 1, characterized in that: The cluster training in step S5 is specifically performed by alternately performing the following iterative processes:

3. The unsupervised reader book review sentiment analysis method according to claim 2, characterized in that: In step S7, for the latent variable y′, the method for calculating the probability that the corresponding category is positive is:

4. The unsupervised reader book review sentiment analysis method according to claim 2, characterized in that: The encoder consists of two independent deep convolutional neural networks, each of which contains a convolution layer, a convolutional pooling layer and a convolutional adaptive pooling layer. They all perform convolution, convolution and pooling, and convolution and adaptive pooling operations on the input comment text matrix in sequence, and predict the mean and variance of the conditional probability distribution p(y|X) respectively.

5. The unsupervised reader book review sentiment analysis method according to claim 2, characterized in that: The decoder consists of two independent deep convolutional neural networks, each of which contains three deconvolution zero-padding layers, which perform three deconvolution and zero-padding operations on the latent variables in sequence to predict the mean and variance of the conditional probability distribution p(X|y) respectively.

Images