Unsupervised opinion summarization generation method and system based on anti-template decoupling

Abstract

The application discloses a kind of based on anti-template decoupling's unsupervised opinion abstract generation method and system, first constructs and trains opinion abstract's anti-template generation model, utilizes contrast learning and variational autoencoder, opinion is regarded as supervision signal, obtains the representation of opinion, takes the average of several opinion representations and decodes, and the anti-template of opinion is initially obtained;Decoupling text representation opinion abstract generation model is further constructed, and the representation of opinion abstract is decoupled into content, template two parts, and the representation vector of content is obtained;Finally, the unsupervised opinion abstract model is trained, and the representation vector of content is input into the decoder of unsupervised opinion abstract model, reconstructs original text by decoder, and the generation of opinion abstract is completed.This method generates the "anti-template" representation of text representation by training in text information, removes template information in text representation using orthogonal decoupling mode, so as to reduce redundant information, and the abstract generation is more accurate, and the work efficiency is higher.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of artificial intelligence and natural language processing, and mainly relates to an unsupervised opinion summary generation method and system based on anti-template decoupling. BACKGROUND

[0002] Artificial intelligence is a science that studies, develops theories, methods, technologies and application systems for simulating, extending and expanding human intelligence. With the explosive growth of data, significant improvement in computing power, and the continuous emergence of new algorithms, especially deep learning algorithms, artificial intelligence theory and technology have become increasingly mature, and the application field has also expanded, and has initially met the conditions for landing applications in various fields.

[0003] A text summary is a paragraph of text extracted from one or more texts, which contains important information in the original text and is no more than or much shorter than half of the original text. It can be divided into two categories, namely extractive and generative. The extractive summary is limited by the quality of the text itself, and it is also difficult to get the most concise and refined summary text. With the rapid development of text generation technology and the continuous progress of deep learning technology, the scene and related technology of generative summary have gradually become rich. Unsupervised opinion summary refers to the use of various natural language processing technologies to automatically extract the main points and summaries from a text without labeled data. This field involves natural language processing, machine learning, text mining and other technologies, and the research goal is to replace humans with machines to process and analyze massive amounts of text data. For example, when using a shopping website, an opinion summary generator can read the user's review information of a product and extract the key information of the product (such as the advantages and disadvantages of the product), which can reduce the user's browsing time of the product information, improve the user's operation efficiency, and greatly help people's production and life.

[0004] In the opinion summary field, due to the lack of large-scale opinion summary data sets, the mainstream methods in the academic and industrial fields currently use unsupervised deep learning to generate summaries. However, after the deep learning model integrates a large amount of text information, it often generates broad and redundant information, which affects the quality of the generated summary and is not conducive to user reading and reference. Therefore, how to eliminate the broad and redundant information in the unsupervised opinion summary is an important problem in the opinion summary field. SUMMARY

[0005] The application is just aimed at the problem of too much redundant information in the prior art unsupervised opinion summary, and provides an unsupervised opinion summary generation method and system based on anti-template decoupling. First, an anti-template generation model of the opinion summary is constructed and trained, the opinion is taken as a supervision signal by using contrast learning and a variational autoencoder, the representation of the opinion is obtained, the average of a plurality of opinion representations is taken for decoding, and the anti-template of the opinion is obtained preliminarily. Then, a decoupled text representation opinion summary generation model is constructed, the representation of the opinion summary is decoupled into content and template, and the representation vector of the content is obtained. Finally, an unsupervised opinion summary model is trained, the content representation vector is input into the decoder of the unsupervised opinion summary model, the original text is reconstructed through the decoder, and the generation of the opinion summary is completed. The method trains the anti-template representation of the text representation in the text information, removes the template information in the text representation by using the orthogonal decoupling mode, thereby decoupling the required content information, generating the final summary text, reducing the redundant information, and making the summary generation more accurate and the work efficiency higher.

[0006] In order to achieve the above object, the technical scheme adopted by the application is: an unsupervised opinion summary generation method based on anti-template decoupling, mainly comprising the following steps:

[0007] S1, constructing and training an anti-template generation model of the opinion summary: using contrast learning and a variational autoencoder, taking the opinion as a supervision signal, obtaining the representation of the opinion, taking the average of a plurality of opinion representations for decoding, and preliminarily obtaining the anti-template of the opinion;

[0008] S2, constructing a decoupled text representation opinion summary generation model: the model adds a decoupling module of the text representation in the variational autoencoder, decouples the representation of the opinion summary into content and template, obtains the representation vector of the content, and adds it to the encoder-decoder structure of the unsupervised opinion summary model;

[0009] S3, training an unsupervised opinion summary model: inputting the content representation vector obtained in step S2 into the decoder of the unsupervised opinion summary model, reconstructing the original text through the decoder, and completing the generation of the opinion summary.

[0010] As an improvement of the application, the step S1 specifically comprises:

[0011] S11: constructing a variational autoencoder model, the model is based on a bidirectional long short-term memory network and comprises an encoder and a decoder; inputting the opinion text into the encoder, enabling the encoder to learn the representation of the output text, inputting the representation of the text into the decoder, enabling the decoder to reconstruct the original input text, and minimizing the difference between the output of the decoder and the input of the encoder to enable the model to learn the ability of constructing the text representation;

[0012] S12: Construct a contrastive learning and reconstruct a text loss function so as to calculate the loss function of the variational autoencoder model; the loss function of the model is the sum of the text loss function, the reconstruction loss of the variational autoencoder and the KL divergence constraint of the variational autoencoder;

[0013] S13: Train the variational autoencoder model;

[0014] S14: Input the opinion text into the encoder of the variational autoencoder model trained in step S13 to obtain a text representation, average the text representation to obtain an average representation, obtain an anti-template representation, and obtain an anti-template text after decoding the anti-template representation.

[0015] As an improvement of the present application, in step S11, the encoder uses a bidirectional long short-term memory network, and the decoder uses a unidirectional long short-term memory network.

[0016] As another improvement of the present application, in step S12, the text loss function Specifically,

[0017]

[0018] Wherein, sim is a cosine similarity calculation function, is a text representation in the same batch, n is the number of samples in a batch, τ is a temperature, is an indicator function, and C is a constant.

[0019] The reconstruction loss of the variational autoencoder And the KL divergence constraint of the variational autoencoder Respectively,

[0020]

[0021]

[0022] Wherein, is The probability distribution function of, θ and ψ are respectively the parameters of the encoder and the decoder.

[0023] As another improvement of the present application, the step of decoupling the opinion content representation in step S2 specifically includes:

[0024] S21: Construct an anti-template representation vector And an opinion representation vector z i ;

[0025] S22: Construct an orthogonal decoupling module to decouple the opinion representation vector z i Into the same direction as the anti-template representation And the anti-template representation vector two parts, removing the redundant information in the template representation;

[0026] S23: decoupling the opinion content representation vector.

[0027] As a further improvement of the present application, the step S22 is specifically: i projecting the representation vector z in the direction of p i :

[0028]

[0029] wherein Proj is the projection function:

[0030]

[0031] and then obtaining the content representation vector c orthogonal to z i :

[0032] c i = Proj(z i , (z i -p i )).

[0033] As a further improvement of the present application, the step S3 specifically includes:

[0034] S31: constructing an unsupervised opinion summary model, inputting the content representation vector into the decoder of the model, and reconstructing the original text through the decoder;

[0035] S32: constructing a loss function, wherein the loss function of the unsupervised opinion summary model is the sum of the anti-template reconstruction loss review reconstruction loss and KL divergence constraint ;

[0036] S33: according to the loss function of step S32, completing the model training, optimizing the model, and realizing the generation of the opinion summary.

[0037] As a further improvement of the present application, in the step S32, the review reconstruction loss function is:

[0038]

[0039] wherein φ, is the parameter of the decoder;

[0040] The anti-template reconstruction loss function is:

[0041]

[0042] The KL divergence constraint is specifically:

[0043]

[0044] where θ, φ, are all model learnable parameters.

[0045] To achieve the above object, the technical scheme adopted by the present application is: an unsupervised opinion summary generation system based on anti-template decoupling, comprising a computer program, which, when executed by a processor, implements the steps of any of the above methods.

[0046] Compared with the prior art, the present application has the beneficial effects that:

[0047] (1) The anti-template generator based on variational autoencoder proposed by the present application can extract extensive and redundant "template" information in text information, which is conducive to subsequent decoupling of the text.

[0048] (2) The text content representation extraction method based on orthogonal decoupling adopted by the present application can decouple the content information of the opinion according to the opinion original representation and the anti-template representation, which is conducive to improving the quality and accuracy of the generated summary.

[0049] (3) The present application adds a contrastive learning loss function in the model training of the anti-template generator, so that the model can better learn the "template" information in the comments and improve the performance and robustness of the model. BRIEF DESCRIPTION OF DRAWINGS

[0050] Figure 1 The step flow chart of the unsupervised opinion summary generation method based on anti-template decoupling of the method of the present application;

[0051] Figure 2 The step flow chart of constructing the anti-template text representation in step S1 of the method of the present application;

[0052] Figure 3 The step flow chart of decoupling the opinion content representation by the orthogonal decoupling method in step S2 of the method of the present application;

[0053] Figure 4 The step flow chart of training the unsupervised opinion summary model in step S3 of the method of the present application;

[0054] Figure 5 The calculation schematic diagram of the orthogonal decoupling module in step S2 of the method of the present application. DETAILED DESCRIPTION

[0055] The present application will be further clarified by the following description and embodiments with reference to the accompanying drawings.

[0056] Embodiment 1

[0057] This embodiment takes a product with several pieces of comment information as an example, which needs to be summarized from all the comment information. Based on the unsupervised opinion summary generation method of anti-template decoupling, as shown in Figure 1 , the method mainly includes the following steps:

[0058] S1: Construct and train an anti-template generation model for opinion summary. The model is trained in an unsupervised manner, based on a variational autoencoder (VAE) structure of bidirectional long short-term memory network (Bi-LSTM).

[0059] Anti-template refers to the broad and redundant information contained in the opinion summary data, such as words or sentences like "good review" that do not contain specific information. These information affect the quality of the generated summary and are not conducive to the reading and use of users.

[0060] Variational autoencoder is a deep learning model, usually composed of an encoder-decoder structure. It can be used for data compression and generation. It maps the input data into a latent space and generates new data according to the distribution in the latent space. Variational autoencoder has adaptability and can handle different data types, and can be applied to image, text, speech and other fields. The advantages of this model are that it can realize the functions of generation diversity, data reconstruction, data compression, etc., and can effectively solve the interference problem in the latent variable.

[0061] As shown in Figure 2 , first, an anti-template generation model needs to be trained. Since there is no summary of these opinions in the data set to supervise the model during training, the model is trained in an unsupervised manner. This application uses a variational autoencoder (VAE) structure based on bidirectional long short-term memory network (Bi-LSTM), which consists of an encoder (Encoder) and a decoder (Decoder). During training, the model inputs the comment text into the encoder, expecting the encoder to learn the representation (Embedding) of the input text. Then the text representation is input into the decoder to reconstruct the original input text. By minimizing the difference between the output of the decoder and the input of the encoder, the model learns to construct the text representation. By inputting each comment text into the trained encoder, the representation of the comment in the anti-template generation model is obtained (where i represents the ith comment), and the set of representations of the comments (N is the number of comments) The average value is used to obtain the template representation, which is then input into the decoder to obtain the final anti-template text.

[0062] In a variational autoencoder based on a bidirectional long short-term memory (Bi-LSTM) network, the encoder uses a bidirectional Bi-LSTM network, while the decoder uses a unidirectional Bi-LSTM network. During model training, the text t is... i (where i represents the i-th comment) Input encoder p θ (t i The latent variables (θ being the encoder parameters) are obtained from the input, and then the text t is learned through a linear network. i Distribution After sampling the distribution, the text t can be obtained. i The representation of After obtaining the text representation, a decoder q from a one-way long short-term memory network is used. ψ To reconstruct the input comments.

[0063] Furthermore, to further improve the performance of the anti-template generation model in extracting text representations, this invention uses a loss function based on contrastive learning. Contrastive learning is a method that narrows the distance between positive samples and negative samples in the representation space, where the definition of positive and negative samples depends on the specific task. All text representations in the same batch are processed... Treating each other as positive samples, we construct the loss function accordingly:

[0064]

[0065] Where sim is the cosine similarity calculation function, n is the number of samples in a batch, and τ is the temperature (an adjustable hyperparameter). The function is an indicator function, and C is set to a constant of 1.0.

[0066] In summary, the loss function of the variational autoencoder has three terms:

[0067]

[0068] in For the reconstruction loss of the variational autoencoder, The KL divergence constraints for the variational autoencoder are defined as follows:

[0069]

[0070]

[0071] in for The probability distribution function of the text representation t is usually considered to follow a Gaussian distribution with a mean of 0 and a variance of 1; and θ and ψ are parameters of the encoder and the decoder, respectively.

[0072] After the training of the anti-template generation model is completed, the comments from different products are input into the encoder structure of the model to obtain the text representation of each comment opinion, and the text representations are averaged to obtain an anti-template representation, which contains a general summary pattern (Pattern) from different product information. The general utterance and information are not desired to be seen in the generated summary, and the anti-template text can be obtained by decoding the anti-template representation.

[0073] S2, constructing a decoupled text representation opinion summary generation model: the model adds a decoupled module of the text representation in the variational autoencoder to decouple the representation of the opinion summary into content and template, and obtains the representation vector of the content, which is added to the encoder-decoder structure of the unsupervised opinion summary model; as shown in detail in Figure 3 .

[0074] The decoupled text representation opinion summary generation model is obtained by adding a decoupled module of the text representation in the variational autoencoder of the bidirectional long short-term memory network (Bi-LSTM). During model training, the text t i (input, where i represents the i-th comment) is input into the encoder p θ (t i ) to obtain the hidden variable (θ is the parameter of the encoder), and then a linear network is used to learn the distribution of the text t i representation After sampling the distribution, the representation z i of the text t i is obtained, and the anti-template representation is obtained through a linear network after the text representation.

[0075] After obtaining the anti-template representation , a module based on orthogonal decoupling is introduced, and the representation z i of the comment text after being encoded by the encoder is decoupled into two parts in the same direction as the anti-template representation and perpendicular to the anti-template representation , so as to remove the wide and redundant information in the template representation. Specifically, first, the representation vector z i is projected to the direction to obtain p i :

[0076]

[0077] where Proj is a projection function:

[0078]

[0079] After that, I received Orthogonal content representation vector c after removing redundant information i :

[0080] c i =Proj(z i ,(z i -p i ))

[0081] The calculation diagram of the orthogonal decoupling module is as follows: Figure 5 As shown, Figure 5 The content can be seen to represent c i With template representation p i It is an orthogonal relation, meaning the content represents c. i It does not contain information that overlaps with the template.

[0082] S3, Training the unsupervised opinion summarization model: Input the content representation vector obtained in step S2 into the decoder of the unsupervised opinion summarization model, and reconstruct the original text through the decoder to complete the generation of opinion summarization.

[0083] like Figure 4 As shown, the content representation vector c of the obtained comment information is... i Then, it is input into the decoder structure, and the decoder reconstructs the original text, φ. For the parameters of the decoder, the reconstruction loss function is defined as follows:

[0084]

[0085] By defining the reconstruction loss function as described above, we can guide the text representation z. i and content representation vector c i It contains the content information needed by the user, improving the quality of text generation. The model also obtains an inverse template representation vector. The loss for reconstructing the detemplate text `temp` is defined as follows:

[0086]

[0087] Similar to the process of training the anti-template generator in step S1, the final loss function of the opinion summary generation model mainly includes the template reconstruction loss. Comment reconstruction loss The three parts are: KL divergence constraint, etc.

[0088]

[0089]

[0090] wherein θ, φ, are all model learnable parameters.

[0091] The deep learning model of the method is implemented based on a deep learning framework PyTorch, and when the model is trained and optimized, an Adam optimizer is used, a linear learning rate scheduler is adopted, the initial learning rate is 10 -3 , the iteration number of the model is 8 times, and a Dropout is used as a regularization method to prevent overfitting. In addition, all experiments are performed on an NVIDIA GeForce RTX 3090.

[0092] Test example

[0093] The model of the method is tested on Amazon and Yelp-Res data sets, and the test results are shown in Table 1 and Table 2.

[0094] Table 1 Experimental results on Amazon data set

[0095]

[0096]

[0097] Table 2 Experimental results on Yelp-Res data set

[0098]

[0099] As can be seen from the above table, the performance of the model on the two data sets is better than that of all previous methods.

[0100] In summary, the opinion summary method based on anti-template decoupling text representation is used, the anti-template representation of the generated review representation is trained in the product review information, the template information in the review representation is removed by using the orthogonal decoupling method, so as to decouple the content information required by the user, and then generate the final summary text, remove the redundant information in the template representation, the summary is more accurate, and the work efficiency is higher.

[0101] It should be noted that the above content only illustrates the technical idea of the present application, and cannot be used to limit the protection scope of the present application. For ordinary skilled persons in the art, without departing from the principles of the present application, a number of improvements and refinements can be made, which fall within the protection scope of the claims of the present application.

Claims

1. An unsupervised opinion summarization method based on anti-template decoupling, characterized in that, The method mainly comprises the following steps: S1, constructing and training an anti-template generation model of opinion summary: using contrastive learning and a variational autoencoder, taking opinions as a supervision signal to obtain a representation of the opinions, decoding the average of a plurality of opinion representations to preliminarily obtain an anti-template of the opinions; the anti-template refers to broad and redundant information contained in the opinion summary data, and specifically comprises the following steps: S11: constructing a variational autoencoder model, the model being based on a bidirectional long short-term memory network and comprising an encoder and a decoder; inputting an opinion text into the encoder to enable the encoder to learn a representation of the input text, and inputting the representation of the text into the decoder to enable the decoder to reconstruct the original input text, and minimizing the difference between the output of the decoder and the input of the encoder to enable the model to learn the ability to construct the text representation; S12: constructing a contrastive learning reconstruction text loss function to facilitate calculation of a loss function of the variational autoencoder model; the loss function of the model being a sum of a text loss function, a reconstruction loss of the variational autoencoder and a KL divergence constraint of the variational autoencoder; S13: training the variational autoencoder model; S14: inputting an opinion text into the encoder of the trained variational autoencoder model in step S13 to obtain a text representation, averaging the text representation to obtain an average representation, obtaining an anti-template representation, and decoding the anti-template representation to obtain an anti-template text; S2, constructing a decoupled text representation opinion summary generation model: the model adding a decoupling module of the text representation in the variational autoencoder to decouple the representation of the opinion summary into content and template to obtain a content representation vector, and inputting the content representation vector into an encoder-decoder structure of an unsupervised opinion summary model; S3, training the unsupervised opinion summary model: inputting the content representation vector obtained in step S2 into the decoder of the unsupervised opinion summary model to reconstruct the original text through the decoder to complete generation of the opinion summary.

2. The unsupervised opinion summarization generation method based on anti-template disentanglement according to claim 1, characterized in that: In step S11, the encoder uses a bidirectional long short-term memory network, and the decoder uses a unidirectional long short-term memory network.

3. The unsupervised opinion summarization generation method based on anti-template disentanglement of claim 1, wherein: In the step S12, the text loss function Specifically: ； wherein, is a cosine similarity computation function, is the number of samples in a batch, is temperature, is an indicator function, is a constant; reconstruction loss of the variational autoencoder respectively: ； wherein is a probability distribution function, and are parameters of the encoder and decoder, respectively.

4. The unsupervised summary generation method based on anti-pattern decoupling according to claim 2 or 3, characterized in that: The step of decoupling the opinion content representation in step S2 specifically comprises: S21 : Constructing the anti-template representation vector and opinion representation vector ; S22: Constructing the orthogonal decoupling module, to the opinion representation vector Decoupling into the same direction and the anti-template representation Decoupling into the same direction and the anti-template representation Removing redundant information in the template representation S23: decoupling an opinion content representation vector.

5. The unsupervised opinion summarization generation method based on anti-pattern decoupling according to claim 4, characterized in that: The step S22 is specifically: projecting the representation vector in the direction of to obtain : ； wherein is a projection function: ； Thereafter, the vector of content representation without redundancy information is obtained orthogonal, the vector of content representation without redundancy information : 。 6. The unsupervised opinion summarization generation method based on anti-pattern decoupling according to claim 4, characterized in that: Step S3 specifically comprises: S31: constructing an unsupervised opinion summary model, inputting the content representation vector into the decoder of the model to reconstruct the original text through the decoder; S32: construct a loss function, the loss function of the unsupervised summary model is a reverse template reconstruction loss , review reconstruction loss and KL divergence constraint ; S33: completing model training, optimizing the model and realizing generation of the opinion summary according to the loss function in step S32.

7. The unsupervised summary generation method based on anti-pattern decoupling according to claim 6, characterized in that: In the step S32, the review reconstruction loss function is defined as follows: f = -log (1 - p) ； wherein is a parameter of the decoder; Anti-Template Reconstruction Loss is: ； The KL divergence constraint is specifically: ； wherein, are all model-learnable parameters.

8. An unsupervised opinion summarization system based on anti-pattern decoupling, comprising a computer program, characterized in that: The computer program is executed by the processor to realize the steps of the method in any one of claims 1-7. The computer program is executed by the processor to realize the steps of the method in any one of claims 1-7.

Images