A fake review identification device based on structural attention enhancement mechanism
Through a false comment recognition device based on the structural attention enhancement mechanism, the hierarchical semantic network and the structural attention enhancement mechanism network are used to identify false comments, which solves the problem of false comment recognition, improves the recognition accuracy and model performance, and realizes cross-platform migration use.
Patent Information
- Application Number
- CN202210304001.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-03-25
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2042-03-25
AI Technical Summary
The existing technology is difficult to effectively identify and judge false comments, resulting in a decline in consumer shopping experience, inaccurate market feedback from merchants, reduced platform user stickiness and authenticity of scientific research data.
Using a false comment recognition device based on structural attention enhancement mechanism, through a hierarchical semantic network and structural attention enhancement mechanism network, the word-sentence-segment hierarchical structure and context coherence of comment text is learned to enhance words and sentences. Learn, identify false comments.
It improves the accuracy of false comment recognition and the recognition performance of the model, and can obtain the deep structural semantic expression of text and the correlation quantization matrix of text units, realizes cross-platform migration and use, and improves accuracy, accuracy and recall.
Smart Images

Figure CN114625842B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of false comment detection and recognition, and in particular to a false comment recognition device based on a structural attention enhancement mechanism. Background Art
[0002] Identification of fake reviews is of great significance to eradicating the black and gray industries on the Internet, creating a safe and reliable shopping environment, and providing a pure and reliable data source for scientific research. The vigorous development of the Internet and mobile terminals has changed people's consumption patterns due to the convenience and comfort of e-commerce platforms. Unlike traditional offline physical shopping, when consumers choose products in online browsing, they cannot observe the appearance of the products in person or experience the overall functions of the products in person. They can only obtain product introductions led by the seller from the details page. However, in the product details display page, business merchants will exaggerate the advantages of products and cover up the disadvantages, making it impossible for consumers to effectively obtain real information. Therefore, under this unbalanced shopping balance, the comments of other consumers after the product shopping behavior have become an important source of information for consumers to obtain real information and make purchase decisions. According to the survey, about 81% of users will read reviews before purchasing, and 80% of them believe that the reviews have an impact on their purchasing decisions. Therefore, the content of product reviews has gradually become one of the important reference factors for people to judge the quality of online products.
[0003] As the comments on the platform have become a guide for consumers to shop, people are increasingly relying on the authenticity and reliability of the comments. However, what follows are intentionally misleading distorted comments, invalid comments that are written to get points, and some merchants using fake orders to impersonate users to post exaggerated comments or post defamatory comments under competing products, i.e., false comments. From the perspective of the entire transaction process, buyers experience a decline due to misleading shopping; sellers are unable to obtain accurate market feedback, which affects marketing strategies and increases returns and exchanges; the platform reduces user stickiness and market competitiveness; and data distortion also brings data authenticity and cleaning problems to scientific research. Therefore, judging and identifying false comments has become a task that needs to be solved from many perspectives. Summary of the invention
[0004] The purpose of the present invention is to overcome the shortcomings of the prior art and provide a false review identification device based on a structural attention enhancement mechanism, which can judge and identify false reviews.
[0005] The purpose of the present invention is achieved through the following technical solutions: a false comment identification device based on a structural attention enhancement mechanism, which includes a hierarchical semantic network for hierarchical staged text representation learning, and a structural attention enhancement mechanism network embedded in the hierarchical semantic network to enhance the representation learning of words and sentences under the text structure;
[0006] The hierarchical semantic network uses the pre-trained word vector as the input layer, and learns the text representation of the review text word-sentence-paragraph through the hierarchical neural network. The word-sentence layer uses the word embedding feature to learn the sentence representation of the review text to complete the modeling at the word level. The sentence-paragraph layer generates the overall paragraph representation of the review text through the sentence vector of the word-sentence layer to complete the modeling at the discourse level.
[0007] The structural attention enhancement mechanism network is used to learn the context coherence matrix and enhance the representation of the free word order of text structure units that are not related to the context.
[0008] The hierarchical semantic network includes an Embedding layer, a GRU layer and a Pooling layer; the Embedding layer is used to represent each word in a sentence as a fixed word vector with the same dimension; the GRU layer is used to perform vector concatenation on the word learning vectors of the forward input sequence and the word learning vectors of the backward input sequence to obtain the encoding vector of the word; the Pooling layer performs an average operation on all words after the semantic enhancement of the structural attention enhancement mechanism network through average pooling.
[0009] The structural attention enhancement mechanism network is embedded between the GRU layer and the Pooling layer, and is used to learn the context coherence matrix and enhance the representation of the free word order of the text structure unit without contextual relationship.
[0010] The steps of implementing the hierarchical semantic network specifically include:
[0011] Given a document d = [d1, d2, ..., d n ], each sentence s i All consist of m words [w i1 ,w i2 ,…,w im ]constitute;
[0012] The Embedding layer obtains the vectorized dictionary of words by training the unlabeled comment text corpus through the Word2vec model, and obtains the word w ij Represented as w ij =W E ·I[w ij ], where W E is the word vector matrix obtained through pre-training, and I[·] represents the word w queried through the dictionary library ij The index of
[0013] The GRU layer passes Get the word learning vector and Get the word learning vector of the backward input sequence, and pass Perform vector concatenation to obtain the word encoding vector w ij ,in, Represents the word learning vector of the forward input sequence, Word learning vectors representing the backward input sequence;
[0014] Pooling layer Perform an average operation on all the semantically enhanced words.
[0015] The structural attention enhancement mechanism network includes a unit dependency learning module and a unit semantic enhancement module; the unit dependency learning module is used to calculate the word-level and sentence-level dependency matrices, and construct a discontinuous text structure under free word order in a long distance or without context; the unit semantic enhancement module is used to learn and allocate the quantization matrix between words by constructing a grammatical tree, and generate hierarchical context information for texts with weak structural information.
[0016] The specific steps of implementing the unit dependency learning module include:
[0017] The sentence s output by each step of the GRU layer obtained by the semantic extraction layer of the word i =[w i1 ,w i2 ,…,w ij ,…w im ] as a semantic expression of words k represents the GRU hidden layer dimension;
[0018] Learning the inter-word attention matrix A∈R via the structural attention mechanism m×m Get the dependencies between words and use u ip =φ(W p ·w im +b p )、u iq =φ(W c ·w in +b q )and Learn the matrix A, where A pq Indicates w ip and w iq The parent node probability (1≤p,q≤m,p≠q), φ is a nonlinear activation function, W p ∈R m×m and W c ∈R m×m are the parent and child node transformation mapping matrices, b f and b c is the corresponding bias value;
[0019] By u j =φ(wr ·w ij +b r )and Calculate the probability of the root node, where W r ∈R 1×k is the root node mapping matrix, and the probability of all words in the sentence being the root node is calculated as
[0020] The specific steps implemented by the unit semantic enhancement module include:
[0021] Divide the semantic vector output by the GRU layer into two parts in represents text dependency learning, The word indicates the update semantics;
[0022] pass and Calculate the words in the structure The non-linear context representation vector of all parent and child words in the sentence, Represents the word embedding vector as the root node in the document tree structure.
[0023] The present invention has the following advantages: a false comment identification device based on a structural attention enhancement mechanism focuses on feature extraction of the hierarchical structure of words, sentences and paragraphs of the comment text, and embeds the structural attention enhancement mechanism in the hierarchical representation to enhance the nonlinear semantic expression of weak structural units; compared with previous methods, it can not only obtain the deep structural semantic expression of the text to improve the recognition performance of the model, but also obtain the correlation quantization matrix of the text unit, which is used for the structural induction generation of the text without the help of a third-party analysis platform and additional auxiliary corpus, and can realize the cross-platform migration and use capability. BRIEF DESCRIPTION OF THE DRAWINGS
[0024] Figure 1 Figure 2. Hierarchical network model diagram for embedding structure attention enhancement;
[0025] Figure 2 Flowchart of feature extraction for hierarchical semantic networks;
[0026] Figure 3 Schematic diagram of word-level attention mechanism;
[0027] Figure 4 Schematic diagram of sentence-level attention mechanism;
[0028] Figure 5 Updated flowchart for word enhancement. DETAILED DESCRIPTION
[0029] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the detailed description of the embodiments of the present application provided below in conjunction with the drawings is not intended to limit the scope of protection of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application. The present invention is further described below in conjunction with the drawings.
[0030] The present invention mainly focuses on the 'word-sentence-paragraph' text representation learning of the review text, and embeds the structural attention enhancement mechanism in the representation of the text structure unit at the sentence level and document level respectively, for the enhanced representation learning of the non-linear relationship of the context.
[0031] like Figure 1 As shown, the present invention mainly includes a hierarchical semantic network for hierarchical and staged text representation learning and a structural attention enhancement mechanism for enhanced representation learning of words and sentences under text structure. The left network layer uses the pre-trained word vector as the input layer, and learns the text representation of the comment text word-sentence-paragraph through the hierarchical neural network (HNN). The word-sentence layer mainly uses the word embedding feature to learn the sentence representation of the comment text to complete the modeling at the word level; the sentence-paragraph layer generates the overall paragraph representation of the comment text through the sentence vector of the previous layer to complete the modeling at the discourse level. The embedded structure attention mechanism (StructureAttention Enpower Mechanism) is used to learn the coherence matrix of the context and enhance the representation of the free word order of the text structure units (words and sentences) that are not related to the context. .
[0032] like Figure 2 As shown, given a document d = [d1, d2, ..., d n ], for each sentence s i , consisting of m words [w i1 ,w i2 ,...,w im ] The hierarchical semantic network is designed to describe the hierarchical information of the document's 'word-sentence-segment'. It is mainly composed of the Embedding layer, the GRU layer, the Structure-Attention layer (structure attention enhancement mechanism layer) and the Pooling layer.
[0033] The first is the Embedding layer, which represents each word in the sentence as a fixed word vector with the same dimension. First, the Word2vec model is used to train the unlabeled comment text corpus to obtain the vectorized dictionary of words. ij The expression is:
[0034] w ij =W E ·I[w ij ]
[0035] Where i, j represents the jth word in the i-th sentence in the document, W E is the word vector matrix obtained through pre-training, and I[·] represents the word w queried through the dictionary library ij The index of .
[0036] Then comes the GRU layer. Traditional RNNs will have problems with gradient diffusion and gradient explosion when processing long text input sequences. LSTM and GRU can select historical information of the previous time step through the "gate" control structure to alleviate problems such as the gradient being dominated by short distances in back propagation. Compared with LSTM, GRU has a simpler "gate" structure design and faster training calculation speed.
[0037]
[0038]
[0039]
[0040] in, Represents the word learning vector of the forward input sequence, Represents the word learning vector of the backward input sequence, and the encoding vector w of the word is obtained by vector concatenation ij .
[0041] Finally, for the Pooling layer, in order to obtain the semantic representation of the entire sentence, average pooling is selected to operate on all words after Structure-Attention semantic enhancement:
[0042]
[0043] The process of document-level modeling is roughly the same as that of the sentence-level modeling.
[0044] Different from the global attention mechanism to select words or sentences that contribute to the classification results. Based on the field of fake reviews, the present invention proposes a structural attention mechanism that can learn context-rich structural dependencies. There are three main reasons: First, it is not difficult to find from the current research that the connection between the basic structural units of words and sentences that make up a document can reflect the authenticity of the document; second, the texts edited and fabricated by fake reviewers by browsing real reviews lack the consistency of the context, so by learning the correlation between the context, the consistency of the review text can be studied; third, the use of hierarchical structures can learn deep document semantic expressions, and by capturing the interactive information of the document unit structure, context representations are generated for words or sentences that contribute highly to classification but lack context information, thereby enhancing their semantic expression.
[0045] In the dependency analysis of text units, the main focus is to identify the dependency of one unit on another unit. Figure 3 As shown in Figure 1, the present invention captures the interaction between words in a sentence and generates a context representation for each word with weak structural information to learn a deeper structural semantic representation. The GRU output s at each step obtained at the semantic extraction layer of the word i =[w i1 ,w i2 ,...,w ij ,...w im ] as a semantic expression of words, k represents the GRU hidden layer dimension, and R represents the real number space.
[0046] In order to obtain the dependency relationship between words, the present invention uses the structural attention mechanism to learn the inter-word attention matrix A∈R m×m , A pq Indicates w ip and w iq The parent node probability of (1≤p,q≤m,p≠q). The learning representation process of matrix A is as follows:
[0047] u ip =φ(W p ·w im +b p )
[0048] u iq =φ(W c ·w in +b q )
[0049]
[0050] Among them, φ is a nonlinear activation function, W p ∈R m×m and W c ∈Rm×m are the parent and child node transformation mapping matrices, b f and b c is the corresponding bias value.
[0051] Since the post-order semantic enhancement representation needs to calculate the root node of the tree structure, the probability of the root node is defined by the following formula:
[0052] u j =φ(w r ·w ij +b r )
[0053]
[0054] Among them, W r ∈R 1×k , as the root node mapping matrix. The probability of all words in the sentence being the root node is calculated as
[0055] The main advantage of text represented by dependency relations is that it can construct discontinuous text structures through free word order in long distances or without context. By building a grammar tree, the quantization matrix between words is learned and allocated, thereby generating hierarchical context information for text with weak structural information.
[0056] like Figure 4 As shown, the semantic vector output by GRU is divided into two parts:
[0057]
[0058] in Used for text dependency learning to obtain the contextual nonlinear dependency matrix A and root probability vector Used to update semantic representation of words.
[0059]
[0060]
[0061]
[0062] in and The words in the structure The non-linear context representation vector of all possible parent-child words in the sentence, Represents the word embedding vector as the root node in the document tree structure.
[0063] Through the context nonlinear dependency matrix A and the root probability vector r, the semantics of the current word is supplemented with the context non-free word order. It is used to enhance words that are highly important to the classification results but have weak semantic expression.
[0064] The present invention automatically learns richer structural dependencies at the same time. The key idea is to capture the interaction between words and sentences, and to provide an enhanced representation of non-context word order for each word and sentence with weak structural information. Relying on the output of each time step of the GRU processing text word order, an l×l structural attention matrix A and a text structure tree root node probability vector r are established. For the output of each time step, the semantic supplement of its nonlinear free word order is supplemented by the values of A and r, and they are merged into the semantic representation of the current step. In this way, for those words or sentences that have recognition significance for the final classification layer, their semantic representation can be enhanced in the case of under-learning of network semantics, so that they can participate in the feature vector construction of the final text classification layer. In the comparative experiment with accuracy, precision, recall and F1 comprehensive evaluation index (F1-Score) as evaluation indicators, according to the comparative experimental results of the data set and the existing methods, there is a 2% improvement in multiple indicators.
[0065] The above description is only a preferred embodiment of the present invention. It should be understood that the present invention is not limited to the form disclosed in the present invention, and should not be regarded as excluding other embodiments, but can be used in various other combinations, modifications and environments, and can be modified within the scope of the concept described in the present invention through the above teachings or the technology or knowledge in the relevant field. The changes and modifications made by those skilled in the art do not deviate from the spirit and scope of the present invention, and should be within the scope of protection of the claims attached to the present invention.
Claims
1. A false review identification device based on structural attention enhancement mechanism, characterized by: It includes a hierarchical semantic network for hierarchical and staged text representation learning, and a structural attention enhancement mechanism network embedded in the hierarchical semantic network to enhance the representation learning of words and sentences under the text structure; The hierarchical semantic network uses the pre-trained word vector as the input layer, and learns the text representation of the review text word-sentence-paragraph through the hierarchical neural network. The word-sentence layer uses the word embedding feature to learn the sentence representation of the review text to complete the modeling at the word level. The sentence-paragraph layer generates the overall paragraph representation of the review text through the sentence vector of the word-sentence layer to complete the modeling at the discourse level. The structural attention enhancement mechanism network is used to learn the context coherence matrix and enhance the representation of the free word order of text structure units without contextual relationship; The specific steps of implementing the hierarchical semantic network include: Given a document d = [d1, d2, ..., d n ], each sentence s i All consist of m words [w i1 ,w i2 ,...,w im ]constitute; The Embedding layer obtains the vectorized dictionary of words by training the unlabeled comment text corpus through the Word2vec model, and obtains the word w ij Represented as w ij =W E ·I[w ij ], where W E is the word vector matrix obtained through pre-training, and I[·] represents the word w queried through the dictionary library ij The index of The GRU layer passes Get the word learning vector and Get the word learning vector of the backward input sequence, and pass Perform vector concatenation to obtain the word encoding vector w ij ,in, Represents the word learning vector of the forward input sequence, Word learning vectors representing the backward input sequence; Pooling layer Perform an average operation on all the semantically enhanced words; The structural attention enhancement mechanism network includes a unit dependency learning module and a unit semantic enhancement module; the unit dependency learning module is used to calculate the word-level and sentence-level dependency matrices, and construct a discontinuous text structure under free word order in a long distance or without context; the unit semantic enhancement module is used to learn and allocate the quantization matrix between words by constructing a grammatical tree, and generate hierarchical context information for texts with weak structural information.
2. According to claim 1, a false review identification device based on structural attention enhancement mechanism is characterized by: The hierarchical semantic network includes an Embedding layer, a GRU layer and a Pooling layer; the Embedding layer is used to represent each word in a sentence as a fixed word vector with the same dimension; the GRU layer is used to perform vector concatenation on the word learning vectors of the forward input sequence and the word learning vectors of the backward input sequence to obtain the encoding vector of the word; the Pooling layer performs an average operation on all words after the semantic enhancement of the structural attention enhancement mechanism network through average pooling.
3. The false review identification device based on structural attention enhancement mechanism according to claim 2 is characterized by: The structural attention enhancement mechanism network is embedded between the GRU layer and the Pooling layer, and is used to learn the context coherence matrix and enhance the representation of the free word order of the text structure unit without contextual relationship.
4. According to claim 1, a false review identification device based on structural attention enhancement mechanism is characterized by: The specific steps of implementing the unit dependency learning module include: The sentence s output by each step of the GRU layer obtained by the semantic extraction layer of the word i =[w i1 ,w i2 ,...,w ij ,…w im ] as a semantic expression of words k represents the GRU hidden layer dimension; Learning the inter-word attention matrix A∈R via the structural attention mechanism m×m Get the dependencies between words and use u ip =φ(W p ·w im +b p )、u iq =φ(W c ·w in +b q )and Learn the matrix A, where A pq Indicates w ip and w iq The parent node probability (1≤p,q≤m,p≠q), φ is a nonlinear activation function, W p ∈R m×m and W c ∈R m×m are the parent and child node transformation mapping matrices, b f and b c is the corresponding bias value; By u j =φ(w r ·w ij +b r )and Calculate the probability of the root node, where W r ∈R 1×k is the root node mapping matrix, and the probability of all words in the sentence being the root node is calculated as 5. The false review identification device based on structural attention enhancement mechanism according to claim 1 is characterized by: The specific steps implemented by the unit semantic enhancement module include: Divide the semantic vector output by the GRU layer into two parts in represents text dependency learning, The word indicates the update semantics; pass and Calculate the words in the structure The non-linear context representation vector of all parent and child words in the sentence, Represents the word embedding vector as the root node in the document tree structure.
Citation Information
Patent Citations
Comment text emotion classification model training and emotion classification method and device and equipment
CN108363753A
Text abstract generation method based on sentence association attention mechanism
CN110348016A
