An author attribution method that maximizes syntactic information by combining multiple attention mechanisms

Abstract

This invention discloses a method for author attribution that maximizes syntactic information by combining multiple attention mechanisms. The method includes acquiring text information data of unknown authors, preprocessing the text information data to obtain raw character data and raw syntactic data, and embedding them into character embedding vectors and tag embedding vectors respectively; extracting character 2-gram feature maps from the character embedding vectors and constructing content feature vectors using max pooling; extracting syntactic information features from the tag embedding vectors and calculating attention coefficients between different words to construct syntactic feature vectors; concatenating the content feature vectors and syntactic feature vectors to obtain text style features; and predicting the author attribution of the text information data based on the text style features. This invention can improve the classification accuracy and performance of anonymous text.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology, and more specifically to an author attribution method that combines multiple attention mechanisms to maximize syntactic information. Background Technology

[0002] Author attribution refers to identifying the author of an unknown text among a set of possible authors, with applications ranging from plagiarism detection to forensic linguistics. The key concept behind statistical author attribution is that measuring textual features can distinguish texts written by different authors. These features include indicators of content differences between authors, such as bag-of-words, and stylistic features that reflect a writer's unique writing patterns, such as the use of punctuation, emojis, and whitespace.

[0003] The emergence of social media as a primary mode of communication has challenged the traditional assumption that forensic investigations have access to lengthy texts (i.e., letters and emails). Recent approaches have focused on large-scale author attribution of thousands of authors, but this is costly in the predictive process, a drawback in online scenarios. Furthermore, in addition to stylistic information, word-level thematic information has also proven relevant to author identification.

[0004] Deep learning research has largely neglected author attribution, instead focusing on shaping author style. Kiros et al. conditionally conditionalized word embeddings based on attributes such as style and predicted the author's age, gender, and industry. Zhu et al. converted image descriptions into book sentences by subtracting "style." However, these efforts all focused on shaping author style and neglected author attribution. State-of-the-art author attribution algorithms, in practical applications, must handle potentially thousands of candidate authors and a limited number of examples per author, requiring CPU days to predict feature subsets when calculating pairwise distances between them. Meanwhile, character n-grams have proven to be the most successful feature, but have not been explored in depth. Potthast et al. compared traditional methods on small datasets but did not delve into the role of deep learning methods.

[0005] With the development of deep learning, more and more neural network models are being applied to author attribution tasks. Ruder et al. applied convolutional neural networks to large-scale author attribution, aiming to identify the author of an unknown text among many candidate authors. Their motivation was the ability to process character-level signals and distinguish a large number of categories, while making fast predictions compared to state-of-the-art methods. They conducted extensive evaluations of convolutional neural network-based methods that utilized word and character channels, comparing them with state-of-the-art methods and offering new insights into traditional approaches. However, they only used character features and not character n-gram features. Shrestha et al. proposed a model that uses convolutional neural networks for author attribution of tweets based on character n-grams. They also proposed a strategy to improve the interpretability of the model by determining the importance of the input text fragment in the predicted classification. Experimental evaluations showed that the text convolutional neural network performed competitively, outperforming previous methods. They used character n-gram features, but only a single feature, which cannot fully represent writing style. Hu et al. proposed a novel embedding-based framework called DeepStyle, which uses multi-view representations of user posts and a ternary loss objective function to learn users' writing styles. It then uses an aggregation function to merge individual user post embeddings into user style embeddings. They argue that different users' style embeddings should be widely spaced in the embedding space to distinguish their writing styles. While DeepStyle extracts various features, it only uses shallow features and does not explore deeper ones. Zhang et al. proposed a Syntax-CNN model that encodes the parse tree of a sentence into a learnable distributed representation. Their proposed parse encoding scheme is proven to be lossless. Specifically, an embedding vector is constructed for each word in a sentence, and the path in the parse tree corresponding to that word is encoded. The Syntax-CNN model uses a syntactic component tree to extract syntactic features from the sentence, addressing the problem of most methods using shallow features. However, when dealing with author attribution of short texts, the length of the syntactic path affects the quality of the extracted syntactic features. Furthermore, most current methods only use attention mechanisms as a small part of the model and do not delve into the role of attention mechanisms in author attribution.

[0006] In summary, current methods suffer from problems such as using a single feature, using multiple shallow features, or the extraction of deep features being easily affected. Summary of the Invention

[0007] To address the aforementioned shortcomings in the prior art, this invention provides a method for author attribution that combines multiple attention mechanisms to maximize syntactic information.

[0008] To achieve the above-mentioned objectives, the technical solution adopted by this invention is as follows:

[0009] An author attribution method that combines multiple attention mechanisms to maximize syntactic information includes the following steps:

[0010] S1. Obtain text information data from unknown authors, preprocess the text information data to obtain raw character data and raw syntax data, and embed them into character embedding vectors and tag embedding vectors respectively.

[0011] S2. Extract character 2-gram feature maps from character embedding vectors and construct content feature vectors using the max pooling method;

[0012] S3. Extract syntactic information features from the label embedding vectors, and calculate the attention coefficients between different words to construct syntactic feature vectors;

[0013] S4. Concatenate the content feature vector and the syntactic feature vector to obtain the text style feature, and predict the author attribution of the text information data based on the text style feature.

[0014] Optionally, step S2 specifically includes:

[0015] A self-attention layer is used to obtain the attention coefficients of all vectors in the character embedding vector, resulting in a character feature vector with added attention coefficients;

[0016] Multiple parallel single-layer convolutional neural networks are used to extract character 2-gram feature maps from character feature vectors with added attention coefficients;

[0017] The maximum value of the feature map is selected from the 2-gram feature maps of each character using the max pooling method, and the selected feature maps are concatenated to construct the content feature vector.

[0018] Optionally, a self-attention layer is used to obtain the attention coefficients of all vectors in the character embedding vector, resulting in a character feature vector with added attention coefficients, expressed as:

[0019]

[0020] Among them, h i1 For the i1th character feature vector to which attention coefficients are added, k j1 ,q j1 ,v j1 represents the attention coefficients of all vectors in the character embedding vector, () represents the fully connected operation, and N is the total number of vectors in the character embedding vector.

[0021] Optionally, the step of using multiple parallel single-layer convolutional neural networks to extract character 2-gram feature maps from character feature vectors with added attention coefficients is represented as follows:

[0022]

[0023] Among them, f i2 For 2-gram feature maps, W i2 For the filtering matrix, For convolution operations, X i2 For character feature vectors with attention coefficients added, i2 is the vector index, and w i2 b is the window size of the filtering matrix. i2 This is the bias vector.

[0024] Optionally, step S3 specifically includes:

[0025] A syntactic component tree is constructed based on text information data, and the syntactic component tree is encoded into a distributed representation vector.

[0026] A syntactic dependency tree is constructed based on the dependency relationships of words in the text information data. Each node in the syntactic dependency tree is numbered sequentially, and the structural features of the syntactic dependency tree are extracted to obtain the adjacency matrix of the words.

[0027] The label embedding vector, distributed representation vector, and word adjacency matrix are concatenated, and multiple parallel single-layer convolutional neural networks are used to extract word embedding vectors containing syntactic features.

[0028] A hierarchical attention network is used to calculate the label attention coefficient in the word embedding vector of each syntactic feature, and the corresponding label attention coefficient is added to the word embedding vector of each syntactic feature to obtain the final word embedding vector.

[0029] Bidirectional long short-term memory is used to extract syntactic information from word embedding vectors, and graph attention network is used to calculate the attention coefficients of different neighboring nodes in the syntactic component tree to obtain node feature vectors;

[0030] Add all the node feature vectors together to obtain the syntactic feature vector.

[0031] Optionally, the layered attention network is used to calculate the label attention coefficients in the word embedding vectors of each syntactic feature, and the corresponding label attention coefficients are added to the word embedding vectors of each syntactic feature to obtain the final word embedding vectors. Specifically, this includes:

[0032] The label vectors embedded in the word embedding vectors of each syntactic feature are transformed and represented as follows:

[0033] u o1 =tanh(Wi3 h o1 +b i3 )

[0034] Among them, u o1 For the o1th transformed label vector, W i3 b i3 To randomly initialize parameters, h o1 Let be the o1th label vector;

[0035] A hierarchical attention network is used to calculate the label attention coefficients of the label vectors, denoted as:

[0036]

[0037] Among them, a o2 Let u be the attention coefficient for the o2th label. o1 Let u be the o1th transformed label vector. l2 Let T be the 12th transformed label vector, and let T be the transpose symbol.

[0038] Add the corresponding label attention coefficient to the word embedding vector of each syntactic feature to obtain the final word embedding vector, as follows:

[0039]

[0040] Among them, w o3 For the final embedding vector of the o3th word, h l3 This is the embedding vector for the l3rd word.

[0041] Optionally, the step of extracting syntactic information from word embedding vectors using bidirectional long short-term memory and calculating attention coefficients of different neighboring nodes in the syntactic component tree using a graph attention network to obtain node feature vectors specifically includes:

[0042] Syntactic information is extracted from word embedding vectors using bidirectional long short-term memory, and represented as follows:

[0043] h o4 =[LSTM F (w o4 ):LSTM B (w o4 )]

[0044] Among them, h o4 For the extracted 4th syntactic information, LSTM F For the forward propagation operation, LSTM B For backpropagation, : represents the join operation, w o4 Embed the vector for the final o4th word;

[0045] The attention coefficients of different neighboring nodes in the syntactic constituent tree are calculated using a graph attention network, and are expressed as follows:

[0046]

[0047] Among them, a o4 W represents the attention coefficients of different adjacent nodes in the syntactic component tree. i4 For learnable parameters, a T For a fully connected layer, h o4 For the 4th syntactic information, h j4 This is the j4th syntactic information;

[0048] Based on the extracted syntactic information and the calculated attention coefficient, the node feature vector is calculated, represented as follows:

[0049] p o5 =σ(∑a o5 W i5 h j5 )

[0050] Where, p o5 Let h be the feature vector of the 5th node, σ be the nonlinear activation function, and h be the feature vector of the 5th node. j5 This is the j5th syntactic information.

[0051] Optionally, the step of concatenating content feature vectors and syntactic feature vectors to obtain text style features, and predicting the author attribution of text information data based on text style features, specifically includes:

[0052] The text style feature V is obtained by concatenating the content feature vector e and the syntactic feature vector s. s ;

[0053] Using a classification network, the probability distribution of text attribution to each author is calculated based on text style features, and is represented as follows:

[0054] y k1 =softmax(W i6 V s +b i6 )

[0055] Among them, y k1 Let W be the probability distribution of the text belonging to the k1-th author. i6 and b i6 Initialize parameters randomly;

[0056] The author with the highest probability is selected as the predicted author of the text.

[0057] The present invention has the following beneficial effects:

[0058] (1) This invention uses syntactic dependency tree and syntactic component tree to extract the syntactic features of sentences. These two syntactic trees can compensate for each other's shortcomings, and at the same time, the structural features of the tree are used as an auxiliary feature to improve the classification accuracy.

[0059] (2) This invention combines self-attention, graph attention and text hierarchical attention to enhance the importance of different features to author style features, thereby improving the classification effect. Attached Figure Description

[0060] Figure 1 This is a flowchart illustrating a method for maximizing syntactic information by combining multiple attention mechanisms in an embodiment of the present invention.

[0061] Figure 2 This is a schematic diagram of the network model for steps S2 to S4 in an embodiment of the present invention;

[0062] Figure 3 Here is an example of the syntactic component tree for “There is no absolute claim.” in the embodiments of the invention, where “NP” represents a noun phrase, “VP” represents a verb phrase, “.” represents a punctuation mark, “EX” represents a pronoun, “VBP” represents a copula, “DT” represents a determiner phrase, “JJ” represents an adjective, and “NNS” represents a noun;

[0063] Figure 4 This is an example of the syntactic dependency tree for "There is no absolute claim." in an embodiment of the invention. The arc represents the dependency function between two words, where "expl" represents specific reference, "nusbj" represents subject-verb relationship, "punct" represents punctuation, "det" represents determiner, and "amod" represents adjective modification.

[0064] Figure 5 This is an adjacency matrix representing the word dependencies between the phrase "There is no absolute claim." in the embodiment of the invention, where a value of 1 indicates that two words have a dependency relationship. Figure 2 The two words in the text contain an arc, and the value represents that they do not exist;

[0065] Figure 6 The following are examples of the syntactic component tree structure features of "There is no absolute claim." in the embodiments of the invention, numbered sequentially from top to bottom and from left to right. Detailed Implementation

[0066] The specific embodiments of the present invention are described below to enable those skilled in the art to understand the present invention. However, it should be understood that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the present invention as defined and determined by the appended claims. All inventions utilizing the concept of the present invention are protected.

[0067] like Figure 1 As shown, this embodiment of the invention provides a method for maximizing author attribution of syntactic information by combining multiple attentions, including the following steps S1 to S4:

[0068] S1. Obtain text information data from unknown authors, preprocess the text information data to obtain raw character data and raw syntax data, and embed them into character embedding vectors and tag embedding vectors respectively.

[0069] In an optional embodiment of the present invention, the present invention first obtains text information data of unknown authors, then preprocesses the text information data, the preprocessing operation including processing the original text data into character raw data and syntax raw data; then embedding them respectively to obtain character embedding vectors X = [x1,...,x...]. n ] and label embedding vector L = [l1,...,l m ], where n and m are the number of characters and tags, respectively, and the embedding vector is randomly initialized as a one-dimensional matrix.

[0070] S2. Extract character 2-gram feature maps from character embedding vectors and construct content feature vectors using the max pooling method;

[0071] In an optional embodiment of the present invention, such as Figure 2 As shown, step S2 specifically includes the following sub-steps:

[0072] S2-1. Use a self-attention layer to obtain the attention coefficients of all vectors in the character embedding vector, and obtain the character feature vector with added attention coefficients;

[0073] In this step, because convolutional neural networks can only extract context within a fixed window size when extracting content features, they often struggle to attend to context outside the window, especially as sequence length increases. To address this issue, this invention first uses a self-attention layer before inputting the character embedding vector into the convolutional network layer to obtain a context-rich vector that assigns different importance to different parts. Self-attention allows all vectors in the sequence to receive attention coefficients, thus solving the problem that convolutional neural networks can only focus on a fixed window size. The formula for calculating self-attention is as follows:

[0074] Q = W Q X,

[0075] K2 = W K2 X,

[0076] V = W V X

[0077] Where X is a character input vector obtained through random initialization, and W... Q W k2 and W V The parameter matrix is ​​randomly initialized for learnable purposes, and Q, K2, and V are query vectors used to obtain attention coefficients.

[0078] Calculate the attention coefficients and obtain the output vector, denoted as:

[0079]

[0080] Among them, h i1 For the i1th character feature vector to which attention coefficients are added, k j1 ,q j1 ,v j1 represents the attention coefficients of all vectors in the character embedding vector, j1 is the j1-th vector, s() is the fully connected operation, and N is the total number of vectors in the character embedding vector.

[0081] S2-2, Multiple parallel single-layer convolutional neural networks are used to extract character 2-gram feature maps from character feature vectors with added attention coefficients;

[0082] In this step, since convolutional neural networks can obtain local contextual information from the input feature vector, changes in the window size can alter the information obtained after convolution. This module uses multiple parallel single-layer convolutional neural networks to extract character 2-gram feature maps, represented as follows:

[0083]

[0084] Among them, f i2 For 2-gram feature maps, W i2 For the filtering matrix, For convolution operations, X i2 For character feature vectors with attention coefficients added, i2 is the vector index, and w i2 b is the window size of the filtering matrix. i2 This is the bias vector.

[0085] S2-3. Use the max pooling method to select the maximum value of the feature map from the 2-grams feature map of each character, and concatenate the selected feature maps to construct the content feature vector.

[0086] In this step, multiple convolutional kernels will acquire input matrix information from different perspectives. Therefore, the information in each feature map is different. To obtain the most relevant information from each feature map, this invention uses max pooling to select the maximum value of each character 2-gram feature map, denoted as...

[0087] e m3 =max(f(m3)),m3=1…j6

[0088] Where j6 is the number of feature maps, e m3 It is the maximum value of the m3th feature map.

[0089] This invention uses e m3 The vector representation of the input sequence is represented by a concatenation of elements, denoted as e, which is the content feature vector.

[0090] S3. Extract syntactic information features from the label embedding vectors, and calculate the attention coefficients between different words to construct syntactic feature vectors;

[0091] In an optional embodiment of the present invention, such as Figure 2 As shown, step S3 specifically includes:

[0092] A syntactic component tree is constructed based on text information data, and the syntactic component tree is encoded into a distributed representation vector.

[0093] A syntactic dependency tree is constructed based on the dependency relationships of words in the text information data. Each node in the syntactic dependency tree is numbered sequentially, and the structural features of the syntactic dependency tree are extracted to obtain the adjacency matrix of the words.

[0094] The label embedding vector, distributed representation vector, and word adjacency matrix are concatenated, and multiple parallel single-layer convolutional neural networks are used to extract word embedding vectors containing syntactic features.

[0095] A hierarchical attention network is used to calculate the label attention coefficient in the word embedding vector of each syntactic feature, and the corresponding label attention coefficient is added to the word embedding vector of each syntactic feature to obtain the final word embedding vector.

[0096] Bidirectional long short-term memory is used to extract syntactic information from word embedding vectors, and graph attention network is used to calculate the attention coefficients of different neighboring nodes in the syntactic component tree to obtain node feature vectors;

[0097] Add all the node feature vectors together to obtain the syntactic feature vector.

[0098] Specifically, for syntactic features, this invention designs a syntactic feature extraction module. The syntactic structure of a given sentence can be uniquely represented by a tree, which this invention calls a syntactic component tree. The syntactic component tree is a phrase structure syntactic tree of a sentence. Figure 3 An example of this component tree is given. Figure 3 As shown, each word has a unique syntactic path containing syntactic labels. This invention encodes the sentence's parse tree into a learnable distributed representation and uses three parallel single-layer convolutional neural networks to obtain word embedding vectors containing more syntactic features. However, for short texts, the depth of the syntactic paths is relatively small, meaning the word embedding vectors contain fewer syntactic features.

[0099] To address the aforementioned problems, this invention uses a sentence dependency tree to extract sentence structural information. A sentence dependency tree is a syntactic representation that shows the grammatical relationships between words. Figure 4 Examples of inter-word dependencies are given. Figure 4 As shown, dependency structures are mostly interconnected and have a designated root node; therefore, a dependency tree is a directed acyclic graph. Structural features of a sentence can be extracted based on the dependencies between words. These features are rarely affected by sentence length but are often related to the author's writing style. An adjacency matrix can be obtained from the dependencies between words, such as... Figure 5 As shown, a value of 1 indicates a dependency relationship between two words. This invention uses graph convolutional networks to extract sentence structural information, and the calculation formula is as follows:

[0100]

[0101] in and These are the weight matrix and bias, respectively. N(v) are the neighbors of v, ReLU is the activation function of the rectifier linear unit, and matrix X is... i7 Given an input matrix, each column x i7 This is for encoding node features.

[0102] Similar to standard convolutional networks, stacking GCN layers can merge neighborhoods to a higher degree, as calculated in the following formula:

[0103]

[0104] Where k1 represents the number of layers.

[0105] However, the dependency tree for sentences also has a drawback: the word embedding vectors in a sentence do not contain syntactic features. In a component tree, words are independent and have no relation to each other. While the dependency tree represents the syntactic relationships between words, the words in the dependency tree do not contain syntactic features. Therefore, this invention proposes a joint component tree and dependency tree approach for extracting syntactic features. The component tree adds syntactic features to the word embedding vectors, and the relationships between words in the dependency tree are used to extract the structural features of the sentence, thus complementing the advantages and disadvantages of these two syntactic trees.

[0106] To extract the structural features of a tree, this invention proposes a novel method—numbering the nodes of a tree sequentially from top to bottom and from left to right. For example... Figure 6 As shown, three structural features can be obtained: 1) the number of the tree node; 2) the number of the parent node of the tree node; 3) the number of the child nodes of the tree node. The parent representation of a tree can reconstruct a tree; adding numbers can distinguish identical syntactic tags within the same tree, for example... Figure 5 The embedding vectors of the two NPs in the array will be different after adding numbers.

[0107] This invention argues that not all parts of a sentence are equally relevant to the author's writing style. Therefore, this invention combines multiple attention methods to improve the accuracy of author attribution.

[0108] This invention posits that not all tags in a word's syntactic path are equally important; the same tag may have different importance in different paths. Each text consists of several words, and the importance of each word within the text varies. Therefore, this invention proposes a hierarchical attention network. The attention calculation for tags is as follows:

[0109] u o1 =tanh(W i3 h o1 +b i3 )

[0110] Among them, u o1 W is the transformed label vector. i3 b i3 To randomly initialize parameters, h o1 Let o1 be the label vector, o1 be the o1th transformation variable, and l1 be the l1th input variable;

[0111] A hierarchical attention network is used to calculate the label attention coefficients of the label vectors, denoted as:

[0112]

[0113] Among them, a o2Let o2 be the label attention coefficient, o2 be the o2th attention coefficient, l2 be the l2th input variable, and T be the transpose sign;

[0114] Add the corresponding label attention coefficient to the word embedding vector of each syntactic feature to obtain the final word embedding vector, represented as:

[0115]

[0116] Among them, w i3 For the final embedding vector of the o3th word, h l3 This is the l3rd input vector.

[0117] The calculation of attention between words is similar. A vector representation of each word is obtained, where each label and each word has a different importance. This vector is then fed into a graph convolutional network.

[0118] This invention employs a graph attention network to calculate the attention coefficients of different neighboring nodes in a syntactic component tree. When using graph convolutional neural networks to extract structural features of a sentence, each neighboring node is considered to have the same importance, which is biased. This invention argues that different neighboring nodes have significantly different importance to the same node. Therefore, this invention uses a graph attention network to obtain the attention coefficients of different neighboring nodes, calculated using the following formula:

[0119]

[0120] Among them, a o4 Let W be the attention coefficient of different adjacent nodes in the syntactic component tree. i4 For learnable parameters, a T For a fully connected layer, h o4 For the 4th syntactic information, h j4 This is the j4th syntactic information; used to calculate the output features:

[0121] p o5 =σ(∑a o5 W i5 h j5 )

[0122] Where, p o5 Let h be the feature vector of the 5th node, σ be the nonlinear activation function, and h be the feature vector of the 5th node. j5 This represents the j5th syntactic information. Adding the resulting vectors yields the syntactic feature vector s.

[0123] S4. Concatenate the content feature vector and the syntactic feature vector to obtain the text style feature, and predict the author attribution of the text information data based on the text style feature.

[0124] In an optional embodiment of the present invention, such as Figure 2 As shown, this invention concatenates the obtained content feature vector e and syntactic feature vector s to obtain the author's text style feature V. This feature is then input into the classification module to obtain the probability distribution of the text for each author, calculated as follows:

[0125] y k1 =softmax(W i6 V s +b i6 )

[0126] Where is the probability distribution of the text belonging to the k1-th author, W i6 and b i6 To randomly initialize the parameters, y k1 Let N be an N-dimensional vector, where N is the number of authors.

[0127] The author with the highest probability is selected as the author of the anonymous text.

[0128] This invention was tested on two short text datasets, Twitter and Weibo. The invention uses the metric P@K, where a correct author is considered a correct prediction among the top K most probable predicted authors. The experimental results are shown in Table 1. For all k values ​​of the P@K metric, this invention achieved the best results. To verify the effect of multiple attention mechanisms, this invention removed the attention module. The results are shown in the last row of Table 1, where the P@1 metric improved by 3.9% and 1.6%, respectively, indicating that the proposed method of combining multiple attention mechanisms is effective. To verify the effect of syntactic features, this invention removed the attention module and compared it with the Syntax-CNN model. As shown in Table 1, when extracting deep syntactic features, this invention outperformed Syntax-CNN by 12.7% and 11.3% in the P@1 metric, respectively. The results further demonstrate the importance of syntactic information for author attribution tasks and show that the syntactic information extracted by this invention is richer than that of the Syntax-CNN model.

[0129] Table 1. P@K index of the present invention and various comparative methods

[0130]

[0131] To verify the generalization ability of this invention, relevant experiments were conducted on two blogs datasets. The experimental results are shown in Table 2. The P@1 metric of this invention on the blogs10 and blogs50 datasets is improved by 1.7% and 2.0% respectively compared to Syntax-CNN. Compared with the Twitter and Weibo datasets, the blogs dataset has slightly longer text lengths and different topics. The improved performance indicates that this invention can solve the author attribution problem of datasets with different topics, proving that this invention has strong generalization ability.

[0132] Table 2. P@1 index of the present invention and the comparison method on the blogs dataset.

[0133]

[0134] To verify the effectiveness of the proposed method, the present invention conducted relevant ablation experiments.

[0135] Table 2. P@1 index of syntactic features

[0136]

[0137] To verify the effects of dependency tree features and constituent tree features on writing style characteristics, these features were gradually added in the experiment. The ablation experiment of syntactic features is shown in Table 2. When only dependency tree features were used, the P@1 index increased by 2.1% and 5.2% respectively, indicating that the extracted sentence structure features are helpful for author attribution tasks. Based on the extracted sentence dependency tree features, this invention extracted sentence constituent tree features, and the P@1 index also improved compared to using dependency tree features, by 1.6% and 3.8% respectively. This is consistent with the hypothesis of this invention: a dependency tree is a syntactic representation of the grammatical relationships between words, but the words in the dependency tree do not contain syntactic features, while a syntactic constituent tree is a phrase structure syntactic tree of a sentence, and a syntactic path can add syntactic features to each word in the sentence. Therefore, this invention uses both constituent trees and dependency trees to extract sentence syntactic features, and the advantages and disadvantages of these two types of trees complement each other.

[0138] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1A device that provides the functions specified in one or more boxes.

[0139] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0140] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0141] Specific embodiments have been used to illustrate the principles and implementation methods of this invention. The descriptions of the embodiments above are only for the purpose of helping to understand the method and core ideas of this invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this invention. Therefore, the content of this specification should not be construed as a limitation of this invention.

[0142] Those skilled in the art will recognize that the embodiments described herein are intended to help the reader understand the principles of the invention, and should be understood that the scope of protection of the invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific modifications and combinations based on the technical teachings disclosed in this invention without departing from the spirit of the invention, and these modifications and combinations are still within the scope of protection of this invention.

Claims

1. An authorship method of maximizing syntactic information of a plurality of attentions in combination, characterized by, The method comprises the following steps: S1, obtaining text information data of an unknown author, preprocessing the text information data to obtain character original data and syntax original data, and embedding to obtain character embedding vectors and label embedding vectors respectively; S2, extracting character 2-grams feature maps from the character embedding vectors, and constructing content feature vectors by using a maximum pooling method; S3, extracting syntax information features from the label embedding vectors, and calculating attention coefficients between different words respectively to construct syntax feature vectors; S4, splicing the content feature vectors and the syntax feature vectors to obtain text style features, and predicting the authorship of the text information data according to the text style features; Step S3 specifically comprises: constructing a syntax constituent tree according to the text information data, and encoding the syntax constituent tree into a distributed representation vector; constructing a syntax dependency tree according to the dependency relationship of the words in the text information data, numbering the nodes in the syntax dependency tree in order, extracting the structural features of the syntax dependency tree, and obtaining an adjacency matrix of the words; splicing the label embedding vectors, the distributed representation vector and the adjacency matrix of the words, and extracting word embedding vectors containing syntax features by using multiple parallel single-layer convolutional neural networks; calculating label attention coefficients in the word embedding vectors of each syntax feature by using a hierarchical attention network, and adding the corresponding label attention coefficients to the word embedding vectors of each syntax feature to obtain final word embedding vectors; extracting syntax information from the word embedding vectors by using a bidirectional long short-term memory, and calculating attention coefficients of different adjacent points in the syntax constituent tree by using a graph attention network to obtain node feature vectors; adding all the node feature vectors to obtain a syntax feature vector.

2. The author attribution method of claim 1, wherein, Step S2 specifically comprises: obtaining attention coefficients of all vectors in the character embedding vectors by using a self-attention layer to obtain character feature vectors with added attention coefficients; extracting character 2-grams feature maps from the character feature vectors with added attention coefficients by using multiple parallel single-layer convolutional neural networks; selecting the maximum value of each character 2-grams feature map by using a maximum pooling method, and concatenating the selected feature maps to construct a content feature vector.

3. The author attribution method of claim 2, wherein, The method of obtaining attention coefficients of all vectors in the character embedding vectors by using a self-attention layer to obtain character feature vectors with added attention coefficients is represented as: wherein, is the character feature vector with the attention coefficient added, is the attention coefficient for all vectors in the character embedding vector, is the full connection operation, N is the total number of vectors in the character embedding vector.

4. The method of claim 2, wherein, The method of extracting character 2-grams feature maps from the character feature vectors with added attention coefficients by using multiple parallel single-layer convolutional neural networks is represented as: wherein, is a 2-grams feature map, is a filter matrix, is a convolution operation, is a character feature vector with added attention coefficients, i2 is a vector index, is a window size of the filter matrix, is a bias vector.

5. The author attribution method of claim 1, wherein, The method of calculating label attention coefficients in the word embedding vectors of each syntax feature by using a hierarchical attention network, and adding the corresponding label attention coefficients to the word embedding vectors of each syntax feature to obtain final word embedding vectors specifically comprises: transforming the label vectors in the word embedding vectors of each syntax feature, represented as wherein, is the first transformed label vector, o 1st transformed label vector, , is a random initialization parameter, is the first transformed label vector, o 1st transformed label vector, calculating label attention coefficients of the label vectors by using a hierarchical attention network, represented as wherein, is the first o 2 label attention coefficients, is the first transformed label vector, is the first l 2 transformed label vectors, T is the transpose symbol; adding the corresponding label attention coefficients to the word embedding vectors of each syntax feature to obtain final word embedding vectors, represented as: wherein, is the final word embedding vector, is the final word embedding vector, is the final word embedding vector, l is the final word embedding vector.

6. The author attribution method of claim 1, wherein, The bidirectional long short-term memory is used to extract syntax information from the word embedding vector, and a graph attention network is used to calculate attention coefficients of different adjacent points in the syntax constituent tree to obtain a node feature vector, and specifically includes: Syntax information is extracted from the word embedding vector by using bidirectional long short-term memory, and is represented as wherein, is the extracted first syntactic information, is the forward propagation operation, is the backward propagation operation, is the concatenation operation, is the final first word embedding vector; The graph attention network is used to calculate the attention coefficients of different adjacent points in the syntax constituent tree, and is represented as wherein, is an attention coefficient for different adjacent points in the syntax component tree, is a learnable parameter, is a fully connected layer, is the first syntax information, is the first syntax information; According to the extracted syntax information and the calculated attention coefficients, the node feature vector is calculated, and is represented as wherein, is the o 5th is a non-linear activation function, is the th syntactic information.

7. The author attribution method of claim 1, wherein, The content feature vector and the syntax feature vector are spliced to obtain a text style feature, and the author attribution of the text information data is predicted according to the text style feature, and specifically includes: The content feature vector e and the syntax feature vector s are spliced to obtain a text style feature V s ; A classification network is used to calculate the probability distribution of the attribution of the text to each author according to the text style feature, and is represented as wherein, is the probability distribution of the text attribution to the first author, and is a random initialization parameter; The author corresponding to the maximum probability is selected as the predicted attribution author of the text.

Images