Text analysis method and apparatus, and electronic device

Abstract

Provided are a text analysis method and apparatus, and an electronic device, relating to the technical field of computers. The method comprises: constructing each sentence in text into a text graph, and using words in the text as nodes in the text graph (S201); using a gated graph neural network to determine initial text features of the text graph, and using the initial text features as coarse-grained features of the nodes, the coarse-grained features being used for representing a relationship between each node and nodes surrounding said node (S202); performing weighting operation on the coarse-grained features to obtain fine-grained features, the fine-grained features being used for representing the degree of correlation between the nodes and abnormal text (S203); performing feature fusion on the coarse-grained features and the fine-grained features to obtain a final text feature (S204); and inputting the text feature into a pre-trained text model to obtain a probability value, and comparing the probability value with a preset threshold value, so as to determine whether the text is abnormal text (S205). The method can improve the accuracy of text detection.

Description

Text analysis method, device, and electronic device

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office of the People's Republic of China on December 1, 2023, with application number 202311640631.5 and application name "A text analysis method, device, and electronic device", the entire contents of which are incorporated by reference into this application. Technical Field

[0003] The present application relates to the field of computer technology, and in particular to a text analysis method, device, and electronic device. Background Art

[0004] With the advent of the information and digital age, vast amounts of multimedia information, including text, audio, and video, are transmitted over the internet, bringing numerous benefits to society. However, this also presents numerous information security challenges. From personal privacy data to national secrets, the security of this information, present in every corner of society, has undoubtedly become a critical issue in the digital age that demands significant attention and vigilance. Text steganalysis is a technology designed to safeguard text information security, analyzing whether text has been tampered with to carry malicious or illegal information.

[0005] Deep learning-based text steganalysis methods leverage the self-learning properties of neural networks to train them to capture unusual semantic features in text, achieving text discrimination. These methods typically use recurrent neural networks, long-short-term memory networks, or pre-trained language models as the primary network, employing different convolutional kernels or integrating features from various network layers to maximize the utilization of effective features. Currently, these methods lack the ability to capture semantic features between long-distance words, and feature interactions are often rigid, resulting in limited text discrimination accuracy.

[0006] Summary of the Invention

[0007] The embodiments of the present application provide a text analysis method, apparatus, and electronic device for improving the detection efficiency and accuracy of text analysis.

[0008] In a first aspect, an embodiment of the present application provides a text analysis method, comprising:

[0009] Construct each sentence in the text into a text graph, and use the words in the text as nodes in the text graph;

[0010] A gated graph neural network is used to determine the initial text features of the text graph. The initial text features are used as coarse-grained features of the nodes. The coarse-grained features are used to represent the relationship between the node and the nodes around it.

[0011] The coarse-grained features are weighted to obtain fine-grained features, which are used to represent the degree of relevance between the node and the abnormal text;

[0012] Fusing coarse-grained features with fine-grained features to obtain the final text features;

[0013] The text features are input into the pre-trained text model to obtain the probability value, and the probability value is compared with the preset threshold to determine whether the text is abnormal. The text model uses the Softmax activation function to convert the output of the neural network into a numerical value representing the probability distribution of different categories.

[0014] In this method, by employing a gated graph neural network to determine the initial text features of the text graph and using these initial text features as coarse-grained features for nodes, semantic features can be flexibly captured between long- and short-distance nodes. This provides richer features, flexibly captures more subtle semantic features within the text, and reduces the loss of text features. Furthermore, by performing weighted operations on coarse-grained features to generate fine-grained features, the weights of features of different dimensions between different nodes can be emphasized, increasing the correlation of valid features in feature values ​​and reducing the impact of invalid features on classification. This improves the accuracy of text detection.

[0015] Optionally, each sentence in the text is constructed as a text graph, and the words in the text are used as nodes in the text graph, specifically including:

[0016] Segment the text to obtain multiple words;

[0017] Use sliding window to traverse the text;

[0018] Edges are established between words that appear in the same sliding window, and a text graph and an adjacency matrix are built to represent the relationship between nodes.

[0019] In the above method, by constructing the text into a text graph, it is convenient to subsequently use the text graph to determine the text features of the text, thereby improving the accuracy of text detection.

[0020] Optionally, the gated graph neural network includes multiple layers of nodes, and the gated graph neural network satisfies the following formula:

[0021] in, represents the coarse-grained features of the k-th node, Used to choose to forget the status information of the last time, represents the coarse-grained features of the k-th node in the previous network layer, Indicates that the update gate controls the newly generated information, Represents the information learned in the previous time.

[0022] Optional, Satisfies the following formula:

[0023] Among them, W z represents the trainable weight, u z represents the trainable weights, Represents the interaction relationship between nodes in each layer of the network.

[0024] Optional, Satisfies the following formula:

[0025] in, represents the inbound and outbound edges of the kth node in the adjacency matrix, W a Represents the trainable parameters of the network layer.

[0026] Optional, Satisfies the following formula:

[0027] in, Satisfies the following formula: W r represents the trainable weight, U r represents the trainable weights, Indicates that the reset gate controls the forgetting information.

[0028] Optionally, before using a gated graph neural network to determine initial text features of the text graph and using the initial text features as coarse-grained features of the nodes, the method further includes:

[0029] When the dimensions of multiple word vectors are different, unify the dimensions of each word vector.

[0030] In the above method, by unifying the dimensions of multiple word vectors when the dimensions of multiple word vectors are different, subsequent calculations can be easier and more accurate results can be obtained, which is conducive to improving the accuracy of subsequent text detection.

[0031] Optionally, a weighted operation is performed on the coarse-grained features to obtain fine-grained features, specifically including:

[0032] Perform maximum pooling and average pooling operations on the coarse-grained features to obtain feature values;

[0033] Perform convolution operation and normalization on the eigenvalues ​​to obtain weight parameters;

[0034] Multiply the coarse-grained features by the weight parameters to obtain fine-grained features.

[0035] In the above method, by performing a weighted operation on coarse-grained features to obtain fine-grained features, we can obtain fine-grained features that are more relevant to abnormal text. When the final text features are subsequently derived based on the fine-grained features, the final text features are more relevant to abnormal text, which facilitates the accuracy of subsequent text detection.

[0036] Optionally, the coarse-grained features are fused with the fine-grained features to obtain the final text features, including:

[0037] Perform adaptive average pooling operations on word vectors of different dimensions to obtain adaptive average pooling results;

[0038] Calculate the correlation coefficient between different dimensions of word vectors based on the adaptive average pooling results;

[0039] Add the correlation coefficients of all word vectors to obtain the first text feature;

[0040] The coarse-grained feature value and the fine-grained feature value are jump-connected and added to the first text feature to obtain the final text feature.

[0041] In the above method, since each word is represented by a multi-dimensional word vector, different dimensions represent different abstract features of the word, and the features of certain dimensions more clearly express the text tampering features, which is more helpful for the classification module to make classification judgments. The present application performs adaptive average pooling operations on word vectors of different dimensions to obtain adaptive average pooling results. This can not change the original number of word vector channels, and compresses difficult-to-process high-dimensional features into low-dimensional features of efficient expression, making the text feature expression more concentrated. At the same time, it reduces the parameters in the network, reduces the amount of calculation, makes network processing more efficient, and helps to reduce the overfitting phenomenon in the network training process. By calculating the correlation coefficients between different dimensions of the word vectors based on the adaptive average pooling results, the sparsity of the network can be further improved, while reducing the negative correlation effects of irrelevant information. The final text feature representation is made more accurate.

[0042] Optionally, the above preset function satisfies the following formula:

[0043] Among them, P represents the probability value, W4 represents the trainable parameters of the network layer, and F j Represents fine-grained feature values.

[0044] Optionally, the probability value is compared with a preset threshold to determine whether the text is abnormal text, specifically including:

[0045] When the probability value is greater than or equal to the threshold, the text is determined to be abnormal text;

[0046] When the probability value is less than the threshold, the text is determined to be normal text.

[0047] In the above method, when the probability value is greater than or equal to the threshold, the text is determined to be abnormal text; when the probability value is less than the threshold, the text is determined to be normal text. This allows the user to determine whether the text is abnormal text in a timely manner.

[0048] In a second aspect, an embodiment of the present application provides a text analysis device, comprising:

[0049] A processing module, configured to construct each sentence in the text into a text graph, and use the words in the text as nodes in the text graph;

[0050] A determination module is used to determine the initial text features of the text graph using a gated graph neural network, and use the initial text features as coarse-grained features of the nodes. The coarse-grained features are used to represent the relationship between the node and the nodes around the node;

[0051] A calculation module is used to perform weighted operations on coarse-grained features to obtain fine-grained features. The fine-grained features are used to represent the degree of relevance between the node and the abnormal text;

[0052] The calculation module is also used to fuse the coarse-grained features with the fine-grained features to obtain the final text features;

[0053] The processing module is also used to input text features into a pre-trained text model to obtain a probability value, and compare the probability value with a preset threshold to determine whether the text is abnormal. The text model uses the Softmax activation function to convert the output of the neural network into a numerical value representing the probability distribution of different categories.

[0054] In a third aspect, an embodiment of the present application further provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, the processor implements any one of the text analysis methods described in the first aspect above.

[0055] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the text analysis method of the first aspect is implemented.

[0056] In a fifth aspect, an embodiment of the present application further provides a computer program product, including a computer program, which is executed by a processor to implement the text analysis method as described in any one of the first aspects above.

[0057] The technical effects brought about by any implementation method in the second to fifth aspects can refer to the technical effects brought about by the corresponding implementation method in the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] FIG1 is a schematic diagram of an application scenario of a text analysis method provided in an embodiment of the present application;

[0059] FIG2 is a flow chart of a text analysis method provided in an embodiment of the present application;

[0060] FIG3 is a schematic diagram of a model training provided in an embodiment of the present application;

[0061] FIG4 is a flowchart of an exemplary text analysis method provided in an embodiment of the present application;

[0062] FIG5 is a schematic diagram of a text analysis device provided in an embodiment of the present application;

[0063] FIG6 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0064] To make the objectives, technical solutions, and advantages of this application more clear, this application will be further described in detail below with reference to the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making any creative efforts are within the scope of protection of this application.

[0065] The application scenarios described in the embodiments of this application are intended to more clearly illustrate the technical solutions of the embodiments of this application and do not constitute a limitation on the technical solutions provided by the embodiments of this application. Persons skilled in the art will appreciate that, as new application scenarios emerge, the technical solutions provided by the embodiments of this application are equally applicable to similar technical problems. In the description of this application, unless otherwise specified, "multiple" means two or more.

[0066] With the advent of the information and digital age, vast amounts of multimedia information, including text, audio, and video, are transmitted over the internet, bringing numerous benefits to society. However, this also presents numerous information security challenges. From personal privacy data to national secrets, the security of this information, present in every corner of society, has undoubtedly become a critical issue in the digital age that demands significant attention and vigilance. Text steganalysis is a technology designed to safeguard text information security, analyzing whether text has been tampered with to carry malicious or illegal information.

[0067] Text steganalysis methods based on machine learning extract text statistical features through complex manual calculations, and then input them into classification models such as support vector machines for classification. This type of method relies heavily on the richness of manually calculated features and is difficult to adapt to complex text environments. In addition, manual calculation methods are inefficient.

[0068] Deep learning-based text steganalysis methods leverage the self-learning properties of neural networks to train them to capture unusual semantic features in text, achieving text discrimination. These methods typically use recurrent neural networks, long-short-term memory networks, or pre-trained language models as the primary network, employing different convolutional kernels or integrating features from various network layers to maximize the utilization of effective features. Currently, these methods lack the ability to capture semantic features between long-distance words, and feature interactions are often rigid, resulting in limited text discrimination accuracy.

[0069] In order to solve the above problems, the embodiments of the present application provide a text analysis method, device and electronic device. For example, an image to be detected is obtained. Among them, each sentence in the text is constructed as a text graph, and the words in the text are used as nodes in the text graph; a gated graph neural network is used to determine the initial text features of the text graph, and the initial text features are used as coarse-grained features of the nodes, and the coarse-grained features are used to represent the relationship between the node and the nodes around the node; the coarse-grained features are weighted to obtain fine-grained features, and the fine-grained features are used to represent the degree of relevance between the node and the abnormal text; the coarse-grained features and the fine-grained features are fused to obtain the final text features; the text features are input into a pre-trained text model to obtain a probability value, and the probability value is compared with a preset threshold to determine whether the text is an abnormal text. Among them, the text model uses a Softmax activation function to convert the output of the neural network into a numerical value representing the probability distribution of different categories.

[0070] As shown in Figure 1, an application scenario diagram of an optional text analysis method of an embodiment of the present application includes a server 100 and a terminal 101. The server 100 and the terminal 101 can be communicatively connected through a network to implement the text analysis method of the present application.

[0071] The user can use the server 100 to interact with the terminal 101 via the network, such as receiving or sending messages, etc. Various client applications can be installed on the terminal 101, such as programming applications, web browser applications, search applications, etc.

[0072] In the embodiment of the present application, the server 100 can be implemented as an independent server or a server cluster composed of multiple servers. The terminal 101 can be various electronic devices with a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, desktop computers, etc.

[0073] As shown in FIG2 , a flowchart of a text analysis method provided in an embodiment of the present application may specifically include the following steps.

[0074] Step S201: construct each sentence in the text into a text graph, and use the words in the text as nodes in the text graph.

[0075] In an optional embodiment, the server can use a sliding window to identify text and construct a text graph. For example, the text is segmented to obtain multiple words; the sliding window is used to traverse the text; edges are established between words that appear in the same sliding window, and the corresponding text and an adjacency matrix are constructed to represent the relationship between each node.

[0076] In an optional embodiment, the server may use word vectors to represent the node set, use an adjacency matrix to represent the relationship between nodes in the node set, and use words in the text as nodes to form the node set, specifically including:

[0077] When two words appear simultaneously in the sliding window, the edge connecting the two words is recorded as 1 in the adjacency matrix. When a single word appears in the sliding window, the single word is recorded as 0 in the adjacency matrix.

[0078] For example, for each word in the text, a sliding window size of 2 is set, with each word as a node. Edges are established between all words in the sliding window, and nodes with the same word are merged into a single node to obtain a news text graph. An adjacency matrix is ​​then built based on the text graph and Laplace normalization is performed. The features of each word are then obtained through a standard gated graph neural network layer. The initial word node features can be the GloVe pre-trained word embedding vectors.

[0079] Step S202: Using a gated graph neural network to determine the initial text features of the text graph, and using the initial text features as the coarse-grained features of the nodes.

[0080] Among them, coarse-grained features are used to represent the relationship between a node and the nodes around it.

[0081] In an optional embodiment, assuming that the gated graph neural network has t layers, each node is updated t times, and each time the node is updated in combination with the information of the node's surrounding nodes and the node's information in the previous update process.

[0082] It is understood that the number of layers of the gated graph neural network can be pre-set by those skilled in the art. The number of layers of the gated graph neural network can also be set according to the specific application scenario. This application does not specifically limit this. For example, the number of layers of the gated graph neural network can be 7 layers. For another example, the number of layers of the gated graph neural network can be 9 layers.

[0083] In the above method, the gated graph neural network is used to capture the coarse-grained features of the text, which can flexibly obtain more text semantic features, including significant semantic interactions between adjacent word nodes and potential semantic interactions between long-distance word nodes, reducing feature loss and improving the model's discrimination accuracy.

[0084] Specifically, since the dimensions of word vectors are not uniform, the server can unify the dimensions of each word vector when the dimensions of multiple word vectors are different. For example, all node dimensions can be filled in uniformly first, and the missing parts are filled with 0. The nodes after the unified dimensions are recorded as:

[0085] Where k represents the kth node. Represents the kth node after dimension unification.

[0086] After unifying the dimensions of each node, the server can calculate the interaction relationship between each node and its surrounding nodes. Satisfies the following formula:

[0087] in, Represents the interaction relationship between nodes in each layer of the network. represents the inbound and outbound edges of the kth node in the adjacency matrix, W a Represents the trainable parameters of the network layer. Indicates the status information of the k-th node at the last time.

[0088] In determining each layer of the network, the interaction relationship between nodes is determined and the status of the previous time Afterwards, update gates are used and reset gate Control the newly generated information and the forgotten information. Among them, the Sigmoid activation function is used to control the The amount of information, each element in the output vector has a value between 0 and 1. And the output vector has the same dimension.

[0089] Satisfies the following formula:

[0090] Among them, W z represents the trainable weight, u z Represents trainable weights.

[0091] Satisfies the following formula:

[0092] Among them, W rrepresents the trainable weight, U r Represents trainable weights.

[0093] After confirming After the vector, according to the interaction relationship between the nodes at the current time and the status information of the last time Calculate the new information generated at the current time use As a control coefficient to limit the state information of the last time. Further, according to the state information of the last time and the information learned at the current time Calculate the status information of the current time, Used to choose to remember newly generated information, It is used to choose to forget the state information of the previous time, and finally obtain the state information of the coarse-grained feature of the kth node at the current time. Satisfies the following formula:

[0094] In the above method, a gated graph neural network is used to flexibly obtain the feature relationship between long- and short-distance word nodes. Compared with neural networks such as recurrent neural networks and long short-term memory networks, it can more flexibly obtain more hidden semantic features in the text and reduce feature loss.

[0095] Step S203: Perform weighted operation on the coarse-grained features to obtain fine-grained features.

[0096] Among them, fine-grained features are used to represent the degree of relevance between nodes and abnormal text.

[0097] In an optional embodiment, the relationship features between each node in the text graph and the surrounding nodes, i.e., coarse-grained features, can be obtained by using a gated graph neural network. However, since not all relationship features in the text graph are related to abnormal text, there is no more obvious expression for the abnormal text node features, which is not conducive to subsequent classification judgment. Therefore, the server can perform maximum pooling and average pooling operations on the coarse-grained features to obtain feature values. The feature values ​​are convolved and normalized to obtain weight parameters. The coarse-grained features are multiplied by the weight parameters to obtain fine-grained features.

[0098] For example, the server can assign different weights to words in the text according to preset rules to distinguish the importance of different words in determining whether the text is abnormal.

[0099] It is understandable that the preset rules for allocating weights in the embodiment of the present application are pre-set by those skilled in the art. The above preset rules can also be set according to specific application scenarios.

[0100] The server can use the spatial attention mechanism. First, it takes the feature map G1 obtained after determining the coarse-grained features as input, performs maximum pooling and average pooling operations respectively to obtain a one-dimensional text feature description. After splicing the one-dimensional text feature descriptions, it performs convolution operations and Sigmoid normalization processing to obtain a weight parameter W with a value between 0 and 1.

[0101] The weight parameter W satisfies the following formula: W=Sigmoid(Conv(Concat[MaxPool(G1),AvgPool(G1)]))

[0102] By multiplying the feature map G1 obtained after determining the coarse-grained features with the weight parameter W, we can obtain fine-grained features G2 representing different levels of importance.

[0103] The fine-grained feature G2 satisfies the following formula: G2 = W·G1

[0104] In the above method, a multi-attention mechanism is used to assign weights to coarse-grained features to obtain fine-grained features that represent whether a node is related to abnormal text. Compared with methods such as multi-scale convolution kernels or multi-layer feature map fusion, this method strengthens effective feature expression, highlights the feature expression of different dimensions of different words that contributes more to the classification module, and further improves the accuracy of model discrimination.

[0105] Step 204: Fusing the coarse-grained features with the fine-grained features to obtain the final text features.

[0106] In an optional embodiment, the server can perform adaptive average pooling on word vectors of different dimensions to obtain an adaptive average pooling result. Based on the adaptive average pooling result, the correlation coefficients between word vectors of different dimensions are calculated. The correlation coefficients of all word vectors are summed to obtain the initial text features. The coarse-grained feature values ​​and fine-grained feature values ​​are jump-connected and added to the initial text features to obtain the text features.

[0107] Specifically, because each word is represented by a multi-dimensional word vector, different dimensions represent different abstract features of the word. Features from certain dimensions more clearly express the characteristics of text tampering and are more helpful for the classification module to make classification judgments. Therefore, in addition to weighting each word, we can further weight the dependencies between each word's multi-dimensional vectors to enhance the steganographic characteristics.

[0108] The server can first perform adaptive average pooling operations on word vectors of different dimensions.

[0109] For example, the adaptive average pooling operation on word vectors of different dimensions satisfies the following formula: k = AdaptiveAvgPool(G2)

[0110] This method uses adaptive average pooling on word vectors of the same dimension, compressing difficult-to-process high-dimensional features into more efficiently expressed low-dimensional features without changing the number of channels in the original word vectors. This allows for more focused feature representation, reduces network parameters and computational complexity, and makes network processing more efficient, helping to mitigate overfitting during network training.

[0111] After performing adaptive average pooling operations on word vectors of different dimensions, the server can calculate the correlation coefficient coe between word vectors of different dimensions based on the adaptive average pooling results.

[0112] Among them, coe satisfies the following formula: coe=Sigmoid(W2·ReLU(W1·k))

[0113] In , W1 and W2 are trainable weights.

[0114] W1 and W2 can be used to assign trainable values ​​to different feature dimensions of word vectors.

[0115] In the above method, the ReLU activation function can almost truncate negative values ​​to 0, further improving the sparsity of the network and reducing the negative correlation effect caused by irrelevant information. On this basis, the Sigmoid function is used to normalize the calculation results, which is more conducive to classification by the classification module.

[0116] After determining the correlation coefficient, the server can use it to further strengthen the different dimensions of the word vector representation, obtaining G3 = coe·G2. All word vectors processed by the fine-grained feature update module are then added together to calculate the initial text feature G4.

[0117] After obtaining the initial text feature G4, the feature values ​​of the coarse-grained feature update module and the fine-grained feature update module are skipped, the valid feature values ​​are reused, and added to G4 to obtain the text feature F.

[0118] The text feature F satisfies the following formula: F = W3(G4+Maxpooling(G3)+Maxpoolong(G2))

[0119] Step S205: Input the final text features into the pre-trained text model to obtain a probability value, and compare the probability value with a preset threshold to determine whether the text is abnormal text.

[0120] Among them, the text model uses the Softmax activation function to convert the output of the neural network into a numerical value representing the probability distribution of different categories.

[0121] In an optional embodiment, the server may use a Softmax activation function as a pre-trained text model.

[0122] The following describes how to train a text model:

[0123] Use normal text and abnormal text pairing training, use cross entropy loss as the model's loss function to measure the distance between the model's discrimination results and the text's true label, and reduce the loss value as the goal of network training. The cross entropy loss formula is as follows:

[0124] Among them, x i Represents the true label of the i-th sample, y i Represents the discriminant result of the model, when x i with y i The closer the distribution is, the smaller the loss value is and the more accurate the model judgment is.

[0125] As shown in Figure 3, an embodiment of the present application provides a model training schematic diagram. Figure 3 records the change process of the loss function from the beginning of model training to the 200th iteration. After the 100th round of model training, the model is basically stable and the training ends after the 200th round of training. The value of the loss function in Figure 3 gradually stabilizes as the number of training times increases.

[0126] The feature F learned by the neural network is used as the input of the pre-trained text model and the probability value P is output. The probability value P satisfies the following formula:

[0127] Among them, W4 is the trainable parameter of this layer network.

[0128] After determining the probability value, the server can compare the probability value with a preset threshold to determine whether the text is abnormal text.

[0129] It is understandable that the threshold values ​​in the embodiments of the present application are preset by those skilled in the art and can also be set according to specific application scenarios.

[0130] In one possible case, the probability value is greater than the threshold value and the text is an abnormal text.

[0131] In another possible case, the probability value is less than or equal to the threshold value, and the text is normal text.

[0132] Optionally, to facilitate the user to more intuitively determine whether the text is abnormal, the server can also set the judgment result of the text to be displayed after determining whether the text is abnormal. For example, displaying 1 represents abnormal text and displaying 0 represents normal text.

[0133] As shown in FIG4 , an embodiment of the present application provides a flowchart of an exemplary text analysis method.

[0134] S401, segmenting the text to obtain multiple words;

[0135] S402, traversing the text using a sliding window;

[0136] S403, establishing edges between words that appear in the same sliding window, and establishing a text graph and an adjacency matrix for representing the relationship between nodes;

[0137] S404: using a gated graph neural network to determine initial text features of the text graph, and using the initial text features as coarse-grained features of the nodes;

[0138] S405, performing maximum pooling and average pooling operations on the coarse-grained features to obtain feature values;

[0139] S406, performing convolution operation and normalization processing on the eigenvalues ​​to obtain weight parameters;

[0140] S407, multiplying the coarse-grained feature by the weight parameter to obtain a fine-grained feature;

[0141] S408, performing an adaptive average pooling operation on word vectors of different dimensions to obtain an adaptive average pooling result;

[0142] S409, calculating the correlation coefficients between different dimensions of the word vectors based on the adaptive average pooling results;

[0143] S410, adding the correlation coefficients of all word vectors to obtain a first text feature;

[0144] S411, skip-connecting the coarse-grained feature value and the fine-grained feature value, and adding them to the first text feature to obtain a final text feature;

[0145] S412: Input the final text features into a pre-trained text model to obtain a probability value, and compare the probability value with a preset threshold to determine whether the text is abnormal text.

[0146] FIG5 is a schematic structural diagram of a text analysis device provided in an embodiment of the present application. As shown in FIG5 , the device includes: a processing module 501 , a determination module 502 , and a calculation module 503 .

[0147] Processing module 501, configured to construct each sentence in the text into a text graph, and use the words in the text as nodes in the text graph;

[0148] A determination module 502 is configured to determine initial text features of the text graph using a gated graph neural network, and use the initial text features as coarse-grained features of the nodes, where the coarse-grained features are used to represent the relationship between the node and its surrounding nodes;

[0149] A calculation module 503 is used to perform weighted operations on the coarse-grained features to obtain fine-grained features, where the fine-grained features are used to represent the degree of relevance between the node and the abnormal text;

[0150] The calculation module 503 is further used to fuse the coarse-grained features with the fine-grained features to obtain the final text features;

[0151] The processing module 501 is also used to input the final text features into a pre-trained text model to obtain a probability value, and compare the probability value with a preset threshold to determine whether the text is abnormal text. The text model uses a Softmax activation function to convert the output of the neural network into a numerical value representing the probability distribution of different categories.

[0152] Optionally, each sentence in the text is constructed as a text graph, and the words in the text are used as nodes in the text graph. The processing module 501 is specifically used to:

[0153] Segment the text to obtain multiple words;

[0154] Use sliding window to traverse the text;

[0155] Edges are established between words that appear in the same sliding window, and a text graph and an adjacency matrix are built to represent the relationship between nodes.

[0156] Optionally, the gated graph neural network includes multiple layers of nodes, and the gated graph neural network satisfies the following formula:

[0157] in, represents the coarse-grained features of the k-th node, Used to choose to forget the status information of the last time, represents the coarse-grained features of the k-th node in the previous network layer, Indicates that the update gate controls the newly generated information, Represents the information learned in the previous time.

[0158] Optional, Satisfies the following formula:

[0159] Among them, W zrepresents the trainable weight, u z represents the trainable weights, Represents the interaction relationship between nodes in each layer of the network.

[0160] Optional, Satisfies the following formula:

[0161] in, represents the inbound and outbound edges of the kth node in the adjacency matrix, W a Represents the trainable parameters of the network layer.

[0162] Optional, Satisfies the following formula:

[0163] in, Satisfies the following formula: W r represents the trainable weight, U r represents the trainable weights, Indicates that the reset gate controls the forgetting information.

[0164] Optionally, before determining the initial text features of the text graph using the gated graph neural network and using the initial text features as coarse-grained features of the nodes, the processing module 501 is further configured to:

[0165] When the dimensions of multiple word vectors are different, unify the dimensions of each word vector.

[0166] Optionally, the weighted operation on the coarse-grained features is performed to obtain fine-grained features, and the calculation module 503 is specifically used to:

[0167] Perform maximum pooling and average pooling operations on the coarse-grained features to obtain feature values;

[0168] Perform convolution operation and normalization on the eigenvalues ​​to obtain weight parameters;

[0169] Multiply the coarse-grained features by the weight parameters to obtain fine-grained features.

[0170] Optionally, the above-mentioned text features are obtained by fusing the coarse-grained features with the fine-grained features. The calculation module 503 is specifically used to:

[0171] Perform adaptive average pooling operations on word vectors of different dimensions to obtain adaptive average pooling results;

[0172] Calculate the correlation coefficient between different dimensions of word vectors based on the adaptive average pooling results;

[0173] Add the correlation coefficients of all word vectors to obtain the initial text features;

[0174] The coarse-grained feature values ​​and the fine-grained feature values ​​are skip-connected and added to the initial text features to obtain the text features.

[0175] Optionally, the above preset function satisfies the following formula:

[0176] Among them, P represents the probability value, W4 represents the trainable parameters of the network layer, and F j Represents fine-grained feature values.

[0177] Optionally, the probability value is compared with a preset threshold to determine whether the text is abnormal text. The processing module 501 is specifically used to:

[0178] When the probability value is greater than or equal to the threshold, the text is determined to be abnormal text;

[0179] When the probability value is less than the threshold, the text is determined to be normal text.

[0180] Based on the same technical concept, an electronic device is also provided in an embodiment of the present application, and the electronic device can realize the functions of the aforementioned text analysis device.

[0181] FIG6 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0182] At least one processor 601, and a memory 602 connected to at least one processor 601. In the embodiments of the present application, the specific connection medium between the processor 601 and the memory 602 is not limited. FIG6 takes the connection between the processor 601 and the memory 602 via the bus 600 as an example. The bus 600 is represented by a bold line in FIG6. The connection method between other components is only for schematic illustration and is not intended to be limiting. The bus 600 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, FIG6 only uses a bold line to represent it, but this does not mean that there is only one bus or one type of bus. Alternatively, the processor 601 can also be called a controller, and there is no limitation on the name.

[0183] In this embodiment of the present application, memory 602 stores instructions executable by at least one processor 601. At least one processor 601 can perform the text analysis method discussed above by executing the instructions stored in memory 602. Processor 601 can implement the functions of each module in the apparatus shown in FIG5.

[0184] Among them, the processor 601 is the control center of the device, which can use various interfaces and lines to connect the various parts of the entire control device, and monitor the device as a whole by running or executing instructions stored in the memory 602 and calling data stored in the memory 602, the various functions of the device and processing data.

[0185] In one possible design, processor 601 may include one or more processing units. Processor 601 may integrate an application processor and a modem processor. The application processor primarily processes the operating system, driver interface, and application programs, while the modem processor primarily handles wireless communications. It is understood that the modem processor may not be integrated into processor 601. In some embodiments, processor 601 and memory 602 may be implemented on the same chip. In some embodiments, they may also be implemented on separate chips.

[0186] Processor 601 can be a general-purpose processor, such as a central processing unit (CPU), a digital signal processor, an application-specific integrated circuit, a field-programmable gate array or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component, and can implement or execute the various methods, steps, and logic diagrams disclosed in the embodiments of this application. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the text analysis method disclosed in the embodiments of this application can be directly implemented and executed by a hardware processor, or by a combination of hardware and software modules in the processor.

[0187] The memory 602 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs and modules. The memory 602 may include at least one type of storage medium, such as a flash memory, a hard disk, a multimedia card, a card-type memory, a random access memory (Random Access Memory, RAM), a static random access memory (Static Random Access Memory, SRAM), a programmable read-only memory (Programmable Read Only Memory, PROM), a read-only memory (Read Only Memory, ROM), an electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, EEPROM), a magnetic memory, a disk, an optical disk, etc. The memory 602 is any other medium that can be used to carry or store a desired program code in the form of an instruction or data structure and can be accessed by a computer, but is not limited thereto. The memory 602 in the embodiment of the present application can also be a circuit or any other device that can realize a storage function, for storing program instructions and / or data.

[0188] By programming processor 601, the code corresponding to the text analysis method described in the aforementioned embodiment can be embedded in the chip, enabling the chip to execute the text analysis method of the embodiment shown in FIG3 during operation. Designing and programming processor 601 is well known to those skilled in the art and will not be further described here.

[0189] It should be noted here that the above-mentioned electronic device provided in the embodiment of the present application can implement all the method steps implemented in the above-mentioned method embodiment and can achieve the same technical effect. The parts and beneficial effects of this embodiment that are the same as those in the method embodiment will not be described in detail here.

[0190] An embodiment of the present application further provides a computer-readable storage medium, which stores computer-executable instructions. The computer-executable instructions are used to enable a computer to execute the text analysis method in the above embodiment.

[0191] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0192] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate a device for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0193] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0194] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, whereby the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0195] Obviously, those skilled in the art may make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is intended to include these modifications and variations.

Claims

1. A text analysis method, characterized in that: The method comprises: Construct each sentence in the text into a text graph, and use the words in the text as nodes in the text graph; Using a gated graph neural network to determine initial text features of the text graph, using the initial text features as coarse-grained features of the nodes, the coarse-grained features being used to represent the relationship between the node and nodes surrounding the node; Performing weighted operations on the coarse-grained features to obtain fine-grained features, wherein the fine-grained features are used to represent the degree of relevance between the node and the abnormal text; Fusing the coarse-grained features with the fine-grained features to obtain final text features; The final text feature is input into a pre-trained text model to obtain a probability value, and the probability value is compared with a preset threshold to determine whether the text is an abnormal text. The text model uses a Softmax activation function to convert the output of the neural network into a numerical value representing the probability distribution of different categories.

2. The method according to claim 1, characterized in that: The step of constructing each sentence in the text into a text graph and using the words in the text as nodes in the text graph specifically includes: Segmenting the text to obtain a plurality of words; Using the sliding window to traverse the text; Edges are established between words that appear in the same sliding window, and the text graph and the adjacency matrix used to represent the relationship between each node are established.

3. The method according to claim 1, characterized in that The gated graph neural network includes multiple layers of nodes, and the gated graph neural network satisfies the following formula: in, represents the coarse-grained features of the kth node, Used to choose to forget the status information of the last time. represents the coarse-grained features of the kth node in the previous network layer, Indicates that the update gate controls the newly generated information, Represents the information learned in the previous time.

4. The method according to claim 3, characterized in that: Satisfies the following formula: Among them, W z represents the trainable weight, u z represents the trainable weights, Represents the interaction relationship between nodes in each layer of the network.

5. The method according to claim 4, characterized in that Satisfies the following formula: in, represents the inbound and outbound edges of the kth node in the adjacency matrix, W a Represents the trainable parameters of this layer of the network.

6. The method according to claim 5, characterized in that Satisfies the following formula: in, Satisfies the following formula: W r represents the trainable weight, U r represents the trainable weights, Indicates that the reset gate controls the forgetting information.

7. The method according to claim 1, characterized in that Before determining the initial text features of the text graph by using a gated graph neural network and using the initial text features as the coarse-grained features of the nodes, the method further includes: When the dimensions of multiple word vectors are different, unify the dimensions of each word vector.

8. The method according to claim 1, characterized in that: The step of performing weighted operation on the coarse-grained features to obtain fine-grained features specifically includes: Performing maximum pooling and average pooling operations on the coarse-grained features to obtain feature values; Performing convolution operation and normalization processing on the eigenvalues ​​to obtain weight parameters; The coarse-grained feature is multiplied by the weight parameter to obtain the fine-grained feature.

9. The method according to claim 1, characterized in that: The step of fusing the coarse-grained features with the fine-grained features to obtain text features specifically includes: Perform adaptive average pooling operations on word vectors of different dimensions to obtain adaptive average pooling results; Calculate the correlation coefficients between different dimensions of word vectors according to the adaptive average pooling result; Add the correlation coefficients of all word vectors to get the first text feature; The coarse-grained feature value and the fine-grained feature value are jump-connected and added to the first text feature to obtain the text feature.

10. The method according to claim 1, characterized in that The preset function satisfies the following formula: Where P represents the probability value, W4 represents the trainable parameters of the network layer, and F j represents the fine-grained feature value.

11. The method according to claim 1, characterized in that The comparing the probability value with a preset threshold value to determine whether the text is an abnormal text specifically includes: When the probability value is greater than or equal to the threshold, determining that the text is an abnormal text; When the probability value is smaller than the threshold, the text is determined to be normal text.

12. A text analysis device, characterized in that: include: A processing module is used to construct each sentence in the text into a text graph, and to use the words in the text as the text graph. The nodes in this graph; A determination module, used for determining initial text features of the text graph by using a gated graph neural network, and using the initial text features as coarse-grained features of the nodes, wherein the coarse-grained features are used to represent the relationship between the node and nodes around the node; A calculation module, used for performing weighted operation on the coarse-grained features to obtain fine-grained features, wherein the fine-grained features are used to represent the degree of relevance between the node and the abnormal text; The calculation module is further used to perform feature fusion on the coarse-grained features and the fine-grained features to obtain final text features; The processing module is also used to input the text features into a pre-trained text model to obtain a probability value, and compare the probability value with a preset threshold to determine whether the text is an abnormal text. The text model uses a Softmax activation function to convert the output of the neural network into a numerical value representing the probability distribution of different categories.

13. An electronic device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 11 are implemented.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 11 are implemented.

15. A computer program product, characterized in that When the computer program product is called by a computer, the computer executes the steps of any one of the methods of claims 1 to 11.

Images