An adverse text classification method and system based on a fusion multi-feature map attention mechanism

Abstract

The application provides an unhealthy text classification method fusing a multi-feature map attention mechanism, and the method comprises the following steps: in response to pre-processing of acquired text data; further processing the acquired text data to construct a text graph; performing feature learning on the text graph by using a graph attention mechanism; further embedding the graph attention mechanism into a Transformer network for parallel processing to obtain an output, and simultaneously performing parallel processing by using a Bi_GRU neural network to obtain an output; and splicing the outputs of the MGTransformer and the Bi_GRU to obtain a final text classification result. The application constructs a short text classification model by starting from the features and text semantics of unhealthy text and combining a graph attention mechanism, a Transformer and a Bi_GRU for the classification task of unhealthy text in social media. The model provides a new idea for the application of deep learning to the classification task of unhealthy text, and solves the problems of poor generalization and low precision of the deep learning model in the classification problem of unhealthy text.

Description

Technical Field

[0001] The present invention belongs to the technical field of text classification, and specifically relates to a bad text classification method and system integrating a multi-feature graph attention mechanism. Background Art

[0002] With the rapid adoption of internet technology, social media platforms have become an indispensable tool for daily communication. However, due to the virtual nature of social media and the hidden nature of online information dissemination, some illicit users have exploited these platforms to publish and disseminate negative speech, including various types of harmful information. Harmful information on social media typically spreads through short text messages like chats and comments. These messages are short, feature unconstrained vocabulary, feature sparseness, and possess highly colloquial sentence structures and grammatical variations. Furthermore, these messages are often cryptic, often appearing as homophones or variants of the original harmful words to evade automatic detection.

[0003] There are currently two main methods for classifying bad text that are widely used. One is to use string matching methods to retrieve bad keywords in the text to be tested. If the text sentence contains bad words, the text is considered to be bad text. The disadvantage of this method is that text with bad words is not necessarily bad text, and the error rate is high; the other is to use deep learning model fusion to perform text classification. First, the text tracing model is used to fit the bad text into normal text, and then the traced text data is used as a training set to train a text classifier. Finally, the trained text classifier is used to perform text classification tasks. The disadvantage of this method is that the tracing model is slow to train and the classification accuracy is overly dependent on the results of text tracing.

[0004] Therefore, how to automatically and efficiently classify and filter bad information is of great significance to improving users' normal online social experience and maintaining long-term social stability. If an accurate and efficient bad information detection and classification model can be designed, then the model will have high practical significance and commercial value.

[0005] In view of this, it is very meaningful to propose a bad text classification method and system that integrates multi-feature graph attention mechanism. Summary of the Invention

[0006] In order to solve the problems of high misjudgment rate, slow training of tracing model, excessive dependence of classification accuracy on text tracing results, poor generalization and low precision of deep learning models in existing bad text classification methods, the present invention provides a bad text classification method and system that integrates a multi-feature graph attention mechanism to solve the above-mentioned technical defects.

[0007] In a first aspect, the present invention proposes a bad text classification method integrating a multi-feature graph attention mechanism, the method comprising the following steps:

[0008] In response to preprocessing the acquired text data;

[0009] The acquired text data is further processed to construct a text graph;

[0010] Using a graph attention mechanism to perform feature learning on the text graph;

[0011] Further embedding the graph attention mechanism into the Transformer network for parallel processing to obtain output, while also using the Bi_GRU neural network for parallel processing to obtain output; and

[0012] Concatenate the outputs of MGTransformer and Bi_GRU to obtain the final text classification result.

[0013] Preferably, preprocessing the acquired text data includes removing noise words and formatting sentences, specifically including:

[0014] Use string matching to remove noise words containing special identifiers;

[0015] Regular expressions are used to remove the inherent language sentences in social texts.

[0016] Preferably, the acquired text data is further processed to construct a text graph, specifically including:

[0017] Perform word segmentation and part-of-speech tagging on the acquired text: The segmentation results are converted into low-dimensional and dense word vectors through a pre-trained model. After part-of-speech analysis, redundant conjunctions, auxiliary words, and adverbs are removed, and nouns, verbs, adjectives, and adverbs are used as graph nodes.

[0018] Construct a vocabulary of inflected words: Use the pre-collected vocabulary of inflected words from bad texts. The types of inflected words mainly include shape-shifted words, sound-shifted words, split words, and synonyms.

[0019] Use a sliding window to obtain a text graph of text words: Use a fixed-size sliding window on the text sentence as a rule for constructing edges. If two words are in the same window, an undirected edge is added. Undirected edges are directly added between bad words and their inflections.

[0020] Preferably, performing feature learning on the text graph using the graph attention mechanism includes performing feature aggregation using the graph attention mechanism, specifically including:

[0021] Given node i, its neighborhood is , set node , first calculate the attention coefficient between the node pair (i, j):

[0022] ;

[0023] in, represents any nonlinear activation function, Is a trainable matrix, splicing feature nodes The linear transformation matrix and the learnable attention function Do a dot product operation and then use a nonlinear activation function Activate the obtained nodes to get the original attention score;

[0024] Then normalize the original attention scores of all incoming edges in the node neighborhood to obtain the attention aggregation coefficient, and perform weighted summation on the neighborhood node features to obtain the updated embedding representation of node i:

[0025] .

[0026] More preferably, it also includes:

[0027] To promote the graph attention mechanism to perform feature aggregation more stably, K multi-head attention mechanisms are spliced ​​to enrich the extraction ability of the model. If this layer is the last layer of the model, the node is represented by taking the average value of K heads:

[0028] .

[0029] Preferably, the graph attention mechanism is embedded into the Transformer network for parallel processing to obtain output, specifically including:

[0030] The first layer is the Transformer layer that performs parallel operations on the GAT: using the Transformer encoding layer, the self-attention input part of the Transformer encoding layer is modified to the GAT, realizing parallel calculation of multiple GATs;

[0031] ;

[0032] Where n represents the number of GAT modules;

[0033] The second layer is a fully connected layer and an activation layer. The residual connection and normalization operations in the Transformer are retained. The normalized result is connected to a Feedword layer, which consists of a fully connected layer and a ReLU activation function to obtain the output of the MGAT-Transformer.

[0034] .

[0035] Preferably, the outputs of MGTransformer and Bi_GRU are concatenated to obtain the final text classification result, which specifically includes:

[0036] The output of MGTransformer is horizontally concatenated with the output of Bi_GRU to obtain the final text vector representation;

[0037] The obtained text vector is then connected to a full connection and log_softmax function to predict the classification result and obtain the final category confidence score;

[0038] Finally, the confidence score is used to obtain the classification label corresponding to the text, and the classification result of each text is recorded;

[0039] ;

[0040] in, , the output value is the result of the entire model's prediction of the input text.

[0041] In a second aspect, an embodiment of the present invention further discloses a bad text classification system integrating a multi-feature graph attention mechanism, the system specifically comprising:

[0042] a preprocessing module for preprocessing the acquired text data in response;

[0043] A text graph construction module is used to further process the acquired text data to construct a text graph;

[0044] A feature learning module, configured to perform feature learning on the text graph using a graph attention mechanism;

[0045] The MGTransformer module is used to embed the graph attention mechanism into the Transformer network for parallel processing to obtain output;

[0046] Bi_GRU neural network module, used to use Bi_GRU neural network to perform parallel processing to obtain output;

[0047] The output concatenation module is used to concatenate the outputs of MGTransformer and Bi_GRU to obtain the final text classification results.

[0048] In a third aspect, an embodiment of the present invention provides an electronic device comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by one or more processors, the one or more processors implement the method described in any implementation manner in the first aspect.

[0049] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, having stored thereon a computer program, which, when executed by a processor, implements the method described in any implementation manner of the first aspect.

[0050] Compared with the prior art, the beneficial results of the present application are that:

[0051] (1) The present application proposes an improved graph neural network classification method for the characteristics of bad texts. First, a text graph construction method that fuses the part of speech of words and can freely connect external word libraries is invented. By connecting the deformations of bad keywords as additional free nodes to the text graph, not only the nonlinear relationship between words is fully learned, but also the model's ability to generalize to deformation words is generalized. Second, a bad text classification method that fuses multi-feature graph attention mechanism is proposed. The main part of the model is mainly divided into two feature layers. The first feature layer is a multi-module graph attention mechanism layer. With the help of the transformer, multiple heads can be used to achieve fast parallel characteristics. Parallel multi-module graph attention mechanism extracts text graph features from different subspaces in multiple dimensions. The second feature layer is a Bi GRU layer. On the basis of the multi-module graph attention mechanism, the Bi GRU is parallel to learn the continuous semantic information of the text, make up for the shortcomings of the graph neural network. Finally, the classification data set of sensitive texts is tested, and other sensitive text classification methods are compared to prove the effectiveness of the graph convolution classification model in the classification of bad texts on social media.

[0052] (2) The present application proposes a short text classification model for the classification task of bad texts in social media from the perspective of the characteristics and text semantics of bad texts, combining graph attention mechanism, transformer and Bi GRU. The model provides a new idea for the application of deep learning in bad text classification tasks, and solves the problems of poor generalization and low precision of deep learning models in bad text classification. BRIEF DESCRIPTION OF DRAWINGS

[0053] The accompanying drawings are included to provide a further understanding of embodiments and are incorporated in and constitute a part of this specification. The drawings illustrate embodiments and serve to explain principles of the present application. Other embodiments and many of the intended advantages of the present application will be readily appreciated as the same becomes better understood by reference to the following detailed description when considered in connection with the accompanying drawings. The elements of the drawings are not necessarily to scale relative to each other. Like reference numerals designate corresponding similar parts.

[0054] Figure 1 is an exemplary device architecture diagram in which an embodiment of the present application can be applied;

[0055] Figure 2Schematic diagram of the flow of a bad text classification method integrating a multi-feature graph attention mechanism according to an embodiment of the present invention;

[0056] Figure 3 Schematic diagram of the overall framework of the MGAT_BiGRU model in the bad text classification method integrating the multi-feature graph attention mechanism in an embodiment of the present invention;

[0057] Figure 4 Schematic diagram of a specific model structure in a bad text classification method integrating a multi-feature graph attention mechanism according to an embodiment of the present invention;

[0058] Figure 5 Schematic diagram of the structure of a bad text classification system integrating a multi-feature graph attention mechanism according to an embodiment of the present invention;

[0059] Figure 6 It is a schematic structural diagram of a computer device suitable for implementing an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0060] In the following detailed description, reference is made to the accompanying drawings, which form a part of the detailed description and are illustrated by illustrative specific embodiments in which the present invention may be practiced. To this end, directional terms, such as "top," "bottom," "left," "right," "up," "down," etc., are used with reference to the orientation of the figures being described. Because the components of the embodiments may be positioned in several different orientations, directional terms are used for illustrative purposes and are in no way limiting. It should be understood that other embodiments may be utilized or logical changes may be made without departing from the scope of the present invention. Therefore, the following detailed description should not be adopted in a limiting sense, and the scope of the present invention is defined by the appended claims.

[0061] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is merely illustrative. Any number of terminal devices, networks and servers may be provided as required.

[0062] Figure 1 An exemplary system architecture 100 is shown to which a method for processing information or an apparatus for processing information according to an embodiment of the present invention can be applied.

[0063] like Figure 1 As shown, system architecture 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. Network 104 is a medium for providing communication links between terminal devices 101, 102, 103 and server 105. Network 104 may include various connection types, such as wired or wireless communication links or fiber optic cables.

[0064] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, and 103, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0065] The terminal devices 101 , 102 , and 103 may be various electronic devices with communication functions, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0066] Server 105 may be a server that provides various services, such as a background information processing server that processes verification request information sent by terminal devices 101, 102, and 103. The background information processing server may analyze and perform other processing on the received verification request information and obtain a processing result (e.g., verification success information indicating that the verification request is legitimate).

[0067] It should be noted that the method for processing information provided in the embodiment of the present invention is generally executed by the server 105, and accordingly, the device for processing information is generally set in the server 105. In addition, the method for sending information provided in the embodiment of the present invention is generally executed by the terminal devices 101, 102, and 103, and accordingly, the device for sending information is generally set in the terminal devices 101, 102, and 103.

[0068] It should be noted that a server can be either hardware or software. If the server is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or as a single server. If the server is software, it can be implemented as multiple software programs or software modules (for example, to provide distributed services), or as a single software program or multiple software modules, without specific limitations here.

[0069] There are currently two main methods for classifying bad text that are widely used. One is to use string matching methods to retrieve bad keywords in the text to be tested. If the text sentence contains bad words, the text is considered to be bad text. The disadvantage of this method is that text with bad words is not necessarily bad text, and the error rate is high; the other is to use deep learning model fusion to perform text classification. First, the text tracing model is used to fit the bad text into normal text, and then the traced text data is used as a training set to train a text classifier. Finally, the trained text classifier is used to perform text classification tasks. The disadvantage of this method is that the tracing model is slow to train and the classification accuracy is overly dependent on the results of text tracing.

[0070] To address the above issues, this paper proposes an improved graph neural network classification method based on the characteristics of objectionable text. First, a text graph construction method is invented that integrates word parts of speech and can freely connect to external lexicons. By connecting variants of objectionable keywords as additional free nodes to the text graph, it not only fully learns the nonlinear relationships between words but also generalizes the model's ability to handle variant words. Second, a method for objectionable text classification that integrates a multi-feature graph attention mechanism is proposed. The main body of the model is divided into two feature layers. The first feature layer is a multi-module graph attention mechanism layer. Leveraging the transformer's ability to achieve fast parallelization using multiple heads, the multi-module graph attention mechanism is parallelized to extract text graph features from different subspaces and multiple dimensions. The second feature layer is a Bi_GRU layer. Based on the multi-module graph attention mechanism, Bi_GRU is parallelized to learn the continuous semantic information of the text, compensating for the shortcomings of graph neural networks. Finally, a sensitive text classification dataset is used for testing and comparison with other sensitive text classification methods, demonstrating the effectiveness of the graph convolution classification model in classifying objectionable text on social media.

[0071] Firstly, Figure 2 It is shown that an embodiment of the present invention discloses a bad text classification method integrating a multi-feature graph attention mechanism, such as Figure 2 As shown, the method includes the following steps:

[0072] S101, in response to preprocessing the acquired text data;

[0073] Specifically, this paper starts from the semantic understanding of bad texts and builds a model MGAT_BiGRU suitable for bad short text classification based on GAT (Graph Attention Transformer) and Bi_GRU (Bi Gated Recurrent Unit). The overall framework of the model is as follows Figure 3 shown.

[0074] Furthermore, preprocessing can not only reduce the interference of useless words on the model results, but also reduce redundant edges and points when constructing the text graph in the later stage. In this embodiment, the preprocessing steps of the present invention are mainly two steps: removing interference words and formatting sentences. Among them, interference words mainly refer to special identifiers such as emoticons, "￥", "&", etc., and this paper adopts string matching to remove them; formatted sentences refer to the inherent sentences in social texts, such as sentences such as "xxx joined the group chat", "xxx added you as a friend", etc., and this paper adopts regular expressions to remove them.

[0075] S102, further processing the acquired text data to construct a text graph;

[0076] Specifically, in the embodiment, first, the text is segmented and tagged with parts of speech, the segmented result is converted into a low-dimensional, dense word vector through a pre-training model, after part-of-speech analysis, redundant words with less influence on classification effect such as conjunctions, auxiliary words and adverbs are removed, and nouns, verbs, adjectives and adverbs are taken as graph nodes;

[0077] Secondly, a morphological library required by the word is constructed, and the present application uses a collected bad text morphological library, wherein the types of morphological words mainly include morphological words, phonetic words, split words and synonyms;

[0078] Finally, a text graph of text words is obtained using a sliding window, wherein the rule for constructing edges is to use a fixed-size sliding window on the text sentence, and if two words are in the same window, an undirected edge is added; the bad words and their morphological words are directly added with undirected edges.

[0079] S103, learning features of the text graph by using a graph attention mechanism;

[0080] Specifically, the process of feature aggregation of the graph attention mechanism is as follows:

[0081] Given a node i, its neighborhood is , let node First, the attention coefficient between the node pair (i, j) is calculated:

[0082] ;

[0083] wherein, is a trainable matrix, and the linear transformation matrix of the concatenated feature node is multiplied by the learnable attention function to perform dot product operation, and then the obtained node is activated by using a nonlinear activation function to obtain the original attention score, wherein represents any nonlinear activation function.

[0084] Then, the original attention scores of all incoming edges in the neighborhood of the node are normalized to obtain the attention aggregation coefficient; the neighborhood node features are weighted and summed to obtain the updated embedding representation of the node i:

[0085] ;

[0086] Finally, in order to promote the graph attention mechanism to more stably aggregate features, K multi-head attention mechanisms are concatenated to enrich the extraction ability of the model, and if the layer is the last layer of the model, the average value of the K heads is used to represent the node:

[0087] .

[0088] S104, further embedding the graph attention mechanism into the Transformer network for parallel processing to obtain output, and simultaneously using the Bi_GRU neural network for parallel processing to obtain output; and

[0089] As a preferred embodiment, the graph attention mechanism is embedded into the Transformer network to achieve fast parallel execution of the graph attention mechanisms of multiple modules.

[0090] The present invention mainly includes two parts. The first layer is the Transformer layer that performs parallel operations on GAT. The present invention uses the Transformer encoding layer part, modifies the self-attention input part of the Transformer encoding layer part to GAT, and realizes parallel calculation of multiple GATs; the second layer is the fully connected layer and the activation layer. In order to accelerate the convergence of the model, the present invention retains the residual connection and normalization operation in the Transformer, and finally connects the normalized result to a Feedword layer, which is composed of a fully connected layer and a relu activation function to obtain the output of MGAT-Transformer. The specific model structure is as follows Figure 4 shown.

[0091] Specifically, the MGAT input consists of word embeddings and an adjacency matrix. The adjacency matrix represents information about adjacent words to the changed word. Multiple parallel GATs are used to learn text graph features, and the extracted features are concatenated horizontally. GAT can integrate word embedding information after deformation and sound change. Using the parallel computing power of the Transformer, the GAT learns word associations from different dimensions. The specific formula is as follows:

[0092] ;

[0093] Where n represents the number of GAT modules.

[0094] The results of parallel Transformer concatenation are connected to a feedforward neural network layer. The feedforward neural network mainly consists of two fully connected layers and a Relu activation function. This improves the model's performance by transforming the output space.

[0095]

[0096] Furthermore, since graph data is inherently discrete, it is impossible to model the sequential relationships of sentences or obtain rich contextual semantic information of text sentences. As a time series data processing model, recurrent neural networks have a natural advantage in sequence-related text processing tasks. Therefore, Bi_GRU is parallelized based on MGAT.

[0097] On the one hand, the Bi_GRU neural network can model the temporal information of text, overcoming the limitation of graph neural networks in acquiring contextual semantic information. On the other hand, the combination of GAT and Transformer can deeply mine the characteristics of text graphs while overcoming the shortcomings of recurrent neural networks, which are unable to achieve parallelization and have difficulty capturing long-range dependencies. The two components complement each other.

[0098] S105. Concatenate the outputs of MGTransformer and Bi_GRU to obtain the final text classification result.

[0099] Specifically, first, the output of MGTransformer is horizontally spliced ​​with the output of Bi_GRU to obtain the final text vector representation. Then the obtained text vector is connected to a full connection and log_softmax function to predict the classification result to obtain the final category confidence score. Finally, the confidence score is used to obtain the classification label corresponding to the text, and the classification result of each text is recorded.

[0100]

[0101] in, , the output value is the result of the entire model's prediction of the input text.

[0102] In the second aspect, the embodiment of the present invention also discloses a bad text classification system integrating a multi-feature graph attention mechanism, such as Figure 5 As shown, the system specifically includes: a preprocessing module 61, a text graph construction module 62, a feature learning module 63, an MGTransformer module 64, a Bi_GRU neural network module 65 and an output splicing module 66.

[0103] In another specific embodiment, a preprocessing module 61 is used to preprocess the acquired text data in response; a text graph construction module 62 is used to further process the acquired text data to construct a text graph; a feature learning module 63 is used to perform feature learning on the text graph using a graph attention mechanism; an MGTransformer module 64 is used to embed the graph attention mechanism into the Transformer network for parallel processing to obtain output; a Bi_GRU neural network module 65 is used to perform parallel processing to obtain output; and an output splicing module 66 is used to splice the outputs of the MGTransformer and Bi_GRU to obtain the final text classification result.

[0104] This paper addresses the task of classifying offensive text on social media. By combining graph attention mechanisms, Transformers, and Bi_GRU models, the paper builds a short text classification model based on the characteristics and semantics of offensive text. The main idea is to first model the text as a discretized graph structure. By adding inflected forms of offensive words as free nodes to the text graph, the model's recognition rate for inflected offensive words is improved. By leveraging the parallel advantages of Transformers, multiple GAT modules are used in parallel to extract text graph features, improving the model's training efficiency. Finally, the model is trained in parallel and incorporates a Bi_GRU model to obtain more sequence information between words, compensating for the inability of graph neural networks to obtain text sequence information.

[0105] This model provides a new approach to applying deep learning to the classification of objectionable text, addressing the poor generalization and low precision of deep learning models in this task. However, the technical solution of this invention still has some shortcomings, such as the classification results being overly dependent on the collection of objectionable text lexicons and the difficulty in defining word edge weights. These issues require further research and resolution.

[0106] Reference below Figure 6 , which shows an electronic device (eg Figure 1 A schematic structural diagram of a computer device 700 (a server or terminal device as shown). Figure 6 The electronic device shown is only an example and should not limit the functions and scope of use of the embodiments of the present invention.

[0107] like Figure 6 As shown, computer device 700 includes a central processing unit (CPU) 701 and a graphics processing unit (GPU) 702, which can perform various appropriate actions and processes according to programs stored in read-only memory (ROM) 703 or programs loaded from storage 709 into random access memory (RAM) 706. Various programs and data required for the operation of device 700 are also stored in RAM 704. CPU 701, GPU 702, ROM 703, and RAM 704 are connected to each other via bus 705. Input / output (I / O) interface 706 is also connected to bus 705.

[0108] The following components are connected to the I / O interface 706: an input section 707 including a keyboard, a mouse, and the like; an output section 708 including, for example, a liquid crystal display (LCD), and speakers; a storage section 709 including, for example, a hard disk; and a communication section 710 including, for example, a network interface card such as a LAN card or a modem. The communication section 710 performs communication processing via a network such as the Internet. A drive 711 may also be connected to the I / O interface 706 as needed. Removable media 712, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is installed in the drive 711 as needed, so that computer programs read from the removable media can be installed in the storage section 709 as needed.

[0109] In particular, according to embodiments disclosed herein, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed herein include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for executing the methods illustrated in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 710 and / or installed from removable media 712. When executed by central processing unit (CPU) 701 and graphics processing unit (GPU) 702, the aforementioned functions defined in the methods of the present invention are performed.

[0110] It should be noted that the computer-readable medium described in the present invention may be a computer-readable signal medium or a computer-readable medium, or any combination thereof. Computer-readable media may include, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor devices, apparatuses, or components, or any combination thereof. More specific examples of computer-readable media may include, but are not limited to, an electrical connection having one or more conductors, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution device, apparatus, or component. In the present invention, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such a propagated data signal may take a variety of forms, including, but not limited to, electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution apparatus, device, or device. Program code embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wireline, optical cable, RF, or any suitable combination thereof.

[0111] Computer program code for performing the operations of the present invention may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0112] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the devices, methods and computer program products according to various embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based device that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0113] The modules involved in the embodiments of the present invention may be implemented in software or hardware, and may also be set in a processor.

[0114] As another aspect, the present invention further provides a computer-readable medium, which may be included in the electronic device described in the above embodiment; or may exist independently without being assembled into the electronic device. The computer-readable medium carries one or more programs, and when the one or more programs are executed by the electronic device, the electronic device: pre-processes the acquired text data in response; further processes the acquired text data to construct a text graph; uses a graph attention mechanism to learn features of the text graph; further embeds the graph attention mechanism into a Transformer network for parallel processing to obtain output, and simultaneously uses a Bi_GRU neural network for parallel processing to obtain output; and concatenates the outputs of the MGTransformer and Bi_GRU to obtain a final text classification result.

[0115] The above description is merely an illustration of preferred embodiments of the present invention and the technical principles employed. Those skilled in the art should understand that the scope of the present invention is not limited to technical solutions formed by specific combinations of the aforementioned technical features. It also encompasses other technical solutions formed by any combination of the aforementioned technical features or their equivalents, without departing from the aforementioned inventive concept. For example, a technical solution formed by replacing the aforementioned features with (but not limited to) technical features with similar functions disclosed in this invention.

Claims

1. A bad text classification method integrating multi-feature graph attention mechanism, characterized by: The method comprises the following steps: In response to preprocessing the acquired text data; The acquired text data is further processed to construct a text graph; Using a graph attention mechanism to perform feature learning on the text graph; The graph attention mechanism is further embedded into the Transformer network for parallel processing to obtain output, and the Bi_GRU neural network is also used for parallel processing to obtain output; as well as Concatenate the outputs of MGTransformer and Bi_GRU to obtain the final text classification result; The graph attention mechanism is embedded into the Transformer network for parallel processing to obtain output, specifically including: The first layer is the Transformer layer that performs parallel operations on the GAT: using the Transformer encoding layer, the self-attention input part of the Transformer encoding layer is modified to the GAT, realizing parallel calculation of multiple GATs; ； Where n represents the number of GAT modules; The second layer is a fully connected layer and an activation layer. The residual connection and normalization operations in the Transformer are retained. The normalized result is connected to a Feedword layer, which consists of a fully connected layer and a ReLU activation function to obtain the output of the MGAT-Transformer. ； The output of MGTransformer and Bi_GRU are concatenated to obtain the final text classification result, which specifically includes: The output of MGTransformer is horizontally concatenated with the output of Bi_GRU to obtain the final text vector representation; The obtained text vector is then connected to a full connection and log_softmax function to predict the classification result and obtain the final category confidence score; Finally, the confidence score is used to obtain the classification label corresponding to the text, and the classification result of each text is recorded; ； in, , the output value is the result of the entire model's prediction of the input text.

2. The bad text classification method integrating multi-feature graph attention mechanism according to claim 1 is characterized in that: Preprocessing the acquired text data includes removing noise words and formatting sentences, specifically including: Use string matching to remove noise words containing special identifiers; Regular expressions are used to remove the inherent language sentences in social texts.

3. The bad text classification method integrating multi-feature graph attention mechanism according to claim 1 is characterized in that: The acquired text data is further processed to construct a text graph, including: Perform word segmentation and part-of-speech tagging on the acquired text: The segmentation results are converted into low-dimensional and dense word vectors through a pre-trained model. After part-of-speech analysis, redundant conjunctions, auxiliary words, and adverbs are removed, and nouns, verbs, adjectives, and adverbs are used as graph nodes. Construct a vocabulary of inflected words: Use the pre-collected vocabulary of inflected words from bad texts. The types of inflected words mainly include shape-shifted words, sound-shifted words, split words, and synonyms. Use a sliding window to obtain a text graph of text words: Use a fixed-size sliding window on the text sentence as a rule for constructing edges. If two words are in the same window, an undirected edge is added. Undirected edges are directly added between bad words and their inflections.

4. The bad text classification method integrating multi-feature graph attention mechanism according to claim 1 is characterized in that: Using the graph attention mechanism to perform feature learning on the text graph includes using the graph attention mechanism to perform feature aggregation, specifically including: Given node i, its neighborhood is , set node , first calculate the attention coefficient between the node pair (i, j): ； in, represents any nonlinear activation function, Is a trainable matrix, splicing feature nodes The linear transformation matrix and the learnable attention function Do a dot product operation and then use a nonlinear activation function Activate the obtained nodes to get the original attention score; Then normalize the original attention scores of all incoming edges in the node neighborhood to obtain the attention aggregation coefficient, and perform weighted summation on the neighborhood node features to obtain the updated embedding representation of node i: 。 5. The bad text classification method integrating multi-feature graph attention mechanism according to claim 4 is characterized in that: Also includes: To promote the graph attention mechanism to perform feature aggregation more stably, K multi-head attention mechanisms are spliced ​​to enrich the extraction ability of the model. If a layer is the last layer of the model, the node is represented by taking the average value of K heads: 。 6. A bad text classification system integrating multi-feature graph attention mechanism, characterized by: The method according to any one of claims 1 to 5, wherein the system specifically comprises: a preprocessing module for preprocessing the acquired text data in response; A text graph construction module is used to further process the acquired text data to construct a text graph; A feature learning module, configured to perform feature learning on the text graph using a graph attention mechanism; The MGTransformer module is used to embed the graph attention mechanism into the Transformer network for parallel processing to obtain output; Bi_GRU neural network module, used to use Bi_GRU neural network to perform parallel processing to obtain output; The output concatenation module is used to concatenate the outputs of MGTransformer and Bi_GRU to obtain the final text classification results.

7. An electronic device comprising: one or more processors; a storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images