Cyberbullying detection method based on multi-dimensional feature fusion network of attention mechanism

Abstract

The application provides a multi-dimensional feature fusion network bullying detection method based on an attention mechanism, which comprises the following steps: in the semantic level of comments in social media, in order to reduce resource consumption and solve the problem that the existing detection method using a converter type model needs to truncate a long conversation, thereby affecting the detection accuracy, the conversation is divided into individual words to avoid length limitation, and text features are extracted from the word and sentence angles through a hierarchical architecture and an attention mechanism; in combination with the habitual behavior of cyberbullying, the historical speech of a user is taken as a kind of user behavior habit information and is integrated into semantic features; in the comment time level, a time graph is constructed for each social media conversation, which is used for modeling the repetitive features in the time dimension in the user interaction process; the semantic features and the time features are fused through an attention mechanism to obtain vector representations of the comprehensive semantic features and the time features; and finally, a normalization exponential function layer is used for classification.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, especially to the technical field of cyberbullying detection and the field of natural language processing. The main core is to use user comment data in social media platform, use neural network to construct features for comments, train multi-feature fusion network model, and finally use the model to detect cyberbullying behavior. BACKGROUND

[0002] With the development of Internet technology, the number of social media users is increasing. Information exchange and social activities are becoming more and more frequent in the network space. The popularity of the Internet and the rapid rise of social media provide people with more extensive communication channels. However, the development of the Internet has also brought a series of serious challenges, one of which is the significant increase in cyberbullying. Cyberbullying, with its diversification and concealment, has gradually become a serious challenge in the digital society. In addition, network content is exposed to a large number of audiences and is difficult to delete, and information can be forwarded, liked or shared, which greatly increases the impact of offensive or harmful information. Cyberbullying is usually defined as a group or individual using electronic communication to repeatedly or continuously target victims who cannot easily defend themselves, and cyberbullying involves forms such as digital platforms, such as social networking sites, chat rooms, blogs, instant messaging applications, and text messages, in the form of text, pictures or videos. Cyberbullying cases have high legal protection costs, and there are often news reports of cyberbullying on the Internet, but only a small number of victims choose to use legal weapons to protect themselves. Therefore, it is of great significance to research automated cyberbullying detection tools to accurately and effectively detect cyberbullying and promptly stop cyberbullying against victims.

[0003] Cyberbullying methods based on traditional machine learning have limited effectiveness in dealing with complex and subtle language expressions, especially in understanding context and implied meaning. Offensive content often does not directly appear obvious derogatory or insulting keywords. Instead, these bullying behaviors may be carried out through hints, sarcasm, or implicit statements, which makes it difficult for keyword-based feature extraction methods to effectively capture. In addition, traditional machine learning methods often require more manual feature engineering when dealing with unstructured and large-scale data, which limits their application effectiveness in dynamic and rapidly developing network environments.

[0004] With the use of deep learning architecture, other more complex features such as polymorphism, dynamics, hierarchy, and interaction have also been studied. Cyberbullying often occurs in unstructured text. Deep learning models can automatically learn and extract complex features from text data, but the obvious repetition of cyberbullying makes it difficult to capture the repetition of cyberbullying behavior by directly applying deep learning to cyberbullying detection.

[0005] It is important to understand how users interact with each other in social media conversations. One important difference between cyberbullying and offline bullying is its persistence and repetition. Psychological and social science research has shown that cyberbullying is carried out repeatedly by one or more individuals against a victim. Research on user interaction allows the present method to describe repetition through content and temporal analysis. Recent research results show that in bullying conversations, user interaction is a strong signal for detecting cyberbullying instances, and the intensity of user activity in bullying conversations is essentially different from that in non-bullying conversations. The development of multi-feature cyberbullying detection greatly improves the detection effect, but in feature representation learning and feature vector fusion, there are still the following challenges: the fusion strategy of feature data is too simple; and the mutual relationship between different feature data is not fully considered in the feature fusion process. Therefore, there is still a wide research space in the fusion representation of multi-feature data. SUMMARY

[0006] The present invention aims to solve the above problems and provide an effective cyberbullying detection method.

[0007] A multi-dimensional feature fusion cyberbullying detection method based on an attention mechanism is proposed to solve the widespread cyberbullying behavior on social media platforms. The goal of the invention is to accurately and timely detect bullying behavior on social media and purify the network space.

[0008] The present invention innovatively proposes a multi-dimensional feature fusion cyberbullying detection method based on an attention mechanism, which mines the semantic information and temporal information of comment text in multiple dimensions. The method mainly includes three parts: first, in the semantic layer of comments in social media, to address the problem of existing detection methods using Transformer-based models that need to truncate long conversations to reduce resource consumption, which affects detection accuracy, the present method first decomposes the conversation into individual words to avoid length restrictions, and extracts text features from both word and sentence perspectives through a hierarchical architecture and attention mechanism. At the same time, the present method incorporates the user's historical statements as a kind of user behavior habit information into the semantic features, taking into account the habitual nature of cyberbullying behavior. Second, in the comment time layer, the present method constructs a time graph for each social media conversation to model the repetitive features in the temporal dimension of user interaction. Finally, the semantic features and temporal features are fused through an attention mechanism to obtain a vector representation of the comprehensive semantic features and temporal features. The final classification is performed through a normalization exponential function (softmax) layer.

[0009] To achieve the above purpose, the cyberbullying detection method of the present invention comprises:

[0010] (1) In order to reduce resource consumption and avoid the problem of affecting the detection accuracy caused by truncating long sessions, the method proposes a semantic context modeling module based on a hierarchical attention mechanism, which avoids length limitation by decomposing the session. Specifically, the method uses a pre-trained transformer-based bidirectional encoder representation model (BERT) for word embedding, and then uses a hierarchical architecture and attention mechanism to form sentences from words, and then form sessions to build text features. This hierarchical architecture gradually extracts social media session features. The method solves the problem that existing detection methods using Transformer-based models are difficult to handle long inputs.

[0011] (2) In view of the repetition of cyberbullying behavior, the method analyzes the Instagram dataset and the Twitter dataset to explore the temporal attribute pattern of cyberbullying. The method designs a time series graph interaction learning module to model the time proximity of user activities through a graph neural network.

[0012] (3) In view of the fusion of semantic features and time features, the method uses a multi-channel information cross-encoding mechanism of the attention mechanism to fuse semantic features and time features to obtain a vector representation of the integrated semantic features and time features. And through a multilayer perceptron for classification prediction, the method achieves high effectiveness of cyberbullying behavior detection. The method conducts experiments on the Instagram and Twitter datasets and compares various baseline models. The method has obtained good accuracy and recall rate, which confirms that it can effectively detect cyberbullying on social networking platforms. BRIEF DESCRIPTION OF DRAWINGS

[0013] Figure 1 The overall flowchart of the present application;

[0014] Figure 2 Schematic diagram of using BERT for word embedding;

[0015] Figure 3 Architecture diagram of multi-channel cross-encoding mechanism;

[0016] Figure 4 Ablation analysis sample of a semantic analysis module;

[0017] Figure 5 Comparison chart of three fusion methods experiments. DETAILED DESCRIPTION

[0018] As Figure 1As shown, a multi-dimensional feature fusion network bullying detection method based on attention mechanism. The multi-feature fusion network bullying detection task is defined as: given a session corpus C, a related label set Y, a user group U and a historical comment H, the goal of this work is to classify social media sessions into bullying / non-bullying categories using multi-feature information extracted from comments, time, users and user historical speeches. For a social media platform, it constitutes a session corpus C, which consists of N sessions, each session consists of n comments, denoted as S={c1, c2, …, c n} where c i is the i-th comment in the session. Each session has its label Y, indicating whether the session is a bullying session y=1 or a non-bullying session y=0. User u makes a comment at time t i In addition, user u also has a set of historical comments, which are connected into a paragraph by this method, consisting of q words

[0019] The present application comprises three parts of semantic context modeling module, time sequence diagram interaction learning module and feature fusion module, referring to Figure 1 The present application based on attention mechanism multi-dimensional feature fusion network bullying detection method comprises the following steps:

[0020] Step 1, use the pre-trained model BERT to extract basic semantic feature information, and convert the p-word comment c= Into a high-dimensional vector through text word embedding. The process is shown in Figure 2 As shown in the figure, the document composed of k words is inputted Through the BERT model, the corresponding word vector B={b1, b2, …, b p ,} is obtained.

[0021] Step 2, input the word vector obtained in the above step into the bidirectional gated recurrent neural network (Bi-GRU) to extract more complete context information. The character-level gated recurrent unit (GRU) layer can encode the long-term contextual dependency between words. Compared with LSTM, GRU has a simpler structure, only containing two gating units of update gate and reset gate, so it has fewer parameters, and the training and inference speed is slightly faster. A set of bidirectional GRU is used to model the word sequence from both forward and backward directions. The i-th word vector feature t i The forward and backward hidden state vectors are obtained by bidirectional GRU, and then spliced in order. Such splicing operation makes the output of each time step contain information in both forward and backward directions. As shown in equation (1).

[0022] ​

[0023] Step 3, the vector obtained in the previous step is input into the attention layer. The hidden state output by the bidirectional GRU in the previous step contains the information of the entire input sequence, but not all information is equally important for the cyberbullying detection task. Through the attention mechanism, the model can automatically learn the importance of different positions. The deep learning model based on attention mechanism can extract important features in the text by training word vectors to identify the importance of words in the text. In this layer, the role of the attention mechanism is to find the key words in the text, which are often important information for classification. For the jth review, the review learned through the attention mechanism is j As shown in equations (2) to (4).

[0024] d i = w i tanh(v T x t i + b) (2)

[0025]

[0026] Step 4, the vector obtained in the previous step is input into the sentence-level bidirectional GRU to model the review in chronological order. The Bi-GRU output of the jth review is represented as:

[0027] Similarly, the method connects the forward hidden state and the backward hidden state to obtain the vector of sentence j, The intermediate layer representation of the jth review in the social media conversation can be obtained using equation (5).

[0028]

[0029] Step 5, the vector obtained in the previous step is input into the sentence-level attention layer, not all reviews contribute to cyberbullying. The main purpose of this layer is to identify the contribution of reviews to the detection task at the sentence level. Through the attention mechanism, find the key sentences in the review. These sentences are important for the classification of the cyberbullying detection task. The semantic vector r j .

[0030] Step 6, User history review encoding. Feature extraction is performed on each review by incorporating the language behavior reflected in the user's history reviews. The number of reviews a user posts for a post is limited, and the purpose of modeling the user's history reviews is to better capture the user's language behavior and the personality traits reflected in their history reviews. Research has found that the bullying behavior of bullies is strongly associated with hostility and personal character. Bullies are more emotional and often have irrational behavior, so they are more likely to participate in online bullying activities. In addition, research has found that using historical information to predict user personality on social media is effective. First, the learned user features are linked to the corresponding reviews in the current session. Then, the same semantic context modeling method described above is used to model the user's history. Specifically, given the history of n users in the session as input, where each history review contains q words, the output context history sequence is {h1, h2, …, hn}. The final user representation is then calculated as follows: n}.

[0031] When examining the user's history reviews, an important phenomenon is that some users exhibit similar language behavior over time. For example, a user always bullies others with aggressive language when participating in online discussions. In contrast, users who occasionally bully others may only make one offensive comment. In this case, the history reviews may not have enough information to identify the user's bullying tendency in the current session. Considering the relative importance of historical information for each user, this method uses a gating mechanism here to automatically weigh the importance of reviews in the current session and the user's history. For the jth review in the session s, the combined user representation u j is represented as (6) to (7).

[0032] u j = β j r j + (1 - β j ) h j (6)

[0033] β j = σ(W h r j + W k h j + b c ) (7)

[0034] where W h , W k , b c are trainable parameters, σ is a sigmoid activation function, and β j controls the weight.

[0035] So after getting the context history sequence, the user's history utterance and utterance in the conversation are input into the attention module, and the user's history representation h and the current semantic representation are aggregated into a unified user representation u j , for the work below.

[0036] Step 7, feature extraction of user's interaction in the conversation by using graph neural network. This method represents the social network graph composed of user's post and user's reply as G=(V,E), where the node set is V={v1,v2,…,v m ,} and m is the number of nodes, each node represents a user. The edge set of the social network is E={e1,e2,…,e n ,} and n is the number of edges, each edge represents the time interval of user's comment, and the two ends of the edge are the comment initiator and the commented user. Define T v and T e as the representation set of node type and edge type, and define two surjective relationship functions f v :V→T v and f e :V→T e . For each node v i ∈V, f v (v i )∈T v , and for each edge e i ∈E, f e (e i )∈T e . Then, the social network graph is represented by a weight matrix A, where if A ij is not zero, it means that there is an edge from node v i to v j . Finally, a feature matrix is used to represent the features of each node in the social network graph, where d0 is the input feature dimension. Assuming that when a comment from v i to v j is sent by a user, the two nodes v i and v j are associated, and a directed edge is constructed in the order of sending the comment, which is called the association path. Obviously, at t=0, the weight matrix A is a zero matrix. Then, when two nodes v i to v j comment, an edge A ij is established, which is the time interval of the two comments.

[0037] This method first constructs a temporal graph, where nodes are timestamps of user comments and edges are time intervals between comments. Each comment in the graph is intrinsically connected, thus they are not constrained by sparse and uneven response relationships between users. The time interval between two adjacent user comments reflects the temporal characteristics of the session, i.e., temporal proximity, which is automatically determined and propagated through the edges of the temporal graph. The core of the temporal graph interaction learning module is to leverage GAT (Generative Atlas of Comments) to aggregate neighbor information (i.e., comments) based on their proximity.

[0038] User interactions are implicitly learned by capturing comment interactions in a temporal graph. Each node, except the root node, has a parent node, where this method establishes connectivity between two comments. In the temporal graph, comments are treated as graph nodes, and the time intervals between them are considered edges. The different saliency levels of adjacent nodes are weighted by their edge weights. The GAT network encodes the temporal levels between the center node and its neighbors by incorporating the temporal correlation between the center node and its neighbors into the calculation of edge weights. Then, neighboring nodes are aggregated based on their edge weights to propagate the interaction signals to the center node. Furthermore, this method considers self-propagation to preserve the original content information of each node. Multiple layers are stacked. For the j-th comment in the temporal graph, this method denotes its node vector as g. j The calculation is shown in (8).

[0039]

[0040] In the formula, N j Let W be the set of adjacent nodes of the j-th comment. c t is the transformation matrix of the graph network k,j =t j -t k This is the time interval between two comments. π(t) k,j ) indicates the connection node v j and v k The weight of the edge. It is a learnable factor that controls the relative contribution of each neighbor to the central node.

[0041] Temporal proximity is a key characteristic of repetition. Cyberbullying is a continuous phenomenon, not a one-off event. By capturing the intervals between each pair of comments, a steady stream of replies and sudden bursts of comments can be identified. Therefore, the temporal progression factor is further integrated into the constructed timeline, as shown in Equation (9).

[0042] π time (t k，j ) = W t t k,j =W t (t j -t k (9)

[0043] where W is the time transform coefficient, t k,j is the time interval.

[0044] Step 8, the method proposes a multi-channel cross-encoding mechanism to fuse the features of the two layers. Let the model learn which feature to focus on by itself, use the multi-head attention mechanism to extract information from different feature spaces, and then use the dot product attention mechanism to multiply each feature's attention score with the respective feature vector and then add it up for weighted fusion. As shown in Figure 3 .

[0045] The geometric meaning of using vector dot product in this method is that the projection of vector x in the direction of vector y multiplied by vector y can reflect the similarity of the two vectors. The larger the vector dot product result, the more similar the two vectors. The following will introduce in detail how to use the multi-head attention mechanism to assign weights to the text features and time features.

[0046] The multi-head attention mechanism divides the model into multiple heads, forming multiple subspaces, so that the model can pay attention to different aspects of information. Specifically, the input u is divided into m parts, and the dot product is calculated with the weight W i to create an input vector W i u, and form {Q i , K i , V i}, it is found through experiments that m = 8 is a suitable value; then the scaled dot product attention method is used to calculate the attention weight of each vector. Next, the Softmax operation is performed, which normalizes the similarity of multiple keys corresponding to a query. After normalization, each value is a weight coefficient greater than 0 and less than 1, and the sum is 1. This result can be understood as a weight matrix. Use the just obtained weight matrix to multiply V to calculate the weighted sum. The whole process is actually encoding the input data, and the basis for encoding is the correlation between the elements in the data. In order to enhance the ability of information extraction, the above encoding process is repeated multiple times in parallel, and the results of multiple encodings are added together to get the final output.

[0047] In this way, the semantic vector matrix u goes through the attention mechanism to get X u , and g goes through the attention mechanism to get X g . Next, the two vectors are compressed from multi-dimensional to one-dimensional. As shown in formulas (10) and (11). Then add the two one-dimensional vectors obtained to get the fusion vector Z. Finally, input Z into the Softmax classification module for classification. As shown in formula (12).

[0048] Z u = X u• flatten() (10)

[0049] Z g = X g • flatten() (11)

[0050] Z = Z u + Z g (12)

[0051] Step 9, the purpose of the conversation classification module is to divide the social media conversation into bullying conversation and non-bullying conversation. In order to expand the task of multi-classification of the conversation in the future, the Softmax classifier is used, which is a method of mapping the output of neural network to probability distribution. Let the training set {(x 1 , y 1 ),…,(x m , y m )} and its class label set y i = {1, 2,…, k}, k is the number of labels. Softmax uses the hypothesis function to process each input x, evaluates the probability value p(y = j | x) of belonging to each class j, selects the maximum probability class, and this class is the final classification of x. θ is the parameter vector of the model, and the cost function is shown in equation (13).

[0052]

[0053] where 1{expression} ∈ {0, 1}, the function determines the result according to the true or false value of the logical expression, and the result is 1 when the expression is true, otherwise 0. The latter part of the formula is a decay factor, which is used to avoid the local optimum of the former part. When the decay factor is added and the parameter λ > 0, equation (13) is a strict convex function. Next, the gradient descent method is used to derive it, and the derivative formula of the cost function is shown in equation (14).

[0054]

[0055] The category distribution probability matrix is obtained, which represents the evaluation probability of the text x belonging to each category. The Softmax function determines that the text belongs to the category label with the maximum probability, and thus the whole text classification process is completed. The predicted label y is obtained, where y = 1 represents bullying.

[0056] The effectiveness of the network bullying detection method is verified by combining real data.

[0057] The dataset information is shown in Table 1. These datasets are generated by randomly sampling from the Twitter dataset. For the users appearing in the dataset, we obtained their follow relationships through the API provided by Twitter on September 10, 2018.

[0058] Table 1 Instagram and Twitter dataset information

[0059]

[0060] The method model is mainly built and trained using the Keras deep learning framework of TensorFlow. The sklearn library is used for data mining and data analysis, and its built-in SVM, logistic regression, random forest, etc. algorithms are used for comparative experiments. For each dataset, the method randomly extracts 80% of the conversations for training, 10% for validation, and 10% for testing. The method assumes that each conversation is independent and does not consider cross-conversation correlation. The conversation length is set to 140, and the sentence length is set to 30. If it exceeds, it is cropped, and if it is insufficient, it is padded with zeros. The method uses a pre-trained BERT model to generate 768-dimensional word vectors as text input. To prevent the model from overfitting, the method uses a Dropout strategy with a setting of 0.3, which randomly inactivates 30% of the neurons. The optimizer is the Adam optimizer, the learning rate is set to 0.00002, and the learning rate decay weight is set to 0.01. The number of iterations is set to 5. The number of samples processed per batch is set to 16. Each experiment is repeated 5 times, and its average performance and standard deviation are recorded.

[0061] The method uses a variety of evaluation indicators in the cyberbullying detection task, including precision, recall, F1 score, and AUC (Area Under the ROC Curve).

[0062] Table 2 Classification problem prediction results and actual label combination

[0063]

[0064] True positive (TP) indicates that the actual label is true and the prediction is positive; false positive (FP) indicates that the actual label is false and the prediction is negative; true negative (TN) indicates that the actual label is true and the prediction is negative; false negative (FN) indicates that the actual label is false and the prediction is negative.

[0065] Precision: refers to the proportion of samples that the model predicts as positive examples, which are actually positive examples. The calculation formula is shown in (15).

[0066]

[0067] Recall: also known as the recall rate, refers to the proportion of all actual positive examples that the model successfully predicts as positive examples. The calculation formula is shown in (16).

[0068]

[0069] F1 value: is the harmonic mean of precision and recall, used to evaluate the performance of the model. The calculation formula is shown in (17).

[0070]

[0071] The F1 value combines precision and recall to more comprehensively evaluate the performance of the model.

[0072] AUC: used to evaluate the overall performance of the classification model at different thresholds. Usually draw the ROC curve

[0073] (Receiver Operating Characteristic curve) and calculate the area under it. The ROC curve takes the false positive rate as the horizontal axis and the true positive rate as the vertical axis. The value of AUC ranges from 0 to 1, the larger the value, the better the performance of the model. The English name of this method is A Multi-dimensional Comment feature Fusion cyberbullying detection method based on Attention mechanism, hereinafter referred to as MCFA.

[0074] Table 3 Experimental results of different methods on Instagram dataset

[0075] Precision (%) Recall (%) F1 AUC LR 65.34 50.12 60.18 71.46 SVM 68.16 56.56 61.73 75.94 XGBoost 70.25 68.46 69.45 79.31 TextCNN 77.47 70.15 73.91 84.55 BiLSTM 78.18 65.74 72.33 81.78 Soni & Sign 76.91 74.35 75.23 82.16 HANCD 77.43 81.58 79.47 89.43 MCFA 81.63 82.64 81.27 90.64

[0076] Table 4 Experimental results of different methods on Twitter dataset

[0077] Precision (%) Recall (%) F1 AUC LR 54.19 52.47 53.34 65.22 SVM 58.32 60.38 59.64 72.57 XGBoost 68.64 64.26 66.51 73.75 TextCNN 71.46 61.72 67.81 76.34 BiLSTM 70.42 62.81 66.34 80.16 Soni & Sign 72.96 64.48 67.17 81.74 HANCD 75.43 72.71 73.16 80.43 MCFA 80.13 74.34 76.83 82.48

[0078] Compared with other network bullying baseline methods considering time analysis, the proposed method finds that MCFA can achieve the best performance in most cases under different data sets and training proportions. This is because the semantic modeling module based on BERT can understand the semantic information of the conversation from the perspective of words and sentences, and effectively model user interactions through attention neural networks. And through the attention mechanism, it can effectively combine the semantic information in the conversation and the time characteristics of the conversation. The joint modeling of the two aspects of bullying indication helps to better understand the repetitive characteristics of network bullying behavior. And while learning user interactions, the personality characteristics and language behavior of users are modeled through the user's historical comments.

[0079] Table 5 Experimental results of different methods on Instagram dataset and Twitter dataset

[0080]

[0081]

[0082] The experimental results obtained by training on the full dataset combined from the two datasets show that the results of all methods have 2%-5% accuracy improvement and 3%-6% recall improvement. This indicates the effectiveness of using larger datasets to further improve the performance of network bullying detection.

[0083] The proposed method selects a conversation with a length of 1845 words in the Instagram dataset, as shown in Figure 4 If the network bullying event occurs outside the range of 512 words, and the hierarchical modeling module is not used, the comments related to network bullying are deleted after truncation of the long text, and the BERT model will judge the conversation as a non-bullying conversation. When using the hierarchical architecture to model the conversation, the conversation is first divided into words, and then the words are input into the bidirectional GRU and attention layer. Next, the conversation is input into the bidirectional GRU and attention layer, and finally into the fully connected layer for classification, and the classification of the conversation is bullying conversation, which is consistent with the artificial label.

[0084] To verify the effectiveness of the feature fusion module proposed by the proposed method, the feature fusion module is replaced with addition operation and concatenation operation respectively, and experiments are conducted. The experimental results are shown in Figure 5 Through experiments, it is proved that the feature fusion based on attention mechanism can perform more effectively in terms of accuracy, recall rate, F1 value and AUC, compared with the other two methods.

[0085] The selection of feature fusion modules has a significant impact on model performance, and attention-based feature fusion can fully consider the importance between different features, thus better extracting and utilizing feature information, making the model perform better when dealing with complex tasks. This result further verifies the effectiveness of attention mechanism in feature fusion.

Claims

1. An attention mechanism-based multi-dimensional feature fusion network bullying detection method, characterized in that, The method comprises the steps of: Step S1. Using a pre-trained transformer-based bidirectional encoder representation model (BERT) to extract basic semantic feature information and input into a bidirectional gated recurrent neural network (Bi-GRU) to extract more complete context information, and adding language behaviors reflected by user historical comments to extract features of each comment; Step S2. Using a graph neural network to extract features of user interactions in the conversation; Step S3. Fusing the features of the two dimensions to classify the conversation; In step S1, the method for using a pre-trained model BERT to extract basic semantic feature information is as follows: where c is an input; p is the number of words; k words as input BERT model to obtain the corresponding word vector B = {b1, b2, …, b p} In step S1, the method for inputting into a Bi-GRU bidirectional gated recurrent neural network to extract more complete context information and adding language behaviors reflected by user historical comments to extract features of each comment comprises: The word vector is input into a Bi-GRU bidirectional gated recurrent neural network, and a set of bidirectional gated recurrent neural networks (GRU) are used to model the word sequence from the forward and reverse directions: The i-th word vector feature obtains forward and reverse hidden state vectors through the bidirectional GRU, and then is spliced in order; Through attention mechanism learning of each comment, a comment with target features is obtained: The method for attention mechanism learning of each comment is as follows: e i = w i tanh(v T x t i + b) The reviews are modeled in chronological order using bidirectional GRUs, and the bidirectional GRU output of the jth review is represented as: and the vector of the jth review is read as: The intermediate layer representation of the comment j in the conversation is: determining a review having the target feature through an attention layer, and determining a vector r of the review j ; In step S1, the method for adding language behaviors reflected by user historical comments to extract features of each comment further comprises: Through modeling of the historical comments of the user, a language behavior and personal characteristics reflected by the historical comments of the user are captured: Given the history of n users in a session as input, where each history review contains q words, the output context history sequence is {h1, h2, …, h n} For the jth comment in the session s, the merged user representation u j : u j = β j ⊙r j +(1-β j )⊙h j β j = σ(W h r j + W k h j + b c ) where W h , W k , b c are trainable parameters, σ is a sigmoid activation function, and β j is a control weight. inputting the user's historical utterances and utterances in the conversation into an attention layer, aggregating the user's historical representation h and the current semantic representation into a unified user representation u j ; The user and the conversation of the user are constructed into a social network graph: G=(V,E) The node set is V = {v1, v2, ..., v}. m In a social network, m represents the number of nodes, and each node represents a user. The set of edges in the social network is denoted as E = {e1, e2, ..., e}. n ,}, where n is the number of edges, each edge represents the time interval of a user's comment, and the two ends of the edge are the user who initiated the comment and the user who was commented on; Definition T v and T e are the sets of representations of node types and edge types, respectively, while two mapping functions f v : V→T v and f e : E→T e are defined; for each node v i ∈V, f v (v i )∈T v , and for each edge e i ∈E, f e (e i )∈T e ; The social network graph is configured with a weight A: where if A ij is non-zero, then there is an edge from node v i to v j ; The feature representation of each node in the social network graph is: Computing a time interval A of a session of a user ij ; Further comprising: Based on time, the jth review in the user's session is calculated, and the vector of the jth review in the user's session is denoted as g j : where N j is the set of jth comment adjacent nodes, W c is the transformation matrix of the graph network, t k,j = t j -t k is the time interval between two comments; π(t k,j ) represents the weight of the edge connecting nodes v j and v k ​ Further comprising: The time factor is integrated into the constructed time graph, and is represented as: W t t k,j = W t (t j -t k ) where W is a time transform coefficient, t k,j is a time interval; In step S2, the method for using a graph neural network to extract features of user interactions in the conversation is represented as: Z u = X u · flatten() Z g = X g · flatten() In the formula, the semantic vector matrix u is obtained through attention mechanism to obtain X u , g is obtained through attention mechanism to obtain X g . 2.The cyberbullying detection method based on the attention mechanism-based multi-dimensional feature fusion network according to claim 1, characterized in that, In step S3, the method for fusing the features of the two dimensions to classify the conversation comprises: Compute fused vector Z: Z = Z u + Z g ; With a normalized exponential function (Softmax) classifier, given a training set {(x 1 ,y 1 ),…,(x m ,y m )} and its class label set y i ={1,2,…,k}, k is the number of labels; The cost function formula is: where θ is the parameter vector of the model; 1{expression}∈{0,1}, the function determines the result according to the true or false value of the logical expression, the result is 1 when the expression is true, otherwise 0; is a decay factor, and it is a strictly convex function when a decay factor is added and the parameter λ>

0. The gradient descent method is used to derive it: The category distribution probability matrix is obtained, which represents the evaluation probability of the text x belonging to each category. The Softmax function determines that the text belongs to the category label with the largest probability, and completes the classification process of the entire conversation.

Images