Microblog network behavior detection method based on feature fusion and SVM

Abstract

The invention discloses a microblog network behavior detection method based on feature fusion and SVM, and relates to the field of artificial intelligence, and the method comprises the steps: employing a TF-IDF algorithm to carry out the feature extraction of comment data, and obtaining a TF-IDF feature matrix reflecting the importance of vocabularies in a document; carrying out feature extraction on the comment data by utilizing a pre-trained BERT-Base-Chine model, so as to obtain a BERT feature matrix containing semantic information; splicing the TF-IDF dense feature matrix and the BERT feature matrix according to columns to form a fused comprehensive feature matrix; inputting the integrated feature matrix after fusion into a pre-trained dichotomy SVM (Support Vector Machine) model; inputting the recognized comprehensive feature matrix of the comment subset containing the target behavior content into a pre-trained multi-classification SVM model, performing multi-classification on each comment in the subset, and recognizing a network behavior type contained in each comment; aiming at the problem of low microblog multi-network behavior type detection precision in the prior art, the multi-network behavior detection precision is improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and more specifically, to a microblog network behavior detection method based on feature fusion and SVM. Background Art

[0002] At present, the detection of microblog network behavior mainly adopts methods such as keyword matching. Keyword matching is prone to high rates of missed detection and misjudgment, and it is difficult to accurately identify complex and changeable network behaviors. In addition, most of the existing research on microblog network behavior focuses on a single type of illegal behavior, such as identifying spam comments, and lacks systematic research and comprehensive detection of multiple network behaviors.

[0003] In recent years, the development of machine learning and natural language processing technology has provided new ideas for microblog network behavior detection. Some researchers have tried to use machine learning models such as support vector machines (SVM) and convolutional neural networks (CNN) to classify microblog comment content and have achieved certain results. However, most of these methods directly use the original comment text as input and do not fully explore the semantic information and structural features of the comment data, resulting in the need to improve the detection accuracy. In addition, for the various types of network behaviors in the microblog comment area, existing methods are difficult to perform comprehensive and fine-grained recognition. Summary of the invention

[0004] 1. Technical issues to be solved

[0005] In response to the problem of low accuracy in detecting multiple network behavior types on Weibo in the prior art, the present application provides a Weibo network behavior detection method based on feature fusion and SVM, which uses a pre-trained binary classification SVM model to identify whether a comment contains a target behavior, and further uses a multi-classification SVM model to perform multiple classifications of network behavior types on comments containing the target behavior, and determines whether the comment area contains the target network behavior, thereby improving the detection accuracy of multiple network behaviors.

[0006] 2. Technical solution

[0007] The purpose of this application is achieved through the following technical solutions.

[0008] The present specification provides a microblog network behavior detection method based on feature fusion and SVM, including: obtaining comment data of a target comment area; using a TF-IDF algorithm to extract features from the comment data to obtain a TF-IDF feature matrix that reflects the importance of words in a document; using a pre-trained BERT-Base-Chinese model to extract features from the comment data to obtain a BERT feature matrix containing semantic information; converting the TF-IDF sparse feature matrix into a TF-IDF dense feature matrix; splicing the TF-IDF dense feature matrix and the BERT feature matrix by column to form a fused comprehensive feature matrix; inputting the fused comprehensive feature matrix into a pre-trained binary classification SVM model, classifying each comment in the matrix, determining whether the comment contains target behavior content, and counting the number of comments containing target behavior content and their proportion in all comments; inputting the comprehensive feature matrix of the identified comment subset containing target behavior content into a pre-trained multi-classification SVM model, performing multi-classification on each comment in the subset, and identifying the network behavior type contained in each comment;

[0009] Furthermore, it also includes: based on the classification results, counting the number of comments of each network behavior type and its proportion in the comment subset containing the target behavior content; comparing the number of comments of each network behavior type obtained by statistics with a preset number threshold, and the proportion of each network behavior type with a preset proportion threshold; when the number of at least one network behavior type exceeds the preset number threshold and the proportion exceeds the corresponding proportion threshold, it is determined that the corresponding target network behavior exists in the target comment area.

[0010] Furthermore, the method also includes preprocessing the comment data: using the replace function to remove all spaces in the comment data as the cleaned comment data; establishing a dictionary containing common punctuation marks, and removing punctuation marks in the cleaned comment data according to the established dictionary; based on the comment data without punctuation marks, searching the maximum probability path through dynamic programming to obtain the optimal word segmentation method, and outputting the obtained optimal word segmentation method as the preprocessed comment data.

[0011] Furthermore, a BERT feature matrix containing semantic information is obtained, including: dividing the preprocessed comment data into multiple batches; in each batch, converting each line of comment text in the comment data into a corresponding digital ID sequence; adjusting the length of all digital ID sequences to a preset length through padding or cropping operations to obtain digital ID sequences of uniform length; inputting the digital ID sequences of uniform length into a pre-trained BERT-Base-Chinese model, and outputting feature arrays of the corresponding batches; and integrating the feature arrays of all batches into a BERT dense feature matrix.

[0012] Furthermore, the TF-IDF sparse feature matrix is ​​converted into a TF-IDF dense feature matrix, including: creating a zero value array with the same dimension as the TF-IDF sparse feature matrix as the initial TF-IDF dense feature matrix; traversing the non-zero elements in the TF-IDF sparse feature matrix, obtaining the position index and corresponding value of each non-zero element, and according to the position index of each non-zero element; adding the obtained non-zero element values ​​to the corresponding positions of the created TF-IDF dense feature matrix to obtain the TF-IDF dense feature matrix filled with all non-zero elements.

[0013] Furthermore, the fused comprehensive feature matrix is ​​input into a pre-trained binary classification SVM model, including: training a binary classification SVM model on the COLDataset dataset; using a linear kernel function to transform the feature matrix in the COLDataset dataset to obtain a transformed feature matrix; using the transformed feature matrix and the corresponding binary classification labels to construct an objective function of the SVM model; finding the optimal classification hyperplane parameters by solving a convex quadratic programming problem to train the binary classification SVM model; using the trained binary classification SVM model to classify each comment in the fused feature matrix, and the classification includes target behavior comments and non-target behavior comments.

[0014] Furthermore, the converted feature matrix and the corresponding binary classification labels are used to construct the objective function of the SVM model, including: using the hinge loss function as the loss function of the SVM model to measure the classification error of each sample; constructing the objective function of the SVM model, the objective function consists of two parts: empirical risk and structural risk: the empirical risk uses the average hinge loss (1 / n)*∑L_i of all samples to represent the classification error of the model on the training set; the structural risk uses the L2 norm (1 / 2)*||w||^2 of the weight vector to control the complexity and generalization ability of the model;

[0015] Objective function: min(1 / 2)*||w||^2+(1 / n)*∑L_i.

[0016] Furthermore, the hinge loss function is: L_i = max(0, 1-y_i*f(x_i)), where L_i is the hinge loss of the i-th sample, y_i is the true category label of the sample, which takes a value of 1 or -1, and f(x_i) is the model's predicted value for the i-th sample.

[0017] Furthermore, the sequential minimum optimization (SMO) algorithm is used to solve the convex quadratic programming problem.

[0018] Furthermore, each comment in the subset is multi-classified to identify the type of network behavior contained in each comment, including: using a linear kernel function to transform the feature matrix in the COLDataset data set to obtain the transformed feature matrix; using the transformed feature matrix and the corresponding multi-classification labels to construct the optimization target of the SVM model; finding the optimal classification hyperplane parameters by solving the convex quadratic programming problem to train the multi-classification SVM model; wherein, setting the tolerance less than the threshold is used to control the convergence accuracy; setting the number of iterations to be unlimited to search for the global optimal solution; taking the comprehensive features of the comment subset corresponding to the target behavior output by the binary classification SVM as input; using the trained multi-classification SVM model to multi-classify each comment in the comprehensive feature matrix of the target behavior comment subset; and according to the classification results, counting the number of each type of network behavior comments and the proportion of the corresponding network behavior in the target behavior comment subset.

[0019] 3. Beneficial effects

[0020] Compared with the prior art, the advantages of this application are:

[0021] This application combines the vocabulary features extracted by TF-IDF and the semantic features extracted by BERT in the detection process to form a comprehensive feature matrix. Among them, the TF-IDF feature matrix reflects the importance of vocabulary in the comment data, and the BERT feature matrix contains the semantic information of the comment data. The fusion of the two features makes the representation of the comment data more comprehensive and accurate, thereby improving the accuracy of subsequent network behavior detection.

[0022] This application adopts the idea of ​​hierarchical classification and designs two levels of classifiers: a binary classification SVM model and a multi-classification SVM model. First, the binary classification SVM model is used to identify comments containing the target network behavior from all comments, filtering out a large amount of irrelevant comment data; then, the multi-classification SVM model is used to perform multi-classification of network behavior types on the selected subset of comments containing the target network behavior. Through hierarchical classification, the amount of training data for the multi-classification SVM model is reduced, the training efficiency is improved, and comprehensive recognition of multiple types of network behaviors is achieved.

[0023] In the feature extraction stage, this application densifies the TF-IDF sparse feature matrix and converts it into a TF-IDF dense feature matrix, which reduces storage space and improves computational efficiency. In addition, in the multi-classification stage, a binary SVM model is used to pre-screen a subset of comments containing the target network behavior, which reduces the amount of input data for the multi-classification SVM model, reduces computational complexity, and improves detection efficiency.

[0024] When training the SVM model, this application introduces an L2 regularization term in the loss function to control the structural risk of the model, reduce the value of the weight parameter, reduce the complexity of the model, prevent overfitting, and enhance the generalization ability of the model, so that the trained model can be better applied to new review data.

[0025] The detection method of this application outputs the number of comments and their proportions of various types of network behaviors, and by comparing with the preset threshold, determines whether there is a specific network behavior in the comment area, and the output results are intuitive and easy to understand. When a certain network behavior is identified, the severity of the behavior can be understood based on the number of comments and the proportion, which facilitates subsequent targeted management and control. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 This is a flowchart of a microblog network behavior detection method based on feature fusion and SVM in this application. DETAILED DESCRIPTION

[0027] The present application is described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0028] Figure 1 This is a flowchart of a Weibo network behavior detection method based on feature fusion and SVM in this application. This application is a Weibo network behavior detection system based on feature fusion and SVM. The introduction of feature fusion technology improves the accuracy of SVM. The system first obtains comment data from the Weibo designated comment area through a web crawler for the Weibo platform, saves the original comment content locally, and then performs data preprocessing, including removing useless information (including punctuation and some special expressions) and word segmentation.

[0029] Then, feature extraction is performed on the preprocessed comment data. At this stage, the system uses TF-IDF and BERT to extract text features. TF-IDF focuses on the statistical characteristics of the text. TF (term frequency) represents the frequency of each vocabulary unit (keyword) in a specific document. This number represents the importance of the keyword in the document. The more frequently a word appears in a document, the higher its term frequency and the greater its contribution to the document content. IDF (inverse document frequency) represents the general importance of a word, which is calculated based on the frequency of the word in all documents. If a word appears in many documents, its IDF value will be very low, because such words do not play a big role in distinguishing the content of the document; on the contrary, if a word appears in only a few documents, its IDF value will be very high, because such words are more discriminative.

[0030] By multiplying TF and IDF, we can get the importance score of a word in a document. The higher the score, the more representative the word is for the content of the document. After the TF-IDF feature extraction is completed, a sparse matrix will be output. Here the system will convert it into a dense matrix to pave the way for subsequent feature fusion.

[0031] After the TF-IDF feature extraction is completed, the pre-trained BERT model is used to extract features from the pre-processed text data again. BERT feature extraction focuses on capturing the deep semantic information of the text. This application uses the BERT model as "BERT-Base-Chinese", which is a deep learning model developed by Google in 2018 specifically for Chinese. A large amount of text is pre-trained through the Transformer architecture. The core of Transformer is the self-attention mechanism. The model calculates the relationship weight of each word with all other words in the sentence, and then updates the word representation based on these weights. After being processed by the self-attention mechanism, the representation of each word is no longer isolated, but integrates the information of the entire sentence. As the number of layers increases, the model can capture deeper semantic information, and finally outputs a dense matrix, each element of which contains rich language feature information.

[0032] After extracting features using both methods, the two features are fused, and the system effectively integrates the high-level semantic features output by the BERT model with the statistical features obtained by the TF-IDF vectorizer. Since the result of TF-IDF feature extraction is a sparse matrix, and the result of BERT feature extraction is a dense matrix, the sparse matrix needs to be converted to a dense matrix first. After the conversion is completed, the TF-IDF matrix and the BERT feature matrix are stacked horizontally to form a longer feature vector. This fused feature vector contains the word frequency information and deep semantic information of the original text.

[0033] The BERT model can provide an embedded representation of each word in the text in context, and these embeddings can capture complex semantic relationships. The TF-IDF method focuses on reflecting word frequency and inverse document frequency, highlighting the importance of keywords in the text. By merging these two feature vectors, the system constructs a more comprehensive text feature representation, which not only contains the semantic information of the text, but also word frequency statistics. This fusion strategy enables the classification model to analyze text from multiple dimensions, improving the accuracy of identifying network behavior.

[0034] After feature fusion, the system inputs the comprehensive feature vector into the pre-trained SVM classification model to predict behavioral tendencies. Finally, the system presents the detection results in an intuitive chart form through statistical analysis and visualization modules, allowing users to quickly grasp the severity of network behavior and topic distribution.

[0035] In terms of judging online behavior, in order to formulate a more scientific judgment threshold for the system, after cooperating with legal professionals, we tracked and investigated 46 events involving online behavior, screened out the 20 most representative cases, and calculated the proportion of offensive speech in the comment area. After quantitative analysis, it was found that the proportion of offensive speech in most cases of online violence was around 0.22. Further analysis of the proportion of offensive speech in these 20 cases, after removing the two highest and two lowest values, calculated their mean to be 0.2674, and the standard deviation to be 0.0718. Subtracting one standard deviation from the mean, we can get a threshold of 0.1956. Using this threshold as the standard for judging online violence, if the number of offensive speeches in a comment area is greater than 10 and the proportion of offensive speeches is greater than 0.1956, it will be judged by the system as "there is online behavior".

[0036] In addition, the system uses a visualization module to display the detection results in an intuitive chart form, allowing users to quickly grasp the severity of network behavior and topic distribution. It can also classify and store the identified offensive comments by topic, which facilitates further analysis and research. The design of the entire system fully considers the scientificity and practicality of the technology, aiming to provide effective technical support for the purification and supervision of the network environment.

[0037] Compared with a single feature, the fusion of two features can improve the accuracy of the SVM classifier in identifying offensive speech. TF-IDF helps to identify keywords, while BERT can provide a deeper level of semantic understanding. The combination of the two can reduce misjudgments and missed judgments. The experimental results also show that the SVM classifier after feature fusion can obtain better evaluation indicators on the specified test set.

[0038] The specific test results are shown in the following table:

[0039]

[0040]

[0041] From the results, we can see that on the same test set, the SVM based on feature fusion performs best in the offensive speech classification task, with an accuracy of 0.8012 and an F1 score of 0.7968.

[0042] When it comes to network behavior detection, SVM is more interpretable than the currently widely used deep learning models and can save running time and memory space.

[0043] Specifically, the advantages of SVM over deep learning models in terms of classification speed and memory usage mainly stem from its lower model complexity and number of parameters. SVM handles nonlinear problems through kernel tricks without explicitly calculating points in high-dimensional space, which greatly reduces the amount of computation.

[0044] In addition, the training process of SVM is to solve a convex optimization problem, and it can stop once the optimal solution is found, while the deep learning model needs to optimize a large number of parameters through multiple iterations, which takes longer. In the prediction stage, SVM only needs to consider the support vector, so it has a small memory usage and a faster prediction speed. In contrast, the deep learning model needs to store and process the parameters of all layers, resulting in a large memory usage and a relatively slow prediction speed.

[0045] The invention of the present application and its implementation methods are described schematically above. The description is not restrictive. Without departing from the spirit or basic features of the present application, the present application can be implemented in other specific forms. What is shown in the accompanying drawings is only one of the implementation methods of the invention of the present application. The actual structure is not limited to this. Any figure mark in the claims should not limit the claims involved. Therefore, if a person of ordinary skill in the art is inspired by it, without departing from the purpose of the present invention, a structural method and an embodiment similar to the technical solution are designed without creativity, which should all fall within the scope of protection of this patent. In addition, the word "including" does not exclude other elements or steps, and the word "one" before the element does not exclude the inclusion of "multiple" elements. The multiple elements stated in the product claim can also be implemented by one element through software or hardware. The words first, second, etc. are used to indicate names, and do not indicate any specific order.

Claims

1. A microblog network behavior detection method based on feature fusion and SVM, comprising: Get the comment data of the target comment area; The TF-IDF algorithm is used to extract features from the comment data to obtain the TF-IDF feature matrix that reflects the importance of words in the document; Use the pre-trained BERT-Base-Chinese model to extract features from the comment data and obtain a BERT feature matrix containing semantic information; Convert the TF-IDF sparse feature matrix to the TF-IDF dense feature matrix; Concatenate the TF-IDF dense feature matrix and the BERT feature matrix by column to form a fused comprehensive feature matrix; The fused comprehensive feature matrix is ​​input into the pre-trained binary classification SVM model to classify each comment in the matrix, determine whether the comment contains the target behavior content, and count the number of comments containing the target behavior content and their proportion in all comments; The comprehensive feature matrix of the identified comment subset containing the target behavior content is input into the pre-trained multi-classification SVM model, and each comment in the subset is multi-classified to identify the type of network behavior contained in each comment.

2. The microblog network behavior detection method based on feature fusion and SVM according to claim 1 is characterized in that: Also includes: Based on the classification results, count the number of comments of each network behavior type and its proportion in the subset of comments containing the target behavior content; The number of comments of each network behavior type obtained by statistics is compared with the preset number threshold, and the proportion of each network behavior type is compared with the preset proportion threshold. When the number of at least one network behavior type exceeds the preset number threshold and the proportion exceeds the corresponding proportion threshold, it is determined that the target comment area has the corresponding target network behavior.

3. The microblog network behavior detection method based on feature fusion and SVM according to claim 2 is characterized in that: Before using the TF-IDF algorithm for feature extraction, the comment data must be preprocessed: Use the replace function to remove all spaces in the comment data as the cleaned comment data; Establish a dictionary containing common punctuation marks, and remove the punctuation marks in the cleaned comment data according to the established dictionary; According to the comment data with punctuation removed, the maximum probability path is searched through dynamic programming to obtain the optimal word segmentation method, and the obtained optimal word segmentation method is output as the preprocessed comment data.

4. The microblog network behavior detection method based on feature fusion and SVM according to claim 3 is characterized by: Get the BERT feature matrix containing semantic information, including: Divide the preprocessed review data into multiple batches; In each batch, convert each line of comment text in the comment data into a corresponding sequence of digital IDs; By padding or trimming operations, the lengths of all digital ID sequences are adjusted to a preset length to obtain digital ID sequences with uniform lengths; Input a digital ID sequence of uniform length into the pre-trained BERT-Base-Chinese model and output the feature array of the corresponding batch; Combine the feature arrays of all batches into a BERT dense feature matrix.

5. The microblog network behavior detection method based on feature fusion and SVM according to claim 4 is characterized in that: Convert the TF-IDF sparse feature matrix to the TF-IDF dense feature matrix, including: Create a zero-value array with the same dimension as the TF-IDF sparse feature matrix as the initial TF-IDF dense feature matrix; Traverse the non-zero elements in the TF-IDF sparse feature matrix, obtain the position index and corresponding value of each non-zero element, according to the position index of each non-zero element; The obtained non-zero element values ​​are added to the corresponding positions of the created TF-IDF dense feature matrix to obtain the TF-IDF dense feature matrix filled with all non-zero elements.

6. The microblog network behavior detection method based on feature fusion and SVM according to claim 5 is characterized by: The fused comprehensive feature matrix is ​​input into the pre-trained binary classification SVM model, including: Train a binary classification SVM model on the COLDataset dataset; The linear kernel function is used to transform the feature matrix in the COLDataset dataset to obtain the transformed feature matrix; Use the converted feature matrix and the corresponding binary classification labels to construct the objective function of the SVM model; By solving the convex quadratic programming problem, the optimal classification hyperplane parameters are found and the binary classification SVM model is trained; The trained binary classification SVM model is used to classify each comment in the fused feature matrix, which includes target behavior comments and non-target behavior comments.

7. The microblog network behavior detection method based on feature fusion and SVM according to claim 6 is characterized by: Use the converted feature matrix and the corresponding binary classification labels to construct the objective function of the SVM model, including: The hinge loss function is used as the loss function of the SVM model to measure the classification error of each sample; Construct the objective function of the SVM model, which consists of two parts: empirical risk and structural risk: The empirical risk is the average hinge loss of all samples. Represents the classification error of the model on the training set; Structural risk uses the L2 norm of the weight vector Used to control the complexity and generalization ability of the model; Objective function:

8. The microblog network behavior detection method based on feature fusion and SVM according to claim 7 is characterized in that: The hinge loss function is: L i =max(0,1-y i ×f(x i )) Among them, L i is the hinge loss of the i-th sample, y i is the true category label of the sample, which takes the value of 1 or -1, f(x i ) is the model’s predicted value for the i-th sample.

9. The microblog network behavior detection method based on feature fusion and SVM according to claim 7 is characterized in that: The sequential minimum optimization (SMO) algorithm is used to solve the convex quadratic programming problem.

10. The microblog network behavior detection method based on feature fusion and SVM according to claim 9 is characterized in that: The comprehensive feature matrix of the identified comment subset containing the target behavior content is input into the pre-trained multi-classification SVM model, and each comment in the subset is multi-classified to identify the network behavior type contained in each comment, including: The feature matrix in the COLDataset dataset is transformed using a linear kernel function to obtain the transformed feature matrix; Use the converted feature matrix and the corresponding multi-classification labels to construct the optimization target of the SVM model; By solving the convex quadratic programming problem, the optimal classification hyperplane parameters are found and the multi-classification SVM model is trained. The tolerance is set to be less than the threshold to control the convergence accuracy. The number of iterations is set to be unlimited to search for the global optimal solution. The comprehensive features of the review subset corresponding to the target behavior output by the binary classification SVM are used as input; Using the trained multi-classification SVM model, perform multi-classification on each comment in the comprehensive feature matrix of the target behavior comment subset; According to the classification results, the number of comments on each type of network behavior and the proportion of the corresponding network behavior in the target behavior comment subset are counted.

Images