Chinese fraudulent language detection method and system based on two channels

Abstract

The invention discloses a Chinese fraudulent language detection method and system based on two channels, and the method comprises the steps: obtaining text data, generating a word vector through a BERT model, inputting the word vector into two parallel channels for processing, and enabling the first channel to generate a first feature vector through a capsule network; and the second channel combines a bidirectional gating loop unit, a text convolutional neural network and a multi-head attention mechanism to generate a second feature vector, after the two feature vectors are fused, the two feature vectors are input into the full-connection layer for feature mapping, and a Chinese fraudulent language detection result is output. Wide application prospects are realized.

Description

Technical Field

[0001] The present invention belongs to the technical field of natural language processing text recognition, and in particular relates to a dual-channel based Chinese offensive language detection method and system. Background Art

[0002] With the rapid development of social media and online platforms, offensive speech has become an increasing problem in Chinese online communities, becoming a serious social problem. Especially in various forums and comment areas, these offensive speech not only destroys the atmosphere of online communication, but also may have an adverse impact on users' mental health. Due to the complex language structure of Chinese, the large number of dialects, and the existence of a large number of online terms and metaphors, detecting offensive speech in Chinese faces special challenges.

[0003] At present, most of the research on offensive speech detection is mainly focused on the English context, and relatively few studies are conducted in the Chinese context. There are significant differences between Chinese and English in terms of expression, grammatical structure, and offensive speech types, so research on offensive speech detection in Chinese has important practical significance and application value.

[0004] Traditional offensive speech detection methods mainly rely on rule-based methods and keyword matching techniques. These methods use pre-built vocabulary lists to identify specific words in the text to mark offensive content. However, these methods often have difficulty effectively identifying obscure or complex offensive expressions due to their lack of understanding of the context. With the development of machine learning technology, researchers have begun to explore the use of machine learning algorithms for offensive speech detection. These methods have improved the detection effect to a certain extent, but are still limited by the understanding of the deep meaning of the language and the capture of contextual information. The rise of deep learning has brought new opportunities for offensive speech detection. Pre-trained word vector models, such as BERT, can automatically learn complex language patterns and capture the nuances and contextual relationships in the text, thereby significantly improving the accuracy of offensive speech detection.

[0005] Although some research has achieved certain results in offensive speech detection in specific fields, research on the Chinese environment is still insufficient. As the language with the largest number of speakers, studying offensive speech detection methods in Chinese has important practical significance. In addition, previous research mainly focused on simple comment text detection, lacking effective recognition of the deep semantics and implicit expressions of the text. Summary of the invention

[0006] In order to solve the above problems, the present invention provides a dual-channel Chinese offensive language detection method, comprising the following steps:

[0007] Step S1: Obtain text data to be detected;

[0008] Step S1-1: using a Chinese offensive language dataset initially selected from multiple online platforms, the Chinese offensive language dataset is initially screened to exclude data with unclear semantics or incorrect annotations, and obtain a sub-dataset;

[0009] Step S1-2: preprocessing the sub-dataset;

[0010] Step S1-2-1: Sub-dataset preprocessing includes cleaning, word segmentation, tagging and standardization, and the preprocessed sub-dataset is divided into a training set, a validation set and a test set according to the proportion;

[0011] Step S2: Use the BERT model to process the sub-dataset text and generate a word vector Ebert that captures the semantic details in the context;

[0012] Step S2-1: The specific method of generating word vector Ebert is as follows:

[0013] Step S2-2: For each text X, it is first tokenized. After tokenization, each token is mapped to a unique index;

[0014] Step S2-3: Add a CLS tag at the beginning of the sentence and a SEP tag at the end of the sentence. The sentence is converted into a format that the BERT model can understand after word segmentation and standardization. After being processed by all Transformer layers, the word vector Ebert of the overall semantic information is obtained;

[0015] Step S3: input the word vector Ebert into two parallel channels for processing;

[0016] Step S3-1: The first channel is processed by the capsule network to generate the first feature vector Vcapsule; the second channel is first processed by the bidirectional gated recurrent unit BiGRU, then by the text convolutional neural network TextCNN, and finally by the multi-head attention mechanism to generate the second feature vector Vmha;

[0017] Step S3-2: The capsule network in the first channel is used to capture the spatial relationship between word vectors, enhance the feature expression capability through a dynamic routing mechanism, take the word vector Ebert as input, generate a capsule feature vector Vcapsule, and use it as the first feature vector;

[0018] Before entering the capsule network, the word vector Ebert is usually preprocessed, including rescaling the dimension and linear transformation to adapt to the input requirements of the capsule network;

[0019] E′ bert =E bert W trans +b trans

[0020] Where Wtrans and btrans are the weight and bias of the conversion layer respectively;

[0021] In each iteration, the output of the capsule network is dynamically sent to capsules in higher layers through the routing process;

[0022] The routing process includes calculating the agreement score between capsules, which determines the contribution of the lower-layer capsule to the output of the upper-layer capsule;

[0023]

[0024] Where cij is the connection weight of capsule i to capsule j, bij is the initial logarithmic prior probability, which is dynamically adjusted through the iterative process;

[0025] Then, vector conversion and activation are implemented. Each capsule output vector sj is obtained by multiplying all input vectors u^j|i with the corresponding routing weight cij and then summing them up:

[0026]

[0027] A compressed nonlinear activation function is used to transform sj into the final capsule output vj:

[0028]

[0029] vj is the final output first feature vector Vcapsule, that is, Vcapsule is obtained from the top-level capsule network;

[0030] Step S3-3: The feature vector generation method of the second channel includes the following steps:

[0031] Taking the word vector Ebert as input, BiGRU processes the input word vector Ebert through the forward and backward GRU units to capture the long-distance dependencies of the text, and the output feature vector is represented as HBiGRU;

[0032] H BiGRU =BiGRU(E bert )

[0033] The BiGRU feature vector VBiGRU is generated by selecting the hidden state of the last time step or applying additional operations, where the output ht at each time step is the forward and backward Output connections;

[0034]

[0035] Through TextCNN, local features of text are captured and TextCNN feature vectors are generated;

[0036] TextCNN processes the input BiGRU feature vector VBiGRU by applying multiple convolution kernels and corresponding maximum pooling operations to capture local features;

[0037] For each convolution kernel k, a convolution operation is performed and a nonlinear activation function ReLU is applied to obtain the convolution output result Ck, where Wk and bk are the weight and bias of the convolution kernel k respectively;

[0038] C k =ReLU(W k *V BiGRU +b k )

[0039] Apply the maximum pooling operation to the convolutional layer output Ck to obtain the most important features;

[0040] P k =MaxPool(C k )

[0041] Concatenate the pooled outputs of all convolution kernels k to form the feature vector VTextCNN of TextCNN;

[0042] VTextCNN is used as the input of the multi-head attention layer, and information is processed in parallel by multiple independent attention heads. The formula of the attention mechanism is:

[0043]

[0044] Where Q, K, and V represent the query vector, key vector, and value vector, respectively. QKT is the dot product of the query vector and the key vector, indicating the similarity. dk is the dimension of the key vector. Softmax is a normalization function used to convert the similarity into a probability distribution.

[0045] Each head calculates the attention score and updates the feature representation, and then the outputs of all heads are concatenated and linearly transformed to the original dimension, that is:

[0046] head i =Attention(V TextCNN W i Q ,V TextCNN W i K ,V TextCNN W i V )

[0047] V mha=Concat(head1,......,head h )W O

[0048] Among them, WQi and WO are learnable weight matrices, Vmha is the feature representation after being processed by the multi-head attention mechanism, and Vmha is used as the second feature vector;

[0049] Step S4: Fusing the first eigenvector and the second eigenvector to obtain a fused eigenvector Vcombined;

[0050] The feature vector Vcapsule from the capsule network of the first channel and the feature vector Vmha obtained by BiGRU, TextCNN and attention mechanism of the second channel are concatenated to form a comprehensive feature vector Vcombined. The concatenation operation is expressed by the formula:

[0051]

[0052] In the formula, : represents the concatenation operation, which retains all elements of the two vectors;

[0053] Step S5: input the fused feature vector Vcombined into the fully connected layer for feature mapping, and output the text detection result;

[0054] The fused feature vector Vcombined is input to a fully connected layer with multiple neurons. The fully connected layer uses a softmax activation function for feature mapping to output the classification result of the text. The output of the fully connected layer is then passed to a batch normalization layer, which standardizes the data, stabilizes the training process of the neural network and significantly accelerates the convergence speed of the model.

[0055] In offensive language detection tasks, it is usually mapped to two categories: offensive and non-offensive;

[0056] The output is converted into a probability distribution through the Softmax activation function, indicating the probability that the input belongs to each category of the sentence:

[0057] Y = softmax(W fc V combined +b fc )

[0058] Where Wfc and bfc are the weight and bias of the fully connected layer, respectively, and Y represents the classification output;

[0059] In order to improve the training stability of the model and accelerate convergence, a batch normalization layer is added after the fully connected layer. The batch normalization layer optimizes the training process by adjusting and scaling the distribution of input features:

[0060] V norm =BanchNorm(V combined )

[0061] The final output is a prediction result, which determines whether the text contains offensive language.

[0062] Preferably, the BERT model is a pre-trained language representation model based on the Transformers architecture. By pre-training on a large-scale text corpus, the learned language representation can be transferred to downstream NLP tasks.

[0063] In the preferred mode, the combination of the BERT model and the dual-channel model enhances the recognition of offensive language while improving the understanding of the semantic depth of the text.

[0064] Preferably, the capsule network is a network structure designed to capture hierarchical relationships in data. In text processing, the capsule network can effectively simulate the hierarchical relationships and dynamic features between words.

[0065] Preferably, the Chinese offensive language dataset includes: 37,480 sentences, covering offensive topics related to race, gender and region.

[0066] In a preferred embodiment, the sub-dataset includes 25,726 sentences, 12,723 of which have offensive labels, i.e., data with labels of 1, and 13,003 of which have non-offensive labels, i.e., data with labels of 0.

[0067] Preferably, the sub-datasets are divided into 80%, 10% and 10% proportions to ensure that the performance of the model on different sub-datasets can be fully evaluated.

[0068] In a preferred embodiment, the specific configuration of the BERT model includes: the number of hidden layers is 12, the hidden layer size is 768, that is, the vector dimension of each word or word fragment is 768, and the number of attention heads is 12.

[0069] In a preferred embodiment, the format that the BERT model can understand includes conversion into a word index, and necessary padding and truncation are performed to meet the maximum length requirement of the model. The input index is converted into a d-dimensional vector through an embedding layer. The embedding layer of the BERT model includes the sum of word embedding, position embedding, and segment embedding.

[0070] In the preferred mode, the multi-head attention mechanism is used to optimize the TextCNN feature vector VTextCNN to generate a multi-head attention feature vector. The multi-head attention mechanism maps the input features to multiple subspaces, calculates the attention in parallel, and then concatenates the results.

[0071] A dual-channel Chinese offensive language detection system, including: BERT module, capsule network module, BiGRU module, TextCNN module, multi-head attention mechanism module;

[0072] Step M1: Obtain text data to be detected;

[0073] Step M1-1: Use the Chinese offensive language dataset to initially select from multiple online platforms. The Chinese offensive language dataset is initially screened to exclude data with unclear semantics or incorrect annotations to obtain a sub-dataset;

[0074] Step M1-2: preprocessing the sub-dataset;

[0075] Step M1-2-1: Sub-dataset preprocessing includes cleaning, word segmentation, tagging and standardization, and the preprocessed sub-dataset is divided into training set, validation set and test set according to the proportion;

[0076] Step M2: The BERT module is implemented by processing the sub-dataset and generating word vectors Ebert that capture the semantic details in the context;

[0077] Step M2-1: The specific method of generating word vector Ebert is as follows:

[0078] Step M2-2: For each text X, it is first tokenized. After tokenization, each token is mapped to a unique index.

[0079] Step M2-3: Add a CLS tag at the beginning of the sentence and a SEP tag at the end of the sentence. The sentence is converted into a format that the BERT model can understand after word segmentation and standardization. After being processed by all Transformer layers, the word vector Ebert of the overall semantic information is obtained;

[0080] Step M3: Input the word vector Ebert into two parallel channels for processing;

[0081] Step M3-1: The first channel is processed by the capsule network module to generate the first feature vector Vcapsule; the second channel is first processed by the bidirectional gated recurrent unit BiGRU module, then by the text convolutional neural network TextCNN module, and finally by the multi-head attention mechanism module to generate the second feature vector Vmha;

[0082] Step M3-2: The capsule network module is specifically implemented as follows: the capsule network module in the first channel is used to capture the spatial relationship between word vectors, and can enhance the feature expression capability through a dynamic routing mechanism. The word vector Ebert is used as input to generate a capsule feature vector Vcapsule, and it is used as the first feature vector;

[0083] Before entering the capsule network module, the word vector Ebert is usually preprocessed, including rescaling the dimension and linear transformation to adapt to the input requirements of the capsule network;

[0084] E′ bert =E bert W trans +b trans

[0085] Where Wtrans and btrans are the weight and bias of the conversion layer respectively;

[0086] In each iteration, the output of the capsule network module is dynamically sent to the capsules in the higher layer through the routing process;

[0087] The routing process usually includes calculating the agreement score between capsules, which determines the contribution of the lower-layer capsule to the output of the upper-layer capsule;

[0088]

[0089] Where cij is the connection weight of capsule i to capsule j, bij is the initial logarithmic prior probability, which is dynamically adjusted through the iterative process;

[0090] Then, vector conversion and activation are implemented. Each capsule output vector sj is obtained by multiplying all input vectors u^j|i with the corresponding routing weight cij and then summing them up:

[0091]

[0092] A compressed nonlinear activation function is used to transform sj into the final capsule output vj:

[0093]

[0094] vj is the final output first feature vector Vcapsule, that is, Vcapsule is obtained from the top-level capsule network;

[0095] Step M3-3: The feature vector generation method of the second channel includes the following steps:

[0096] The specific implementation of the BiGRU module is as follows: the word vector Ebert is taken as input, and the BiGRU module processes the input word vector Ebert through the forward and backward GRU units to capture the long-distance dependency of the text, and the output feature vector is represented as HBiGRU;

[0097] H BiGRU =BiGRU(E bert )

[0098] The feature vector VBiGRU of the BiGRU module is generated by selecting the hidden state of the last time step or applying additional operations, where the output ht at each time step is the forward and backward Output connections;

[0099]

[0100] The specific implementation method of the TextCNN module is as follows: through the TextCNN module, the local features of the text are captured and the feature vector of the TextCNN module is generated;

[0101] The TextCNN module processes the input feature vector VBiGRU of the BiGRU module by applying multiple convolution kernels and corresponding max pooling operations to capture local features;

[0102] For each convolution kernel k, a convolution operation is performed and a nonlinear activation function ReLU is applied to obtain the convolution output result Ck, where Wk and bk are the weight and bias of the convolution kernel k respectively;

[0103] C k =ReLU(W k *V BiGRU +b k )

[0104] Apply the maximum pooling operation to the convolutional layer output Ck to obtain the most important features;

[0105] P k =MaxPool(C k )

[0106] Concatenate the pooled outputs of all convolution kernels k to form the feature vector VTextCNN of the TextCNN module;

[0107] The multi-head attention mechanism module is implemented as follows: VTextCNN is used as the input of the multi-head attention layer, and information is processed in parallel through multiple independent attention heads. The formula of the attention mechanism is:

[0108]

[0109] Where Q, K, and V represent the query vector, key vector, and value vector, respectively. QKT is the dot product of the query vector and the key vector, indicating the similarity. dk is the dimension of the key vector. Softmax is a normalization function used to convert the similarity into a probability distribution.

[0110] Each head calculates the attention score and updates the feature representation, and then the outputs of all heads are concatenated and linearly transformed to the original dimension, that is:

[0111] headi =Attention(V TextCNN W i Q ,V TextCNN W i K ,V TextCNN W i V )

[0112] V mha =Concat(head1,......,head h )W O

[0113] Among them, WQi and WO are learnable weight matrices, Vmha is the feature representation after being processed by the multi-head attention mechanism, and Vmha is used as the second feature vector;

[0114] Step M4: Fusing the first eigenvector and the second eigenvector to obtain a fused eigenvector Vcombined;

[0115] The feature vector Vcapsule from the capsule network module of the first channel and the feature vector Vmha obtained by the BiGRU module, the TextCNN module and the attention mechanism module of the second channel are concatenated to form a comprehensive feature vector Vcombined. The concatenation operation is expressed by the formula:

[0116]

[0117] In the formula, : represents the concatenation operation, which retains all elements of the two vectors;

[0118] Step M5: Input the fused feature vector Vcombined into the fully connected layer for feature mapping, and output the text detection result;

[0119] The fused feature vector Vcombined is input to a fully connected layer with multiple neurons. The fully connected layer uses a softmax activation function for feature mapping to output the classification result of the text. The output of the fully connected layer is then passed to a batch normalization layer, which standardizes the data, stabilizes the training process of the neural network and significantly accelerates the convergence speed of the model.

[0120] In offensive language detection tasks, it is usually mapped to two categories: offensive and non-offensive;

[0121] The output is converted into a probability distribution through the Softmax activation function, indicating the probability that the input belongs to each category of the sentence:

[0122] Y = softmax(Wfc V combined +b fc )

[0123] Where Wfc and bfc are the weight and bias of the fully connected layer, respectively, and Y represents the classification output;

[0124] In order to improve the training stability of the model and accelerate convergence, a batch normalization layer is added after the fully connected layer. The batch normalization layer optimizes the training process by adjusting and scaling the distribution of input features:

[0125] V norm =BanchNorm(V combined )

[0126] The final output is a prediction result, which determines whether the text contains offensive language.

[0127] Beneficial purpose of the present invention: The present invention adopts a deep learning neural network and a pre-trained BERT model, and effectively captures the semantic information of the text by using the context word vector generated by the BERT model. This process not only enhances the ability to recognize subtle differences in offensive Chinese language, but also improves the understanding of the deep meaning of the text. The present invention adopts a parallel dual-channel processing mechanism, and uses capsule networks and BiGRU and TextCNN models to extract feature information, which can comprehensively capture text features from different angles, understand the deep semantic structure of the text, and accurately grasp the local detail features. This parallel processing method not only improves the processing efficiency, but also significantly enhances the comprehensive learning ability of the model. The introduced multi-head attention mechanism can effectively optimize the feature representation of the model, so that the model can focus more on the key information in the text. When processing complex sentence structures or ambiguous expressions, the introduction of the multi-head attention mechanism significantly improves the judgment accuracy and robustness of the model. The model structure and processing strategy of the present invention enable it to have a strong generalization ability, and can effectively adapt to text data of different types and styles, including but not limited to Chinese texts in various network environments such as social media comments and forum posts. This is particularly important for real-time monitoring and automated processing of large-scale network text data. Deep learning technology effectively reduces the reliance on manual screening and preprocessing, greatly improving the efficiency of offensive language detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0128] Figure 1 is a flow chart of offensive language detection in the present invention;

[0129] Figure 2 Structural diagram of the dual-channel language detection model in the present invention. DETAILED DESCRIPTION

[0130] According to Figure 1The present invention provides a dual-channel Chinese offensive language detection method, comprising the following steps:

[0131] Step S1: Obtain text data to be detected;

[0132] Step S1-1: The Chinese offensive language dataset used in the present invention is initially selected from multiple online social platforms, including 37,480 sentences, covering offensive topics related to race, gender and region. These sentences are initially screened to exclude data with unclear semantics or incorrect annotations, and finally a dataset containing 25,726 sentences is determined. In the dataset, there are 12,723 pieces of data with offensive labels, i.e., labels 1, and 13,003 pieces of data with non-offensive labels, i.e., labels 0;

[0133] Step S1-2: preprocessing the data set;

[0134] Step S1-2-1: The data set preprocessing step includes cleaning, word segmentation, tagging and standardization. The preprocessed data set is randomly divided into training set, validation set and test set in the proportion of 80%, 10% and 10% to ensure that the performance of the model on different data subsets can be fully evaluated;

[0135] Step S2: Use the BERT model to process the dataset and generate word vectors Ebert that capture the semantic details in the context;

[0136] Step S2-1: The specific method of generating word vector Ebert is as follows:

[0137] Step S2-2: BERT model, the specific configuration includes: the number of hidden layers is 12, the hidden layer size is 768, that is, the vector dimension of each word or word fragment is 768, and the number of attention heads is 12;

[0138] Step S2-3: For each text X, it is first tokenized. After tokenization, each token is mapped to a unique index, which is done by looking up BERT’s built-in vocabulary.

[0139] Step S2-4: Add a CLS tag at the beginning of the sentence and a SEP tag at the end of the sentence. These tags not only help the model identify the beginning and end of the input data, but also the output vector of the CLS tag, that is, the representation at the sentence level, will be used for subsequent classification tasks. The sentence is converted into a format that the BERT model can understand after word segmentation and standardization, including conversion into a word index, and necessary padding and truncation to meet the maximum length requirement of the model. The input index is converted into a d-dimensional vector through the embedding layer. The embedding layer of the BERT model includes the sum of word embedding, position embedding and segment embedding. Then, after being processed by all Transformer layers, the word vector Ebert of the overall semantic information is obtained;

[0140] Step S3: Input the word vector Ebert into Figure 2 The dual-channel language detection model is used for processing;

[0141] Step S3-1: The first channel is processed by the capsule network to generate the first feature vector Vcapsule; the second channel is first processed by the bidirectional gated recurrent unit BiGRU, then by the text convolutional neural network TextCNN, and finally by the multi-head attention mechanism to generate the second feature vector Vmha;

[0142] Step S3-2: The capsule network in the first channel is used to capture the spatial relationship between word vectors, and can enhance the feature expression ability through a dynamic routing mechanism. The word vector Ebert is used as input to generate a capsule feature vector Vcapsule, which is used as the first feature vector;

[0143] Before entering the capsule network, the word vector Ebert is usually preprocessed, including rescaling the dimension and linear transformation to adapt to the input requirements of the capsule network;

[0144] E′ bert =E bert W trans +b trans

[0145] Where Wtrans and btrans are the weight and bias of the conversion layer respectively;

[0146] In each iteration, the output of the capsule network is dynamically sent to capsules in higher layers through the routing process;

[0147] The routing process usually includes calculating the agreement score between capsules, which determines the contribution of the lower-layer capsule to the output of the upper-layer capsule;

[0148]

[0149] Where cij is the connection weight of capsule i to capsule j, bij is the initial logarithmic prior probability, which is dynamically adjusted through the iterative process;

[0150] Then, vector conversion and activation are implemented. Each capsule output vector sj is obtained by multiplying all input vectors u^j|i with the corresponding routing weight cij and then summing them up:

[0151]

[0152] A compressed nonlinear activation function is used to transform sj into the final capsule output vj:

[0153]

[0154] vj is the final output first feature vector Vcapsule, that is, Vcapsule is obtained from the top-level capsule network;

[0155] Step S3-3: The feature vector generation method of the second channel includes the following steps:

[0156] Taking the word vector Ebert as input, BiGRU processes the input word vector Ebert through the forward and backward GRU units to capture the long-distance dependencies of the text, and the output feature vector is represented as HBiGRU;

[0157] H BiGRU =BiGRU(E bert )

[0158] The BiGRU feature vector VBiGRU is generated by selecting the hidden state of the last time step or applying additional operations, where the output ht at each time step is the forward and backward Output connections;

[0159]

[0160] Through TextCNN, local features of text are captured and TextCNN feature vectors are generated;

[0161] TextCNN processes the input BiGRU feature vector VBiGRU by applying multiple convolution kernels and corresponding maximum pooling operations to capture local features;

[0162] For each convolution kernel k, a convolution operation is performed and a nonlinear activation function ReLU is applied to obtain the convolution output result Ck, where Wk and bk are the weight and bias of the convolution kernel k respectively;

[0163] C k =ReLU(W k *VBiGRU +b k )

[0164] Apply the maximum pooling operation to the convolutional layer output Ck to obtain the most important features;

[0165] P k =MaxPool(C k )

[0166] Concatenate the pooled outputs of all convolution kernels k to form the feature vector VTextCNN of TextCNN;

[0167] The multi-head attention mechanism is used to optimize the TextCNN feature vector VTextCNN and generate a multi-head attention feature vector. The multi-head attention mechanism maps the input features to multiple subspaces, calculates the attention in parallel, and then concatenates the results. VTextCNN is used as the input of the multi-head attention layer, and information is processed in parallel through multiple independent attention heads. The formula of the attention mechanism is:

[0168]

[0169] Where Q, K, and V represent the query vector, key vector, and value vector, respectively. QKT is the dot product of the query vector and the key vector, indicating the similarity. dk is the dimension of the key vector. Softmax is a normalization function used to convert the similarity into a probability distribution.

[0170] Each head calculates the attention score and updates the feature representation, and then the outputs of all heads are concatenated and linearly transformed to the original dimension, that is:

[0171] head i =Attention(V TextCNN W i Q ,V TextCNN W i K ,V TextCNN W i V )

[0172] V mha =Concat(head1,......,head h )W O

[0173] Among them, WQi and WO are learnable weight matrices, Vmha is the feature representation after being processed by the multi-head attention mechanism, and Vmha is used as the second feature vector;

[0174] Step S4: Fusing the first eigenvector and the second eigenvector to obtain a fused eigenvector Vcombined;

[0175] The feature vector Vcapsule from the capsule network of the first channel and the feature vector Vmha obtained by BiGRU, TextCNN and attention mechanism of the second channel are concatenated to form a comprehensive feature vector Vcombined. The concatenation operation is expressed by the formula:

[0176]

[0177] In the formula, : represents the concatenation operation, which retains all elements of the two vectors;

[0178] Step S5: input the fused feature vector Vcombined into the fully connected layer for feature mapping, and output the text detection result;

[0179] The fused feature vector Vcombined is input to a fully connected layer with multiple neurons. The fully connected layer uses a softmax activation function for feature mapping to output the classification result of the text. The output of the fully connected layer is then passed to a batch normalization layer, which standardizes the data, stabilizes the training process of the neural network and significantly accelerates the convergence speed of the model.

[0180] In offensive language detection tasks, it is usually mapped to two categories: offensive and non-offensive;

[0181] The output is converted into a probability distribution through the Softmax activation function, indicating the probability that the input belongs to each category of the sentence:

[0182] Y = softmax(W fc V combined +b fc )

[0183] Where Wfc and bfc are the weight and bias of the fully connected layer, respectively, and Y represents the classification output;

[0184] In order to improve the training stability of the model and accelerate convergence, a batch normalization layer is added after the fully connected layer. The batch normalization layer optimizes the training process by adjusting and scaling the distribution of input features:

[0185] V norm =BanchNorm(V combined )

[0186] The final output is a prediction result, which determines whether the text contains offensive language.

[0187] The above are only preferred specific implementation modes of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical solutions and inventive concepts of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A dual-channel Chinese offensive language detection method, characterized in that: The steps include: Step S1: Obtain text data to be detected; Step S1-1: using a Chinese offensive language dataset initially selected from multiple online platforms, the Chinese offensive language dataset is initially screened to exclude data with unclear semantics or incorrect annotations, and obtain a sub-dataset; Step S1-2: preprocessing the sub-dataset; Step S1-2-1: Sub-dataset preprocessing includes cleaning, word segmentation, tagging and standardization, and the preprocessed sub-dataset is divided into a training set, a validation set and a test set according to the proportion; Step S2: Use the BERT model to process the sub-dataset text and generate a word vector Ebert that captures the word meaning details in the context; Step S2-1: The specific method of generating word vector Ebert is as follows: Step S2-2: For each text X, it is first tokenized. After tokenization, each token is mapped to a unique index; Step S2-3: Add a CLS tag at the beginning of the sentence and a SEP tag at the end of the sentence. The sentence is converted into a format that the BERT model can understand after word segmentation and standardization. After being processed by all Transformer layers, the word vector Ebert of the overall semantic information is obtained; Step S3: input the word vector Ebert into two parallel channels for processing; Step S3-1: The first channel is processed by the capsule network to generate the first feature vector Vcapsule; the second channel is first processed by the bidirectional gated recurrent unit BiGRU, then by the text convolutional neural network TextCNN, and finally by the multi-head attention mechanism to generate the second feature vector Vmha; Step S3-2: The capsule network in the first channel is used to capture the spatial relationship between word vectors, enhance the feature expression capability through a dynamic routing mechanism, take the word vector Ebert as input, generate a capsule feature vector Vcapsule, and use it as the first feature vector; Before entering the capsule network, the word vector Ebert is usually preprocessed, including rescaling the dimension and linear transformation to adapt to the input requirements of the capsule network; HAVE BEEN bert ′=E bert W trans +b trans Where Wtrans and btrans are the weight and bias of the conversion layer respectively; In each iteration, the output of the capsule network is dynamically sent to capsules in higher layers through the routing process; The routing process includes calculating the agreement score between capsules, which determines the contribution of the lower-layer capsule to the output of the upper-layer capsule; Where cij is the connection weight of capsule i to capsule j, bij is the initial logarithmic prior probability, which is dynamically adjusted through the iterative process; Then, vector conversion and activation are implemented. Each capsule output vector sj is obtained by multiplying all input vectors u^j|i with the corresponding routing weight cij and then summing them up: A compressed nonlinear activation function is used to transform sj into the final capsule output vj: vj is the final output first feature vector Vcapsule, that is, Vcapsule is obtained from the top-level capsule network; Step S3-3: The feature vector generation method of the second channel includes the following steps: Taking the word vector Ebert as input, BiGRU processes the input word vector Ebert through the forward and backward GRU units to capture the long-distance dependencies of the text, and the output feature vector is represented as HBiGRU; H BiGRU =BigGRU(E bert ) The BiGRU feature vector VBiGRU is generated by selecting the hidden state of the last time step or applying additional operations, where the output ht at each time step is the forward and backward Output connections; Through TextCNN, local features of text are captured and TextCNN feature vectors are generated; TextCNN processes the input BiGRU feature vector VBiGRU by applying multiple convolution kernels and corresponding maximum pooling operations to capture local features; For each convolution kernel k, a convolution operation is performed and a nonlinear activation function ReLU is applied to obtain the convolution output result Ck, where Wk and bk are the weight and bias of the convolution kernel k respectively; C k =ReLU(W k *V BiGRU +b k ) Apply the maximum pooling operation to the convolutional layer output Ck to obtain the most important features; P k =MaxPool(C k ) Concatenate the pooled outputs of all convolution kernels k to form the feature vector VTextCNN of TextCNN; VTextCNN is used as the input of the multi-head attention layer, and information is processed in parallel by multiple independent attention heads. The formula of the attention mechanism is: Where Q, K, and V represent the query vector, key vector, and value vector, respectively. QKT is the dot product of the query vector and the key vector, indicating the similarity. dk is the dimension of the key vector. Softmax is a normalization function used to convert the similarity into a probability distribution. Each head calculates the attention score and updates the feature representation, and then the outputs of all heads are concatenated and linearly transformed to the original dimension, that is: head i =Attention(V TextCNN W i Q ,V TextCNN W i K ,V TextCNN W i V ) V mha =Concat(head1,......,head h )W O Among them, WQi and WO are learnable weight matrices, Vmha is the feature representation after being processed by the multi-head attention mechanism, and Vmha is used as the second feature vector; Step S4: Fusing the first eigenvector and the second eigenvector to obtain a fused eigenvector Vcombined; The feature vector Vcapsule from the capsule network of the first channel and the feature vector Vmha obtained by BiGRU, TextCNN and attention mechanism of the second channel are concatenated to form a comprehensive feature vector Vcombined. The concatenation operation is expressed by the formula: In the formula, : represents the concatenation operation, which retains all elements of the two vectors; Step S5: input the fused feature vector Vcombined into the fully connected layer for feature mapping, and output the text detection result; The fused feature vector Vcombined is input to a fully connected layer with multiple neurons. The fully connected layer uses a softmax activation function for feature mapping to output the classification result of the text. The output of the fully connected layer is then passed to a batch normalization layer, which standardizes the data, stabilizes the training process of the neural network and significantly accelerates the convergence speed of the model. In offensive language detection tasks, it is usually mapped to two categories: offensive and non-offensive; The output is converted into a probability distribution through the Softmax activation function, indicating the probability that the input belongs to each category of the sentence: Y =softmax(W fc V combined +b fc ) Where Wfc and bfc are the weight and bias of the fully connected layer, respectively, and Y represents the classification output; In order to improve the training stability of the model and accelerate convergence, a batch normalization layer is added after the fully connected layer. The batch normalization layer optimizes the training process by adjusting and scaling the distribution of input features: V norm =BanchNorm(V combined ) The final output is a prediction result, which determines whether the text contains offensive language.

2. The dual-channel Chinese offensive language detection method according to claim 1, characterized in that: The BERT model is a pre-trained language representation model based on the Transformers architecture. By pre-training on a large-scale text corpus, the learned language representation can be transferred to downstream NLP tasks.

3. The dual-channel Chinese offensive language detection method according to claim 1, characterized in that: The combination of the BERT model and the dual-channel model enhances the recognition of offensive language while improving the understanding of the semantic depth of the text.

4. The dual-channel Chinese offensive language detection method according to claim 1, characterized in that: Capsule network is a network structure designed to capture hierarchical relationships in data. In text processing, capsule network can effectively simulate the hierarchical relationships and dynamic features between words.

5. The dual-channel Chinese offensive language detection method according to claim 1, characterized in that: The Chinese offensive language dataset includes: 37,480 sentences, covering offensive topics related to race, gender and region.

6. The dual-channel Chinese offensive language detection method according to claim 1, characterized in that: The sub-dataset contains 25,726 sentences, of which 12,723 have offensive labels, i.e., data with labels of 1, and 13,003 have non-offensive labels, i.e., data with labels of 0.

7. The dual-channel Chinese offensive language detection method according to claim 1, characterized in that: The sub-datasets are divided into 80%, 10%, and 10% proportions to ensure that the performance of the model on different sub-datasets can be fully evaluated.

8. The dual-channel Chinese offensive language detection method according to claim 1, characterized in that: The specific configuration of the BERT model includes: the number of hidden layers is 12, the hidden layer size is 768, that is, the vector dimension of each word or word fragment is 768, and the number of attention heads is 12.

9. The dual-channel Chinese offensive language detection method according to claim 1, characterized in that: The format that the BERT model can understand includes conversion to word index, and necessary padding and truncation to meet the maximum length requirement of the model. The input index is converted to a d-dimensional vector through the embedding layer. The embedding layer of the BERT model includes the sum of word embedding, position embedding and segment embedding.

10. A dual-channel Chinese offensive language detection system, characterized in that: include: BERT module, capsule network module, BiGRU module, TextCNN module, multi-head attention mechanism module; Step M1: Obtain text data to be detected; Step M1-1: Use the Chinese offensive language dataset to initially select from multiple online platforms. The Chinese offensive language dataset is initially screened to exclude data with unclear semantics or incorrect annotations to obtain a sub-dataset; Step M1-2: preprocessing the sub-dataset; Step M1-2-1: Sub-dataset preprocessing includes cleaning, word segmentation, tagging and standardization, and the preprocessed sub-dataset is divided into training set, validation set and test set according to the proportion; Step M2: The BERT module is implemented by processing the sub-dataset and generating word vectors Ebert that capture the semantic details in the context; Step M2-1: The specific method of generating word vector Ebert is as follows: Step M2-2: For each text X, it is first tokenized. After tokenization, each token is mapped to a unique index. Step M2-3: Add a CLS tag at the beginning of the sentence and a SEP tag at the end of the sentence. The sentence is converted into a format that the BERT model can understand after word segmentation and standardization. After being processed by all Transformer layers, the word vector Ebert of the overall semantic information is obtained; Step M3: Input the word vector Ebert into two parallel channels for processing; Step M3-1: The first channel is processed by the capsule network module to generate the first feature vector Vcapsule; the second channel is first processed by the bidirectional gated recurrent unit BiGRU module, then by the text convolutional neural network TextCNN module, and finally by the multi-head attention mechanism module to generate the second feature vector Vmha; Step M3-2: The capsule network module is specifically implemented as follows: the capsule network module in the first channel is used to capture the spatial relationship between word vectors, and can enhance the feature expression capability through a dynamic routing mechanism. The word vector Ebert is used as input to generate a capsule feature vector Vcapsule, and it is used as the first feature vector; Before entering the capsule network module, the word vector Ebert is usually preprocessed, including rescaling the dimension and linear transformation to adapt to the input requirements of the capsule network; HAVE BEEN bert ′=E bert W trans +b trans Where Wtrans and btrans are the weight and bias of the conversion layer respectively; In each iteration, the output of the capsule network module is dynamically sent to the capsules in the higher layer through the routing process; The routing process usually includes calculating the agreement score between capsules, which determines the contribution of the lower-layer capsule to the output of the upper-layer capsule; Where cij is the connection weight of capsule i to capsule j, bij is the initial logarithmic prior probability, which is dynamically adjusted through the iterative process; Then, vector conversion and activation are implemented. Each capsule output vector sj is obtained by multiplying all input vectors u^j|i with the corresponding routing weight cij and then summing them up: A compressed nonlinear activation function is used to transform sj into the final capsule output vj: vj is the final output first feature vector Vcapsule, that is, Vcapsule is obtained from the top-level capsule network; Step M3-3: The feature vector generation method of the second channel includes the following steps: The specific implementation of the BiGRU module is as follows: the word vector Ebert is taken as input, and the BiGRU module processes the input word vector Ebert through the forward and backward GRU units to capture the long-distance dependency of the text, and the output feature vector is represented as HBiGRU; H BiGRU =BigGRU(E bert ) The feature vector VBiGRU of the BiGRU module is generated by selecting the hidden state of the last time step or applying additional operations, where the output ht at each time step is the forward and backward Output connections; The specific implementation method of the TextCNN module is as follows: through the TextCNN module, the local features of the text are captured and the feature vector of the TextCNN module is generated; The TextCNN module processes the input feature vector VBiGRU of the BiGRU module by applying multiple convolution kernels and corresponding max pooling operations to capture local features; For each convolution kernel k, a convolution operation is performed and a nonlinear activation function ReLU is applied to obtain the convolution output result Ck, where Wk and bk are the weight and bias of the convolution kernel k respectively; C k =ReLU(W k *V BiGRU +b k ) Apply the maximum pooling operation to the convolutional layer output Ck to obtain the most important features; P k =MaxPool(C k ) Concatenate the pooled outputs of all convolution kernels k to form the feature vector VTextCNN of the TextCNN module; The multi-head attention mechanism module is implemented as follows: VTextCNN is used as the input of the multi-head attention layer, and information is processed in parallel through multiple independent attention heads. The formula of the attention mechanism is: Where Q, K, and V represent the query vector, key vector, and value vector, respectively. QKT is the dot product of the query vector and the key vector, indicating the similarity. dk is the dimension of the key vector. Softmax is a normalization function used to convert the similarity into a probability distribution. Each head calculates the attention score and updates the feature representation, and then the outputs of all heads are concatenated and linearly transformed to the original dimension, that is: head i =Attention(V TextCNN W i Q ,V TextCNN W i K ,V TextCNN W i V ) V mha =Concat(head1,......,head h )W O Among them, WQi and WO are learnable weight matrices, Vmha is the feature representation after being processed by the multi-head attention mechanism, and Vmha is used as the second feature vector; Step M4: Fusing the first eigenvector and the second eigenvector to obtain a fused eigenvector Vcombined; The feature vector Vcapsule from the capsule network module of the first channel and the feature vector Vmha obtained by the BiGRU module, the TextCNN module and the attention mechanism module of the second channel are concatenated to form a comprehensive feature vector Vcombined. The concatenation operation is expressed by the formula: In the formula, : represents the concatenation operation, which retains all elements of the two vectors; Step M5: Input the fused feature vector Vcombined into the fully connected layer for feature mapping, and output the text detection result; The fused feature vector Vcombined is input to a fully connected layer with multiple neurons. The fully connected layer uses a softmax activation function for feature mapping to output the classification result of the text. The output of the fully connected layer is then passed to a batch normalization layer, which standardizes the data, stabilizes the training process of the neural network and significantly accelerates the convergence speed of the model. In offensive language detection tasks, it is usually mapped to two categories: offensive and non-offensive; The output is converted into a probability distribution through the Softmax activation function, indicating the probability that the input belongs to each category of the sentence: Y =softmax(W fc V combined +b fc ) Where Wfc and bfc are the weight and bias of the fully connected layer, respectively, and Y represents the classification output; In order to improve the training stability of the model and accelerate convergence, a batch normalization layer is added after the fully connected layer. The batch normalization layer optimizes the training process by adjusting and scaling the distribution of input features: V norm =BanchNorm(V combined ) The final output is a prediction result, which determines whether the text contains offensive language.

Images