A method and system for detecting associative sensitive words
By learning the usage of normal sentences and error detection of mask language models, combined with twin neural networks and self-attention mechanisms, accurate detection of variant characters is achieved, and the dependence on predefined libraries and a large number of in-depth knowledge is solved. The problem of labeling data improves detection efficiency and accuracy.
Patent Information
- Application Number
- CN202411111844.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-14
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2044-08-14
AI Technical Summary
The existing sensitive word detection methods rely on predefined libraries and a large number of labeled data, making it difficult to detect substitutions of variant characters, homophones and shapes, resulting in high missed detection rates and error detection rates, and difficult data acquisition and high data labeling costs.
By learning how to use normal sentences, using mask language model for error detection, combining twin neural networks and self-attention mechanisms to judge the similarity of Chinese characters, generate suspected sensitive words and make similar matches, and achieve accurate information on sensitive words containing variant characters. Test.
It improves the detection ability of sensitive words for variant characters, reduces the workload of manual review, and is suitable for content review and information filtering, ensuring the comprehensiveness and accuracy of the inspection.
Smart Images

Figure CN118627507B_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present application relate to, but are not limited to, the field of machine learning technology, and in particular, to a method and system for detecting associative sensitive words. Background Art
[0002] In the field of natural language processing (NLP), sensitive word detection is an important research direction and is widely used in content review, information filtering, network security and other fields. Traditional sensitive word detection methods usually rely on predefined sensitive word libraries to detect sensitive words in texts by direct matching. However, these methods have several obvious defects: first, the sensitive word library needs to be updated frequently to cope with the emergence of new sensitive words and variant words; second, the direct matching method is difficult to detect sensitive words replaced by variant characters, homophones and similar characters, resulting in high missed detection rate and false detection rate.
[0003] In recent years, in order to improve the accuracy and coverage of sensitive word detection, models based on deep learning have gradually attracted attention. These models can automatically learn complex features in text through large-scale data training, thereby improving the effect of sensitive word detection. However, most existing methods rely on a large amount of labeled data for training, which has problems such as difficulty in data acquisition and high data labeling costs in practical applications. Summary of the invention
[0004] In view of the above problems existing in the prior art, this application proposes a method and system for detecting associative sensitive words. This method predicts the Chinese characters that may contain errors in sentences by learning the usage of normal sentences, and combines these Chinese characters into suspected sensitive words for similar matching, thereby detecting sensitive words containing variant characters. The technical solution adopted by this application is as follows:
[0005] A method for detecting associative sensitive words, the method comprising the following steps:
[0006] S1, obtaining a data set containing variant characters, and preprocessing the data set containing variant characters to generate a sensitive word sentence data set containing variant characters;
[0007] S2. Use a masked language model to perform error detection on the sensitive word sentence data set, detect spelling errors and grammatical errors caused by variants, and generate a set of suspected sensitive words;
[0008] S3, sorting the sensitive word library, and performing similarity matching between the suspected sensitive word set and the sensitive words in the sorted sensitive word library to generate a sensitive word set;
[0009] S4. The input word is feature extracted through the Chinese character feature extraction layer to obtain feature encoding, and the Chinese character similarity is judged based on the twin neural network and self-attention mechanism to match sensitive words, and finally the sensitive word detection result is obtained.
[0010] Furthermore, the step S1 specifically includes the following sub-steps:
[0011] S101, simplifying the collected sensitive words, removing redundant variant sensitive words and normal words;
[0012] S102, sorting the processed data by blocks according to pinyin, number of strokes and word length;
[0013] S103, performing partial or complete variant replacement on the sorted data by synonym replacement, variant character replacement, or similar-sounding and similar-shaped character replacement to generate a sensitive word sentence data set containing variant characters.
[0014] Furthermore, the step S2 specifically includes the following sub-steps:
[0015] S201, performing masking processing on the sentences in the sensitive word sentence dataset, and inputting the masked sentences and their embedded representations into a masked language model to obtain a predicted probability distribution score;
[0016] S202, comparing the predicted probability distribution score with the average absolute deviation, and marking words below the standard value as wrong words;
[0017] S203: Restoring the erroneous word to the phrase corresponding to the word, and generating a set of suspected sensitive words.
[0018] Further, in step S4, the determination of Chinese character similarity based on the twin neural network and the self-attention mechanism for sensitive word matching specifically includes:
[0019] Construct a twin neural network, and extract the common features of the feature encoding of the words in the sensitive word set and the input words through two convolutional neural network sub-networks with shared parameters;
[0020] A self-attention mechanism is introduced to calculate the attention energy of the feature encoding position of each word in the common feature, and the attention energy is normalized into an attention weight through a Softmax function;
[0021] The outputs of the two convolutional neural network sub-networks are weighted and fused to obtain the two attention output vectors that finally represent the self-attention weight information;
[0022] The difference between the two attention output vectors is calculated, and the similarity prediction value is obtained through the fully connected layer and the Sigmoid activation function. The similarity of the Chinese characters is judged according to the similarity prediction value to perform sensitive word matching, and finally the sensitive word detection result is obtained.
[0023] A detection system for associative sensitive words, used to implement the above method, the system comprises:
[0024] A data acquisition module, used for acquiring a data set containing variant characters, and preprocessing the data set containing variant characters to generate a sensitive word sentence data set containing variant characters;
[0025] An error detection module, used to perform error detection on the sensitive word sentence data set using a masked language model, detect spelling errors and grammatical errors caused by variants, and generate a set of suspected sensitive words;
[0026] A sensitive word matching module, used for similarly matching the suspected sensitive word set with the sorted sensitive words in the sensitive word library to generate a sensitive word set;
[0027] The Chinese character similarity matching module is used to extract features of input words through the Chinese character feature extraction layer to obtain feature encoding, and judge the similarity of Chinese characters based on the twin neural network and self-attention mechanism to match sensitive words, and finally obtain sensitive word detection results.
[0028] Furthermore, the data acquisition module includes:
[0029] The data filtering submodule is used to simplify the collected sensitive words and remove unnecessary variant sensitive words and normal words;
[0030] The text segmentation submodule is used to sort the data processed by the data filtering submodule in blocks according to pinyin, number of strokes and word length;
[0031] The part-of-speech tagging submodule is used to perform partial or complete variant replacement on the sorted data by synonym replacement, variant character replacement, or similar-sounding and similar-shaped character replacement to generate a sensitive word sentence dataset containing variant characters.
[0032] Furthermore, the error detection module includes:
[0033] A masking scoring submodule, used to perform masking on the sentences in the sensitive word sentence dataset, and input the masked sentences and their embedded representations into a masked language model to obtain a predicted probability distribution score;
[0034] A word detection submodule, used for comparing the predicted probability distribution score with the mean absolute deviation, and marking words below the standard value as wrong words;
[0035] The word combination submodule is used to restore the erroneous word to the phrase corresponding to the word and generate a set of suspected sensitive words.
[0036] Furthermore, the method of judging the similarity of Chinese characters based on the twin neural network and the self-attention mechanism to match sensitive words specifically includes:
[0037] Construct a twin neural network, and extract the common features of the feature encoding of the words in the sensitive word set and the input words through two convolutional neural network sub-networks with shared parameters;
[0038] A self-attention mechanism is introduced to calculate the attention energy of the feature encoding position of each word in the common feature, and the attention energy is normalized into an attention weight through a Softmax function;
[0039] The outputs of the two convolutional neural network sub-networks are weighted and fused to obtain the two attention output vectors that finally represent the self-attention weight information;
[0040] The difference between the two attention output vectors is calculated, and the similarity prediction value is obtained through the fully connected layer and the Sigmoid activation function. The similarity of Chinese characters is judged based on the similarity prediction value to perform sensitive word matching, and finally the sensitive word detection result is obtained.
[0041] An electronic device comprises: a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the above method when executing the computer program.
[0042] A computer-readable medium stores a computer-executable program, wherein the computer-executable program is used to enable a computer to execute the above method.
[0043] Through the embodiments of the present application, the following technical effects can be obtained: by learning the usage of normal sentences for error detection, and combining word feature encoding and similarity judgment, accurate matching of sensitive words containing variant characters can be achieved, so that sensitive words recognized by readers' association can be automatically marked. This method not only overcomes the limitations of traditional methods that rely on a large number of sensitive data sets for training, but also improves the detection ability of variant characters and sensitive words, and is suitable for content review, information filtering and other fields. Specifically, the system uses a masked language model for error detection, combines a twin neural network and a self-attention mechanism to judge the similarity of Chinese characters, and can effectively identify spelling errors and grammatical collocation errors caused by variant characters, and accurately match sensitive words. Through automated processing, the system greatly improves detection efficiency and reduces the workload of manual review, while maintaining efficient detection capabilities when the amount of data is small. In particular, the optimized processing of variant characters ensures the comprehensiveness and accuracy of detection, making it widely applicable to multiple application scenarios such as content review of social media platforms and information filtering within enterprises.
[0044] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] In order to more clearly illustrate the technical solutions in the embodiments of the present application, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
[0046] Figure 1 This is a schematic diagram of the process framework of the sensitive word detection method;
[0047] Figure 2 It is a schematic diagram of the structure of the detection system;
[0048] Figure 3 Schematic diagram of an error detection example.
[0049] Description of reference numerals:
[0050] 101 data acquisition module, 102 error detection module, 103 word combination module, 104 Chinese character similarity identification module, 105 sensitive word matching module containing variant characters. DETAILED DESCRIPTION
[0051] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0052] It should be understood that in the description of the embodiments of the present application, the meaning of multiple (or multiple) is more than two, greater than, less than, and exceeding are understood to exclude the number itself, and above, below, and within are understood to include the number itself. If there is a description of "first", "second", etc., it is only used for the purpose of distinguishing technical features, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.
[0053] In the first aspect, the present application proposes a method for detecting associative sensitive words, such as Figure 1 As shown, the method comprises the following steps:
[0054] S1, obtaining a data set containing variant characters, and preprocessing the data set containing variant characters to generate a sensitive word sentence data set containing variant characters;
[0055] The step S1 specifically includes the following sub-steps:
[0056] S101, simplifying the collected sensitive words, removing redundant variant sensitive words and normal words;
[0057] S102, used for sorting the data processed by the data filtering submodule in blocks according to pinyin, number of strokes and word length;
[0058] S103, performing partial or complete variant replacement on the sorted data by synonym replacement, variant character replacement, or similar-sounding and similar-shaped character replacement to generate a sensitive word sentence data set containing variant characters.
[0059] Sensitive words are collected through the above data acquisition module, and the collected sensitive words are simplified, redundant variant sensitive words and normal words are removed, and they are sorted in blocks according to pinyin, number of strokes and word length. For the existing sensitive word library on the network, which often contains sensitive words that have been replaced by variants, as well as a large number of normal words, the collected sensitive words are processed, redundant variants and normal words are removed, and they are sorted in blocks according to pinyin, number of strokes and word length, which is conducive to improving the matching efficiency in the later stage. When collecting sensitive words, it is also possible to crawl domestic websites (such as post bars, Weibo comment areas and game exchange areas, etc.), and collect some data sets that can be used for sensitive word detection through manual screening and filtering. Then crawl foreign websites (such as Twitter tweets and comment data), filter data with unclear meanings or chaotic formats, and retain data containing English sensitive words. Because English sensitive words are less restricted abroad than Chinese sensitive words, there are fewer types, and it is more difficult to form variants. For each English sensitive word, a Chinese sensitive word with similar meaning is selected to pair it with it, and a large number of collected English sensitive data sets are translated and randomly replaced by synonyms to generate the corresponding Chinese sensitive word data set. Through this method, the system can effectively identify English sensitive words, which not only improves the system's ability to detect multilingual sensitive words, but also ensures the comprehensiveness and accuracy of the detection. In particular, through random synonym replacement and variant replacement, the system can capture more variant forms, further improving the coverage and accuracy of sensitive word detection. In addition, the above-mentioned data acquisition module replaces part or all of the variants of the collected sensitive word data set according to the method of word replacement, and obtains the final sensitive word sentence data set containing variant characters. This variant replacement method can not only capture more variant forms, further improve the coverage and accuracy of sensitive word detection, but also effectively deal with spelling errors and grammatical collocation errors caused by variant characters. In addition, some normal sentences and normal sentences replaced by variants are added to the above-mentioned sensitive word sentence data set to test the false alarm rate of the detection system. In this way, the system can effectively distinguish normal sentences from sensitive word sentences, reduce false alarms, and improve the accuracy and reliability of detection. In particular, by introducing normal sentences and normal sentences replaced by variants, the system can better cope with complex language environments in practical applications and ensure the accuracy of detection results.
[0060] In the data preprocessing stage, a natural language processing tool is used to obtain the word segmentation results and part-of-speech information of the sentence. For example, the natural language processing tool is the HanLP tool. By performing word segmentation and part-of-speech tagging on the sentence, the system can better understand the structure and semantics of the sentence, thereby providing more accurate basic data in the subsequent error detection and sensitive word matching process.
[0061] S2. Use a masked language model to perform error detection on the sensitive word sentence data set, detect spelling errors and grammatical errors caused by variants, and generate a set of suspected sensitive words;
[0062] The step S2 specifically includes the following sub-steps:
[0063] S201, performing masking processing on the sentences in the sensitive word sentence dataset, and inputting the masked sentences and their embedded representations into a masked language model to obtain a predicted probability distribution score;
[0064] S202, comparing the predicted probability distribution score with the average absolute deviation, and marking words below the standard value as wrong words;
[0065] S203: Restoring the erroneous word to the phrase corresponding to the word, and generating a set of suspected sensitive words.
[0066] The masking of the sentences in the sensitive word sentence data set specifically includes:
[0067] Get the input sentence ,in is the first words;
[0068] Mask a word in the sentence and replace it with the mask token , the mask tag represents the word at that position predicted by the mask language model.
[0069] The input is scored in the masked language model, specifically including:
[0070] The masked language model encodes the sentence through a multi-layer Transformer encoder to generate a series of hidden states , where each hidden state It captures rich information related to the sentence context; by Apply a fully connected layer and use the softmax function to calculate the predicted probability distribution score for each word;
[0071] set up is the embedding matrix of all words in the vocabulary of the masked language model, with dimension ,in is the embedding dimension, is the number of words in the vocabulary; let is a weight matrix with dimension , used to hide the state Project to the same dimension as the word embedding and compute the predicted probability distribution score.
[0072] The comparing the predicted probability distribution score with the mean absolute deviation specifically includes:
[0073] The average of the predicted probability distribution scores is calculated and compared with the mean absolute deviation, which is calculated as follows:
[0074] ;
[0075] ;
[0076] ;
[0077] in, is the length of the sequence, Represents a given sentence In the case of The probability of is the average value, is the mean absolute deviation, is a word detection function, when the indicator function True if the conditions in brackets are met. is a custom threshold parameter.
[0078] The reference basis for restoring the erroneous words includes the predicted probability distribution score, word segmentation results and part-of-speech tagging. The following is a method for restoring the erroneous words according to different situations:
[0079] ① When the abnormal character appears at the beginning or end, if the adjacent characters are not segmented into words, the abnormal character and the adjacent characters are combined into a suspected sensitive word; if the adjacent characters are segmented into words that are not human names, and the word does not contain the abnormal character, the abnormal character and the word in which the adjacent characters are located are combined into a suspected sensitive word; if the adjacent characters are segmented into words that are not human names, and the word contains the abnormal character, the word containing the abnormal character is regarded as a suspected sensitive word.
[0080] ② If the abnormal character is segmented into a word, the word will be regarded as a suspected sensitive word.
[0081] ③ If the abnormal word is preceded and followed by words that have not been segmented into words, the word with the predicted score is combined with the abnormal word to form a suspected sensitive word.
[0082] ④ If the abnormal character is a word that is segmented into a word on one side and a word that is not segmented into a word on the other side, the abnormal character and the word that is not segmented into a word are combined as a suspected sensitive word. If the prediction score of the word that is segmented into a word is relatively low, the abnormal character and the word that is segmented into a word are combined as a suspected sensitive word.
[0083] ⑤ If the abnormal word is preceded and followed by words that are segmented into words, the word containing the word with a lower prediction score is combined with the abnormal word as a suspected sensitive word.
[0084] By using the masked language model to perform error detection on the sensitive word sentence dataset, we can effectively detect spelling errors and grammatical errors caused by variants, and generate a set of suspected sensitive words. This step not only improves the accuracy and efficiency of error detection, but also ensures high-precision matching of variant words and sensitive words through multi-layer Transformer encoders and self-attention mechanisms, thereby significantly improving the application effect of the system in multiple fields such as content review and information filtering.
[0085] S3, sorting the sensitive word library, and performing similarity matching between the suspected sensitive word set and the sensitive words in the sorted sensitive word library to generate a sensitive word set;
[0086] In the above steps, the sensitive word library is collected and sorted, and the library is streamlined to ensure the efficiency of matching; the sensitive words in the library are sorted by strokes and number of characters, and the word restriction area when matching sensitive words is limited; after completing the processing of the sensitive word library, the suspected sensitive word set combined by the word combination submodule is matched with the sensitive word library for similar sound or similar shape. If each word of the two words in the match reaches the similarity threshold, the current word is determined to be a variant of the sensitive word, and finally the sensitive word detection containing variant characters is completed, and a sensitive word set is generated. This step significantly improves the accuracy and efficiency of sensitive word matching by streamlining and sorting the sensitive word library, combining the similar sound and similar shape matching technology, and ensures that the system can efficiently and accurately detect sensitive words containing variant characters.
[0087] S4. The input word is feature extracted through the Chinese character feature extraction layer to obtain feature encoding, and the Chinese character similarity is judged based on the twin neural network and self-attention mechanism to match sensitive words, and finally the sensitive word detection result is obtained.
[0088] The feature coding includes pronunciation feature coding of similar-sounding characters and shape feature coding of similar-shaped characters;
[0089] The pronunciation features include initial consonants, final consonants and tones in Chinese Pinyin, and the fuzzy sounds are optimized; the character shape features include Chinese character structure, four-corner coding and stroke features;
[0090] In step S4, the method of judging the similarity of Chinese characters based on the twin neural network and the self-attention mechanism to match sensitive words specifically includes:
[0091] Construct a twin neural network, and extract the common features of the feature encoding of the words in the sensitive word set and the input words through two convolutional neural network sub-networks with shared parameters;
[0092] A self-attention mechanism is introduced to calculate the attention energy of the feature encoding position of each word in the common feature, and the attention energy is normalized into attention weights through a Softmax function; the outputs of the two convolutional neural network sub-networks are weightedly fused to obtain two attention output vectors that finally represent the self-attention weight information;
[0093] The difference between the two attention output vectors is calculated, and the similarity prediction value is obtained through the fully connected layer and the Sigmoid activation function. The similarity of Chinese characters is judged based on the similarity prediction value to perform sensitive word matching, and finally the sensitive word detection result is obtained.
[0094] The Chinese character similarity matching module uses a twin neural network and self-attention mechanism to compare the similarity of two Chinese characters. This method aims to extract common features from the input code and judge the similarity of two Chinese characters based on these features. First, a twin neural network structure is constructed, which contains two sub-networks of the same convolutional neural network. The two sub-networks correspond to the two input codes respectively. They share parameters so that the common features of the input codes can be extracted. Subsequently, a self-attention mechanism is introduced to capture the long-range dependencies in the input sequence. The mechanism first calculates the attention energy of each position and then normalizes it to the attention weight through the Softmax function. Finally, the outputs of the convolutional neural network sub-networks are weighted fused using these attention weights to obtain the final representation. After the two input codes are processed by the same twin neural network structure and self-attention mechanism, two attention output vectors are obtained. By calculating the difference between the two vectors and passing through a fully connected layer and a Sigmoid activation function, their similarity prediction value is obtained, and the similarity of the two Chinese character codes is judged based on this prediction value. By combining the pronunciation feature encoding of similar-sounding characters and the glyph feature encoding of similar-looking characters, the system can more comprehensively capture the multi-dimensional features of Chinese characters, thereby improving the accuracy and reliability of sensitive word matching. In particular, by introducing the self-attention mechanism, the system can capture long-distance dependencies in the input sequence, further improving the accuracy of similarity judgment. Therefore, this step achieves efficient judgment of Chinese character similarity and sensitive word matching through the combination of Chinese character feature extraction, twin neural networks and self-attention mechanism, which not only improves the system's ability to detect variant characters and sensitive words, but also ensures the accuracy and reliability of the detection results. It is suitable for content review, information filtering and other fields, and provides strong support for research and application in related fields.
[0095] In practical applications, the detection method of associative sensitive words of the present application can be used to monitor online content and offline files, help identify and filter potentially harmful content containing sarcasm, and effectively improve the accuracy and efficiency of content review. The method of the present invention also has broad application prospects in content distribution networks (CDNs).
[0096] Specifically, the content audit of the CDN system can be real-time or offline. Real-time auditing is based on traffic, for example, it is monitored and audited in real time during data transmission. Offline auditing is based on files, and the data is analyzed and processed after storage. Real-time auditing can instantly detect and filter bad content for traffic, including content that cache nodes are ready to distribute, as well as user comments and user-uploaded content, to ensure the timeliness and security of the content; while offline auditing is suitable for in-depth analysis and batch processing of stored files, such as files stored by CDN's intelligent backup source. By combining real-time and offline auditing methods, the system can fully cover various content auditing needs. Content auditing can be performed on edge nodes or on intelligent backup sources. By performing content auditing on different parts of the CDN network architecture, CDN service providers can ensure the security and compliance of transmitted content while improving the overall performance of the system and user satisfaction.
[0097] In a second aspect, the present application proposes a detection system for associative sensitive words, the system comprising a data acquisition module, an error detection module, a word combination module, a Chinese character similarity matching module, and a sensitive word matching module;
[0098] The data acquisition module is used to acquire a data set containing variant characters, and pre-process the data set containing variant characters to generate a sensitive word sentence data set containing variant characters;
[0099] The data acquisition module includes a data filtering submodule, a text segmentation submodule, and a part-of-speech tagging submodule;
[0100] The data filtering submodule is used to simplify the collected sensitive words and remove unnecessary variant sensitive words and normal words;
[0101] The text segmentation submodule is used to sort the data processed by the data filtering submodule in blocks according to pinyin, number of strokes and word length;
[0102] The part-of-speech tagging submodule is used to perform partial or complete variant replacement on the block-sorted data by means of synonym replacement, variant character replacement, or similar-sounding and similar-shaped character replacement to generate a sensitive word sentence data set containing variant characters.
[0103] Sensitive words are collected through the above data acquisition module, and the collected sensitive words are simplified, redundant variant sensitive words and normal words are removed, and they are sorted in blocks according to pinyin, number of strokes and word length. For the existing sensitive word library on the network, which often contains sensitive words that have been replaced by variants, as well as a large number of normal words, the collected sensitive words are processed, redundant variants and normal words are removed, and they are sorted in blocks according to pinyin, number of strokes and word length, which is conducive to improving the matching efficiency in the later stage. When collecting sensitive words, it is also possible to crawl domestic websites (such as post bars, Weibo comment areas and game exchange areas, etc.), and collect some data sets that can be used for sensitive word detection through manual screening and filtering. Then crawl foreign websites (such as Twitter tweets and comment data), filter data with unclear meanings or chaotic formats, and retain data containing English sensitive words. Because English sensitive words are less restricted abroad than Chinese sensitive words, there are fewer types, and it is more difficult to form variants. For each English sensitive word, a Chinese sensitive word with similar meaning is selected to pair it with it, and a large number of collected English sensitive data sets are translated and randomly replaced by synonyms to generate the corresponding Chinese sensitive word data set. In addition, the data acquisition module replaces part or all of the variants of the collected sensitive word data set by word replacement to obtain a final sensitive word sentence data set containing variant characters. In addition, some normal sentences and normal sentences replaced by variants are added to the sensitive word sentence data set to test the false alarm rate of the detection system.
[0104] The error detection module is used to perform error detection on the sensitive word sentence data set using a masked language model, detect spelling errors and grammatical errors caused by variants, and generate a set of suspected sensitive words;
[0105] The error detection module includes a mask scoring submodule, a word detection submodule, and a word combination submodule;
[0106] The masked scoring submodule is used to perform masking on the sentences in the sensitive word sentence data set, and input the masked sentences and their embedded representations into a masked language model to obtain a predicted probability distribution score; the masked language model is a Chinesebert model;
[0107] The masking of the sentences in the sensitive word sentence data set specifically includes:
[0108] Get the input sentence ,in is the first words;
[0109] Mask a word in the sentence and replace it with the mask token , the mask tag represents the word at that position predicted by the mask language model.
[0110] The input is scored in the masked language model, specifically including:
[0111] The masked language model encodes the sentence through a multi-layer Transformer encoder to generate a series of hidden states , where each hidden state It captures rich information related to the sentence context; by Apply a fully connected layer and use the softmax function to calculate the predicted probability distribution score for each word;
[0112] set up is the embedding matrix of all words in the vocabulary of the masked language model, with dimension ,in is the embedding dimension, is the number of words in the vocabulary; let is a weight matrix with dimension , used to hide the state Project to the same dimension as the word embedding and compute the predicted probability distribution score.
[0113] The word detection submodule is used to compare the predicted probability distribution score with the average absolute deviation, and mark the words below the standard value as wrong words;
[0114] The comparing the predicted probability distribution score with the mean absolute deviation specifically includes:
[0115] The average of the predicted probability distribution scores is calculated and compared with the mean absolute deviation, which is calculated as follows:
[0116] ;
[0117] ;
[0118] ;
[0119] in, is the length of the sequence, Represents a given sentence In the case of The probability of is the average value, is the mean absolute deviation, is a word detection function, when the indicator function True if the conditions in brackets are met. is a custom threshold parameter.
[0120] The word combination submodule is used to restore the erroneous word to the corresponding phrase of the word and generate a set of suspected sensitive words;
[0121] The reference basis for restoring the erroneous words includes the predicted probability distribution score, word segmentation results and part-of-speech tagging. The following is a method for restoring the erroneous words according to different situations:
[0122] ① When the abnormal character appears at the beginning or end, if the adjacent characters are not segmented into words, the abnormal character and the adjacent characters are combined into a suspected sensitive word; if the adjacent characters are segmented into words that are not human names, and the word does not contain the abnormal character, the abnormal character and the word in which the adjacent characters are located are combined into a suspected sensitive word; if the adjacent characters are segmented into words that are not human names, and the word contains the abnormal character, the word containing the abnormal character is regarded as a suspected sensitive word.
[0123] ② If the abnormal character is segmented into a word, the word will be regarded as a suspected sensitive word.
[0124] ③ If the abnormal word is preceded and followed by words that have not been segmented into words, the word with the predicted score is combined with the abnormal word to form a suspected sensitive word.
[0125] ④ If the abnormal character is a word that is segmented into a word on one side and a word that is not segmented into a word on the other side, the abnormal character and the word that is not segmented into a word are combined as a suspected sensitive word. If the prediction score of the word that is segmented into a word is relatively low, the abnormal character and the word that is segmented into a word are combined as a suspected sensitive word.
[0126] ⑤ If the abnormal word is preceded and followed by words that are segmented into words, the word containing the word with a lower prediction score is combined with the abnormal word as a suspected sensitive word.
[0127] The sensitive word matching module is used to organize the sensitive word library, and perform similarity matching between the suspected sensitive word set and the sensitive words in the organized sensitive word library to generate a sensitive word set;
[0128] The sensitive word matching module collects and organizes the sensitive word library, simplifies the library to ensure the efficiency of matching; sorts the sensitive words in the library by strokes and number of characters, and limits the word limit area when matching sensitive words; after completing the processing of the sensitive word library, the suspected sensitive word set combined by the word combination submodule is matched with the sensitive word library for similar sound or similar shape. If each character of the two words in the match reaches the similarity threshold, the current word is determined to be a variant of the sensitive word, and finally the sensitive word detection containing variant characters is completed, and a sensitive word set is generated.
[0129] The Chinese character similarity matching module is used to extract features of input words through the Chinese character feature extraction layer to obtain feature coding, and judge the similarity of Chinese characters based on the twin neural network and self-attention mechanism to perform sensitive word matching, and finally obtain sensitive word detection results.
[0130] The feature coding includes pronunciation feature coding of similar-sounding characters and shape feature coding of similar-shaped characters;
[0131] The pronunciation features include initial consonants, final consonants and tones in Chinese Pinyin, and the fuzzy sounds are optimized; the character shape features include Chinese character structure, four-corner coding and stroke features;
[0132] The method of judging the similarity of Chinese characters based on the twin neural network and the self-attention mechanism to match sensitive words specifically includes:
[0133] Construct a twin neural network, and extract the common features of the feature encoding of the words in the sensitive word set and the input words through two convolutional neural network sub-networks with shared parameters;
[0134] A self-attention mechanism is introduced to calculate the attention energy of the feature encoding position of each word in the common feature, and the attention energy is normalized into attention weights through a Softmax function; the outputs of the two convolutional neural network sub-networks are weightedly fused to obtain two attention output vectors that finally represent the self-attention weight information;
[0135] The difference between the two attention output vectors is calculated, and the similarity prediction value is obtained through the fully connected layer and the Sigmoid activation function. The similarity of Chinese characters is judged based on the similarity prediction value to perform sensitive word matching, and finally the sensitive word detection result is obtained.
[0136] The Chinese character similarity matching module uses a twin neural network and self-attention mechanism to compare the similarity of two Chinese characters. This method aims to extract common features from the input code and judge the similarity of two Chinese characters based on these features. First, a twin neural network structure is constructed, which contains two sub-networks of the same convolutional neural network. The two sub-networks correspond to the two input codes respectively. They share parameters so that the common features of the input codes can be extracted. Subsequently, a self-attention mechanism is introduced to capture the long-range dependencies in the input sequence. The mechanism first calculates the attention energy of each position and then normalizes it to the attention weight through the Softmax function. Finally, the outputs of the convolutional neural network sub-networks are weighted fused using these attention weights to obtain the final representation. After the two input codes are processed by the same twin neural network structure and self-attention mechanism, two attention output vectors are obtained. By calculating the difference between the two vectors and passing through a fully connected layer and a Sigmoid activation function, their similarity prediction value is obtained, and the similarity of the two Chinese character codes is judged based on this prediction value. This method can effectively extract common features from the input codes and judge the similarity of the two Chinese characters based on these features. It has high performance and accuracy in the task of Chinese character similarity discrimination, providing strong support for research and applications in related fields.
[0137] exist Figure 3 In the error detection example, first, a fusion layer is obtained by encoding the character layer, the glyph layer, and the pinyin layer. The Chinesebert pre-trained model (publicly downloadable) containing phonetic and glyph information trained with position encoding is used. Chinesebert is used to mask and score each Chinese character in the sentence to obtain the Chinese characters with abnormal scores. The scores are combined with the word segmentation part-of-speech information to restore the Chinese characters to their phrases. In summary, the present application provides a sensitive word detection system containing variant characters. By learning the usage of normal sentences, it predicts the erroneous Chinese characters that may exist in the sentences, and then combines these Chinese characters into suspected sensitive words for similar matching containing variant characters, and finally obtains sensitive word detection results.
[0138] In a third aspect, the present application proposes an electronic device, comprising: a memory and one or more processors.
[0139] One or more application programs are stored in the memory, and the one or more application programs are suitable for being executed by the one or more processors to implement the method described in the first aspect.
[0140] The electronic device includes: a processor and a memory, wherein the processor and the memory are connected, for example, via a bus.
[0141] The structure of the electronic device does not constitute a limitation on the embodiments of the present application.
[0142] The processor may be a CPU, a general-purpose processor, a DSP, an ASIC, an FPGA or other programmable logic device, a transistor logic device, a hardware component or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. The processor may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.
[0143] The bus may include a path to transmit information between the above components. The bus may be a PCI bus or an EISA bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc.
[0144] The memory can be a ROM or other type of static storage device that can store static information and instructions, a RAM or other type of dynamic storage device that can store information and instructions, or an EEPROM, CD-ROM or other optical disk storage, optical disk storage (including compressed optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.), magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited to this.
[0145] In a fourth aspect, the present application proposes a computer-readable storage medium having a computer program stored thereon, wherein the computer program can be loaded by a processor and execute the method described in the first aspect.
[0146] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the embodiments of the present application, rather than to limit them. Although the present application is described in detail with reference to the aforementioned embodiments, a person of ordinary skill in the art should understand that the technical solutions described in the aforementioned embodiments can still be modified, or some of the technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application. Any changes or replacements that can be easily thought of by a person of ordinary skill in the art within the technical scope disclosed in the present application should be covered within the scope of protection of the present application.
Claims
1. A method for detecting associative sensitive words, characterized in that: The method comprises the following steps: S1, obtaining a data set containing variant characters, and preprocessing the data set containing variant characters to generate a sensitive word sentence data set containing variant characters; S2. Use a masked language model to perform error detection on the sensitive word sentence data set, detect spelling errors and grammatical errors caused by variants, and generate a set of suspected sensitive words; S3, sorting the sensitive word library, and performing similarity matching between the suspected sensitive word set and the sensitive words in the sorted sensitive word library to generate a sensitive word set; S4, extracting features of input words through the Chinese character feature extraction layer to obtain feature encoding, and judging the similarity of Chinese characters based on the twin neural network and self-attention mechanism to match sensitive words, and finally obtaining the sensitive word detection result; The step S1 specifically includes the following sub-steps: S101, simplifying the collected sensitive words, removing redundant variant sensitive words and normal words; S102, sorting the processed data by blocks according to pinyin, number of strokes and word length; S103, performing partial or complete variant replacement on the sorted data by synonym replacement, variant character replacement, or similar-sounding and similar-shaped character replacement to generate a sensitive word sentence data set containing variant characters.
2. The method according to claim 1, characterized in that The step S2 specifically includes the following sub-steps: S201, performing masking processing on the sentences in the sensitive word sentence dataset, and inputting the masked sentences and their embedded representations into a masked language model to obtain a predicted probability distribution score; S202, comparing the predicted probability distribution score with the average absolute deviation, and marking words below the standard value as wrong words; S203: Restoring the erroneous word to the phrase corresponding to the word, and generating a set of suspected sensitive words.
3. The method according to claim 1, characterized in that In step S4, the method of judging the similarity of Chinese characters based on the twin neural network and the self-attention mechanism to match sensitive words specifically includes: Construct a twin neural network, and extract the common features of the feature encoding of the words in the sensitive word set and the input words through two convolutional neural network sub-networks with shared parameters; A self-attention mechanism is introduced to calculate the attention energy of the feature encoding position of each word in the common feature, and the attention energy is normalized into an attention weight through a Softmax function; The outputs of the two convolutional neural network sub-networks are weighted and fused to obtain the two attention output vectors that finally represent the self-attention weight information; The difference between the two attention output vectors is calculated, and the similarity prediction value is obtained through the fully connected layer and the Sigmoid activation function. The similarity of the Chinese characters is judged according to the similarity prediction value to perform sensitive word matching, and finally the sensitive word detection result is obtained.
4. A detection system for associative sensitive words, used to implement the method of any one of claims 1 to 3, the system comprising: A data acquisition module, used for acquiring a data set containing variant characters, and preprocessing the data set containing variant characters to generate a sensitive word sentence data set containing variant characters; An error detection module, used to perform error detection on the sensitive word sentence data set using a masked language model, detect spelling errors and grammatical errors caused by variants, and generate a set of suspected sensitive words; A sensitive word matching module, used for similarly matching the suspected sensitive word set with the sorted sensitive words in the sensitive word library to generate a sensitive word set; The Chinese character similarity matching module is used to extract features of input words through the Chinese character feature extraction layer to obtain feature encoding, and judge the similarity of Chinese characters based on the twin neural network and self-attention mechanism to match sensitive words, and finally obtain sensitive word detection results.
5. The system according to claim 4, characterized in that The data acquisition module comprises: The data filtering submodule is used to simplify the collected sensitive words and remove unnecessary variant sensitive words and normal words; The text segmentation submodule is used to sort the data processed by the data filtering submodule in blocks according to pinyin, number of strokes and word length; The part-of-speech tagging submodule is used to perform partial or complete variant replacement on the sorted data by synonym replacement, variant character replacement, or similar-sounding and similar-shaped character replacement to generate a sensitive word sentence dataset containing variant characters.
6. The system according to claim 4, characterized in that The error detection module comprises: A masking scoring submodule, used to perform masking on the sentences in the sensitive word sentence dataset, and input the masked sentences and their embedded representations into a masked language model to obtain a predicted probability distribution score; A word detection submodule, used for comparing the predicted probability distribution score with the mean absolute deviation, and marking words below the standard value as wrong words; The word combination submodule is used to restore the erroneous word to the phrase corresponding to the word and generate a set of suspected sensitive words.
7. The system according to claim 4, characterized in that The method of judging the similarity of Chinese characters based on the twin neural network and the self-attention mechanism to match sensitive words specifically includes: Construct a twin neural network, and extract the common features of the feature encoding of the words in the sensitive word set and the input words through two convolutional neural network sub-networks with shared parameters; A self-attention mechanism is introduced to calculate the attention energy of the feature encoding position of each word in the common feature, and the attention energy is normalized into an attention weight through a Softmax function; The outputs of the two convolutional neural network sub-networks are weighted and fused to obtain the two attention output vectors that finally represent the self-attention weight information; The difference between the two attention output vectors is calculated, and the similarity prediction value is obtained through the fully connected layer and the Sigmoid activation function. The similarity of Chinese characters is judged based on the similarity prediction value to perform sensitive word matching, and finally the sensitive word detection result is obtained.
8. An electronic device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method according to any one of claims 1 to 3 when executing the computer program.
9. A computer-readable storage medium storing a computer-executable program, wherein the computer-executable program is used to enable a computer to execute the method according to any one of claims 1 to 3.
