Sensitive word recognition method and device

The feature fusion of text through the BiGRU-Transformer model solves the problem of low recognition accuracy of sensitive word in the prior art, realizes more efficient automatic recognition of sensitive words, and improves recognition accuracy and robustness.

CN118916736APending Publication Date: 2024-11-08SOUTH CHINA NORMAL UNIV
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202410899299.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-04
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

In the prior art, when sensitive word recognition processes language variability, ambiguity and new words and new words, it has low accuracy in metaphor and code language recognition, especially in a specific context, and low manual review efficiency and false judgment rate. high.

Method used

A multi-dimensional feature fusion network based on BiGRU-Transformer is adopted to vectorize the text to be recognized, and the text sequence and position features are extracted, and the feature fusion is used by BiGRU and Transformer modules to realize the automatic recognition of sensitive words.

Benefits of technology

It improves the accuracy and robustness of sensitive word recognition, reduces the burden of manual review, and enhances the recognition ability in complex contexts.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118916736A_ABST
    Figure CN118916736A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of text recognition, and provides a sensitive word recognition method and device, computer equipment and a storage medium. The method comprises the steps of performing vectorization processing on a to-be-recognized text to obtain a text feature vector, processing the text feature vector through a pre-trained sensitive word recognition model, and obtaining a sensitive word recognition result according to a processing result. Wherein the sensitive word recognition model is a model based on BiGRU (Big Generalized Rate Unit)-Transform. After vectorization processing is carried out on a to-be-recognized text, feature extraction and feature fusion are carried out on text sequence features and position features by utilizing a BiGRU-Transform multi-dimensional feature fusion network, so that automatic recognition of sensitive words is realized, and the accuracy of sensitive word recognition is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the technical field of text recognition, and in particular to a sensitive word recognition method, apparatus, computer equipment and storage medium. Background Art

[0002] With the rapid development of information technology, the frequency of Internet information exchange has increased, and the problem of sensitive information dissemination in the network environment has become more serious. In the existing technology, information on the network environment is usually reviewed manually, which has low review efficiency. Summary of the invention

[0003] In view of the above-mentioned deficiencies in the prior art, the present invention provides a sensitive word method, device, equipment and medium.

[0004] A sensitive word identification method, the method comprising:

[0005] Get the text to be recognized;

[0006] Vectorizing the text to be recognized to obtain a text feature vector;

[0007] Processing the text feature vector through a pre-trained sensitive word recognition model, and obtaining a sensitive word recognition result according to the processing result;

[0008] Among them, the sensitive word recognition model is a BiGRU-Transformer-based model; the sensitive word recognition model is trained by a text data set containing sensitive word tags.

[0009] In one embodiment of the present invention, vectorizing the text to be recognized to obtain a text feature vector includes:

[0010] Inputting the text to be recognized into a word vector model for processing to obtain a word vector sequence;

[0011] Calculating the word frequency and inverse document frequency of the text to be recognized;

[0012] The text feature vector is obtained according to the word vector sequence, word frequency and inverse document frequency.

[0013] In one embodiment of the present invention, the text to be recognized contains multiple words, each of which has a corresponding word vector sequence, word frequency and inverse document frequency; the text feature vector is obtained according to the word vector sequence, word frequency and inverse document frequency, including:

[0014] Multiplying the word vector of each word in the to-be-recognized text by the corresponding word frequency and inverse document frequency to obtain a weighted word vector sequence of each word;

[0015] According to the weighted word vector sequence of each word, a text feature vector of the text to be recognized is obtained.

[0016] In one embodiment of the present invention, the sensitive word recognition model includes a word order recognition module, a Transformer module and a feature fusion module. The text feature vector is processed by the pre-trained sensitive word recognition model, and the sensitive word recognition result is obtained according to the processing result, including:

[0017] Inputting the text feature vector into the word order recognition module for processing, and obtaining a word order vector according to the processing result;

[0018] Inputting the text feature vector into the Transformer module for processing, and obtaining a position vector according to the processing result;

[0019] The word order vector and the position vector are processed by the feature fusion module, and the sensitive word recognition result is obtained according to the output result.

[0020] In one embodiment of the present invention, the word order recognition module includes a BiGRU submodule and a double-layer attention mechanism submodule, and the text feature vector is input into the word order recognition module for processing, and a word order vector is obtained according to the processing result, including:

[0021] Inputting the text feature vector into the BiGRU submodule to obtain a state sequence of the text feature vector;

[0022] Attention weights are assigned to the state sequence through a double-layer attention mechanism sub-module to obtain the word sequence vector.

[0023] In one embodiment of the present invention, the Transformer module includes a position encoding submodule, an encoding submodule and a decoding submodule, and the text feature vector is input into the Transformer module for processing, and the position vector is obtained according to the processing result, including:

[0024] Processing the text feature vector by the position encoding submodule to obtain a word embedding vector of the text feature vector;

[0025] After adding position coding to the word embedding vector through the encoding submodule, the output probability prediction is performed through the decoding submodule, and the position vector is obtained according to the output result.

[0026] In one embodiment of the present invention, the step of processing the word order vector and the position vector by the feature fusion module and obtaining the sensitive word recognition result according to the output result includes:

[0027] Integrate the word order vector and the position vector through a weight matrix to obtain a fused feature vector;

[0028] Linear transformation and classification are performed on the fused feature vector, and the sensitive word recognition result is obtained according to the classification result.

[0029] A sensitive word recognition device, the device comprising:

[0030] A text acquisition unit, used for acquiring the text to be recognized;

[0031] A vectorization processing unit, used for performing vectorization processing on the text to be recognized to obtain a text feature vector;

[0032] The sensitive word recognition unit processes the text feature vector through a pre-trained sensitive word recognition model, and obtains a sensitive word recognition result according to the processing result; wherein the sensitive word recognition model is a BiGRU-Transformer-based model; the sensitive word recognition model is obtained by training a text data set containing sensitive word tags.

[0033] A computer device includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method embodiment when executing the computer program.

[0034] A computer-readable storage medium stores a computer program, which implements the steps of the above method embodiment when executed by a processor.

[0035] The present invention obtains a text feature vector by vectorizing the text to be recognized, processes the text feature vector by a pre-trained sensitive word recognition model, and obtains a sensitive word recognition result according to the processing result. Among them, the sensitive word recognition model is a model based on BiGRU-Transformer. After the text to be recognized is vectorized, the BiGRU-Transformer multidimensional feature fusion network is used to extract and fuse text sequence features and position features, so as to realize automatic recognition of sensitive words and improve the accuracy of sensitive word recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 A schematic diagram of a process flow of a sensitive word identification method provided in one embodiment of the present invention;

[0037] Figure 2 A structural block diagram of a sensitive word recognition model provided in one embodiment of the present invention;

[0038] Figure 3 A structural block diagram of a Transformer module provided in one embodiment of the present invention;

[0039] Figure 4 A structural block diagram of a sensitive word identification device provided in one embodiment of the present invention;

[0040] Figure 5 An internal structure diagram of a computer device provided in one embodiment of the present invention. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solution and advantages of the present application more clear, the exemplary embodiments of the present disclosure will be described in more detail with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited to the embodiments described here.

[0042] The terms used in this disclosure are for the purpose of describing specific embodiments only and are not intended to limit the disclosure. The singular forms of "a", "said" and "the" used in this disclosure and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0043] In the existing technology, in areas where information is growing explosively, such as social media, instant messaging, and online platforms, it is more difficult to identify sensitive words. The commonly used manual review method is inefficient and has a high misjudgment rate. Using a deep neural network model to identify sensitive words can automate the review process and reduce the burden of manual review.

[0044] However, the existing sensitive word recognition technology still has limitations in dealing with the variability and ambiguity of language and the rapid emergence of new words and phrases, especially when facing metaphors and code words in a specific context, the recognition accuracy is often low. Therefore, this application provides a sensitive word recognition method based on BiGRU-Transformer, which improves the accuracy of sensitive word recognition by integrating multi-dimensional feature networks into sensitive word recognition.

[0045] In one embodiment, Figure 1 As shown, a sensitive word method is provided, and the method is applied to a server as an example for explanation, including the following steps:

[0046] S110, obtaining text to be recognized.

[0047] The text to be recognized may contain multiple sentences and words.

[0048] In a specific implementation, the server may obtain the text to be recognized from the terminal in real time, or may collect the pre-stored text to be recognized from the database.

[0049] S120, performing vectorization processing on the text to be recognized to obtain a text feature vector;

[0050] Among them, vectorization processing refers to converting text data into numerical vectors. A text feature vector can be a set of multiple word vectors. Word vectors are used to map words or phrases to dense vectors in real vector space. These vectors can capture the semantic relationship between words, so that semantically similar words are close in position in the vector space.

[0051] The text can be vectorized by using one or more analysis methods such as bag-of-words model, n-gram model, word embeddings, etc.

[0052] In a specific implementation, the server can vectorize each word in the text to be recognized to obtain a text feature vector.

[0053] S130, processing the text feature vector through a pre-trained sensitive word recognition model, and obtaining a sensitive word recognition result according to the processing result.

[0054] Among them, the sensitive word recognition model is based on the BiGRU-Transformer model. This model is a multi-dimensional feature fusion network that uses BiGRU to capture the context information of the text sequence, and uses a double-layer attention mechanism to achieve multi-scale understanding of the text on this basis. Transformer enhances the model's ability to capture long-distance dependencies, uses a self-attention mechanism, can simultaneously consider information at all positions, better handle long sequences, and has a strong model generalization ability.

[0055] The sensitive word recognition model is trained by a text dataset containing sensitive word tags. The text dataset can be a general text dataset with sensitive word tags added, or a special text dataset based on a specific technical field, a specific industry, a specific life scenario, etc.

[0056] The sensitive word recognition result may be a binary classification result, i.e., "yes" or "no", or may be confidence data of a sensitive word. The sensitive word recognition result may be a collection of recognition results of multiple words.

[0057] In the specific implementation, the server can input the text feature vector into the BiGRU-Transformer-based model, perform feature extraction and feature fusion based on the text sequence features and position features through the model, and obtain the sensitive word recognition result based on the output result.

[0058] In the above sensitive word recognition method, the text to be recognized is vectorized to obtain a text feature vector, and the text feature vector is processed by a pre-trained sensitive word recognition model, and the sensitive word recognition result is obtained according to the processing result. Among them, the sensitive word recognition model is a BiGRU-Transformer-based model. After the text to be recognized is vectorized, the BiGRU-Transformer multidimensional feature fusion network is used to extract and fuse text sequence features and position features, so as to realize automatic recognition of sensitive words and improve the accuracy of sensitive word recognition.

[0059] In one embodiment, the step of determining to vectorize the text to be recognized in S120 to obtain a text feature vector includes:

[0060] The text to be recognized is input into the word vector model for processing to obtain a word vector sequence; the word frequency and inverse document frequency of the text to be recognized are calculated; and the text feature vector is obtained based on the word vector sequence, word frequency and inverse document frequency.

[0061] In this embodiment, the server can perform vectorization processing on each word in the text to be recognized by using a word vector model and a TF-IDF algorithm to obtain a text feature vector.

[0062] Specifically, the word vector model can be a continuous bag of words (CBOW) model in the Word2Vec model. The server can load the Word2Vec model, use the preprocessed text to be recognized as an index, and then use the model to generate a word vector sequence for each word, thereby capturing the contextual meaning of the vocabulary.

[0063] TF-IDF (term frequency-inverse document frequency) can calculate weights based on the frequency of a word in a document and its rarity in the entire document collection. The TF-IDF value is used to evaluate the importance of a word in a document or a document in a document collection. The server can calculate the term frequency TF, inverse document frequency IDF, and TF-IDF value for each word in the text to be recognized. The term frequency indicates the number of times a word appears in a document. Usually, the term frequency TF (t, d) will be normalized to eliminate the influence of document length. Common normalization methods include maximum frequency normalization and document length normalization. This application uses a normalization method, using the logarithmic form of the term frequency or the smoothed term frequency, refer to formula (1)

[0064] TF(t, d) = log(1 + the number of times word t appears in document d) (1)

[0065] Calculate the inverse document frequency (IDF) of a word. The inverse document frequency reflects the general importance of a word. Refer to formula (2).

[0066]

[0067] Combine TF and IDF to calculate the TF-IDF value of a word in a specific document, refer to formula (3).

[0068] (TF-IDF)(t,d,D)=TF(t,d)×IDF(t,D) (3)

[0069] In some embodiments, the server can obtain a text feature vector based on a word vector sequence, word frequency, and inverse document frequency. Specifically, the text to be recognized contains multiple words, and each word has a corresponding word vector sequence, word frequency, and inverse document frequency. The server can multiply the word vector of each word in the text to be recognized with the corresponding word frequency and inverse document frequency to obtain a weighted word vector sequence for each word, and obtain a text feature vector of the text to be recognized based on the weighted word vector sequence for each word. Combining word frequency and semantic information helps to enhance the co-occurrence pattern of balanced words, emphasize the importance of document-specific context, and enhance the recognition ability of the model in complex contexts.

[0070] For example, the server can use TF-IDF weighted Word2Vec. Multiply the Word2Vec vector of each word by its TF-IDF value to obtain a weighted word vector sequence for the word. The vector can be represented as X = [X1, X2, ..., X T ].

[0071] The solution of the above embodiment improves the text recognition capability in complex contexts by combining word frequency and semantic information through TF-IDF weighted Word2Vec.

[0072] In one embodiment, the server can perform vectorization processing after preprocessing the text to be recognized. Specifically, retain the Chinese characters, letters, numbers and necessary spaces in the text to be recognized, remove HTML tags, special characters, and standardize punctuation. For numbers, determine the context in which they appear and decide whether to retain them. For English text, convert all English characters to lowercase. Use jieba word segmentation for Chinese text, analyze the statistical probability and contextual information between words, and cut the Chinese text into words. It helps computers better understand and process text data. Apply the spaCy library to English text for word form restoration, and restore the vocabulary to the baseline form, such as "running" → "run". By preprocessing the text to be recognized and performing word recognition, a good text foundation is provided for vectorization and noise is reduced.

[0073] In one embodiment, reference Figure 2 ,The sensitive word recognition model includes word order recognition module, Transformer module and feature fusion module.

[0074] Among them, the word order recognition module includes a bidirectional gated recurrent unit (BiGRU) module and a double-layer attention mechanism module.

[0075] BiGRU is a recurrent neural network that contains a reset gate, an update gate, and a merge gate. It consists of two independent GRU units, one for forward propagation in time series and the other for backward propagation. Through this bidirectional structure, the forward and reverse context information of the data is captured at the same time, allowing the model to learn richer word order features and context dependencies. The gated recurrent unit (GRU) is mainly composed of two gates, namely the reset gate and the update gate. The reset gate determines the amount of past information that needs to be forgotten, and the update gate determines the amount of information that needs to be passed between the previous time step and the current time step.

[0076] The dual-layer attention mechanism mainly consists of two stages, local attention and global attention. In the local attention stage, attention is focused on a single time step and focuses on the vocabulary dimension, assigning different weights to each time step in the word vector sequence; the global attention stage is performed on the output of all time steps, assigning different weights to the hidden states of all time steps, and focusing on the sentence dimension. Through the dual-layer attention mechanism, multi-dimensional feature analysis from fine-grained to coarse-grained is achieved, which helps to improve the generalization ability of the model.

[0077] Among them, reference Figure 3 , the Transformer module is a model based on the Encoder-Decoder framework. The EncoderBlock (encoding submodule) encoder receives the output of the Input embedding as input, and transforms and encodes these embedded vectors through a multi-layer self-attention mechanism and a feedforward neural network. The Decoder Block (decoding submodule) receives the output of the encoding submodule as context information, and predicts the next output word or token based on this context information and the generated output sequence (the target sequence during training and the sequence generated by the model itself during inference). In order to share weights, the Output Embedding uses the same (or similar) weight matrix as the Input embedding to convert the words or tokens in the output vocabulary into vectors, which are used to calculate the output probability distribution before the Softmax layer.

[0078] The key components of the Transformer module include self-attention mechanism (Self-Attention), multi-head attention mechanism (Multi-Head Attention), positional encoding (Positional Encoding) and feedforward neural network (FeedForward Neural Network). The self-attention mechanism calculates the attention weight for each word vector sequence and dynamically adjusts the weight distribution of different words, thereby effectively capturing potential sensitive information and improving the recognition accuracy and efficiency of the model; while the multi-head attention mechanism maps the input to different representation spaces and independently calculates the attention weight; positional encoding adds a learnable vector to the embedding of each position to clarify the position information in the word vector sequence, so that the model can understand the order of words; the feedforward neural network (FFN) contains two linear transformations, which increase the nonlinear features of the sequence after attention processing, so as to learn complex feature combinations and improve the expressiveness of the model.

[0079] The feature fusion module is a weighted fusion module. The main function of the weighted fusion module is to integrate the features learned from two different modes, dynamically balance their proportions by adjusting the weight matrix W2, and finally form a comprehensive feature output. This step is the key innovation of the model, which realizes the fusion of multi-dimensional features, strengthens the model's ability to combine local context, global context perception, and long-distance dependency, and effectively improves the accuracy and robustness of the model.

[0080] The sensitive word recognition model also includes a fully connected layer and an output module. The fully connected layer (Dense Layer) receives and integrates the vectors output by the word order recognition module and the Transformer module, learns the weight parameters through the weight matrix W, performs linear transformation, and further compresses and integrates these features to make them more suitable for the final classification task. Since sensitive word recognition is a binary classification task, for example, an activation function (Sigmoid function) can be used for binary classification. The Sigmoid function maps the output to a probability value in the interval (0, 1), that is, the probability that the input sample belongs to the positive class. Therefore, when the output of the Sigmoid function is greater than 0.5, it is predicted to be a positive class (sensitive word); otherwise, it is predicted to be a negative class (non-sensitive word).

[0081] The sensitive word recognition model of the above-mentioned embodiment aims to accurately identify sensitive information in the text through multi-level semantic understanding and feature extraction. Its overall framework consists of three core parts: the first part is a combination of the BiGRU module and the double-layer attention mechanism. In this combination, the BiGRU module captures the forward and backward context information in the text sequence at the same time, and provides a hidden state that takes both the past and the future into consideration for each time step; and the double-layer attention mechanism analyzes from the vocabulary dimension and the sentence dimension on this basis to achieve multi-dimensional feature analysis. The second part is the Transformer module, which can efficiently extract features and improve the long-distance dependencies of the model. The third part is composed of the fusion attention layer, the fully connected layer and the output, which dynamically adjusts the weights of the feature vectors output by the first and second parts to integrate and better realize sensitive word recognition and control.

[0082] In one embodiment, the step of determining in S130 that the text feature vector is processed by a pre-trained sensitive word recognition model and obtaining a sensitive word recognition result according to the processing result includes:

[0083] Inputting the text feature vector into the word order recognition module for processing, and obtaining a word order vector according to the processing result;

[0084] The text feature vector is input into the Transformer module for processing, and a position vector is obtained according to the processing result; the word order vector and the position vector are processed by the feature fusion module, and the sensitive word recognition result is obtained according to the output result.

[0085] Specifically, the server can input the text feature vector into the BiGRU submodule to capture the forward and reverse context information of the data, so that the model can learn richer word sequence features and up-down dependencies, and obtain the state sequence of the text feature vector. The output of this module is the hidden state sequence of each time step, denoted as O = [O1, O2, ..., O T ], where T is the sequence length.

[0086] Furthermore, the server can assign attention weights to the state sequence through the double-layer attention mechanism submodule to obtain the word sequence vector. The server can output O = [O1, O2, ..., O T ], assigning weights to the hidden states of each time step. The attention weight distribution is given by the Softmax function, and its size depends on the similarity between the hidden state and the query vector. Then the global attention weight is calculated over the entire sequence, and the attention weight distribution is obtained by the Softmax function. Finally, a weighted sum is performed and the vector y is obtained by integrating the weight matrix W1.

[0087] The server inputs the text feature vector into the Transformer module for processing. The text feature vector can be processed by the position encoding submodule to obtain the word embedding vector of the text feature vector; after adding the position encoding to the word embedding vector by the encoding submodule, the output probability prediction is performed by the decoding submodule, and the position vector is obtained according to the output result. The input encoding part of the Transformer module includes the word embedding vector and the position encoding. The text feature vector is obtained by searching the word embedding matrix to obtain a dense vector representation, that is, the word embedding vector; then a position encoding is added to each word embedding vector. The position encoding is a periodic sequence based on sine and cosine functions, which can ensure that the model distinguishes the same words in different positions. The encoding module of the Transformer module consists of 6 identical encoders, each of which is composed of a self-attention layer containing a masking mechanism and a feedforward neural network; similarly, the decoding module is composed of 6 identical decoders stacked together. The difference between the decoder and the encoder is that a multi-head attention layer containing a masking mechanism is added.

[0088] The scheme of the above-mentioned embodiment extracts the word order characteristics and position characteristics of the text respectively through the word order recognition module and the Transformer module, and introduces the method of multi-dimensional attention mechanism in each module, so that the model can focus on important information fragments at different granularities, distinguish the internal importance of each time step through the local attention layer, and realize the capture of dependencies across sentences through the global attention layer, thereby improving the accuracy and robustness of the model in the sensitive word recognition task.

[0089] In one embodiment, the word order vector and the position vector are processed by the feature fusion module, and the sensitive word recognition result is obtained according to the output result, including:

[0090] The word order vector and position vector are integrated through the weight matrix to obtain a fused feature vector; linear transformation and classification are performed on the fused feature vector, and the sensitive word recognition result is obtained according to the classification result.

[0091] In this embodiment, the word order vector output by the word order recognition module and the position vector output by the Transformer module can be integrated through the weight matrix W2 to obtain a fused feature vector, realize multi-dimensional feature fusion, strengthen the model's ability to understand local context, global context perception, and long-distance dependence, and improve the accuracy and robustness of the model. The fully connected layer (Dense Layer) of the sensitive word recognition model receives and integrates the fused feature vector, learns the weight parameters through the weight matrix W, performs a linear transformation on it, and further compresses and integrates these features to make it more suitable for the final classification task. Since sensitive word recognition is a binary classification task, the Sigmoid function is selected for binary classification. When the output of the Sigmoid function is greater than 0.5, it is predicted to be a positive class (sensitive word); otherwise, it is predicted to be a negative class (non-sensitive word).

[0092] The solution of the above embodiment, through the feature fusion module, fuses the word order vector output by the word order recognition module and the position vector output by the Transformer module, thereby realizing multi-dimensional feature fusion and improving the accuracy and robustness of sensitive word recognition.

[0093] In one embodiment, the server can train a sensitive word recognition model. Before model training, determine the training configuration. For example, the initial learning rate can be set to 1e-5, the batch size can be set to 128, the number of iterations can be initially set to 20 epochs, the optimizer can select AdamW, and the early stopping strategy and regularization can be applied to reduce the risk of overfitting. During model training, the server can input the obtained general text training set into the sensitive word recognition model for training after preprocessing and preliminary feature extraction, and optimize the model weight value; the validation set is used for hyperparameter adjustment and monitoring to improve the accuracy of the final judgment.

[0094] In one embodiment, when performing sensitive word recognition, the acquired text test set can be input into a trained sensitive word recognition model after preprocessing and preliminary feature extraction to obtain sensitive word judgments, and finally the matched and located sensitive words are replaced with preset texts, thereby effectively identifying sensitive words and conducting timely management and control.

[0095] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the order of the actions described, because according to the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the present invention.

[0096] In one embodiment, Figure 4As shown, a sensitive word device is provided, and the device 4600 includes:

[0097] The text acquisition unit 410 is used to acquire the text to be recognized;

[0098] A vectorization processing unit 420 is used to perform vectorization processing on the text to be recognized to obtain a text feature vector;

[0099] The sensitive word recognition unit 430 processes the text feature vector through a pre-trained sensitive word recognition model, and obtains a sensitive word recognition result according to the processing result; wherein the sensitive word recognition model is a BiGRU-Transformer-based model; the sensitive word recognition model is obtained by training a text data set containing sensitive word tags.

[0100] In one embodiment, the vectorization processing unit 420 is further used to input the text to be recognized into a word vector model for processing to obtain a word vector sequence; calculate the word frequency and inverse document frequency of the text to be recognized; and obtain the text feature vector based on the word vector sequence, word frequency and inverse document frequency.

[0101] In one embodiment, the text to be recognized contains multiple words, each of which has a corresponding word vector sequence, word frequency and inverse document frequency; the vectorization processing unit 420 is further used to multiply the word vector of each word in the text to be recognized with the corresponding word frequency and inverse document frequency to obtain a weighted word vector sequence of each word; based on the weighted word vector sequence of each word, a text feature vector of the text to be recognized is obtained.

[0102] In one embodiment, the sensitive word recognition model includes a word order recognition module, a Transformer module and a feature fusion module. The sensitive word recognition unit 430 includes: a word order vector acquisition subunit, which is used to input the text feature vector into the word order recognition module for processing, and obtain the word order vector according to the processing result; a position vector subunit, which is used to input the text feature vector into the Transformer module for processing, and obtain the position vector according to the processing result; a feature fusion subunit, which is used to process the word order vector and the position vector through the feature fusion module, and obtain the sensitive word recognition result according to the output result.

[0103] In one embodiment, the word order recognition module includes a BiGRU submodule and a two-layer attention mechanism submodule, and a word order vector acquisition subunit, which is further used to input the text feature vector into the BiGRU submodule to obtain a state sequence of the text feature vector; and assign attention weights to the state sequence through the two-layer attention mechanism submodule to obtain the word order vector.

[0104] In one embodiment, the Transformer module includes a position encoding submodule, an encoding submodule and a decoding submodule, and a position vector acquisition subunit, which is used to process the text feature vector through the position encoding submodule to obtain a word embedding vector of the text feature vector; after adding position encoding to the word embedding vector through the encoding submodule, the output probability prediction is performed through the decoding submodule, and the position vector is obtained according to the output result.

[0105] In one embodiment, the feature fusion subunit is further used to integrate the word order vector and the position vector through a weight matrix to obtain a fused feature vector; perform linear transformation and classification on the fused feature vector, and obtain the sensitive word recognition result according to the classification result.

[0106] For the specific definition of the sensitive word recognition device, please refer to the definition of the sensitive word recognition method above, which will not be repeated here. Each module in the above-mentioned sensitive word recognition device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0107] The present invention can be applied to computer devices, such as smart phones, tablet computers, laptop computers, desktop computers, rack servers, blade servers, tower servers or cabinet servers (including independent servers or server clusters composed of multiple servers) that can execute programs. The computer device of this embodiment includes at least but is not limited to: a memory and a processor that can communicate with each other through a system bus, such as Figure 5 It should be pointed out that Figure 5Only a computer device with a memory and a processor component is shown, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. The memory (i.e., a readable storage medium) includes a flash memory, a hard disk, a multimedia card, a card-type memory (e.g., SD or DX memory, etc.), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a disk, an optical disk, etc. In some embodiments, the memory may be an internal storage unit of a computer device, such as a hard disk or a memory of the computer device. In other embodiments, the memory may also be an external storage device of a computer device, such as a plug-in hard disk equipped on the computer device, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. Of course, the memory may also include both an internal storage unit of a computer device and an external storage device thereof. In this embodiment, the memory is generally used to store an operating system and various application software installed on a computer device, such as a sensitive word recognition model, feature vector data, etc. In addition, the memory can also be used to temporarily store various types of data that have been output or are about to be output. The processor can be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip in some embodiments. The processor is generally used to control the overall operation of a computer device. In this embodiment, the processor is used to run the program code stored in the memory or process data to implement a sensitive word recognition method.

[0108] Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0109] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0110] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0111] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing related hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods.

[0112] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A sensitive word identification method, characterized in that: The method comprises: Get the text to be recognized; Vectorizing the text to be recognized to obtain a text feature vector; Processing the text feature vector through a pre-trained sensitive word recognition model, and obtaining a sensitive word recognition result according to the processing result; Among them, the sensitive word recognition model is a BiGRU-Transformer-based model; the sensitive word recognition model is trained by a text data set containing sensitive word tags.

2. The method according to claim 1, characterized in that The vectorization process of the text to be recognized to obtain a text feature vector includes: Inputting the text to be recognized into a word vector model for processing to obtain a word vector sequence; Calculating the word frequency and inverse document frequency of the text to be recognized; The text feature vector is obtained according to the word vector sequence, word frequency and inverse document frequency.

3. The method according to claim 2, characterized in that The text to be recognized contains a plurality of words, each of which has a corresponding word vector sequence, word frequency and inverse document frequency; the text feature vector is obtained according to the word vector sequence, word frequency and inverse document frequency, including: Multiplying the word vector of each word in the to-be-recognized text by the corresponding word frequency and inverse document frequency to obtain a weighted word vector sequence of each word; According to the weighted word vector sequence of each word, a text feature vector of the text to be recognized is obtained.

4. The method according to claim 1, characterized in that: The sensitive word recognition model includes a word order recognition module, a Transformer module and a feature fusion module. The text feature vector is processed by the pre-trained sensitive word recognition model, and a sensitive word recognition result is obtained according to the processing result, including: Inputting the text feature vector into the word order recognition module for processing, and obtaining a word order vector according to the processing result; Inputting the text feature vector into the Transformer module for processing, and obtaining a position vector according to the processing result; The word order vector and the position vector are processed by the feature fusion module, and the sensitive word recognition result is obtained according to the output result.

5. The method according to claim 4, characterized in that The word order recognition module includes a BiGRU submodule and a double-layer attention mechanism submodule. The text feature vector is input into the word order recognition module for processing, and a word order vector is obtained according to the processing result, including: Inputting the text feature vector into the BiGRU submodule to obtain a state sequence of the text feature vector; Attention weights are assigned to the state sequence through a double-layer attention mechanism sub-module to obtain the word sequence vector.

6. The method according to claim 4, characterized in that The Transformer module includes a position encoding submodule, an encoding submodule and a decoding submodule. The text feature vector is input into the Transformer module for processing, and a position vector is obtained according to the processing result, including: Processing the text feature vector by the position encoding submodule to obtain a word embedding vector of the text feature vector; After adding position coding to the word embedding vector through the encoding submodule, the output probability prediction is performed through the decoding submodule, and the position vector is obtained according to the output result.

7. The method according to any one of claims 3 to 5, characterized in that: The step of processing the word order vector and the position vector by the feature fusion module and obtaining the sensitive word recognition result according to the output result includes: Integrate the word order vector and the position vector through a weight matrix to obtain a fused feature vector; Linear transformation and classification are performed on the fused feature vector, and the sensitive word recognition result is obtained according to the classification result.

8. A sensitive word recognition device, characterized in that: The device comprises: A text acquisition unit, used for acquiring the text to be recognized; A vectorization processing unit, used for performing vectorization processing on the text to be recognized to obtain a text feature vector; The sensitive word recognition unit processes the text feature vector through a pre-trained sensitive word recognition model, and obtains a sensitive word recognition result according to the processing result; wherein the sensitive word recognition model is a BiGRU-Transformer-based model; the sensitive word recognition model is obtained by training a text data set containing sensitive word tags.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Citation Information

Patent Citations

  • Electric power audit text entity recognition method, device and equipment and storage medium

    CN116245107A

  • Information processing method and system based on big data

    CN117236323A