Sensitive word type determination method and device

Abstract

Embodiments of the invention provide a sensitive word type determination method and apparatus. The method comprises the steps of obtaining a to-be-recognized text vector; inputting the to-be-recognized text vector into a three-branch attention model to obtain a sensitive word recognition result and an entity word recognition result, and obtaining a weighted vector of each branch; determining a first sensitive word type corresponding to a sensitive word vector from preset sensitive word types according to the similarity between the sensitive word vector in the sensitive word recognition result and feature vectors of the preset sensitive word types; and according to an entity word vector in the entity word recognition result, the feature vector of the preset sensitive word type and the weighted vector of each branch, performing sensitive word type recognition on the entity word vector to obtain a second sensitive word type corresponding to the entity word vector.

Description

Technical Field

[0001] The present application relates to the field of data security technology, and in particular to a method and device for determining the type of sensitive words. Background Art

[0002] With the rapid development of information technology, data privacy issues are receiving increasing attention. Personal privacy data includes but is not limited to personal identity information, contact information, location data, etc. The leakage of this information may lead to the violation of personal privacy and property loss. Traditional privacy protection methods often rely on predefined privacy data types and rules, which cannot meet the needs of diversified data types and personalized privacy protection needs. Summary of the invention

[0003] In view of the above problems, embodiments of the present application provide a method and device for determining the type of sensitive words that overcome the above problems or at least partially solve the above problems.

[0004] In a first aspect, an embodiment of the present application provides a method for determining a sensitive word type, the method comprising:

[0005] Get the text vector to be recognized;

[0006] Input the text vector to be recognized into the three-branch attention model to obtain sensitive word recognition results and entity word recognition results, and obtain the weighted vector of each branch;

[0007] Determining a first sensitive word type corresponding to the sensitive word vector from the preset sensitive word types according to the similarity between the sensitive word vector in the sensitive word recognition result and the feature vector of the preset sensitive word type;

[0008] According to the entity word vector in the entity word recognition result, the feature vector of the preset sensitive word type and the weighted vector of each branch, the entity word vector is subjected to sensitive word type recognition to obtain a second sensitive word type corresponding to the entity word vector.

[0009] Optionally, the text vector to be recognized is input into a three-branch attention model to obtain a sensitive word recognition result and an entity word recognition result, and obtain a weighted vector of each branch, including:

[0010] Input the text vector to be recognized into the sensitive word recognition branch in the three-branch attention model to obtain a first weighted vector and a sensitive word recognition result;

[0011] Inputting the text vector to be recognized into the entity word recognition branch in the three-branch attention model to obtain a second weighted vector and an entity word recognition result;

[0012] The text vector to be recognized is input into the privacy sentence recognition branch in the three-branch attention model to obtain a third weighted vector.

[0013] Optionally, the step of inputting the to-be-recognized text vector into a sensitive word recognition branch in a three-branch attention model to obtain a first weighted vector and a sensitive word recognition result includes:

[0014] Input the text vector to be recognized into the sensitive word recognition branch in the three-branch attention model, perform word segmentation processing on the text vector to be recognized, and obtain multiple word vectors;

[0015] Perform word embedding processing on each word vector to obtain a word embedding matrix corresponding to the word vector;

[0016] Performing a rearrangement operation, a feature decomposition, and a convolution operation on the word embedding matrix in sequence to obtain a first feature matrix, and calculating a first attention weight of the first feature matrix;

[0017] Calculate a first weighted vector according to the first feature matrix and the first attention weight;

[0018] The sensitive word recognition results of the multiple word vectors are determined according to the similarity between the first weighted vector of each word vector and the weighted vector of the preset sensitive word.

[0019] Optionally, the step of inputting the text vector to be recognized into an entity word recognition branch in a three-branch attention model to obtain a second weighted vector and an entity word recognition result includes:

[0020] Input the text vector to be recognized into the entity word recognition branch in the three-branch attention model, perform word segmentation processing on the text vector to be recognized, and obtain multiple word vectors;

[0021] Perform feature extraction on each word vector to obtain the word vector feature matrix;

[0022] Performing a rearrangement operation, a feature decomposition, and a convolution operation on the word vector feature matrix in sequence to obtain a second feature matrix, and calculating a second attention weight of the second feature matrix;

[0023] Calculate a second weighted vector according to the second feature matrix and the second attention weight;

[0024] The second weighted vector of each word vector is input into the conditional random field model to obtain entity word recognition results after entity word annotation of the multiple word vectors.

[0025] Optionally, the step of inputting the to-be-recognized text vector into a privacy sentence recognition branch in a three-branch attention model to obtain a third weighted vector comprises:

[0026] Inputting the text vector to be recognized into the privacy sentence recognition branch in the three-branch attention model, performing sentence processing on the text vector to be recognized to obtain multiple sentence vectors;

[0027] Perform feature extraction on each sentence vector to obtain a sentence vector feature matrix;

[0028] Performing a rearrangement operation, a feature decomposition, and a convolution operation on the sentence vector feature matrix in sequence to obtain a third feature matrix, and calculating a third attention weight of the third feature matrix;

[0029] According to the third attention weight and hidden state corresponding to each sentence vector, the third weighted vector is calculated.

[0030] Optionally, determining a first sensitive word type corresponding to the sensitive word vector from the preset sensitive word types according to the similarity between the sensitive word vector in the sensitive word recognition result and a feature vector of a preset sensitive word type includes:

[0031] Performing vector conversion on the sensitive word vector in the sensitive word recognition result to obtain a sensitive word feature vector;

[0032] Calculating a first similarity between the sensitive word feature vector and a feature vector of each preset sensitive word type;

[0033] The sensitive word type of the feature vector of the preset sensitive word type corresponding to the largest first similarity is determined as the first sensitive word type corresponding to the sensitive word vector.

[0034] Optionally, the step of performing sensitive word type identification on the entity word vector according to the entity word vector in the entity word recognition result, the feature vector of the preset sensitive word type, and the weighted vector of each branch to obtain a second sensitive word type corresponding to the entity word vector includes:

[0035] Performing vector conversion on the entity word vector in the entity word recognition result to obtain an entity word feature vector;

[0036] Calculating a second similarity between the entity word feature vector and a feature vector of each preset sensitive word type;

[0037] When the maximum second similarity is within a preset similarity range, calculating a fourth weighted vector according to the first weighted vector, the second weighted vector, the third weighted vector and a fourth attention weight;

[0038] The fourth weighted vector is input into the distillation model to identify the sensitive word type, and the second sensitive word type corresponding to the entity word vector is obtained.

[0039] In a second aspect, an embodiment of the present application further provides a device for determining a sensitive word type, the device comprising:

[0040] The first acquisition module acquires the text vector to be recognized;

[0041] The second acquisition module is used to input the text vector to be recognized into the three-branch attention model to obtain sensitive word recognition results and entity word recognition results, and obtain weighted vectors of each branch;

[0042] A first determination module, configured to determine a first sensitive word type corresponding to the sensitive word vector from the preset sensitive word types according to a similarity between the sensitive word vector in the sensitive word recognition result and a feature vector of a preset sensitive word type;

[0043] The second determination module is used to identify the sensitive word type of the entity word vector according to the entity word vector in the entity word recognition result, the feature vector of the preset sensitive word type and the weighted vector of each branch, so as to obtain the second sensitive word type corresponding to the entity word vector.

[0044] In a third aspect, an embodiment of the present application further provides an electronic device, including a memory, a transceiver, and a processor:

[0045] A memory for storing a computer program; a transceiver for sending and receiving data under the control of a processor; and a processor for reading the computer program in the memory and executing the method as described in the first aspect above.

[0046] In a fourth aspect, an embodiment of the present application further provides a processor-readable storage medium, wherein the processor-readable storage medium stores a computer program, and the computer program is used to enable the processor to execute the method described in the first aspect above.

[0047] In the above embodiment of the present application, a text vector to be identified is obtained, and the text vector to be identified is input into a three-branch attention model to obtain a sensitive word identification result and an entity word identification result, and a weighted vector of each branch is obtained, that is, the three-branch attention model can automatically identify sensitive information and entity information. In addition, according to the similarity between the sensitive word vector in the sensitive word identification result and the feature vector of the preset sensitive word type, the first sensitive word type corresponding to the sensitive word vector is determined from the preset sensitive word type, and according to the entity word vector in the entity word identification result, the feature vector of the preset sensitive word type and the weighted vector of each branch, the entity word vector is subjected to sensitive word type identification to obtain the second sensitive word type corresponding to the entity word vector, that is, the type of new sensitive information can be automatically discovered and learned, and the dynamic expansion capability can adapt to the ever-changing privacy protection needs, and can also improve the accuracy and flexibility of sensitive information identification, solve the problem that traditional privacy protection methods often rely on predefined sensitive data types and rules, and cannot meet the diversification of data types and the personalized needs of privacy protection needs, and reduce the maintenance cost of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0049] Figure 1 One of the flowcharts of the method for determining the sensitive word type provided in the embodiment of the present application;

[0050] Figure 2 Flow chart 2 of the method for determining the sensitive word type provided in the embodiment of the present application;

[0051] Figure 3 Flow chart 3 of the method for determining the sensitive word type provided in the embodiment of the present application;

[0052] Figure 4 Flow chart 4 of the method for determining the sensitive word type provided in an embodiment of the present application;

[0053] Figure 5 Flow chart 5 of the method for determining the sensitive word type provided in the embodiment of the present application;

[0054] Figure 6 Flow chart six of the method for determining the sensitive word type provided in an embodiment of the present application;

[0055] Figure 7A structural block diagram of a device for determining sensitive word types provided in an embodiment of the present application;

[0056] Figure 8 A structural block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0057] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0058] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0059] The following is a detailed description of the method for determining the sensitive word type provided in the embodiment of the present application through specific embodiments and application scenarios in conjunction with the accompanying drawings.

[0060] Specifically, the present application embodiment provides a method for determining the type of sensitive words, such as Figure 1 , Figure 2 and Figure 6 As shown, the following steps may be specifically included:

[0061] Step 101, obtaining a text vector to be recognized.

[0062] Specifically, Figure 2 , Figure 3 and Figure 6As shown, the text to be recognized is obtained, and the text to be recognized is preprocessed. The preprocessing includes but is not limited to: performing a noise removal operation on the text to be recognized to remove irrelevant noise information in the text to be recognized; and performing a special character removal operation on the text to be recognized to remove special characters and punctuation marks in the text to be recognized; and performing a stop word removal operation on the text to be recognized to delete common high-frequency irrelevant words in the text to be recognized; and performing a typo correction operation on the text to be recognized to detect and correct spelling errors in the text to be recognized; and performing a text filtering operation on the text to be recognized to further clean up the text to be recognized and retain meaningful content in the text to be recognized. After the above operations, the initial text data x1, x2, ...x are obtained. n .

[0063] For the initial text data x1, x2, ...x n Use BERT-Base-Chinese to encode the data and obtain the text vectors h1, h2, …h to be recognized n , the specific formula is as follows:

[0064] h1,h2,…h n = Bert(x1,x2,…x n )

[0065] Among them, BERT-Base-Chinese is a pre-trained language representation model (Bidirectional EncoderRepresentations from Transformers, BERT) for Chinese text. It is pre-trained on Chinese text and can learn the semantic and grammatical information of Chinese text.

[0066] Step 102, input the text vector to be recognized into the three-branch attention model, obtain the sensitive word recognition result and the entity word recognition result, and obtain the weighted vector of each branch.

[0067] Specifically, the text vector to be recognized is input into the three branches of the three-branch attention model respectively, and the weighted vectors corresponding to the three branches are obtained, and the sensitive word recognition results and the entity word recognition results are obtained.

[0068] Step 103: Determine a first sensitive word type corresponding to the sensitive word vector from the preset sensitive word types according to the similarity between the sensitive word vector in the sensitive word recognition result and the feature vector of the preset sensitive word type.

[0069] Specifically, Figure 2As shown, the sensitive word recognition result includes sensitive words and corresponding sensitive word vectors. Each sensitive word vector is matched with the feature vectors of multiple preset sensitive word types respectively to determine the preset sensitive word type to which the sensitive word vector belongs among the multiple preset sensitive word types, that is, the first sensitive word type.

[0070] Step 104, based on the entity word vector in the entity word recognition result, the feature vector of the preset sensitive word type and the weighted vector of each branch, the entity word vector is subjected to sensitive word type recognition to obtain a second sensitive word type corresponding to the entity word vector.

[0071] Specifically, Figure 2 As shown, the entity word recognition result includes the entity word and the corresponding entity word vector. The sensitive word type can be identified through the entity word vector, the feature vectors of multiple preset sensitive word types, and the weighted vectors of each branch to obtain a new sensitive word type corresponding to the entity word vector, that is, the second sensitive word type, to expand the existing sensitive word type.

[0072] In the above embodiment of the present application, a text vector to be identified is obtained, and the text vector to be identified is input into a three-branch attention model to obtain a sensitive word identification result and an entity word identification result, and a weighted vector of each branch is obtained, that is, the three-branch attention model can automatically identify sensitive information and entity information. In addition, according to the similarity between the sensitive word vector in the sensitive word identification result and the feature vector of the preset sensitive word type, the first sensitive word type corresponding to the sensitive word vector is determined from the preset sensitive word type, and according to the entity word vector in the entity word identification result, the feature vector of the preset sensitive word type and the weighted vector of each branch, the entity word vector is subjected to sensitive word type identification to obtain the second sensitive word type corresponding to the entity word vector, that is, the type of new sensitive information can be automatically discovered and learned, and the dynamic expansion capability can adapt to the ever-changing privacy protection needs, and can also improve the accuracy and flexibility of sensitive information identification, solve the problem that traditional privacy protection methods often rely on predefined sensitive data types and rules, and cannot meet the diversification of data types and the personalized needs of privacy protection needs, and reduce the maintenance cost of the system.

[0073] As an optional specific embodiment, the step 102 inputs the text vector to be recognized into the three-branch attention model to obtain sensitive word recognition results and entity word recognition results, and obtains the weighted vector of each branch, specifically including steps 1021 to 1023:

[0074] Step 1021, input the text vector to be recognized into the sensitive word recognition branch in the three-branch attention model to obtain a first weighted vector and a sensitive word recognition result.

[0075] Specifically, Figure 2 and Figure 6 As shown, the text vector to be recognized is input into the sensitive word recognition branch in the three-branch attention model, the first weighted vector is calculated by the sensitive word recognition branch, and the sensitive word recognition result is obtained based on the first weighted vector.

[0076] Step 1022, input the text vector to be recognized into the entity word recognition branch in the three-branch attention model to obtain a second weighted vector and an entity word recognition result.

[0077] Specifically, Figure 2 and Figure 6 As shown, the text vector to be recognized is input into the entity word recognition branch in the three-branch attention model, the second weighted vector is calculated by the entity word recognition branch, and the entity word recognition result is obtained based on the second weighted vector.

[0078] Step 1023: input the text vector to be recognized into the privacy sentence recognition branch in the three-branch attention model to obtain a third weighted vector.

[0079] Specifically, Figure 2 and Figure 6 As shown, the text vector to be recognized is input into the privacy sentence recognition branch in the three-branch attention model, and the third weighted vector is calculated through the privacy sentence recognition branch.

[0080] Furthermore, the step 1021 inputs the text vector to be recognized into the sensitive word recognition branch in the three-branch attention model to obtain a first weighted vector and a sensitive word recognition result, including:

[0081] Input the text vector to be recognized into the sensitive word recognition branch in the three-branch attention model, perform word segmentation processing on the text vector to be recognized, and obtain multiple word vectors;

[0082] Perform word embedding processing on each word vector to obtain a word embedding matrix corresponding to the word vector;

[0083] Performing a rearrangement operation, a feature decomposition, and a convolution operation on the word embedding matrix in sequence to obtain a first feature matrix, and calculating a first attention weight of the first feature matrix;

[0084] Calculate a first weighted vector according to the first feature matrix and the first attention weight;

[0085] The sensitive word recognition results of the multiple word vectors are determined according to the similarity between the first weighted vector of each word vector and the weighted vector of the preset sensitive word.

[0086] Specifically, Figure 3 and Figure 4 As shown in , the first branch of the three-branch attention model is the sensitive word recognition branch, such as Figure 4 The sensitive word recognition branch first performs word segmentation on the text vector to be recognized to obtain multiple word vectors. In the first branch, word embedding GloVe is used to learn word embedding, which encodes local context information and global word co-occurrence statistics. GloVe generates word vectors based on the frequency F of word occurrence. The specific formula is as follows:

[0087]

[0088] Among them, P ij It is obtained by the number of times the i-th word vector and the j-th word vector appear together in the preset context window, that is, the probability that the i-th word vector appears in the context of the j-th word vector;

[0089] P i is the probability of the i-th word vector appearing in the text vector to be recognized.

[0090] The core idea of ​​GloVe is to use these co-occurrence frequencies F(w i ,w j ) to generate word vectors. GloVe can construct a word vector space, in which the similarity between words is expressed as distance. GloVe learns word vectors through the objective function J, as shown in the following formula:

[0091]

[0092] Among them, X ij is the number of co-occurrences between the i-th word vector and the j-th word vector, that is, the number of times the i-th word vector and the j-th word vector appear together in the preset context window;

[0093] V is the size of the text vector to be recognized;

[0094] w is the word vector;

[0095] b is the bias term;

[0096] f(X ij ) is a weighted function used to reduce the impact of frequently co-occurring words on the objective function;

[0097] X max is the preset maximum co-occurrence threshold;

[0098] α is a preset hyperparameter, 0<α<1.

[0099] The word vector is embedded by GloVe, and the result is a word embedding matrix E. c ∈Rn×d , the specific formula is as follows:

[0100] E c =Glove(T)

[0101] In the sensitive word recognition branch, the word embedding matrix E c Perform a permutation operation to keep the interaction between word embedding dimensions and focus on the semantic features in the word embedding matrix. Perform feature decomposition (Z-Pool) on the permuted matrix, and then perform a convolution operation to obtain the first feature matrix Z1, as shown in the following formula:

[0102] Z-Pool(H)=[MaxPool 0d (H),AvgPool 0d (H)]

[0103] Among them, Z-Pool splices the outputs of maximum pooling and average pooling together, then convolves to obtain the first feature matrix Z1, and calculates the first attention weight A1 of the first feature matrix Z1, as shown in the following formula:

[0104] a1=softmax(W1Z1+b1)

[0105] Among them, W1 is the weight matrix learned in the attention mechanism, which is learned through the back-propagation algorithm during the model training process; in the forward propagation process of the model, the weight matrix W1 and the bias term b1 are assigned by random initialization. Subsequently, each time the model calculates the attention weight and output, the gradient is calculated based on the current loss function, and the weight is updated through back-propagation. Here, W1 and b1 are the weights and bias terms updated through back-propagation after the loss function is calculated.

[0106] Then the first attention weight A1 is applied to the first feature matrix Z1 after convolution, and processed by the activation function Sigmoid to obtain the first weighted vector Attn1(Z), which is as follows:

[0107] Attn1(Z)=A1·Sigmoid(Z1);

[0108] The data captured after dimension interaction is restored to its original state through rearrangement operation, that is, the dimensions and positions are restored.

[0109] Calculate the similarity between the first weighted vector of each word vector and the weighted vector of the preset sensitive word, and specifically extract the similarity through the Euclidean distance:

[0110]

[0111] Among them, d1(v1,v2) is the Euclidean distance between vector v1 and vector v2, v1 is the word vector, v2 is the weighted vector of the preset sensitive word, k is the dimension index representing the vector, and C is the dimension size of the vector.

[0112] Then, multiple d1s are sorted by size, and the smallest d1 (i.e., the one with the shortest Euclidean distance and the largest similarity) is compared with the first preset threshold λ1. If d1<λ1, it means that the word corresponding to the word vector is a sensitive word, thereby obtaining the sensitive word recognition result.

[0113] Furthermore, the step 1022 inputs the text vector to be recognized into the entity word recognition branch in the three-branch attention model to obtain a second weighted vector and an entity word recognition result, including:

[0114] Input the text vector to be recognized into the entity word recognition branch in the three-branch attention model, perform word segmentation processing on the text vector to be recognized, and obtain multiple word vectors;

[0115] Perform feature extraction on each word vector to obtain the word vector feature matrix;

[0116] Performing a rearrangement operation, a feature decomposition, and a convolution operation on the word vector feature matrix in sequence to obtain a second feature matrix, and calculating a second attention weight of the second feature matrix;

[0117] Calculate a second weighted vector according to the second feature matrix and the second attention weight;

[0118] The second weighted vector of each word vector is input into the conditional random field model to obtain entity word recognition results after entity word annotation of the multiple word vectors.

[0119] Specifically, Figure 3 and Figure 4 As shown in , the second branch is the entity word recognition branch, such as Figure 5 As shown in Figure 1, it specifically includes: input layer, convolutional neural network (Text Convolutional Neural Network, TextCNN) layer, BiLSTM layer, information attention mechanism layer, fully connected layer and conditional random field (Conditional Random Field, CRF) layer. Figure 4In branch b, the entity word recognition branch first performs word segmentation on the text vector to be recognized to obtain multiple word vectors. The TextCNN layer uses a series of convolution kernels of different sizes to extract features from word vectors to capture multi-level context information in the text vector to be recognized. For example, five convolution kernels can be used, namely 3×k, 4×k, 5×k, 6×k and 7×k. Each convolution operation can be regarded as a feature extraction process, as shown in the following formula:

[0120] c i =f(W·X i:i+h-1 +b)

[0121] Among them, it is assumed that the size of a convolution kernel is n×k;

[0122] W2 is the weight matrix of the convolution kernel;

[0123] b2 is the bias term;

[0124] f is the activation function Relu;

[0125] X i:i+h-1 is the word embedding matrix E c The submatrix from row i to row i+h in .

[0126] The pooling layer uses the maximum pooling method for the output area and outputs the feature information after pooling. The specific formula is as follows:

[0127] c max =max(c1,c2,…,c n-h+1 )

[0128] BiLSTM (directional Long Short-Term Memory) is composed of forward LSTM and backward LSTM. By combining two independent LSTM networks, BiLSTM effectively extracts the bidirectional feature information of the input sequence and obtains the word vector feature matrix.

[0129] Among them, BiLSTM is composed of forward LSTM and backward LSTM.

[0130] In the entity word recognition branch, the word vector feature matrix is ​​rearranged to capture the interaction between different word vectors and calculate the dependency of each time step on other time steps. The rearranged matrix is ​​then subjected to feature decomposition (Z-Pool) and then convolution to obtain the second feature matrix Z2, as shown in the following formula:

[0131] Z-Pool(H)=[MaxPool 0d (H),AvgPool 0d (H)]

[0132] Among them, Z-Pool splices the outputs of maximum pooling and average pooling together, then convolves to obtain the second feature matrix Z2, and calculates the second attention weight A2 of the second feature matrix Z2, as shown in the following formula:

[0133] A2=softmax(W2Z2+b2)

[0134] Among them, W2 is the weight matrix learned in the attention mechanism, which is learned through the back-propagation algorithm during the model training process; in the forward propagation process of the model, the weight matrix W2 and the bias term b2 are assigned by random initialization. Subsequently, each time the model calculates the attention weight and output, it calculates the gradient based on the current loss function and updates the weight through back-propagation. Here, W2 and b2 are the weights and bias terms updated through back-propagation after the loss function is calculated.

[0135] Then the first attention weight A2 is applied to the second feature matrix Z2 after convolution, and processed by the activation function Sigmoid to obtain the second weighted vector Attn2(Z), which is as follows:

[0136] Attn2(Z)=A2·Sigmoid(Z2);

[0137] The data captured after dimension interaction is restored to its original state through rearrangement operation, that is, the dimensions and positions are restored.

[0138] The constraints and dependencies between labels are learned through CRF, and the maximum likelihood estimation is used for evaluation to obtain the entity word labeling results and the entity word recognition results after entity word labeling.

[0139] Furthermore, the step 1023 inputs the text vector to be recognized into the privacy sentence recognition branch in the three-branch attention model to obtain a third weighted vector, including:

[0140] Inputting the text vector to be recognized into the privacy sentence recognition branch in the three-branch attention model, performing sentence processing on the text vector to be recognized to obtain multiple sentence vectors;

[0141] Perform feature extraction on each sentence vector to obtain a sentence vector feature matrix;

[0142] Performing a rearrangement operation, a feature decomposition, and a convolution operation on the sentence vector feature matrix in sequence to obtain a third feature matrix, and calculating a third attention weight of the third feature matrix;

[0143] According to the third attention weight and hidden state corresponding to each sentence vector, the third weighted vector is calculated.

[0144] Specifically, the third branch is the privacy sentence recognition branch, such as Figure 4 The privacy sentence recognition branch first divides the text vector to be recognized into sentences to obtain multiple sentence vectors. Each sentence vector is feature decomposed (Z-Pool), and then convolution operation is performed to obtain the third feature matrix Z3, which is as follows:

[0145] Z-Pool(H)=[MaxPool 0d (H),AvgPool 0d (H)]

[0146] Among them, Z-Pool splices the outputs of the maximum pooling and the average pooling together, and then convolves to obtain the third feature matrix Z3, and calculates the attention score e of the third feature matrix Z3 j , the specific formula is as follows:

[0147]

[0148] Among them, h j is the third feature matrix Z3 of the j-th sentence vector;

[0149] is the learned attention weight parameter used to measure the contribution of the sentence, and the score function is the dot product similarity measure.

[0150] And, the attention score e j Converted to the third attention weight A3:

[0151]

[0152] Where m is the number of sentence vectors.

[0153] Then the third attention weight A3 is used to perform weighted summation on the hidden state of the sentence vector to obtain the third weighted vector Attn3(Z):

[0154]

[0155] It should be noted that α j That is the third attention weight A3; h j is the hidden state of the jth sentence vector.

[0156] As an optional specific embodiment, the step 103 determines the first sensitive word type corresponding to the sensitive word vector from the preset sensitive word types according to the similarity between the sensitive word vector in the sensitive word recognition result and the feature vector of the preset sensitive word type, including:

[0157] Performing vector conversion on the sensitive word vector in the sensitive word recognition result to obtain a sensitive word feature vector;

[0158] Calculating a first similarity between the sensitive word feature vector and a feature vector of each preset sensitive word type;

[0159] The sensitive word type of the feature vector of the preset sensitive word type corresponding to the largest first similarity is determined as the first sensitive word type corresponding to the sensitive word vector.

[0160] Specifically, the sensitive word type identifier is composed of an attention weight calculation module, a sensitive word type identification module, a sensitive word type calculation module and a sensitive word type judgment module. In the sensitive word type identification module, the sensitive word vector in the sensitive word identification result is vectorized to obtain a sensitive word feature vector. The existing preset sensitive word type is represented in the feature space to obtain a feature vector of the preset sensitive word type. The sensitive word feature vector and the feature vector of each preset sensitive word type are Euclidean distance calculated (similarity is expressed by Euclidean distance) to obtain the distance d2 (i.e., the first similarity), and the sensitive word type of the feature vector of the preset sensitive word type corresponding to the smallest d2 (i.e., the largest first similarity) is used as the first sensitive word type corresponding to the sensitive word vector.

[0161] As an optional specific embodiment, the step 104 performs sensitive word type identification on the entity word vector according to the entity word vector in the entity word recognition result, the feature vector of the preset sensitive word type, and the weighted vector of each branch to obtain the second sensitive word type corresponding to the entity word vector, including:

[0162] Performing vector conversion on the entity word vector in the entity word recognition result to obtain an entity word feature vector;

[0163] Calculating a second similarity between the entity word feature vector and a feature vector of each preset sensitive word type;

[0164] When the maximum second similarity is within a preset similarity range, calculating a fourth weighted vector according to the first weighted vector, the second weighted vector, the third weighted vector and a fourth attention weight;

[0165] The fourth weighted vector is input into the distillation model to identify the sensitive word type, and the second sensitive word type corresponding to the entity word vector is obtained.

[0166] Specifically, in the attention weight calculation module, the first weighted vector Attn1(Z), the second weighted vector Attn2(Z), the third weighted vector Attn3(Z) and the fourth attention weight A4 are calculated to obtain the fourth weighted vector Attn4(Z):

[0167] Attn4(Z)=A4(σ(Attn1(Z))+σ(Attn2(Z))+σ(Attn3(Z)))

[0168] Among them, σ is the activation function Sigmoid.

[0169] In the sensitive word type calculation module, the entity word vector in the entity word recognition result is converted into an entity word feature vector, and the entity word feature vector and the feature vector of each preset sensitive word type are subjected to Euclidean distance calculation (i.e., similarity calculation) to obtain the distance d3 (i.e., the second similarity).

[0170] In the sensitive word type judgment module, the sensitive word type decision is made according to d3. If the second similarity is within the preset similarity range (i.e., the second preset threshold μ α ≤d3≤third preset threshold μ β ), the fourth attention weight A4 is used as input, and the distillation model is used to identify the new sensitive word type to obtain the second sensitive word type corresponding to the entity word vector. If the second similarity is not within the preset similarity range (i.e., d3> the third preset threshold μ β ), the entity word corresponding to this entity word vector is output as a common entity word.

[0171] In summary, the above embodiments of the present application can automatically identify sensitive information and entity information through the three-branch attention model, and can automatically discover and learn new types of sensitive information. The dynamic expansion capability can adapt to the ever-changing privacy protection needs, and can also improve the accuracy and flexibility of sensitive information identification, solving the problem that traditional privacy protection methods often rely on predefined sensitive data types and rules, and cannot meet the diversification of data types and the personalized needs of privacy protection needs, thereby reducing the maintenance cost of the system. In addition, by using the Euclidean distance method, the similarity between the text to be identified and the words in the sensitive word library is calculated, and potential sensitive words can be quickly and accurately judged. With a modular design, each functional module is independent and complementary, which is convenient for flexible integration and expansion in different application scenarios. Through reasonable module division and interface design, it can be easily integrated with the existing privacy protection system to improve the overall performance of the system.

[0172] The above describes the method for determining the sensitive word type provided in the embodiment of the present application. The following describes the device for determining the sensitive word type provided in the embodiment of the present application in conjunction with the accompanying drawings.

[0173] like Figure 7 As shown, the embodiment of the present application further provides a device 700 for determining the type of sensitive words, the device comprising:

[0174] The first acquisition module 701 acquires a text vector to be recognized;

[0175] The second acquisition module 702 is used to input the text vector to be recognized into the three-branch attention model to obtain sensitive word recognition results and entity word recognition results, and obtain weighted vectors of each branch;

[0176] A first determination module 703, configured to determine a first sensitive word type corresponding to the sensitive word vector from the preset sensitive word types according to the similarity between the sensitive word vector in the sensitive word recognition result and the feature vector of the preset sensitive word type;

[0177] The second determination module 704 is used to identify the sensitive word type of the entity word vector according to the entity word vector in the entity word recognition result, the feature vector of the preset sensitive word type and the weighted vector of each branch, and obtain the second sensitive word type corresponding to the entity word vector.

[0178] Optionally, the second acquisition module 702 is specifically configured to:

[0179] Input the text vector to be recognized into the sensitive word recognition branch in the three-branch attention model to obtain a first weighted vector and a sensitive word recognition result;

[0180] Inputting the text vector to be recognized into the entity word recognition branch in the three-branch attention model to obtain a second weighted vector and an entity word recognition result;

[0181] The text vector to be recognized is input into the privacy sentence recognition branch in the three-branch attention model to obtain a third weighted vector.

[0182] Optionally, when the second acquisition module 702 inputs the text vector to be recognized into the sensitive word recognition branch in the three-branch attention model to obtain the first weighted vector and the sensitive word recognition result, it is specifically used to:

[0183] Input the text vector to be recognized into the sensitive word recognition branch in the three-branch attention model, perform word segmentation processing on the text vector to be recognized, and obtain multiple word vectors;

[0184] Perform word embedding processing on each word vector to obtain a word embedding matrix corresponding to the word vector;

[0185] Performing a rearrangement operation, a feature decomposition, and a convolution operation on the word embedding matrix in sequence to obtain a first feature matrix, and calculating a first attention weight of the first feature matrix;

[0186] Calculate a first weighted vector according to the first feature matrix and the first attention weight;

[0187] The sensitive word recognition results of the multiple word vectors are determined according to the similarity between the first weighted vector of each word vector and the weighted vector of the preset sensitive word.

[0188] Optionally, when the second acquisition module 702 inputs the text vector to be recognized into the entity word recognition branch in the three-branch attention model to obtain the second weighted vector and the entity word recognition result, it is specifically used to:

[0189] Input the text vector to be recognized into the entity word recognition branch in the three-branch attention model, perform word segmentation processing on the text vector to be recognized, and obtain multiple word vectors;

[0190] Perform feature extraction on each word vector to obtain the word vector feature matrix;

[0191] Performing a rearrangement operation, a feature decomposition, and a convolution operation on the word vector feature matrix in sequence to obtain a second feature matrix, and calculating a second attention weight of the second feature matrix;

[0192] Calculate a second weighted vector according to the second feature matrix and the second attention weight;

[0193] The second weighted vector of each word vector is input into the conditional random field model to obtain entity word recognition results after entity word annotation of the multiple word vectors.

[0194] Optionally, when the second acquisition module 702 inputs the text vector to be recognized into the privacy sentence recognition branch in the three-branch attention model to obtain the third weighted vector, it is specifically used to:

[0195] Inputting the text vector to be recognized into the privacy sentence recognition branch in the three-branch attention model, performing sentence processing on the text vector to be recognized to obtain multiple sentence vectors;

[0196] Perform feature extraction on each sentence vector to obtain a sentence vector feature matrix;

[0197] Performing a rearrangement operation, a feature decomposition, and a convolution operation on the sentence vector feature matrix in sequence to obtain a third feature matrix, and calculating a third attention weight of the third feature matrix;

[0198] According to the third attention weight and hidden state corresponding to each sentence vector, the third weighted vector is calculated.

[0199] Optionally, the first determining module 703 is specifically configured to:

[0200] Performing vector conversion on the sensitive word vector in the sensitive word recognition result to obtain a sensitive word feature vector;

[0201] Calculating a first similarity between the sensitive word feature vector and a feature vector of each preset sensitive word type;

[0202] The sensitive word type of the feature vector of the preset sensitive word type corresponding to the largest first similarity is determined as the first sensitive word type corresponding to the sensitive word vector.

[0203] Optionally, the second determining module 704 is specifically configured to:

[0204] Performing vector conversion on the entity word vector in the entity word recognition result to obtain an entity word feature vector;

[0205] Calculating a second similarity between the entity word feature vector and a feature vector of each preset sensitive word type;

[0206] When the maximum second similarity is within a preset similarity range, calculating a fourth weighted vector according to the first weighted vector, the second weighted vector, the third weighted vector and a fourth attention weight;

[0207] The fourth weighted vector is input into the distillation model to identify the sensitive word type, and the second sensitive word type corresponding to the entity word vector is obtained.

[0208] It should be noted here that the above-mentioned sensitive word type determination device provided in the embodiment of the present application can implement all the method steps implemented in the above-mentioned sensitive word type determination method embodiment, and can achieve the same technical effect. The parts and beneficial effects of this embodiment that are the same as those of the method embodiment will not be described in detail here.

[0209] It should be noted that the division of units in the embodiments of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, each functional unit in each embodiment of the present application may be integrated into a processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0210] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a processor-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store program codes.

[0211] like Figure 8 As shown, an embodiment of the present application further provides an electronic device, including a memory 820, a transceiver 810, and a processor 800:

[0212] A memory 820, for storing computer programs;

[0213] A transceiver 810, configured to transmit and receive data under the control of the processor;

[0214] The processor 800 is configured to read the computer program in the memory and execute the steps of the method for determining the sensitive word type as described in any of the above embodiments.

[0215] Among them, Figure 8 In the embodiment, the bus architecture may include any number of interconnected buses and bridges, specifically one or more processors represented by processor 800 and various circuits of memory represented by memory 820 are linked together. The bus architecture may also link together various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art and are therefore not further described herein. The bus interface provides an interface. The transceiver 810 may be a plurality of components, namely, a transmitter and a receiver, providing a unit for communicating with various other devices on a transmission medium, which transmission medium may include a wireless channel, a wired channel, an optical cable, and other transmission media. The processor 800 is responsible for managing the bus architecture and general processing, and the memory 820 may store data used by the processor 800 when performing operations.

[0216] The processor 800 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or a complex programmable logic device (CPLD). The processor may also adopt a multi-core architecture.

[0217] The processor calls the computer program stored in the memory to execute the method for determining the sensitive word type provided in the embodiment of the present application according to the obtained executable instructions. The processor and the memory can also be arranged physically separately.

[0218] It should be noted here that the above-mentioned electronic device provided in the embodiment of the present application can implement all the method steps implemented in the above-mentioned sensitive word type determination method embodiment, and can achieve the same technical effect. The parts and beneficial effects of this embodiment that are the same as the method embodiment will not be described in detail here.

[0219] An embodiment of the present application further provides a processor-readable storage medium, wherein the processor-readable storage medium stores a computer program, and the computer program is used to enable the processor to execute the above-mentioned method for determining the sensitive word type.

[0220] The processor-readable storage medium can be any available medium or data storage device that can be accessed by the processor, including but not limited to magnetic storage (such as floppy disks, hard disks, magnetic tapes, magneto-optical disks (MO), etc.), optical storage (such as CD, DVD, BD, HVD, etc.), and semiconductor storage (such as ROM, EPROM, EEPROM, non-volatile memory (NANDFLASH), solid-state drive (SSD)), etc.

[0221] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) that contain computer-usable program code.

[0222] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer executable instructions. These computer executable instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0223] These processor executable instructions may also be stored in a processor readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the processor readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0224] These processor-executable instructions may also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0225] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for determining the type of sensitive words, characterized in that: The method comprises: Get the text vector to be recognized; Input the text vector to be recognized into the three-branch attention model to obtain sensitive word recognition results and entity word recognition results, and obtain the weighted vector of each branch; Determining a first sensitive word type corresponding to the sensitive word vector from the preset sensitive word types according to the similarity between the sensitive word vector in the sensitive word recognition result and the feature vector of the preset sensitive word type; According to the entity word vector in the entity word recognition result, the feature vector of the preset sensitive word type and the weighted vector of each branch, the entity word vector is subjected to sensitive word type recognition to obtain a second sensitive word type corresponding to the entity word vector.

2. The method according to claim 1, characterized in that The text vector to be recognized is input into the three-branch attention model to obtain sensitive word recognition results and entity word recognition results, and obtain the weighted vectors of each branch, including: Input the text vector to be recognized into the sensitive word recognition branch in the three-branch attention model to obtain a first weighted vector and a sensitive word recognition result; Inputting the text vector to be recognized into the entity word recognition branch in the three-branch attention model to obtain a second weighted vector and an entity word recognition result; The text vector to be recognized is input into the privacy sentence recognition branch in the three-branch attention model to obtain a third weighted vector.

3. The method according to claim 2, characterized in that The step of inputting the text vector to be recognized into the sensitive word recognition branch in the three-branch attention model to obtain a first weighted vector and a sensitive word recognition result includes: Input the text vector to be recognized into the sensitive word recognition branch in the three-branch attention model, perform word segmentation processing on the text vector to be recognized, and obtain multiple word vectors; Perform word embedding processing on each word vector to obtain a word embedding matrix corresponding to the word vector; Performing a rearrangement operation, a feature decomposition, and a convolution operation on the word embedding matrix in sequence to obtain a first feature matrix, and calculating a first attention weight of the first feature matrix; Calculate a first weighted vector according to the first feature matrix and the first attention weight; The sensitive word recognition results of the multiple word vectors are determined according to the similarity between the first weighted vector of each word vector and the weighted vector of the preset sensitive word.

4. The method according to claim 2, characterized in that: The step of inputting the text vector to be recognized into the entity word recognition branch in the three-branch attention model to obtain a second weighted vector and an entity word recognition result includes: Input the text vector to be recognized into the entity word recognition branch in the three-branch attention model, perform word segmentation processing on the text vector to be recognized, and obtain multiple word vectors; Perform feature extraction on each word vector to obtain the word vector feature matrix; Performing a rearrangement operation, a feature decomposition, and a convolution operation on the word vector feature matrix in sequence to obtain a second feature matrix, and calculating a second attention weight of the second feature matrix; Calculate a second weighted vector according to the second feature matrix and the second attention weight; The second weighted vector of each word vector is input into the conditional random field model to obtain entity word recognition results after entity word annotation of the multiple word vectors.

5. The method according to claim 2, characterized in that: The step of inputting the to-be-recognized text vector into the privacy sentence recognition branch in the three-branch attention model to obtain a third weighted vector includes: Inputting the text vector to be recognized into the privacy sentence recognition branch in the three-branch attention model, performing sentence processing on the text vector to be recognized to obtain multiple sentence vectors; Perform feature extraction on each sentence vector to obtain a sentence vector feature matrix; Performing a rearrangement operation, a feature decomposition, and a convolution operation on the sentence vector feature matrix in sequence to obtain a third feature matrix, and calculating a third attention weight of the third feature matrix; According to the third attention weight and hidden state corresponding to each sentence vector, the third weighted vector is calculated.

6. The method according to claim 2, characterized in that The determining, according to the similarity between the sensitive word vector in the sensitive word recognition result and the feature vector of the preset sensitive word type, the first sensitive word type corresponding to the sensitive word vector from the preset sensitive word type includes: Performing vector conversion on the sensitive word vector in the sensitive word recognition result to obtain a sensitive word feature vector; Calculating a first similarity between the sensitive word feature vector and a feature vector of each preset sensitive word type; The sensitive word type of the feature vector of the preset sensitive word type corresponding to the largest first similarity is determined as the first sensitive word type corresponding to the sensitive word vector.

7. The method according to claim 2, characterized in that The step of performing sensitive word type identification on the entity word vector according to the entity word recognition result, the feature vector of the preset sensitive word type, and the weighted vector of each branch to obtain a second sensitive word type corresponding to the entity word vector includes: Performing vector conversion on the entity word vector in the entity word recognition result to obtain an entity word feature vector; Calculating a second similarity between the entity word feature vector and a feature vector of each preset sensitive word type; When the maximum second similarity is within a preset similarity range, calculating a fourth weighted vector according to the first weighted vector, the second weighted vector, the third weighted vector and a fourth attention weight; The fourth weighted vector is input into the distillation model to identify the sensitive word type, and the second sensitive word type corresponding to the entity word vector is obtained.

8. A device for determining the type of sensitive words, characterized in that: The device comprises: The first acquisition module acquires the text vector to be recognized; The second acquisition module is used to input the text vector to be recognized into the three-branch attention model to obtain sensitive word recognition results and entity word recognition results, and obtain weighted vectors of each branch; A first determination module, configured to determine a first sensitive word type corresponding to the sensitive word vector from the preset sensitive word types according to a similarity between the sensitive word vector in the sensitive word recognition result and a feature vector of a preset sensitive word type; The second determination module is used to identify the sensitive word type of the entity word vector according to the entity word vector in the entity word recognition result, the feature vector of the preset sensitive word type and the weighted vector of each branch, so as to obtain the second sensitive word type corresponding to the entity word vector.

9. An electronic device, characterized in that: Including memory, transceiver, processor: Memory for storing computer programs; a transceiver, for transmitting and receiving data under the control of the processor; A processor is used to read the computer program in the memory and execute the method for determining the sensitive word type as described in any one of claims 1 to 7.

10. A processor-readable storage medium, characterized in that: The processor-readable storage medium stores a computer program, and the computer program is used to enable the processor to execute the method for determining the sensitive word type according to any one of claims 1 to 7.

Images