Methods and apparatus for determining sensitive word types

Abstract

The embodiment of the application provides a sensitive word type determination method and device, the method comprises: obtaining a to-be-recognized text vector; inputting the to-be-recognized text vector into a three-branch attention model to obtain a sensitive word recognition result and an entity word recognition result, and obtain a weighting vector of each branch; determining a first sensitive word type corresponding to a sensitive word vector in the sensitive word recognition result from a preset sensitive word type according to the similarity between the sensitive word vector and a feature vector of the preset sensitive word type; and performing sensitive word type recognition on an entity word vector in the entity word recognition result according to the entity word vector, the feature vector of the preset sensitive word type and the weighting vector of each branch, to obtain a second sensitive word type corresponding to the entity word vector.

Description

Technical Field

[0001] This application relates to the field of data security technology, and in particular to a method and apparatus for determining sensitive word types. Background Technology

[0002] With the rapid development of information technology, data privacy issues are receiving increasing attention. Personal privacy data includes, but is not limited to, personal identification information, contact information, and location data. The leakage of this information may lead to the infringement of personal privacy and financial losses. Traditional privacy protection methods often rely on predefined types of privacy data and rules, which cannot meet the diverse needs of data types and the personalized needs of privacy protection. Summary of the Invention

[0003] In view of the above problems, embodiments of this application provide a method and apparatus for determining sensitive word types that overcomes or at least partially solves the above problems.

[0004] In a first aspect, embodiments of this application provide a method for determining the type of sensitive words, the method comprising:

[0005] Obtain the vector of the text to be recognized;

[0006] The text vector to be identified is input into a three-branch attention model to obtain the sensitive word recognition result and the entity word recognition result, and the weighted vector of each branch is obtained.

[0007] Based on the similarity between the sensitive word vector in the sensitive word recognition result and the feature vector of the preset sensitive word type, the first sensitive word type corresponding to the sensitive word vector is determined from the preset sensitive word types;

[0008] Based on the entity word vectors in the entity word recognition results, the feature vectors of the preset sensitive word types, and the weighted vectors of each branch, the entity word vectors are subjected to sensitive word type recognition to obtain the second sensitive word type corresponding to the entity word vectors.

[0009] Optionally, the step of inputting the text vector to be identified into a three-branch attention model to obtain sensitive word recognition results and entity word recognition results, and obtaining the weighted vector of each branch, includes:

[0010] The text vector to be identified is input into the sensitive word identification branch of the three-branch attention model to obtain the first weighted vector and the sensitive word identification result;

[0011] The text vector to be identified is input into the entity word recognition branch of the three-branch attention model to obtain the second weighted vector and the entity word recognition result;

[0012] The text vector to be identified is input into the privacy sentence recognition branch of the three-branch attention model to obtain the third weighted vector.

[0013] Optionally, the step of inputting the text vector to be identified into the sensitive word identification branch of the three-branch attention model to obtain the first weighted vector and the sensitive word identification result includes:

[0014] The text vector to be identified is input into the sensitive word identification branch of the three-branch attention model, and the text vector to be identified is segmented to obtain multiple word vectors;

[0015] Each word vector is subjected to word embedding processing to obtain the word embedding matrix corresponding to the word vector;

[0016] The word embedding matrix is ​​sequentially rearranged, decomposed, and convolved to obtain a first feature matrix, and the first attention weight of the first feature matrix is ​​calculated.

[0017] Calculate the first weighted vector based on the first feature matrix and the first attention weight;

[0018] The sensitive word recognition result of the multiple word vectors is determined based on the similarity between the first weighted vector of each word vector and the weighted vector of the preset sensitive words.

[0019] Optionally, the step of inputting the text vector to be identified into the entity word recognition branch of the three-branch attention model to obtain the second weighted vector and the entity word recognition result includes:

[0020] The text vector to be identified is input into the entity word recognition branch of the three-branch attention model, and the text vector to be identified is segmented to obtain multiple word vectors;

[0021] Feature extraction is performed on each word vector to obtain the word vector feature matrix;

[0022] The word vector feature matrix is ​​sequentially rearranged, decomposed, and convolved to obtain a second feature matrix, and the second attention weight of the second feature matrix is ​​calculated.

[0023] Calculate the second weighted vector based on the second feature matrix and the second attention weight;

[0024] The second weighted vector of each word vector is input into the conditional random field model to obtain the entity word recognition result after the entity word annotation of the multiple word vectors.

[0025] Optionally, the step of inputting the text vector to be identified into the privacy sentence recognition branch of the three-branch attention model to obtain the third weighted vector includes:

[0026] The text vector to be identified is input into the privacy sentence recognition branch of the three-branch attention model, and the text vector to be identified is segmented into sentences to obtain multiple sentence vectors.

[0027] Feature extraction is performed on each sentence vector to obtain the sentence vector feature matrix;

[0028] The sentence vector feature matrix is ​​sequentially rearranged, decomposed, and convolved to obtain a third feature matrix, and the third attention weight of the third feature matrix is ​​calculated.

[0029] Calculate the third weighted vector based on the third attention weight and hidden state corresponding to each sentence vector.

[0030] Optionally, determining the first sensitive word type corresponding to the sensitive word vector from the preset sensitive word types based on the similarity between the sensitive word vector in the sensitive word recognition result and the feature vector of the preset sensitive word type includes:

[0031] The sensitive word vectors in the sensitive word identification results are transformed into sensitive word feature vectors;

[0032] Calculate the first similarity between the sensitive word feature vector and the feature vector of each preset sensitive word type;

[0033] The sensitive word type of the feature vector corresponding to the highest first similarity is determined as the first sensitive word type corresponding to the sensitive word vector.

[0034] Optionally, the step of performing sensitive word type identification on the entity word vector based on the entity word vector in the entity word recognition result, the feature vector of the preset sensitive word type, and the weighted vector of each branch to obtain the second sensitive word type corresponding to the entity word vector includes:

[0035] The entity word vectors in the entity word recognition results are transformed into entity word feature vectors.

[0036] Calculate the second similarity between the entity word feature vector and the feature vector of each preset sensitive word type;

[0037] If the maximum second similarity is within a preset similarity range, the fourth weighted vector is calculated based on the first weighted vector, the second weighted vector, the third weighted vector, and the fourth attention weight.

[0038] The fourth weighted vector is input into the distillation model to identify sensitive word types, thereby obtaining the second sensitive word type corresponding to the entity word vector.

[0039] Secondly, embodiments of this application also provide a device for determining sensitive word types, the device comprising:

[0040] The first acquisition module acquires the vector of the text to be recognized;

[0041] The second acquisition module is used to input the text vector to be identified into the three-branch attention model to obtain the sensitive word recognition result and the entity word recognition result, and to obtain the weighted vector of each branch;

[0042] The first determining module is used to determine the first sensitive word type corresponding to the sensitive word vector from the preset sensitive word types based on the similarity between the sensitive word vector in the sensitive word recognition result and the feature vector of the preset sensitive word type;

[0043] The second determining module is used to identify the sensitive word type of the entity word vector based on the entity word vector in the entity word recognition result, the feature vector of the preset sensitive word type, and the weighted vector of each branch, so as to obtain the second sensitive word type corresponding to the entity word vector.

[0044] Thirdly, embodiments of this application also provide an electronic device, including a memory, a transceiver, and a processor:

[0045] A memory for storing computer programs; a transceiver for sending and receiving data under the control of a processor; and a processor for reading the computer programs from the memory and executing the method described in the first aspect above.

[0046] Fourthly, embodiments of this application also provide a processor-readable storage medium storing a computer program for causing the processor to perform the method described in the first aspect above.

[0047] In the embodiments described above, a text vector to be identified is obtained, and this vector is input into a three-branch attention model to obtain sensitive word recognition results and entity word recognition results, as well as weighted vectors for each branch. This means the three-branch attention model can automatically identify sensitive information and entity information. Furthermore, based on the similarity between the sensitive word vector in the sensitive word recognition results and the feature vector of a preset sensitive word type, a first sensitive word type corresponding to the sensitive word vector is determined from the preset sensitive word types. Based on the entity word vector in the entity word recognition results, the feature vector of the preset sensitive word type, and the weighted vectors of each branch, the entity word vector is subjected to sensitive word type recognition to obtain a second sensitive word type corresponding to the entity word vector. This allows for the automatic discovery and learning of new sensitive information types, and its dynamic expansion capability can adapt to constantly changing privacy protection needs. It also improves the accuracy and flexibility of sensitive information identification, solving the problem that traditional privacy protection methods often rely on predefined sensitive data types and rules, failing to meet the diverse data types and personalized privacy protection requirements, and reducing system maintenance costs. Attached Figure Description

[0048] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0049] Figure 1 One of the flowcharts for a method of determining sensitive word types provided in the embodiments of this application;

[0050] Figure 2 The second flowchart of the method for determining sensitive word types provided in the embodiments of this application;

[0051] Figure 3 The third flowchart of the method for determining sensitive word types provided in the embodiments of this application;

[0052] Figure 4 Flowchart four of the methods for determining sensitive word types provided in the embodiments of this application;

[0053] Figure 5 The fifth flowchart of the method for determining sensitive word types provided in the embodiments of this application;

[0054] Figure 6 Flowchart six of the method for determining sensitive word types provided in the embodiments of this application;

[0055] Figure 7A structural block diagram of the device for determining sensitive word types provided in the embodiments of this application;

[0056] Figure 8 A structural block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0057] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0058] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0059] The method for determining sensitive word types provided in this application will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0060] Specifically, embodiments of this application provide a method for determining the type of sensitive words, such as... Figure 1 , Figure 2 and Figure 6 As shown, the specific steps may include the following:

[0061] Step 101: Obtain the vector of the text to be recognized.

[0062] Specifically, such as Figure 2 , Figure 3 and Figure 6As shown, the text to be recognized is obtained, and the text is preprocessed, including but not limited to: noise removal (removing irrelevant noise); special character removal (removing special characters and punctuation); stop word removal (removing common high-frequency irrelevant words); spelling correction (detecting and correcting spelling errors); and text filtering (further cleaning the text and retaining meaningful content). After the above operations, the initial text data x1, x2, ... x is obtained. n .

[0063] Given the initial text data x1, x2, ... x n Data encoding is performed using BERT-Base-Chinese to obtain the text vectors h1, h2, ... h to be recognized. n The specific formula is as follows:

[0064] h1, h2, ... h n = Bert(x1,x2,…x n )

[0065] Among them, BERT-Base-Chinese is a pre-trained version of the Bidirectional Encoder Representations from Transformers (BERT) model for Chinese text. It is pre-trained on Chinese text and can learn the semantic and syntactic information of Chinese text.

[0066] Step 102: Input the text vector to be identified into the three-branch attention model to obtain the sensitive word recognition result and the entity word recognition result, and obtain the weighted vector of each branch.

[0067] Specifically, the text vector to be identified is input into the three branches of the three-branch attention model to obtain the weighted vectors corresponding to the three branches, and the results of sensitive word recognition and entity word recognition are obtained.

[0068] Step 103: Based on the similarity between the sensitive word vector in the sensitive word recognition result and the feature vector of the preset sensitive word type, determine the first sensitive word type corresponding to the sensitive word vector from the preset sensitive word types.

[0069] Specifically, such as Figure 2As shown, the sensitive word identification result includes the sensitive word and its corresponding sensitive word vector. Each sensitive word vector is matched with the feature vectors of multiple preset sensitive word types to determine the preset sensitive word type to which the sensitive word vector belongs, i.e., the first sensitive word type.

[0070] Step 104: Based on the entity word vector in the entity word recognition result, the feature vector of the preset sensitive word type, and the weighted vector of each branch, perform sensitive word type recognition on the entity word vector to obtain the second sensitive word type corresponding to the entity word vector.

[0071] Specifically, such as Figure 2 As shown, the entity word recognition result includes entity words and their corresponding entity word vectors. Sensitive word types can be identified using entity word vectors, feature vectors of multiple preset sensitive word types, and weighted vectors of each branch, to obtain new sensitive word types corresponding to the entity word vectors, i.e., the second sensitive word type, thus expanding the existing sensitive word types.

[0072] In the embodiments described above, a text vector to be identified is obtained, and this vector is input into a three-branch attention model to obtain sensitive word recognition results and entity word recognition results, as well as weighted vectors for each branch. This means the three-branch attention model can automatically identify sensitive information and entity information. Furthermore, based on the similarity between the sensitive word vector in the sensitive word recognition results and the feature vector of a preset sensitive word type, a first sensitive word type corresponding to the sensitive word vector is determined from the preset sensitive word types. Based on the entity word vector in the entity word recognition results, the feature vector of the preset sensitive word type, and the weighted vectors of each branch, the entity word vector is subjected to sensitive word type recognition to obtain a second sensitive word type corresponding to the entity word vector. This allows for the automatic discovery and learning of new sensitive information types, and its dynamic expansion capability can adapt to constantly changing privacy protection needs. It also improves the accuracy and flexibility of sensitive information identification, solving the problem that traditional privacy protection methods often rely on predefined sensitive data types and rules, failing to meet the diverse data types and personalized privacy protection requirements, and reducing system maintenance costs.

[0073] As an optional specific embodiment, step 102 inputs the text vector to be identified into a three-branch attention model to obtain sensitive word recognition results and entity word recognition results, and obtains the weighted vectors of each branch, specifically including steps 1021 to 1023:

[0074] Step 1021: Input the text vector to be identified into the sensitive word identification branch of the three-branch attention model to obtain the first weighted vector and the sensitive word identification result.

[0075] Specifically, such as Figure 2 and Figure 6 As shown, the text vector to be identified is input into the sensitive word identification branch in the three-branch attention model. The first weighted vector is calculated through the sensitive word identification branch, and the sensitive word identification result is obtained based on the first weighted vector.

[0076] Step 1022: Input the text vector to be identified into the entity word recognition branch in the three-branch attention model to obtain the second weighted vector and the entity word recognition result.

[0077] Specifically, such as Figure 2 and Figure 6 As shown, the text vector to be recognized is input into the entity word recognition branch in the three-branch attention model. The second weighted vector is calculated through the entity word recognition branch, and the entity word recognition result is obtained based on the second weighted vector.

[0078] Step 1023: Input the text vector to be identified into the privacy sentence recognition branch of the three-branch attention model to obtain the third weighted vector.

[0079] Specifically, such as Figure 2 and Figure 6 As shown, the text vector to be identified is input into the privacy sentence recognition branch in the three-branch attention model, and the third weighted vector is calculated through the privacy sentence recognition branch.

[0080] Further, step 1021 inputs the text vector to be identified into the sensitive word identification branch of the three-branch attention model to obtain the first weighted vector and the sensitive word identification result, including:

[0081] The text vector to be identified is input into the sensitive word identification branch of the three-branch attention model, and the text vector to be identified is segmented to obtain multiple word vectors;

[0082] Each word vector is subjected to word embedding processing to obtain the word embedding matrix corresponding to the word vector;

[0083] The word embedding matrix is ​​sequentially rearranged, decomposed, and convolved to obtain a first feature matrix, and the first attention weight of the first feature matrix is ​​calculated.

[0084] Calculate the first weighted vector based on the first feature matrix and the first attention weight;

[0085] The sensitive word recognition result of the multiple word vectors is determined based on the similarity between the first weighted vector of each word vector and the weighted vector of the preset sensitive words.

[0086] Specifically, such as Figure 3 and Figure 4 As shown, the first branch of the three-branch attention model is the sensitive word recognition branch, such as... Figure 4 As shown in branch a, the sensitive word identification branch first performs word segmentation on the text vector to be identified, obtaining multiple word vectors. In the first branch, word embedding GloVe is used to learn word embeddings, encoding both local contextual information and global word co-occurrence statistics. GloVe generates word vectors based on the frequency F of word occurrences, as shown in the following formula:

[0087]

[0088] Among them, P ij It is obtained by the number of times the i-th word vector and the j-th word vector co-occur within the preset context window, that is, the probability that the i-th word vector appears in the context of the j-th word vector;

[0089] P i It is the probability of the i-th word vector appearing in the text vector to be recognized.

[0090] The core idea of ​​GloVe is to use these co-occurrence frequencies F(w i ,w j GloVe generates word vectors using a target function J. GloVe constructs a word vector space where the similarity between words is represented by distance. GloVe learns word vectors using the target function J, as shown in the following formula:

[0091]

[0092] Among them, X ij It is the number of times the i-th word vector and the j-th word vector co-occur, that is, the number of times the i-th word vector and the j-th word vector appear together within the preset context window;

[0093] V is the size of the text vector to be recognized;

[0094] w is the word vector;

[0095] b is the bias term;

[0096] f(X ij () is a weighting function used to mitigate the impact of frequently co-occurring words on the objective function;

[0097] X max It is a preset maximum co-occurrence threshold;

[0098] α is a preset hyperparameter, 0 < α < 1.

[0099] Word embeddings are performed on the word vectors using GloVe, resulting in a word embedding matrix E. c ∈Rn×d The specific formula is as follows:

[0100] E c =Glove(T)

[0101] In the sensitive word recognition branch, the word embedding matrix E c A permutation operation is performed to preserve the interaction capture between word embedding dimensions, focusing on the semantic features in the word embedding matrix. The permuted matrix is ​​then subjected to feature decomposition (Z-Pool) followed by convolution to obtain the first feature matrix Z1, as shown in the following formula:

[0102] Z-Pool(H) = [MaxPool] 0d (H),AvgPool 0d (H)]

[0103] Z-Pool concatenates the outputs of max pooling and average pooling, then convolves them to obtain the first feature matrix Z1, and calculates the first attention weight A1 of the first feature matrix Z1, as shown in the following formula:

[0104] a1 = softmax(W1Z1 + b1)

[0105] Here, W1 is the weight matrix learned in the attention mechanism, which is learned through the backpropagation algorithm during model training. During the forward propagation of the model, the weight matrix W1 and the bias term b1 are assigned values ​​through random initialization. Subsequently, each time the model calculates the attention weights and output, the gradient is calculated based on the current loss function, and the weights are updated through backpropagation. Here, W1 and b1 are the weights and bias terms updated through backpropagation after the loss function is calculated.

[0106] Then, the first attention weight A1 is applied to the first feature matrix Z1 after convolution, and processed by the Sigmoid activation function to obtain the first weighted vector Attn1(Z), as shown in the following formula:

[0107] Attn1(Z) = A1·Sigmoid(Z1);

[0108] The data that has been captured through dimensional interaction is restored to its original state through a rearrangement operation, that is, the dimensions and positions are restored.

[0109] Calculate the similarity between the first weighted vector of each word vector and the weighted vector of the preset sensitive words, specifically using Euclidean distance to represent the similarity:

[0110]

[0111] Where d1(v1,v2) is the Euclidean distance between vectors v1 and v2, v1 is a word vector, v2 is a weighted vector of preset sensitive words, k is the dimension index of the vector, and C is the dimension size of the vector.

[0112] Then, the multiple d1 values ​​are sorted by size, and the smallest d1 (i.e. the closest in Euclidean distance and the highest similarity) is compared with the first preset threshold λ1. If d1 < λ1, it means that the word corresponding to the word vector is a sensitive word, thus obtaining the sensitive word identification result.

[0113] Further, step 1022 inputs the text vector to be identified into the entity word recognition branch of the three-branch attention model to obtain the second weighted vector and the entity word recognition result, including:

[0114] The text vector to be identified is input into the entity word recognition branch of the three-branch attention model, and the text vector to be identified is segmented to obtain multiple word vectors;

[0115] Feature extraction is performed on each word vector to obtain the word vector feature matrix;

[0116] The word vector feature matrix is ​​sequentially rearranged, decomposed, and convolved to obtain a second feature matrix, and the second attention weight of the second feature matrix is ​​calculated.

[0117] Calculate the second weighted vector based on the second feature matrix and the second attention weight;

[0118] The second weighted vector of each word vector is input into the conditional random field model to obtain the entity word recognition result after the entity word annotation of the multiple word vectors.

[0119] Specifically, such as Figure 3 and Figure 4 As shown, the second branch is the entity recognition branch, such as... Figure 5 As shown, it specifically includes: an input layer, a Text Convolutional Neural Network (TextCNN) layer, a BiLSTM layer, an information attention mechanism layer, a fully connected layer, and a Conditional Random Field (CRF) layer. Figure 4In the b branch of the TextCNN layer, the entity recognition branch first performs word segmentation on the text vector to be recognized, obtaining multiple word vectors. The TextCNN layer uses a series of convolutional kernels of different sizes to extract features from the word vectors, capturing multi-level contextual information in the text vector to be recognized. For example, five convolutional kernels can be used, namely 3×k, 4×k, 5×k, 6×k, and 7×k. Each convolution operation can be regarded as a feature extraction process, as shown in the following formula:

[0120] c i =f(W·X) i:i+h-1 +b)

[0121] Here, it is assumed that the size of a convolutional kernel is n×k;

[0122] The weight matrix of the W2 convolution kernel;

[0123] b2 is the bias term;

[0124] f is the ReLU activation function;

[0125] X i:i+h-1 It is the word embedding matrix E c The submatrix from row i to row i+h.

[0126] The pooling layer applies max pooling to the output region, outputting pooled feature information, as shown in the following formula:

[0127] c max =max(c1,c2,…,c n-h+1 )

[0128] BiLSTM (directional Long Short-Term Memory) is a combination of forward LSTM and backward LSTM. By combining two independent LSTM networks, BiLSTM effectively extracts bidirectional feature information of the input sequence and obtains the word vector feature matrix.

[0129] BiLSTM is composed of a forward LSTM and a backward LSTM.

[0130] In the entity recognition branch, the word vector feature matrix is ​​rearranged to capture the interactions between different word vectors and to calculate the dependencies between each time step and other time steps. Then, the rearranged matrix undergoes feature decomposition (Z-Pool) followed by convolution to obtain the second feature matrix Z2, as shown in the following formula:

[0131] Z-Pool(H) = [MaxPool] 0d (H),AvgPool 0d (H)]

[0132] Z-Pool concatenates the outputs of max pooling and average pooling, then convolves them to obtain the second feature matrix Z2, and calculates the second attention weight A2 of the second feature matrix Z2, as shown in the following formula:

[0133] A2 = softmax(W2Z2 + b2)

[0134] Here, W2 is the weight matrix learned in the attention mechanism, which is learned through the backpropagation algorithm during model training. During the forward propagation of the model, the weight matrix W2 and the bias term b2 are assigned values ​​through random initialization. Subsequently, each time the model calculates the attention weights and output, the gradient is calculated based on the current loss function, and the weights are updated through backpropagation. Here, W2 and b2 are the weights and bias terms updated through backpropagation after the loss function is calculated.

[0135] Then, the first attention weight A2 is applied to the second feature matrix Z2 after convolution, and processed by the Sigmoid activation function to obtain the second weighted vector Attn2(Z), as shown in the following formula:

[0136] Attn2(Z) = A2·Sigmoid(Z2);

[0137] The data that has been captured through dimensional interaction is restored to its original state through a rearrangement operation, that is, the dimensions and positions are restored.

[0138] The constraints and dependencies between labels are learned through CRF, and the results are evaluated by maximum likelihood estimation to obtain the entity word annotation results and the entity word recognition results after annotation.

[0139] Further, step 1023 inputs the text vector to be identified into the privacy sentence recognition branch of the three-branch attention model to obtain a third weighted vector, including:

[0140] The text vector to be identified is input into the privacy sentence recognition branch of the three-branch attention model, and the text vector to be identified is segmented into sentences to obtain multiple sentence vectors.

[0141] Feature extraction is performed on each sentence vector to obtain the sentence vector feature matrix;

[0142] The sentence vector feature matrix is ​​sequentially rearranged, decomposed, and convolved to obtain a third feature matrix, and the third attention weight of the third feature matrix is ​​calculated.

[0143] Calculate the third weighted vector based on the third attention weight and hidden state corresponding to each sentence vector.

[0144] Specifically, the third branch is the privacy sentence recognition branch, such as... Figure 4 As shown in branch c, the privacy sentence recognition branch first segments the text vector to be recognized into multiple sentence vectors. Each sentence vector undergoes feature decomposition (Z-Pool) and then convolution to obtain the third feature matrix Z3, as shown in the following formula:

[0145] Z-Pool(H) = [MaxPool] 0d (H),AvgPool 0d (H)]

[0146] Z-Pool concatenates the outputs of max pooling and average pooling, then convolves them to obtain the third feature matrix Z3, and calculates the attention score e of the third feature matrix Z3. j The specific formula is as follows:

[0147]

[0148] Among them, h j Z3 is the third feature matrix of the j-th sentence vector;

[0149] These are the learned attention weight parameters used to measure the contribution of a sentence, and the score function is a dot product similarity measure.

[0150] Furthermore, the attention score e j Convert to third attention weight A3:

[0151]

[0152] Where m is the number of sentence vectors.

[0153] Then, the hidden states of the sentence vector are weighted and summed using the third attention weight A3 to obtain the third weighted vector Attn3(Z):

[0154]

[0155] It should be noted that α j This is the third attention weight A3; h j It is the hidden state of the j-th sentence vector.

[0156] As an optional specific embodiment, step 103, based on the similarity between the sensitive word vector in the sensitive word recognition result and the feature vector of a preset sensitive word type, determines the first sensitive word type corresponding to the sensitive word vector from the preset sensitive word types, including:

[0157] The sensitive word vectors in the sensitive word identification results are transformed into sensitive word feature vectors;

[0158] Calculate the first similarity between the sensitive word feature vector and the feature vector of each preset sensitive word type;

[0159] The sensitive word type of the feature vector corresponding to the highest first similarity is determined as the first sensitive word type corresponding to the sensitive word vector.

[0160] Specifically, the sensitive word type identifier consists of an attention weight calculation module, a sensitive word type identification module, a sensitive word type calculation module, and a sensitive word type judgment module. In the sensitive word type identification module, the sensitive word vectors in the sensitive word identification results are transformed into sensitive word feature vectors. Existing preset sensitive word types are represented in the feature space to obtain feature vectors for those preset sensitive word types. Euclidean distance is calculated between the sensitive word feature vectors and the feature vectors of each preset sensitive word type (similarity is represented by Euclidean distance), yielding a distance d2 (i.e., the first similarity). The sensitive word type of the feature vector corresponding to the preset sensitive word type with the smallest d2 (i.e., the largest first similarity) is taken as the first sensitive word type corresponding to that sensitive word vector.

[0161] As an optional specific embodiment, step 104, based on the entity word vector in the entity word recognition result, the feature vector of the preset sensitive word type, and the weighted vector of each branch, performs sensitive word type identification on the entity word vector to obtain the second sensitive word type corresponding to the entity word vector, including:

[0162] The entity word vectors in the entity word recognition results are transformed into entity word feature vectors.

[0163] Calculate the second similarity between the entity word feature vector and the feature vector of each preset sensitive word type;

[0164] If the maximum second similarity is within a preset similarity range, the fourth weighted vector is calculated based on the first weighted vector, the second weighted vector, the third weighted vector, and the fourth attention weight.

[0165] The fourth weighted vector is input into the distillation model to identify sensitive word types, thereby obtaining the second sensitive word type corresponding to the entity word vector.

[0166] Specifically, in the attention weight calculation module, the obtained first weighted vector Attn1(Z), second weighted vector Attn2(Z), third weighted vector Attn3(Z), and fourth attention weight A4 are calculated to obtain the fourth weighted vector Attn4(Z):

[0167] Attn4(Z)=A4(σ(Attn1(Z))+σ(Attn2(Z))+σ(Attn3(Z)))

[0168] Where σ is the activation function Sigmoid.

[0169] In the sensitive word type calculation module, the entity word vectors in the entity word recognition results are converted into entity word feature vectors, and the entity word feature vectors and the feature vectors of each preset sensitive word type are calculated using Euclidean distance (i.e., similarity calculation) to obtain the distance d3 (i.e., the second similarity).

[0170] In the sensitive word type determination module, a sensitive word type decision is made based on d3. If the second similarity is within the preset similarity range (i.e., the second preset threshold μ), the decision is made accordingly. α ≤d3≤Third preset threshold μ β If the fourth attention weight A4 is used as input, the distillation model is used to identify the new sensitive word type, resulting in the second sensitive word type corresponding to the entity word vector. If the second similarity is not within the preset similarity range (i.e., d3 > the third preset threshold μ), then... β If the entity vector is 0, then the entity word corresponding to this entity word vector will be output as a regular entity word.

[0171] In summary, the embodiments described above in this application, through a three-branch attention model, can automatically identify sensitive information and entity information, and can automatically discover and learn new types of sensitive information. Its dynamic scalability can adapt to constantly changing privacy protection needs, and it also improves the accuracy and flexibility of sensitive information identification. This solves the problem that traditional privacy protection methods often rely on predefined sensitive data types and rules, failing to meet the diverse data types and personalized privacy protection requirements, thus reducing system maintenance costs. Furthermore, by using the Euclidean distance method to calculate the similarity between the text to be identified and words in the sensitive word library, potential sensitive words can be quickly and accurately identified. The modular design, with each functional module being independent and complementary, facilitates flexible integration and expansion in different application scenarios. Through reasonable module division and interface design, it can be easily integrated with existing privacy protection systems, improving overall system performance.

[0172] The above describes the method for determining sensitive word types provided in the embodiments of this application. The following will describe the device for determining sensitive word types provided in the embodiments of this application with reference to the accompanying drawings.

[0173] like Figure 7 As shown in the figure, this application embodiment also provides a sensitive word type determination device 700, the device comprising:

[0174] The first acquisition module 701 acquires the vector of the text to be recognized;

[0175] The second acquisition module 702 is used to input the text vector to be identified into the three-branch attention model to obtain the sensitive word recognition result and the entity word recognition result, and to obtain the weighted vector of each branch;

[0176] The first determining module 703 is used to determine the first sensitive word type corresponding to the sensitive word vector from the preset sensitive word types based on the similarity between the sensitive word vector in the sensitive word recognition result and the feature vector of the preset sensitive word type;

[0177] The second determining module 704 is used to perform sensitive word type identification on the entity word vector based on the entity word vector in the entity word recognition result, the feature vector of the preset sensitive word type, and the weighted vector of each branch, so as to obtain the second sensitive word type corresponding to the entity word vector.

[0178] Optionally, the second acquisition module 702 is specifically used for:

[0179] The text vector to be identified is input into the sensitive word identification branch of the three-branch attention model to obtain the first weighted vector and the sensitive word identification result;

[0180] The text vector to be identified is input into the entity word recognition branch of the three-branch attention model to obtain the second weighted vector and the entity word recognition result;

[0181] The text vector to be identified is input into the privacy sentence recognition branch of the three-branch attention model to obtain the third weighted vector.

[0182] Optionally, when the second acquisition module 702 inputs the text vector to be identified into the sensitive word identification branch of the three-branch attention model to obtain the first weighted vector and the sensitive word identification result, it is specifically used for:

[0183] The text vector to be identified is input into the sensitive word identification branch of the three-branch attention model, and the text vector to be identified is segmented to obtain multiple word vectors;

[0184] Each word vector is subjected to word embedding processing to obtain the word embedding matrix corresponding to the word vector;

[0185] The word embedding matrix is ​​sequentially rearranged, decomposed, and convolved to obtain a first feature matrix, and the first attention weight of the first feature matrix is ​​calculated.

[0186] Calculate the first weighted vector based on the first feature matrix and the first attention weight;

[0187] The sensitive word recognition result of the multiple word vectors is determined based on the similarity between the first weighted vector of each word vector and the weighted vector of the preset sensitive words.

[0188] Optionally, when the second acquisition module 702 inputs the text vector to be recognized into the entity word recognition branch of the three-branch attention model to obtain the second weighted vector and the entity word recognition result, it is specifically used for:

[0189] The text vector to be identified is input into the entity word recognition branch of the three-branch attention model, and the text vector to be identified is segmented to obtain multiple word vectors;

[0190] Feature extraction is performed on each word vector to obtain the word vector feature matrix;

[0191] The word vector feature matrix is ​​sequentially rearranged, decomposed, and convolved to obtain a second feature matrix, and the second attention weight of the second feature matrix is ​​calculated.

[0192] Calculate the second weighted vector based on the second feature matrix and the second attention weight;

[0193] The second weighted vector of each word vector is input into the conditional random field model to obtain the entity word recognition result after the entity word annotation of the multiple word vectors.

[0194] Optionally, when the second acquisition module 702 inputs the text vector to be identified into the privacy sentence recognition branch of the three-branch attention model to obtain the third weighted vector, it is specifically used for:

[0195] The text vector to be identified is input into the privacy sentence recognition branch of the three-branch attention model, and the text vector to be identified is segmented into sentences to obtain multiple sentence vectors.

[0196] Feature extraction is performed on each sentence vector to obtain the sentence vector feature matrix;

[0197] The sentence vector feature matrix is ​​sequentially rearranged, decomposed, and convolved to obtain a third feature matrix, and the third attention weight of the third feature matrix is ​​calculated.

[0198] Calculate the third weighted vector based on the third attention weight and hidden state corresponding to each sentence vector.

[0199] Optionally, the first determining module 703 is specifically used for:

[0200] The sensitive word vectors in the sensitive word identification results are transformed into sensitive word feature vectors;

[0201] Calculate the first similarity between the sensitive word feature vector and the feature vector of each preset sensitive word type;

[0202] The sensitive word type of the feature vector corresponding to the highest first similarity is determined as the first sensitive word type corresponding to the sensitive word vector.

[0203] Optionally, the second determining module 704 is specifically used for:

[0204] The entity word vectors in the entity word recognition results are transformed into entity word feature vectors.

[0205] Calculate the second similarity between the entity word feature vector and the feature vector of each preset sensitive word type;

[0206] If the maximum second similarity is within a preset similarity range, the fourth weighted vector is calculated based on the first weighted vector, the second weighted vector, the third weighted vector, and the fourth attention weight.

[0207] The fourth weighted vector is input into the distillation model to identify sensitive word types, thereby obtaining the second sensitive word type corresponding to the entity word vector.

[0208] It should be noted that the sensitive word type determination device provided in this application embodiment can implement all the method steps implemented in the above sensitive word type determination method embodiment and can achieve the same technical effect. Here, the parts that are the same as those in the method embodiment and the beneficial effects will not be described in detail.

[0209] It should be noted that the division of units in the embodiments of this application is illustrative and only represents one logical functional division. In actual implementation, other division methods may be used. Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated units described above can be implemented in hardware or as software functional units.

[0210] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a processor-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0211] like Figure 8 As shown, embodiments of this application also provide an electronic device, including a memory 820, a transceiver 810, and a processor 800:

[0212] Memory 820 is used to store computer programs;

[0213] Transceiver 810 is used to send and receive data under the control of the processor;

[0214] Processor 800 is configured to read a computer program from memory and execute the steps of the method for determining the type of sensitive words as described in any of the above embodiments.

[0215] Among them, Figure 8 In this context, the bus architecture can include any number of interconnected buses and bridges, specifically linking various circuits together, represented by one or more processors (processor 800) and memory (memory 820). The bus architecture can also link various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. The bus interface provides an interface. The transceiver 810 can be multiple elements, including transmitters and receivers, providing a unit for communicating with various other devices over transmission media, including wireless channels, wired channels, optical fibers, etc. The processor 800 is responsible for managing the bus architecture and general processing, and the memory 820 can store data used by the processor 800 during operation.

[0216] The processor 800 can be a central processing unit (CPU), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or a complex programmable logic device (CPLD). The processor can also adopt a multi-core architecture.

[0217] The processor executes the sensitive word type determination method provided in this application embodiment by calling a computer program stored in memory, according to the obtained executable instructions. The processor and memory can also be physically separated.

[0218] It should be noted that the electronic device provided in this application embodiment can implement all the method steps implemented in the above-mentioned sensitive word type determination method embodiment, and can achieve the same technical effect. Here, the parts that are the same as those in the method embodiment and the beneficial effects will not be described in detail.

[0219] Embodiments of this application also provide a processor-readable storage medium storing a computer program for causing the processor to execute the aforementioned method for determining sensitive word types.

[0220] The processor-readable storage medium can be any available medium or data storage device that the processor can access, including but not limited to magnetic memory (e.g., floppy disk, hard disk, magnetic tape, magneto-optical disk (MO)), optical memory (e.g., CD, DVD, BD, HVD), and semiconductor memory (e.g., ROM, EPROM, EEPROM, non-volatile memory (NAND FLASH), solid-state drive (SSD)).

[0221] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0222] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-executable instructions. These computer-executable instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0223] These processor-executable instructions may also be stored in a processor-readable memory that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the processor-readable memory produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0224] These processors can execute instructions that can also be loaded onto a computer or other programmable data processing device, causing a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable device for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0225] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for determining a sensitive word type, characterized in that, The method comprises: obtaining a to-be-recognized text vector; inputting the to-be-recognized text vector into a three-branch attention model to obtain a sensitive word recognition result and an entity word recognition result, and to obtain a weighting vector of each branch, comprising: inputting the to-be-recognized text vector into a sensitive word recognition branch of the three-branch attention model to obtain a first weighting vector and the sensitive word recognition result; inputting the to-be-recognized text vector into an entity word recognition branch of the three-branch attention model to obtain a second weighting vector and the entity word recognition result; and inputting the to-be-recognized text vector into a private sentence recognition branch of the three-branch attention model to obtain a third weighting vector; determining a first sensitive word type corresponding to a sensitive word vector in the sensitive word recognition result from a preset sensitive word type according to a similarity between the sensitive word vector and a feature vector of the preset sensitive word type; performing sensitive word type recognition on the entity word vector according to the entity word vector in the entity word recognition result, the feature vector of the preset sensitive word type, and the weighting vector of each branch to obtain a second sensitive word type corresponding to the entity word vector, comprising: performing vector conversion on the entity word vector in the entity word recognition result to obtain an entity word feature vector; calculating a second similarity between the entity word feature vector and a feature vector of each preset sensitive word type; in a case where a maximum second similarity is within a preset similarity range, calculating a fourth weighting vector according to the first weighting vector, the second weighting vector, the third weighting vector, and a fourth attention weight; and inputting the fourth weighting vector into a distillation model to perform sensitive word type recognition to obtain the second sensitive word type corresponding to the entity word vector.

2. The method of claim 1, wherein, The inputting the to-be-recognized text vector into the sensitive word recognition branch of the three-branch attention model to obtain the first weighting vector and the sensitive word recognition result comprises: inputting the to-be-recognized text vector into the sensitive word recognition branch of the three-branch attention model to perform word segmentation processing on the to-be-recognized text vector to obtain a plurality of word vectors; performing word embedding processing on each word vector to obtain a word embedding matrix corresponding to the word vector; sequentially performing rearrangement operation, feature decomposition, and convolution operation on the word embedding matrix to obtain a first feature matrix, and calculating a first attention weight of the first feature matrix; calculating a first weighting vector according to the first feature matrix and the first attention weight; determining a sensitive word recognition result of the plurality of word vectors according to a similarity between the first weighting vector of each word vector and a weighting vector of a preset sensitive word.

3. The method of claim 1, wherein, The inputting the to-be-recognized text vector into the entity word recognition branch of the three-branch attention model to obtain the second weighting vector and the entity word recognition result comprises: inputting the to-be-recognized text vector into the entity word recognition branch of the three-branch attention model to perform word segmentation processing on the to-be-recognized text vector to obtain a plurality of word vectors; performing feature extraction on each word vector to obtain a word vector feature matrix; The word vector feature matrix is sequentially subjected to rearrangement operation, feature decomposition and convolution operation to obtain a second feature matrix, and a second attention weight of the second feature matrix is calculated; A second weighted vector is calculated according to the second feature matrix and the second attention weight; The second weighted vector of each word vector is input into a conditional random field model to obtain an entity word recognition result of the entity word after entity word labeling of the plurality of word vectors.

4. The method of claim 1, wherein, The third weighted vector is obtained by inputting the to-be-recognized text vector into a privacy sentence recognition branch of the three-branch attention model, comprising: The to-be-recognized text vector is input into the privacy sentence recognition branch of the three-branch attention model to perform sentence processing on the to-be-recognized text vector to obtain a plurality of sentence vectors; Feature extraction is performed on each sentence vector to obtain a sentence vector feature matrix; The sentence vector feature matrix is sequentially subjected to rearrangement operation, feature decomposition and convolution operation to obtain a third feature matrix, and a third attention weight of the third feature matrix is calculated; The third weighted vector is calculated according to the third attention weight and the hidden state corresponding to each sentence vector.

5. The method of claim 1, wherein, The first sensitive word type corresponding to the sensitive word vector is determined from the preset sensitive word types according to the similarity between the sensitive word vector in the sensitive word recognition result and the feature vector of the preset sensitive word type, comprising: The sensitive word vector in the sensitive word recognition result is converted into a sensitive word feature vector; A first similarity between the sensitive word feature vector and the feature vector of each preset sensitive word type is calculated; The sensitive word type of the feature vector of the preset sensitive word type corresponding to the maximum first similarity is determined as the first sensitive word type corresponding to the sensitive word vector.

6. A device for determining a sensitive word type, characterized in that The apparatus comprises: A first obtaining module for obtaining a to-be-recognized text vector; A second obtaining module for inputting the to-be-recognized text vector into a three-branch attention model to obtain a sensitive word recognition result and an entity word recognition result, and obtaining a weighted vector of each branch, comprising: The to-be-recognized text vector is input into a sensitive word recognition branch of the three-branch attention model to obtain a first weighted vector and a sensitive word recognition result; the to-be-recognized text vector is input into an entity word recognition branch of the three-branch attention model to obtain a second weighted vector and an entity word recognition result; and the to-be-recognized text vector is input into a privacy sentence recognition branch of the three-branch attention model to obtain a third weighted vector; A first determining module for determining a first sensitive word type corresponding to a sensitive word vector from preset sensitive word types according to a similarity between the sensitive word vector in the sensitive word recognition result and a feature vector of the preset sensitive word type; A second determining module for performing sensitive word type identification on an entity word vector according to the entity word vector in the entity word recognition result, the feature vector of the preset sensitive word type and the weighted vector of each branch to obtain a second sensitive word type corresponding to the entity word vector, comprising: The entity word vector in the entity word recognition result is vector converted to obtain an entity word feature vector; a second similarity of the entity word feature vector and a feature vector of each preset sensitive word type is calculated; in a case where the maximum second similarity is within a preset similarity range, a fourth weighting vector is calculated according to the first weighting vector, the second weighting vector, the third weighting vector, and a fourth attention weight; the fourth weighting vector is input to a distillation model for sensitive word type recognition to obtain a second sensitive word type corresponding to the entity word vector.

7. An electronic device, comprising: The device comprises a memory, a transceiver, and a processor: The memory is configured to store a computer program. The transceiver is configured to transceive data under the control of the processor. The processor is configured to read the computer program in the memory and perform the sensitive word type determination method according to any one of claims 1 to 5.

8. A processor-readable storage medium, comprising: The processor readable storage medium stores a computer program for causing the processor to perform the sensitive word type determination method according to any one of claims 1 to 5.

Images