Dynamic desensitization method for structured data
Through the combination of TextCNN text classification model and regular expression, the problems of low efficiency and low accuracy of sensitive data recognition in the existing technology are solved, and the automation of structured data and customized user desensitization are realized, and the recognition efficiency and accuracy are improved. and reduces the system code coupling.
Patent Information
- Application Number
- CN202410936669.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-12
- Publication Date
- 2024-12-03
AI Technical Summary
In the prior art, sensitive data recognition cannot be automated, the recognition efficiency is low, the recognition accuracy is low, and the data desensitization method is highly coupled with the system code.
The TextCNN text classification model is used to identify and classify structured data in combination with Sem2Vec method, and the sensitive data identified through regular expression matching is set. The desensitization algorithm is set according to the user's personalized needs, and the result set processing is used for the database middleware to achieve desensitization. Responsive.
It realizes automation of structured data and customized user desensitization, improves the efficiency and accuracy of sensitive data recognition, reduces the time-consuming regular expression matching, and reduces the system code coupling.
Smart Images

Figure CN119066698A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of data desensitization, and in particular relates to a dynamic desensitization method for structured data. Background Art
[0002] With the popularization of modern communication technology and mobile Internet devices, personal sensitive information is increasing. The rapid development of technology has improved people's quality of life and work efficiency, but it has also brought the risk of personal information leakage. Data desensitization technology provides an innovative protection method that can provide personalized protection according to different user needs and provide additional data security.
[0003] Currently, common sensitive data identification products have problems with low recognition efficiency and accuracy, and cannot achieve automatic recognition. Traditional sensitive data identification methods use regular expressions for matching. This method uses a large number of regular expressions to match texts one by one, which is time-consuming and has low recognition efficiency. In addition, many data desensitization methods are processed at the application layer, which is highly coupled with the system code. Summary of the invention
[0004] The purpose of the present invention is to provide a dynamic desensitization method for structured data in view of the above-mentioned deficiencies in the prior art, so as to solve the problems in the prior art that sensitive data recognition cannot be automated, has low recognition efficiency and low recognition accuracy.
[0005] In order to achieve the above object, the technical solution adopted by the present invention is:
[0006] A dynamic desensitization method for structured data, comprising the following steps:
[0007] S1. Collect and obtain user data;
[0008] S2, use the TextCNN text classification model to identify and classify the data, and output the probability of each classification category;
[0009] S3, based on different category probabilities, using regular expressions to match data corresponding to the category probabilities and identifying the category of the corresponding data;
[0010] S4. Set desensitization algorithms for different types of data based on user personalized needs;
[0011] S5. Based on the desensitization algorithm, the identified sensitive data is desensitized by using the result set processing method.
[0012] Furthermore, the TextCNN text classification model in S2 includes:
[0013] The embedding layer uses the Sem2Vec method to convert the input data into word vectors and form a sentence matrix based on multiple word vectors;
[0014] The convolution layer uses multiple convolution kernels of different sizes to extract key information from the sentence matrix, obtain semantic information at different levels of abstraction, and output a variety of data;
[0015] The pooling layer performs pooling on multiple data and uses the torch.cat function to connect the three data together;
[0016] The fully connected layer fully connects the connected data and uses the softmax activation function to normalize the fully connected data to obtain a probability distribution vector, that is, the probability of each classification category.
[0017] Furthermore, the Sem2Vec method is used to generate embedding vectors with semantic constraints, which combine the semantic libraries of multiple dictionaries and synonym tables to generate word vectors.
[0018] Furthermore, in S3, the category probability values are sorted, and a regular expression is used to match the category data corresponding to the top three category probability values to identify the category of the corresponding data.
[0019] Furthermore, S4 includes, according to the personalized needs of users: masking, encrypting or replacing sensitive words;
[0020] The desensitizing algorithms set for different types of data include: summary encryption desensitizing algorithm, symmetric key encryption desensitizing algorithm, national commercial password desensitizing algorithm, shape-preserving encryption desensitizing algorithm or order-preserving encryption desensitizing algorithm.
[0021] Furthermore, the symmetric key encryption desensitization algorithm includes any one of AES, DES, 3DES, RC4, substitution encryption and rearrangement encryption.
[0022] Furthermore, S5 specifically includes:
[0023] When performing data query, the client sends the original data request directly to the database server. When the database server returns the query result, the database proxy intercepts the message returned by the database server, that is, the returned query result, and desensitizes the message according to the algorithm in S2~S4, and then returns the desensitized message to the client.
[0024] The dynamic desensitization method for structured data provided by the present invention has the following beneficial effects:
[0025] 1. The present invention desensitizes structured data and realizes automation and user customization; uses the TextCNN text classification model to identify sensitive data, thereby improving the efficiency and accuracy of sensitive data identification.
[0026] 2. The invention uses the Sem2Vec method in the data preprocessing stage, which is more stable and efficient than the traditional word vector embedding method.
[0027] 3. The present invention combines the use of the TextCNN text classification model and regular expressions. Compared with the use of traditional single regular expressions, it can greatly reduce the time consumption of regular expression matching and reduce computing power. At the same time, it can also improve recognition efficiency and improve the accuracy of sensitive data recognition.
[0028] 4. The data desensitization of the present invention is processed in the form of database middleware, specifically using a result set processing method, which has a low degree of coupling with the system code and is convenient for processing and improvement. BRIEF DESCRIPTION OF THE DRAWINGS
[0029] Figure 1 This is a system architecture diagram of the dynamic desensitization method for structured data of the present invention.
[0030] Figure 2 The present invention is a flow chart of the dynamic desensitization method for structured data.
[0031] Figure 3 This is a diagram showing the composition of the TextCNN text classification model of the present invention.
[0032] Figure 4 This is a hierarchical functional structure diagram of the TextCNN text classification model of the present invention.
[0033] Figure 5 It is the overall structure diagram of AES of the present invention.
[0034] Figure 6 The data flow diagram of the result set desensitization processing of the present invention.
[0035] Figure 7 This is a flow chart for realizing data parsing of the present invention. DETAILED DESCRIPTION
[0036] The specific implementation modes of the present invention are described below so that those skilled in the art can understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific implementation modes. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations utilizing the concept of the present invention are protected.
[0037] Example 1
[0038] This embodiment provides a dynamic desensitization method for structured data. This embodiment combines the use of the TextCNN text classification model and regular expressions to improve the recognition accuracy, and uses database middleware for processing, thus getting rid of the high coupling between the data desensitization method and the system application. Figure 2 , which specifically includes the following steps:
[0039] Step S1, collecting and obtaining user data;
[0040] The target data in this embodiment is structured data.
[0041] Step S2: Use the TextCNN text classification model to identify and classify the data, and output the probability of each classification category;
[0042] This embodiment first performs data preprocessing to convert the data into vectors.
[0043] Specifically, the Sem2Vec method is used for word embedding, which effectively generates embedding vectors with semantic constraints; it uses a semantic library that combines multiple dictionaries and synonyms to generate semantic constraints for word vectors. They are implemented as semantic constraints with different strengths (heavy constraints and light constraints) to limit the original embedding problem. The two constraints can also interact with each other to increase their average information content.
[0044] Then use the TextCNN text classification model to predict the possible sensitive type of the data, refer to Figure 3 ,TextCNN applies convolutional neural network (CNN) to text classification tasks, and uses multiple kernels of different sizes to extract key information in sentences, so as to better capture local correlations.
[0045] The core idea of the convolutional neural network is to capture local features. For text, local features are sliding windows composed of several words, similar to N-gram. The advantage of convolutional neural networks is that they can automatically combine and filter N-gram features to obtain semantic information at different levels of abstraction.
[0046] refer to Figure 4 , TextCNN text classification model, first you need to configure the model parameters, as follows:
[0047] batch_size=512, the batch size is 512, and 512 sets of data are processed at a time;
[0048] pad_size=64, the length of each sentence (short padding and long cutting);
[0049] embed=100, word vector dimension, word vector dimension in the embedding layer;
[0050] filter_sizes=(2,3,5), the size of the convolution kernel, three sizes of convolution kernels are used;
[0051] num_filters=64, number of convolution kernels (number of channels);
[0052] dropout=0.3, parameter setting of dropout layer;
[0053] num_classes = None, the number of categories in the final output, which is assigned at runtime;
[0054] n_vocab = None, vocabulary size, assigned at runtime
[0055] The initialization and functions of each layer in the model are as follows:
[0056] Embedding layer: Use torch.nn.Embedding function, set the embedding layer dictionary size to n_vocab (i.e. vocabulary size), the size of each embedding vector to embed (i.e. word vector dimension), and the padding index size to n_vocab-1; n_vocab-1 means vocabulary size minus one.
[0057] refer to Figure 4 The first layer is the 7 by 5 sentence matrix on the left of the figure. Each row is a word vector with a dimension of 5, which can be compared to the original pixel points in the image. This layer uses the Sem2Vec method to obtain the word vector and then the sentence matrix.
[0058] Convolution layer: Use torch.nn.Conv2d function, set the number of input channels to 1, set the number of channels generated by convolution to num_filters (i.e. the number of convolution kernels), and set the size of the convolution kernel to (k, embed), where k represents the values in filter_sizes (convolution kernel size), and embed is the word vector dimension, because it must be the same size as the embedding vector in the Embedding layer, so that the entire vector of a word can be fully read.
[0059] Then use the torch.nn.functional.relu function to process the convolution layer and add a nonlinear activation function to the convolution layer to effectively prevent overfitting.
[0060] Linear layer: Use torch.nn.Linear function, the size of each input sample is set to num_filters*len(filter_sizes), which is the number of convolution output channels multiplied by the number of convolution kernels of different sizes, and the size of the output sample is set to num_classes, which is the number of categories of the final output.
[0061] Pooling layer MaxPolling: The third layer is a 1-max pooling layer. Sentences of different lengths can be converted into fixed-length content after passing through the pooling layer.
[0062] Full Connection and Softmax: The last layer is a fully connected softmax layer, which outputs the probability of each category.
[0063] Specifically, when performing data classification and identification:
[0064] The input data is a matrix with the dimension [batch_size, sequence_length]. The numbers in the input matrix represent the index (subscript) of a word in the entire vocabulary.
[0065] The input data first passes through the embedding layer to convert each index into a word vector, so the entire data has an additional dimension and becomes [batch_size, sequence_length, embedding_size];
[0066] Then use the unsqueeze(1) function to increase the data dimension to [batch_size, 1, sequence_length, embedding_size].
[0067] Then, the input data is passed through nn.Conv2d(1,64,(2,100)); nn.Conv2d(1,64,(3,100)); nn.Conv2d(1,64,(5,100)), that is, the input data is copied into 3 copies and passed through the convolution layer with kernel size of (2,100), (3,100), (5,100) respectively, and three different data [batch_size,64,height(different here),1] are obtained. Then, the ReLU nonlinear activation function is used to prevent overfitting without changing the dimension. Then, the squeeze(3) function is used to reduce the data dimension by one dimension, and it becomes [batch_size,64,height(different here)*1], which is convenient for subsequent operations.
[0068] Then the three data are pooled through the pooling layer and nn.functional.max_pool1d(input, input.size(2)) respectively. The final data dimensions are consistent, which is [batch_size, 64, 1*1]. Then the squeeze(2) function is used to reduce the data dimension by one, becoming [batch_size, 64*1*1]. Finally, the torch.cat function is used to concatenate the three data together.
[0069] Add a dropout layer with the parameter set to 0.3 to prevent overfitting.
[0070] Finally, through a fully connected layer, after nn.Linear (64*3, number of categories) full connection, finally using the softmax activation function, all numerical vectors are normalized into probability distribution vectors, that is, the probability of each classification category is output.
[0071] Step S3: Based on different category probabilities, a regular expression is used to match the data corresponding to the category probability to identify the category of the corresponding data;
[0072] Specifically, this embodiment sorts the category probability values based on the category probabilities in step S2, takes the top three types of probability values, and uses regular expressions to match the three categories of data to identify the category of the corresponding data.
[0073] For example, the three types with the highest probability are mobile phone number, ID number and email address. Then, the type of this group of data can be identified by using regular matching based on mobile phone number, regular matching based on ID number and regular matching based on email address respectively.
[0074] A regular expression represents a search pattern defined by a sequence of characters. If a pattern is found in one type of text that does not occur in another type of text, a regular expression is a good approach because it can find all occurrences of the pattern and thus detect only that type of text.
[0075] Many text types, such as dates, ID numbers, email addresses, etc., follow a pattern or must be one of several formats that data of that type can appear in. For example, ID numbers always have a strict format, while dates have multiple formats, but it is easy to create regular expressions that take all of these formats into account at the same time, thereby successfully identifying all occurrences of these patterns.
[0076] Table 1 shows the regular expressions for several common text types.
[0077] Table 1 Common regular expressions for sensitive data
[0078]
[0079] Step S4: according to the personalized needs of users, set desensitization algorithms for different types of data;
[0080] refer to Figure 1 This embodiment can be customized by users and adopts a variety of desensitization methods, including masking, encryption and replacement. Therefore, a desensitization strategy module is introduced to allow users to customize the desensitization method, and the system also presets desensitization strategies for users. For example, name-masking, phone-replacement, address-encryption. Prepare for the next step of data desensitization, making it more flexible and customizable.
[0081] The main function of the desensitization strategy module is to implement user-defined desensitization strategies. That is, to set the desensitization method to be used for different types of sensitive data, such as partial masking for mobile phone numbers, full masking for ID numbers, and replacement for names. The system saves the user's customized desensitization strategy. When performing dynamic data desensitization, the desensitization method is selected according to the desensitization strategy set by the user. At the same time, the system has a built-in default desensitization strategy to ensure that the system can perform desensitization processing normally even if the user does not set it.
[0082] The data desensitization module integrates multiple encryption algorithms, including: summary encryption desensitization algorithm, symmetric key encryption desensitization algorithm, national commercial password desensitization algorithm, shape-preserving encryption desensitization algorithm and order-preserving encryption desensitization algorithm.
[0083] The encryption desensitization algorithm based on symmetric encryption is the most widely used in data desensitization technology based on encryption algorithms. Its main application scenario is in internal systems where keys do not need to be exchanged with each other.
[0084] At present, the encryption desensitization algorithms based on symmetric encryption built into the data desensitization module mainly include: AES, DES, 3DES, RC4, as well as substitution encryption (Caesar cipher), rearrangement encryption (fence cipher N type and V type), etc.
[0085] Among them, AES is a block cipher, which means dividing the plaintext into groups, each group is of equal length, and encrypting a group of data each time until the entire plaintext is encrypted. In the AES standard specification, the block length can only be 128 bits, that is, each block is 16 bytes (each byte is 8 bits). The key length can be 128 bits, 192 bits, or 256 bits. The recommended number of encryption rounds is different for different key lengths. 128, 192, and 256-bit keys correspond to 10, 12, and 14 rounds of encryption, respectively.
[0086] The processing unit of AES is byte. The 128-bit input plaintext group P and input key K are divided into 16 bytes, which are recorded as P=P0 P1...P15 and K=K0 K1...K15 respectively. For example, the plaintext group is P=abcdefghijklmnop, where the character a corresponds to P0 and p corresponds to P15. Generally, the plaintext group is described by a square matrix with bytes as the unit, which is called the state matrix. In each round of the algorithm, the content of the state matrix keeps changing, and the final result is output as the ciphertext.
[0087] The overall structure of AES is as follows Figure 5 As shown in the figure, W[0,3] refers to the 128-bit key composed of W[0], W[1], W[2] and W[3] in series. The round functions of the first to ninth rounds of encryption are the same, including four operations: byte substitution, row shift, column mixing and round key addition. The last round of iteration does not perform column mixing. In addition, before the first round of iteration, the plaintext and the original key are first XORed.
[0088] The following introduces the four operation stages of one round of AES, which fully confuse the input bits.
[0089] Byte substitution;
[0090] AES byte substitution is actually a simple table lookup operation. AES defines an S-box and an inverse S-box.
[0091] Row shift operation;
[0092] The row shift is a simple left circular shift operation. When the key length is 128 bits, the 0th row of the state matrix is shifted left by 0 bytes, the 1st row is shifted left by 1 byte, the 2nd row is shifted left by 2 bytes, and the 3rd row is shifted left by 3 bytes.
[0093] The inverse transformation of the row shift is to perform the opposite shift operation on each row in the state matrix. For example, in AES-128, the 0th row of the state matrix is shifted right by 0 bytes, the 1st row is shifted right by 1 byte, the 2nd row is shifted right by 2 bytes, and the 3rd row is shifted right by 3 bytes.
[0094] Column mixing;
[0095] The column mixing transformation is achieved by matrix multiplication. The state matrix after row shifting is multiplied with the fixed matrix to obtain the confused state matrix, such as:
[0096]
[0097] Among them, the multiplication and addition of matrix elements are binary operations defined based on GF(2^8), not multiplication and addition in the usual sense.
[0098] The inverse column-mixing transform can be defined by the matrix multiplication shown below:
[0099]
[0100] Round key plus
[0101] Round key addition is to perform bit-by-bit XOR operation on the 128-bit round key Ki and the data in the state matrix. The round key addition process can be viewed as the result of word-by-word XOR, or as a byte-level or bit-level operation. In other words, it can be viewed as the XOR operation of the 32-bit word composed of S0 S1 S2 S3 and W[4i].
[0102] The inverse operation of round key addition is exactly the same as the forward round key addition operation, because the inverse operation of XOR is itself. Round key addition is very simple, but it can affect every bit in the S array.
[0103] Step S5: Desensitize the identified sensitive data using a result set processing method based on a desensitization algorithm;
[0104] like Figure 6 As shown, when performing data query, the original data request is directly sent to the database server. When the database returns the query result, the database proxy intercepts it, desensitizes it, and then sends it to the system.
[0105] This system adopts a dynamic data desensitization method based on result set processing. The specific implementation method is to parse and rewrite the message returned by the database. The response message can be divided into six categories, and the specific contents are shown in Table 2.
[0106] Table 2 Functional description of command request message command
[0107]
[0108] Among them, the Result Set package is mainly parsed, and the Result Set Package structure is shown in Table 3;
[0109] Table 3 Result Set Package message structure
[0110]
[0111] The Field structure contains information such as the database name, table name, and field name. Multiple Row Data structures also include multiple field values, corresponding to the n-byte parameter part. These values constitute a complete row of data. The implementation process of row data parsing is as follows: Figure 7 As shown in the figure, the row data is parsed, sensitive data information is found and modified, and then it is restored to the row data to complete a desensitization operation.
[0112] Although the specific implementation of the invention is described in detail in conjunction with the drawings, it should not be understood as limiting the scope of protection of this patent. Within the scope described in the claims, various modifications and variations that can be made by those skilled in the art without creative work still fall within the scope of protection of this patent.
Claims
1. A dynamic desensitization method for structured data. It is characterized in that The following steps are involved: S1. Collect and obtain user data; S2, use the TextCNN text classification model to identify and classify the data, and output the probability of each classification category; S3, based on different category probabilities, using regular expressions to match data corresponding to the category probabilities and identifying the category of the corresponding data; S4. Set desensitization algorithms for different types of data based on user personalized needs; S5. Based on the desensitization algorithm, the identified sensitive data is desensitized by using the result set processing method.
2. The dynamic desensitization method for structured data according to claim 1, It is characterized in that The TextCNN text classification model in S2 includes: The embedding layer uses the Sem2Vec method to convert the input data into word vectors and form a sentence matrix based on multiple word vectors; The convolution layer uses multiple convolution kernels of different sizes to extract key information from the sentence matrix, obtain semantic information at different levels of abstraction, and output a variety of data; The pooling layer performs pooling on multiple data and uses the torch.cat function to connect the three data together; The fully connected layer fully connects the connected data and uses the softmax activation function to normalize the fully connected data to obtain a probability distribution vector, that is, the probability of each classification category.
3. The dynamic desensitization method for structured data according to claim 2, Features: In the embedding layer, the Sem2Vec method is used to generate embedding vectors with semantic constraints, which combine the semantic libraries of multiple dictionaries and synonym tables to generate word vectors.
4. The dynamic desensitization method for structured data according to claim 1, It is characterized in that In S3, the category probability values are sorted, and a regular expression is used to match the category data corresponding to the top three category probability values to identify the category of the corresponding data.
5. The dynamic desensitization method for structured data according to claim 1, It is characterized in that The step S4 includes, according to the user's personalized needs: covering, encrypting or replacing sensitive words; The desensitizing algorithms set for different types of data include: summary encryption desensitizing algorithm, symmetric key encryption desensitizing algorithm, national commercial password desensitizing algorithm, shape-preserving encryption desensitizing algorithm or order-preserving encryption desensitizing algorithm.
6. The dynamic desensitization method for structured data according to claim 5, It is characterized in that The symmetric key encryption desensitization algorithm includes any one of AES, DES, 3DES, RC4, substitution encryption and rearrangement encryption.
7. The dynamic desensitization method for structured data according to claim 1, It is characterized in that The S5 specifically includes: When performing data query, the client sends the original data request directly to the database server. When the database server returns the query result, the database proxy intercepts the message returned by the database server, that is, the returned query result, and desensitizes the message according to the algorithm in S2~S4, and then returns the desensitized message to the client.
Citation Information
Cited By
Desensitization SQL script generation method and device and electronic equipment
CN119311720A
Self-adaptive government affair shared data conformal encryption method and system
CN120705910A
