Data desensitization method, system and equipment based on privacy protection and storage medium

Abstract

The invention discloses a data desensitization method, system and device based on privacy protection and a storage medium, and the method comprises the steps: obtaining to-be-desensitized data, and carrying out the preprocessing of the to-be-desensitized data, and obtaining the preprocessed to-be-desensitized data; inputting the preprocessed data to be desensitized into a pre-trained recognition model for sensitive data recognition to obtain sensitive data; judging the category of the sensitive data, and generating an optimal desensitization strategy based on the category of the sensitive data; and based on the generated optimal desensitization strategy, desensitizing the sensitive information to obtain desensitized data. According to the method, the desensitization strategy is flexibly selected or adjusted according to different categories of the sensitive data, so that the privacy is protected, the availability of the data is reserved to the greatest extent, and diversified data processing requirements are met.

Description

Technical Field

[0001] The present application relates to the field of data security technology, and in particular to a data desensitization method, system, device and storage medium based on privacy protection. Background Art

[0002] In the digital age, data has become the core resource for business operations and decision-making. However, the widespread use of data is also accompanied by unprecedented challenges in privacy protection. A large amount of sensitive information contained in the data, such as personal identity information, financial records, health data, etc., if directly stored or transmitted without proper processing, ignoring the effective protection of sensitive information, is very likely to cause privacy leakage risks, resulting in security risks in the process of data sharing, analysis or storage; therefore, data desensitization technology came into being, aiming to protect the privacy rights and interests of data subjects by processing sensitive data.

[0003] Most existing data desensitization technologies rely on a single set of desensitization rules, which are difficult to adapt to complex and changing data environments and increasingly stringent requirements. Different sensitive data types (such as ID card numbers, mobile phone numbers, email addresses, etc.) often require different desensitization strategies due to their unique sensitivity and usage requirements. In order to protect privacy while maximizing the availability of data, in order to achieve the best balance between desensitization effect and data availability. Therefore, how to dynamically adjust the desensitization strategy to ensure the best balance between desensitization effect and data availability is an urgent problem to be solved. Summary of the invention

[0004] The main purpose of this application is to overcome the shortcomings and deficiencies of the prior art and to provide a data desensitization method, system, device and storage medium based on privacy protection. By flexibly selecting or adjusting the desensitization strategy according to different categories of sensitive data, it ensures that while protecting privacy, the availability of data is retained to the maximum extent possible to meet diverse data processing needs.

[0005] In order to achieve the above objectives, this application adopts the following technical solutions:

[0006] In a first aspect, the present application provides a data desensitization method based on privacy protection, comprising the following steps:

[0007] Obtaining the data to be desensitized, and preprocessing the data to be desensitized to obtain the preprocessed data to be desensitized;

[0008] Inputting the pre-processed data to be desensitized into a pre-trained recognition model to perform sensitive data recognition to obtain sensitive data;

[0009] Determine the category to which the sensitive data belongs, and generate an optimal desensitization strategy based on the category to which the sensitive data belongs;

[0010] Based on the optimal desensitization strategy generated, the sensitive information is desensitized to obtain desensitized data.

[0011] As a preferred technical solution, the obtaining of the data to be desensitized includes obtaining user data, transaction data and communication data.

[0012] As a preferred technical solution, before inputting the sensitive data into a pre-trained recognition model for sensitive data recognition, the method further includes:

[0013] Segmenting the preprocessed data to be desensitized to obtain a sequence of words or phrases;

[0014] Constructing the words or phrases into a vocabulary; wherein each word or phrase in the vocabulary is assigned an index;

[0015] The words or phrases in the vocabulary are mapped into vectors of fixed dimension to form a numerical matrix of the text sequence.

[0016] As a preferred technical solution, the pre-processed data to be desensitized is input into a pre-trained recognition model to identify sensitive data to obtain sensitive data, including:

[0017] Searching for a text vector corresponding to the word or phrase by the index of the word or phrase in the vocabulary;

[0018] Arrange the text vectors of all words or phrases in order to form a two-dimensional text matrix;

[0019] Extracting local features of the text vector to generate a first feature map;

[0020] Extract important features from the first feature map to generate a second feature map;

[0021] Integrate the first feature map and the second feature map to obtain a global feature;

[0022] Converting the global features into a one-dimensional feature vector, and performing a nonlinear transformation on the one-dimensional feature vector to obtain high-level features;

[0023] The high-level features are passed through a softmax function to obtain the class probabilities of output sensitive data and non-sensitive data;

[0024] If the probability of sensitive data is greater than the preset threshold, it is determined to be sensitive data.

[0025] As a preferred technical solution, the categories to which the sensitive data belongs include personal identity information, financial data, geographic location information and network identification information.

[0026] As a preferred technical solution, the determining the category to which the sensitive data belongs includes:

[0027] Analyze the data of each sensitive data category to obtain the characteristics of each sensitive data category;

[0028] According to the characteristics of each sensitive data category, define a regular expression for each sensitive data category;

[0029] Traversing the sensitive data, and matching the sensitive data with a regular expression of the sensitive data category;

[0030] If the sensitive data successfully matches the regular expression of a sensitive data category, the sensitive data is classified into the corresponding sensitive data category.

[0031] As a preferred technical solution, the desensitization strategy includes but is not limited to: character replacement, mask processing, fuzzy processing and data encryption.

[0032] In a second aspect, the present application provides a data desensitization system based on privacy protection, which is applied to the data desensitization method based on privacy protection, including a preprocessing module, a sensitive data identification module, a desensitization strategy generation module and a desensitization module;

[0033] The preprocessing module is used to obtain the data to be desensitized, and preprocess the data to be desensitized to obtain the preprocessed data to be desensitized;

[0034] The sensitive data identification module is used to input the pre-processed data to be desensitized into a pre-trained recognition model to perform sensitive data identification to obtain sensitive data;

[0035] The desensitization strategy generation module is used to determine the category to which the sensitive data belongs, and generate an optimal desensitization strategy based on the category to which the sensitive data belongs;

[0036] The desensitizing module is used to desensitize sensitive information based on the generated optimal desensitizing strategy to obtain desensitized data.

[0037] As a preferred technical solution, the obtaining of the data to be desensitized includes obtaining user data, transaction data and communication data.

[0038] As a preferred technical solution, it also includes a sensitive data preprocessing module, and the sensitive data preprocessing module is used to:

[0039] Segmenting the preprocessed data to be desensitized to obtain a sequence of words or phrases;

[0040] Constructing the words or phrases into a vocabulary; wherein each word or phrase in the vocabulary is assigned an index;

[0041] The words or phrases in the vocabulary are mapped into vectors of fixed dimension to form a numerical matrix of the text sequence.

[0042] As a preferred technical solution, the module for identifying sensitive data is specifically used for:

[0043] Searching for a text vector corresponding to the word or phrase by the index of the word or phrase in the vocabulary;

[0044] Arrange the text vectors of all words or phrases in order to form a two-dimensional text matrix;

[0045] Extracting local features of the text vector to generate a first feature map;

[0046] Extract important features from the first feature map to generate a second feature map;

[0047] Integrate the first feature map and the second feature map to obtain a global feature;

[0048] Converting the global features into a one-dimensional feature vector, and performing a nonlinear transformation on the one-dimensional feature vector to obtain high-level features;

[0049] The high-level features are passed through a softmax function to obtain the class probabilities of output sensitive data and non-sensitive data;

[0050] If the probability of sensitive data is greater than the preset threshold, it is determined to be sensitive data.

[0051] As a preferred technical solution, the desensitization strategy generation module is also specifically used to: analyze the data of each sensitive data category to obtain the characteristics of each sensitive data category;

[0052] According to the characteristics of each sensitive data category, define a regular expression for each sensitive data category;

[0053] Traversing the sensitive data, and matching the sensitive data with a regular expression of the sensitive data category;

[0054] If the sensitive data successfully matches the regular expression of a sensitive data category, the sensitive data is classified into the corresponding sensitive data category.

[0055] The categories to which the sensitive data belongs include personal identity information, financial data, geographic location information and network identification information.

[0056] The desensitization strategy includes but is not limited to: character replacement, mask processing, fuzzy processing and data encryption.

[0057] In a third aspect, the present application provides an electronic device, the electronic device comprising:

[0058] at least one processor; and,

[0059] a memory communicatively connected to the at least one processor; wherein,

[0060] The memory stores computer program instructions that can be executed by the at least one processor, and the computer program instructions are executed by the at least one processor to enable the at least one processor to perform the privacy protection-based data desensitization method.

[0061] In a fourth aspect, the present application provides a computer-readable storage medium storing a program, which, when executed by a processor, implements the data desensitization method based on privacy protection.

[0062] In summary, compared with the prior art, the effective effects brought about by the technical solution provided by this application include at least:

[0063] The present application proposes a data desensitization method based on privacy protection, which obtains the data to be desensitized, and pre-processes the data to be desensitized to obtain the pre-processed data to be desensitized; the pre-processed data to be desensitized is input into a pre-trained recognition model to identify sensitive data to obtain sensitive data; the category to which the sensitive data belongs is determined, and based on the category to which the sensitive data belongs, an optimal desensitization strategy is generated; based on the generation of the optimal desensitization strategy, the sensitive information is desensitized to obtain the desensitized data. Compared with the traditional method that relies on a single desensitization rule, the present application uses a pre-trained recognition model for sensitive data identification, which can more accurately identify sensitive data, and determine the category to which the identified sensitive data belongs, and automatically and flexibly select or adjust the desensitization strategy according to the category to which it belongs, ensuring that while protecting privacy, the availability of data is retained to the maximum extent, and better balancing the relationship between data privacy protection and data availability. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0065] Figure 1 A flowchart of a data desensitization method based on privacy protection provided in one embodiment of the present application;

[0066] Figure 2 A flowchart of preprocessing sensitive data provided by an embodiment of the present application;

[0067] Figure 3 A sensitive data identification flow chart provided for one embodiment of the present application;

[0068] Figure 4 A flow chart for determining the category to which sensitive data belongs provided in one embodiment of the present application;

[0069] Figure 5 A block diagram of a data desensitization system based on privacy protection provided in one embodiment of the present application;

[0070] Figure 6 A structural diagram of an electronic device provided for one embodiment of the present application. DETAILED DESCRIPTION

[0071] In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present application.

[0072] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments.

[0073] In the current digital age, data is a core asset, and its security and compliance management are becoming increasingly important. Existing data desensitization technologies are generally limited to a single preset desensitization rule. Although this static and fixed processing method can protect sensitive information to a certain extent, it seems to be powerless in the face of complex and changing data environments and increasingly stringent compliance requirements. With the continuous enrichment and refinement of data types, such as sensitive data such as ID card numbers, mobile phone numbers, and email addresses, they each carry different privacy risks and usage values; for example, ID card numbers are the only identifier of personal identity, and strong encryption or replacement strategies need to be adopted. Its desensitization processing must not only prevent direct leakage, but also facilitate identity verification when necessary; mobile phone numbers and email addresses are often used as communication tools. When desensitizing, it is necessary to protect privacy and consider retaining a certain format for data analysis or communication, so that users can be re-identified or contacted when necessary, while avoiding malicious use. Therefore, when identifying different sensitive data, the use of a fixed single desensitization strategy can no longer meet actual needs, and it is impossible to achieve the best balance between desensitization effect and data availability.

[0074] In order to solve the above technical problems, when sensitive data is identified, the present application further determines the category to which the sensitive data belongs, and generates a desensitization strategy in a targeted manner according to the specific category to which the sensitive data belongs, thereby reducing the risk of data leakage. While ensuring that privacy is fully protected, the availability and analysis value of the data are retained to the maximum extent, achieving the optimal balance between desensitization effect and data availability, and meeting diverse data processing needs.

[0075] Desensitization in this application is a data processing technology designed to protect sensitive information from unauthorized access or disclosure. During data sharing, storage, processing or transmission, desensitization technology modifies or hides sensitive parts of the data to ensure that such sensitive information will not be improperly used or disclosed to unauthorized individuals or organizations.

[0076] The technical solutions provided by the embodiments of this specification are described in detail below in conjunction with the accompanying drawings.

[0077] See also Figure 1 In one embodiment of the present application, a data desensitization method based on privacy protection is provided, comprising the following steps:

[0078] S1. Obtain the data to be desensitized, and preprocess the data to be desensitized to obtain the preprocessed data to be desensitized.

[0079] In this embodiment, Kafka can be used as a message queue system to collect and obtain the data to be desensitized in real time. Kafka is a distributed message system based on a publish / subscribe model that can process high-throughput data streams to effectively meet large-scale data processing.

[0080] Furthermore, obtaining the data to be desensitized includes but is not limited to obtaining user data, transaction data, and communication data.

[0081] Among them, user data includes: user's personal information, such as account information, name, age, gender, ID number, telephone number, email address, etc.

[0082] Transaction data includes: bank card number, payment account number, transaction amount, transaction time, transaction counterparty, etc.

[0083] Communication data includes: email content, text messages, instant messages, call logs, voicemails, etc.

[0084] The preprocessing refers to cleaning and standardizing the acquired data to be desensitized to remove duplicate and invalid data and ensure data quality.

[0085] Specifically, data cleaning is to comprehensively scan the acquired data to be desensitized, identify and remove duplicate data and erroneous data, and fill in missing values ​​to improve the accuracy and completeness of the data.

[0086] Data from different sources and in different formats need to be standardized. Standardized data includes data format conversion (such as standardization of date and time formats), data type specification (such as ensuring the accuracy and range of numerical data), and data coding standardization (such as using a unified coding standard to encode address, name and other information). Through standardization, the heterogeneity between data can be eliminated, which facilitates subsequent data analysis and desensitization processing.

[0087] S2. Input the pre-processed data to be desensitized into a pre-trained recognition model to perform sensitive data recognition to obtain sensitive data.

[0088] Furthermore, before the sensitive data is input into the pre-trained recognition model for sensitive data recognition, the process also includes: pre-processing the sensitive data, see Figure 2 , the steps include:

[0089] S21.1 segment the preprocessed data to be desensitized to obtain a sequence of words or phrases.

[0090] Furthermore, word segmentation aims to divide continuous text into a sequence of words or phrases with practical meaning or grammatical function. For English text, spaces are used as separators to preliminarily divide the sequence of English words or phrases. For numbers, regular expressions can be used to match the number sequence in the text. For the matched number sequence, different number types have different format requirements. The format verification is performed according to actual needs. For example: the format of a mobile phone number is usually 11 digits; the format of an ID card number is 18 digits; an email address is not a pure number sequence and is often mixed with numbers. The format of an email address is usually username@domain name, where both the username and the domain name can contain numbers; in addition, the identified number sequence needs to be marked as an independent unit to maintain the integrity of the number sequence. For Chinese text, a dictionary-based word segmentation algorithm can be used for word segmentation. When using a dictionary-based algorithm to segment Chinese text, it can be implemented according to different matching directions, mainly including forward maximum matching, reverse maximum matching and bidirectional maximum matching. The steps include:

[0091] First, determine the word segmentation granularity according to needs, whether it is coarse-grained word segmentation (such as sentence-based) or fine-grained word segmentation (such as word-based);

[0092] Based on the determined word segmentation granularity, set the maximum word length variable according to the length of the longest word in the dictionary;

[0093] Use the forward maximum matching method: scan the text to be segmented from left to right, and take the length of the maximum word length as the matching field each time;

[0094] Match the matching field with the entries in the dictionary. If the match is successful, the matching field is segmented as a word and the text pointer is updated (moved to the end of the word). If the match is unsuccessful, the length of the matching field is shortened (minus one character) and the matching process is repeated until a matching entry is found or the length of the matching field is 1.

[0095] If the last remaining characters cannot form an entry in the dictionary, they are considered as a separate word (usually a character). Repeat the above steps for the remaining text until the entire text has been scanned.

[0096] When using the reverse maximum matching method: scan the text to be segmented from right to left, and take the length of the maximum word length as the matching field each time;

[0097] Match the matching field with the entry in the dictionary. If the match is successful, segment the matching field as a word and update the text pointer (move the text pointer forward to the starting position of the word).

[0098] If the match is unsuccessful, the length of the matching field is shortened (minus one character) and the matching process is repeated until a matching term is found or the matching field length is 1;

[0099] If the last remaining characters cannot form an entry in the dictionary, they are considered as a separate word (usually a character). Repeat the above steps for the remaining text until the entire text has been scanned.

[0100] The bidirectional maximum matching method is to compare the word frequencies of each word in the word segmentation result obtained by the forward maximum matching method and the word segmentation result obtained by the reverse maximum matching method, and select the word segmentation result with a higher total word frequency.

[0101] S21.2. Construct the words or phrases into a vocabulary; wherein each word or phrase in the vocabulary is assigned an index.

[0102] Furthermore, when building the vocabulary, first traverse the word or phrase sequence after word segmentation, add each word or phrase to the vocabulary, and assign it an index value. The index value is assigned as an integer that increases from 0, ensuring that each word or phrase can be uniquely identified and accessed in the vocabulary through its index value.

[0103] S21.3. Map the words or phrases in the vocabulary into vectors of fixed dimension to form a numerical matrix of the text sequence.

[0104] Furthermore, first, an empty numerical matrix is ​​initialized according to the length of the text sequence (i.e., the number of words or phrases) and the dimension of the word vector. The number of rows of the numerical matrix corresponds to the number of words or phrases in the text sequence, and the number of columns corresponds to the dimension of the word vector. Then, each word or phrase in the text sequence is traversed, and its corresponding word vector is copied to the corresponding row of the numerical matrix to ensure that each word or phrase is correctly mapped to its corresponding vector.

[0105] Specifically, the pre-processed data to be desensitized is input into a pre-trained recognition model for sensitive data recognition. The recognition model uses a convolutional neural network, including an embedding layer, a convolution layer, a pooling layer, and a fully connected layer. For specific steps, please refer to Figure 3 ,include:

[0106] S22.1. Search for the text vector corresponding to the word or phrase through the index of the word or phrase in the vocabulary.

[0107] Furthermore, the conversion of words or phrases into text vectors is performed in the embedding layer of the convolutional neural network; first, the embedding layer initializes an embedding matrix according to the size of the vocabulary and the dimension of the embedding vector; wherein each row of the embedding matrix is ​​a text vector embedded in a word or phrase.

[0108] Second, for each word or phrase in the input text, the text vector corresponding to the word or phrase is looked up in the embedding matrix by the index of the word or phrase in the vocabulary.

[0109] S22.2. Arrange the text vectors of all words or phrases in order to form a two-dimensional text matrix.

[0110] Furthermore, the width of the two-dimensional text matrix is ​​the dimension of the text vector, and the height is the number of words or phrases in the text.

[0111] S22.3. Extract local features of the text vector and generate a first feature map.

[0112] Furthermore, the local features of the text vector are extracted in the convolution layer in the convolutional neural network; first, a plurality of convolution kernels are arranged in the convolution layer, each of which has a specific width (smaller than the width of the text matrix) and the same dimension (height) as the text vector; each convolution kernel is slid on the text matrix, and the dot product between the convolution kernel and the corresponding area of ​​each text matrix is ​​calculated to capture each text feature of different length (i.e., local feature), and generate a first feature map.

[0113] In addition, the local features in this embodiment refer to data features related to sensitive information.

[0114] S22.4. Extract important features from the first feature map to generate a second feature map.

[0115] Furthermore, extracting important features from the first feature map is performed in a pooling layer in a convolutional neural network, and reducing the dimension of the first feature map and extracting the most important features through maximum pooling or average pooling; specifically, defining a pooling window on the first feature map, and the size and step size of the pooling window are pre-set; for each pooling window on the first feature map, performing a pooling operation (such as maximum pooling or average pooling), wherein maximum pooling selects the maximum value in the window as output, and average pooling calculates the average value in the window as output; applying the pooling operation to all positions on the first feature map to generate a second feature map of smaller size.

[0116] S22.5. Integrate the first feature map and the second feature map to obtain a global feature.

[0117] S22.6. Convert the global features into a one-dimensional feature vector, and perform nonlinear transformation on the one-dimensional feature vector to obtain high-level features.

[0118] Furthermore, in the fully connected layer, the global features are converted into one-dimensional feature vectors, and then nonlinear transformations are performed through multiple fully connected layers to finally output the category probabilities of sensitive data and non-sensitive data to achieve the classification task.

[0119] The nonlinear transformation is performed through one or more fully connected layers, and the steps include:

[0120] For each feature in the one-dimensional feature vector, each neuron in the fully connected layer performs a weighted sum of each feature and adds a bias term. This process is a linear transformation, that is, the product of the input vector and the weight matrix plus the bias vector.

[0121] The result of the weighted summation is transformed through a nonlinear activation function; since the superposition of linear transformations is still linear, and the recognition model needs to be able to capture nonlinear relationships, the nonlinear activation function introduces nonlinear factors into the recognition model, enabling the recognition model to learn and represent complex feature combinations.

[0122] Through multiple fully connected layers, deeper nonlinear transformations can be performed; that is, the output of each fully connected layer will be used as the input of the next fully connected layer to gradually extract higher-level feature representations.

[0123] By performing nonlinear transformations through multiple fully connected layers, the recognition model can learn and represent complex nonlinear relationships, thereby improving classification accuracy and generalization ability.

[0124] S22.7. The high-level features are passed through a softmax function to obtain output class probabilities of sensitive data and non-sensitive data.

[0125] Furthermore, the last fully connected layer will pass the high-level features through the softmax function, which can convert the input vector into a probability distribution and finally output the category probabilities of sensitive data and non-sensitive data.

[0126] S22.8. If the probability of sensitive data is greater than the preset threshold, it is determined to be sensitive data.

[0127] S3. Determine the category to which the sensitive data belongs, and generate an optimal desensitization strategy based on the category to which the sensitive data belongs.

[0128] Furthermore, the categories to which sensitive data belongs include personally identifiable information, financial data, geographic location information, and network identification information.

[0129] Among them, personal identity information is directly related to the individual's identity, including: name, address, telephone number, email address, date of birth, gender, marital status, etc.

[0130] Financial data refers to an individual’s economic status, including: bank account number, credit card number, income, transfer records, transaction amounts, etc.

[0131] Geographic location information includes: the user's real-time location, frequently visited places, home address, etc.

[0132] Network identification information refers to information used to identify and distinguish different devices, users or network components in a network environment. This information plays a vital role in network communication, data exchange, device identification, etc.; network identification information includes: IP address (short for Internet Protocol address, which refers to the unique digital identifier assigned to each device or network interface on the network), MAC address (short for media access control address, which is a unique identifier used by network devices at the physical layer or data link layer below the network layer), host name, etc. (host name refers to the name of the computer on the network, which is used to identify a specific device in the network).

[0133] Specifically, to determine the category to which the sensitive data belongs, please refer to Figure 4 , the steps include:

[0134] S31. Analyze the data of each sensitive data category to obtain the characteristics of each sensitive data category;

[0135] S32. Define a regular expression for each sensitive data category according to the characteristics of each sensitive data category;

[0136] S33, traversing the sensitive data, and matching the sensitive data with the regular expression of the sensitive data category;

[0137] S34. If the sensitive data successfully matches a regular expression of a sensitive data category, the sensitive data is classified into a corresponding sensitive data category.

[0138] The desensitization strategies in this application include but are not limited to: character replacement, masking, fuzzification and data encryption.

[0139] For example, when the sensitive data belongs to the category of personal identity information, character replacement is used for desensitization. For key numbers in the ID number (such as the date of birth), specific characters (such as asterisks *) are used to replace them, and only a few digits are retained for verification (such as the last four digits); for example, the ID number 123456199001011234 is desensitized to 123456**********1234. For the desensitization of the name, the first letter and some characters of the surname are retained, and the rest are replaced with specific characters. For example, "Zhang San" is desensitized to "Zhang**".

[0140] When the sensitive data is classified as financial data, data encryption is used, where the data encryption uses a symmetric encryption algorithm, including:

[0141] First, a random 256-bit key is generated and converted into a Base64-encoded string; Base64 encoding not only reduces the length of the key in text format, but also allows the key to be securely transmitted in systems that do not support binary data.

[0142] Get the file containing sensitive data, encrypt the file with sensitive data using the specified AES-256-CBC encryption algorithm, and generate an encrypted file.

[0143] Among them, AES-256-CBC combines the 256-bit key length of the Advanced Encryption Standard (AES) and the Cipher Block Chaining (CBC) mode to provide strong encryption protection. In addition, during the encryption process, in addition to the key, a salt value is added. The salt value is a random value that works with the key to increase the randomness and complexity of the encryption process. Even if two identical files are encrypted with the same key, different encryption results will be generated, thereby improving security. After encryption, the original data file is converted into an encrypted file, which cannot be directly read by unauthorized personnel. For example, suppose there is a financial data file containing bank card numbers and transaction amounts that needs to be encrypted. The file records sensitive information such as "Bank card number: 4111111111111111, transaction amount: 12345.67"; after encryption, the bank card number "4111111111111111" and the transaction amount "12345.67" will become a string of encrypted text that cannot be directly recognized, such as the bank card number becomes "41111111111D2k8f", and the transaction amount becomes "1E45gH".

[0144] When sensitive data in an encrypted file needs to be accessed, the same key is used to decrypt the sensitive data, allowing authorized personnel to access and use the data; ensuring the security of financial data during storage and transmission, and preventing the risk of data leakage and unauthorized access.

[0145] When sensitive data belongs to the category of geographic location information, a fuzzy desensitization strategy can be used for processing. For example, the precise longitude and latitude coordinates can be converted into a larger geographic area or grid code, such as blurring the precise location at the city level to the district level; or while retaining the approximate direction of the original location, a small random offset is made to the longitude and latitude, making it impossible to accurately locate the specific location.

[0146] When the sensitive data belongs to the category of network identification information, the masking desensitization strategy can be used for processing; the core idea of ​​masking is to cover or replace the sensitive part of the data. For example, for IP addresses, since they have a clear hierarchical structure and numerical range, they can be processed by partial masking. For example, the last segment or segments of the IP address can be replaced with 0 or a specific placeholder, such as masking "192.168.1.100" to "192.168.1.XXX", which not only retains the information of the network segment, but also hides the specific device address. The MAC address is the physical identification of the device. Therefore, when performing masking, the last few bits of the MAC address can be replaced with specific placeholders, such as "XX" or "00", to hide the specific device identification; for example, the original MAC address: 00-1A-2B-3C-4D-5E, the MAC address after masking (hiding the last two bits): 00-1A-2B-3C-4D-XX.

[0147] In addition, the present application can also obtain the optimal desensitization strategy by adopting the random forest algorithm model, that is, generating the optimal desensitization strategy based on the prediction results of the output of the random forest algorithm model.

[0148] By accurately determining the category of sensitive data (personal identity information, financial data, geographic location information, network identification information, etc.), we can customize and generate the optimal desensitization strategy, significantly improving the flexibility and security of data processing, effectively preventing the risk of leakage of sensitive information, and ensuring the privacy protection of data during sharing, storage or transmission, while retaining the analytical value of the data.

[0149] S4. Based on the generated optimal desensitization strategy, the sensitive information is desensitized to obtain desensitized data.

[0150] After desensitizing, this embodiment also includes securely storing the desensitized data and introducing a multi-level security protection mechanism. The multi-level security protection mechanism includes but is not limited to encryption, access control, and data integrity check to enhance the security protection of the desensitized data.

[0151] For example, set up fine-grained access control policies to ensure that only authorized users can access desensitized data. Fine-grained access control policies can include the following aspects:

[0152] (1) User and group management:

[0153] User: Create an IAM (Identity and Access Management) user for each individual and assign them unique credentials (such as access key, password).

[0154] Group: A group is a collection of users with similar responsibilities or who need to access the same resources. Users with similar responsibilities are organized into groups and permission policies are assigned to the groups. The permission policies define which resources the users in the group can access and which operations they can perform, so as to clearly divide the permission scopes of different user groups.

[0155] (2) IAM policy: IAM policy exists in the form of a document in JSON format, which defines which operations are allowed or denied. The policy can be attached to users and groups. IAM policy includes management policy and inline policy;

[0156] Managed Policies: Provided predefined policies or user-defined policies.

[0157] Inline Policies: Policies that are embedded directly into a single user, group, or role.

[0158] (3) Roles are used to temporarily obtain specific permissions, usually for applications, services, or external users to access AWS (Amazon Web Services) resources. For example, when an application or external user accesses AWS resources through a role, AWS generates a set of temporary security credentials for the entity, including an access key ID, a secret access key, and a session token. These credentials have a limited validity period and contain only the permissions required to perform a specific task.

[0159] (4) Multi-factor authentication: It forces users to provide additional authentication information (such as a one-time password) when logging in to enhance security.

[0160] (5) Conditional statements: Use conditional statements to set more granular access control in the policy. Conditions can include: limiting the time range of access, enabling multi-factor authentication, and / or limiting access from a specific VPC; where VPC refers to virtual private cloud, which is a cloud computing service.

[0161] (6) Access key management: Regularly rotate access keys: Regularly change user access keys to reduce the risk of key leakage; limit usage boundaries to prevent abuse of permissions.

[0162] (7) Resource-based policies: These include object storage service bucket policies and database policies. Object storage service bucket policies are used to store and retrieve any number of data objects. Bucket policies allow control over access to buckets and their contents. For example, you can define who can access objects in a bucket, what access permissions are available (such as read, write, delete, etc.), and whether these permissions are subject to specific conditions (such as time, IP address, etc.). Database policies can control access to instances.

[0163] This embodiment uses the SHA-256 hash algorithm to perform data integrity check, and the steps include:

[0164] First, the original sensitive data (whether it is a file, string or other form of data) needs to be hashed using the SHA-256 hash algorithm to generate a unique 256-bit (32-byte) hash value; the generated SHA-256 hash value is stored or transmitted together with the original data; when the integrity of the data needs to be verified, the receiver or verifier will use the SHA-256 hash algorithm to hash the received data again to generate a new hash value, and then compare this newly generated hash value with the previously stored or transmitted hash value. If the two hash values ​​are the same, it means that the data has not changed during the transmission or storage process and the data is complete. If the two hash values ​​are different, it means that the data may have been tampered with or damaged during the transmission or storage process and the data is incomplete.

[0165] In summary, the present application provides a data desensitization method based on privacy protection. The present application adopts a recognition model to identify sensitive data, and determines the category to which the identified sensitive data belongs, and automatically selects the optimal desensitization strategy according to the category to which it belongs. The desensitization strategy includes replacement, encryption, blurring, etc., which better balances the relationship between data privacy protection and data availability, and at the same time, meets diverse data processing needs. It ensures that while protecting privacy, the availability of data is retained to the maximum extent; in addition, the present application also introduces a multi-level security protection mechanism after desensitization, including but not limited to transmission encryption, storage encryption, access control, etc., to build a security line for the desensitized data, effectively preventing the security risks such as interception and tampering that may be encountered during the transmission and storage of data, and further consolidating the security and integrity of the data.

[0166] It should be noted that, for the sake of convenience of description, the aforementioned method embodiments are all expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited to the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously.

[0167] Based on the same idea as the data desensitization method based on privacy protection in the above-mentioned embodiment, the present application also provides a data desensitization system based on privacy protection, which can be used to execute the above-mentioned data desensitization method based on privacy protection. For ease of explanation, the structural diagram of the embodiment of the data desensitization system based on privacy protection only shows the parts related to the embodiment of the present application. Those skilled in the art can understand that the illustrated structure does not constitute a limitation on the system, and may include more or fewer components than shown in the diagram, or combine certain components, or arrange the components differently.

[0168] See also Figure 5 In another embodiment of the present application, a data desensitization system based on privacy protection is provided, the system comprising a pre-processing module 101, a sensitive data identification module 102, a desensitization strategy generation module 103 and a desensitization module 104;

[0169] The preprocessing module 101 is used to obtain the data to be desensitized, and preprocess the data to be desensitized to obtain the preprocessed data to be desensitized;

[0170] The sensitive data identification module 102 is used to input the pre-processed data to be desensitized into a pre-trained recognition model to perform sensitive data identification to obtain sensitive data;

[0171] The desensitization strategy generation module 103 is used to determine the category to which the sensitive data belongs, and generate an optimal desensitization strategy based on the category to which the sensitive data belongs;

[0172] The desensitizing module 104 is used to desensitize the sensitive information based on the generated optimal desensitizing strategy to obtain desensitized data.

[0173] As a preferred technical solution, the obtaining of the data to be desensitized includes obtaining user data, transaction data and communication data.

[0174] As a preferred technical solution, it also includes a sensitive data preprocessing module, and the sensitive data preprocessing module is used to:

[0175] Segmenting the preprocessed data to be desensitized to obtain a sequence of words or phrases;

[0176] Constructing the words or phrases into a vocabulary; wherein each word or phrase in the vocabulary is assigned an index;

[0177] The words or phrases in the vocabulary are mapped into vectors of fixed dimension to form a numerical matrix of the text sequence.

[0178] As a preferred technical solution, the sensitive data identification module 102 is specifically used for:

[0179] Searching for a text vector corresponding to the word or phrase by the index of the word or phrase in the vocabulary;

[0180] Arrange the text vectors of all words or phrases in order to form a two-dimensional text matrix;

[0181] Extracting local features of the text vector to generate a first feature map;

[0182] Extract important features from the first feature map to generate a second feature map;

[0183] Integrate the first feature map and the second feature map to obtain a global feature;

[0184] Converting the global features into a one-dimensional feature vector, and performing a nonlinear transformation on the one-dimensional feature vector to obtain high-level features;

[0185] The high-level features are passed through a softmax function to obtain the class probabilities of output sensitive data and non-sensitive data;

[0186] If the probability of sensitive data is greater than the preset threshold, it is determined to be sensitive data.

[0187] As a preferred technical solution, the desensitization strategy generation module 103 is also specifically used for:

[0188] Analyze the data of each sensitive data category to obtain the characteristics of each sensitive data category;

[0189] According to the characteristics of each sensitive data category, define a regular expression for each sensitive data category;

[0190] Traversing the sensitive data, and matching the sensitive data with a regular expression of the sensitive data category;

[0191] If the sensitive data successfully matches the regular expression of a sensitive data category, the sensitive data is classified into the corresponding sensitive data category.

[0192] The categories to which the sensitive data belongs include personal identity information, financial data, geographic location information and network identification information.

[0193] The desensitization strategy includes but is not limited to: character replacement, mask processing, fuzzy processing and data encryption.

[0194] It should be noted that the data desensitization system based on privacy protection of the present application corresponds one-to-one to the data desensitization method based on privacy protection of the present application. The technical features and beneficial effects described in the embodiment of the data desensitization method based on privacy protection are applicable to the embodiment of the data desensitization system based on privacy protection. For specific contents, please refer to the description in the embodiment of the method of the present application. It will not be repeated here. This is hereby declared.

[0195] In addition, in the implementation of the data desensitization system based on privacy protection in the above-mentioned embodiment, the logical division of each program module is only an example. In actual applications, the above-mentioned functions can be assigned to different program modules as needed, for example, for the configuration requirements of the corresponding hardware or the convenience of software implementation. That is, the internal structure of the data desensitization system based on privacy protection is divided into different program modules to complete all or part of the functions described above.

[0196] See also Figure 6 In another embodiment, an electronic device for implementing a data desensitization method based on privacy protection is provided. The electronic device may include a processor, a memory, and a bus, and may also include a computer program stored in the memory and executable on the processor, such as a data desensitization program based on privacy protection.

[0197] Exemplarily, in this embodiment, the computer program may be divided into one or more modules, which are stored in the memory and executed by the processor to complete the present application. The one or more module elements may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program in the device.

[0198] The electronic device may be a computing device such as a desktop computer, a notebook, a palm computer, a cloud server, etc. The electronic device may include, but is not limited to, a processor and a memory.

[0199] In some embodiments, the processor may be composed of an integrated circuit, for example, a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center (Control Unit) of the device, and uses various interfaces and lines to connect the various components of the entire device; by running or executing the program or module stored in the memory, and calling the data stored in the memory, the various functions of the electronic device and the processing data are executed.

[0200] The memory can be used to store the computer program and / or module, and the processor realizes various functions of the device by running or executing the computer program and / or module stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required for a function, etc.; in addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0201] Wherein, the memory may be an internal storage unit of an electronic device in some embodiments, such as a mobile hard disk of the electronic device. The memory may also be an external storage device of an electronic device in other embodiments, such as a plug-in mobile hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device. Furthermore, the memory may also include both an internal storage unit of the electronic device and an external storage device. The memory may be used not only to store application software and various types of data installed in the electronic device, such as the code of a data desensitization program based on privacy protection, but also to temporarily store data that has been output or is to be output.

[0202] Figure 6 Only an electronic device with components is shown, and those skilled in the art will understand that Figure 6 The structure shown does not constitute a limitation on the electronic device, and may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.

[0203] The data desensitization program based on privacy protection stored in the memory of the electronic device is a combination of multiple instructions, and when running in the processor, it can achieve:

[0204] Obtaining the data to be desensitized, and preprocessing the data to be desensitized to obtain the preprocessed data to be desensitized;

[0205] Inputting the pre-processed data to be desensitized into a pre-trained recognition model to perform sensitive data recognition to obtain sensitive data;

[0206] Determine the category to which the sensitive data belongs, and generate an optimal desensitization strategy based on the category to which the sensitive data belongs;

[0207] Based on the optimal desensitization strategy generated, the sensitive information is desensitized to obtain desensitized data.

[0208] Accordingly, the present application also provides a computer-readable storage medium, which includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the privacy protection-based data desensitization method as described in any of the above embodiments.

[0209] The computer program includes computer program code, which may be in source code form, object code form, executable file or some intermediate form, etc. The computer readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), flash memory, mobile hard disk, multimedia card, card-type memory (e.g., SD or DX memory, etc.), magnetic memory, magnetic disk, optical disk, etc.

[0210] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0211] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0212] The above embodiments are preferred implementation modes of the present application, but the implementation modes of the present application are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present application should be equivalent replacement methods and are included in the protection scope of the present application.

Claims

1. A data desensitization method based on privacy protection, characterized in that: The steps include: Obtaining the data to be desensitized, and preprocessing the data to be desensitized to obtain the preprocessed data to be desensitized; Inputting the pre-processed data to be desensitized into a pre-trained recognition model to perform sensitive data recognition to obtain sensitive data; Determine the category to which the sensitive data belongs, and generate an optimal desensitization strategy based on the category to which the sensitive data belongs; Based on the optimal desensitization strategy generated, the sensitive information is desensitized to obtain desensitized data.

2. The data desensitization method based on privacy protection according to claim 1 is characterized in that: The obtaining of the data to be desensitized includes obtaining user data, transaction data and communication data.

3. The data desensitization method based on privacy protection according to claim 1 is characterized in that: Before inputting the sensitive data into a pre-trained recognition model for sensitive data recognition, the method further includes: Segmenting the preprocessed data to be desensitized to obtain a sequence of words or phrases; Constructing the words or phrases into a vocabulary; wherein each word or phrase in the vocabulary is assigned an index; The words or phrases in the vocabulary are mapped into vectors of fixed dimension to form a numerical matrix of the text sequence.

4. The data desensitization method based on privacy protection according to claim 3 is characterized in that: The pre-processed data to be desensitized is input into a pre-trained recognition model to identify sensitive data, thereby obtaining sensitive data, including: Searching for a text vector corresponding to the word or phrase by the index of the word or phrase in the vocabulary; Arrange the text vectors of all words or phrases in order to form a two-dimensional text matrix; Extracting local features of the text vector to generate a first feature map; Extract important features from the first feature map to generate a second feature map; Integrate the first feature map and the second feature map to obtain a global feature; Converting the global features into a one-dimensional feature vector, and performing a nonlinear transformation on the one-dimensional feature vector to obtain high-level features; The high-level features are passed through a softmax function to obtain the class probabilities of output sensitive data and non-sensitive data; If the probability of sensitive data is greater than the preset threshold, it is determined to be sensitive data.

5. The data desensitization method based on privacy protection according to claim 1 is characterized in that: The categories to which the sensitive data belongs include personally identifiable information, financial data, geographic location information, and network identification information.

6. According to the data desensitization method based on privacy protection according to claim 1, it is characterized in that: The determining the category to which the sensitive data belongs includes: Analyze the data of each sensitive data category to obtain the characteristics of each sensitive data category; According to the characteristics of each sensitive data category, define a regular expression for each sensitive data category; Traversing the sensitive data, and matching the sensitive data with a regular expression of the sensitive data category; If the sensitive data successfully matches the regular expression of a sensitive data category, the sensitive data is classified into the corresponding sensitive data category.

7. The data desensitization method based on privacy protection according to claim 1 is characterized in that: The desensitization strategy includes but is not limited to: character replacement, mask processing, fuzzy processing and data encryption.

8. The data desensitization system based on privacy protection is characterized by: The data desensitization method based on privacy protection applied to any one of claims 1 to 7 comprises a preprocessing module, a sensitive data identification module, a desensitization strategy generation module and a desensitization module; The preprocessing module is used to obtain the data to be desensitized, and preprocess the data to be desensitized to obtain the preprocessed data to be desensitized; The sensitive data identification module is used to input the pre-processed data to be desensitized into a pre-trained recognition model to perform sensitive data identification to obtain sensitive data; The desensitization strategy generation module is used to determine the category to which the sensitive data belongs, and generate an optimal desensitization strategy based on the category to which the sensitive data belongs; The desensitizing module is used to desensitize sensitive information based on the generated optimal desensitizing strategy to obtain desensitized data.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores computer program instructions that can be executed by the at least one processor, and the computer program instructions are executed by the at least one processor so that the at least one processor can execute the privacy protection-based data desensitization method as described in any one of claims 1-7.

10. A computer-readable storage medium storing a program, characterized in that: When the program is executed by a processor, the data desensitization method based on privacy protection described in any one of claims 1 to 7 is implemented.

Images