Sensitive data desensitization method

Abstract

The invention relates to a sensitive data desensitization method which comprises the following steps: S1, identifying a database type of a to-be-processed data source, and selecting a connection mode corresponding to the database type to establish connection with the data source; s2, analyzing a data structure of a data table in the to-be-data source, and identifying sensitive data contained in data of different data structures; and S3, aiming at the sensitive data in different data structures, adopting corresponding desensitization strategies, and determining a display mode of the desensitization data during desensitization display. According to the method, the hierarchical design is adopted, the data structure of the data source is analyzed, the sensitive data in different data structures are identified, and the sensitive data of the different data structures are processed according to the set desensitization strategy, so that the desensitization processing efficiency is improved.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and specifically to a method for de-identifying sensitive data. Background Technology

[0002] In the field of data security, data anonymization is a key protection technique.

[0003] Research has found that existing desensitization techniques typically employ static desensitization, pre-setting certain fields as sensitive data and displaying the sensitive data by masking the original information content.

[0004] Existing desensitization methods are not flexible enough. When dealing with a large amount of data from different data sources, using static settings will greatly reduce the efficiency of desensitization. Summary of the Invention

[0005] This invention provides a method for de-identifying sensitive data. When dealing with a large amount of data from different data sources, a layered design is adopted to parse the data structure of the data sources, identify sensitive data in different data structures, and process the sensitive data in different data structures according to the set de-identification strategy, thereby improving the efficiency of de-identification processing.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: Firstly, this application provides a method for de-identifying sensitive data, including: S1, identify the database type of the data source to be processed, and select the connection method corresponding to the database type to establish a connection with the data source; S2, parse the data structure of the data table in the data source and identify the sensitive data contained in the data of different data structures; S3: For sensitive data in different data structures, adopt corresponding desensitization strategies to determine the display method of desensitized data during desensitization display.

[0007] In one implementation, in S1, the database type of the data source includes relational databases, non-relational databases, and file data sources.

[0008] In one implementation, the data structure includes numeric characters and strings; Identify sensitive data contained in data with different data structures, including: using regular expressions to match numeric characters representing identity IDs or account IDs, and matching email accounts represented in strings.

[0009] In one implementation, the desensitization strategy includes encrypting some or all fields in the sensitive data using an encryption algorithm; or replacing some or all fields in the sensitive data with the same or different symbols.

[0010] In one implementation, the method further includes: encapsulating the process of the method into a de-identification task, and performing the de-identification task on multiple data sources at a set frequency.

[0011] In a second aspect, a project progress evaluation system is provided, including a processor and a memory, wherein the memory stores a computer program, and the processor executes the computer program to implement the method described in the first aspect.

[0012] The present invention has the following advantages due to the adoption of the above technical solutions: This method employs a layered design when dealing with a large amount of data from different data sources. It parses the data structure of the data sources, identifies sensitive data in different data structures, and processes the sensitive data in different data structures according to the set de-identification strategy, thereby improving the efficiency of de-identification processing. Attached Figure Description

[0013] Figure 1 This is a sensitive data desensitization method provided in the embodiments of the present invention. Detailed Implementation

[0014] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the described embodiments of the present invention are within the scope of protection of the present invention.

[0015] To address the shortcomings and problems of existing technologies, this application provides a method for de-identifying sensitive data, comprising: S1, identify the database type of the data source to be processed, and select the connection method corresponding to the database type to establish a connection with the data source; S2, parse the data structure of the data table in the data source and identify the sensitive data contained in the data of different data structures; S3: For sensitive data in different data structures, adopt corresponding desensitization strategies to determine the display method of desensitized data during desensitization display.

[0016] Corresponding to the above method, this invention provides a data anonymization system based on a layered architecture, mainly divided into a data source adaptation layer, a data parsing layer, a sensitive data identification layer, an anonymization strategy execution layer, and a data output layer. The data source adaptation layer supports access to various data sources, including relational databases (such as MySQL and Oracle), non-relational databases (such as MongoDB), and file data sources; the data parsing layer is responsible for structural parsing of the databases and tables in the selected data source; the sensitive data identification layer uses a specific algorithm to identify the type of data within the table; the anonymization strategy execution layer calls the corresponding anonymization strategy based on the identification results; and the data output layer feeds back the anonymized data to the designated target location.

[0017] The above system and method are described below in a more detailed embodiment.

[0018] Detailed Implementation Examples The data anonymization system's processing flow includes: 1) Selection of data source, database and data table Employing standard database connectivity technologies such as JDBC (Java Database Connectivity) and ODBC (Open Database Connectivity), combined with API interface calls, the system enables connection and management of various data sources. Users can easily select the target data source through a visual interface, thereby locating the specific database and tables. The system establishes a data connection pool in the background to ensure efficient and stable data access.

[0019] 2) Sensitive data identification Regular expression matching: For structured data such as ID cards and email addresses, predefined regular expression templates are used. For example, the regular expression for matching ID card numbers is (^\d{15}$)|(^\d{18}$)|(^\d{17}(\d|X|x)$), and the regular expression for matching email addresses is ^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}$. The system iterates through the data table fields, performs pattern matching using the regular expression engine, and identifies data that matches the rules as sensitive data.

[0020] 3) Implementation of desensitization strategy Masking for data such as bank card numbers: A masking method is used to display the data, keeping only the first 4 and last 4 digits, and replacing the middle digits with "*". For example, "622202100100010001" will be displayed as "6222********0001" after masking.

[0021] Encryption and desensitization: Sensitive fields are encrypted using AES256 encryption. For example, 622202100100010001 will be displayed as desensitized data. "ghsMjCUn4gqg3J2pHI9HtA==".

[0022] In the several embodiments provided by this invention, it should be understood that the disclosed methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0023] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for de-identifying sensitive data, characterized in that, include: S1, identify the database type of the data source to be processed, and select the connection method corresponding to the database type to establish a connection with the data source; S2, parse the data structure of the data table in the data source and identify the sensitive data contained in the data of different data structures; S3: For sensitive data in different data structures, adopt corresponding desensitization strategies to determine the display method of desensitized data during desensitization display.

2. The sensitive data desensitization method according to claim 1, characterized in that, In S1, the database type of the data source includes relational databases, non-relational databases, and file data sources.

3. The sensitive data desensitization method according to claim 2, characterized in that, The data structure includes numeric characters and strings; Identify sensitive data contained in data with different data structures, including: using regular expressions to match numeric characters representing identity IDs or account IDs, and matching email accounts represented in strings.

4. The sensitive data desensitization method according to claim 3, characterized in that, The desensitization strategy includes using encryption algorithms to encrypt some or all fields in the sensitive data; or replacing some or all fields in the sensitive data with the same or different symbols.

5. The sensitive data desensitization method according to claim 1, characterized in that, The method further includes: encapsulating the process of the method into a de-identification task, and performing the de-identification task on multiple data sources at a set frequency.

6. A project progress evaluation system, characterized in that, It includes a processor and a memory, the memory storing a computer program, the processor executing the computer program to implement the method according to any one of claims 1 to 5.

Images