Data processing method and device, electronic equipment, medium and program product

Abstract

The invention provides a data processing method and device, electronic equipment, a medium and a program product, and the method comprises the steps: obtaining table information composed of at least one data table and data in each data table in a target database, carrying out the sensitive information matching of the table information, determining a sensitive data list with the sensitive information, and storing the sensitive data list in the target database; and generating a desensitization statement based on the sensitive data list, and performing desensitization processing on the target database based on the desensitization statement to obtain a desensitized target database. According to the method, all the sensitive data in the target database can be automatically recognized for desensitization processing, the corresponding desensitization statements are generated, all types of data in the target database can be covered, and then the data desensitization efficiency based on the desensitization statements is improved.

Description

Technical Field

[0001] This application relates to the technical field of data processing, and in particular, to a data processing method, apparatus, electronic device, medium, and program product. Background Art

[0002] In the field of Internet technology, data security is a very important link. Currently, desensitizing data is an important protection measure in the data security link, that is, sensitive data related to user privacy in the data is modified or concealed through desensitization rules or algorithms.

[0003] Related data desensitization methods mainly involve manually designing desensitization scripts by staff and desensitizing sensitive data in the data based on the desensitization scripts.

[0004] However, in the process of data interaction, the amount of data is large and the data types are complex. The method of manually designing desensitization scripts requires a large amount of manpower, and the data types and quantities that the desensitization scripts can handle are small, resulting in low desensitization efficiency. Summary of the Invention

[0005] This application provides a data processing method, apparatus, electronic device, medium, and program product to solve the technical problem of low efficiency of related data desensitization processing methods.

[0006] In a first aspect, this application provides a data processing method, including:

[0007] Obtain the table information of the target database; wherein, the table information includes at least one data table and the data in each data table;

[0008] Perform sensitive information matching on the table information to determine a list of sensitive data with sensitive information;

[0009] Generate a desensitization statement based on the list of sensitive data, and the desensitization statement is used to desensitize the target database to obtain a desensitized target database.

[0010] Optionally, before performing sensitive information matching on the table information, it includes:

[0011] Identify the data types of the data in the table information;

[0012] Determine a target data type among the data types, and determine the data corresponding to the target data type in the table information as non-sensitive data;

[0013] Delete the non-sensitive data to obtain new table information.

[0014] Optionally, the sensitive data list that performs sensitive information matching on the table information to determine the existence of sensitive information includes:

[0015] When the data volume of the table information is less than or equal to the first threshold, select the first target data of the first quantity;

[0016] Based on the regular expression, perform sensitive information matching on each of the first target data to determine the sensitive data list with sensitive information.

[0017] Optionally, the performing sensitive information matching on each of the first target data based on the regular expression to determine the sensitive data list with sensitive information includes:

[0018] Based on the regular expression, perform sensitive information matching on each of the first target data to obtain the first matching result of each of the first target data;

[0019] When the first matching result of the first target data indicates the inclusion of sensitive information, determine the column where the first target data is located in the table information as the first target column;

[0020] Generate the sensitive data list based on each data included in each of the first target columns.

[0021] Optionally, the performing sensitive information matching on the table information to determine the sensitive data list with sensitive information includes:

[0022] When the data volume of the table information is greater than the first threshold, in the order of each of the first target data in the table information, perform sensitive information matching on each of the first target data based on the regular expression in sequence to obtain the second matching result of each of the first target data;

[0023] When the second matching result of the first target data indicates the inclusion of sensitive information, determine the first target data as the second target data;

[0024] When the quantity of the second target data reaches the first quantity, stop executing the steps of performing sensitive information matching on each of the first target data based on the regular expression in sequence in the order of each of the first target data in the table information to obtain the second matching result of each of the first target data and subsequent steps, and determine the columns where each of the second target data is located as the second target columns;

[0025] Determine the sensitive data list based on each data included in the second target columns of the first quantity.

[0026] Optionally, generating a desensitization statement based on the sensitive data list, where the desensitization statement is used to desensitize the target database to obtain a desensitized target database, including:

[0027] Generating a corresponding desensitization statement based on the type of the target database and the desensitization data list; wherein, the desensitization statement is used to desensitize each data in the sensitive data list in the target database to obtain a desensitized target database; the desensitization process includes at least one of encryption processing, replacement processing, and fuzzing processing.

[0028] Optionally, before obtaining the table information of the target database, including:

[0029] Obtaining the configuration information of the target database;

[0030] Based on the configuration information, establishing a connection relationship with the target database.

[0031] In a second aspect, the present application provides a data processing device, including:

[0032] A first acquisition module, configured to acquire table information of a target database; wherein, the table information includes at least one data table and data in each data table;

[0033] A matching module, configured to perform sensitive information matching on the table information to determine a sensitive data list with sensitive information;

[0034] A generation module, configured to generate a desensitization statement based on the sensitive data list, where the desensitization statement is used to desensitize the target database to obtain a desensitized target database.

[0035] In a third aspect, an embodiment of the present application provides an electronic device, including: a processor and a memory communicatively connected to the processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory, so that the processor executes the data processing method described in the first aspect and various possible designs of the first aspect above.

[0036] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored, and when the processor executes the computer-executable instructions, the data processing method described in the first aspect and various possible designs of the first aspect above is implemented.

[0037] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, and when the computer program is executed by a processor, the data processing method described in the first aspect and various possible designs of the first aspect above is implemented.

[0038] A data processing method, apparatus, electronic device, medium, and program product provided by the present application obtain table information in a target database, which consists of at least one data table and the data in each data table, perform sensitive information matching on the table information to determine a list of sensitive data with sensitive information, generate a desensitization statement based on the list of sensitive data, and perform desensitization processing on the target database based on the desensitization statement to obtain a desensitized target database, thereby automatically identifying each sensitive data in the target database and generating a corresponding desensitization statement, which can cover various types of data in the target database, thereby improving the efficiency of data desensitization based on the desensitization statement. Description of the Drawings

[0039] The drawings here are incorporated into the specification and form a part of this specification, showing embodiments consistent with the present application, and are used together with the specification to explain the principles of the present application.

[0040] Figure 1 It is an application scenario diagram of a related data desensitization processing method;

[0041] Figure 2 It is one of the flow diagrams of the data processing method provided by the embodiments of the present application;

[0042] Figure 3 It is another flow diagram of the data processing method provided by the embodiments of the present application;

[0043] Figure 4 It is an application diagram of the data processing method provided by the embodiments of the present application;

[0044] Figure 5 It is the third flow diagram of the data processing method provided by the embodiments of the present application;

[0045] Figure 6 It is the fourth flow diagram of the data processing method provided by the embodiments of the present application;

[0046] Figure 7 It is the structural diagram of the data processing apparatus provided by the embodiments of the present application;

[0047] Figure 8 It is the structural diagram of the electronic device provided by the embodiments of the present application.

[0048] Through the above drawings, specific embodiments of the present application have been shown, and there will be more detailed descriptions hereinafter. These drawings and the written description are not intended to limit the scope of the concept of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. Detailed Embodiments

[0049] Exemplary embodiments will be described in detail herein, and examples thereof are shown in the accompanying drawings. When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0050] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data that have been authorized by the user or fully authorized by all parties. Moreover, the collection, use, and processing of relevant data need to comply with relevant laws, regulations, and standards, and corresponding operation entrances are provided for users to choose to authorize or reject.

[0051] In the field of Internet technology, data security is a very important link. Currently, data desensitization is an important protection measure in the data security link, that is, sensitive data related to user privacy in the data is modified or concealed through desensitization rules or algorithms.

[0052] For related data desensitization methods, mainly through staff manually designing desensitization scripts, and based on the desensitization scripts, desensitize the sensitive data in the data. However, in the process of data interaction, the amount of data is large and the data types are complex. And the way of manually designing desensitization scripts by humans requires a large amount of manpower, and the data types and quantities that can be processed by the manually designed desensitization scripts are small, resulting in a low completion efficiency of the desensitization process.

[0053] See Figure 1 , among the data that can be covered by the desensitization script manually designed by the staff ( Figure 1 represented by a dotted circle in Figure 1 ), the data types are few and the amount of data is small. While the data transmitted by the user in the process of data interaction (

[0054] Figure 1 represented by a solid circle in

[0054] <{0000121}> includes a large amount of data and the data types are complex and rich. This leads to the existence of some un-desensitized data after desensitizing the data to be desensitized through the manually designed desensitization script, resulting in a low completion efficiency of the desensitization process.

[0054] In view of the above problems, the present application proposes a data processing method, apparatus, electronic device, storage medium, and program product. The method includes: obtaining table information in a target database, which consists of at least one data table and the data in each data table; performing sensitive information matching on the table information to determine a list of sensitive data with sensitive information; generating a desensitization statement based on the list of sensitive data, so as to perform desensitization processing on the target database based on the desensitization statement to obtain a desensitized target database, thereby automatically identifying each piece of sensitive data in the target database and generating corresponding desensitization statements, which can cover various types of data in the target database, thus improving the efficiency of data desensitization based on the desensitization statement.

[0055] The following uses specific embodiments to elaborate in detail on the technical solution of the present application and how the technical solution of the present application solves the above technical problems. These several specific embodiments below can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below with reference to the accompanying drawings.

[0056] Figure 2 It is a schematic flowchart of a data processing method provided by an embodiment of the present application. Refer to Figure 2 This method includes the following steps:

[0057] S101. Obtain the table information of the target database; wherein, the table information includes at least one data table and the data in each data table.

[0058] Specifically, the target database is used to indicate one or more databases for which data desensitization processing operations are to be performed. Each target database stores at least one data table, and each data table stores multiple pieces of data. By obtaining at least one data table in the target database and the data stored in each data table, the table information of the target database is obtained, providing a data basis for subsequent operations of data desensitization processing and improving the comprehensiveness and accuracy of desensitization processing.

[0059] Optionally, the table information further includes, but is not limited to, information such as the name of the data table, field name, and corresponding data type.

[0060] In practical applications, the data desensitization processing method follows three core principles: 1. Maintain the integrity of the data format to ensure that the desensitized data still conforms to the data verification standard and does not affect the normal operation of the device; 2. Based on the specific requirements of the application scenario, retain the original information value of the data during desensitization processing to maintain the effectiveness and authenticity of the data; 3. After the desensitized data is reversely restored or analyzed, sensitive information such as the associated user identity cannot be identified.

[0061] Optionally, data desensitization processing methods are mainly divided into two types: static desensitization and dynamic desensitization. Among them, static desensitization processing mainly performs pre-desensitization processing on the original data to generate an independent desensitized dataset, which is then safely transmitted to downstream systems or other user environments for use, that is, the principle of "desensitize first, then use". The data after sensitive processing is usually exported to an offline environment to ensure security. Dynamic desensitization processing is mainly based on specific usage scenarios, user identities, permissions and other sensitive information. At the moment when the data is accessed or processed in real time, it dynamically performs desensitization processing based on preset rules, so that the data remains in its original state in the production environment and is only instantaneously desensitized according to actual application requirements when it needs to be displayed or transmitted, that is, it has the flexibility of "instant desensitization on demand". Dynamic desensitization processing can keep the data in the production environment all the time while meeting the requirements of secure access and compliance.

[0062] Exemplarily, the data processing method provided by the embodiments of the present application can be completed based on an application development framework (such as Spring Boot).

[0063] The data processing method provided by the embodiments of the present application can be applicable to various methods such as static desensitization and dynamic desensitization, with high integration, simple operation, adaptable to various database types, capable of self-adaptively matching all sensitive information, and supporting custom desensitization rules, thereby improving the efficiency and accuracy of data desensitization processing.

[0064] S102. Perform sensitive information matching on the table information to determine a list of sensitive data with sensitive information.

[0065] Specifically, perform sensitive information matching processing on the table information of the target database to obtain at least one data column containing sensitive information, so as to obtain the corresponding list of sensitive data.

[0066] Optionally, sensitive information can be used to indicate data related to the privacy information of users, including but not limited to at least one of the user's name, age, home address, ID number, phone number, family members, etc.

[0067] S103. Generate a desensitization statement based on the list of sensitive data, and the desensitization statement is used to perform desensitization processing on the target database to obtain a desensitized target database.

[0068] Specifically, generate a desensitization SQL statement based on the list of sensitive data, and the desensitization SQL statement is used to automatically perform desensitization processing on each data in the list of sensitive data in the target database to obtain a desensitized target database.

[0069] Optionally, the data in the sensitive data list can be desensitized through desensitization statements, and then the desensitized sensitive data table can replace the sensitive data list in the target database, so as to obtain the desensitized target database.

[0070] Optionally, in the target database, each column of data in the sensitive data list can be determined, and the above-mentioned columns of data in the target database can be desensitized based on the desensitized data to obtain the desensitized target database. In this way, the desensitized target database can also be referred to as a desensitized data table.

[0071] For example, the target database includes four columns of data, namely L1, L2, L3, and L4. Among them, L2 and L3 are columns with sensitive data, and the corresponding sensitive data list can be obtained based on L2 and L3. At this time, 1. The L2 and L3 in the sensitive data list can be directly desensitized to obtain a desensitized data table, and the desensitized data table (including desensitized L2 and desensitized L3) can replace the original L2 and L3 columns in the target database to obtain the desensitized target database (including L1, desensitized L2, desensitized L3, and L4); 2. In the target database, determine the positions of the L2 and L3 columns and the data contained in the L2 and L3 columns, and directly desensitize each data in the L2 and L3 columns to obtain the desensitized target database (including L1, desensitized L2, desensitized L3, and L4).

[0072] In some embodiments, before performing sensitive information matching on the table information, it includes:

[0073] Identify the data types of each piece of data in the table information;

[0074] Determine the target data type among each data type, and determine the data corresponding to the target data type in the table information as non-sensitive data;

[0075] Delete the non-sensitive data to obtain new table information.

[0076] Specifically, when the amount of data in the target database is large, there may be some invalid data (that is, data that does not contain sensitive information) among them, and the existence of invalid data affects the desensitization efficiency. The type of invalid data can be obtained in advance and used as the target data type. Identify the data types of each piece of data in the table information of the target database, match the target data type among each data type, and determine the data corresponding to the target data type in the table information (such as each piece of data in the column where the target data type is located) as non-sensitive data, and delete the non-sensitive data to obtain new table information, further improving the desensitization efficiency of the target database.

[0077] Exemplarily, the invalid data may include at least one of a date field, a large text field, a numeric field, etc. Among them, the date field includes data such as year, month, and day; the large text field includes data that does not contain personal identity information, financial information, or other sensitive information, such as descriptive text, general logs, etc.; the numeric field includes data that is not related to individual identity information (such as ID card number, financial document card number, etc.), such as serial numbers, numbers, counters, etc.

[0078] By comprehensively scanning all field types in the target table, classifying the data, differentiating the valid data from the invalid data, identifying and removing the invalid data of these non-sensitive fields, the amount of data for the sensitive data identification operation can be reduced, the time consumed for the sensitive data identification operation can be reduced, and the efficiency of the sensitive data identification operation can be improved.

[0079] Figure 3 Another process schematic diagram of a data processing method provided by an embodiment of this application. Refer to Figure 3 , step S102 of this method includes the following steps:

[0080] S1021. When the data volume of the table information is less than or equal to the first threshold, select the first quantity of first target data.

[0081] Specifically, when the data volume of the target database is too large, the time required for desensitization processing of the entire table is relatively long and the computational amount is relatively large. The data volume in the table information of the target database can be obtained. When the data volume of the table information is less than or equal to the first threshold, it is determined that the data volume of the target database is not large, and the first quantity of first target data can be randomly selected to improve the randomness and representativeness of the data sensitive information matching process.

[0082] Specifically, the first threshold can be used to indicate the threshold for measuring the data volume of the target database, and it can be specifically set according to the actual situation. For example, the first threshold is set to 10 million, or the first threshold is set to 50 million.

[0083] Specifically, the first quantity can be specifically set according to the actual situation. Exemplarily, the first quantity is a preset proportion of the data volume of the target database. For example, the first quantity is 1% of the data volume of the target database. When the data volume of the target database is 1 million, the first quantity is 10,000. Another example is that the first quantity is the number of columns in the target database. For example, if the target database includes 100 columns of data, the first quantity is 100.

[0084] Optionally, the first quantity can be greater than the number of columns in the target database or less than the number of columns in the target database.

[0085] Optionally, the first target data can be data in the same column or data in different columns.

[0086] S1022. Match the sensitive information for each of the first target data based on a regular expression, and determine a list of sensitive data with sensitive information.

[0087] Specifically, match the sensitive information for each of the first target data based on a regular expression, find out the first target data with sensitive information, and determine a list of sensitive data based on the first target data with sensitive information.

[0088] Optionally, the sensitive information can be specifically set according to the actual situation, and it includes at least one of name, ID number, date of birth, phone number, email address, financial card number, etc.

[0089] In some embodiments, the step of matching the sensitive information for each of the first target data based on a regular expression and determining a list of sensitive data with sensitive information includes:

[0090] Match the sensitive information for each of the first target data based on the regular expression to obtain a first matching result for each of the first target data;

[0091] When the first matching result of the first target data indicates the inclusion of sensitive information, determine the column where the first target data is located in the table information as the first target column;

[0092] Generate the list of sensitive data based on each data included in each of the first target columns.

[0093] Specifically, match the sensitive information for each of the first target data based on a regular expression to obtain a first matching result for each of the first target data. The first matching result includes at least one of including sensitive information, not including sensitive information, and being uncertain.

[0094] Specifically, find out the first target data whose first matching result indicates the inclusion of sensitive information, determine the column where the above first target data is located as the first target column, obtain each data in the first target column, and generate a list of sensitive data. In this way, the list of sensitive data includes at least one data column with sensitive information.

[0095] Optionally, determine the column where the first target data with an uncertain first matching result is located as the first target column.

[0096] Optionally, the regular expression rule set of sensitive information can be specifically set according to the actual situation.

[0097] Exemplarily, if the user's identity ID is 15 - digit or 18 - digit, the regular expression may include , where the last digit of the 18 - digit number can be a digit or a letter 。

[0098] Exemplarily, if the user's phone number is 11 digits, the regular expression may include: , which is used to match a phone number starting with 1, with any digit between 3 and 9 as the second digit, followed by 9 digits. Alternatively, the regular expression may include: , to match multiple phone number formats, including area code - number, fixed phone number, mobile terminal number, and other types of data.

[0099] Exemplarily again, the regular expression for the user's name may be: , which is used to match names containing Chinese characters. The regular expression for an email address may be: , to match the standard email address format. The regular expression for a social software number may be: , to match a social software number starting with a letter, followed by 5 - 19 letters, digits, underscores, or hyphens.

[0100] By adaptively configuring the regular expression rule set for sensitive information, performing regular expression matching operations on each piece of first - target data, and generating a corresponding sensitive data list if sensitive information is matched, the recognition efficiency of sensitive information is improved, and thus the efficiency of data desensitization processing is improved.

[0101] In some embodiments, the matching of sensitive information in the table information to determine the sensitive data list with sensitive information includes:

[0102] When the data volume of the table information is greater than the first threshold, in the order of each piece of first - target data in the table information, based on the regular expression, perform sensitive information matching on each piece of first - target data in turn to obtain the second matching result of each piece of first - target data;

[0103] When the second matching result of the first - target data indicates that it contains sensitive information, determine the first - target data as the second - target data;

[0104] When the number of the second - target data reaches the first quantity, stop executing the steps of performing sensitive information matching on each piece of first - target data in turn based on the regular expression in the order of each piece of first - target data in the table information to obtain the second matching result of each piece of first - target data and subsequent steps, and determine the column where each piece of second - target data is located as the second - target column;

[0105] Based on the data included in each of the second - target columns with the first quantity, determine the sensitive data list.

[0106] Specifically, when the data volume of the table information in the target database is greater than the first threshold, obtain the order of each first target data in the table information, and based on the regular expression in the order of the above-mentioned first target data, perform sensitive information matching on each first target data in turn to obtain the second matching result of each first target data.

[0107] Optionally, the order of each first target data in the table information may include any one of the row order of each first target data in the table information and the column order of each first target data in the table information.

[0108] Optionally, in the table information, determine each first target data whose second matching result indicates the inclusion of sensitive data as the second target data, and obtain the quantity of the second target data. When the quantity of the second target data reaches the first quantity, stop performing the operations of performing sensitive information matching on the first target data based on the regular expression, obtaining the second matching result of each first target data and subsequent steps, and determine the column where each second target data is located as the second target column, so as to obtain the first quantity of second target columns. Obtain each data in each second target column, and generate a sensitive data list based on the data in the first quantity of second target columns.

[0109] The sensitive information matching process classified based on the actual data volume in the target database reduces the processing volume of sensitive data matching and improves the efficiency of sensitive data matching.

[0110] Figure 4 It is an application scenario diagram of the data processing method provided by the embodiment of the present application.

[0111] As Figure 4 shown in 401, the target database includes 5 columns of data, namely L1, L2, L3, L4, and L5. Among them, it is recognized that the data L1 is a serial number and does not involve data such as user privacy information. Determine the data in L1 as invalid data and delete it, and perform sensitive information matching on the column data such as L2, L3, L4, and L5 (as shown in 402). During the matching, it is determined that there is sensitive information involving user privacy in L2 (including the user's name) and L3 (including the user's age). Then, determine L2 and L3 as the sensitive data list (as shown in 403), and perform data desensitization processing on L2 and L3 ( Figure 4 shown by replacement processing), and obtain the desensitized L2 and L3 (as shown in 404), so as to obtain the desensitized target database (as shown in 405).

[0112] Figure 5 It is another process schematic diagram of the data processing method provided by the embodiment of the present application. Refer to Figure 5 , step S103 of this method includes the following steps:

[0113] S1031. Generate corresponding desensitization statements based on the type of the target database and the desensitized data list. Among them, the desensitization statements are used to desensitize each data in the sensitive data list in the target database to obtain a desensitized target database. The desensitization process includes at least one of encryption processing, replacement processing, and blurring processing.

[0114] Specifically, obtain the type of the target database, and generate corresponding desensitization SQL statements according to the type of the target database and the desensitized data list. The desensitization SQL statements are specifically used to automatically desensitize each data in the sensitive data list in the target database to obtain a desensitized target database.

[0115] Among them, the desensitized target database can indicate a data table composed of only the columns in the sensitive data list that have undergone desensitization processing, or can indicate each data table in the target database (including the columns in the sensitive data list that have undergone desensitization processing).

[0116] Optionally, the type of the target database includes but is not limited to Oracle, PostgreSQL, MySQL, etc.

[0117] Optionally, the desensitization process includes at least one of encryption processing, replacement processing, and blurring processing (such as masking, etc.).

[0118] In some embodiments, the types of the data in the sensitive data list can be determined, and corresponding desensitization processing can be performed according to the types of the data. For example, if data A is of the integer (int) data type, a random int data is generated to replace data A to achieve blurring processing of data A.

[0119] By determining the type of the target database, targeted dynamic generation of automated desensitization statements is achieved, so that the generated desensitization statements can meet the specific requirements of data desensitization, and strictly comply with the security operation specifications and performance optimization principles of the database, making the desensitization process both efficient and safe, avoiding potential risks that may affect the stability, integrity, and performance of the database, and can be applicable to different database types. And through the automated processing flow, the full amount of data is adaptively parsed and desensitization statements are generated, greatly reducing the time of manual operation and the risk of possible data omission, improving the efficiency of identifying sensitive data in the database, as well as the accuracy of desensitization processing and based on the generated desensitization statements.

[0120] Figure 6 Another process schematic diagram of a data processing method provided by an embodiment of the present application. Refer to Figure 6 This method includes the following steps:

[0121] S201. Obtain the configuration information of the target database;

[0122] S202. Based on the configuration information, establish a connection relationship with the target database.

[0123] Specifically, obtain the configuration information of the target database. The configuration information of the target data includes, but is not limited to, at least one of the database address URL, username, and password.

[0124] Among them, the configuration information is used to indicate the data that can be used to establish a connection relationship with the database, facilitating the execution of database operations.

[0125] Specifically, based on the configuration information of the target database, establish a connection relationship with the target database.

[0126] Exemplarily, by using annotations to map the data source configuration in the lightweight data serialization language file (YAML Ain't Markup Language, YAML) file to the attributes of the Java class, the database configuration information can be obtained through the above attributes, and a database connection can be established. For example, when starting data desensitization processing, the current terminal can automatically scan the application (application.yml) configuration file and obtain the configuration information to initialize the data source information required by the desensitization handler, thereby establishing a connection relationship with the target database.

[0127] Optionally, when the establishment of the connection relationship with the target database fails, it is possible to detect whether the configuration information of the target database is correct.

[0128] In the embodiments of the present application, by automatically obtaining the full-scale table information of the target database, adaptively matching sensitive fields according to the actual data volume of the statistical full-scale table, automatically forming a sensitive data list and dynamically generating corresponding desensitization statements, the efficiency of data desensitization data recognition is improved, thereby improving the efficiency and accuracy of desensitization processing based on the generated desensitization statements, protecting user privacy, and improving the user experience.

[0129] It should be noted that for the foregoing method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should know that the present application is not limited by the described action sequence, because according to the present application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to the present application.

[0130] It should be further noted that although the steps in the flowchart are sequentially displayed according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear description in this document, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowchart may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed alternately or in turn with at least a part of other steps or sub-steps or stages of other steps.

[0131] Figure 7 This is a schematic structural diagram of a data processing device provided by an embodiment of the present application. Refer to Figure 7 , the data processing device includes:

[0132] The first acquisition module 701 is configured to acquire table information of a target database; wherein, the table information includes at least one data table and data in each of the data tables;

[0133] The matching module 702 is configured to perform sensitive information matching on the table information to determine a list of sensitive data with sensitive information;

[0134] The generation module 703 is configured to generate a desensitization statement based on the list of sensitive data, and the desensitization statement is used to perform desensitization processing on the target database to obtain a desensitized target database.

[0135] In some embodiments, the device includes:

[0136] The recognition module is configured to recognize the data types of the data in the table information;

[0137] The determination module is configured to determine a target data type from the data types, and determine the data corresponding to the target data type in the table information as non-sensitive data;

[0138] The deletion module is configured to delete the non-sensitive data to obtain new table information.

[0139] In some embodiments, the matching module 702 includes:

[0140] The selection unit is configured to select a first quantity of first target data when the data volume in the table information is less than or equal to a first threshold;

[0141] The first matching unit is configured to perform sensitive information matching on each of the first target data based on a regular expression to determine a list of sensitive data with sensitive information.

[0142] In some embodiments, the first matching unit is specifically configured to:

[0143] Based on the regular expression, perform sensitive information matching on each of the first target data to obtain a first matching result for each of the first target data;

[0144] When the first matching result of the first target data indicates the inclusion of sensitive information, determine the column where the first target data is located in the table information as the first target column;

[0145] Generate the sensitive data list based on each data included in each of the first target columns.

[0146] In some embodiments, the matching module 702 includes:

[0147] A second matching unit, configured to, when the data volume of the table information is greater than the first threshold, sequentially perform sensitive information matching on each of the first target data based on the regular expression in the order of each of the first target data in the table information to obtain a second matching result for each of the first target data;

[0148] A first determining unit, configured to, when the second matching result of the first target data indicates the inclusion of sensitive information, determine the first target data as the second target data;

[0149] A second determining unit, configured to, when the number of the second target data reaches the first number, stop executing the steps of sequentially performing sensitive information matching on each of the first target data based on the regular expression in the order of each of the first target data in the table information to obtain a second matching result for each of the first target data and subsequent steps, and determine the columns where each of the second target data is located as the second target columns;

[0150] A third determining unit, configured to determine the sensitive data list based on each data included in the second target columns of the first number.

[0151] In some embodiments, the generating module 703 includes:

[0152] A generating unit, configured to generate a corresponding desensitization statement based on the type of the target database and the desensitized data list; wherein, the desensitization statement is used to perform desensitization processing on each data in the sensitive data list in the target database to obtain a desensitized target database; the desensitization processing includes at least one of encryption processing, replacement processing, and fuzzing processing.

[0153] In some embodiments, the device includes:

[0154] A second acquisition module, configured to acquire configuration information of the target database;

[0155] An establishment module, configured to establish a connection relationship with the target database based on the configuration information.

[0156] It should be understood that the above device embodiments are illustrative, and the devices of the present application can also be implemented in other ways. For example, the division of units / modules in the above embodiments is only a logical function division, and there can be other division methods in actual implementation. For example, multiple units, modules or components can be combined, or can be integrated into another system, or some features can be ignored or not executed.

[0157] Figure 8 This is a schematic structural diagram of the electronic device provided in the embodiment of the present application. As Figure 8 shown, the electronic device may include: a transceiver 801, a processor 802, and a memory 803.

[0158] Among them, the processor 802 executes the computer execution instructions stored in the memory, so that the processor 802 executes the solutions in the above embodiments. The processor 802 may be a general-purpose processor, including a central processing unit CPU, a network processor (NP), etc.; it may also be a digital signal processor DSP, an application-specific integrated circuit ASIC, a field-programmable gate array FPGA, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0159] The memory 803 is connected to the processor 802 through a system bus and completes communication with each other. The memory 803 is used to store computer program instructions.

[0160] The transceiver 801 can be used for table information, configuration information, etc. of the target database.

[0161] The system bus may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The system bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of simplicity, only a thick line is shown in the figure, but it does not mean that there is only one bus or one type of bus. The transceiver is used to implement communication between the database access device and other computers (such as clients, read-write libraries, and read-only libraries). The memory may include a random access memory (RAM), and may also include a non-volatile memory.

[0162] An embodiment of the present application further provides a computer-readable storage medium, in which computer instructions are stored. When the computer instructions run on a computer, the computer is caused to execute the technical solution of the data processing method in the above embodiment.

[0163] An embodiment of the present application further provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium, and when the at least one processor executes the computer program, the technical solution of the data processing method in the above embodiment can be implemented.

[0164] In addition, unless otherwise specified, in each embodiment of the present application, each functional unit / module can be integrated in one unit / module, or each unit / module can exist physically alone, or two or more unit / modules can be integrated together. The above integrated unit / module can be implemented in the form of hardware or in the form of a software program module.

[0165] When the integrated unit / module is implemented in the form of hardware, the hardware can be a digital circuit, an analog circuit, etc. The physical implementation of the hardware structure includes but is not limited to transistors, memristors, etc. Unless otherwise specified, the processor can be any suitable hardware processor, such as a CPU, GPU, FPGA, DSP, and ASIC, etc. Unless otherwise specified, the storage unit can be any suitable magnetic storage medium or magneto-optical storage medium, such as resistive random access memory (RRAM), dynamic random access memory (DRAM), static random access memory (SRAM), enhanced dynamic random access memory (EDRAM), high-bandwidth memory (HBM), hybrid memory cube (HMC), etc.

[0166] When the integrated unit / module is implemented in the form of a software program module and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods in various embodiments of the present application. And the aforementioned memory includes: various media such as USB flash drives, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), mobile hard disks, magnetic disks, or optical discs that can store program codes.

[0167] In the above embodiments, the descriptions of the various embodiments have their own emphases. For the parts not detailed in a certain embodiment, reference can be made to the relevant descriptions of other embodiments. The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0168] Those skilled in the art will readily think of other implementation manners of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any variations, uses, or adaptations of the present application, which follow the general principles of the present application and include the common general knowledge or conventional technical means in the technical field not disclosed in the present application. The specification and the embodiments are only regarded as exemplary, and the true scope and spirit of the present application are pointed out by the following claims.

[0169] It should be understood that the present application is not limited to the exact structure already described and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present application is only limited by the appended claims.

Claims

1. A data processing method, characterized in that, Including: Obtain the table information of the target database; wherein, the table information includes at least one data table and the data in each data table; Perform sensitive information matching on the table information to determine a list of sensitive data with sensitive information; Generate a desensitization statement based on the list of sensitive data, and the desensitization statement is used to desensitize the target database to obtain a desensitized target database.

2. The method according to claim 1, characterized in that, Before performing sensitive information matching on the table information, it includes: Identify the data types of the data in the table information; Determine a target data type among the data types, and determine the data corresponding to the target data type in the table information as non-sensitive data; Delete the non-sensitive data to obtain new table information.

3. The method according to claim 1, wherein The performing sensitive information matching on the table information to determine a list of sensitive data with sensitive information includes: When the data volume in the table information is less than or equal to the first threshold, select a first quantity of first target data; Based on a regular expression, perform sensitive information matching on each first target data to determine a list of sensitive data with sensitive information.

4. The method according to claim 3, wherein The based on a regular expression, performing sensitive information matching on each first target data to determine a list of sensitive data with sensitive information includes: Based on the regular expression, perform sensitive information matching on each first target data to obtain a first matching result of each first target data; When the first matching result of the first target data indicates the inclusion of sensitive information, determine the column where the first target data is located in the table information as the first target column; Generate the list of sensitive data based on the data included in each first target column.

5. The method according to claim 3, characterized in that, The performing sensitive information matching on the table information to determine a list of sensitive data with sensitive information includes: When the data volume in the table information is greater than the first threshold, in the order of each first target data in the table information, based on the regular expression, perform sensitive information matching on each first target data in sequence to obtain a second matching result of each first target data; When the second matching result of the first target data indicates the inclusion of sensitive information, determine the first target data as the second target data; When the quantity of the second target data reaches the first quantity, stop executing the steps of performing sensitive information matching on each first target data in sequence based on the regular expression in the order of each first target data in the table information to obtain the second matching result of each first target data and subsequent steps, and determine the columns where each second target data is located as the second target columns; Determine the list of sensitive data based on the data included in the first quantity of second target columns.

6. The method according to claim 1, characterized in that, The generating a desensitization statement based on the list of sensitive data, and the desensitization statement is used to desensitize the target database to obtain a desensitized target database, includes: Generate corresponding desensitization statements based on the type of the target database and the desensitized data list; wherein, the desensitization statements are used to desensitize each piece of data in the sensitive data list in the target database to obtain a desensitized target database; the desensitization processing includes at least one of encryption processing, replacement processing, and fuzzing processing.

7. The method according to any one of claims 1-6, characterized in that, Before obtaining the table information of the target database, it includes: Obtain the configuration information of the target database; Based on the configuration information, establish a connection relationship with the target database.

8. A data processing device, characterized in that, It includes: A first acquisition module, configured to acquire the table information of the target database; wherein, the table information includes at least one data table and the data in each of the data tables; A matching module, configured to perform sensitive information matching on the table information to determine a sensitive data list with sensitive information; A generation module, configured to generate desensitization statements based on the sensitive data list, and the desensitization statements are used to desensitize the target database to obtain a desensitized target database.

9. An electronic device, characterized in that, It includes: A processor and a memory communicatively connected to the processor; The memory stores computer execution instructions; The processor executes the computer execution instructions stored in the memory to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, Computer execution instructions are stored in the computer-readable storage medium, and when the computer execution instructions are executed by a processor, they are used to implement the method according to any one of claims 1 to 7.

11. A computer program product, characterized in that, It includes a computer program, and when the computer program is executed by a processor, it implements the method according to any one of claims 1-7.

Images