A method, device and storage medium for updating asset catalog with sensitive data marking

Abstract

The present invention provides an asset catalog update method, device and storage medium with sensitive data marking, the steps of the method include: obtaining an initial data catalog corresponding to a database in an initial state, the initial data catalog recording the names of each table in the database and the column names of each column in the table; setting a data capture component in the entry and / or output path of the database, obtaining access requests passing through the database and / or feedback data of the database based on the access request based on the data capture component; parsing the database access request and / or feedback data based on the type of the database to obtain database update information, and updating the initial data catalog based on the content of the database update information to obtain an updated data catalog. The present application can capture database change information in real time, complete the update of the data catalog, does not need to apply for scanning permissions, and can update the data catalog in real time to ensure real-time correspondence between the data catalog and the content in the database.

Description

Technical Field

[0001] The present invention relates to the field of data security technology, and in particular to a method, device and storage medium for updating an asset catalog with sensitive data marking. Background Art

[0002] With the continuous development of the Internet, existing enterprises or organizations, due to their large amount of data, usually build their own data into a database for storage. In order to understand the content in the database, data asset managers usually need to build an asset catalog, and for data security considerations, they need to further mark sensitive data including ID numbers, names, etc. in the asset catalog to strengthen the security management of sensitive data. The existing way to build an asset catalog is usually to scan the database to establish the asset catalog, and to update the asset catalog by periodically scanning the database.

[0003] However, there are limitations to updating the asset catalog by periodically scanning the database. On the one hand, scanning the entire database and all tables requires obtaining database access rights in advance. In some companies, database access rights are controlled by the business department. When the data governance department (generally the enterprise security department) needs to scan the database, it cannot obtain database access rights, or it needs to pass the approval of the internal process before it can obtain them. In addition, database scanning will have an impact on performance, resulting in the business department not allowing frequent scanning, and it is likely that scanning will only be allowed at specified times. Therefore, for the data governance department, the process of collecting data assets is very inefficient, and the marking of sensitive data will also lag accordingly, resulting in reduced data security; on the other hand, the periodic scanning method requires a certain period of time to update the database, and it is impossible to achieve real-time updates to the database asset catalog, and it is difficult to achieve real-time correspondence between the database asset catalog and the internal data of the database. Summary of the invention

[0004] In view of this, an embodiment of the present invention provides an asset catalog update method with sensitive data marking to eliminate or improve one or more defects existing in the prior art.

[0005] One aspect of the present invention provides a method for updating an asset catalog with sensitive data marking, the method comprising:

[0006] Acquire an initial data directory corresponding to the database in an initial state, wherein the initial data directory records metadata information in the database;

[0007] A data capture component is arranged at an entry and / or output path of the database, and an access request of the database and / or feedback data of the database based on the access request is acquired based on the data capture component;

[0008] The database access request and / or feedback data are parsed based on the type of the database to obtain database update information, and the initial data directory is updated based on the content of the database update information to obtain an updated data directory.

[0009] By adopting the above scheme, the database access request can be an operation of adding, deleting, modifying and checking the database. The scheme of the present application can capture the operation information and the database's feedback on the operation in real time. The present application can capture the database change information in real time, and complete the update of the data directory corresponding to the database based on the changes inside the database. On the one hand, there is no need to apply for scanning permissions. On the other hand, the present application can update the data directory in real time to ensure the real-time correspondence between the data directory and the content in the database.

[0010] In some embodiments of the present invention, the initial data directory is updated based on the content of the database update information, and the step of obtaining the updated data directory includes:

[0011] Determine whether the database update information includes sensitive data based on the regular expression feature library, if the database update information includes sensitive data;

[0012] The sensitive data is marked in the update data directory.

[0013] In some embodiments of the present invention, the update status of the database update information includes:

[0014] Add annotations to sensitive data that was not originally annotated;

[0015] Add annotations to sensitive data newly added to the database;

[0016] Remove annotations for non-sensitive data where annotations exist.

[0017] In some embodiments of the present invention, the capture component includes but is not limited to a monitoring component or a proxy component.

[0018] If the capture component is a monitoring component, the monitoring component is set at the network interface of the database, and is used to obtain the access request of the database through the network interface and / or the feedback data of the database based on the access request;

[0019] If the capture component is a proxy component, the proxy component is set on the path between the database end and the client end, forwards the data between the two, and saves the forwarded data to obtain the database access request and / or the database feedback data based on the access request.

[0020] In some embodiments of the present invention, the type of the database includes but is not limited to MySQL or PostgreSQL, and the step of parsing the database access request and / or feedback data based on the type of the database includes:

[0021] If the type of the database is MySQL, the MySQL parser is used for parsing;

[0022] If the type of the database is PostgreSQL, the PostgreSQL parser is used for parsing.

[0023] In some embodiments of the present invention, the content of the database update information includes metadata information of the modified database, the metadata information includes database name / schema name / table name / column name, each database includes multiple schemas, each schema includes multiple tables, and each table includes multiple columns;

[0024] The initial data directory is updated based on the content of the database update information, and the step of obtaining the updated data directory includes:

[0025] Based on the database name / schema name / table name / column name in the metadata information in the database update information, the database name / schema name / table name / column name in the initial data directory is updated to obtain an updated data directory.

[0026] In some embodiments of the present invention, if the content of the database update information is to add / subtract a table or to add / subtract a column in an existing table, the name of the table is added / subtracted or the column name in the existing table is added / subtracted in the initial data directory.

[0027] In some implementations of the present invention, if the content of the database update information is to change the table name or change the column name in an existing table, the table name is changed or the column name in the existing table is changed in the initial data directory.

[0028] In some embodiments of the present invention, the database includes a user table and a system table, the system table is used to record the database where each user table is located, the mode where it is located, the name of the user table and the column name of each column in the user table, and the initial data directory is updated based on the content of the database update information, and the step of obtaining the updated data directory includes:

[0029] The change information about the system table in the content of the database update information is obtained, and the initial data directory is updated based on the change information about the system table to obtain an updated data directory.

[0030] Additional advantages, purposes, and features of the present invention will be described in part in the following description, and will become apparent to those skilled in the art after studying the following, or may be learned from the practice of the present invention. The purposes and other advantages of the present invention can be specifically pointed out and obtained in the specification and the accompanying drawings.

[0031] Those skilled in the art will appreciate that the objectives and advantages that can be achieved with the present invention are not limited to the above specific description, and the above and other objectives that can be achieved by the present invention will be more clearly understood from the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The drawings described herein are used to provide a further understanding of the present invention, constitute a part of this application, and do not constitute a limitation of the present invention.

[0033] Figure 1 It is a schematic diagram of a first implementation of the asset catalog update method with sensitive data marking of the present invention;

[0034] Figure 2 It is a schematic diagram of a second implementation of the asset catalog update method with sensitive data marking of the present invention. DETAILED DESCRIPTION

[0035] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments and the accompanying drawings. Here, the illustrative embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0036] It should also be noted that, in order to avoid obscuring the present invention due to unnecessary details, only structures and / or processing steps closely related to the solutions according to the present invention are shown in the accompanying drawings, while other details that are not closely related to the present invention are omitted.

[0037] It should be emphasized that the term “include / comprises” when used herein refers to the presence of features, elements, steps or components, but does not exclude the presence or addition of one or more other features, elements, steps or components.

[0038] It should also be noted that, unless otherwise specified, the term “connection” herein may refer not only to a direct connection but also to an indirect connection involving an intermediate.

[0039] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In the accompanying drawings, the same reference numerals represent the same or similar components, or the same or similar steps.

[0040] To solve the above problems, Figure 1 , 2As shown, the present invention proposes a method for updating an asset catalog with sensitive data marking, the steps of the method comprising:

[0041] Step S100, obtaining an initial data directory corresponding to the database in the initial state, wherein the initial data directory records metadata information in the database;

[0042] In some embodiments of the present invention, the database in the initial state may be an empty database. When the database is empty, the initial data directory is also empty, that is, the data directory is created when the database is created, so that the two correspond to each other in real time;

[0043] When the database is empty, there is a framework of system tables and user tables inside, but no specific data exists.

[0044] The name of the table may be a table name named after the user involved in the table content, and the column name may be a mobile phone number, bank card number, ID card number, email address, work number, etc.;

[0045] In some embodiments of the present invention, the database in the initial state may also be a database having multiple data tables inside, and the initial data directory corresponding to the database in the initial state may be a data directory obtained by scanning the database in advance.

[0046] Step S200: Setting a data capture component at the input and / or output path of the database, and acquiring a database access request and / or feedback data of the database based on the access request based on the data capture component;

[0047] In some embodiments of the present invention, the capture component may be a plug-in external to the database server, which captures traffic in and out of the database and parses the traffic to obtain database access requests and / or database feedback data based on the access requests.

[0048] In some embodiments of the present invention, if the capture component is set at the entry path of the database, the capture component obtains the access request of the database; if the capture component is set at the output path of the database, the capture component obtains feedback data of the database based on the access request.

[0049] Step S300: parsing the database access request and / or feedback data based on the type of the database to obtain database update information, and updating the initial data directory based on the content of the database update information to obtain an updated data directory.

[0050] In some embodiments of the present invention, the type of the database includes but is not limited to MySQL, MariaDB, PostgreSQL, Oracle, etc.

[0051] In some embodiments of the present invention, updating the initial data directory may include adding the name of a table or a column name in a table to the data directory; deleting the name of a table or a column name in a table; changing the name of a table or a column name in a table.

[0052] By adopting the above scheme, the database access request can be an operation of adding, deleting, modifying and checking the database. The scheme of the present application can capture the operation information and the database's feedback on the operation in real time. The present application can capture the database change information in real time, and complete the update of the data directory corresponding to the database based on the changes inside the database. On the one hand, there is no need to apply for scanning permissions. On the other hand, the present application can update the data directory in real time to ensure the real-time correspondence between the data directory and the content in the database.

[0053] like Figure 2 As shown, in some embodiments of the present invention, the initial data directory is updated based on the content of the database update information, and the step of obtaining the updated data directory includes:

[0054] Determine whether the database update information includes sensitive data based on the regular expression feature library, if the database update information includes sensitive data;

[0055] The sensitive data is marked in the update data directory.

[0056] The database update information may be used to add annotations to data that was not originally annotated, to add annotations to newly added data, or to delete annotations from data that was originally annotated.

[0057] Sensitive data in the database can be identified based on the existing regular expression feature library matching. For example, sensitive data types such as mobile phone numbers, bank card numbers, ID numbers, and email addresses can be matched with table names and column names to ultimately determine whether each field in the database table meets the characteristics of sensitive data.

[0058] With the above solution, the discovery, classification and grading of the database's asset catalog and sensitive data is a key part of data governance and data security protection. Many companies need to fully and clearly understand the data asset catalog and the location of sensitive data in the database. This application can complete the labeling of sensitive data while updating the data catalog, thereby achieving standardized management and maintenance of data with access control policies.

[0059] Traditional solutions require setting up scanning cycles, which leads to a lag in the sorting of existing database asset catalogs and the mechanism for discovering sensitive data. They are also limited to the company's process specifications, inefficient, and inoperable.

[0060] The method proposed in this article is to capture the flow data in real time, parse the data, extract useful data, and then perform feature matching to complete the discovery of sensitive data when the database is accessed. The discovery process is a passive method that does not perform a full scan of the entire database and all tables, and does not require database access permissions. Moreover, we will capture sensitive data as soon as it appears, providing a more real-time and accurate basis for data protection.

[0061] In some embodiments of the present invention, the update status of the database update information includes:

[0062] Add annotations to sensitive data that was not originally annotated;

[0063] Add annotations to sensitive data newly added to the database;

[0064] Remove annotations for non-sensitive data where annotations exist.

[0065] In some embodiments of the present invention, the capture component includes but is not limited to a monitoring component or a proxy component.

[0066] If the capture component is a monitoring component, the monitoring component is set at the network interface of the database, and is used to obtain the access request of the database through the network interface and / or the feedback data of the database based on the access request;

[0067] In some embodiments of the present invention, when a monitoring component is used, a monitoring component can be placed on the host of the database. When data passes through a network interface of the host, our monitoring component can capture the data. The capture mechanism can be based on the principle of network tap, adding a bypass process at the data link layer. When a data packet arrives at the network interface, a copy of the data packet is obtained from the link layer driver. This monitoring component can be placed on the machine where the database is running, or it can be placed on the machine where the application program accessing the database is running.

[0068] If the capture component is a proxy component, the proxy component is set on the path between the database end and the client end, forwards the data between the two, and saves the forwarded data to obtain the database access request and / or the database feedback data based on the access request. The present invention eliminates the requirement for database access rights and makes up for the scenarios where the current discovery method is not applicable. At the same time, the mechanism of sensitive data discovery is carried out continuously and in real time, providing a more real-time and accurate basis for data protection.

[0069] In some embodiments of the present invention, the proxy component is arranged on the path between the database end and the client end, and data is transmitted from the database end to the client end or from the client end to the database end through the proxy component.

[0070] In some embodiments of the present invention, when a proxy component is used, the proxy component is set in the path between the database end and the client. When accessing the database, the database is not accessed directly, but through the proxy component. The proxy component will accept all database access requests from the client and forward the requests to the database server. When the database server returns feedback data, the proxy component will forward the feedback data to the client. While forwarding, the proxy component will copy a copy of the data, thereby capturing all the original data that is flowing.

[0071] By adopting the above scheme, the present invention obtains the update information of the database by adopting the capture component, which not only avoids the use of scanning the entire or partial database to reduce efficiency; but also does not need to set the scanning frequency, ensuring real-time updating of the data directory.

[0072] The proxy mode can not only capture data, but also control the returned data, and can be used in a variety of scenarios. In the monitoring mode, since it is not in the data flow, there is very little additional overhead for accessing data, and the impact on the existing system is very small.

[0073] In some embodiments of the present invention, this solution can use a listening component and a proxy component at the same time. When the listening component and the proxy component exist at the same time, the data collected by the two are parsed and the parsed data are compared. If the parsed data are different, the codes for accessing the listening component and the proxy component are further compared to determine whether the codes are different. If the codes are different, it means that there is a miscommunication.

[0074] By adopting the above scheme, the present invention can compare the data parsed by the monitoring component and the proxy component. The comparison content can be the table name or column name, etc. If there is a difference between the two, it means that there is an error in the parsing process. By comparing the access codes of the two, if the access codes of the two are different, a warning is sent. The management personnel can be notified through the warning message, and the time and content of the above warning message are recorded in the management log, so as to improve the accuracy of data processing and avoid the final recording errors of the data directory due to program errors.

[0075] In some embodiments of the present invention, the type of the database includes but is not limited to MySQL or PostgreSQL, and the step of parsing the database access request and / or feedback data based on the type of the database includes:

[0076] If the type of the database is MySQL, the MySQL parser is used for parsing;

[0077] If the type of the database is PostgreSQL, the PostgreSQL parser is used for parsing.

[0078] By adopting the above scheme, the types of databases are diverse, and the present invention can match different parsers to perform data parsing to ensure processing efficiency.

[0079] In some embodiments of the present invention, the content of the database update information includes metadata information of the modified database, the metadata information includes database name / schema name / table name / column name, each database includes multiple schemas, each schema includes multiple tables, and each table includes multiple columns;

[0080] The initial data directory is updated based on the content of the database update information, and the step of obtaining the updated data directory includes:

[0081] Based on the database name / schema name / table name / column name in the metadata information in the database update information, the database name / schema name / table name / column name in the initial data directory is updated to obtain an updated data directory.

[0082] The database update information may also be data in a column of a modified table, such as ID card number data in an ID card number column.

[0083] In some embodiments of the present invention, if the content of the database update information is to add / subtract a table or to add / subtract a column in an existing table, the name of the table is added / subtracted or the column name in the existing table is added / subtracted in the initial data directory.

[0084] In some implementations of the present invention, if the content of the database update information is to change the table name or change the column name in an existing table, the table name is changed or the column name in the existing table is changed in the initial data directory.

[0085] In some embodiments of the present invention, the database includes a user table and a system table, the system table is used to record the database where each user table is located, the mode where it is located, the name of the user table and the column name of each column in the user table, and the initial data directory is updated based on the content of the database update information, and the step of obtaining the updated data directory includes:

[0086] The change information about the system table in the content of the database update information is obtained, and the initial data directory is updated based on the change information about the system table to obtain an updated data directory.

[0087] An embodiment of the present invention also provides an asset catalog update device with sensitive data marking, which includes a computer device, the computer device includes a processor and a memory, the memory stores computer instructions, the processor is used to execute the computer instructions stored in the memory, and when the computer instructions are executed by the processor, the device implements the steps of the method described above.

[0088] The embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the asset catalog update method with sensitive data marking are implemented. The computer-readable storage medium can be a tangible storage medium, such as a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a floppy disk, a hard disk, a removable storage disk, a CD-ROM, or any other form of storage medium known in the technical field.

[0089] An embodiment of the present invention further provides a computer device, which may include a processor and a memory. The processor and the memory may be connected via a bus or in other ways.

[0090] The processor may be a central processing unit (CPU). The processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or a combination of the above chips.

[0091] As a non-transitory computer-readable storage medium, the memory can be used to store non-transitory software programs, non-transitory computer executable programs and modules, such as program instructions / modules corresponding to the method for shielding buttons of a vehicle-mounted display device in an embodiment of the present invention. The processor executes various functional applications and data processing of the processor by running the non-transitory software programs, instructions and modules stored in the memory to implement the method of the present invention.

[0092] The memory may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created by the processor, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0093] In some embodiments of the present disclosure, the user end may include a processor, a memory, and a transceiver unit, which may include a receiver and a transmitter. The processor, memory, receiver, and transmitter may be connected through a bus system, the memory is used to store computer instructions, and the processor is used to execute the computer instructions stored in the memory to control the transceiver unit to send and receive signals.

[0094] As an implementation method, the functions of the receiver and the transmitter in the present invention may be implemented by a transceiver circuit or a dedicated chip for transceiver, and the processor may be implemented by a dedicated processing chip, a processing circuit or a general-purpose chip.

[0095] As another implementation, it is possible to use a general-purpose computer to implement the authentication device and authentication server provided in the embodiment of the present invention, that is, the program code for implementing the functions of the processor, receiver, and transmitter is stored in a memory, and the general-purpose processor implements the functions of the processor, receiver, and transmitter by executing the code in the memory.

[0096] It should be understood by those skilled in the art that the exemplary components, systems and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software or a combination of the two. Whether it is performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present invention are programs or code segments used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link via a data signal carried in a carrier.

[0097] It should be clear that the present invention is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present invention.

[0098] In the present invention, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with or replace features of other embodiments.

[0099] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the embodiments of the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for updating an asset catalog with sensitive data marking, characterized in that: The steps of the method include, Acquire an initial data directory corresponding to the database in an initial state, wherein the initial data directory records metadata information in the database; A data capture component is arranged at an entry and / or output path of the database, and an access request of the database and / or feedback data of the database based on the access request is acquired based on the data capture component; The database access request and / or feedback data is parsed based on the type of the database to obtain database update information, the initial data directory is updated based on the content of the database update information, and it is determined based on a regular expression feature library whether the database update information includes sensitive data; if the database update information includes sensitive data, the sensitive data is marked in the updated data directory to obtain an updated data directory.

2. The method for updating an asset catalog with sensitive data marking according to claim 1, characterized in that: The update status of the database update information includes: Add annotations to sensitive data that was not originally annotated; Add annotations to sensitive data newly added to the database; Remove annotations for non-sensitive data where annotations exist.

3. The method for updating an asset catalog with sensitive data marking according to claim 1, characterized in that: The capture component includes but is not limited to a monitoring component or a proxy component. If the capture component is a monitoring component, the monitoring component is set at the network interface of the database, and is used to obtain the access request of the database through the network interface and / or the feedback data of the database based on the access request; If the capture component is a proxy component, the proxy component is set on the path between the database end and the client end, forwards the data between the two, and saves the forwarded data to obtain the database access request and / or the database feedback data based on the access request.

4. The method for updating an asset catalog with sensitive data marking according to claim 1, characterized in that: The type of the database includes but is not limited to MySQL or PostgreSQL. The step of parsing the database access request and / or feedback data based on the type of the database includes: If the type of the database is MySQL, the MySQL parser is used for parsing; If the type of the database is PostgreSQL, the PostgreSQL parser is used for parsing.

5. The method for updating an asset catalog with sensitive data marking according to any one of claims 1 to 4, characterized in that: The content of the database update information includes metadata information of the changed database, and the metadata information includes database name / schema name / table name / column name, each database includes multiple schemas, each schema includes multiple tables, and each table includes multiple columns; The initial data directory is updated based on the content of the database update information, and the step of obtaining the updated data directory includes: Based on the database name / schema name / table name / column name in the metadata information in the database update information, the database name / schema name / table name / column name in the initial data directory is updated to obtain an updated data directory.

6. The method for updating an asset catalog with sensitive data marking according to claim 5, characterized in that: If the content of the database update information is to add / subtract a table or to add / subtract a column in an existing table, then the name of the table is added / subtracted or the name of the column in the existing table is added / subtracted in the initial data directory.

7. The method for updating an asset catalog with sensitive data marking according to claim 1, characterized in that: The database includes a user table and a system table. The system table is used to record the database where each user table is located, the mode where it is located, the name of the user table and the column name of each column in the user table. The initial data directory is updated based on the content of the database update information. The step of obtaining the updated data directory includes: The change information about the system table in the content of the database update information is obtained, and the initial data directory is updated based on the change information about the system table to obtain an updated data directory.

8. An asset catalog update device with sensitive data marking, characterized in that: The apparatus comprises a computer device, wherein the computer device comprises a processor and a memory, wherein the memory stores computer instructions, and the processor is used to execute the computer instructions stored in the memory. When the computer instructions are executed by the processor, the apparatus implements the steps of the method as described in any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images