Data tracking methods, apparatus, equipment and computer storage media

Abstract

This invention provides a data tracking method, apparatus, device, and computer storage medium. The method includes: filtering target data from a database to be monitored based on preset matching rules; acquiring multiple computing tasks received by the database to be monitored, each computing task including a starting data node and an ending data node participating in the computation; connecting nodes with the same node name among the starting and ending data nodes included in all computing tasks to obtain data lineage; for each starting data node, if the data carried by the starting data node includes a data tag, adding the data tag to the ending data node corresponding to the starting data node based on the data lineage; filtering out starting and ending data nodes including data tags from the data lineage to obtain data flow path information of the target data. This application embodiment can improve data security.

Description

Technical Field

[0001] This invention belongs to the field of data management technology, and in particular relates to a data tracking method, apparatus, device, and computer storage medium. Background Technology

[0002] Data lineage, as an abstract concept, is used to describe the relationships between data. In data governance and value mining scenarios, operators need to use data lineage to focus on the upstream and downstream information of data resources, that is, where the data comes from, which downstream users continue to use it, and the downstream outputs.

[0003] Existing technologies primarily construct data lineages by extracting field features and then perform node tracing analysis based on table-level or field lineage graphs. However, in the actual data lifecycle of information systems, data undergoes processes such as data retrieval, transformation, and storage in other tables through data task loading. If the data in these tables is not continuously tracked, the transformed data is highly likely to be misused or inadvertently leaked. Therefore, existing data tracking methods have poor ability to continuously track and mark data based on lineage, resulting in low data security. Summary of the Invention

[0004] This application provides a data tracking method, apparatus, device, and computer storage medium to address the problem of poor data security due to weak continuous tracking and tagging capabilities.

[0005] In a first aspect, embodiments of this application provide a data tracking method, including:

[0006] Based on preset matching rules, target data is filtered out from the database to be monitored, and the target data carries preset data tags;

[0007] Obtain multiple computing tasks received by the database to be monitored, each computing task including a starting data node and an ending data node participating in the computing;

[0008] Connect the nodes with the same name among the starting and ending data nodes of all the computing tasks to obtain the data lineage, which includes the initial flow path of the data called by each computing task;

[0009] For each starting data node, if the data carried by the starting data node includes the data tag, the data tag is added to the ending data node corresponding to the starting data node based on the data lineage.

[0010] By filtering out the starting and ending data nodes, including the data markers, from the data lineage, the data flow path information of the target data is obtained.

[0011] Secondly, embodiments of this application provide a data tracking device, the device comprising:

[0012] The tagging module is used to filter out target data from the database to be monitored based on preset matching rules. The target data carries preset data tags.

[0013] The first acquisition module is used to acquire multiple computing tasks received by the database to be monitored, each computing task including a starting data node and an ending data node participating in the computing;

[0014] A connection module is used to connect nodes with the same node name among the starting data nodes and ending data nodes of all the computing tasks to obtain data lineage, wherein the data lineage includes the initial flow path of the data called by each computing task;

[0015] An adding module is used to add the data tag to the corresponding termination data node based on the data lineage, for each starting data node, when the data carried by the starting data node includes the data tag;

[0016] The filtering module is used to filter out the starting data node and the ending data node including the data marker in the data lineage to obtain the data flow path information of the target data.

[0017] Thirdly, embodiments of this application provide a terminal device, the device including: a processor and a memory storing computer program instructions;

[0018] The processor implements the data tracking method as described in the first aspect when executing computer program instructions.

[0019] Fourthly, embodiments of this application provide a computer storage medium on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the data tracking method as described in the first aspect is implemented.

[0020] Fifthly, embodiments of this application provide a computer program product, characterized in that, when the instructions in the computer program product are executed by the processor of an electronic device, the electronic device performs the data tracking method as described in the first aspect.

[0021] The data tracking method, apparatus, device, and computer storage medium of this invention acquire multiple computing tasks received by the database under supervision, connect nodes with the same node name among the starting and ending data nodes of all the computing tasks to obtain data lineage, and filter target data in the database under supervision based on preset matching rules, and add data tags to the target data, thereby enabling the marking of target data in the database under supervision. Furthermore, when the data carried by the starting data node is target data, the data tag of the target data is added to the ending data node based on the data lineage, enabling the data tag to be passed from upstream to downstream of the data, preventing the target data from becoming untraceable after undergoing processes such as data retrieval, transformation, and storage in other tables. By filtering the starting and ending data nodes including the data tag from the data lineage, data flow path information is obtained, enabling the filtering of the target data flow path, improving the ability to continuously track and mark data, and thus improving data security. Attached Figure Description

[0022] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a schematic diagram of the overall process of the data tracking method provided in the embodiments of this application;

[0024] Figure 2 This is a schematic diagram of the data lineage provided in the embodiments of this application;

[0025] Figure 3 This is a schematic diagram of data tag transmission in an example provided in an embodiment of this application;

[0026] Figure 4 This is a schematic diagram of data flow in an example provided in this application embodiment;

[0027] Figure 5 This is a schematic diagram of a process for determining access risks, provided in an example of an embodiment of this application.

[0028] Figure 6 This is a schematic diagram of the structure of the data tracking device provided in the embodiments of this application;

[0029] Figure 7 This is a schematic diagram of the structure of the terminal device provided in the embodiments of this application. Detailed Implementation

[0030] The features and exemplary embodiments of various aspects of this application will be described in detail below. To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain this application and not to limit it. For those skilled in the art, this application can be implemented without some of these specific details. The following description of the embodiments is merely to provide a better understanding of this application by illustrating examples.

[0031] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0032] To address the problems of the prior art, embodiments of this application provide a data tracking method, apparatus, device, and computer storage medium. The following first illustrates examples of the data tracking method applicable to embodiments of this application.

[0033] Figure 1 A schematic flowchart of a data tracking method according to an embodiment of this application is shown. This data tracking method can be applied to electronic devices, such as… Figure 1 As shown, the method includes:

[0034] Step 101: Based on preset matching rules, target data is filtered out from the database to be monitored, and the target data carries preset data tags;

[0035] Step 102: Obtain multiple computing tasks received by the database to be monitored, each computing task including a starting data node and an ending data node participating in the computing;

[0036] Step 103: Connect the nodes with the same node name among the starting data nodes and ending data nodes of all the computing tasks to obtain the data lineage relationship, which includes the initial flow path of the data called by each computing task;

[0037] Step 104: For each starting data node, if the data carried by the starting data node includes the data tag, add the data tag to the ending data node corresponding to the starting data node based on the data lineage.

[0038] Step 105: Filter out the starting data node and ending data node including the data marker from the data lineage to obtain the data flow path information of the target data.

[0039] In step 101, target data is filtered out from the database to be monitored based on preset matching rules, and the target data carries preset data tags;

[0040] The target data mentioned above can be sensitive data, such as mobile phone numbers, International Mobile Subscriber Identity Serial Numbers (IMSIs), identity verification documents, service subscriptions, and smart terminal device information. The matching rules mentioned above can include regular expression matching, string sample library matching, or file template matching. The matching rules are used to identify the target data in the database to be regulated. The data tags mentioned above can be tags added to the target data after it has been matched, used to distinguish the target data from the rest of the data in the database to be regulated.

[0041] In step 102, multiple computing tasks received by the database to be monitored are obtained, and each computing task includes a starting data node and an ending data node participating in the computing.

[0042] In this embodiment, the computing task can be a data retrieval task or a query task sent by an external device to the data to be monitored.

[0043] In one embodiment, step 102 further includes:

[0044] Obtain the access statement and data computation task name for each of the multiple computation tasks received by the regulatory database;

[0045] Parse the access statement and data computation task name of each computation task to obtain the start data name and end data name of each computation task;

[0046] The start data name and end data name of each computing task are stored in a binary structure to obtain the start data node and end data node of each computing task.

[0047] The above process can be achieved by creating a parsing component, which then retrieves the computation tasks. Specifically, during the retrieval of computation tasks, since the database to be monitored requires access through data warehouse tools, the event monitoring mechanism of the data warehouse tools can be utilized to obtain the computation tasks. For example, when data warehouse tools such as Spark and Flink are accessed, it indicates that the database to be monitored has been accessed. Therefore, the plugin can be configured to send the SQL access statements and Flink engine messages from the data warehouse tools to a message queue. The parsing component then retrieves these SQL access statements and Flink engine messages from the message queue, thereby obtaining the computation tasks for the database to be monitored. These SQL access statements and Flink engine messages constitute the computation tasks. It should be noted that there can be multiple computation tasks, and each computation task can contain multiple pairs of start and end data names.

[0048] After the parsing component obtains the computation task, it parses the computation task to obtain the start data name and end data name corresponding to the computation task. Finally, the start data name and end data name are stored in a binary structure to obtain the start data node corresponding to the start data name and the end data node corresponding to the end data name.

[0049] The aforementioned starting data name can be the data to be called by the calculation task, specifically a field name, table name, or database name. The ending data name can be the calculation result of the calculation task, specifically a field name, table name, or database name. It should be noted that the data tag in step 101 can be a tag on any one of the field name, table name, or database name of the data.

[0050] In one example, the calculation task requires data A, which is stored in Table 1 of the database to be monitored, and the calculation result is C, which is stored in Table 2 of the database to be monitored. In this case, the starting data name is Table 1 and the ending data name is Table 2. Alternatively, the calculation task requires data A and B, where B is stored in Table 3 of the database and the calculation result is C. In this case, the starting data names are Table 1 and Table 3, and the ending data name is Table 2.

[0051] In step 103, nodes with the same name among the starting and ending data nodes of all the computing tasks are connected to obtain data lineage, which includes the initial flow path of the data called by each computing task.

[0052] In this embodiment, there may be multiple starting data nodes and multiple ending data nodes. Each node's attributes include information such as data node name, classification / level sensitive data markers, affiliated application system, cluster location of the resource, vendor, or tenant. When the ending data node of one computing task is the starting data node of another computing task, the ending data node and the starting data node will have the same node name. Therefore, the ending data node and the starting data node can be connected. This step is performed for all computing tasks to obtain the data lineage.

[0053] In one example, there are computation tasks A and B. The starting and ending data nodes of computation task A are 1 and 2, respectively, while the starting and ending data nodes of computation task B are 2 and 3, respectively. Thus, the ending data node of computation task A can be connected to the starting data node of computation task B to obtain the data lineage. Through this data lineage, it can be determined that the data flow path is from node 1 to node 2, and then to node 3.

[0054] like Figure 2 As shown, blood relations can be represented by a diagram. Figure 2 In the diagram, "ship" in the first column and "dp-101" in the second column can be a starting and ending data node pair. Similarly, "dp-101" and "dp-101-1min-avg" in the third column can be a starting and ending data node pair. Furthermore, "dp-101-1min-avg" and "cleaned-data" in the fourth column can be a starting and ending data node pair. Therefore, the data flow path sequentially passes through the nodes: ship, dp-101, dp-101-1min-avg, and cleaned-data. These nodes constitute... Figure 2 A complete flow path within the data. Data lineage can include a network of flow relationships composed of the aforementioned flow paths.

[0055] In step 104, for each starting data node, if the data carried by the starting data node includes the data tag, the data tag is added to the ending data node corresponding to the starting data node based on the data lineage.

[0056] In this embodiment, to prevent the data from becoming unlocatable due to deformation or reassembly during transmission, the target data needs to be tracked. Therefore, as the target data is transmitted from the starting data node to the ending data node, a data tag can be simultaneously passed to the ending data node. In other words, in this embodiment, the data tag can be passed along with the source data in the database to be monitored to various intermediate or result data generated from the source data within the data lineage, so that the target data can be tracked subsequently by tracking the data tag.

[0057] If the starting data node carries data including the data tag, it indicates that the starting data node carries target data. This target data may be obtained by directly filtering source data in the database to be monitored, or it may be source data that has been transferred to other starting data nodes. To achieve data tag inheritance, the data tags from all starting data nodes can be added to the corresponding ending data nodes to complete the data tag transfer.

[0058] In step 105, the starting data node and the ending data node including the data tag are filtered out from the data lineage to obtain the data flow path information of the target data.

[0059] In one embodiment, step 105 further includes:

[0060] By filtering out the starting and ending data nodes that share the same sensitivity level from the data lineage, data flow path information is obtained.

[0061] In this embodiment, since data tags include data sensitivity levels, starting and ending data nodes with the same sensitivity level indicate that they have the same data tag. By filtering out starting and ending data nodes with the same sensitivity level, the flow path of the target data corresponding to the data tag can be obtained, thereby obtaining data flow path information.

[0062] In one embodiment, it should be noted that the data flow path information may include a data flow path diagram or a textual description of the data flow path. The data flow path diagram is a directional graph containing the flow path. To better distinguish and view the flow path of the target data, data nodes that are not part of the target data's data lineage can be hidden in the data flow path diagram, thereby reducing the number of nodes in the diagram. Alternatively, data nodes containing the target data can be marked with different colors to better distinguish the flow path of the target data.

[0063] In this embodiment, by acquiring multiple computing tasks received by the database under supervision, and connecting nodes with the same node name among the starting and ending data nodes of all the computing tasks, data lineage is obtained. Based on preset matching rules, target data is filtered out in the database under supervision, and data tags are added to the target data, enabling the marking of target data in the database under supervision. Furthermore, when the data carried by the starting data node is target data, the data tag of the target data is added to the ending data node based on the data lineage, enabling the data tag to be passed from upstream to downstream of the data. This prevents the target data from becoming untraceable after undergoing processes such as data retrieval, transformation, and storage in other tables. By filtering out the starting and ending data nodes including the data tag from the data lineage, data flow path information is obtained, enabling the filtering of the target data flow path, improving the ability to continuously track and mark data, and thus enhancing data security.

[0064] In one embodiment of this application, step 101 further includes:

[0065] A scanning task is created using a preset data tagging engine. The scanning task includes the database to be monitored, the matching rules, and the scanning cycle.

[0066] The scanning task is executed to obtain scanning results, which include intermediate data obtained by the data tagging engine from scanning the database to be monitored within the scanning cycle based on the matching rules.

[0067] The intermediate data is labeled with the data tag to obtain the target data.

[0068] In this embodiment, the engine performs the following steps for data marking: First, the engine program configures a set of data template matching rules for sensitive information in the communications field, such as mobile phone numbers, IMSI, identity verification, service subscriptions, and smart terminal device information. The matching rules include regular expression matching, string sample library matching, and file template matching. Second, for the compliant and regulated database, the engine creates a data content scanning task. The scanning task includes parameters such as the object to be scanned, the matching rules, and the scanning period. Third, after the scanning task is completed, the data that matches the matching rules is marked as target data in the scanning results, and data tags are added to the metadata storage area. Fourth, within the scope of the regulated database, the data marking engine periodically executes scanning tasks to identify the storage distribution information of target data in each table of the database.

[0069] The sensitive data marking elements mentioned in the third step include: resource domain ID, business ID, category ID, sensitivity level ID, and random number, in the form of: resource domain ID_business ID_category ID_sensitivity level ID_random number.

[0070] A globally unique asset management tag, or data tag, is added to the data. During the subsequent lifecycle of the data, including storage, circulation, calculation, open sharing, and destruction, applications accessing the data asset can obtain information such as the data's ownership domain, responsible party, data type, and sensitivity level by parsing the tag value. This designed data attribute serves as a classification and grading tag for sensitive data, and its structure is shown in Table 1 below.

[0071] sourceid projectid classifyid sensitivityid randomnum

[0072] Table 1

[0073] In Table 1, the attributes are, in order, resource domain ID, business ID, category ID, sensitivity level ID, and random number.

[0074] In this embodiment, by adding data tags to the target data, the target data can be filtered, and the target data can be tracked through the data tags, thereby improving the tracking and positioning capabilities of the target data.

[0075] In one embodiment of this application, after step 104 and before step 105, the data tracking method further includes:

[0076] In the case where the termination data node has multiple corresponding starting data nodes, the sensitivity level of the data tag in the multiple starting data nodes is obtained;

[0077] Add the data tag with the higher sensitivity level from the multiple starting data nodes to the ending data node.

[0078] In this embodiment, the computation task may obtain the computation result by calling two data. In this case, there are two starting data nodes and one ending data node. When the data carried by the two starting data nodes are both target data, the ending data node only inherits the data tag with the higher sensitivity level from the data tags corresponding to the two target data.

[0079] In one example, assuming the target data is sensitive data and the data is marked with a sensitive tag, please refer to [link to relevant documentation]. Figure 3After the data tagging engine scans the database, the original data fields A, B, and D that match the rules are detected and marked as sensitive data by the engine, with sensitivity levels (sensitivityid) of level 1, level 2, and level 3, respectively. After the parsing component draws the sensitive data flow path based on the data lineage, the sensitivityid tag is passed downstream (i.e., the data tag is passed to the terminating data node) using the sensitive data as the anchor point. According to the sensitivity tag inheritance mechanism, with the lineage flow, field C, the common downstream node of fields A and B, inherits the relatively high sensitivity level between its upstream nodes, i.e., level 2; field E, the common downstream node of fields C and D, inherits the relatively high sensitivity level between its upstream nodes, i.e., level 3. Simultaneously, the sensitivity level of the table is set to be consistent with the field with the highest sensitivity level. Therefore, the sensitivity level of Table 1 is level 2, the sensitivity level of Table 2 is level 2, and the sensitivity level of Table 3 is level 3. In the above embodiment, Table 1, Table 2, and Table 3 are all corresponding starting or ending data nodes.

[0080] In this embodiment, by passing the data marker with a higher sensitivity level in the starting data node to the ending data node, it is ensured that while the data marker is passed to the downstream node of the target data, the sensitivity level of the target data will not change, thereby improving the tracking and positioning capability of the target data.

[0081] In one embodiment of this application, after filtering out the starting data node and ending data node including the data marker from the data lineage to obtain the data flow path information of the target data, the method further includes:

[0082] Based on the data flow path information, the sensitive data in the database to be monitored and the sensitivity level of the sensitive data are determined. The sensitive data is data carrying a data tag.

[0083] Obtain the user access logs of the database to be monitored;

[0084] In the event of data transfer of the sensitive data, the sensitivity level corresponding to the sensitive data and the user access log are matched with the preset security compliance policy to determine the data transfer risk of the sensitive data.

[0085] In this embodiment, since the database under supervision may interact with external systems based on various computing tasks, such as data outflow or data inflow, in order to reduce the risk of data transfer between the database under supervision and external systems, this embodiment needs to monitor the risk of data transfer between the database under supervision and external systems.

[0086] Specifically, the data carried by each node in the data flow path diagram is the sensitive data in the database to be monitored, which carries data tags. These data tags include a sensitivity level; therefore, when monitoring the flow risks of the database to be monitored, only this sensitive data needs to be monitored. User access logs can include data outflow and inflow records of the database to be monitored, as well as host information accessing the database, such as the host's security level. It should be noted that user access logs are acquired in real time, therefore data flow risks are also monitored in real time.

[0087] In one example, see Figure 3 , Figure 3 This is a diagram illustrating the data flow between the database to be monitored and external systems. The external systems may include host node 1, host node 2, and host node 3, while the data acquisition, exchange, and storage area contains the sensitive data to be exchanged.

[0088] The aforementioned security compliance strategies include data security compliance flow strategies and user security compliance access to sensitive data strategies. In this embodiment, if the sensitivity level and user access logs comply with the aforementioned security compliance strategies, then the corresponding node is determined to have data flow risks.

[0089] The data security compliance flow strategy specifically includes:

[0090] (1) Data traffic exit policy: Sensitive data is included in the data resource filing list that is permitted to be opened outside the system;

[0091] (2) Data traffic admission policy: The target host IP of real-time traffic inflow is in the host asset IP list of the data subscription user;

[0092] The user security and compliance access policy for sensitive data specifically includes: a data leakage prevention and protection policy for data transfer, specifically:

[0093] More sensitive data is only allowed to flow into high-security hosts;

[0094] Sensitive data is only allowed to flow into hosts with medium or higher security levels;

[0095] Data of a generally sensitive level is allowed to flow into hosts with a normal or higher security level.

[0096] Among them, the more sensitive level data can be the level 3 data in the above embodiments, the sensitive level data can be the level 2 data in the above embodiments, and the general sensitive level data can be the level 1 data in the above embodiments.

[0097] In one embodiment, after determining the data transfer risk of the sensitive data by matching the sensitivity level corresponding to the sensitive data and the user access log with a preset security compliance policy when the sensitive data undergoes data transfer, the method further includes:

[0098] If there is a risk of data transfer for the sensitive data, the risk of data transfer will be marked on the data node corresponding to the sensitive data in the data transfer path diagram.

[0099] In this embodiment, in order to facilitate the viewing of risks during the data transfer process, nodes with transfer risks can be marked by using different warning colors, or risks can be marked by text. This application does not impose any restrictions on this.

[0100] In this embodiment, by identifying the transfer risks of each data node and marking the transfer risks, the security of the data transfer process can be improved.

[0101] In one embodiment of this application, after filtering out the starting data node and ending data node including the data marker from the data lineage to obtain the data flow path information of the target data, the method further includes:

[0102] Obtain the user access logs of the database to be monitored;

[0103] Extract the access objects and access behaviors of each user account from the user access logs;

[0104] When the access target is data with data tags in the database to be monitored, the access risk of the user account is determined based on the preset permission information of each user account, the access target, and the access behavior.

[0105] In this embodiment, in order to prevent the leakage of sensitive data in the database under supervision when the database under supervision is accessed, it is necessary to monitor the access risk of each user account accessing the database under supervision.

[0106] Specifically, the access risks of the aforementioned user accounts include the risk of unauthorized access to data resources, the risk of non-sensitive users accessing sensitive resources, and the risk of accessing sensitive data without vault approval. The access targets include sensitive data accessed by the user; the access behavior includes access behavior that triggers vault approval when the user account accesses the database; and the preset permission information includes the user account's resource permission list, the user's sensitivity level, and the vault approval policy for accessing sensitive data.

[0107] In the process of determining access risks, the risks of non-sensitive users accessing sensitive resources, users accessing data resources without authorization, and accessing sensitive data without the approval of the vault are determined in sequence.

[0108] First, identify the risks of non-sensitive users accessing sensitive resources, specifically including:

[0109] Step 1: Based on the user's sensitivity level permissions, determine whether the user account is a sensitive account with a sensitive level of the target data;

[0110] Step 2: If the user account is not a sensitive account at the target data sensitivity level, determine that the user account has a non-sensitive account operating sensitive data.

[0111] Secondly, identify the risk of users accessing data resources without authorization, specifically including:

[0112] Step 1: If the user account is a sensitive account with a sensitive level for the target data, determine whether the user account has access to the target data based on the resource permission list;

[0113] Step 2: If the user account does not have access to the target data, determine that the user account is performing unauthorized operations involving sensitive data.

[0114] Finally, the risk of accessing sensitive user account data without vault approval was identified, specifically including:

[0115] Step 1: If the user account has access to the target data, determine whether the vault approval log should be used to access sensitive data when the user account receives the successful vault authorization log for that account.

[0116] Step 2: If a user account accesses sensitive data without receiving the successful vault authorization log for that account, it is determined that the user account has a risk of violating vault access rules.

[0117] In this embodiment, database security is improved by identifying access risks associated with user accounts.

[0118] Figure 6 A schematic diagram of the structure of a data tracking device provided in one embodiment of this application is shown. For ease of explanation, only the parts related to the embodiment of this application are shown.

[0119] Reference Figure 6 The data tracking device 600 may include:

[0120] The tagging module 601 is used to filter out target data in the database to be monitored based on preset matching rules, wherein the target data carries preset data tags;

[0121] The first acquisition module 602 is used to acquire multiple computing tasks received by the database to be monitored, each computing task including a starting data node and an ending data node participating in the computing;

[0122] The connection module 603 is used to connect nodes with the same node name among the starting data nodes and ending data nodes of all the computing tasks to obtain data lineage, wherein the data lineage includes the initial flow path of the data called by each computing task;

[0123] Adding module 604 is used to add the data tag to the corresponding termination data node based on the data lineage, when the data carried by the starting data node includes the data tag for each starting data node.

[0124] The filtering module 605 is used to filter out the starting data node and the ending data node including the data marker in the data lineage relationship, and obtain the data flow path information of the target data.

[0125] Optionally, the marking module 601 further includes:

[0126] A submodule is created to create a scanning task using a preset data tagging engine. The scanning task includes the database to be monitored, the matching rules, and the scanning cycle.

[0127] An execution submodule is used to execute the scanning task and obtain scanning results, the scanning results including intermediate data obtained by the data tagging engine based on the matching rules during the scanning cycle from the database to be monitored;

[0128] The first acquisition submodule is used to add the data tag to the intermediate data to obtain the target data.

[0129] Optionally, the first acquisition module 602 further includes:

[0130] The second acquisition submodule is used to acquire the access statement and data calculation task name of each of the multiple computing tasks received by the regulatory database.

[0131] The parsing submodule is used to parse the access statement and data calculation task name of each computing task to obtain each pair of start data name and end data name of each computing task.

[0132] The storage submodule is used to store each pair of start data names and end data names of each computing task in a binary structure to obtain the start data node and end data node of each computing task.

[0133] Optionally, the data tracking device 600 further includes:

[0134] The second acquisition module is used to acquire the sensitivity level of the data tag in the multiple starting data nodes when the termination data node has multiple corresponding starting data nodes;

[0135] An adding module is used to add the data marker with the higher sensitivity level among the multiple starting data nodes to the ending data node.

[0136] Optionally, the filtering module 605 is specifically used for:

[0137] By filtering out the starting and ending data nodes that share the same sensitivity level from the data lineage, data flow path information is obtained.

[0138] Optionally, the data tracking device 600 further includes:

[0139] The third acquisition module is used to acquire the sensitivity level of each data node in the data flow path information and the user access logs of the database to be monitored.

[0140] The first determination module is used to determine the data flow risk of each data node based on the sensitivity level, the user access logs, and the preset security compliance policy.

[0141] Optionally, the data tracking device 600 further includes:

[0142] The annotation module is used to annotate the data flow risk of each node in the data flow path information.

[0143] Optionally, the data tracking device 600 further includes:

[0144] The fourth acquisition module is used to acquire user access logs of the database to be monitored;

[0145] The extraction module is used to extract the access objects and access behaviors of each user account in the user access log;

[0146] The second determining module is used to determine the access risk of a user account based on the preset permission information of each user account, the access object, and the access behavior.

[0147] It should be noted that the data tracking device 600 is a device corresponding to the data tracking method described above. All implementation methods in the above method embodiments are applicable to the embodiments of this device and can achieve the same technical effect.

[0148] Figure 7A schematic diagram of the hardware structure of the terminal device provided in an embodiment of this application is shown.

[0149] The terminal device may include a processor 701 and a memory 702 storing computer program instructions.

[0150] Specifically, the processor 701 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0151] Memory 702 may include mass storage for data or instructions. For example, and not limitingly, memory 702 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 702 may include removable or non-removable (or fixed) media. Where appropriate, memory 702 may be internal or external to the integrated gateway disaster recovery device. In a particular embodiment, memory 702 is non-volatile solid-state memory.

[0152] In a particular embodiment, memory 702 may include read-only memory (ROM), random access memory (RAM), disk storage media device, optical storage media device, flash memory device, electrical, optical, or other physical / tangible memory storage device. Thus, generally, memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to one aspect of this disclosure.

[0153] The processor 701 implements any of the data tracking methods described in the above embodiments by reading and executing computer program instructions stored in the memory 702.

[0154] In one example, the terminal device may also include a communication interface 703 and a bus 710. Wherein, as... Figure 7 As shown, the processor 701, memory 702, and communication interface 703 are connected through bus 710 and complete communication with each other.

[0155] The communication interface 703 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of this application.

[0156] Bus 710 includes hardware, software, or both, that couples components of an online data traffic metering device together. For example, and not limitingly, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 710 may include one or more buses. Although specific buses are described and illustrated in embodiments of this application, any suitable bus or interconnect is contemplated herein.

[0157] Furthermore, in conjunction with the data tracking methods in the above embodiments, this application embodiment can provide a computer storage medium for implementation. The computer storage medium stores computer program instructions; when these computer program instructions are executed by a processor, they implement any of the data tracking methods in the above embodiments.

[0158] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.

[0159] The functional blocks shown in the above block diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. Programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0160] It should also be noted that the exemplary embodiments mentioned in this application describe methods or systems based on a series of steps or apparatus. However, this application is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0161] The aspects of this disclosure have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data tracking device to produce a machine such that these instructions, executable via the processor of the computer or other programmable data tracking device, enable the implementation of the function / action specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by special-purpose hardware performing the specified function or action, or can be implemented by a combination of special-purpose hardware and computer instructions.

[0162] The above are merely specific embodiments of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the protection scope of this application.

Claims

1. A data tracking method, characterized in that, The methods include: Based on preset matching rules, target data is filtered out from the database to be monitored, and the target data carries preset data tags; Obtain the access statement and data computation task name for each of the multiple computation tasks received by the regulatory database; Parse the access statement and data computation task name of each computation task to obtain each pair of start data name and end data name for each computation task; Store each pair of start data names and end data names of each computing task in a binary structure to obtain the start data node and end data node of each computing task. Connect the nodes with the same name among the starting and ending data nodes of all the computing tasks to obtain the data lineage, which includes the initial flow path of the data called by each computing task; For each starting data node, if the data carried by the starting data node includes the data tag, the data tag is added to the ending data node corresponding to the starting data node based on the data lineage. The starting data node and ending data node, including the data marker, are selected from the data lineage to obtain the data flow path information of the target data. The data flow path information includes a data flow path diagram. In the data flow path diagram, data nodes that do not have the data marker are hidden, or data nodes that have the data marker are marked with different colors.

2. The data tracking method as described in claim 1, characterized in that, The process of filtering target data from the database to be monitored based on preset matching rules includes: A scanning task is created using a preset data tagging engine. The scanning task includes the database to be monitored, the matching rules, and the scanning cycle. The scanning task is executed to obtain scanning results, which include intermediate data obtained by the data tagging engine from scanning the database to be monitored within the scanning cycle based on the matching rules. The intermediate data is labeled with the data tag to obtain the target data.

3. The data tracking method as described in claim 1, characterized in that, The data tag format includes: resource domain, business, category, sensitivity level, and random number.

4. The data tracking method as described in claim 3, characterized in that, After adding the data tag to the termination data node corresponding to the starting data node, and before filtering out the starting data node and termination data node including the data tag from the data lineage to obtain the data flow path information of the target data, the method further includes: In the case where the termination data node has multiple corresponding starting data nodes, the sensitivity level of the data tag in the multiple starting data nodes is obtained; Add the data tag with the higher sensitivity level from the multiple starting data nodes to the ending data node.

5. The data tracking method as described in claim 4, characterized in that, The step of filtering out the starting and ending data nodes, including the data marker, from the data lineage to obtain the data flow path information of the target data includes: By filtering out the starting and ending data nodes with the same sensitivity level from the data lineage, the data flow path information is obtained.

6. The data tracking method as described in claim 1, characterized in that, After filtering out the starting and ending data nodes including the data markers from the data lineage to obtain the data flow path information of the target data, the method further includes: Based on the data flow path information, the sensitive data in the database to be monitored and the sensitivity level of the sensitive data are determined. The sensitive data is data carrying a data tag. Obtain the user access logs of the database to be monitored; In the event of data transfer of the sensitive data, the sensitivity level corresponding to the sensitive data and the user access log are matched with the preset security compliance policy to determine the data transfer risk of the sensitive data.

7. The data tracking method as described in claim 6, characterized in that, In the event of data transfer of the sensitive data, after matching the sensitivity level corresponding to the sensitive data and the user access log with the preset security compliance policy to determine the data transfer risk of the sensitive data, the method further includes: If there is a risk of data transfer for the sensitive data, the risk of data transfer will be marked on the data node corresponding to the sensitive data in the data transfer path information.

8. The data tracking method as described in claim 1, characterized in that, After filtering out the starting and ending data nodes including the data markers from the data lineage to obtain the data flow path information of the target data, the method further includes: Obtain the user access logs of the database to be monitored; Extract the access objects and access behaviors of each user account from the user access logs; When the access target is data with data tags in the database to be monitored, the access risk of the user account is determined based on the preset permission information of each user account, the access target, and the access behavior.

9. A data tracking device, characterized in that, The device includes: The tagging module is used to filter out target data from the database to be monitored based on preset matching rules. The target data carries preset data tags. The first acquisition module is used to acquire the access statement and data calculation task name of each of the multiple computing tasks received by the regulatory database. Used to parse the access statement and data computation task name of each computation task to obtain each pair of start data name and end data name for each computation task; This is used to store each pair of start data names and end data names of each computing task in a binary structure, thereby obtaining the start data node and end data node of each computing task. A connection module is used to connect nodes with the same node name among the starting data nodes and ending data nodes of all the computing tasks to obtain data lineage, wherein the data lineage includes the initial flow path of the data called by each computing task; An adding module is used to add the data tag to the corresponding termination data node based on the data lineage, for each starting data node, when the data carried by the starting data node includes the data tag; The filtering module is used to filter out the starting data node and the ending data node including the data marker in the data lineage relationship, and obtain the data flow path information of the target data, wherein the data flow path information includes a data flow path diagram; Data differentiation module: used to hide data nodes that do not have the data marker in the data flow path diagram, or to mark data nodes that have the data marker with different colors.

10. A terminal device, characterized in that, The device includes: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, it implements the data tracking method as described in any one of claims 1-8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions that, when executed by a processor, implement the data tracking method as described in any one of claims 1-8.

Images