Method and device for constructing data genealogy, medium, equipment and product

Abstract

Embodiments of the present specification provide a data bloodline graph construction method, a data bloodline graph construction device, a computer readable storage medium, an electronic device and a computer program product, the method comprising: determining target data to be detected and a detection node for detecting the target data, wherein the detection node is located in a target application program or in the target application program and a target database. In the running process of the target application program, the detection nodes in the target data propagation path are determined in all detection nodes to obtain an actual propagation node sequence of the target data, wherein the detection nodes in the actual propagation node sequence have an upstream and downstream relationship about the target data propagation path. Further, according to the actual propagation node sequence of the target data and the upstream and downstream relationship about the target data propagation path contained therein, a data bloodline graph about the target data can be determined.

Description

Technical Field

[0001] This specification relates to the field of data processing technology, and in particular to a method for constructing a data lineage map, a device for constructing a data lineage map, a computer-readable storage medium, an electronic device, and a computer program product. Background Technology

[0002] A data lineage graph refers to the relationships formed between data points during their generation, processing, flow, and eventual disappearance, similar to kinship in human society. Data lineage products provided by related technologies generally specifically refer to lineage graphs between data within an offline data warehouse, and do not cover data lineage graphs generated during application runtime. To further enhance data management, there is an urgent need in related technologies for a data lineage graph construction scheme applicable to the application runtime process.

[0003] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this specification, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0004] The purpose of this specification is to provide a method for constructing a data lineage map, a device for constructing a data lineage map, a computer-readable storage medium, an electronic device, and a computer program product, and to provide a data lineage map construction scheme suitable for application operation, which at least to a certain extent improves the accuracy and efficiency of data management.

[0005] Other features and advantages of this specification will become apparent from the following detailed description, or may be learned in part by practice of this specification.

[0006] According to one aspect of this specification, a method for constructing a data lineage graph is provided. The method includes: determining target data and determining detection nodes related to the target data, wherein the detection nodes are located in a target application, or the detection nodes are located in both the target application and a target database; during the operation of the target application, determining detection nodes in the propagation path of the target data among all detection nodes to obtain an actual propagation node sequence of the target data, wherein there are upstream and downstream relationships between the detection nodes in the actual propagation node sequence related to the propagation path of the target data; and determining a data lineage graph of the target data based on the actual propagation node sequence of the target data.

[0007] According to another aspect of this specification, an apparatus for constructing a data lineage map is provided, the apparatus comprising: a first determining module, a second determining module, and a construction module.

[0008] The first determining module is configured to: determine target data and determine detection nodes related to the target data, wherein the detection nodes are located in the target application or in both the target application and the target database; the second determining module is configured to: during the operation of the target application, determine the detection nodes in the propagation path of the target data among all detection nodes to obtain the actual propagation node sequence of the target data, wherein there are upstream and downstream relationships between the detection nodes in the actual propagation node sequence regarding the propagation path of the target data; and the construction module is configured to: determine the data lineage map of the target data based on the actual propagation node sequence of the target data.

[0009] According to another aspect of this specification, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method for constructing a data lineage map as described in the above embodiments.

[0010] According to one aspect of this specification, a computer-readable storage medium is provided, wherein instructions are stored therein, which, when executed on a computer or processor, cause the computer or processor to perform the method for constructing a data lineage map as described in the above embodiments.

[0011] According to another aspect of this specification, a computer program product containing instructions is provided that, when the computer program product is run on a computer or processor, causes the computer or processor to perform the method for constructing a data lineage map as described in the above embodiments.

[0012] The data lineage mapping construction method, data lineage mapping construction apparatus, computer-readable storage medium, electronic device, and computer program product provided in the embodiments of this specification have the following technical effects:

[0013] In the exemplary embodiments provided in this specification, target data to be detected and detection nodes for detecting the target data are determined. These detection nodes are located within the target application, or within both the target application and the target database. Thus, the target data can be detected through the set detection nodes when it propagates within the application, between applications, or between the application and the database. During the operation of the target application, detection nodes along the propagation path of the target data are determined from all detection nodes, resulting in an actual propagation node sequence for the target data. The detection nodes in this actual propagation node sequence have upstream and downstream relationships along the propagation path of the target data. Furthermore, based on the actual propagation node sequence of the target data and the upstream and downstream relationships it contains, a data lineage graph of the target data can be determined. The technical solutions provided in the embodiments of this specification can conveniently construct a lineage graph of the dynamic flow of data during the operation of an application within a specified range, thereby intuitively and efficiently determining the source and destination of each target data, improving the accuracy and efficiency of data use and management.

[0014] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this specification. Attached Figure Description

[0015] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this specification and, together with the description, serve to explain the principles of this specification. It is obvious that the drawings described below are merely some embodiments of this specification, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0016] Figure 1 This is a flowchart illustrating a method for constructing a data lineage map according to an embodiment of this specification.

[0017] Figure 2 This is a schematic diagram of the target data to be detected and the node to be tested, provided as an embodiment of this specification.

[0018] Figure 3 This is a flowchart illustrating a method for determining a propagation node sequence according to an embodiment of this specification.

[0019] Figure 4 This is a schematic diagram of the propagation node paths for different target data provided in one embodiment of this specification.

[0020] Figure 5 This is a schematic diagram of a bloodline model provided in one embodiment of this specification.

[0021] Figure 6 This is a flowchart illustrating a method for constructing a data lineage map, provided as another embodiment of this specification.

[0022] Figure 7 This is a schematic diagram of the propagation node paths for different target data, provided as another embodiment of this specification.

[0023] Figure 8 This is a schematic diagram of the interface for a data lineage map provided in one embodiment of this specification.

[0024] Figure 9 This is a schematic diagram of the interface for a data lineage map provided in another embodiment of this specification.

[0025] Figure 10 A schematic diagram of a data lineage mapping construction apparatus that can be applied to an embodiment of this specification is shown.

[0026] Figure 11 A schematic diagram of a data lineage mapping construction apparatus is shown, which can be applied to another embodiment of this specification.

[0027] Figure 12 The schematic diagram illustrates the structure of an electronic device according to an exemplary embodiment of this specification. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of this specification clearer, the embodiments of this specification will be described in further detail below with reference to the accompanying drawings.

[0029] In the following description, when referring to the accompanying drawings, the same numbers in different drawings denote the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this specification. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this specification as detailed in the appended claims.

[0030] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided to make this specification more comprehensive and complete, and to fully convey the concept of example embodiments to those skilled in the art. The described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a full understanding of the embodiments described herein. However, those skilled in the art will recognize that the technical solutions described herein may be practiced with one or more of the specific details omitted, or other methods, components, apparatus, steps, etc., may be employed. In other instances, well-known technical solutions are not shown or described in detail to avoid obscuring various aspects of this specification.

[0031] Furthermore, the accompanying drawings are merely illustrative diagrams of this specification and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0032] Data lineage maps for offline data warehouses provided in related technologies are generally constructed based on static code analysis, specifically by analyzing the source code. However, due to the complexity of code design, there are scenarios where accurate analysis is impossible, resulting in data lineage maps with low accuracy.

[0033] This specification provides a method for constructing a data pedigree map, an apparatus for constructing a data pedigree map, a computer-readable storage medium, an electronic device, and a computer program product, which can solve the aforementioned problems existing in related technologies. Specifically, this specification first introduces an embodiment of the method for constructing a data pedigree map.

[0034] in, Figure 1 This is a schematic flowchart illustrating a method for constructing a data lineage map according to an embodiment of this specification. Exemplarily, the execution entity of the evaluation and detection method provided in this embodiment is a device with computing power, such as a server. (See reference...) Figure 1 The embodiment shown in the figure includes: S110-S140.

[0035] In S110, target data and detection nodes related to the target data are determined, wherein the detection nodes are located in the target application or in the target application and the target database.

[0036] The aforementioned applications are computer programs designed to perform one or more specific tasks. For example, to implement a user login account function, three applications are involved. Specifically, application A is responsible for front-end rendering (e.g., the URL of the login webpage, the content displayed on the login interface), application B is responsible for account login (e.g., application A calls the interface exposed by application B with login request, user account, and password information, and then application B performs account login), and application C is responsible for real-name authentication information (application B queries user information based on the account, then calls the interface of application C to ask if the user has been real-name authenticated, and further returns the query result to application B, then application B processes the query result and returns the login information to application A).

[0037] In an exemplary embodiment, the target data mentioned above is the data to be detected, which can be one or multiple. The lineage relationship of the target data is determined by the scheme provided in the embodiments of this specification, thereby facilitating the management of the target data. Assuming that the database field "mysqlM1-tableT1-fieldF1" is user privacy data, its use requires user authorization, it can be used as the target data to be detected in this embodiment. The lineage graph construction scheme provided in the embodiments of this specification is used to determine the lineage relationship of this data. For example, the lineage relationship can be used to determine that during the operation of related applications, the interface app2.interfaceI3 of the second application and the interface app3.interfaceI6 of the third application both use this data. Furthermore, these two interfaces need to be carefully managed to ensure that user authorization is obtained when using the data. The target data is determined according to actual needs in the embodiments of this specification. After determining the sub-model of the target data through the scheme provided in the embodiments of this specification, targeted management of the target data can be achieved, which is beneficial to improving the accuracy and efficiency of data management.

[0038] For example, refer to Figure 2 The target data 22 and target data 24 to be detected were identified, and detection nodes for detecting the aforementioned target data were also determined. To enable tracking of the target data path during application execution, in this embodiment, the detection nodes can be set in application 210 and / or application 220. Besides setting detection nodes only in the application, detection nodes can also be set in both the application and the database simultaneously. (See reference...) Figure 2 While setting monitoring nodes in application 210 and / or application 220, monitoring nodes can also be set in database 230. Specifically, when the monitoring nodes are located in the target application, the monitoring nodes for the target data are set to one or more of the following: application interface, application interface fields, messages sent by the application, messages subscribed to by the application, fields of messages sent by the application, and fields of messages subscribed to by the application; when the monitoring nodes are located in both the target application and the target database, the monitoring nodes for the target data are set to one or more of the following: application interface, application interface fields, messages sent by the application, messages subscribed to by the application, fields of messages sent by the application, fields of messages subscribed to by the application, tables in the database, and table fields in the database. Therefore, the solution provided in the embodiments of this specification can determine field-level sub-models within / between online applications.

[0039] For example, in this embodiment, the detection nodes set for the detection nodes of target data 22 and target data 24 are the same. (See reference...) Figure 2 In application 210, detection nodes A, B, C, and D are set; in application 220, detection nodes H, I, and J are set; and in database 230, detection nodes O, P, and Q are set. Furthermore, during the operation of application 210 and / or application 220, the propagation node sequence of the target data 22 and the propagation node sequence of the target data 24 can be determined respectively through dynamic taint data analysis.

[0040] Continue to refer to Figure 1 In S120, during the operation of the target application, the detection nodes in the propagation path of the target data are determined among all detection nodes, and the actual propagation node sequence of the target data is obtained.

[0041] In an exemplary embodiment, Figure 3 The flowchart illustrating a method for determining a propagation node sequence according to an embodiment of this specification can be used as a specific implementation of S120. (See reference...) Figure 3 The embodiment shown in the figure includes: S1202 and S1204.

[0042] In S1202, an information acquisition agent module is determined based on the target data, and the information acquisition agent module is deployed to the target application where the detection node is located; and in S1204, during the operation of the target application, information about the detection nodes in the propagation path of the target data is obtained through the information acquisition agent module, and the actual propagation node sequence of the target data is obtained.

[0043] For example, firstly, an information acquisition agent module (Agent) for taint analysis can be implemented through coding, specifically including target data information and detection node information, such as setting tags for target data 22 and target data 24 and identifying each detection node. Further, the aforementioned information acquisition agent module (Agent) is deployed to the target application or the target database where the aforementioned detection nodes reside, such as in application 210, application 220, and database 230. Thus, during the operation of application 210 and application 220, the information acquisition agent module (Agent) can determine whether target data 22 has passed through any detection node, and the upstream and downstream relationships of the relevant detection nodes; similarly, the information acquisition agent module (Agent) can determine the detection nodes that target data 24 has passed through, and the upstream and downstream relationships of the relevant detection nodes.

[0044] For example, Figure 4 The propagation paths of target data 22 and target data 24 during the operation of applications 210 and 220 are illustrated. For example, target data 22 is propagated from detection node A in application 210 to detection nodes H and J in application 220 during application operation, and further propagated to detection node P in database 230. It can be seen that there are upstream and downstream relationships between the detection nodes in the above actual propagation node sequence regarding the propagation path of the target data. For example, taking detection node J as the lineage center, detection node H is its first-level upstream node, and detection node A is its second-level upstream node; of course, detection node H can also be taken as the lineage center, in which case detection node A is its first-level upstream node, and detection node J is its first-level downstream node.

[0045] In this embodiment, the information acquisition agent module can dynamically acquire the detection nodes that the target data passes through during application execution. This allows for the acquisition of the actual detection nodes the target data traverses, thus obtaining the "actual" data lineage and resulting in high accuracy. Therefore, with thorough application evaluation and comprehensive use case coverage, the information acquisition agent module can obtain the propagation node sequence of the target data, and its analysis results demonstrate high recall and accuracy.

[0046] However, in the process of using the information acquisition agent module to obtain the propagation path of target data, there are issues such as insufficient evaluation of relevant applications and incomplete coverage of execution use cases, resulting in a low "actual" data lineage recall rate. In another exemplary embodiment, static code analysis and the information acquisition agent module can be combined to determine the propagation node sequence of target data, thereby improving the recall rate. Specifically:

[0047] Static code analysis is performed on the target application to obtain the inferred propagation node sequence of the target data; and, based on the inferred propagation node sequence and the actual propagation node sequence, the data lineage map of the target data is determined.

[0048] In an embodiment that uses both static code analysis and dynamic taint analysis techniques, pruning and fusion of pedigree results can improve the accuracy and recall of the pedigree map.

[0049] Continue to refer to Figure 1 In S130, a data lineage map of the target data is determined based on the actual propagation node sequence of the target data.

[0050] The pedigree map of the aforementioned target data contains multiple sub-models. In this embodiment, each sub-model of the target data can be determined first, and then the pedigree map of the target data can be constructed using all sub-models. For example, Figure 5 This is a schematic diagram of a bloodline model provided in one embodiment of this specification. (See reference) Figure 5 Each of the aforementioned lineage sub-models 500 (hereinafter referred to as "sub-models") includes: a source node 510, a target node 520, and the propagation type of the target data transmitted from the source node to the target node. The propagation type indicates the type of propagation of the target data from the source node to the target node, and may include, for example, message propagation type, parameter passing type, data query type, and data update type.

[0051] In an exemplary embodiment, Figure 6 The flowchart illustrating a method for constructing a data pedigree map, provided in another embodiment of this specification, can be used as a specific implementation of S130. (See reference...) Figure 6 The embodiment shown in the figure includes: S1302-S13012.

[0052] In S1302, it is determined whether there are N downstream nodes for the target detection node in the propagation node sequence.

[0053] In the case where only an information acquisition agent is used in the process of obtaining the propagation node sequence of the target data, the "propagation node sequence" mentioned above can refer to the "actual propagation node sequence" mentioned above. In the case where both an information acquisition agent and static code analysis are used in the process of obtaining the propagation node sequence of the target data, the "propagation node sequence" mentioned above can be determined by a combination of the "actual propagation node sequence" mentioned above and the "inferred propagation node sequence" mentioned above.

[0054] For example, the target detection node mentioned above can be any detection node in the propagation node sequence of target data. Further, the downstream node of the target detection node refers to a "first-level downstream node," which can be one or more, or may not exist; that is, N takes the value of a positive integer.

[0055] In an exemplary embodiment, if there are N downstream nodes for the target detection node in the above-mentioned actual propagation node sequence, it indicates that the current target detection node is an end node and cannot be used as a source node for a blood model; it can be used as a target detection node with other target detection nodes as source nodes. Therefore, S13012 is executed to end the generation process of the sub-model for the above-mentioned target node.

[0056] In an exemplary embodiment, if the target detection node has N downstream nodes in the actual propagation node sequence, and the target detection node is not the last detection node in the sequence, then N sub-models with the target detection node as the source node can be determined based on the target detection node and the N downstream nodes. For example, if the target detection node S has one downstream node, denoted as detection node N, then a sub-model with the target detection node S as the source node and any one of its downstream nodes as the target node can be determined based on the target detection node S and detection node N. If the target detection node S has two downstream nodes (denoted as detection node N1 and detection node N2 respectively), then a sub-model with the target detection node S as the source node and detection node N1 as the target node can be determined based on the target detection node S and detection node N1, and another sub-model with the target detection node S as the source node and detection node N2 as the target node can be determined based on the target detection node S and detection node N2.

[0057] For example, S1304-S1308 provides a specific implementation method for determining the sub-model:

[0058] In S1304, the aforementioned target detection node is taken as the source node of the i-th sub-model, and the i-th downstream node of the aforementioned target detection node is determined as the target node of the i-th sub-model, where i takes the value from 1 to N and is an integer containing 1 and N.

[0059] For example, refer to Figure 4 For target data 24, detection node Q in its detection node sequence is determined as the aforementioned target detection node. It can be seen that detection node Q has two downstream nodes: detection node O and detection node P. Further, referring to... Figure 7 Based on detection node Q (target detection node) and detection node P, a sub-model 710 is determined with detection node Q as the source node and detection node P as the target node; based on detection node Q (target detection node) and detection node O, another sub-model 720 is determined with detection node Q as the source node and detection node O as the target node.

[0060] For example, refer again Figure 4 For target data 24, detection node P in its detection node sequence is determined as the aforementioned target detection node. It is evident that detection node P has a downstream node: detection node B. Further, refer to... Figure 7 Based on detection node P (target detection node) and detection node B, a sub-model 730 is determined with detection node P as the source node and detection node B as the target node.

[0061] Since each sub-model includes not only a source node and a target node, but also the propagation type of the target data during the process of being transmitted from the source node to the target node, after determining the source node and the target node in the sub-model, the propagation type in the sub-model is then determined through S1306 and its embodiments.

[0062] In S1306, based on the data type of the target data in the target detection node and the data type of the target data in the i-th downstream node, the propagation type of the i-th sub-model, with the target detection node as the source node and the i-th downstream node as the target node, is determined. And in S1308, based on the propagation type of the target detection node, the i-th downstream node, and the i-th sub-model, the i-th sub-model, with the target detection node as the source node and the i-th downstream node as the target node, is determined.

[0063] In this embodiment, the propagation type of the sub-model is determined based on the data type of the target data in the source node and the data type in the target node within the same sub-model. As mentioned above, the propagation type indicates the type of propagation of the target data from the source node to the target node, and may include, for example, message propagation type, parameter passing type, data query type, and data update type. Specifically, for the aforementioned sub-model 710, if the data type of the target data 24 in the detection node Q (source node) is an interface parameter, and the data type in the detection node P (target node) is also an interface parameter, then the propagation type of the sub-model 710 can be determined to be the parameter passing type. Furthermore, the determination method for each propagation type will be described in detail in the following embodiments.

[0064] Continue to refer to Figure 6 In S13010, a data lineage map of the target data is constructed by using a sub-model with each detection node in the above-mentioned actual propagation node sequence as the source node.

[0065] For example, using the common detection nodes among the aforementioned sub-models as the intersection, the sub-models that use each detection node in the aforementioned propagation node sequence as the source node are fused to obtain a data lineage map of the aforementioned target data. (Reference) Figure 7 If the common detection node between sub-model 710 and sub-model 720 is detection node Q, then the intersection of the two sub-models is used as the detection node Q; if the common detection node between sub-model 710 and sub-model 730 is detection node P, then the intersection of the two sub-models is used as the detection node P. In this way, multiple sub-models about the same target data can be fused to obtain the lineage map of the target data.

[0066] The following describes an example of determining the propagation type in each sub-model:

[0067] In an exemplary embodiment, Table 1 shows the source node and the data type of the target data in the source node, the target node and the data type of the target data in the target node, wherein the data in the same row belongs to the same sub-model.

[0068] Table 1

[0069]

[0070]

[0071] Furthermore, the propagation type of each sub-model is determined based on the data in Table 1:

[0072] If the aforementioned detection node is located in the aforementioned target application and target database, the propagation type of the i-th sub-model with the aforementioned target detection node as the source node is determined based on the node type of the aforementioned target detection node and the node type of the aforementioned i-th downstream node, as shown in Table 2 below:

[0073] Table 2

[0074]

[0075]

[0076] Referring to Table 2, when all detection nodes are located in the target application, the propagation type of the sub-model is determined based on the data type of the target data in the source node of the sub-model and the data type of the target data in the target node of the sub-model. Specific implementation method:

[0077] When the data type in the source node is an interface parameter or a message body: if the data type in the target node is an interface parameter, the propagation type of the sub-model is parameter passing; if the data type in the target node is a message body, the propagation type of the sub-model is message propagation.

[0078] If the data type in the source node is an interface return value: if the data type in the target node is an interface return value, then the propagation type of the sub-model is data query type; if the data type in the target node is a message body, then the propagation type of the sub-model is message propagation type.

[0079] When the detection node is located in both the target application and the target database, the propagation type of the sub-model is determined based on the data type of the target data in the source node of the sub-model and the data type of the target data in the target node of the sub-model. Specific implementation method:

[0080] When the data type in the source node is an interface parameter or a message body: if the data type in the target node is an interface parameter, the propagation type of the sub-model is parameter passing; if the data type in the target node is a message body, the propagation type of the sub-model is message propagation; if the data type in the target node is database data, the propagation type of the sub-model is data update.

[0081] When the data type in the source node is an interface return value or database data: if the data type in the target node is an interface return value, the propagation type of the sub-model is data query type; if the data type in the target node is a message body, the propagation type of the sub-model is message propagation type; if the data type in the target node is database data, the propagation type of the sub-model is data update type.

[0082] If the data type in the source node is database data, and the data type in the target node is interface input parameter, then the propagation type of the sub-model is parameter passing type.

[0083] In an exemplary embodiment, based on the propagation type determination method shown in Table 2, and according to the data of each data type shown in Table 1, the following is determined: Figure 8 The bloodline chart shown.

[0084] refer to Figure 8 For the sub-model corresponding to the first row in Table 1: the source node is "app1.interfaceI1" 812, where the data "app1.interfaceI1-paramP1.name" has a data type of "interface input parameter"; the target node is "app2.interfaceI2" 820, where the data "app2.interfaceI2-paramP1.name" also has a data type of "interface input parameter". Therefore, the propagation type in this sub-model is "parameter passing". Figure 8 As shown, the data propagation type from detection node 812 to detection node 820 is "parameter transfer".

[0085] For the sub-model corresponding to row 2 in Table 1: the source node is "app1.interfaceI1" 812, where the data "app1.interfaceI1-paramP1.name" has a data type of "interface input parameter"; the target node is "app1.mqM1" 814, where the data "app1.mqM1-topicT1" has a data type of "message body". Therefore, the propagation type in this sub-model is "message propagation". Figure 8 As shown, the data propagation type from detection node 812 to detection node 814 is "message propagation".

[0086] For the sub-model corresponding to row 3 in Table 1: the source node is "app2.interfaceI2" 820, where the data "app2.interfaceI2-paramP1.name" has a data type of "interface input parameter"; the target node is "app3.interfaceI5" 830, where the data "app3.interfaceI5-paramP2.username" has a data type of "interface input parameter". Therefore, the propagation type in this sub-model is "parameter passing". Figure 8 As shown, the data propagation type from detection node 820 to detection node 830 is "parameter transfer".

[0087] For the sub-model corresponding to row 4 in Table 1: the source node is "app2.interfaceI2" 820, where the data "app2.interfaceI2-paramP1.name" has a data type of "interface input parameter"; the target node is "mysqlM1" 840, where the data "mysqlM1-tableT1-fieldF1" has a data type of "database data". Therefore, the propagation type in this sub-model is "data update". Figure 8 As shown, the data propagation type from detection node 820 to detection node 840 is "data update".

[0088] For the sub-model corresponding to row 5 in Table 1: the source node is "mysqlM1" 840, where the data "mysqlM1-tableT1-fieldF1" has a data type of "database data"; the target node is "app2.interfaceI3" 824, where the data "app2.interfaceI3-resultR1.name" has a data type of "interface return value". Therefore, the propagation type in this sub-model is "data query". Figure 8 As shown, the data propagation type from detection node 840 to detection node 824 is "data query".

[0089] For the sub-model corresponding to row 6 in Table 1: the source node is "app1.mqM1" 814, where the data type of "app1.mqM1-topicT1" is "message body"; the target node is "app2.mqM1" 822, where the data type of "app2.mqM1-topicT2" is also "message body". Therefore, the propagation type in this sub-model is "message propagation". Figure 8 As shown, the data propagation type from detection node 814 to detection node 822 is "message propagation".

[0090] For the sub-model corresponding to row 7 in Table 1: the source node is "app2.mqM1" 822, where the data "app2.mqM1-topicT2" has a data type of "message body"; the target node is "mysqlM1" 840, where the data "mysqlM1-tableT1-fieldF1" has a data type of "database data". Therefore, the propagation type in this sub-model is "data update". Figure 8 As shown, the data propagation type from detection node 822 to detection node 840 is "data update".

[0091] For the sub-model corresponding to row 8 in Table 1: the source node is "app1.mqM1" (814), where the data type of "app1.mqM1-topicT1" is "message body"; the target node is "app2.interfaceI4" (826), where the data type of "app2.interfaceI4-paramP1.name" is "interface parameter". Therefore, the propagation type in this sub-model is "parameter passing". Figure 8 As shown, the data propagation type from detection node 814 to detection node 826 is "parameter transfer".

[0092] For the sub-model corresponding to row 9 in Table 1: the source node is "app2.interfaceI3" 824, where the data "app2.interfaceI3-resultR1.name" has a data type of "interface return value"; the target node is "app3.interfaceI6" 832, where the data "app3.interfaceI6-resultR2.name" has a data type of "interface return value". Therefore, the propagation type in this sub-model is "data query". Figure 8 As shown, the data propagation type from detection node 824 to detection node 832 is "data query".

[0093] For the sub-model corresponding to row 10 in Table 1: the source node is “mysqlM3-tableT3-”860, where the data “mysqlM3-tableT3-fieldF3” has a data type of “database data”; the target node is “app3.interfaceI6”832, where the data “app3.interfaceI6-resultR2.name” has a data type of “interface return value”; therefore, the propagation type in this sub-model is “data query”. Figure 8 As shown, the data propagation type from detection node 860 to detection node 832 is "data query".

[0094] For the sub-model corresponding to row 11 in Table 1: the source node is "mysqlM3-tableT2" 850, where the data "mysqlM3-tableT2-fieldF3" has a data type of "database data"; the target node is "app3.interfaceI6" 832, where the data "app3.interfaceI6-resultR2.name" has a data type of "interface return value". Therefore, the propagation type in this sub-model is "data query". Figure 8 As shown, the data propagation type from detection node 850 to detection node 832 is "data query".

[0095] For the sub-model corresponding to row 12 in Table 1: the source node is "app3.interfaceI5" 830, where the data "app3.interfaceI5-paramP2.username" has a data type of "interface input parameter"; the target node is "mysqlM3-tableT2" 850, where the data "mysqlM3-tableT2-fieldF3" has a data type of "database data". Therefore, the propagation type in this sub-model is "data update". Figure 8 As shown, the data propagation type from detection node 830 to detection node 850 is "data update".

[0096] Based on the sub-model corresponding to each row above, it is constructed as follows: Figure 8 The data shown is a pedigree chart. (Reference) Figure 8 Detection nodes belonging to different applications / databases can be distinguished by different display methods, so that users can clearly determine which applications and databases the propagation path of each target data passes through.

[0097] In an exemplary embodiment, Figure 9 This displays another interface related to the data sub-model. (Reference) Figure 9 Users can select the type of detection node they want to view by checking boxes. Examples include: online applications, online application interfaces, online tables, and online table fields. For example, if the target data to be viewed is data s, then data s serves as the "lineage center," based on... Figure 8 The data sub-model shown can display the upstream and downstream nodes of data s, as well as the number and names of entities in each upstream and downstream node.

[0098] like Figure 9As shown, data s, acting as the lineage center, has two layers of upstream nodes and one layer of downstream nodes. The first-layer upstream nodes contain one entity named "OPQ", the second-layer upstream nodes contain one entity named "ABC", and the first-layer downstream nodes do not contain any entities. This can be further illustrated by... Figure 9 The interface shown allows users to easily obtain data such as upstream and downstream nodes of the target data, facilitating targeted control of the relevant target data and / or detection nodes.

[0099] It should be noted that the above figures are merely illustrative of the processes included in the methods according to exemplary embodiments of this specification, and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Furthermore, it is readily understood that these processes may, for example, be executed synchronously or asynchronously in multiple modules.

[0100] The following are embodiments of the apparatus described in this specification, which can be used to execute the embodiments of the methods described in this specification. For details not disclosed in the apparatus embodiments of this specification, please refer to the embodiments of the methods described in this specification.

[0101] in, Figure 10 A schematic diagram of a data pedigree mapping construction apparatus applicable to one embodiment of this specification is shown. Please refer to... Figure 10 The data lineage mapping device shown in the figure can be implemented as a whole or part of an electronic device through software, hardware, or a combination of both. It can also be integrated as an independent module on a server or in an electronic device.

[0102] The data lineage mapping construction device 1000 in the embodiments of this specification includes: a first determining module 1010, a second determining module 1020, and a construction module 1030.

[0103] The first determining module 1010 is configured to: determine target data and determine detection nodes related to the target data, wherein the detection nodes are located in the target application or in both the target application and the target database; the second determining module 1020 is configured to: during the operation of the target application, determine the detection nodes in the propagation path of the target data among all detection nodes to obtain the actual propagation node sequence of the target data, wherein there are upstream and downstream relationships between the detection nodes in the actual propagation node sequence related to the propagation path of the target data; and the construction module 1030 is configured to: determine the data lineage map of the target data based on the actual propagation node sequence of the target data.

[0104] In an exemplary embodiment, Figure 11A schematic diagram illustrating a structural arrangement for constructing a data pedigree map according to another exemplary embodiment of this specification is shown. Please refer to... Figure 11 :

[0105] In an exemplary embodiment, based on the foregoing scheme, the data lineage map of the target data includes multiple sub-models, and each sub-model includes a source node;

[0106] The aforementioned construction module 1030 includes: a sub-model determination unit 10302 and a bloodline map construction unit 10304.

[0107] The sub-model determination unit 10302 is used to: determine N sub-models with the target detection node as the source node based on the target detection node and the N downstream nodes when there are N downstream nodes in the actual propagation node sequence; wherein N is a positive integer; and the lineage map construction unit 10304 is used to: construct a data lineage map of the target data using sub-models with each detection node in the actual propagation node sequence as the source node.

[0108] In an exemplary embodiment, based on the foregoing scheme, each of the above sub-models further includes: a target node, and the propagation type of the target data transmitted from the source node to the target node;

[0109] The aforementioned sub-model determination unit 10302 includes: target node determination sub-unit 3022, propagation type determination sub-unit 3024, and sub-model determination sub-unit 3026.

[0110] The target node determination subunit 3022 is configured to: use the target detection node as the source node of the i-th sub-model, and determine the i-th downstream node of the target detection node as the target node of the i-th sub-model, where i is an integer from 1 to N, including 1 and N; the propagation type determination subunit 3024 is configured to: determine the propagation type of the i-th sub-model with the target detection node as the source node and the i-th downstream node as the target node based on the data type of the target data in the target detection node and the data type of the target data in the i-th downstream node; and the sub-model determination subunit 3026 is configured to: determine the i-th sub-model with the target detection node as the source node and the i-th downstream node as the target node based on the propagation type of the target detection node, the i-th downstream node, and the i-th sub-model.

[0111] In an exemplary embodiment, based on the foregoing scheme, when the detection node is located in the target application, the propagation type determination subunit 3024 is specifically used for:

[0112] If the data type of the target data in the target detection node is an interface parameter or a message body, and the data type of the target data in the i-th downstream node is an interface parameter, then the propagation type of the i-th sub-model with the target detection node as the source node and the i-th downstream node as the target node is the parameter passing type; if the data type of the target data in the i-th downstream node is a message body, then the propagation type of the i-th sub-model with the target detection node as the source node and the i-th downstream node as the target node is the message propagation type.

[0113] If the data type of the target data in the target detection node is an interface return value, and the data type of the target data in the i-th downstream node is an interface return value, then the propagation type of the i-th sub-model with the target detection node as the source node and the i-th downstream node as the target node is the data query type. If the data type of the target data in the i-th downstream node is a message body, then the propagation type of the i-th sub-model with the target node as the source node and the i-th downstream node as the target node is the message propagation type.

[0114] In an exemplary embodiment, based on the foregoing scheme, when the detection node is located in the target application and the target database, the propagation type determination subunit 3024 is specifically used for:

[0115] If the data type of the target data in the target detection node is an interface parameter or a message body, and the data type of the target data in the i-th downstream node is an interface parameter, then the propagation type of the i-th sub-model with the target detection node as the source node and the i-th downstream node as the target node is the parameter passing type. If the data type of the target data in the i-th downstream node is a message body, then the propagation type of the i-th sub-model with the target detection node as the source node and the i-th downstream node as the target node is the message propagation type. If the data type of the target data in the i-th downstream node is database data, then the propagation type of the i-th sub-model with the target detection node as the source node and the i-th downstream node as the target node is the data update type.

[0116] If the data type of the target data in the target detection node is an interface return value or database data, and the data type of the target data in the i-th downstream node is an interface return value, then the propagation type of the i-th sub-model with the target detection node as the source node and the i-th downstream node as the target node is the data query type. If the data type of the target data in the i-th downstream node is a message body, then the propagation type of the i-th sub-model with the target detection node as the source node and the i-th downstream node as the target node is the message propagation type. If the data type of the target data in the i-th downstream node is database data, then the propagation type of the i-th sub-model with the target detection node as the source node and the i-th downstream node as the target node is the data update type.

[0117] If the data type of the target data in the target detection node is database data, and the data type of the target data in the i-th downstream node is interface input parameter, then the propagation type of the i-th sub-model with the target detection node as the source node and the i-th downstream node as the target node is the parameter passing type.

[0118] In an exemplary embodiment, based on the aforementioned scheme, the above-mentioned lineage map construction unit 10304 is specifically used to: perform fusion processing on the sub-models with each detection node in the above-mentioned actual propagation node sequence as the source node, taking the common detection nodes among the above-mentioned sub-models as the intersection, to obtain a data lineage map of the above-mentioned target data.

[0119] In an exemplary embodiment, based on the foregoing scheme, when the detection node is located in the target application, the detection node for the target data is set with one or more of the following information: application interface, application interface fields, messages sent by the application, messages subscribed to by the application, fields of messages sent by the application, and fields of messages subscribed to by the application; and when the detection node is located in both the target application and the target database, the detection node for the target data is set with one or more of the following information: application interface, application interface fields, messages sent by the application, messages subscribed to by the application, fields of messages sent by the application, fields of messages subscribed to by the application, tables in the database, and table fields in the database.

[0120] In an exemplary embodiment, based on the foregoing scheme, the second determining module 1020 is specifically used to: determine the information acquisition agent module according to the target data, and deploy the information acquisition agent module to the target application where the detection node is located; and, during the operation of the target application, obtain information about the detection nodes in the propagation path of the target data through the information acquisition agent module to obtain the actual propagation node sequence of the target data.

[0121] In an exemplary embodiment, based on the foregoing scheme, the second determining module 1020 is specifically used to: perform static code analysis on the target application to obtain the inferred propagation node sequence of the target data; and determine the data lineage map of the target data based on the inferred propagation node sequence and the actual propagation node sequence.

[0122] It should be noted that the data lineage map construction device provided in the above embodiments is only illustrated by the division of the above functional modules when executing the data lineage map construction method. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0123] Furthermore, the data pedigree mapping device and data pedigree mapping method embodiments provided in the above embodiments belong to the same concept. Therefore, for details not disclosed in the device embodiments of this specification, please refer to the data pedigree mapping method embodiments described above, which will not be repeated here.

[0124] The example numbers in this specification are for descriptive purposes only and do not represent the superiority or inferiority of the examples.

[0125] This specification also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of any of the methods described above.

[0126] Figure 12 This schematic diagram illustrates the structure of an electronic device according to an exemplary embodiment of this specification. Please refer to... Figure 12 As shown, the electronic device 1200 includes a processor 1201 and a memory 1202.

[0127] In this embodiment, processor 1201 is the control center of the computer system and can be a processor of a physical machine or a processor of a virtual machine. Processor 1201 may include one or more processing cores, such as a 4-core processor or an 8-core processor. Processor 1201 may be implemented using at least one hardware form selected from Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA). Processor 1201 may also include a main processor and a coprocessor; the main processor is used to process data in the wake-up state, and the coprocessor is a low-power processor used to process data in the standby state.

[0128] In the embodiments described in this specification, the processor 1201 is specifically used for:

[0129] The process involves identifying target data and detection nodes related to that target data, wherein the detection nodes are located in a target application or in both the target application and a target database; during the operation of the target application, detection nodes along the propagation path of the target data are identified from all detection nodes to obtain an actual propagation node sequence for the target data, wherein upstream and downstream relationships exist between the detection nodes in the actual propagation node sequence; and, based on the actual propagation node sequence of the target data, a data lineage map of the target data is determined.

[0130] Furthermore, the aforementioned pedigree graph of the target data contains multiple sub-models, each of which includes a source node;

[0131] The above-mentioned determination of the data lineage graph of the target data based on the actual propagation node sequence of the target data includes: when there are N downstream nodes of the target detection node in the actual propagation node sequence, determining N sub-models with the target detection node as the source node based on the target detection node and the N downstream nodes, where N is a positive integer; and constructing the data lineage graph of the target data through the sub-model with each detection node in the actual propagation node sequence as the source node.

[0132] Furthermore, each of the above sub-models also includes: a target node, and the propagation type of the target data transmitted from the source node to the target node;

[0133] The above-mentioned determination of N sub-models with the target detection node as the source node based on the target detection node and the N downstream nodes includes: taking the target detection node as the source node of the i-th sub-model, and determining the i-th downstream node of the target detection node as the target node of the i-th sub-model, where i is an integer from 1 to N, including 1 and N; determining the propagation type of the i-th sub-model with the target detection node as the source node based on the data type of the target data in the target detection node and the data type of the target data in the i-th downstream node; and determining the i-th sub-model with the target detection node as the source node and the i-th downstream node as the target node based on the propagation type of the target detection node, the i-th downstream node, and the i-th sub-model.

[0134] Furthermore, when the aforementioned detection node is located in the target application, determining the propagation type of the i-th sub-model with the target detection node as the source node and the i-th downstream node as the target node based on the data type of the target data in the target detection node and the data type of the target data in the i-th downstream node includes: if the data type of the target data in the target detection node is an interface parameter or a message body, and if the data type of the target data in the i-th downstream node is an interface parameter, then the propagation type of the i-th sub-model with the target detection node as the source node and the i-th downstream node as the target node is a parameter passing type; if the data type of the target data in the i-th downstream node is an interface parameter, then the propagation type of the i-th sub-model with the target detection node as the source node and the i-th downstream node as the target node is a parameter passing type. If the data type in the target data is a message body, then the propagation type of the i-th sub-model with the target detection node as the source node and the i-th downstream node as the target node is the message propagation type; and, if the data type of the target data in the target detection node is an interface return value, and the data type of the target data in the i-th downstream node is an interface return value, then the propagation type of the i-th sub-model with the target detection node as the source node and the i-th downstream node as the target node is the data query type; if the data type of the target data in the i-th downstream node is a message body, then the propagation type of the i-th sub-model with the target detection node as the source node and the i-th downstream node as the target node is the message propagation type.

[0135] Furthermore, when the detection node is located in the target application and the target database, determining the propagation type of the i-th sub-model with the target detection node as the source node and the i-th downstream node as the target node based on the data type of the target data in the target detection node and the data type of the target data in the i-th downstream node includes: if the data type of the target data in the target detection node is an interface parameter or a message body, and the data type of the target data in the i-th downstream node is an interface parameter, then the propagation type of the i-th sub-model with the target detection node as the source node and the i-th downstream node as the target node is a parameter passing type; if the data type of the target data in the i-th downstream node is a message body, then the propagation type of the i-th sub-model with the target detection node as the source node and the i-th downstream node as the target node is a message propagation type; if the data type of the target data in the i-th downstream node is database data, then the propagation type of the i-th sub-model with the target detection node as the source node and the i-th downstream node as the target node is... Data update type; when the data type of the target data in the target detection node is an interface return value or database data, if the data type of the target data in the i-th downstream node is an interface return value, then the propagation type of the i-th sub-model with the target detection node as the source node and the i-th downstream node as the target node is the data query type; if the data type of the target data in the i-th downstream node is a message body, then the propagation type of the i-th sub-model with the target detection node as the source node and the i-th downstream node as the target node is the message propagation type; if the data type of the target data in the i-th downstream node is database data, then the propagation type of the i-th sub-model with the target detection node as the source node and the i-th downstream node as the target node is the data update type; and, when the data type of the target data in the target detection node is database data, if the data type of the target data in the i-th downstream node is an interface input parameter, then the propagation type of the i-th sub-model with the target detection node as the source node and the i-th downstream node as the target node is the parameter passing type.

[0136] Furthermore, the above-mentioned construction of a data lineage map of the target data by using sub-models with each detection node in the above-mentioned actual propagation node sequence as the source node includes: using the common detection nodes among the above-mentioned sub-models as the intersection, performing fusion processing on the sub-models with each detection node in the above-mentioned actual propagation node sequence as the source node to obtain a data lineage map of the target data.

[0137] Furthermore, when the aforementioned detection node is located in the target application, the detection node for the aforementioned target data is set to one or more of the following information: application interface, application interface fields, messages sent by the application, messages subscribed to by the application, fields of messages sent by the application, and fields of messages subscribed to by the application; and when the aforementioned detection node is located in both the aforementioned target application and the target database, the detection node for the aforementioned target data is set to one or more of the following information: application interface, application interface fields, messages sent by the application, messages subscribed to by the application, fields of messages sent by the application, fields of messages subscribed to by the application, tables in the database, and table fields in the database.

[0138] Furthermore, during the operation of the target application, identifying the detection nodes in the propagation path of the target data among all detection nodes to obtain the actual propagation node sequence of the target data includes: determining an information acquisition agent module based on the target data and deploying the information acquisition agent module to the target application where the detection nodes are located; and during the operation of the target application, obtaining information about the detection nodes in the propagation path of the target data through the information acquisition agent module to obtain the actual propagation node sequence of the target data.

[0139] Furthermore, determining the data lineage graph of the target data based on the actual propagation node sequence of the target data includes: performing static code analysis on the target application to obtain the inferred propagation node sequence of the target data; and determining the data lineage graph of the target data based on the inferred propagation node sequence and the actual propagation node sequence.

[0140] Memory 1202 may include one or more computer-readable storage media, which may be non-transitory. Memory 1202 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments of this specification, the non-transitory computer-readable storage media in memory 1202 is used to store at least one instruction for execution by processor 1201 to implement the methods in the embodiments of this specification.

[0141] In some embodiments, the electronic device 1200 further includes a peripheral device interface 1203 and at least one peripheral device. The processor 1201, memory 1202, and peripheral device interface 1203 can be connected via a bus or signal line. Each peripheral device can be connected to the peripheral device interface 1203 via a bus, signal line, or circuit board. Specifically, the peripheral device includes at least one of a display screen 1204, a camera 1205, and an audio circuit 1206.

[0142] Peripheral interface 1203 can be used to connect at least one input / output (I / O) related peripheral device to processor 1201 and memory 1202. In some embodiments of this specification, processor 1201, memory 1202, and peripheral interface 1203 are integrated on the same chip or circuit board; in some other embodiments of this specification, any one or two of processor 1201, memory 1202, and peripheral interface 1203 can be implemented on separate chips or circuit boards. This specification does not specifically limit the embodiments in this regard.

[0143] Display screen 1204 is used to display a user interface (UI). The UI may include graphics, text, icons, videos, and any combination thereof. When display screen 1204 is a touch display screen, it also has the ability to collect touch signals on or above its surface. These touch signals can be input as control signals to processor 1201 for processing. In this case, display screen 1204 can also be used to provide virtual buttons and / or a virtual keyboard, also known as soft buttons and / or a soft keyboard. In some embodiments of this specification, there may be one display screen 1204, which serves as the front panel of electronic device 1200; in other embodiments, there may be at least two display screens 1204, respectively disposed on different surfaces of electronic device 1200 or in a folded design; in still other embodiments, display screen 1204 may be a flexible display screen, disposed on a curved or folded surface of electronic device 1200. Furthermore, display screen 1204 may also be configured as a non-rectangular irregular shape, i.e., a non-rectangular screen. The display screen 1204 can be made of materials such as liquid crystal display (LCD) and organic light-emitting diode (OLED).

[0144] Camera 1205 is used to capture images or videos. Optionally, camera 1205 includes a front-facing camera and a rear-facing camera. Typically, the front-facing camera is located on the front panel of the electronic device, and the rear-facing camera is located on the back of the electronic device. In some embodiments, there are at least two rear-facing cameras, which are any one of a main camera, a depth-sensing camera, a wide-angle camera, and a telephoto camera, to achieve background blurring by fusion of the main camera and the depth-sensing camera, panoramic shooting by fusion of the main camera and the wide-angle camera, virtual reality (VR) shooting, or other fusion shooting functions. In some embodiments of this specification, camera 1205 may also include a flash. The flash can be a single-color temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm light flash and a cool light flash, which can be used for light compensation at different color temperatures.

[0145] The audio circuit 1206 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, and convert the sound waves into electrical signals that are input to the processor 1201 for processing. For stereo sound acquisition or noise reduction purposes, there may be multiple microphones, each located in a different part of the electronic device 1200. The microphone may also be an array microphone or an omnidirectional microphone.

[0146] Power supply 1207 is used to supply power to various components in electronic device 1200. Power supply 1207 can be AC ​​power, DC power, a disposable battery, or a rechargeable battery. When power supply 1207 includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery that is charged via a wired line, and a wireless rechargeable battery is a battery that is charged via a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0147] The block diagrams of the electronic device shown in the embodiments of this specification do not constitute a limitation on the electronic device 1200. The electronic device 1200 may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0148] In the description of this specification, it should be understood that the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Those skilled in the art can understand the specific meaning of these terms in this specification based on the specific circumstances. Furthermore, in the description of this specification, unless otherwise stated, "multiple" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship.

[0149] This specification also provides a computer-readable storage medium storing instructions that, when executed on a computer or processor, cause the computer or processor to perform one or more steps in the above embodiments. If the constituent modules of the above-described data pedigree mapping construction apparatus are implemented as software functional units and sold or used as independent products, they can be stored in the aforementioned computer-readable storage medium.

[0150] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When these computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this specification are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in or transmitted through a computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The aforementioned available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., Digital Versatile Discs (DVDs)), or semiconductor media (e.g., Solid State Disks (SSDs)).

[0151] It should be noted that the above description describes specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims may be performed in a different order than that shown in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0152] The above description is merely a specific embodiment of this specification, but the scope of protection of this specification is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this specification should be included within the scope of protection of this specification. Therefore, equivalent variations made in accordance with the claims of this specification are still within the scope of this specification.

Claims

1. A method for constructing a data pedigree map, wherein, The method includes: Determine target data and determine detection nodes related to the target data, wherein the detection nodes are located in the target application, or the detection nodes are located in both the target application and the target database; During the operation of the target application, the detection nodes in the target data propagation path are determined among all detection nodes to obtain the actual propagation node sequence of the target data, wherein there are upstream and downstream relationships between the detection nodes in the actual propagation node sequence regarding the target data propagation path; Based on the actual propagation node sequence of the target data, determine the data lineage map of the target data; The data lineage map of the target data includes multiple sub-models, and each sub-model includes a source node; Determining the data lineage map of the target data based on the actual propagation node sequence of the target data includes: In the actual propagation node sequence, if the target detection node has N downstream nodes, N sub-models are determined based on the target detection node and the N downstream nodes, with the target detection node as the source node, where N is a positive integer. A data lineage map of the target data is constructed by using a sub-model with each detection node in the actual propagation node sequence as the source node. Each of the sub-models further includes: a target node, and the propagation type of the target data transmitted from the source node to the target node; The step of determining N sub-models, with the target detection node as the source node, based on the target detection node and the N downstream nodes, includes: The target detection node is used as the source node of the i-th sub-model, and the i-th downstream node of the target detection node is determined as the target node of the i-th sub-model, where i is an integer from 1 to N, including 1 and N. Based on the data type of the target data in the target detection node and the data type of the target data in the i-th downstream node, determine the propagation type of the i-th sub-model with the target detection node as the source node and the i-th downstream node as the target node; Based on the propagation type of the target detection node, the i-th downstream node, and the i-th sub-model, determine the i-th sub-model with the target detection node as the source node and the i-th downstream node as the target node.

2. The method according to claim 1, wherein, When the detection node is located in the target application, determining the propagation type of the i-th sub-model with the target detection node as the source node and the i-th downstream node as the target node, based on the data type of the target data in the target detection node and the data type of the target data in the i-th downstream node, includes: When the data type of the target data in the target detection node is an interface parameter or a message body, if the data type of the target data in the i-th downstream node is an interface parameter, then the propagation type of the i-th sub-model with the target detection node as the source node and the i-th downstream node as the target node is the parameter transmission type; if the data type of the target data in the i-th downstream node is a message body, then the propagation type of the i-th sub-model with the target detection node as the source node and the i-th downstream node as the target node is the message propagation type. If the data type of the target data in the target detection node is an interface return value, and the data type of the target data in the i-th downstream node is an interface return value, then the propagation type of the i-th sub-model with the target detection node as the source node and the i-th downstream node as the target node is the data query type. If the data type of the target data in the i-th downstream node is a message body, then the propagation type of the i-th sub-model with the target detection node as the source node and the i-th downstream node as the target node is the message propagation type.

3. The method according to claim 2, wherein, When the detection node is located in the target application and the target database, determining the propagation type of the i-th sub-model with the target detection node as the source node and the i-th downstream node as the target node, based on the data type of the target data in the target detection node and the data type of the target data in the i-th downstream node, includes: When the data type of the target data in the target detection node is an interface parameter or a message body, if the data type of the target data in the i-th downstream node is an interface parameter, then the propagation type of the i-th sub-model with the target detection node as the source node and the i-th downstream node as the target node is the parameter transmission type; if the data type of the target data in the i-th downstream node is a message body, then the propagation type of the i-th sub-model with the target detection node as the source node and the i-th downstream node as the target node is the message propagation type; if the data type of the target data in the i-th downstream node is database data, then the propagation type of the i-th sub-model with the target detection node as the source node and the i-th downstream node as the target node is the data update type. If the data type of the target data in the target detection node is an interface return value or database data, and the data type of the target data in the i-th downstream node is an interface return value, then the propagation type of the i-th sub-model with the target detection node as the source node and the i-th downstream node as the target node is the data query type. If the data type of the target data in the i-th downstream node is a message body, then the propagation type of the i-th sub-model with the target detection node as the source node and the i-th downstream node as the target node is the message propagation type. If the data type of the target data in the i-th downstream node is database data, then the propagation type of the i-th sub-model with the target detection node as the source node and the i-th downstream node as the target node is the data update type. If the data type of the target data in the target detection node is database data, and the data type of the target data in the i-th downstream node is interface input parameter, then the propagation type of the i-th sub-model with the target detection node as the source node and the i-th downstream node as the target node is the parameter transmission type.

4. The method according to any one of claims 1 to 3, wherein, The step of constructing a data lineage map of the target data using a sub-model with each detection node in the actual propagation node sequence as the source node includes: Using the common detection nodes among the sub-models as the intersection, the sub-models with each detection node in the actual propagation node sequence as the source node are fused to obtain a data lineage map of the target data.

5. The method according to any one of claims 1 to 3, wherein, When the detection node is located in the target application, the detection node for the target data is set with one or more of the following information: application interface, fields of the application interface, messages sent by the application, messages subscribed to by the application, fields of messages sent by the application, and fields of messages subscribed to by the application. When the detection node is located in the target application and the target database, the detection node for the target data is set with one or more of the following information: application interface, application interface fields, messages sent by the application, messages subscribed by the application, fields of messages sent by the application, fields of messages subscribed by the application, tables in the database, and table fields in the database.

6. The method according to any one of claims 1 to 3, wherein, During the execution of the target application, the detection nodes in the target data propagation path are determined among all detection nodes to obtain the actual propagation node sequence of the target data, including: Based on the target data, an information acquisition agent module is determined, and the information acquisition agent module is deployed to the target application where the detection node is located. During the operation of the target application, the information acquisition agent module obtains information about the detection nodes in the target data propagation path, thereby obtaining the actual propagation node sequence of the target data.

7. The method according to any one of claims 1 to 3, wherein, Determining the data lineage map of the target data based on the actual propagation node sequence of the target data includes: Static code analysis is performed on the target application to obtain the inference propagation node sequence of the target data; Based on the inferred propagation node sequence and the actual propagation node sequence, a data lineage map of the target data is determined.

8. An apparatus for constructing a data pedigree map, wherein, The device includes: The first determining module is configured to: determine target data and determine a detection node related to the target data, wherein the detection node is located in a target application, or the detection node is located in both the target application and the target database; The second determining module is used to: during the operation of the target application, determine the detection nodes in the target data propagation path among all detection nodes, and obtain the actual propagation node sequence of the target data, wherein there are upstream and downstream relationships between the detection nodes in the actual propagation node sequence and the target data propagation path; The construction module is used to: determine the data lineage map of the target data based on the actual propagation node sequence of the target data; The pedigree graph of the target data includes multiple sub-models, each sub-model including a source node; the construction module further includes a sub-model determination unit and a pedigree graph construction unit. The sub-model determination unit is used to: determine N sub-models with the target detection node as the source node based on the target detection node and the N downstream nodes when there are N downstream nodes in the actual propagation node sequence; where N is a positive integer. The lineage graph construction unit is used to: construct a data lineage graph of the target data by using a sub-model with each detection node in the actual propagation node sequence as the source node; Each of the sub-models further includes: a target node, and the propagation type of the target data transmitted from the source node to the target node; the sub-model determination unit includes a target node determination sub-unit, a propagation type determination sub-unit, and a sub-model determination sub-unit; The target node determination subunit is used to: use the target detection node as the source node of the i-th sub-model, and determine the i-th downstream node of the target detection node as the target node of the i-th sub-model, where i takes the value from 1 to N and is an integer containing 1 and N; The propagation type determination subunit is used to: determine the propagation type of the i-th sub-model with the target detection node as the source node and the i-th downstream node as the target node based on the data type of the target data in the target detection node and the data type of the target data in the i-th downstream node; The sub-model determining sub-unit is used to: determine the i-th sub-model with the target detection node as the source node and the i-th downstream node as the target node based on the propagation type of the target detection node, the i-th downstream node, and the i-th sub-model.

9. A computer-readable storage medium storing instructions that, when executed on a computer or processor, cause the computer or processor to perform the method for constructing a data lineage map as described in any one of claims 1 to 7.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein, When the processor executes the computer program, it implements the method for constructing a data lineage map as described in any one of claims 1 to 7.

11. A computer program product comprising instructions that, when run on a computer or processor, causes the computer or processor to perform the method for constructing a data lineage map as described in any one of claims 1 to 7.

Images