Data blood relationship model determination method and device, computing equipment and storage medium

Abstract

The invention provides a method and device for determining a data consanguinity model, computing equipment and a storage medium. The method comprises the steps of obtaining byte code information contained in a byte code file in a to-be-analyzed service system; wherein the byte code information comprises a plurality of method functions of the business system to be analyzed; determining a data consanguinity sub-model corresponding to each method function according to a dependency relationship among a byte code instruction, an operand stack and a local variable table of each method function in the byte code information; and determining a data blood relationship model of the to-be-analyzed service system according to the dependency relationship among the method functions in the byte code information and the data blood relationship sub-models corresponding to the method functions. According to the scheme, the data blood relationship model can be accurately and effectively constructed without using an abstract syntax tree.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, apparatus, computing device and storage medium for determining a data lineage model. Background Art

[0002] As online Internet businesses continue to grow, the number of application business systems and codes is growing exponentially. Faced with such a large amount of code, the R&D team often builds a data lineage model to quickly locate the scope of code defects, assess the scope of business data changes (downward tracking, upward tracing), and generate test cases.

[0003] Existing technologies often construct data lineage models through the abstract syntax tree method, but the abstract syntax tree has the following disadvantages: (1) it cannot represent the data isolation characteristics between objects; (2) it cannot distinguish the edge types of the data lineage model; (3) it cannot represent the relationship between function methods and databases; (4) it cannot simplify the code for third-party data packages.

[0004] So, how to accurately and effectively build a data lineage model without using an abstract syntax tree remains to be solved. Summary of the invention

[0005] The present application provides a method, apparatus, computing device and storage medium for determining a data lineage model, which can accurately and effectively construct a data lineage model without using an abstract syntax tree.

[0006] In the first aspect, an embodiment of the present application provides a method for determining a data lineage model, which can be executed by a device for determining a data lineage model, and the device for determining a data lineage model can be a terminal device or a module for a terminal device, or a server or a module for a server. The present application does not limit the execution subject of the method. The method includes: obtaining bytecode information contained in a bytecode file in a business system to be analyzed; the bytecode information includes multiple method functions of the business system to be analyzed; according to the dependency relationship between the bytecode instructions, operand stack and local variable table of each method function in the bytecode information, determining the data lineage sub-model corresponding to each method function; according to the dependency relationship between each method function in the bytecode information, and the data lineage sub-model corresponding to each method function, determining the data lineage relationship model of the business system to be analyzed.

[0007] The above scheme can accurately and effectively determine the data lineage sub-model corresponding to each method function based on the dependency relationship between the bytecode instructions, operand stack and local variable table of each method function in the bytecode information, avoiding the need in the prior art to first obtain the abstract syntax tree and then construct a data lineage model based on the abstract syntax tree, thereby avoiding the defects of the abstract syntax tree.

[0008] In a possible implementation method, for any bytecode instruction in the method function, a first node corresponding to the bytecode instruction is determined; based on the dependency relationship among the bytecode instruction, the operand stack and the local variable table, a second node that has a dependency relationship with the first node is determined; a connection relationship between the first node and the second node is constructed; and based on the connection relationship between nodes formed by the bytecode instructions of the method function, a data lineage sub-model corresponding to the method function is determined.

[0009] The above solution can achieve accurate and effective determination of the second node, and further can achieve accurate and effective determination of the connection relationship between the first node and the second node.

[0010] In a possible implementation method, the instruction type and the number of operands required during the execution process are determined according to the bytecode instruction; according to the number of operands required during the execution process, a corresponding number of execution operands are obtained from the operand stack; according to the execution operands, the index information corresponding to the execution operands in the local variable table is determined; according to the index information and the instruction type, the first N bytecode instructions that have a dependency relationship with the bytecode instruction are determined; wherein N is a positive integer; according to the first N bytecode instructions, the second node that has a dependency relationship with the first node is determined.

[0011] The above scheme can accurately and effectively determine the execution operand through the operand stack, and can accurately and effectively determine the index information of the execution operand based on the execution operand and the local variable table. According to the index information, it can accurately determine the first N bytecode instructions that have a dependency relationship with the bytecode instruction, and further can accurately and effectively determine the second node that has a dependency relationship with the first node.

[0012] In a possible implementation method, if the bytecode instruction is an object initialization instruction, then the identifier of the object instance initialized by the object initialization instruction is determined; based on the identifier of the object instance, the information of the first node corresponding to the object initialization instruction is determined.

[0013] The above solution sets the identifier of the object instance initialized by the object initialization instruction, which can solve the problem that the data isolation between objects cannot be represented in the abstract syntax tree.

[0014] In one possible implementation method, if the bytecode instruction is not the object initialization instruction but depends on the object initialization instruction, then the identifier of the object instance corresponding to the object initialization instruction is obtained; based on the identifier of the object instance corresponding to the object initialization instruction, the first node corresponding to the bytecode instruction is determined.

[0015] The above scheme sets the identifier of the object instance corresponding to the object initialization instruction for the dependent object initialization instruction. On the one hand, the dependency relationship between different nodes can be determined based on the identifier of the object instance; on the other hand, it solves the problem that data isolation between objects cannot be represented in the abstract syntax tree.

[0016] In one possible implementation method, if the bytecode instruction is a calling third-party instruction, it is determined whether the caller of the calling third-party instruction is located in a third-party configuration file; the third-party configuration file contains multiple third-party identifiers; if the caller of the calling third-party instruction is located in the third-party configuration file, simulation code is used to replace the instruction logic corresponding to the calling third-party instruction, and the first node corresponding to the calling third-party instruction is determined based on the simulation code.

[0017] The above solution uses simulation code to determine the first node corresponding to the call to the third-party instruction, thereby avoiding the problem that the abstract syntax tree cannot simplify the code for the third-party data package. The traditional abstract syntax tree will further parse the code in the third-party data package, but the code in the third-party data package is basically useless for building the data lineage relationship model of the business system to be analyzed. Therefore, if the data lineage relationship model is built using the abstract syntax tree method, it will undoubtedly increase the size of the data lineage relationship model, resulting in the inability to quickly locate code defects and difficulty in assessing the impact of business data changes.

[0018] In a possible implementation method, the third-party configuration file is set; wherein each third-party identifier in the third-party configuration file corresponds to a plurality of third-party data packets having the same function.

[0019] In the above solution, multiple third-party data packages with the same function correspond to one third-party identifier, which can reduce the size of the third-party configuration file and the storage space; and can also reduce many problems caused by large differences in third-party codes.

[0020] In a possible implementation method, the connection relationship between the first node and the second node is determined based on the information of the first node, and the index number and instruction type of the bytecode instruction corresponding to the second node.

[0021] The above scheme can accurately and effectively determine the connection relationship between the first node and the second node, thereby solving the problem that the abstract syntax tree cannot distinguish the edge types of the data lineage model.

[0022] In a possible implementation method, based on the information of the first node, a set of dependent nodes that have a dependent relationship with the first node is determined; the dependent node set includes at least one second node that has a dependent relationship with the first node; if there is only one second node in the dependent node set whose bytecode instruction has an instruction type of local variable transfer to stack instruction ALOAD, then the connection relationship between the first node and the second node is set to a reference relationship, and the connection relationship between the first node and other nodes in the dependent node set is set to a numerical relationship; if there are multiple second nodes in the dependent node set whose bytecode instruction has an instruction type of ALOAD, then the connection relationship between the first node and the third node is set to a reference relationship, and the connection relationship between the first node and other nodes in the dependent node set is a numerical relationship; wherein the third node is a second node whose index number is the first numerical value.

[0023] The above scheme can accurately and effectively determine the connection relationship between the first node and the second node; further, the connection relationship between the first node and the second node is divided into a reference relationship and a numerical relationship, and the data lineage model can be traced according to different connection relationships, thereby improving the traceability efficiency.

[0024] In a possible implementation method, if the bytecode instruction corresponding to the first node is a call interface instruction, the database node is determined based on the information of the first node and the information of the second node; and a connection relationship between the database node, the first node and the second node is established.

[0025] The above solution can achieve accurate and effective determination of database nodes, thereby solving the problem that the relationship between function methods and databases cannot be represented in the abstract syntax tree.

[0026] In a possible implementation method, it is determined whether the caller of the calling interface instruction is located in a database configuration file; the database configuration file includes information of a database corresponding to the business system to be analyzed;

[0027] If the caller of the calling interface instruction is located in the database configuration file, determining whether the bytecode instruction corresponding to the second node is ALOAD; if the bytecode instruction corresponding to the second node is ALOAD, and the variable type of the variable obtained according to the ALOAD is located in the configuration file of the domain model, determining a fourth node having the same identifier as the object instance corresponding to the second node; the configuration file of the domain model includes table information of the database;

[0028] The database node is determined according to the second node and the fourth node.

[0029] The above solution can achieve accurate and effective determination of database nodes, thereby solving the problem that the relationship between function methods and databases cannot be represented in the abstract syntax tree.

[0030] In a possible implementation method, it is determined whether there are database field nodes and database table nodes in the data lineage sub-model; the database field node is used to indicate operations on database fields; the database table node is used to indicate table information of the database; wherein the database node includes the database field node and the database table node; if the database field node does not exist, the information of the database field node is determined according to the fourth node; if the database field node does not exist, the information of the database table node is determined according to the second node.

[0031] The above scheme can realize accurate and effective determination of database field nodes and database table nodes, and further can realize accurate and effective determination of database nodes based on database field nodes and database table nodes.

[0032] In a second aspect, an embodiment of the present application provides a device for determining a data lineage model, comprising: an acquisition unit and a determination unit. The acquisition unit is used to acquire bytecode information contained in a bytecode file in a business system to be analyzed; the bytecode information contains multiple method functions of the business system to be analyzed; the determination unit is used to determine the data lineage sub-model corresponding to each method function according to the dependency relationship between the bytecode instructions, operand stack, and local variable table of each method function in the bytecode information; the data lineage relationship model of the business system to be analyzed is determined according to the dependency relationship between the various method functions in the bytecode information and the data lineage sub-model corresponding to each method function.

[0033] In a possible implementation method, the determination unit is used to determine, for any bytecode instruction in the method function, a first node corresponding to the bytecode instruction; determine a second node that has a dependency relationship with the first node based on a dependency relationship among the bytecode instruction, the operand stack, and the local variable table; construct a connection relationship between the first node and the second node; and determine a data lineage sub-model corresponding to the method function based on the inter-node connection relationship formed by each bytecode instruction of the method function.

[0034] In a possible implementation method, the determination unit is used to determine the instruction type and the number of operands required during the execution process according to the bytecode instruction; obtain the corresponding number of execution operands from the operand stack according to the number of operands required during the execution process; determine the index information corresponding to the execution operands in the local variable table according to the execution operands; determine the first N bytecode instructions that have a dependency relationship with the bytecode instruction according to the index information and the instruction type; wherein N is a positive integer; and determine the second node that has a dependency relationship with the first node according to the first N bytecode instructions.

[0035] In a possible implementation method, the determination unit is used to determine the identifier of the object instance initialized by the object initialization instruction if the bytecode instruction is an object initialization instruction; and determine the information of the first node corresponding to the object initialization instruction based on the identifier of the object instance.

[0036] In a possible implementation method, the determination unit is used to obtain the identifier of the object instance corresponding to the object initialization instruction if the bytecode instruction is not the object initialization instruction but depends on the object initialization instruction; and determine the first node corresponding to the bytecode instruction based on the identifier of the object instance corresponding to the object initialization instruction.

[0037] In one possible implementation method, the determination unit is used to determine whether the caller of the third-party instruction is located in a third-party configuration file if the bytecode instruction is a third-party calling instruction; the third-party configuration file contains multiple third-party identifiers; if the caller of the third-party calling instruction is located in the third-party configuration file, use simulation code to replace the instruction logic corresponding to the third-party calling instruction, and determine the first node corresponding to the third-party calling instruction based on the simulation code.

[0038] In a possible implementation method, the determining unit is used to set the third-party configuration file; wherein each third-party identifier in the third-party configuration file corresponds to a plurality of third-party data packets having the same function.

[0039] In a possible implementation method, the determining unit is used to determine the connection relationship between the first node and the second node based on the information of the first node and the index number and instruction type of the bytecode instruction corresponding to the second node.

[0040] In a possible implementation method, the determination unit is used to determine a set of dependent nodes that have a dependent relationship with the first node based on information of the first node; the dependent node set includes at least one second node that has a dependent relationship with the first node; if there is only one second node in the dependent node set whose bytecode instruction has an instruction type of local variable transfer to stack instruction ALOAD, then the connection relationship between the first node and the second node is set to a reference relationship, and the connection relationship between the first node and other nodes in the dependent node set is set to a numerical relationship; if there are multiple second nodes in the dependent node set whose bytecode instruction has an instruction type of ALOAD, then the connection relationship between the first node and the third node is set to a reference relationship, and the connection relationship between the first node and other nodes in the dependent node set is a numerical relationship; wherein the third node is a second node whose index number is the first numerical value.

[0041] In a possible implementation method, the determination unit is used to determine the database node based on information of the first node and information of the second node if the bytecode instruction corresponding to the first node is a call interface instruction; and to establish a connection relationship between the database node, the first node and the second node.

[0042] In a possible implementation method, the determining unit is used to determine whether the caller of the calling interface instruction is located in a database configuration file; the database configuration file includes information about a database corresponding to the business system to be analyzed;

[0043] If the caller of the calling interface instruction is located in the database configuration file, determine whether the bytecode instruction corresponding to the second node is ALOAD; if the bytecode instruction corresponding to the second node is ALOAD, and the variable type of the variable obtained according to the ALOAD is located in the configuration file of the domain model, then determine the fourth node with the same identifier as the object instance corresponding to the second node; the configuration file of the domain model includes the table information of the database; determine the database node based on the second node and the fourth node.

[0044] In a possible implementation method, the determination unit is used to determine whether there is a database field node and a database table node in the data lineage sub-model; the database field node is used to indicate operations on a database field; the database table node is used to indicate table information of a database; wherein the database node includes the database field node and the database table node; if the database field node does not exist, the information of the database field node is determined based on the fourth node; if the database field node does not exist, the information of the database table node is determined based on the second node.

[0045] In a third aspect, an embodiment of the present application further provides a computing device, including:

[0046] A memory for storing program instructions;

[0047] The processor is used to call the program instructions stored in the memory, and execute any method of implementing the above-mentioned first aspect according to the obtained program instructions.

[0048] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, in which computer-readable instructions are stored. When a computer reads and executes the computer-readable instructions, any method of the above-mentioned first aspect is implemented.

[0049] In a fifth aspect, an embodiment of the present application provides a computer program product, comprising a computer program executable by a computer device, wherein when the program is run on the computer device, the computer device executes any method for implementing the above-mentioned first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 A flowchart of a method for determining a data lineage model provided in an embodiment of the present application;

[0051] Figure 2 A flowchart of a method for determining a data lineage model provided in an embodiment of the present application;

[0052] Figure 3 A schematic diagram of the structure of bytecode information provided in an embodiment of the present application;

[0053] Figure 4 A code example diagram provided for an embodiment of the present application;

[0054] Figure 5 A schematic diagram of a bytecode instruction and its corresponding operand stack and local variable table change process provided in an embodiment of the present application;

[0055] Figure 6 A schematic diagram of the structure of a data lineage model provided in an embodiment of the present application;

[0056] Figure 7 A code example diagram provided for an embodiment of the present application;

[0057] Figure 8 A code example diagram provided for an embodiment of the present application;

[0058] Fig. 9 A schematic diagram of the structure of bytecode information provided in an embodiment of the present application;

[0059] Fig.10A schematic diagram of the structure of bytecode information provided in an embodiment of the present application;

[0060] Fig.11 A schematic diagram of the structure of bytecode information provided in an embodiment of the present application;

[0061] Fig.12 A schematic diagram of the structure of bytecode information provided in an embodiment of the present application;

[0062] FIG. 13( a ) is a schematic diagram of the structure of a data lineage model provided in an embodiment of the present application;

[0063] FIG13( b ) is a schematic diagram of the structure of a data lineage model provided in an embodiment of the present application;

[0064] Fig.14 A code example diagram provided for an embodiment of the present application;

[0065] Fig.15 A schematic diagram of the structure of bytecode information provided in an embodiment of the present application;

[0066] Fig.16 A schematic diagram of the structure of a data lineage model provided in an embodiment of the present application;

[0067] Fig.17 A schematic diagram of the structure of a database configuration file provided in an embodiment of the present application;

[0068] Fig.18 A flowchart of a method for determining a database node provided in an embodiment of the present application;

[0069] Fig.19 A code example diagram provided for an embodiment of the present application;

[0070] Fig. 20 A code example diagram provided for an embodiment of the present application;

[0071] Fig.21 A code example diagram provided for an embodiment of the present application;

[0072] Fig. 22 A schematic diagram of the structure of bytecode information provided in an embodiment of the present application;

[0073] Fig.23 A schematic diagram of the structure of a data lineage model provided in an embodiment of the present application;

[0074] Fig.24 A schematic diagram of the structure of a device for determining a data lineage model provided in an embodiment of the present application;

[0075] Fig.25A schematic diagram of the structure of a device for determining a data lineage model provided in an embodiment of the present application. DETAILED DESCRIPTION

[0076] The following is a brief description of the basic terms of this application.

[0077] Object-oriented language: It is a type of programming language that uses objects as the basic unit of program structure. It means that the design used for description is centered on objects, and objects are the basic components of the program at runtime. Object-oriented languages ​​provide components such as classes and inheritance, and have four main characteristics: recognition, polymorphism, classification, and inheritance.

[0078] Data Lineage Relation: Data lineage relationship refers to the relationship between data in the process of data generation, processing, circulation and extinction, which is similar to the blood relationship in human society.

[0079] Abstract Syntax Tree (AST): AST for short, it is an abstract representation of the syntax structure of the source code. It represents the syntax structure of the programming language in a tree form, and each node on the tree represents a structure in the source code. Each node in the abstract syntax tree represents a syntax construct in the source code, such as variable declaration, expression, function call, control structure, etc. The root node of the tree usually represents the entire source code file, while the child nodes represent specific syntax elements and their relationships. For example, an abstract syntax tree node for a function declaration may contain multiple child nodes, such as function name, parameter list, and function body. The abstract syntax tree is a tree structure that contains all the syntax information of the code, and it is a direct translation of the code. The method for determining data lineage based on the abstract syntax tree is: based on the information of the abstract syntax tree, obtain the function statements in the business system to be analyzed, perform data lineage analysis on the function statements, and determine the data lineage of the business system to be analyzed.

[0080] Data Access Object (DAO): DAO is a crucial component that encapsulates all access to data sources. The DAO layer is an intermediate layer in the application architecture, located between the business logic layer and the data storage layer.

[0081] Entity Class: represents a table in the database. Each entity class corresponds to a database table. The attributes of the entity class correspond to the columns of the database table.

[0082] Figure 1A flow chart of a method for determining a data lineage model provided in an embodiment of the present application, the method can be executed by a device for determining a data lineage model, the device for determining a data lineage model can be a terminal device or a module for a terminal device, or a server or a module for a server. The present application does not limit the execution subject of the method.

[0083] The method comprises the following steps:

[0084] Step 101, obtaining bytecode information contained in a bytecode file in a business system to be analyzed.

[0085] The bytecode information includes multiple method functions of the business system to be analyzed.

[0086] The business system to be analyzed can be any system, for example, it can be the background service system of a certain application, or the business system that provides a certain service, etc. The business system to be analyzed can be a system composed of an independent system, or a system composed of multiple different systems that can complete a certain business. For example, the trigger position of business 1 is in application A. When the user triggers business A, it can pass through the background service system of application A and the business service system where business 1 is located, and finally reach the service system of business 1. In this way, it is necessary to pass through the three systems of the above-mentioned background service system, business service system and service system of business 1. The bytecode file can be a file for storing bytecode information. In actual applications, the bytecode file can be a file with a .class suffix. In actual applications, the bytecode file can be a class file. The bytecode information can be binary information composed of program code and / or data pairs. The class file can be a file compiled by a computer, and can be a file composed of a language that can be recognized by a computer.

[0087] Optionally, in the specific implementation process, the bytecode file in the system to be analyzed can be obtained. For example, a file with a file suffix of .class can be selected from the business system to be analyzed, wherein the .class file can be a bytecode file. Each bytecode file can be parsed to obtain the bytecode information contained therein, wherein the bytecode information can include multiple method functions of the business system to be analyzed, and each method function includes but is not limited to class name information, the name of the inherited parent class, the generic information of the class, the attribute list of the class, the class function list and the implemented interface class name list, etc. The class attribute list can include attribute types and attribute names, etc., and the class function list can include function names, input parameters and return parameters, etc.

[0088] Step 102, determining the data lineage sub-model corresponding to each method function according to the dependency relationship among the bytecode instructions, operand stack and local variable table of each method function in the bytecode information.

[0089] Among them, the operand stack is part of the frame structure in the Java virtual machine (JVM), which is used to store local variables, operands, and intermediate results during the execution of the method. It is a last-in-first-out (LIFO) data structure, similar to the abstract data type of the stack. In Java, the main functions of the operand stack are: storing intermediate results: when performing arithmetic operations, logical operations, etc., the operand stack is used to store operands and operation results. Supporting method calls: when calling a method, the operand stack is used to pass parameters and receive return values. Implementing control flow: In the operand stack, control flow statements such as conditional jumps and loops can be easily implemented through push and pop operations.

[0090] Local Variables: Local Variables, also known as local variable arrays or local variable tables, are mainly used to store method parameters and local variables defined in the method body. These data types include various basic data types, object references, and returnAddress (return value) types. The local variable table is an important part of the stack frame. It is used to save function parameters and local variables. The variables in the local variable table are only valid in the current function call. When the function call ends, the local variable table will be destroyed along with the destruction of the function stack frame.

[0091] Exemplarily, the data lineage sub-model corresponding to each method function is as follows: a=b+c, ​​then it can be determined that the lineage of object a comes from object b and object c, and its lineage relationship is b+c. The lineage relationship of object b, the lineage relationship of object c, and the lineage relationship of b+c can be further determined in the above manner, and the lineage relationship is deduced and analyzed accordingly, and finally the data lineage sub-model of the method function a=b+c is determined. Different method functions correspond to different data lineage sub-models. According to the dependency relationship between each method function and the data lineage sub-model corresponding to each method function, the data lineage relationship model of the business system to be analyzed is determined.

[0092] Step 103, determining the data lineage model of the business system to be analyzed according to the dependency relationship between the various method functions in the bytecode information and the data lineage sub-model corresponding to each method function.

[0093] In a possible implementation method, the dependency relationship between various method functions is determined by a business management method, and then the data lineage relationship model of the business system to be analyzed is determined.

[0094] In another possible implementation method, the dependency relationship between the method functions is determined by the relationship between the parameters passed between the method functions, and then the data lineage relationship model of the business system to be analyzed is determined. This application does not limit the method for determining the dependency relationship between the method functions.

[0095] The above scheme can accurately and effectively determine the data lineage sub-model corresponding to each method function based on the dependency relationship between the bytecode instructions, operand stack and local variable table of each method function in the bytecode information, avoiding the need in the prior art to first obtain the abstract syntax tree and then construct a data lineage model based on the abstract syntax tree, thereby avoiding the defects of the abstract syntax tree.

[0096] In the above step 102, the specific implementation method of the data lineage sub-model corresponding to each method function is determined according to the dependency relationship between the bytecode instructions, operand stack and local variable table of each method function in the bytecode information, such as Figure 2 As shown, the following steps are included:

[0097] Step 201: for any bytecode instruction in the method function, determine the first node corresponding to the bytecode instruction.

[0098] In a possible implementation method, the information of the first node includes one or more of the following: a node identifier, a name, a bytecode instruction, a source code line number, and a class and method to which it belongs.

[0099] The node identification, namely the node ID, is used to determine the uniqueness of the node.

[0100] Optionally, the node identifier is determined according to one or more of the following: the class name, method signature, bytecode instruction index number, and object instance identifier corresponding to the method function. Exemplarily, the class name, method signature, bytecode instruction index number, and object instance identifier are string concatenated, and the concatenated result is hashed to obtain the node identifier.

[0101] Optionally, the class name and method signature can be obtained by ASM when parsing the Java Class file to construct ClassNode and MethodNode. The instruction code index number can be obtained by traversing the Java method (MethodNode) instruction set. ASM is a general Java bytecode manipulation and analysis framework. It can be used to modify existing classes or dynamically generate classes directly in binary form. ASM provides some common bytecode transformation and analysis algorithms, from which custom complex transformation and code analysis tools can be built.

[0102] Optionally, the bytecode instruction index number is used to indicate the index position of the Java bytecode instruction in the current parsing method.

[0103] In a possible implementation method, the bytecode instructions are traversed and analyzed. When the bytecode instruction type is an instruction for operating variables (such as ISTORE and ILOAD), the name is the corresponding variable name, such as a, b, and c. When the bytecode instruction is a calculation instruction (such as IADD), the name is the instruction abbreviation enumeration name, such as add.

[0104] Optionally, in addition to variable operation instructions and calculation instructions, there are also method call instructions. When a method call instruction is encountered, if the called method is the project method (which can be determined by the package path), the analysis will continue in the method. If the called method comes from a third-party package, an alternative method is used to simplify the internal logic, which can indicate the data flow.

[0105] In one possible implementation method, in the process of building a data lineage model, it is not necessary to generate a node for each instruction, but only to generate nodes for instructions that operate variables.

[0106] In a possible implementation method, the instruction name is used to indicate a Java bytecode instruction name.

[0107] In a possible implementation method, the bytecode instructions are traversed and analyzed, and when a LINENUMBER bytecode instruction is encountered, the source code line number of the bytecode instruction is saved in the memory, and when the subsequent bytecode instructions are traversed, the source code line number is obtained from the memory. When the LINENUMBER bytecode instruction is traversed again, the source code line number in the memory is updated.

[0108] In a possible implementation method, the belonging class is used to indicate the class to which the current bytecode instruction belongs, that is, the class currently being analyzed.

[0109] Optionally, the bytecode instructions, source code line numbers, and the classes and methods to which they belong can all be obtained naturally when traversing the bytecode instructions.

[0110] In a possible implementation method, in the data lineage model, edge information includes a starting point, an end point, a type, etc., and the edge type is a numerical type by default.

[0111] Step 202: Determine a second node that has a dependency relationship with the first node based on the dependency relationship among the bytecode instruction, the operand stack, and the local variable table.

[0112] In a possible implementation method, the instruction type and the number of operands required during the execution process are determined according to the bytecode instruction; according to the number of operands required during the execution process, a corresponding number of execution operands are obtained from the operand stack; according to the execution operands, the index information corresponding to the execution operands in the local variable table is determined; according to the index information and the instruction type, the first N bytecode instructions that have a dependency relationship with the bytecode instruction are determined; wherein N is a positive integer; according to the first N bytecode instructions, the second node that has a dependency relationship with the first node is determined.

[0113] Step 203: construct a connection relationship between the first node and the second node.

[0114] Step 204, based on the inter-node connection relationship formed by each bytecode instruction of the method function, determine the data lineage sub-model corresponding to the method function.

[0115] The above steps 201 to 204 are described in detail below using a specific example.

[0116] First, obtain the jar package of the application system and load the jar package, and traverse the Java Class files in the jar package. By analyzing the Java Class files, obtain the bytecode instructions, operand stack and local variable table of the Java method, and analyze the method bytecode instructions, such as Figure 3 As shown, Figure 3 The corresponding method function is as follows Figure 4 As shown, it is mainly used to obtain c=a+b.

[0117] Figure 3In the example, Ln (where n is a number from 0 to 4) and LINENUMBER bytecode instructions are not involved in the principle introduction of this embodiment and are temporarily ignored. First, it can be known from the MAXSTACK=2 instruction that the maximum depth of the operand stack is 2. It can be known from the MAXLOCALS=4 instruction that the maximum size of the local variable table is 4. And from the four LOCALVARIABLE instructions, it can be known that the variable names are this, a, b, and c, and the corresponding types are reference types (cn / webank / cnc / datatrace / demo / TestCase), integer, integer, and integer; at the same time, the scope of the local variable and the index position in the local variable table can also be known. MAXSTACK and MAXLOCAL are verified by the JVM during class loading to ensure security and can be ignored when building a lineage model. Each LOCALVARIABLE in the local variable table represents a variable, and the variable information, such as the variable name and variable type, can be obtained through this information.

[0118] In a possible implementation method, the bytecode instruction and its corresponding operand stack and local variable table change process in this example are as follows: Figure 5 As shown, the determined data lineage model is as follows Figure 6 shown.

[0119] like Figure 5-1 As shown, initially the operand stack is empty and the local variable table does not contain the values ​​of local variables.

[0120] 1. When executing the LDC 100000 bytecode instruction, 100000 is pushed into the operand stack, such as Figure 5-2 shown.

[0121] 2. When executing the ISTORE 1 bytecode instruction, an element in the operand stack is used to pop the top element 100000 in the operand stack and save it to the variable with index position 1 in the local variable table, such as Figure 5-3 shown.

[0122] 3. When executing the ICONST_2 bytecode instruction, push 2 into the operand stack, such as Figure 5-4 shown.

[0123] 4. When executing the ISTORE_2 bytecode instruction, an element in the operand stack is used to pop the top element 2 in the operand stack and save it to the variable with index position 2 in the local variable table, such as Figure 5-5 shown.

[0124] 5. When the ILOAD 1 bytecode instruction is executed, the variable with index position 1 is read from the local variable table and pushed into the operand stack, such as Figure 5-6 shown.

[0125] 6. When the ILOAD 2 bytecode instruction is executed, the variable with index position 2 is read from the local variable table and pushed into the operand stack, such as Figure 5-7 shown.

[0126] 7. When executing the IADD bytecode instruction, two elements in the operand stack are needed. The two elements in the operand stack are popped and added. The result 100002 is pushed into the operand stack. For example, Figure 5-8 shown.

[0127] 8. When executing the ISTORE 3 bytecode instruction, you need to use an element in the operand stack, pop an element from the operand stack, and save the element to the variable at index position 3 in the local variable table, such as Figure 5-9 shown.

[0128] The variable c is at index 3 in the local variable table. The variable at index 3 in the local variable table is assigned the top element of the operand stack by the ISTORE3 bytecode instruction. When the ISTORE 3 bytecode is executed, the top element of the stack is generated and pushed to the top of the stack by the previous instruction of the ISTORE 3 bytecode, the IADD bytecode instruction.

[0129] The input for executing the IADD bytecode instruction is the two top stack elements. When executing the IADD bytecode, the two top stack elements are retrieved from the local variable table through the first two instructions of the IADD bytecode, ILOAD 1 and ILOAD 2. ILOAD 1 retrieves the variable a at index position 1 in the local variable table and pushes it to the top of the stack.

[0130] The variable at index 1 in the local variable table is assigned the top element of the operand stack by the ISTORE 1 bytecode instruction before ILOAD 1. When the ISTORE 1 bytecode is executed, the top element of the stack is generated and pushed onto the top of the stack by the previous instruction of the ISTORE 1 bytecode, the LDC 100000 bytecode instruction. The same can be said for ILOAD 2.

[0131] The unified processing rule is to analyze the variable reading instructions (such as ILOAD, etc.), the storage variable instructions (such as ISTORE, etc.) and the operand stack when executing the storage variable instructions.

[0132] The variable name is obtained by reading the local variable table to specify the index position variable instruction (such as ILOAD n), and the variable name is obtained by the previous instruction of storing the local variable table to specify the index position variable (such as ISTORE n).

[0133] Then get the previous instruction of the instruction that stores the variable at the specified index position in the local variable table. If the instruction has output (the output result is pushed to the top of the data stack), then according to the number of top stack elements required by the instruction, find the previous variable reading class instructions of the corresponding number, and so on. If the instruction does not have output, continue to search for the previous instruction until an instruction with output is found.

[0134] Through analysis, we can obtain the dependency relationship between bytecode instructions, operand stack elements, and local variable table elements.

[0135] The ISTORE 3 instruction depends on the element pushed onto the top of the stack by the IADD instruction. The IADD instruction depends on the two elements at the top of the stack before the instruction is executed. By analogy, the dependency of variables can be determined. Variable c depends on a and b and is obtained by adding a and b.

[0136] Taking this example as an example, how to determine the second node that has a dependency relationship with the first node according to the dependency relationship between the bytecode instruction, the operand stack and the local variable table is explained in detail. If the IADD bytecode instruction is executed, it is determined according to the IADD bytecode instruction that the bytecode instruction requires two operands, then the execution operands obtained from the operand stack are 100000 and 2; the execution operand 100000 corresponds to the index number 1 in the local variable table, and the execution operand 2 corresponds to the index number 2 in the local variable table. Then, according to the execution operands corresponding to the index numbers and the IADD instruction types, the first two bytecode instructions that have a dependency relationship with the IADD bytecode instruction are determined to be ILOAD 1 and ILOAD 2; the node corresponding to ILOAD 1 is determined to be the second node, and the node corresponding to ILOAD 2 is determined to be the second node. For the convenience of expression, this application refers to the nodes that have a dependency relationship with the first node as the second node, but this application does not limit the number of second nodes, because there may be multiple nodes that have a dependency relationship with the first node.

[0137] In one embodiment, in the above step 201, determining the first node corresponding to the bytecode instruction includes: if the bytecode instruction is an object initialization instruction, determining the identifier of the object instance initialized by the object initialization instruction; and determining the information of the first node corresponding to the object initialization instruction according to the identifier of the object instance. That is, the information of the first node includes the identifier of the object instance in addition to the node identifier, name, bytecode instruction, source code line number, and one or more of the class and method to which it belongs.

[0138] Optionally, the node ID of the first node is determined according to an identifier of the object instance of the first node.

[0139] Optionally, if the bytecode instruction is not the object initialization instruction but depends on the object initialization instruction, then the identifier of the object instance corresponding to the object initialization instruction is obtained; and based on the identifier of the object instance corresponding to the object initialization instruction, the first node corresponding to the bytecode instruction is determined.

[0140] Object-oriented programming emphasizes object encapsulation, inheritance, and polymorphism, while procedural programming emphasizes the order and steps of the process. Correspondingly, there are also significant differences in building data lineage models. Take the following object-oriented methods setName and getName as examples. Figure 7 and Figure 8 As shown in the figure, the setName method sets the name attribute of the User type object, and the getName method obtains the name attribute of the User type object. Calling the getName method needs to return the value set by calling the setName method of the specified object. For example, calling the getName method of the user1 object needs to return the value kevin set by calling the setName method of the user1 object, and calling the getName method of the user2 object needs to return the value james set by calling the setName method of the user2 object.

[0141] However, user1 is initialized as an object instance through the object initialization instruction, i.e., the new instruction, and user2 is also initialized as another object instance through the object initialization instruction, i.e., the new instruction. After user1 is assigned the value of kevin and user2 is assigned the value of james, since the traditional abstract syntax tree cannot represent the data isolation characteristics between objects, in the data lineage model, it is impossible to distinguish which object instance the value of user1.getName obtains, that is, it is impossible to distinguish whether user1.getName obtains kevin or james. Similarly, it is impossible to distinguish whether user2.getName obtains kevin or james, resulting in errors in the nodes of the data lineage model and the connection relationships between the nodes. What's more, if the object instance plays a greater role in the entire function method, the data lineage model may not be formed.

[0142] This application obtains Figure 7 The corresponding bytecode information, such as Fig. 9 and Fig.10 As shown; get Figure 8 The corresponding bytecode information, such as Fig.11 and Fig.12 shown.

[0143] Specifically, the bytecode (class) files in the application jar package are traversed, and the bytecode instructions in the bytecode files are parsed one by one. When parsing the object initialization instruction (such as Fig.11, NEW instruction on line 23), a unique ID of the object instance, i.e., the identifier of the object instance, is generated and stored in the initialization node of the object (such as the NEW node in the upper left corner of Figure 13(a)), and added to the bloodline model.

[0144] Continue parsing subsequent instructions. If the current instruction (such as Fig.11 , ASTORE instruction on line 26) depends on the object initialization instruction (such as Fig.11 , NEW instruction on line 23). According to the index and instruction type of the object initialization instruction, the node of the object initialization instruction is queried to obtain the object instance ID, and the object instance ID is assigned to the current (instruction) node.

[0145] When the parsed object (user1) calls the object method (setName) to set the class attribute (kevin) instruction (such as Fig.11 , when INVOKEVIRTUAL is called on line 31, it enters the setName method of User to continue parsing.

[0146] When parsing the setName method of User, we first know from the method signature and the local variable table that the method has two input parameters, namely the object this and name, which are located at index 0 and 1 respectively (in the local variable table, the input parameter variables are stored at the front). Create the corresponding PARAM parameter node. The PARAM parameter node name is the same as the corresponding variable name in the local variable table, and the instruction name is PARAM.

[0147] The PARAM node named this at index 0 is associated with the method called (such as Fig.11 , the first input parameter ALOAD (such as INVOKEVIRTUAL) in line 31 Fig.11 , line 29 instruction) connection.

[0148] Similarly, connect the PARAM node at index 1 and call this method (such as Fig.11 , the second input parameter LDC (such as INVOKEVIRTUAL) in line 31 Fig.11 , line 29 instruction) connection.

[0149] Parse the first instruction of the setName method of User ( Fig.10 In line 39, we know that the instruction is to read the parameter at index 0 in the local variable table, so we create an ALOAD node and connect it to the PARAM node named this at index 0. Fig.10 The node generated by the ALOAD 1 instruction in line 40 is connected to the PARAM node named this at index 1.

[0150] When parsed to Fig.10 , the PUTFIELD instruction on line 41 generates a (PUTFIELD) node for setting class attributes, binds the object instance ID to the node, and stores the binding relationship in the global cache (the binding relationship can be maintained in the database or the parser memory).

[0151] When the parsed object calls the object method to obtain the class attribute instructions (such as Fig.11 , INVOKEVIRTUAL instruction on line 46), enters the calling method, and when parsing to the GEFIELD instruction (such as Fig. 9 , line 27), query the object initialization node information in the bloodline model diagram according to the dependent object information to obtain the object instance ID, and obtain the node for setting the class attribute in the global cache according to the class attribute name (the PUTFIELD node mentioned in the previous article). Connect the two associated nodes, and add a data flow edge that sets the type attribute node to the node that gets the class attribute. For example, "User.name PUTFIELD" points to "User.nameGETFIELD", indicating that the "User.name GETFIELD" data comes from "User.name PUTFIELD".

[0152] In one embodiment, in the above step 201, determining the first node corresponding to the bytecode instruction includes: if the bytecode instruction is a calling third-party instruction, determining whether the caller of the calling third-party instruction is located in a third-party configuration file; the third-party configuration file contains multiple third-party identifiers; if the caller of the calling third-party instruction is located in the third-party configuration file, using simulation code to replace the instruction logic corresponding to the calling third-party instruction, and determining the first node corresponding to the calling third-party instruction based on the simulation code.

[0153] Optionally, due to the large differences in third-party codes, this application uses the interface mode for subsequent expansion. The interface implementer can query the data lineage model input node and customize the expression node relationship.

[0154] Optionally, the third-party configuration file is set; wherein each third-party identifier in the third-party configuration file corresponds to multiple third-party data packets with the same function. Exemplarily, multiple third-party data packets all contain a copy function, but the name of the copy function of each third-party data packet is different, for example, the name of the copy function corresponding to third-party data packet 1 is copy1, the name of the copy function corresponding to third-party data packet 2 is copy2, and the name of the copy function corresponding to third-party data packet 3 is copy3, then the third-party identifier corresponding to the copy function is set to copy, and the third-party identifier corresponds to the copy functions in third-party data packet 1, third-party data packet 2, and third-party data packet 3, respectively.

[0155] Optionally, the method corresponding to the caller of the third-party instruction has a one-to-one mapping relationship with the simulation method.

[0156] For example, Fig.14 As shown, when the method ( Fig.14 BeanUtils.copyProperties(record, creditOccupationReq) is called in line 266 of the code. BeanUtils is a tool class in the third-party jar package (springframework). Its copyProperties method is implemented through reflection and other technologies. The code is complex and has poor readability, and the meaning of the method cannot be intuitively reflected. The traditional abstract syntax tree method needs to enter the third-party data package to further parse the code in the third-party data package. However, the code in the third-party data package is not very useful for analyzing the data lineage model of the business system. The data lineage model generated according to the abstract syntax tree is relatively large, but has poor usability.

[0157] Get Fig.14 The bytecode information corresponding to the code, such as Fig.15 shown.

[0158] in Fig.15 Line 732 indicates that the copyProperties method of org / springframework / beans / BeanUtils is called. It should be noted that the calling instructions refer to related instructions such as INVOKESPECIAL, INVOKESTATIC, and INVOKEVIRTUAL. Here we collectively refer to them as calling third-party instructions.

[0159] When building a data lineage model, obtain the caller of the third-party instruction, such as Fig.15Line 732, INVOKESTATIC calls the org / springframework / beans / BeanUtils directive. Query the third-party configuration file to determine whether the fully qualified name of the caller's class begins with the characters in the fully qualified name prefix list of this third-party class. If a match is found, the internal implementation logic is replaced with simulation code to simplify the model complexity.

[0160] like Fig.14 Line 266 indicates that all attributes of record are copied to the creditOccupationReq object. Take one of the serviceCode attributes as an example. Fig.16 The assignment node (node ​​ID: 104994798) in obtains the creditOccupationReq object reference and the serviceCode attribute of the record object respectively, and performs the copy operation. Fig.16 Based on Fig.15 The data lineage model constructed from Fig.16 It can be seen that this application does not further parse the third-party data packet, but determines the first node corresponding to the call of the third-party instruction based on the third-party identifier (copyProperties).

[0161] In one embodiment, in the above step 203, building a connection relationship between the first node and the second node includes: determining the connection relationship between the first node and the second node based on information of the first node, and the index number and instruction type of the bytecode instruction corresponding to the second node.

[0162] The edge types of traditional syntax abstract trees are all numerical types, that is, data types. The data lineage model built based on the syntax abstract tree cannot distinguish the edge types. Of course, after the data lineage model is successfully built, it can be manually set later; however, since the data lineage model is relatively large and has many branches, if it relies entirely on manual settings, it will not only be time-consuming and labor-intensive, but also extremely error-prone.

[0163] In the present application, during the code execution phase, a data lineage model can be generated based on bytecode information, and the edge type of the data lineage model can be determined.

[0164] by Figure 7 and Figure 8 For example, an object of type User2 is created, and the name attribute of user2 is set to james. When user2 calls the User#setName method, user2 is passed to this in the setName method as an object of type User, and the parameter name is assigned to the name attribute of user2 through this.name = name.

[0165] When it is necessary to trace the value of user1.getName(), trace upward according to the getName() node in the lower left corner of Figure 13(a). Tracing back to User.name (GETFIELD node), the GETFIELD node depends on two nodes, one is the PUTFIELD node and the other is the ALOAD node. The edge relationship between the GETFIELD node and the PUTFIELD node is of the data type (i.e., represented by the solid line), and the edge relationship between the GETFIELD node and the ALOAD node is of the reference type (i.e., represented by the dotted line). When tracing back to the ALOAD node, because the edge relationship between the GETFIELD node and the ALOAD node is of the reference type, there is no need to trace upward; when tracing back to the PUTFIELD node, because the edge relationship between the GETFIELD node and the PUTFIELD node is of the value type, it is necessary to trace upward until the final node kevin (LDC) is traced back.

[0166] In other words, it is not necessary to distinguish the edge types in the data lineage model. All edge types are numerical types. Therefore, each time the source is traced, it is necessary to trace back to the final node, which is inefficient. This application fully expresses the code semantics through different edge types and improves the traceability efficiency.

[0167] Optionally, based on the information of the first node, a set of dependent nodes that have a dependency relationship with the first node is determined; the dependent node set includes at least one second node that has a dependency relationship with the first node; if there is only one second node in the dependent node set whose bytecode instruction has an instruction type of local variable transfer to stack instruction ALOAD, then the connection relationship between the first node and the second node is set to a reference relationship, and the connection relationship between the first node and other nodes in the dependent node set is set to a numerical relationship; if there are multiple second nodes in the dependent node set whose bytecode instruction has an instruction type of ALOAD, then the connection relationship between the first node and the third node is set to a reference relationship, and the connection relationship between the first node and other nodes in the dependent node set is a numerical relationship; wherein the third node is a second node whose index number is the first numerical value.

[0168] Exemplarily, the PUTFIELD node on the left in FIG. 13(a) is the first node, and the dependent node set that has a dependent relationship with the PUTFIELD node is {(this, ALOAD) node, (name, ALOAD) node}, and the instruction type of the bytecode instructions of the two nodes in the dependent node set is ALOAD, then the connection relationship between the node with index number 0 (i.e., the (this, ALOAD) node) and the PUTFIELD node is set to a reference relationship, and the node with index number 0 is also the first node on the left that has a dependent relationship with the first node. The connection relationship between the node with index number 1 (i.e., the (name, ALOAD) node) and the PUTFIELD node is set to a numerical relationship.

[0169] In Figure 13(a), the GETFIELD node on the left is the first node, and the dependent node set that has a dependency relationship with the GETFIELD node is (PUTFIELD node, ALOAD node). There is only one node in the dependent node set whose bytecode instruction has an instruction type of ALOAD. Therefore, the connection relationship between the ALOAD node and the GETFIELD node is set to a reference relationship, and the connection relationship between the PUTFIELD node and the GETFIELD node is set to a numerical relationship.

[0170] Figure 13(b) is based on Fig.11 The data lineage model constructed by another object instance user2 in the bytecode information in the bytecode information has the same connection relationship between the nodes as above, which will not be repeated here. It can be clearly seen that the two object instances user1 and user2 correspond to two data lineage models respectively, and the connection relationship between the nodes is different.

[0171] In one embodiment, in the above step 203, a connection relationship between the first node and the second node is constructed, including: if the bytecode instruction corresponding to the first node is a call interface instruction, determining the database node according to the information of the first node and the information of the second node; and constructing a connection relationship between the database node, the first node and the second node.

[0172] The traditional abstract syntax tree cannot represent the relationship between a function and a database (or other global data). Therefore, the data lineage model constructed based on the abstract syntax tree cannot represent the relationship between a function and a DB table. This application constructs the relationship between a function and a DB table (or other global data) by declaring globally unique nodes for the DB table (or other global data) and fields.

[0173] Optionally, a database configuration file and a domain model configuration file are set. The database configuration file includes information about the database corresponding to the business system to be analyzed; the domain model configuration file includes table information of the database. Exemplarily, the database configuration file and the domain model configuration file are as follows: Fig.17 As shown. The daoPackageList configuration item indicates the fully qualified name prefix of the class of the data access layer, that is, the database configuration file. If the fully qualified name of the class begins with the characters in this list, then the class is a class of the data access layer, and the methods in it are methods for operating the DB. The dtoPackageList configuration item indicates the fully qualified name prefix of the class of the domain model, that is, the configuration file of the domain model. If the fully qualified name of the class begins with the characters in this list, then the class is a class of the domain model. Among them, the domain model is a representation of the database table in Java.

[0174] In a possible implementation method, methods for operating the DB and domain model classes are identified by configuring the data access layer path, method name prefix, domain model (Entity) layer path, etc.

[0175] In one possible implementation method, the database node is determined as follows: Fig.18 As shown, the following steps are included:

[0176] Step 1801, determine whether the caller of the calling interface instruction is located in the database configuration file.

[0177] The database configuration file includes information about the database corresponding to the business system to be analyzed.

[0178] Step 1802: If the caller of the calling interface instruction is located in the database configuration file, determine whether the bytecode instruction corresponding to the second node is ALOAD.

[0179] The configuration file of the domain model includes table information of the database.

[0180] In a possible implementation method, it is determined whether the instruction type of the bytecode instruction corresponding to the second node having a dependency relationship with the first node is ALOAD, if so, step 1803 is executed; if not, the process ends. In theory, if the bytecode instruction of the first node is a call interface instruction, the bytecode instruction of the second node is ALOAD.

[0181] Step 1803: If the bytecode instruction corresponding to the second node is ALOAD, and the variable type of the variable obtained according to the ALOAD is located in the configuration file of the domain model, determine a fourth node having the same identifier as the object instance corresponding to the second node.

[0182] In one possible implementation method, if the bytecode instruction corresponding to the second node is ALOAD, the local variable table is read according to the ALOAD instruction, and the variable type of the variable obtained by ALOAD is obtained from the local variable table. If the variable type of the variable is located in the configuration file of the domain model, the node with the same identifier as the object instance corresponding to the second node in the data lineage model is queried.

[0183] Step 1804: determine the database node according to the second node and the fourth node.

[0184] In a possible implementation method, the database node is determined according to the second node and the fourth node, including: judging whether there is a database field node and a database table node in the data lineage sub-model; the database field node is used to indicate operations on database fields; the database table node is used to indicate table information of the database; wherein the database node includes the database field node and the database table node; if the database field node does not exist, determining the information of the database field node according to the fourth node; if the database field node does not exist, determining the information of the database table node according to the second node.

[0185] Optionally, the database field node is set as a node having a dependency relationship with the fourth node; and the database table node is set as a node having a dependency relationship with the second node.

[0186] For example, Fig.19 As shown, Fig.19 In line 284 of AccountServiceImpl.java, tmAccountO is a memory object of the TM_ACCOUNT_O table, and the creditLimit attribute (i.e., the CREDIT_LIMIT field of the TM_ACCOUNT_O table) is set. Among them, the TM_ACCOUNT_O table is a database table, and the CREDIT_LIMIT field is a field of the TM_ACCOUNT_O table.

[0187] Fig. 20 This is the implementation code corresponding to the saveOrUpdate function. Fig.21 This is the implementation code corresponding to the saveOrUpdateAccount function. Fig. 22 The bytecode information corresponding to saveOrUpdateAccount, Fig.23 Based on Fig. 22 Constructed data lineage model.

[0188] like Fig.23As shown, when parsing the INVOKEINTERFACE bytecode instruction on line 195, an INVOKEINTERFAC type node is created (such as Fig.23 Node ID: -1623815529), the INVOKEINTERFACE instruction is the call interface instruction.

[0189] Identify the INVOKEINTERFACE bytecode instruction calling object type as cn / webank / cnc / cps / infrastructure / dao / TmAccountDao. Query the database configuration file to know that this class is a class of the data access layer. Query the dependent nodes of the INVOKEINTERFACE node (such as Fig.23 The node ID is -743788686), and the fully qualified name of its class matches the fully qualified name prefix of the class in the domain model. Then, it is determined whether there is a globally unique database table node in the data lineage model (such as Fig.23 Middle node ID: 1144730246), if it does not exist, create it.

[0190] Optionally, the database table node is created by: the name of the database table node is the same as the name of the assignment node (ie Fig.23 In the example of tmAccount( ) of node ID: -743788686, the instruction name is set to the first name, such as PO_DEFINE. Of course, the instruction name can also be set to other names, which is not limited in this application. The source code line number is set to -1, and of course, the source code line number can also be set to other values, which is not limited in this application. The node identifier of the database table node is set according to the name of the database table node, the instruction name, and the source code line number.

[0191] Optionally, connect dependent nodes (such as Fig.23 Node ID: -743788686) and DB table node (such as Fig.23 Middle node ID: 1144730246).

[0192] Optionally, according to the object instance identifier of the second node (node ​​ID: -743788686), a node (such as Fig.23 Middle node ID: -929446741), marked as the fourth node, based on the fourth node to determine whether there is a database field node (such as Fig.23 Node ID: -723132615), if it does not exist, create it.

[0193] Optionally, the database field node is created by: the name of the database field node is the same as the name of the assignment node ( Fig.23 The instruction name is set to the second name, such as PO_FIELD. Of course, the instruction name can also be set to other names, which is not limited by this application. The source code line number is set to -1, and of course the source code line number can also be set to other values, which is not limited by this application. The node identifier of the database field node is set according to the name of the database field node, the instruction name, and the source code line number.

[0194] Optionally, connect the assignment node (node ​​ID: -929446741, node ID: -929443823) and the database field node (node ​​ID: -723132615).

[0195] Optionally, connect the Database Field node (Node ID: -723132615) with the Database Table node (Node ID: 1144730246).

[0196] The TmAccountO.creditLimit PO_FILED node (node ​​ID: -723132615) represents the CREDIT_LIMIT field of the TM_ACCOUNT_O table. Through its incoming edge, it can be traced back to line 383 of AccountServiceImpl.java. Line 284 of AccountServiceImpl.java assigns a value to the CREDIT_LIMIT field of the TM_ACCOUNT_O table. The data flow and object reference flow can be traced according to the edge type.

[0197] At the same time, the tmAccountO PO_DEFINE node represents the TM_ACCOUNT_O table. The TmAccountO.creditLimitPO_FILED node (node ​​ID: -723132615) and its relationship express the subordinate relationship between the table and the table field, providing strong support for the R&D team to analyze the source of the table field data.

[0198] Based on the same technical concept, Fig.24 The following is an exemplary diagram of a data lineage model determination device 2400 provided in an embodiment of the present application. Fig.24As shown, it includes: an acquisition unit 2401 and a determination unit 2402. The acquisition unit 2401 is used to acquire the bytecode information contained in the bytecode file of the business system to be analyzed; the bytecode information contains multiple method functions of the business system to be analyzed; the determination unit 2402 is used to determine the data lineage sub-model corresponding to each method function according to the dependency relationship between the bytecode instructions, operand stack and local variable table of each method function in the bytecode information; according to the dependency relationship between each method function in the bytecode information and the data lineage sub-model corresponding to each method function, determine the data lineage relationship model of the business system to be analyzed.

[0199] In a possible implementation method, the determination unit 2402 is used to determine, for any bytecode instruction in the method function, a first node corresponding to the bytecode instruction; determine a second node that has a dependency relationship with the first node based on a dependency relationship among the bytecode instruction, the operand stack, and the local variable table; construct a connection relationship between the first node and the second node; and determine a data lineage sub-model corresponding to the method function based on the inter-node connection relationship formed by each bytecode instruction of the method function.

[0200] In a possible implementation method, the determination unit 2402 is used to determine the instruction type and the number of operands required during the execution process according to the bytecode instruction; obtain the corresponding number of execution operands from the operand stack according to the number of operands required during the execution process; determine the index information corresponding to the execution operands in the local variable table according to the execution operands; determine the first N bytecode instructions that have a dependency relationship with the bytecode instruction according to the index information and the instruction type; wherein N is a positive integer; and determine the second node that has a dependency relationship with the first node according to the first N bytecode instructions.

[0201] In a possible implementation method, the determination unit 2402 is used to determine the identifier of the object instance initialized by the object initialization instruction if the bytecode instruction is an object initialization instruction; and determine the information of the first node corresponding to the object initialization instruction based on the identifier of the object instance.

[0202] In a possible implementation method, the determination unit 2402 is used to obtain the identifier of the object instance corresponding to the object initialization instruction if the bytecode instruction is not the object initialization instruction but depends on the object initialization instruction; and determine the first node corresponding to the bytecode instruction based on the identifier of the object instance corresponding to the object initialization instruction.

[0203] In one possible implementation method, the determination unit 2402 is used to determine whether the caller of the third-party instruction is located in a third-party configuration file if the bytecode instruction is a calling third-party instruction; the third-party configuration file contains multiple third-party identifiers; if the caller of the third-party instruction is located in the third-party configuration file, use simulation code to replace the instruction logic corresponding to the calling third-party instruction, and determine the first node corresponding to the calling third-party instruction based on the simulation code.

[0204] In a possible implementation method, the determining unit 2402 is configured to set the third-party configuration file; wherein each third-party identifier in the third-party configuration file corresponds to a plurality of third-party data packets having the same function.

[0205] In a possible implementation method, the determining unit 2402 is used to determine the connection relationship between the first node and the second node based on the information of the first node and the index number and instruction type of the bytecode instruction corresponding to the second node.

[0206] In a possible implementation method, the determination unit 2402 is used to determine a set of dependent nodes that have a dependent relationship with the first node based on information of the first node; the dependent node set includes at least one second node that has a dependent relationship with the first node; if there is only one second node in the dependent node set whose bytecode instruction has an instruction type of local variable transfer to stack instruction ALOAD, then the connection relationship between the first node and the second node is set to a reference relationship, and the connection relationship between the first node and other nodes in the dependent node set is a numerical relationship; if there are multiple second nodes in the dependent node set whose bytecode instruction has an instruction type of ALOAD, then the connection relationship between the first node and the third node is set to a reference relationship, and the connection relationship between the first node and other nodes in the dependent node set is a numerical relationship; wherein the third node is a second node whose index number is the first numerical value.

[0207] In a possible implementation method, the determination unit 2402 is used to determine the database node based on the information of the first node and the information of the second node if the bytecode instruction corresponding to the first node is a call interface instruction; and to establish a connection relationship between the database node, the first node and the second node.

[0208] In a possible implementation method, the determining unit 2402 is used to determine whether the caller of the calling interface instruction is located in a database configuration file; the database configuration file includes information about the database corresponding to the business system to be analyzed; if

[0209] If the caller of the calling interface instruction is located in the database configuration file, determine whether the bytecode instruction corresponding to the second node is ALOAD; if the bytecode instruction corresponding to the second node is ALOAD, and the variable type of the variable obtained according to the ALOAD is located in the configuration file of the domain model, then determine the fourth node with the same identifier as the object instance corresponding to the second node; the configuration file of the domain model includes the table information of the database; determine the database node based on the second node and the fourth node.

[0210] In a possible implementation method, the determination unit 2402 is used to determine whether there is a database field node and a database table node in the data lineage sub-model; the database field node is used to indicate operations on database fields; the database table node is used to indicate table information of the database; wherein the database node includes the database field node and the database table node; if the database field node does not exist, the information of the database field node is determined according to the fourth node; if the database field node does not exist, the information of the database table node is determined according to the second node.

[0211] Based on the same technical concept, the embodiment of the present application provides a device 2500 for determining a data lineage model. The device 2500 for determining a data lineage model may be, for example, a computing device. Fig.25 As shown, a device 2500 for determining a data lineage model includes at least one processor 2501 and a memory 2502 connected to the at least one processor. The specific connection medium between the processor 2501 and the memory 2502 is not limited in the embodiment of the present application. Fig.25 For example, the processor 2501 and the memory 2502 are connected via a bus. The bus can be divided into an address bus, a data bus, a control bus, etc.

[0212] In an embodiment of the present application, the memory 2502 stores instructions that can be executed by at least one processor 2501. The at least one processor 2501 can execute the above-mentioned method for determining a data lineage model by executing the instructions stored in the memory 2502.

[0213] Among them, the processor 2501 is a control center of a device 2500 for determining a data lineage model, and can use various interfaces and lines to connect various parts of the computer device, and perform resource settings by running or executing instructions stored in the memory 2502 and calling data stored in the memory 2502. Optionally, the processor 2501 may include one or more determination units, and the processor 2501 may integrate an application processor and a modem processor, wherein the application processor mainly processes an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 2501. In some embodiments, the processor 2501 and the memory 2502 may be implemented on the same chip, and in some embodiments, they may also be implemented separately on independent chips.

[0214] Processor 2501 can be a general-purpose processor, such as a central processing unit (CPU), a digital signal processor, an application-specific integrated circuit (ASIC), a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, and can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present application. A general-purpose processor can be a microprocessor or any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware processor for execution, or can be executed by a combination of hardware and software modules in the processor.

[0215] Memory 2502, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer executable programs and modules. Memory 2502 may include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card-type memory, random access memory (Random Access Memory, RAM), static random access memory (Static Random Access Memory, SRAM), programmable read-only memory (Programmable Read Only Memory, PROM), read-only memory (Read Only Memory, ROM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, EEPROM), magnetic memory, disk, optical disk, etc. Memory 2502 is any other medium that can be used to carry or store a desired program code in the form of an instruction or data structure and can be accessed by a computer, but is not limited thereto. The memory 2502 in the embodiment of the present application can also be a circuit or any other device that can realize a storage function, for storing program instructions and / or data.

[0216] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer-executable program. The computer-executable program is used to enable a computer to execute a method for determining a data lineage model listed in any of the above methods.

[0217] An embodiment of the present application provides a computer program product, including a computer program executable by a computer device. When the program is run on the computer device, the computer device executes a method for determining a data lineage model listed in any of the above-mentioned methods.

[0218] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0219] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0220] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0221] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0222] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for determining a data lineage model, characterized in that: include: Acquire bytecode information contained in a bytecode file in the business system to be analyzed; wherein the bytecode information includes multiple method functions of the business system to be analyzed; Determine a data lineage sub-model corresponding to each method function according to the dependency relationship among the bytecode instructions, operand stack and local variable table of each method function in the bytecode information; According to the dependency relationship between each method function in the bytecode information and the data lineage sub-model corresponding to each method function, the data lineage model of the business system to be analyzed is determined.

2. The method according to claim 1, characterized in that The step of determining the data lineage sub-model corresponding to each method function according to the dependency relationship among the bytecode instructions, operand stack and local variable table of each method function in the bytecode information comprises: For any bytecode instruction in the method function, determining a first node corresponding to the bytecode instruction; Determine a second node having a dependency relationship with the first node according to the dependency relationship among the bytecode instruction, the operand stack and the local variable table; Establishing a connection relationship between the first node and the second node; Based on the inter-node connection relationship formed by each bytecode instruction of the method function, the data lineage sub-model corresponding to the method function is determined.

3. The method according to claim 2, characterized in that The determining, according to the dependency relationship among the bytecode instruction, the operand stack, and the local variable table, a second node having a dependency relationship with the first node comprises: Determining the instruction type and the number of operands required during execution according to the bytecode instruction; According to the number of operands required in the execution process, obtaining a corresponding number of execution operands from the operand stack; Determine, according to the execution operand, index information corresponding to the execution operand in the local variable table; Determine, according to the index information and the instruction type, the first N bytecode instructions that have a dependency relationship with the bytecode instruction; wherein N is a positive integer; According to the first N bytecode instructions, the second node having a dependency relationship with the first node is determined.

4. The method according to claim 2, characterized in that The determining the first node corresponding to the bytecode instruction includes: If the bytecode instruction is an object initialization instruction, determining an identifier of an object instance initialized by the object initialization instruction; According to the identifier of the object instance, information of the first node corresponding to the object initialization instruction is determined.

5. The method according to claim 4, characterized in that The method further comprises: If the bytecode instruction is not the object initialization instruction but depends on the object initialization instruction, obtaining an identifier of the object instance corresponding to the object initialization instruction; According to the identifier of the object instance corresponding to the object initialization instruction, a first node corresponding to the bytecode instruction is determined.

6. The method according to claim 2, characterized in that The determining the first node corresponding to the bytecode instruction includes: If the bytecode instruction is a third-party calling instruction, determining whether the caller of the third-party calling instruction is located in a third-party configuration file; the third-party configuration file includes a plurality of third-party identifiers; If the caller of the calling third-party instruction is located in the third-party configuration file, the instruction logic corresponding to the calling third-party instruction is replaced by the simulation code, and the first node corresponding to the calling third-party instruction is determined according to the simulation code.

7. The method according to claim 6, characterized in that Before determining the first node corresponding to the bytecode instruction, the method further includes: The third-party configuration file is set; wherein each third-party identifier in the third-party configuration file corresponds to a plurality of third-party data packets having the same function.

8. The method according to claim 2, characterized in that The establishing a connection relationship between the first node and the second node includes: A connection relationship between the first node and the second node is determined based on the information of the first node and the index number and instruction type of the bytecode instruction corresponding to the second node.

9. The method according to claim 8, characterized in that The determining, according to the first node and the index number and the instruction type of the bytecode instruction corresponding to the second node, a connection relationship between the first node and the second node includes: Determine, according to the information of the first node, a set of dependent nodes having a dependent relationship with the first node; the set of dependent nodes includes at least one second node having a dependent relationship with the first node; If there is only one second node in the dependent node set whose bytecode instruction has an instruction type of ALOAD, the connection relationship between the first node and the second node is set to a reference relationship, and the connection relationship between the first node and other nodes in the dependent node set is set to a numerical relationship; If there are multiple second nodes in the dependent node set whose bytecode instructions have an instruction type of ALOAD, the connection relationship between the first node and the third node is set to a reference relationship, and the connection relationship between the first node and other nodes in the dependent node set is a numerical relationship; wherein the third node is the second node whose index number is the first numerical value.

10. The method according to claim 2, characterized in that The establishing a connection relationship between the first node and the second node includes: If the bytecode instruction corresponding to the first node is a call interface instruction, determining a database node according to information of the first node and information of the second node; A connection relationship between the database node, the first node, and the second node is established.

11. The method according to claim 10, characterized in that The determining the database node according to the information of the first node and the information of the second node includes: Determine whether the caller of the calling interface instruction is located in the database configuration file; the database configuration file includes information of the database corresponding to the business system to be analyzed; If the caller of the calling interface instruction is located in the database configuration file, determining whether the bytecode instruction corresponding to the second node is ALOAD; If the bytecode instruction corresponding to the second node is ALOAD, and the variable type of the variable obtained according to the ALOAD is located in the configuration file of the domain model, then determining a fourth node having the same identifier as the object instance corresponding to the second node; the configuration file of the domain model includes table information of the database; The database node is determined according to the second node and the fourth node.

12. The method according to claim 11, characterized in that The step of determining the database node according to the second node and the fourth node includes: Determine whether the data lineage sub-model has a database field node and a database table node; the database field node is used to indicate an operation on a database field; the database table node is used to indicate table information of a database; wherein the database node includes the database field node and the database table node; If the database field node does not exist, determining information of the database field node according to the fourth node; If the database field node does not exist, information of the database table node is determined according to the second node.

13. A device for determining a data lineage model, characterized in that: Including acquisition unit and determination unit: The acquisition unit is used to acquire bytecode information contained in a bytecode file in the business system to be analyzed; wherein the bytecode information includes multiple method functions of the business system to be analyzed; The determination unit is used to determine the data lineage sub-model corresponding to each method function based on the dependency relationship between the bytecode instructions, operand stack and local variable table of each method function in the bytecode information; and determine the data lineage relationship model of the business system to be analyzed based on the dependency relationship between the method functions in the bytecode information and the data lineage sub-model corresponding to each method function.

14. A computing device, characterized in that include: A memory for storing program instructions; A processor is used to call the program instructions stored in the memory, and execute the method according to any one of claims 1 to 12 according to the obtained program instructions.

15. A computer-readable storage medium, characterized in that: The method comprises computer-readable instructions, and when a computer reads and executes the computer-readable instructions, the method according to any one of claims 1 to 12 is implemented.

16. A computer program product, characterized in that The invention comprises a computer program executable by a computer device, and when the program is run on the computer device, the computer device is caused to execute the steps of the method according to any one of claims 1 to 12.

Images