Data blood relationship construction method suitable for complex storage process

Abstract

The invention discloses a data blood relationship construction method suitable for a complex storage process, and belongs to the technical field of data processing. The method specifically comprises the steps that a to-be-analyzed database storage process source code is analyzed, and an original SQL statement set, a process internal and cross-process calling relation structure, context hierarchical structure information of statements, a database table identifier, a field identifier, a grammar structure position identifier of a field in the statement to which the field belongs and a grammar structure unit identifier are obtained; dividing the field into an input field or an output field; splitting the structured statement into statement fragments, identifying the field-level dependency relationship of the fragments, and labeling the type of the field-level dependency relationship; and constructing a field granularity data consanguinity map and a table granularity data consanguinity map. The method has the beneficial effects that the accuracy and coverage rate of blood relationship identification can be remarkably improved, the dependence on rule configuration and manual analysis is reduced, and the feasibility of data blood relationship analysis in a large-scale data system is enhanced.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a method for constructing data lineages suitable for complex storage processes. Background Technology

[0002] In data-intensive systems, data lineage describes the dependency paths and transformation logic of data from source to destination, and is of great value for tasks such as data governance, quality tracking, and change management. Especially in enterprise database environments, a large amount of data processing logic is encapsulated in SQL stored procedures, which have complex structures, flexible syntax, and hidden dependency chains. This makes it difficult for traditional lineage identification methods to accurately reconstruct the upstream and downstream relationships between fields and tables. Existing methods are mostly based on fixed rules or syntax tree parsing, suitable for stored procedures with simple structures. However, when faced with semantically complex stored procedures containing multi-table JOINs, nested subqueries, dynamic field mappings, or aggregation calculations, they often fail to achieve comprehensive and accurate lineage extraction, resulting in dependency omissions and structural fragmentation. Furthermore, these solutions lack deep semantic analysis capabilities at the field level, failing to reconstruct the true dependencies between fields, greatly limiting their practicality in highly complex environments. Therefore, there is an urgent need for a lineage construction method that can effectively adapt to complex stored procedure structures and possesses higher automation and accuracy to support the reliability and controllability requirements of modern data systems. Summary of the Invention

[0003] To address the shortcomings of existing technologies in constructing data lineage relationships, incomplete structure identification, and inaccurate extraction of field dependencies when dealing with complex database stored procedures, this invention proposes a data lineage construction method suitable for complex stored procedures. By performing syntactic structure analysis and context-level modeling on the SQL statements in the stored procedure, combined with field syntax position identifiers and call relationship structures, the method identifies the role of fields in the data flow, constructs a set of field-level dependencies, and further automatically generates data lineage diagrams at the field and table granularities based on the proposed dependencies.

[0004] To achieve the above-mentioned objective, the present invention provides a data lineage construction method suitable for complex stored procedures, the method comprising the following steps: Step S1: Parse the source code of the stored procedure in the database to be analyzed and obtain the following information: the original set of SQL statements, the call relationship structure within and across procedures, the context hierarchy structure information of each SQL statement, the database table identifier and field identifier, the syntactic structure position identifier of the field in its SQL statement, and the syntactic structure unit identifier of the SQL syntax. Step S2: Based on the field identifiers and their syntax structure position identifiers extracted in Step S1, determine the role of the field in the data stream and classify it into input fields or output fields; Step S3: Identify the dynamic SQL statements in the original SQL statement set. The dynamic SQL statements include SQL statements constructed by string concatenation and called by execution statements such as EXECUTE IMMEDIATE and sp_executesql. Through parameter parsing and structure restoration, they are transformed into analyzable standard SQL statements. Step S4: Based on the syntactic structure unit identifier, the structured SQL statements in the original SQL statement set are split into statement fragments; for each fragment, combining the call relationship structure, context hierarchy structure information, and the syntactic structure position identifier of the field, the field-level dependency relationship between its internal input field and output field is identified, and the dependency relationship type is labeled to form a field-level dependency relationship set. The dependency relationship types include value passing relationship, computation relationship, and reference relationship; wherein, the structured SQL statement includes the standard SQL statement obtained by restoring static SQL statement and dynamic SQL statement; Step S5: Based on the set of field-level dependencies output in Step S4, construct a field-level data lineage graph with fields as nodes and a table-level data lineage graph with tables as nodes.

[0005] In a preferred embodiment of the present invention, in step S1, the original SQL statement set, context hierarchy structure information, syntax structure location identifiers, and syntax structure unit identifiers are extracted using a syntax structure analysis method; the call relationship structure is obtained through inter-procedure call graph analysis. Step S2 specifically includes the following steps: Step S201: Determine the syntax unit type based on the field's syntax structure position identifier and determine its initial role preference: If a field is located in the target column of a SELECT clause, the assignment target of a SET clause, or the assignment target of an INSERT / UPDATE statement, it is marked as an output preference field; If a field is located in a WHERE clause, JOIN clause, GROUP BY clause, ORDER BY clause, or function / stored procedure input parameter, it is marked as an input preference field. Step S202: For the output tendency field, analyze the expression structure to which it belongs: If the field is a directly assigned object (such as TargetField = SourceField), then it is determined to be an output field; If a field is involved in the calculation expression and is the result carrier, then it is determined to be an output field; Step S203: For the input preference field, verify its validity by combining the nesting level and control flow branch identifier in the context hierarchy information of the fragment to which it belongs. If the field is not covered by higher-level logic or the branch is invalid, it is determined to be an input field. Step S204: In nested structures or related logic, fields that serve as both upstream outputs and downstream inputs are marked with bidirectional flow attributes.

[0006] In step S3, the specific steps for forming an analyzable standard SQL statement are as follows: Step S301: Identify dynamic SQL statements in the original SQL statement set; identify code segments in the original SQL statements that contain dynamic SQL execution statements, including SQL statements generated and executed using EXECUTE IMMEDIATE, sp_executesql, or string concatenation; Step S302: Extract parameter placeholders from the dynamic SQL statement, and bind them to actual input fields or variables based on context hierarchy information and call relationship structure to build a parameter binding mapping table; Step S303: Restore the dynamic SQL statement after binding parameters to a standard SQL statement, as an analyzable structured SQL statement.

[0007] As a preferred embodiment of the present invention, step S4 specifically comprises: Step S401: Based on the syntax structure unit identifier and context hierarchy structure information, the structured SQL statement is split into statement fragments, where each subquery, common table expression (CTE) unit, and join logic block (JOIN-ON group) is divided into independent fragments, and a unique fragment identifier is generated for each independent fragment, and its nesting level and the identifier of the control flow branch to which it belongs are recorded. Step S402: For each segment, identify all output fields within it based on the grammatical structure position identifier, establish intra-segment dependencies between output fields and input fields, and label the intra-segment dependency types; Step S403: If the input field of the current segment comes from the output field of other segments, establish a cross-segment dependency chain according to the call relationship structure and segment identifier; if cross-procedure calls are involved, parse the binding relationship between the output field of the called procedure and the input field of the current segment according to the call relationship structure; for fields with the bidirectional flow attribute, associate their upstream output segment with their downstream input segment. Step S404: Merge all intra-segment dependencies and cross-segment dependency chains to generate a set of field-level dependencies.

[0008] In step S401, the division of the associated logic blocks is performed according to the following rules: Divide multiple tables directly connected by the ON condition and their associated conditions into an independent segment; When nested JOIN structures exist, recursively split the innermost JOIN group into child segments and establish a parent-child segment hierarchy. Label each JOIN group segment with the main table identifier and the connection direction (LEFT / RIGHT / INNER).

[0009] The establishment of the cross-segment dependency chain in step S403 specifically includes the following steps: Step a: Extract the fragment identifier and its output field identifier of the upstream output fragment, and the fragment identifier and its input field identifier of the downstream input fragment; Step b: Based on the semantic context of the dependency path between the input and output fields, select the corresponding field mapping rule, including: In subquery scenarios, you can bind aliases for fields defined in the subquery to the locations in the outer query that reference those fields. In the context of common table expressions, establish a mapping or positional matching relationship between the output fields of the common table expression and the referenced fields. In the case of temporary tables, the mapping relationship between fields is generated by associating the table identifier and field identifier of the temporary table; Step c: Generate a structured cross-segment dependency chain, which includes: the identifier of the upstream output segment and the output field identifier, the identifier of the downstream input segment and the input field identifier, and the dependency relationship type.

[0010] As a preferred embodiment of the present invention, step S5 includes the following steps: Step S501: Using field identifiers as graph nodes, construct directed edges based on each dependency in the field-level dependency set, label the dependency types, and generate a field-granular data lineage graph; Step S502: Based on the database table identifier to which the field belongs, aggregate field nodes within the same table into a single table node; upgrade field-level dependencies to table-level dependencies and construct a table-level data lineage graph; Step S503: In the field granularity data lineage diagram, record the statement fragment identifier, context level, and call path to which the field node belongs, in order to support dependency path tracing and visualization.

[0011] The beneficial effects of this invention are as follows: The method of this invention is particularly suitable for stored procedures with complex structures, including multi-table joins, nested subqueries, dynamic SQL statements, common table expressions (CTEs), and field aggregation operations. It can significantly improve the accuracy and coverage of lineage identification, reduce reliance on rule configuration and manual analysis, and enhance the feasibility of data lineage analysis in large-scale data systems. Specifically, this invention introduces an SQL statement fragment segmentation mechanism based on syntactic structural units and context levels, combined with field syntax position identifiers and call relationship structures, to accurately identify the input and output roles of fields and their dependency types. This allows for the reconstruction of the true data flow relationship at the field granularity. Furthermore, this invention supports structural restoration and semantic binding of dynamic SQL statements, enabling parts that were originally impossible to analyze statically to be included in the scope of lineage analysis. It can also automatically construct field-level and table-level data lineage graphs, improving the completeness, accuracy, and automation of lineage identification. Attached Figure Description

[0012] Figure 1 The above are flowcharts of Embodiments 1 and 2 of the present invention. Detailed Implementation

[0013] To clearly illustrate the technical features of this solution, the following detailed implementation method will be used to explain the solution.

[0014] Example 1 This invention provides a method for constructing data lineage for complex stored procedures, the method comprising the following steps: Step S1: Parse the source code of the stored procedure in the database to be analyzed to obtain the following information: the original set of SQL statements, the call relationship structure within and across procedures, the context hierarchy structure information of each SQL statement, the database table identifier and field identifier, the syntactic structure position identifier of the field in its respective SQL statement, and the syntactic structure unit identifier of the SQL syntax; among them, the original set of SQL statements, the context hierarchy structure information, the syntactic structure position identifier, and the syntactic structure unit identifier are extracted using syntactic structure analysis methods; the call relationship structure is obtained through inter-procedure call graph analysis.

[0015] Step S2: Based on the field identifiers and their syntax structure position identifiers extracted in Step S1, determine the role of the field in the data stream and classify it as an input field or an output field; specifically, this includes the following steps: Step S201: Determine the syntax unit type based on the field's syntax structure position identifier and determine its initial role preference: If a field is located in the target column of a SELECT clause, the assignment target of a SET clause, or the assignment target of an INSERT / UPDATE statement, it is marked as an output preference field; If a field is located in a WHERE clause, JOIN clause, GROUP BY clause, ORDER BY clause, or function / stored procedure input parameter, it is marked as an input preference field. Step S202: For the output tendency field, analyze the expression structure to which it belongs: If a field is a directly assigned object, then it is determined to be an output field; If a field is involved in the calculation expression and is the result carrier, then it is determined to be an output field; Step S203: For the input preference field, verify its validity by combining the nesting level and control flow branch identifier in the context hierarchy information of the fragment to which it belongs. If the field is not covered by higher-level logic or the branch is invalid, it is determined to be an input field. Step S204: In nested structures or related logic, fields that serve as both upstream outputs and downstream inputs are marked with bidirectional flow attributes.

[0016] Step S3: Identify the dynamic SQL statements in the original SQL statement set, and form analyzable standard SQL statements through parameter parsing and structure restoration; specifically: Step S301: Identify dynamic SQL statements in the original SQL statement set; Step S302: Extract parameter placeholders from the dynamic SQL statement, and bind them to the actual input fields based on the context hierarchy structure information and the call relationship structure to build a parameter binding mapping table; Step S303: Restore the dynamic SQL statement after binding parameters to a standard SQL statement, as an analyzable structured SQL statement.

[0017] Step S4: Based on the syntactic structure unit identifier, the structured SQL statements in the original SQL statement set are split into statement fragments; for each fragment, combining the call relationship structure, context hierarchy structure information, and field syntactic structure position identifiers, the field-level dependencies between its internal input and output fields are identified, and the dependency type is labeled to form a set of field-level dependencies; the structured SQL statements include standard SQL statements obtained by restoring static SQL statements and dynamic SQL statements; specifically, the following steps are included: Step S401: Based on the syntax structure unit identifier and context hierarchy structure information, the structured SQL statement is split into statement fragments, where each subquery, common table expression unit, and relational logic block is divided into an independent fragment, and a unique fragment identifier is generated for each independent fragment, recording its nesting level and the identifier of the control flow branch to which it belongs; the division of the relational logic block is performed according to the following rules: Divide multiple tables directly connected by the ON condition and their associated conditions into an independent segment; When nested JOIN structures exist, recursively split the innermost JOIN group into child segments and establish a parent-child segment hierarchy. Label each JOIN group segment with its main table identifier and connection direction.

[0018] Step S402: For each segment, identify all output fields within it based on the grammatical structure position identifier, establish intra-segment dependencies between output fields and input fields, and label the dependency type; Step S403: If the input field of the current segment originates from the output field of another segment, establish a cross-segment dependency chain based on the call relationship structure and the segment identifier; if cross-procedure calls are involved, resolve the binding relationship between the output field of the called procedure and the input field of the current segment based on the call relationship structure; for fields with bidirectional flow attributes, associate their upstream output segment with their downstream input segment; establishing a cross-segment dependency chain specifically includes the following steps: Step a: Extract the fragment identifier and its output field identifier of the upstream output fragment, and the fragment identifier and its input field identifier of the downstream input fragment; Step b: Based on the semantic context of the dependency path between the input and output fields, select the corresponding field mapping rule, including: In subquery scenarios, you can bind aliases for fields defined in the subquery to the locations in the outer query that reference those fields. In the context of common table expressions, establish a mapping or positional matching relationship between the output fields of the common table expression and the referenced fields. In the case of temporary tables, the mapping relationship between fields is generated by associating the table identifier and field identifier of the temporary table; Step c: Generate a structured cross-segment dependency chain. The cross-segment dependency chain includes: the identifier of the upstream output segment and the output field identifier, the identifier of the downstream input segment and the input field identifier, and the dependency relationship type.

[0019] Step S404: Merge all intra-segment dependencies and cross-segment dependency chains to generate a set of field-level dependencies.

[0020] Step S5: Based on the set of field-level dependencies output in Step S4, construct a field-level data lineage graph with fields as nodes and a table-level data lineage graph with tables as nodes; specifically including the following steps: Step S501: Using field identifiers as graph nodes, construct directed edges based on each dependency in the field-level dependency set, label the dependency types, and generate a field-granular data lineage graph; Step S502: Based on the database table identifier to which the field belongs, aggregate field nodes within the same table into a single table node; upgrade field-level dependencies to table-level dependencies and construct a table-level data lineage graph; Step S503: In the field granularity data lineage diagram, record the statement fragment identifier, context level, and call path to which the field node belongs.

[0021] Example 2 This embodiment provides a data lineage construction method suitable for complex stored procedures, possessing high adaptability and semantic reconstruction capabilities. By introducing strategies such as hierarchical structural modeling, syntactic role annotation, dynamic statement standardization, and cross-fragment semantic mapping, it achieves accurate extraction of field-level data dependencies and lineage graph construction in complex stored procedure environments. Specifically, it includes the following steps: Step 1: Parse the source code of the stored procedure in the database to be analyzed. Use syntax structure analysis and inter-procedure call graph analysis to obtain the following information: the original set of SQL statements, the call relationship structure within and across procedures, the context hierarchy structure information of each SQL statement, database table identifiers and field identifiers, the syntactic structure position of the field in its respective SQL statement, and the syntactic structure unit identifier of the SQL syntax. Step two: By recognizing the positional identifiers of field syntax structures, combined with nesting levels and control flow structures, automatic classification and validation of fields in input and output roles are achieved. This effectively overcomes the misjudgment problems of traditional keyword-based judgment methods in nested logic and multi-path statements, laying a semantic foundation for the accurate construction of subsequent dependencies. Specifically, firstly, the syntax unit type is deduced based on the positional identifiers of the field syntax structure to determine its initial role tendency: 1. If a field is located in the target column of a SELECT clause, the assignment target of a SET clause, or the assignment target of an INSERT / UPDATE statement, it is marked as an output target field; 2. If a field is located in a WHERE clause, JOIN clause, GROUP BY clause, ORDER BY clause, or function / stored procedure input parameter, it is marked as an input preference field. Then, for the output tendency field, analyze the expression structure to which it belongs: 1. If the field is a directly assigned object (e.g., TargetField = SourceField), then it is determined to be an output field; 2. If a field participates in the calculation expression and is the result carrier, then it is determined to be an output field; Furthermore, for input preference fields, their validity is verified by combining the nesting level and control flow branch identifier in the context hierarchy information of the fragment to which they belong. If the field is not covered by higher-level logic or the branch is invalid, it is determined to be an input field. Finally, in nested structures or related logic, fields that serve as both upstream outputs and downstream inputs are marked with bidirectional flow attributes.

[0022] Step 3: Dynamic SQL statements cannot be directly analyzed statically. This embodiment extracts parameter placeholders and establishes binding mapping relationships, enabling dynamic SQL statements to be restored to standard SQL templates for unified analysis. First, it identifies dynamic SQL statement representations constructed through string concatenation and invoked by execution statements such as EXECUTE IMMEDIATE and sp_executesql. Then, through parameter parsing and structure restoration, it transforms these representations into analyzable standard SQL representations. Specifically: First, identify dynamic SQL statements within the original SQL statement set. Then, identify code segments containing dynamic SQL execution statements within the original SQL statements. Dynamic SQL execution statements include SQL statements generated and executed using EXECUTE IMMEDIATE, sp_executesql, or string concatenation. Next, extract parameter placeholders from the dynamic SQL statements and bind them to actual input fields or variables based on contextual hierarchy and call relationship structure, constructing a parameter binding mapping table. Finally, restore the parameter-bound dynamic SQL statements to standard SQL statements, as analyzable structured SQL statements. Subsequently, the standard SQL statements are used as structured SQL for statement fragment segmentation and dependency analysis in subsequent steps. Based on the parameter binding mapping table, the obtained field dependencies are mapped back to the corresponding fields in the original dynamic SQL fragments, achieving the integration of field-level dependencies.

[0023] Step 4: Based on the syntactic unit identifier, the structured SQL statements (including static SQL statements and standard SQL statements derived from dynamic SQL statements) in the original SQL statement set are split into a set of statement fragments. For each fragment, combining the call relationship structure, context hierarchy structure information, and the syntactic position identifier of the field, the field-level dependency relationship between its internal input and output fields is identified, and the dependency relationship type (including value passing relationship, computation relationship, and reference relationship) is labeled, forming a set of field-level dependency relationships. The statement structure fragment partitioning method proposed in this embodiment is based on the syntactic unit identifier and context hierarchy structure, supporting the independent identification and numbering of subqueries, common table expressions, and multi-level JOIN structures. This method makes the field dependency construction process structurally controllable and supports upstream and downstream semantic association queries of field dependencies through fragment nesting information.

[0024] 1. Based on the syntactic structure unit identifier and context hierarchy information, the structured SQL statement is split into statement fragments. Each subquery, common table expression (CTE) unit, and join logic block (JOIN-ON group) is divided into independent fragments. A unique fragment identifier is generated for each independent fragment, and its nesting level and the identifier of the control flow branch to which it belongs are recorded. The division of join logic block (JOIN-ON group) is performed according to the following rules: multiple tables directly connected by the ON condition and their join conditions are divided into an independent fragment. When there is a nested JOIN structure, the innermost JOIN group is recursively split into sub-fragments, and a parent-child fragment hierarchy relationship is established. Finally, each JOIN group fragment is labeled with the main table identifier and the join direction (LEFT / RIGHT / INNER).

[0025] 2. For each segment, identify all output fields within it based on the syntactic structure position identifier, establish intra-segment dependencies between output fields and input fields, and label the intra-segment dependency types; 3. If the input field of the current segment originates from the output field of another segment, establish a cross-segment dependency chain based on the call relationship structure and segment identifier. If cross-procedure calls are involved, resolve the binding relationship between the output field of the called procedure and the input field of the current segment based on the call relationship structure. For fields with bidirectional flow attributes, associate their upstream output segment with their downstream input segment. Specifically, establishing a cross-segment dependency chain involves first extracting the segment identifier and its output field identifier of the upstream output segment, and the segment identifier and its input field identifier of the downstream input segment. Then, based on the semantic context of the dependency path between the input and output fields, select the corresponding field mapping rule. In a subquery scenario, bind the field alias defined in the subquery to the position of the referenced field in the outer query. In a common table expression scenario, establish a name mapping or position matching relationship between the output field of the common table expression and the referenced field. In a temporary table scenario, generate a field mapping relationship by associating the table identifier and field identifier of the temporary table, forming a structured cross-segment dependency chain (including the identifier and output field identifier of the upstream output segment, the identifier and input field identifier of the downstream input segment, and the dependency relationship type). Finally, all intra-segment dependencies and cross-segment dependency chains are merged to generate a set of field-level dependencies. This step employs various field mapping strategies for different semantic scenarios, including subquery alias binding, common table expression field position mapping, and temporary table field parsing, thereby improving the accuracy and coverage of cross-segment dependency identification between fields.

[0026] Step 5: Based on the set of field-level dependencies, construct a field-level data lineage graph with fields as nodes and a table-level data lineage graph with tables as nodes. Using field identifiers as graph nodes, construct directed edges according to each dependency in the field-level dependency set, and label the dependency type to generate a field-level data lineage graph. Based on the database table identifier to which the field belongs, aggregate field nodes within the same table into a single table node. Upgrade the field-level dependencies to table-level dependencies to construct a table-level data lineage graph. In the field-level data lineage graph, record the statement fragment identifier, context level, and call path to which the field node belongs to support dependency path tracing and visualization. This invention, by constructing two levels of lineage graphs at the field and table levels, supports both refined field tracing and the generation of table-level data asset views. It also allows embedding statement fragment identifiers and context metadata in the field graph, facilitating graph structure visualization, dependency path querying, and conflict diagnosis.

[0027] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for constructing data lineage suitable for complex stored procedures, characterized in that, The construction method includes the following steps: Step S1: Parse the source code of the stored procedure in the database to be analyzed and obtain the following information: the original set of SQL statements, the call relationship structure within and across procedures, the context hierarchy structure information of each SQL statement, the database table identifier and field identifier, the syntactic structure position identifier of the field in its SQL statement, and the syntactic structure unit identifier. Step S2: Based on the field identifiers and their syntax structure position identifiers extracted in Step S1, determine the role of the field in the data stream and classify it into input fields or output fields; Step S3: Identify the dynamic SQL statements in the original SQL statement set, and form analyzable standard SQL statements through parameter parsing and structure restoration; Step S4: Based on the syntactic structure unit identifier, the structured SQL statements in the original SQL statement set are split into statement fragments; for each fragment, the field-level dependency relationship between its input field and output field is identified by combining the call relationship structure, context hierarchy structure information and the syntactic structure position identifier of the field, and the dependency relationship type is marked to form a field-level dependency relationship set; wherein, the structured SQL statement includes the standard SQL statement obtained by restoring static SQL statement and dynamic SQL statement; Step S5: Based on the set of field-level dependencies output in Step S4, construct a field-level data lineage graph with fields as nodes and a table-level data lineage graph with tables as nodes.

2. The construction method according to claim 1, characterized in that, In step S1, the original SQL statement set, context hierarchy structure information, syntax structure position identifier and syntax structure unit identifier are extracted using a syntax structure analysis method; the call relationship structure is obtained through inter-procedure call graph analysis.

3. The construction method according to claim 1, characterized in that, Step S2 specifically includes the following steps: Step S201: Determine the syntax unit type based on the field's syntax structure position identifier and determine its initial role preference: If a field is located in the target column of a SELECT clause, the assignment target of a SET clause, or the assignment target of an INSERT / UPDATE statement, it is marked as an output preference field; If a field is located in a WHERE clause, JOIN clause, GROUP BY clause, ORDER BY clause, or function / stored procedure input parameter, it is marked as an input preference field. Step S202: For the output tendency field, analyze the expression structure to which it belongs: If a field is a directly assigned object, then it is determined to be an output field; If a field is involved in the calculation expression and is the result carrier, then it is determined to be an output field; Step S203: For the input preference field, verify its validity by combining the nesting level and control flow branch identifier in the context hierarchy information of the fragment to which it belongs. If the field is not covered by higher-level logic or the branch is invalid, it is determined to be an input field. Step S204: In nested structures or related logic, fields that serve as both upstream outputs and downstream inputs are marked with bidirectional flow attributes.

4. The method according to claim 1, characterized in that, In step S3, the specific steps for forming an analyzable standard SQL statement are as follows: Step S301: Identify dynamic SQL statements in the original SQL statement set; Step S302: Extract parameter placeholders from the dynamic SQL statement, and bind them to the actual input fields based on the context hierarchy structure information and the call relationship structure to build a parameter binding mapping table; Step S303: Restore the dynamic SQL statement after binding parameters to a standard SQL statement, as an analyzable structured SQL statement.

5. The construction method according to claim 1, characterized in that, Step S4 specifically involves: Step S401: Based on the syntax structure unit identifier and context hierarchy structure information, the structured SQL statement is split into statement fragments, where each subquery, common table expression unit, and relational logic block is divided into independent fragments, and a unique fragment identifier is generated for each independent fragment, and its nesting level and the identifier of the control flow branch to which it belongs are recorded. Step S402: For each segment, identify all output fields within it based on the grammatical structure position identifier, establish intra-segment dependencies between output fields and input fields, and label the dependency type; Step S403: If the input field of the current segment comes from the output field of other segments, establish a cross-segment dependency chain according to the call relationship structure and segment identifier; if cross-procedure calls are involved, parse the binding relationship between the output field of the called procedure and the input field of the current segment according to the call relationship structure; for fields with the bidirectional flow attribute, associate their upstream output segment with their downstream input segment. Step S404: Merge all intra-segment dependencies and cross-segment dependency chains to generate a set of field-level dependencies.

6. The construction method according to claim 5, characterized in that, The establishment of the cross-segment dependency chain in step S403 specifically includes the following steps: Step a: Extract the fragment identifier and its output field identifier of the upstream output fragment, and the fragment identifier and its input field identifier of the downstream input fragment; Step b: Based on the semantic context of the dependency path between the input and output fields, select the corresponding field mapping rule, including: In subquery scenarios, you can bind aliases for fields defined in the subquery to the locations in the outer query that reference those fields. In the context of common table expressions, establish a mapping or positional matching relationship between the output fields of the common table expression and the referenced fields. In the case of temporary tables, the mapping relationship between fields is generated by associating the table identifier and field identifier of the temporary table; Step c: Generate a structured cross-segment dependency chain, which includes: the identifier of the upstream output segment and the output field identifier, the identifier of the downstream input segment and the input field identifier, and the dependency relationship type.

7. The construction method according to claim 1, characterized in that, Step S5 includes the following steps: Step S501: Using field identifiers as graph nodes, construct directed edges based on each dependency in the field-level dependency set, label the dependency types, and generate a field-granular data lineage graph; Step S502: Based on the database table identifier to which the field belongs, aggregate field nodes within the same table into a single table node; upgrade field-level dependencies to table-level dependencies and construct a table-level data lineage graph; Step S503: In the field granularity data lineage diagram, record the statement fragment identifier, context level, and call path to which the field node belongs.

Images