Data provenance method, apparatus, medium, and device

Abstract

Embodiments of the present specification disclose a data tracing method and device, a storage medium and equipment. The method comprises: parsing a to-be-traced structured query language (SQL), to obtain an abstract syntax tree corresponding to the to-be-traced SQL; based on the abstract syntax tree, obtaining predetermined element information corresponding to the to-be-traced SQL; based on the predetermined element information and the syntax structure of the to-be-traced SQL, constructing a reverse SQL corresponding to the to-be-traced SQL; and performing data tracing processing based on the reverse SQL.

Description

Technical Field

[0001] This specification relates to the field of big data technology, and in particular to a data traceability method, apparatus, storage medium and device. Background Technology

[0002] With the development of internet technology, a large amount of data is generated every day, and how to trace the source of this data has become a focus of attention.

[0003] Data tracing can reconstruct the flow path and evolution process of data. One technical solution involves using a reverse lookup method to trace the source of target data, obtaining the tracing results. For example, by reverse-engineering the data transformation process, the original data can be traced back to the target data. However, current reverse lookup methods for tracing target data have relatively low accuracy in obtaining the tracing results.

[0004] Therefore, improving the accuracy of data traceability results has become a pressing technical challenge. Summary of the Invention

[0005] This specification provides a data traceability method, apparatus, storage medium, and device that can improve the accuracy of traceability results and quickly and accurately locate the source of data.

[0006] Firstly, embodiments of this specification provide a data tracing method, including:

[0007] The structured query language (SQL) to be traced is parsed to obtain the abstract syntax tree corresponding to the SQL to be traced.

[0008] Based on the abstract syntax tree, obtain the predetermined element information corresponding to the SQL to be traced;

[0009] Based on the predetermined element information and the syntax structure of the SQL to be traced, construct the reverse SQL corresponding to the SQL to be traced;

[0010] Data tracing is performed based on the reverse SQL.

[0011] Secondly, embodiments of this specification provide a data traceability device, including:

[0012] The parsing module is used to parse the structured query language (SQL) to be traced and obtain the abstract syntax tree corresponding to the SQL to be traced.

[0013] The element acquisition module is used to acquire the predetermined element information corresponding to the SQL to be traced based on the abstract syntax tree;

[0014] The reverse function construction module is used to construct the reverse SQL corresponding to the SQL to be traced based on the predetermined element information and the syntax structure of the SQL to be traced.

[0015] The source tracing module is used to perform data source tracing based on the reverse SQL.

[0016] Thirdly, embodiments of this specification provide a computer storage medium storing a plurality of instructions adapted for loading by a processor and executing the steps of the method described above.

[0017] Fourthly, embodiments of this specification provide a computer program product containing instructions that, when run on a computer or processor, cause the computer or processor to perform the steps of the method described above.

[0018] Fifthly, embodiments of this specification provide an electronic device, including: a processor and a memory; wherein the memory stores a computer program adapted to be loaded by the processor and to execute the steps of the method described above.

[0019] According to the technical solution of the embodiments of this specification, on the one hand, by parsing the SQL to be traced to obtain the corresponding abstract syntax tree, and based on the abstract syntax tree, the predetermined element information corresponding to the SQL to be traced can be obtained, and the key elements in the SQL to be traced can be accurately determined; on the other hand, based on the predetermined element information and the abstract syntax tree, the reverse SQL corresponding to the SQL to be traced is constructed, and based on the reverse SQL, the execution result corresponding to the SQL to be traced is subjected to data tracing processing. Since the reverse SQL can quickly and accurately query the original data of the execution result of the SQL to be traced, the accuracy of the tracing result of the data tracing processing can be improved, and the source of the data in the execution result can be quickly and accurately located; furthermore, since the data tracing processing of the data in the execution result is only performed when needed, the storage space occupied by the data tracing processing can be reduced. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments or prior art of this specification, the drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 A schematic diagram illustrating an application scenario provided by an embodiment of this specification;

[0022] Figure 2A flowchart illustrating a data tracing method provided in an embodiment of this specification;

[0023] Figure 3 This is a schematic diagram of the process for constructing reverse SQL according to the embodiments provided in this specification;

[0024] Figure 4 This is a flowchart illustrating the process of stitching together various traceability elements in a multi-table traceability element model according to the embodiments provided in this specification.

[0025] Figure 5 This is a flowchart illustrating the process of stitching together elements in a multi-layered nested source element model, as provided in the embodiments of this specification.

[0026] Figure 6 A flowchart illustrating another data tracing method provided in the embodiments of this specification;

[0027] Figure 7 A schematic diagram of an abstract syntax tree provided for embodiments of this specification;

[0028] Figure 8 This is a schematic diagram of the data traceability device provided in the embodiments of this specification;

[0029] Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this specification. Detailed Implementation

[0030] To make the features and advantages of this specification more apparent and understandable, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this specification without creative effort are within the scope of protection of this specification.

[0031] First, the terms and concepts used in one or more embodiments of this specification will be explained.

[0032] AST (Abstract Syntax Tree): A tree-like representation of a syntax structure.

[0033] Antlr: A language tool that can construct language recognizers, such as compilers and parsers, from grammatical descriptions of languages ​​like Java, C++, or C#. These language recognizers are used to read, process, execute, or translate structured text or binary files.

[0034] Data tracing: a part of data governance, which mainly involves tracking the origin of data and reproducing the historical state of the data.

[0035] SQL (Structured Query Language) sourcing: A specific scenario for data sourcing, using the syntax and execution results of SQL statements to obtain the original data in the corresponding data table.

[0036] Reverse lookup method: This method involves constructing a reverse function to invert the query or performing reverse derivation based on the data transformation process to trace back to the original data from the result.

[0037] Reverse SQL: SQL used to obtain the original data of a forward SQL statement; one of the main methods for SQL tracing.

[0038] Data lineage: The main way to present data tracing results is through data graphs, which show the source and destination of the data.

[0039] Because annotation methods require additional storage space, they are not suitable for fine-grained data, especially for tracing the origin of large datasets. Therefore, one technical solution uses reverse lookup to trace the origin of target data. For example, this involves constructing an inverse function to reverse the query or performing reverse derivation based on the data transformation process to trace the result back to the original data. However, as data lineage becomes increasingly complex, the reverse lookup method in the above-mentioned technical solution struggles to quickly and accurately locate the source of data in the execution result.

[0040] Based on the above, this specification provides a data tracing method. On the one hand, by parsing the SQL to be traced, a corresponding abstract syntax tree is obtained. Based on the abstract syntax tree, predetermined element information corresponding to the SQL to be traced is obtained, which can accurately determine the key elements in the SQL to be traced. On the other hand, based on the predetermined element information and the syntax structure of the SQL to be traced, a reverse SQL corresponding to the SQL to be traced is constructed. Based on the reverse SQL, data tracing processing is performed on the execution result corresponding to the SQL to be traced. Since the reverse SQL can quickly and accurately query the original data of the execution result of the SQL to be traced, the accuracy of the tracing result of the data tracing processing can be improved, thereby quickly and accurately locating the source of the data in the execution result. Furthermore, since data tracing processing is performed on the data in the execution result only when needed, the storage space occupied by data tracing processing can be reduced.

[0041] The technical solutions of the embodiments of this specification will now be described in detail with reference to the accompanying drawings.

[0042] Figure 1 This is a schematic diagram illustrating an application scenario provided by an embodiment of this specification.

[0043] like Figure 1 As shown, this application scenario includes an SQL configuration and execution module 110, a database 120, an SQL tracing module 130, and a tracing result display module 140. The SQL configuration and execution module 110 is used to configure the SQL to be traced and to run the configured SQL, retrieving the execution result of the SQL from the database 120. The database 120 stores multiple tables containing the original data corresponding to the execution result of the SQL to be traced. The SQL tracing module 130 performs data tracing processing on the data in the execution result of the SQL to be traced based on its syntax structure and execution result. The data display module 140 displays the execution result of the SQL to be traced and the results of the data tracing processing.

[0044] It should be noted that the steps in the data tracing method in the example embodiments of this specification may be partially executed by the client, partially executed by the server, or entirely executed by the server or entirely by the client. This specification does not impose any special limitations on this.

[0045] based on Figure 1 The application scenarios shown will be discussed below. Figures 2-7 This specification provides a detailed description of the data traceability method provided in the embodiments. It should be noted that the above application scenarios are shown only to facilitate understanding of the spirit and principles of this specification, and the embodiments are not limited in any way. On the contrary, the embodiments of this specification can be applied to any applicable scenario.

[0046] Figure 2 This is a flowchart illustrating a data tracing method provided in an embodiment of this specification. This data tracing method can be executed by a device with computing capabilities, such as a terminal device or a server. Figure 2 As shown, the data tracing method in the embodiments of this specification may include the following steps S210 to S240.

[0047] In step S210, the source SQL to be traced is parsed to obtain the abstract syntax tree corresponding to the source SQL.

[0048] In the example embodiment, an abstract syntax tree (API) is a tree-like representation used to describe the syntactic structure of SQL. Each node in the API represents a syntactic structure or element in SQL; for example, a node in the API can be a table name, a table operation name, or a field. Lexical analysis is performed on the source SQL to be traced based on the lexical rules of the SQL language to obtain the symbol stream corresponding to the source SQL. Syntactic analysis is then performed on the symbol stream based on the syntactic rules of the SQL language to generate the API corresponding to the source SQL.

[0049] For example, a lexical analyzer analyzes each character in the source SQL to be traced, converting each character into a symbol that conforms to the lexical rules of the SQL language, forming a symbol stream corresponding to the source SQL to be traced; based on the symbol stream corresponding to the source SQL to be traced, a syntax analyzer is used to construct an abstract syntax tree of the source SQL to be traced, using each symbol in the symbol stream as a syntax node.

[0050] In step S220, based on the abstract syntax tree, the predetermined element information corresponding to the SQL to be traced is obtained.

[0051] In the example embodiment, the predetermined element information refers to the key elements of the SQL to be traced. These include elements such as table name, query conditions, fields, and table relationships. It should be noted that although the example given uses table name, fields, and table relationships as the predetermined element information, those skilled in the art should understand that the predetermined element information may also include other appropriate element information, such as association conditions or the element's hierarchical level, which is also within the scope of the embodiments in this specification.

[0052] Furthermore, in the example embodiment, predetermined element information corresponding to the source SQL is obtained from the abstract syntax tree based on the syntax rules of the source SQL. For example, the data operation type of the source SQL is determined, which can be a create, update, read, or delete operation. Based on the syntax rules corresponding to the data operation type, the predetermined element information corresponding to the source SQL is obtained from the abstract syntax tree.

[0053] For example, pre-set syntax rule files corresponding to various data operation types. The syntax rule files include elements such as data operation name, table name, and field elements. Based on the data operation type corresponding to the SQL to be traced, obtain the corresponding syntax rule file. Based on the pre-defined element names in the syntax rule file, such as table name, traverse the abstract syntax tree and obtain the pre-defined element information corresponding to the SQL to be traced from the abstract syntax tree, such as the table name or table query conditions in the SQL to be traced.

[0054] In step S230, based on the predetermined element information and the syntax structure of the SQL, the reverse SQL corresponding to the SQL to be traced is constructed.

[0055] In the example embodiment, based on the syntax structure of SQL, the syntax structure type corresponding to the SQL to be traced is determined. The syntax structure type is one of the following: single table type, multi-table type, and multi-level nested type. Based on the syntax structure type and predetermined element information, the reverse SQL corresponding to the SQL to be traced is constructed.

[0056] For example, if the syntax structure type is a single-table type, then the various elements in the predetermined element information are concatenated based on the SQL syntax structure of the single-table type to generate the reverse SQL corresponding to the SQL to be traced. For example, suppose the SQL to be traced is a single-table type, such as select id from student where name='jack', and the predetermined elements include the data operation type select, the table name student, the query condition where, and the field name. Based on the SQL syntax structure of the single-table type, the various elements in the predetermined element information are concatenated to generate the reverse SQL corresponding to the SQL to be traced, i.e., select * from student where name='jack', where "*" is the fill element in the reverse SQL, indicating that it can be replaced with the corresponding field in the data table.

[0057] In step S240, data tracing is performed based on reverse SQL.

[0058] In the example embodiment, reverse SQL refers to SQL used to retrieve the raw data from the forward SQL statement. Reverse SQL may include filling elements such as the symbol "*", retrieving the corresponding fields of the data table from the data table's metadata; filling the retrieved fields into the fill elements of the reverse SQL; and executing the reverse SQL for data tracing processing.

[0059] For example, suppose the data table is a student table, and the fields of the data table include unique identifier ID, name, class, etc. Suppose the SQL to be traced is a single table type, such as select id from student where name = 'jack'. The query result of the SQL to be traced is the unique identifier ID of the name "jack". The reverse SQL is select * from student where name = 'jack'. In response to the data tracing request of the SQL to be traced, the reverse SQL is executed, and the execution result of the reverse SQL is obtained, that is, the result of the data tracing processing. The execution result of the reverse SQL is a row of records in the student table with the name "jack".

[0060] according to Figure 2The technical solution in the example embodiment, on the one hand, obtains the corresponding abstract syntax tree by parsing the SQL to be traced, and obtains the predetermined element information corresponding to the SQL to be traced based on the abstract syntax tree, which can accurately determine the key elements in the SQL to be traced; on the other hand, based on the predetermined element information and the syntax structure of the SQL to be traced, a reverse SQL corresponding to the SQL to be traced is constructed, and data tracing processing is performed on the execution result corresponding to the SQL to be traced based on the reverse SQL. Since the reverse SQL can quickly and accurately query the original data of the execution result of the SQL to be traced, the accuracy of the tracing result of the data tracing processing can be improved, and the source of the data in the execution result can be quickly and accurately located; furthermore, since data tracing processing is performed on the data in the execution result only when needed, the storage space occupied by data tracing processing can be reduced.

[0061] Furthermore, in the example embodiment, the data tracing method further includes: displaying the processing results of the data tracing process through a data lineage graph. Displaying the tracing results based on the data lineage graph can visually and intuitively show the source and data flow path of the target data, thereby supporting row-level data tracing.

[0062] Figure 3 This is a schematic diagram of the process for constructing reverse SQL according to the embodiments provided in this specification.

[0063] Reference Figure 3 As shown, in step S310, the syntax structure type corresponding to the SQL to be traced is determined based on the syntax structure of the SQL to be traced.

[0064] In some example implementations, the syntax structure type of the SQL to be traced is one of the following: single-table, multi-table, or multi-level nested. Table 1 below shows the three syntax structure types of SQL:

[0065] Table 1. Three types of SQL syntax structures

[0066]

[0067] In the example embodiment, based on the syntax structure of the SQL to be traced, the target elements of the SQL to be traced are determined, the target elements of the SQL to be traced are obtained from the abstract syntax tree of the SQL to be traced, and the syntax structure type corresponding to the SQL to be traced is determined based on the number of target elements obtained.

[0068] For example, suppose the SQL to be traced is a query SQL. Based on the syntax structure of the query SQL, such as the query syntax structure shown below, the target elements of the SQL to be traced are determined to be the table name and the select keyword. The table name and select keyword in the SQL to be traced are obtained. If there is only one table name and only one select keyword, the syntax structure type corresponding to the SQL to be traced is determined to be a single-table type; if there are two or more table names and only one select keyword, the syntax structure type corresponding to the SQL to be traced is determined to be a multi-table type; if there are two or more select keywords, the syntax structure corresponding to the SQL to be traced is determined to be a multi-level nested type. The following shows a query syntax structure of SQL provided in an embodiment of this specification:

[0069] select[all|distinct] <expression> [, <expression>]...

[0070] from [,

[0071] <condition>[and|or][ <condition>]...

[0072] group by <expression> [, <expression>...

[0073] having...

[0074] order by...

[0075] In step S320, a traceability element model corresponding to the SQL to be traced is constructed based on the syntax structure type and predetermined element information.

[0076] In the example embodiment, the traceability element model includes multiple elements corresponding to the SQL to be traced. Elements are selected from predetermined element information based on the syntax structure type, and the traceability element model corresponding to the SQL to be traced is constructed based on the selected elements. For example, if the syntax structure type of the SQL to be traced is a single-table type, then the single-table traceability elements in the predetermined element information are determined based on the single-table type, and the single-table traceability element model corresponding to the SQL to be traced is constructed. The following illustrates a traceability element model provided in this embodiment, which includes attributes of multiple traceability elements:

[0077]

[0078] The traceability element model includes traceability elements such as table name, alias, select keyword list, where keyword, and join keyword.

[0079] For example, a source element model corresponding to the above-mentioned syntax structure type is pre-set. The source element model includes the attributes of multiple elements of the SQL corresponding to the syntax structure type. Let the syntax structure type be a single table type and the data operation type be a read operation type, i.e., a select operation. The attribute values ​​of the elements corresponding to the source element model of the single table type are obtained from the pre-defined element information. For example, the value of the table name, the value of the query condition, etc. The single table source element model corresponding to the SQL to be traced is constructed based on the obtained attribute values ​​of the elements. That is, the obtained element values ​​are assigned to the source element model to generate the single table source element model corresponding to the SQL to be traced.

[0080] Furthermore, in the example embodiment, if the syntax structure type of the SQL to be traced is a multi-table type, then the multi-table traceability elements in the predetermined element information are determined based on the multi-table type. The multi-table traceability elements include multi-table related elements. Based on the multi-table traceability elements, a multi-table traceability element model corresponding to the SQL to be traced is constructed. If the syntax structure type of the SQL to be traced is a multi-level nested type, the nested traceability elements in the predetermined element information are determined based on the multi-level nested type. The nested traceability elements include the element's level information. Based on the nested traceability elements, a multi-level nested traceability element model corresponding to the SQL to be traced is constructed.

[0081] In step S330, based on the SQL syntax structure corresponding to the syntax structure type, each traceability element in the traceability element model is concatenated to generate the reverse SQL corresponding to the SQL to be traced.

[0082] In the example embodiment, the SQL syntax structure corresponding to the syntax structure type of the SQL to be traced is determined. Based on the SQL syntax structure corresponding to the syntax structure type of the SQL to be traced, the various traceability elements in the traceability element model are concatenated to generate the reverse SQL corresponding to the SQL to be traced.

[0083] For example, various SQL syntax structures are pre-defined for different syntax types. If the syntax structure of the SQL to be traced is a single-table type, then the traceability elements in the single-table traceability element model are concatenated based on the SQL syntax structure of the single-table type to generate the reverse SQL corresponding to the SQL to be traced. If the syntax structure of the SQL to be traced is a multi-table type, and the predefined element information includes multi-table related element information, then the traceability elements in the multi-table traceability element model are concatenated based on the SQL syntax structure of the multi-table type and the multi-table related element information; generating the reverse SQL for each data table corresponding to the SQL to be traced. If the syntax structure of the SQL to be traced is a multi-level nested type, and the predefined element information includes the element's level information, then the traceability elements in the multi-level nested traceability element model are concatenated layer by layer based on the SQL syntax structure of the multi-level nested type and the element's level information; generating the reverse SQL for each level corresponding to the SQL to be traced.

[0084] according to Figure 3 The technical solution in the example embodiment constructs a traceability element model corresponding to the SQL to be traced based on the syntax structure type and predetermined element information, which can efficiently and accurately determine the key elements of the SQL to be traced; based on the SQL syntax structure corresponding to the syntax structure type, the traceability element model is spliced ​​together to generate the reverse SQL corresponding to the SQL to be traced, which can efficiently generate the reverse SQL corresponding to the SQL to be traced, thereby enabling the source of data in the execution result to be quickly and accurately located based on the reverse SQL; furthermore, since SQL can operate on data at the row level of the data table, it can support row-level data traceability.

[0085] Figure 4 This is a schematic diagram illustrating the process of splicing together various traceability elements in a multi-table traceability element model according to the embodiments provided in this specification.

[0086] Reference Figure 4 As shown, in step S410, based on the abstract syntax tree, the table association conditions corresponding to the multi-table association element information are determined.

[0087] In the example implementation, the table join conditions corresponding to the multi-table join element information are determined from the parsing structure of the SQL to be traced, i.e., the abstract syntax tree. For example, if the multi-table join elements are join elements and where or on elements, and the SQL to be traced is "select student.name from student left join score on student.id=score.student_id", then based on the multi-table join element information, the corresponding table join conditions are determined from the SQL to be traced, such as "join score on student.id=score.student_id".

[0088] In step S420, based on the table join conditions, the subquery conditions of the reverse SQL for each of the multiple data tables are determined.

[0089] In the example embodiment, the table join conditions involve multiple data tables. Based on the table join conditions in the source SQL to be traced, the subquery conditions of the reverse SQL for each data table are determined.

[0090] For example, suppose the source SQL to be traced is "select student.name from student left joins score on student.id = score.student_id", then the table join condition is "join score on student.id = score.student_id", involving the data tables student and score. For the student table, the subquery condition for generating the reverse SQL is "select student_id from score"; for the score table, the subquery condition is "select id from student".

[0091] In step S430, based on the subquery conditions of the reverse SQL for each data table and the SQL syntax structure of the multi-table type, the traceability elements corresponding to each data table in the multi-table traceability element model are spliced ​​together.

[0092] In the example embodiment, the traceability elements corresponding to each data table in the multi-table traceability element model are determined. Based on the subquery conditions of the reverse SQL for each data table and the SQL syntax structure of the multi-table type, the traceability elements corresponding to each data table in the multi-table traceability element model are concatenated to generate the reverse SQL for each data table corresponding to the SQL to be traced.

[0093] For example, suppose the source SQL to be traced is "select student.name from student left joins score on student.id = score.student_id". For the student table, the subquery condition of the reverse SQL is "select student_id from score". Then, for the student table, the reverse SQL generated after concatenation is "select student.*from student where student.id in(select student_id from score)". For the score table, the subquery condition of the reverse SQL is "select id from student". Then, for the score table, the reverse SQL generated after concatenation is "select score.*from score wherescore.student_id in(select id from student)".

[0094] according to Figure 4 The technical solution in the example embodiment, on the one hand, determines the table association conditions corresponding to the multi-table association element information based on the abstract syntax tree, and determines the subquery conditions of the reverse SQL for each data table based on the table association conditions, which can accurately determine the subquery conditions of the reverse SQL for each data table; on the other hand, based on the subquery conditions of the reverse SQL for each data table and the SQL syntax structure of the multi-table type, the traceability elements corresponding to each data table in the multi-table traceability element model are concatenated, which can generate the reverse SQL for each data table corresponding to the SQL to be traced, thereby enabling the source of data in the execution result of the multi-table type SQL to be quickly and accurately located based on the reverse SQL.

[0095] Figure 5 This is a flowchart illustrating the process of stitching together elements in a multi-layered nested traceability element model, as provided in the embodiments of this specification.

[0096] Reference Figure 5 As shown, in step S510, the level of each traceability element in the multi-layer nested traceability element model is determined based on the element's level information.

[0097] In the example embodiment, the element level information of each element is obtained from the parsed structure of the SQL to be traced, and the level of each traceable element in the multi-level nested traceable element model is determined based on the element level information. For example, the element level information corresponding to the traceable element is obtained based on the identification information of the traceable element, and the level of the traceable element is determined based on the element level information.

[0098] In step S520, the syntax structure type of SQL at each level is determined based on the syntax structure of SQL at each level, in order from the inner layer to the outer layer.

[0099] In the example embodiment, the innermost level is set to 1, and the levels increase by 1 sequentially from the innermost to the outermost level. Following the order from memory to the outermost level, the SQL at each level in the source SQL to be traced is determined. Based on the SQL syntax structure of each level, the syntax structure type of the SQL at each level is determined. The syntax structure type can be one of the following: single-table type, multi-table type, or multi-level nested type.

[0100] For example, suppose a SQL statement at a certain level is a query SQL statement. Based on the syntax structure of this query SQL statement, the target elements of the query SQL statement are determined to be the table name and the select keyword. If there is only one table name and one select keyword in the query SQL statement, then the syntax structure type of the query SQL statement is determined to be a single-table type; if there are two or more table names and one select keyword, then the syntax structure type of the query SQL statement is determined to be a multi-table type; if there are two or more select keywords, then the syntax structure type of the query SQL statement is determined to be a multi-level nested type.

[0101] In step S530, based on the syntax structure type of the SQL at each level, the traceability elements in the multi-level nested traceability element model corresponding to each level are concatenated to generate the reverse SQL at each level corresponding to the SQL to be traced.

[0102] In the example embodiment, based on the syntax structure type of SQL at each level, the syntax structure corresponding to SQL at each level is determined. Based on the syntax structure of SQL at each level, the traceability elements in the multi-level nested traceability element model corresponding to each level are concatenated to generate the reverse SQL at each level corresponding to the SQL to be traced.

[0103] For example, if the syntax structure of the inner SQL is a single-table type, then the source elements in the corresponding multi-level nested source element model are concatenated based on the single-table type SQL syntax structure to generate the reverse SQL corresponding to the inner SQL. If the syntax structure of the inner SQL is a multi-table type, then the source elements in the corresponding multi-level nested source element model are concatenated based on the multi-table type SQL syntax structure and multi-table association element information to generate the reverse SQL corresponding to the inner SQL.

[0104] For example, suppose the SQL to be traced is "select student.age,temp.value from student,(selectvalue,student_id from score where score.grade=60)where student.id=temp2.student_id", where the part in parentheses is the inner SQL, and the reverse SQL corresponding to the inner SQL is "select*fromscore where score.grade=60(inner)"; the reverse SQL corresponding to the outer SQL is "select*from studentwhere student.id in(select student_id from score where score.grade=60)(outer)".

[0105] according to Figure 5 The technical solution in the example embodiment, on the one hand, determines the level of each traceability element in the multi-layer nested traceability element model based on the element's level information, which can accurately determine the level of each traceability element in the element model; on the other hand, according to the order from the inner layer to the outer layer, the syntax structure type of each level is determined based on the syntax structure of the SQL at each level, which can accurately determine the syntax structure type of the SQL at each level; furthermore, based on the syntax structure type of the SQL at each level, the traceability elements in the multi-layer nested traceability element model corresponding to each level are spliced ​​together, which can generate the reverse SQL corresponding to each level of the SQL to be traced, thereby enabling the source of data in the execution result of the multi-layer nested SQL to be quickly and accurately located based on the reverse SQL.

[0106] Furthermore, in the example embodiment, the inner SQL subquery is used as a temporary table to concatenate the source elements in the corresponding multi-level nested source element model of the outer layer; the temporary table in the concatenated reverse SQL is replaced with the SQL subquery. For example, starting from the innermost layer of the source SQL to be traced in the multi-level nested type, the reverse SQL of each level of SQL is constructed sequentially. If the inner SQL subquery is a single-table type SQL, the reverse SQL generation processing logic of the single-table type SQL is called; otherwise, the reverse SQL generation processing logic of the multi-table type SQL is called. When constructing the outer layer, the inner SQL subquery is used as a temporary table until the entire multi-level nested query is split into a single-table query; then, the temporary table is replaced with the original level of SQL in the multi-level nested source element model.

[0107] For example, suppose the SQL to be traced is "select student.age,temp.value from student,(select value,student_id from score where score.grade=60)temp wherestudent.id=temp2.student_id", where the part in parentheses is the inner SQL, which is used as a temporary table temp. The reverse SQL corresponding to this inner SQL is "select * from score where score.grade=60(inner)"; the reverse SQL corresponding to the outer SQL is "select * from student where student.id in(select student_id from score where score.grade=60)(outer)", where the part in parentheses is the inner SQL corresponding to the temporary table.

[0108] According to the technical solution in the above example embodiment, by using the subquery SQL in memory as a temporary table to construct the outer reverse SQL, the processing data for constructing the outer reverse SQL can be reduced, thereby improving the processing efficiency of generating reverse SQL.

[0109] Figure 6 This is a flowchart illustrating another data tracing method provided in the embodiments of this specification.

[0110] Reference Figure 6 As shown, in step S605, input the SQL to be traced.

[0111] In the example embodiment, SQL can be entered through a graphical user interface, for example, by entering the SQL to be traced through the graphical user interface of an SQL database. For example, the SQL to be traced is "CREATE TABLE t1(id int,name varchar)".

[0112] It should be noted that the SQL to be traced can be MySQL, or other appropriate SQL such as HiveSQL, OdpsSQL, FlinkSQL, SQL Server, SparkSQL, etc.

[0113] In step S610, the source SQL to be traced is parsed.

[0114] In the example implementation, the source SQL to be traced is parsed using the Antlr parser. The Antlr parser can define lexical rules for recognizing character streams and parsing rules for interpreting symbol streams. Through the .g4 file, the Antlr parser automatically generates the corresponding lexer and parser. The lexer and parser can then be used to transform the source SQL into an Abstract Syntax Tree (AST).

[0115] For example, suppose the input SQL to be traced is "CREATE TABLE t1(id int,name varchar)", which is then parsed by the Antlr parser to generate... Figure 7 The abstract syntax tree shown.

[0116] It should be noted that although the Antlr parser has been used as an example, those skilled in the art should understand that other suitable parsers, such as FastSql AST or Calcite AST, can also be used to parse the source SQL, which is also within the scope of the embodiments in this specification.

[0117] In step S615, the table name and corresponding metadata in the parsing result are obtained.

[0118] In the example implementation, based on the SQL syntax rules corresponding to the abstract syntax tree, key elements of the source SQL structure to be traced can be obtained from the abstract syntax tree, such as table names, field definitions, and join relationships. For example, based on the SQL syntax rules in Antlr's g4 file, key elements of the source SQL structure to be traced can be obtained from the abstract syntax tree. The g4 file is Antlr's syntax rule file, containing lexical rules and syntax rules.

[0119] Furthermore, in the example embodiment, metadata corresponding to the data table is obtained, such as the various fields of the data table. By obtaining the metadata corresponding to the data table, it can be used to supplement the fill element "*" in the reverse SQL, for example, to adapt to the "select*" scenario in the reverse SQL, and fill the fill element "*" in that scenario with the corresponding fields of the data table.

[0120] In step S620, the source SQL to be traced is classified according to its syntax structure.

[0121] In the example embodiment, referring to Table 1 above, the syntax structure type is one of the following: single-table type, multi-table type, and multi-level nested type. Taking the SQL to be traced as the SQL corresponding to a MySQL database as an example, the SQL to be traced is classified according to the syntax structure of MySQL.

[0122] It should be noted that the SQL to be traced can be MySQL, or other appropriate SQL such as HiveSQL, OdpsSQL, FlinkSQL, SQL Server, SparkSQL, etc.

[0123] In step S625, it is determined whether the SQL to be traced is a multi-level nested model.

[0124] In the example embodiment, based on the syntax structure of the SQL to be traced, the target elements of the SQL to be traced are determined, the target elements of the SQL to be traced are obtained from the abstract syntax tree of the SQL to be traced, and the syntax structure type corresponding to the SQL to be traced is determined based on the number of target elements obtained.

[0125] For example, suppose the SQL to be traced is a query SQL. Based on the syntax structure of the query SQL, such as the query syntax structure shown below, the target elements of the SQL to be traced are determined to be the table name and the select keyword. The table name and select keyword of the data table in the SQL to be traced are obtained. If there are more than two select keywords, the syntax structure corresponding to the SQL to be traced is determined to be a multi-level nested type.

[0126] In step S630, the reverse SQL of the source SQL to be traced is constructed for a multi-level nested type.

[0127] In the example embodiment, the reverse SQL of the source SQL of the multi-level nested type is constructed by the following steps: (1) Obtain the table name, fields, level (the innermost subquery is defined as level 1, and so on until the outermost level), and original SQL of the level from the parsing result of the source SQL. Obtain all fields corresponding to each table from the metadata. (2) Determine the nested source elements in the predetermined element information based on the multi-level nested type. The nested source elements include the level information of the element. Based on the nested source elements, construct the multi-level nested source element model corresponding to the source SQL of the multi-level nested type, namely the TraceFactor model. (3) Starting from the innermost level, construct the reverse SQL of each level in sequence. If the inner SQL subquery is a single-table type SQL, call the reverse SQL generation processing logic of the single-table type SQL. Otherwise, call the reverse SQL generation processing logic of the multi-table type. When constructing the outer level, treat the inner SQL subquery as a temporary table until the entire multi-level nested query is split into a single-table query. Then, replace the temporary table with the original level SQL part of the multi-level nested source element model.

[0128] For example, suppose the SQL to be traced is "select student.age,temp.value from student,(select value,student_id from score where score.grade=60)temp wherestudent.id=temp2.student_id", where the part in parentheses is the inner SQL, which is used as a temporary table temp. The reverse SQL corresponding to this inner SQL is "select * from score where score.grade=60(inner)"; the reverse SQL corresponding to the outer SQL is "select * from student where student.id in(select student_id from score where score.grade=60)(outer)", where the part in parentheses is the inner SQL corresponding to the temporary table.

[0129] In step S635, it is determined whether the SQL to be traced is a multi-table type.

[0130] In the example implementation, if the data table has two or more table names and there is only one select keyword, then the syntax structure type corresponding to the SQL to be traced is determined to be a multi-table type. If the data table has only one table name and there is only one select keyword, then the syntax structure type corresponding to the SQL to be traced is determined to be a single-table type.

[0131] In step S640, the reverse SQL of the source SQL to be traced is constructed for multiple table types.

[0132] In the example embodiment, the reverse SQL of the SQL to be traced for multiple table types is constructed through the following steps: (1) Obtain the table name, the table name after the join, the join conditions, etc. from the parsing result of the SQL to be traced, and obtain the fields corresponding to each data table from the metadata of each data table. (2) Determine the multi-table traceability elements in the predetermined element information based on the multi-table type. The multi-table traceability elements include multi-table association elements. Based on the multi-table traceability elements, construct the multi-table element model of the SQL to be traced for multiple table types. (3) Determine the subquery conditions of the reverse SQL for each data table based on the table association conditions in the SQL to be traced. Based on the subquery conditions of the reverse SQL for each data table and the SQL syntax structure of the multi-table type, concatenate the traceability elements corresponding to each data table in the multi-table traceability element model to generate the reverse SQL of each data table corresponding to the SQL to be traced.

[0133] For example, suppose the source SQL to be traced is "select student.name from student left join scoreon student.id=score.student_id", and for the data table student, the subquery condition of the reverse SQL is "select student_id from score". Then, for the data table student, the reverse SQL generated after concatenation is "select student.*from student where student.id in(select student_id from score)". For the data table score, the subquery condition of the reverse SQL is "select id from student". Then, for the data table score, the reverse SQL generated after concatenation is "select score.*from score wherescore.student_id in(select id from student)".

[0134] In step S645, the reverse SQL of the source SQL to be traced is constructed for a single-table type.

[0135] In the example embodiment, the reverse SQL of the SQL to be traced in a single table type is constructed through the following steps: (1) Obtain the table name, the table name after the join, the join conditions, etc. from the parsing result of the SQL to be traced, and obtain the fields corresponding to each table from the metadata. (2) Determine the single table traceability elements in the predetermined element information based on the single table type, and construct the single table traceability element model corresponding to the SQL to be traced. (3) Concatenate each traceability element in the single table traceability element model based on the SQL syntax structure of the single table type to generate the reverse SQL corresponding to the SQL to be traced.

[0136] For example, suppose the SQL to be traced is a single-table type, such as `select id from student where name = 'jack'`. The predefined elements include the data operation type `select`, the table name `student`, the query condition `where`, and the field `name`. Based on the SQL syntax structure of the single-table type, the various elements in the predefined element information are concatenated to generate the reverse SQL corresponding to the SQL to be traced, i.e., `select * from student where name = 'jack'`. Here, "*" is the fill element in the reverse SQL, indicating that it can be replaced with the field in the corresponding data table.

[0137] In step S650, the reverse SQL is output.

[0138] In the example embodiment, reverse SQL is output on the graphical user interface.

[0139] In step S655, SQL tracing is performed based on the reverse SQL.

[0140] In an example embodiment, the reverse SQL may include filling elements such as the symbol "*", obtaining the corresponding field of the data table from the metadata of the data table; filling the obtained field into the filling elements of the reverse SQL; executing the reverse SQL, and performing data tracing processing on the execution result corresponding to the SQL to be traced.

[0141] For example, suppose the data table is a student table, and the fields of the data table include unique identifier ID, name, class, etc. Suppose the SQL to be traced is a single table type, such as select id from student where name = 'jack'. The query result of the SQL to be traced is the unique identifier ID of the name "jack". The reverse SQL is select * from student where name = 'jack'. In response to the data tracing request of the SQL to be traced, the reverse SQL is executed, and the execution result of the reverse SQL is obtained, that is, the result of the data tracing processing. The execution result of the reverse SQL is a row of records in the student table with the name "jack".

[0142] according to Figure 6 The technical solution in the example embodiment, on the one hand, obtains the corresponding abstract syntax tree by parsing the SQL to be traced, and obtains the predetermined element information corresponding to the SQL to be traced based on the abstract syntax tree, which can accurately determine the key elements in the SQL to be traced; on the other hand, based on the predetermined element information and the syntax structure of the SQL to be traced, a reverse SQL corresponding to the SQL to be traced is constructed, and data tracing processing is performed on the execution result corresponding to the SQL to be traced based on the reverse SQL. Since the reverse SQL can quickly and accurately query the original data of the execution result of the SQL to be traced, the source of the data in the execution result can be quickly and accurately located; furthermore, since data tracing processing is performed on the data in the execution result only when needed, the storage space occupied by data tracing processing can be reduced.

[0143] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0144] Below, we will combine Figure 8 as well as Figure 1 The system architecture shown in this specification provides a detailed description of the data traceability device provided in the embodiments. It should be noted that... Figure 8 The data traceability device in this document is used to execute the instructions. Figures 2-7 The methods shown in the embodiments are illustrated for ease of explanation, showing only the parts related to the embodiments of this specification. For specific technical details not disclosed, please refer to this specification. Figures 2-7 The example shown.

[0145] Please see Figure 8 This is a schematic diagram of the structure of a data traceability device provided in an embodiment of this specification. Figure 8 As shown, the data traceability device 800 in this embodiment may include:

[0146] Parsing module 810 is used to parse the structured query language SQL to be traced and obtain the abstract syntax tree corresponding to the SQL to be traced.

[0147] The element acquisition module 820 is used to acquire the predetermined element information corresponding to the SQL to be traced based on the abstract syntax tree;

[0148] The reverse function construction module 830 is used to construct the reverse SQL corresponding to the SQL to be traced based on the predetermined element information and the syntax structure of the SQL to be traced.

[0149] The traceability processing module 840 is used to perform data traceability processing based on the reverse SQL.

[0150] In some example embodiments, based on the above scheme, the inverse function construction module 830 includes:

[0151] The structure type determination unit is used to determine the syntax structure type corresponding to the SQL to be traced based on the syntax structure of the SQL to be traced.

[0152] The reverse SQL determination unit is used to construct the reverse SQL corresponding to the SQL to be traced based on the syntax structure type and the predetermined element information.

[0153] In some example embodiments, based on the above scheme, the syntax structure type is one of the following: single-table type, multi-table type, and multi-level nested type.

[0154] In some example embodiments, based on the above scheme, the reverse SQL determination unit includes:

[0155] The element model construction unit is used to construct the source element model corresponding to the SQL to be traced based on the syntax structure type and the predetermined element information.

[0156] The splicing unit is used to splice the various traceability elements in the traceability element model based on the SQL syntax structure corresponding to the syntax structure type, and generate the reverse SQL corresponding to the SQL to be traced.

[0157] In some example embodiments, based on the above scheme, the syntax structure type is a single-table type, and the feature model construction unit is configured as follows:

[0158] Based on the single table type, determine the single table traceability elements in the predetermined element information;

[0159] Based on the single-table traceability elements, construct the single-table traceability element model corresponding to the SQL to be traced.

[0160] The splicing unit is configured as follows:

[0161] Based on the SQL syntax structure of the single-table type, the various traceability elements in the single-table traceability element model are concatenated to generate the reverse SQL corresponding to the traceability SQL.

[0162] In some example embodiments, based on the above scheme, the syntax structure type is a multi-table type, and the feature model construction unit is configured as follows:

[0163] Based on the multi-table type, the multi-table traceability elements in the predetermined element information are determined, and the multi-table traceability elements include multi-table related elements;

[0164] Based on the multi-table tracing elements, a multi-table tracing element model corresponding to the SQL to be traced is constructed.

[0165] The splicing unit is configured as follows:

[0166] Based on the SQL syntax structure of the multi-table type and the multi-table association elements, the various traceability elements in the multi-table traceability element model are spliced ​​together.

[0167] In some example embodiments, based on the above scheme, the splicing unit is further configured as follows:

[0168] Based on the abstract syntax tree, determine the table association conditions corresponding to the multi-table association elements;

[0169] Based on the table association conditions, determine the subquery conditions for the reverse SQL for each of the multiple data tables;

[0170] Based on the subquery conditions of the reverse SQL for each of the data tables and the SQL syntax structure of the multi-table type, the traceability elements corresponding to each of the data tables in the multi-table traceability element model are concatenated.

[0171] The reverse SQL determination unit is configured as follows:

[0172] Generate the reverse SQL for each data table corresponding to the SQL to be traced.

[0173] In some example embodiments, based on the above scheme, the syntax structure type is a multi-level nested type, and the feature model building unit is configured as follows:

[0174] Based on the multi-level nesting type, nested tracing elements in the predetermined element information are determined, and the nested tracing elements include the element's level information;

[0175] Based on the nested tracing elements, a multi-level nested tracing element model corresponding to the SQL to be traced is constructed.

[0176] The splicing unit is configured as follows:

[0177] Based on the SQL syntax structure of the aforementioned multi-level nested type and the hierarchical information of the elements, the traceability elements in the multi-level nested traceability element model are concatenated layer by layer.

[0178] The reverse SQL determination unit is configured as follows:

[0179] Generate the reverse SQL for each level corresponding to the SQL to be traced.

[0180] In some example embodiments, based on the above scheme, the splicing unit is further configured as follows:

[0181] The hierarchy of each traceability element in the multi-layered nested traceability element model is determined based on the hierarchical information of the element.

[0182] Based on the syntax structure of SQL at each level, determine the syntax structure type of SQL at each level, following the order from the innermost to the outermost layer.

[0183] Based on the syntax structure type of SQL at each level, the traceability elements in the multi-level nested traceability element model corresponding to each level are spliced ​​together.

[0184] In some example embodiments, based on the above scheme, the splicing unit is further configured as follows:

[0185] The inner SQL subquery is used as a temporary table to concatenate the source elements in the corresponding multi-layer nested source element model of the outer layer.

[0186] Replace the temporary table in the concatenated reverse SQL with the SQL subquery.

[0187] In some example embodiments, based on the above scheme, the reverse SQL includes fill elements, which are used to replace fields in the corresponding data table, and the tracing processing module 840 is configured as follows:

[0188] Obtain the fields corresponding to the data table from the metadata of the data table;

[0189] The field is populated into the fill element of the reverse SQL;

[0190] Execute the reverse SQL and perform data tracing processing on the execution result corresponding to the SQL to be traced.

[0191] In some example embodiments, based on the above scheme, the parsing module 810 is configured as follows:

[0192] Lexical analysis of the SQL to be traced is performed based on the lexical rules of the SQL language to obtain the symbol stream corresponding to the SQL to be traced.

[0193] Based on the syntax rules of the SQL language, the symbol stream is parsed to generate an abstract syntax tree corresponding to the SQL to be traced.

[0194] In some example embodiments, based on the above scheme, the data traceability device 800 further includes:

[0195] The display module is used to show the processing results of the data tracing process through a data lineage graph.

[0196] According to this instruction manual Figure 8 The technical solution of the embodiment has two aspects. On the one hand, by parsing the SQL to be traced to obtain the corresponding abstract syntax tree, and based on the abstract syntax tree, the predetermined element information corresponding to the SQL to be traced can be obtained, which can accurately determine the key elements in the SQL to be traced. On the other hand, based on the predetermined element information and the syntax structure of the SQL to be traced, the reverse SQL corresponding to the SQL to be traced is constructed. Based on the reverse SQL, the execution result corresponding to the SQL to be traced is subjected to data tracing processing. Since the reverse SQL can quickly and accurately query the original data of the execution result of the SQL to be traced, the source of the data in the execution result can be quickly and accurately located. Furthermore, since the data tracing processing in the execution result is only performed when needed, the storage space occupied by the data tracing processing can be reduced.

[0197] The above is a schematic scheme of a data traceability device according to an embodiment of this specification. It should be noted that the technical solution of this data traceability device and the technical solution of the data traceability method described above belong to the same concept. For details not described in detail in the technical solution of the data traceability device, please refer to the description of the technical solution of the data traceability method described above.

[0198] This specification also provides a computer storage medium that can store multiple program instructions adapted to be loaded and executed by a processor as described above. Figures 2-7 The method steps of the illustrated embodiment can be found in the following documentation for detailed execution. Figures 2-7 The specific details of the illustrated embodiments will not be elaborated here.

[0199] This specification also provides a computer program product that stores at least one instruction, which is loaded and executed by a processor as described above. Figures 2-7 The data tracing method described in the illustrated embodiment can be found in the following documentation for its specific execution process. Figures 2-7 The specific details of the illustrated embodiments will not be elaborated here.

[0200] Please refer to Figure 9 This diagram illustrates the structure of an electronic device provided in an exemplary embodiment of this specification. The electronic device in this specification may include one or more of the following components: a processor 910, a memory 920, an input device 930, an output device 940, and a bus 950. The processor 910, memory 920, input device 930, and output device 940 may be connected via the bus 950.

[0201] Processor 910 may include one or more processing cores. Processor 910 connects to various parts of the electronic device through various interfaces and lines, and performs various functions and processes data of electronic device 900 by running or executing instructions, programs, code sets, or instruction sets stored in memory 920, and by calling data stored in memory 920. Optionally, processor 910 may be implemented using at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), or programmable logic array (PLA). Processor 910 may integrate one or more of the following: central processing unit (CPU), graphics processing unit (GPU), and modem. The CPU mainly handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the displayed content; and the modem is used for wireless communication. It is understood that the modem may also not be integrated into processor 910 and may be implemented separately through a communication chip.

[0202] The memory 920 may include random access memory (RAM) or read-only memory (ROM). Optionally, the memory 920 may include a non-transitory computer-readable storage medium. The memory 920 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 920 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (e.g., touch function, sound playback function, image playback function, etc.), instructions for implementing the various method embodiments described below, etc. The operating system may be an Android system, including systems deeply developed based on the Android system, an iOS system, including systems deeply developed based on the iOS system, or other systems.

[0203] In order for the operating system to distinguish the specific application scenarios of third-party applications, it is necessary to establish data communication between the third-party applications and the operating system. This would allow the operating system to obtain the current scenario information of the third-party applications at any time, and then perform targeted system resource adaptation based on the current scenario.

[0204] The input device 930 is used to receive input instructions or data, and includes, but is not limited to, a keyboard, mouse, camera, microphone, or touch device. The output device 940 is used to output instructions or data, and includes, but is not limited to, a display device and a speaker. In one example, the input device 930 and the output device 940 can be combined, and both the input device 930 and the output device 940 can be a touch display screen.

[0205] In addition, those skilled in the art will understand that the structure of the electronic device shown in the above figures does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements. For example, the electronic device may also include radio frequency circuits, input units, sensors, audio circuits, Wireless Fidelity (WiFi) modules, power supplies, Bluetooth modules, etc., which will not be described in detail here.

[0206] exist Figure 9 In the illustrated electronic device, the processor 910 can be used to call the data tracing application stored in the memory 920 and specifically perform the following operations:

[0207] The structured query language (SQL) to be traced is parsed to obtain the abstract syntax tree corresponding to the SQL to be traced.

[0208] Based on the abstract syntax tree, obtain the predetermined element information corresponding to the SQL to be traced;

[0209] Based on the predetermined element information and the syntax structure of the SQL to be traced, construct the reverse SQL corresponding to the SQL to be traced;

[0210] Data tracing is performed based on the reverse SQL.

[0211] In some example embodiments, when the processor 910 executes the reverse SQL corresponding to the SQL to be traced based on the predetermined element information and the syntax structure of the SQL to be traced, it specifically performs the following operations:

[0212] Based on the syntax structure of the SQL to be traced, determine the syntax structure type corresponding to the SQL to be traced;

[0213] Based on the grammatical structure type and the predetermined element information, construct the reverse SQL corresponding to the SQL to be traced.

[0214] In some example embodiments, the syntax structure type is one of the following: single-table type, multi-table type, and multi-level nested type.

[0215] In some example embodiments, when the processor 910 executes the reverse SQL corresponding to the source SQL based on the syntax structure type and the predetermined element information, it specifically performs the following operations:

[0216] Based on the grammatical structure type and the predetermined element information, construct the source element model corresponding to the SQL to be traced;

[0217] Based on the SQL syntax structure corresponding to the syntax structure type, the traceability elements in the traceability element model are concatenated to generate the reverse SQL corresponding to the SQL to be traced.

[0218] In some example embodiments, the syntax structure type is a single-table type. When the processor 910 executes the step of constructing the traceability element model corresponding to the SQL to be traced based on the syntax structure type and the predetermined element information, it specifically performs the following operations:

[0219] Based on the single table type, determine the single table traceability elements in the predetermined element information;

[0220] Based on the single-table traceability elements, construct the single-table traceability element model corresponding to the SQL to be traced.

[0221] The step of concatenating the various traceability elements in the traceability element model based on the SQL syntax structure corresponding to the syntax structure type includes:

[0222] Based on the SQL syntax structure of the single-table type, the various traceability elements in the single-table traceability element model are concatenated to generate the reverse SQL corresponding to the traceability SQL.

[0223] In some example embodiments, the syntax structure type is a multi-table type. When the processor 910 executes the step of constructing the source element model corresponding to the SQL to be traced based on the syntax structure type and the predetermined element information, it specifically performs the following operations:

[0224] Based on the multi-table type, the multi-table traceability elements in the predetermined element information are determined, and the multi-table traceability elements include multi-table related elements;

[0225] Based on the multi-table tracing elements, a multi-table tracing element model corresponding to the SQL to be traced is constructed.

[0226] The step of concatenating the various traceability elements in the traceability element model based on the SQL syntax structure corresponding to the syntax structure type includes:

[0227] Based on the SQL syntax structure of the multi-table type and the multi-table association elements, the various traceability elements in the multi-table traceability element model are spliced ​​together.

[0228] In some example embodiments, when the processor 910 executes the SQL syntax structure based on the multi-table type and the multi-table association elements to concatenate the various traceability elements in the multi-table traceability element model, it specifically performs the following operations:

[0229] Based on the abstract syntax tree, determine the table association conditions corresponding to the multi-table association elements;

[0230] Based on the table association conditions, determine the subquery conditions for the reverse SQL for each of the multiple data tables;

[0231] Based on the subquery conditions of the reverse SQL for each of the data tables and the SQL syntax structure of the multi-table type, the traceability elements corresponding to each of the data tables in the multi-table traceability element model are concatenated.

[0232] The generation of the reverse SQL corresponding to the SQL to be traced includes:

[0233] Generate the reverse SQL for each data table corresponding to the SQL to be traced.

[0234] In some example embodiments, the syntax structure type is a multi-level nested type. When the processor 1110 executes the step of constructing the source element model corresponding to the SQL to be traced based on the syntax structure type and the predetermined element information, it specifically performs the following operations:

[0235] Based on the multi-level nesting type, nested tracing elements in the predetermined element information are determined, and the nested tracing elements include the element's level information;

[0236] Based on the nested tracing elements, a multi-level nested tracing element model corresponding to the SQL to be traced is constructed.

[0237] The step of concatenating the various traceability elements in the traceability element model based on the SQL syntax structure corresponding to the syntax structure type includes:

[0238] Based on the SQL syntax structure of the aforementioned multi-level nested type and the hierarchical information of the elements, the traceability elements in the multi-level nested traceability element model are concatenated layer by layer.

[0239] The generation of the reverse SQL corresponding to the SQL to be traced includes:

[0240] Generate the reverse SQL for each level corresponding to the SQL to be traced.

[0241] In some example embodiments, when the processor 910 executes the SQL syntax structure based on the multi-level nested type and the element's hierarchical information to concatenate each traceable element in the multi-level nested traceable element model layer by layer, it specifically performs the following operations:

[0242] The hierarchy of each traceability element in the multi-layered nested traceability element model is determined based on the hierarchical information of the element.

[0243] Based on the syntax structure of SQL at each level, determine the syntax structure type of SQL at each level, following the order from the innermost to the outermost layer.

[0244] Based on the syntax structure type of SQL at each level, the traceability elements in the multi-level nested traceability element model corresponding to each level are spliced ​​together.

[0245] In some example embodiments, when the processor 910 executes the syntax structure type of the SQL based on each level and concatenates the traceability elements in the multi-level nested traceability element model corresponding to each level, it specifically performs the following operations:

[0246] The inner SQL subquery is used as a temporary table to concatenate the source elements in the corresponding multi-layer nested source element model of the outer layer.

[0247] Replace the temporary table in the concatenated reverse SQL with the SQL subquery.

[0248] In some example embodiments, the reverse SQL includes fill elements used to replace fields in the corresponding data table. When the processor 910 executes the data tracing processing based on the reverse SQL, it specifically performs the following operations:

[0249] Obtain the fields corresponding to the data table from the metadata of the data table;

[0250] The field is populated into the fill element of the reverse SQL;

[0251] Execute the reverse SQL and perform data tracing processing on the execution result corresponding to the SQL to be traced.

[0252] In some example embodiments, when the processor 910 parses the structured query language (SQL) to be traced and obtains the abstract syntax tree corresponding to the SQL to be traced, it specifically performs the following operations:

[0253] Lexical analysis of the SQL to be traced is performed based on the lexical rules of the SQL language to obtain the symbol stream corresponding to the SQL to be traced.

[0254] Based on the syntax rules of the SQL language, the symbol stream is parsed to generate an abstract syntax tree corresponding to the SQL to be traced.

[0255] In some example embodiments, the processor 910 also performs the following operations:

[0256] The processing results of the data tracing process are displayed using a data lineage graph method.

[0257] The above is an illustrative embodiment of an electronic device according to this specification. It should be noted that the technical solution of this electronic device and the technical solution of the data traceability method described above belong to the same concept. Details not described in detail in the technical solution of the electronic device can be found in the description of the technical solution of the data traceability method described above.

[0258] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium for the computer program can be a magnetic disk, optical disk, read-only memory, or random access memory, etc.

[0259] The above-disclosed embodiments are merely preferred embodiments of this specification and should not be construed as limiting the scope of this specification. Therefore, any equivalent variations made in accordance with the claims of this specification shall still fall within the scope of this specification.

[0260] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.< / expression> < / expression> < / condition> < / condition> ...where < / expression> < / expression>

Claims

1. A data tracing method, comprising: The structured query language (SQL) to be traced is parsed to obtain the abstract syntax tree corresponding to the SQL to be traced. Based on the abstract syntax tree, obtain the predetermined element information corresponding to the SQL to be traced; Based on the predetermined element information and the syntax structure of the SQL to be traced, construct the reverse SQL corresponding to the SQL to be traced; Data tracing is performed based on the reverse SQL, where the reverse SQL refers to the SQL used to obtain the original data of the forward SQL statement; The reverse SQL includes a fill element, which is used to replace the corresponding field in the data table. The data tracing process based on the reverse SQL includes: obtaining the field corresponding to the data table from the metadata of the data table; filling the field into the fill element of the reverse SQL; executing the reverse SQL and performing data tracing processing on the execution result corresponding to the SQL to be traced. The step of constructing the reverse SQL corresponding to the source SQL based on the predetermined element information and the syntax structure of the source SQL to be traced includes: determining the syntax structure type corresponding to the source SQL based on the syntax structure of the source SQL; constructing the reverse SQL corresponding to the source SQL based on the syntax structure type and the predetermined element information; wherein the syntax structure type is one of the following: single-table type, multi-table type, and multi-level nested type. The step of constructing the reverse SQL corresponding to the SQL to be traced based on the syntax structure type and the predetermined element information includes: constructing a traceability element model corresponding to the SQL to be traced based on the syntax structure type and the predetermined element information; and concatenating each traceability element in the traceability element model based on the SQL syntax structure corresponding to the syntax structure type to generate the reverse SQL corresponding to the SQL to be traced.

2. The method according to claim 1, wherein, The syntax structure type is a single-table type. The step of constructing the source element model corresponding to the SQL to be traced, based on the syntax structure type and the predetermined element information, includes: Based on the single table type, determine the single table traceability elements in the predetermined element information; Based on the single-table traceability elements, construct the single-table traceability element model corresponding to the SQL to be traced. The step of concatenating the various traceability elements in the traceability element model based on the SQL syntax structure corresponding to the syntax structure type includes: Based on the SQL syntax structure of the single-table type, the various traceability elements in the single-table traceability element model are concatenated to generate the reverse SQL corresponding to the traceability SQL.

3. The method according to claim 1, wherein, The syntax structure type is a multi-table type. The step of constructing the source element model corresponding to the SQL to be traced, based on the syntax structure type and the predetermined element information, includes: Based on the multi-table type, the multi-table traceability elements in the predetermined element information are determined, and the multi-table traceability elements include multi-table related elements; Based on the multi-table tracing elements, a multi-table tracing element model corresponding to the SQL to be traced is constructed. The step of concatenating the various traceability elements in the traceability element model based on the SQL syntax structure corresponding to the syntax structure type includes: Based on the SQL syntax structure of the multi-table type and the multi-table association elements, the various traceability elements in the multi-table traceability element model are spliced ​​together.

4. The method according to claim 3, wherein, The process of concatenating the various source elements in the multi-table source element model based on the SQL syntax structure of the multi-table type and the multi-table association elements includes: Based on the abstract syntax tree, determine the table association conditions corresponding to the multi-table association elements; Based on the table association conditions, determine the subquery conditions for the reverse SQL for each of the multiple data tables; Based on the subquery conditions of the reverse SQL for each of the data tables and the SQL syntax structure of the multi-table type, the traceability elements corresponding to each of the data tables in the multi-table traceability element model are concatenated. The generation of the reverse SQL corresponding to the SQL to be traced includes: Generate the reverse SQL for each data table corresponding to the SQL to be traced.

5. The method according to claim 1, wherein, The syntax structure type is a multi-level nested type. The step of constructing the source element model corresponding to the SQL to be traced, based on the syntax structure type and the predetermined element information, includes: Based on the multi-level nesting type, nested tracing elements in the predetermined element information are determined, and the nested tracing elements include the element's level information; Based on the nested tracing elements, a multi-level nested tracing element model corresponding to the SQL to be traced is constructed. The step of concatenating the various traceability elements in the traceability element model based on the SQL syntax structure corresponding to the syntax structure type includes: Based on the SQL syntax structure of the aforementioned multi-level nested type and the hierarchical information of the elements, the traceability elements in the multi-level nested traceability element model are concatenated layer by layer. The generation of the reverse SQL corresponding to the SQL to be traced includes: Generate the reverse SQL for each level corresponding to the SQL to be traced.

6. The method according to claim 5, wherein, The SQL syntax structure based on the multi-level nested type and the hierarchical information of the elements are used to concatenate each source element in the multi-level nested source element model layer by layer, including: The hierarchy of each traceability element in the multi-layered nested traceability element model is determined based on the hierarchical information of the element. Based on the syntax structure of SQL at each level, determine the syntax structure type of SQL at each level, following the order from the innermost to the outermost layer. Based on the syntax structure type of SQL at each level, the traceability elements in the multi-level nested traceability element model corresponding to each level are spliced ​​together.

7. The method according to claim 6, wherein, The method of concatenating the traceability elements in the multi-level nested traceability element model corresponding to each level, based on the syntax structure type of SQL at each level, includes: The inner SQL subquery is used as a temporary table to concatenate the source elements in the corresponding multi-layer nested source element model of the outer layer. Replace the temporary table in the concatenated reverse SQL with the SQL subquery.

8. The method according to any one of claims 1 to 7, wherein, The process of parsing the structured query language (SQL) to be traced to obtain the abstract syntax tree corresponding to the SQL to be traced includes: Lexical analysis of the SQL to be traced is performed based on the lexical rules of the SQL language to obtain the symbol stream corresponding to the SQL to be traced. Based on the syntax rules of the SQL language, the symbol stream is parsed to generate an abstract syntax tree corresponding to the SQL to be traced.

9. The method according to any one of claims 1 to 7, wherein, The method further includes: The processing results of the data tracing process are displayed using a data lineage graph method.

10. A data traceability device, comprising: The parsing module is used to parse the structured query language (SQL) to be traced and obtain the abstract syntax tree corresponding to the SQL to be traced. The element acquisition module is used to acquire the predetermined element information corresponding to the SQL to be traced based on the abstract syntax tree; The reverse function construction module is used to construct the reverse SQL corresponding to the SQL to be traced based on the predetermined element information and the syntax structure of the SQL to be traced. The data tracing module is used to perform data tracing processing based on the reverse SQL, where the reverse SQL refers to the SQL used to obtain the original data of the forward SQL statement. The reverse SQL includes a fill element, which is used to replace the corresponding field in the data table. The data tracing process based on the reverse SQL includes: obtaining the field corresponding to the data table from the metadata of the data table; filling the field into the fill element of the reverse SQL; executing the reverse SQL and performing data tracing processing on the execution result corresponding to the SQL to be traced. The step of constructing the reverse SQL corresponding to the source SQL based on the predetermined element information and the syntax structure of the source SQL to be traced includes: determining the syntax structure type corresponding to the source SQL based on the syntax structure of the source SQL; constructing the reverse SQL corresponding to the source SQL based on the syntax structure type and the predetermined element information; wherein the syntax structure type is one of the following: single-table type, multi-table type, and multi-level nested type. The step of constructing the reverse SQL corresponding to the SQL to be traced based on the syntax structure type and the predetermined element information includes: constructing a traceability element model corresponding to the SQL to be traced based on the syntax structure type and the predetermined element information; and concatenating each traceability element in the traceability element model based on the SQL syntax structure corresponding to the syntax structure type to generate the reverse SQL corresponding to the SQL to be traced.

11. A computer storage medium storing a plurality of instructions adapted for loading by a processor and performing the steps of the method as claimed in any one of claims 1 to 9.

12. An electronic device, comprising: A processor and a memory; wherein the memory stores a computer program adapted to be loaded by the processor and to execute the steps of the method as claimed in any one of claims 1 to 9.

13. A computer program product comprising instructions that, when run on a computer or processor, cause the computer or processor to perform the steps of the method as described in any one of claims 1-9.

Images