Data bloodline analysis method and device and electronic equipment
By adding annotations to SQL statements to generate standardized traceability syntax trees and converting them into query matrices, the high complexity and maintenance difficulties caused by different SQL writing habits in traditional methods are solved, achieving efficient and easy-to-maintain data lineage parsing.
Patent Information
- Application Number
- CN202310272552.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-03-17
- Publication Date
- 2026-01-06
- Estimated Expiration
- 2043-03-17
AI Technical Summary
Traditional data lineage analysis methods rely on individual SQL statement writing habits, resulting in high algorithm complexity, maintenance difficulties, and the inability to confirm convergence of the tracing results.
By adding annotations to SQL statements, a standardized tracing syntax tree is generated and transformed into a query matrix to generate a data lineage model, avoiding the need for different SQL writing habits and simplifying the tracing process.
It achieves high efficiency and simplification in data lineage analysis, reduces algorithm complexity, and improves the generation efficiency and maintainability of data lineage models.
Smart Images

Figure CN116467331B_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of big data, and in particular to a data lineage analysis method, apparatus, and electronic device. Background Technology
[0002] In related technologies, data lineage analysis is a crucial step in building a global view of data assets within data asset management. Traditional problem-solving involves constructing a basic algorithm based on typical use cases, directly tracing the source of data within the original SQL statement. This algorithm is then iteratively optimized using actual data until it converges. The robustness of the algorithm heavily relies on the richness of the data. However, the diverse SQL syntax habits of different individuals place a significant burden on the algorithm, resulting in numerous hard-coded processing branches and ultimately a large, complex, and difficult-to-maintain algorithm. Furthermore, while the tracing results can approach convergence, final convergence cannot be confirmed. Summary of the Invention
[0003] This disclosure provides a data lineage analysis method, apparatus, and electronic device to at least solve the problem of poor data lineage analysis performance in related technologies. The technical solution of this disclosure is as follows:
[0004] According to a first aspect of the present disclosure, a data lineage analysis method is provided, comprising:
[0005] Obtain a lineage resolution request, wherein the lineage resolution request contains at least one Structured Query Language (SQL) statement;
[0006] Add annotations to the SQL statements in the lineage resolution request, and generate a standardized tracing syntax tree based on the SQL statements;
[0007] The corresponding data lineage model is generated based on each subquery in the standardized source tracing syntax tree.
[0008] Optionally, the step of adding annotations to the SQL statement in the lineage resolution request specifically includes:
[0009] Inject the corresponding field alias at the source field location;
[0010] Inject a reference alias at the source field location;
[0011] Inject the field name at the group by or order by position.
[0012] Optionally, the steps further include:
[0013] Inject the main clause of the SQL statement into the WITH clause;
[0014] Inject asset metadata.
[0015] Optionally, the step of generating the corresponding data lineage model based on each subquery in the standardized source tracing syntax tree specifically includes:
[0016] The standardized source syntax tree is transformed into a query matrix, wherein each row of the query matrix is a subquery;
[0017] Based on the subqueries in each row of the query matrix, trace the source and merge the same trace results to obtain the target field model and the source field model, and generate the data lineage model.
[0018] Optionally, the method further includes:
[0019] If a subquery in the SQL statement is not related to other statements in the SQL statement, the subquery is traced separately to generate the data lineage model.
[0020] According to a second aspect of the present disclosure, a data lineage analysis apparatus is provided, comprising:
[0021] A receiving module is used to obtain a lineage resolution request, wherein the lineage resolution request contains at least one Structured Query Language (SQL) statement;
[0022] The annotation module is used to add annotations to the SQL statements in the lineage resolution request and generate a standardized tracing syntax tree based on the SQL statements;
[0023] The tracing module is used to generate corresponding data lineage models based on each subquery in the standardized tracing syntax tree.
[0024] According to a third aspect of the present disclosure, an electronic device is provided, characterized in that it includes:
[0025] processor;
[0026] Memory used to store the processor's executable instructions;
[0027] The processor is configured to execute the instructions to implement the method as described in any one of the first aspects.
[0028] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided that, when executed by a processor of an electronic device, enables the electronic device to perform the method as described in any one of the first aspects.
[0029] The technical solutions provided by the embodiments of this disclosure have at least the following beneficial effects:
[0030] This disclosure standardizes the syntax tree by adding annotations to the SQL statements in the lineage resolution request, avoiding increased difficulty in tracing the source due to varying SQL statement writing habits, and improving the efficiency of data lineage resolution.
[0031] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description
[0032] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure, and are not intended to unduly limit this disclosure.
[0033] Figure 1 This is a flowchart illustrating a data lineage analysis method according to an exemplary embodiment.
[0034] Figure 2 This is a flowchart illustrating a data lineage analysis method according to an exemplary embodiment.
[0035] Figure 3 This is a flowchart illustrating a data lineage analysis method according to an exemplary embodiment.
[0036] Figure 4 This is a schematic diagram illustrating a data lineage model according to an exemplary embodiment.
[0037] Figure 5 This is a block diagram illustrating a data lineage analysis apparatus according to an exemplary embodiment.
[0038] Figure 6 This is a block diagram illustrating an apparatus according to an exemplary embodiment.
[0039] Figure 7 This is a block diagram illustrating an apparatus according to an exemplary embodiment. Detailed Implementation
[0040] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings.
[0041] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.
[0042] It should be noted that the user information involved in this disclosure (including but not limited to user device information, user personal information, etc.) is all information authorized by the user or fully authorized by all parties.
[0043] In data asset management, data lineage analysis is a crucial step in building a global view of data assets. SQL lineage relationships are a core dependency in data warehouse model construction. By analyzing and parsing SQL statements, the dependencies and attribute dependencies between tables at different business layers are obtained and visualized, forming a hierarchical relationship diagram of data tables and attributes. This fully demonstrates the mapping relationship between raw field data and the data model. A robust SQL lineage system not only benefits data analysts in understanding business scenarios but also greatly assists in building a layered data warehouse. Furthermore, it plays a vital role in tracing the origins of data quality, and significantly aids in constructing data link diagrams and monitoring data changes.
[0044] Traditional data lineage analysis approaches fall into two categories. The first is hierarchical storage, followed by data processing to trace lineage information layer by layer. This approach suffers from several drawbacks: large data volume, excessive redundancy, complex processing algorithms, and poor real-time performance. When dealing with massive data assets, it may be impossible to complete the full lineage information construction within T+1 days. Another issue is that all scenarios where lineage cannot be traced are treated as missing database / table metadata, which is often not the case, leading to performance degradation due to excessive reliance on database / table metadata. The second approach involves building a basic algorithm for tracing lineage directly within the original SQL statement based on typical use cases. This algorithm is then iteratively optimized using real-world data to achieve convergence. However, this approach has a long development cycle, increasingly complex algorithms, extensive hard-coded data, and high maintenance costs. Handling relationships between fields by name and position in SQL is extremely complex. Most importantly, the algorithm lacks logical consistency and can never exhaustively cover all scenarios.
[0045] The specific process of analyzing data lineage in related technologies is as follows:
[0046] 1) Transform SQL statements into abstract syntax trees.
[0047] 2) Split the abstract syntax tree at the granularity of subqueries.
[0048] 3) Store the subqueries of each layer, as well as the corresponding source and target fields for that layer.
[0049] 4) Penetrate all levels of subqueries to obtain the target field and the innermost source field.
[0050] 5) If penetration is not possible, use database and table metadata to assist in penetration.
[0051] While the hierarchical storage method described above reduces the complexity of tracing, it also leads to excessive information storage, a surge in database pressure, and poor performance in subsequent processing and analysis. Furthermore, the stored information contains a large amount of useless redundancy, especially for nested subqueries with dozens of levels of nesting, resulting in significant storage waste. In scenarios where tracing is impossible, the problem is simply attributed to missing database / table metadata, making it difficult to perform logical deduction based on contextual information for accurate confirmation, leading to excessive queries of database / table metadata.
[0052] Another analytical process for data lineage in related technologies is as follows:
[0053] Implement the basic algorithm based on existing scripts.
[0054] Based on incremental script iterative optimization algorithm
[0055] The above method avoids the storage waste problem of Solution 1 by directly tracing the algorithm's origin and continuously strengthening it with real data. However, the robustness of the algorithm heavily depends on the richness of the data. Different people have different SQL writing habits, which can place a huge burden on the algorithm, resulting in numerous hard-coded processing branches and ultimately a large, complex, and difficult-to-maintain algorithm. Tracing the origin can only approach convergence; it cannot confirm final convergence.
[0056] Figure 1 This is a flowchart illustrating a data lineage analysis method according to an exemplary embodiment, such as... Figure 1 As shown, the method includes the following steps:
[0057] Step 101: Obtain a lineage resolution request, wherein the lineage resolution request contains at least one Structured Query Language (SQL) statement.
[0058] Structured Query Language (SQL) is a database query and programming language used to access, query, update, and manage relational database systems. SQL statements are a language used to manipulate databases.
[0059] The method in this embodiment can be applied to a server. The server can receive lineage resolution requests sent by clients. A lineage resolution request can carry multiple SQL statements. During data processing, a relationship naturally forms between the data, called the data lineage. Data lineage refers to the generation process chain of a table (which tables it depends on, how it was generated, and what processes and stages it went through); and the generation process chain of the tables it depends on. The purpose of this embodiment is to analyze and obtain the data lineage based on the SQL statements in the lineage resolution request.
[0060] Step 102: Add annotations to the SQL statement in the lineage resolution request, and generate a standardized tracing syntax tree based on the SQL statement.
[0061] In this embodiment, the server can perform syntax analysis on each SQL statement in the lineage resolution request to obtain an Abstract Syntax Tree (AST) corresponding to each SQL statement. The AST is a tree-like representation of the abstract syntactic structure of the source code. Each node in the AST represents a structure in the source code.
[0062] In related technologies, because everyone has different SQL statement writing habits, the abstract syntax trees generated when performing lineage analysis on SQL statements that perform the same task may differ, increasing the parsing difficulty and leading to a greater system load. To standardize the abstract syntax tree, annotations are added to the SQL statements in the lineage analysis request, thus standardizing the abstract syntax tree. A standardized source syntax tree can then be generated by performing syntax analysis on the annotated SQL statements.
[0063] Step 103: Generate the corresponding data lineage model based on each subquery in the standardized source tracing syntax tree.
[0064] In this embodiment, after generating a standardized traceability syntax tree, the data can be clearly traced back to its source, and the dependency relationships and attribute dependencies between various business layer tables can be obtained to generate a data lineage model and visualize it, forming a data table and attribute lineage hierarchy diagram to show the mapping relationship between the original field data and the data model.
[0065] Figure 2 This is a flowchart illustrating a data lineage analysis method according to an exemplary embodiment, such as... Figure 2 As shown, Figure 1 Step 102 includes the following steps:
[0066] Step 201: Inject the corresponding field alias at the source field location;
[0067] In this embodiment, in the data selection statement "SELECT <field1>, <field2>, ... FROM ", if the position of each field in the table corresponds to the position of other fields in the table, then the corresponding field alias needs to be injected at the source field position. For example:
[0068] Before injection
[0069] insert into t(k,v)
[0070] select k1,v1 from t1
[0071] union all
[0072] select v1,k1 from t2
[0073] After injection
[0074] insert into t(k,v)
[0075] select k1 as k,v1 as v from t1
[0076] union all
[0077] select v1 as v,k1 as k from t2
[0078] The SQL statements before the injection insert two fields, k and v, into table t using an INSERT statement. Each field corresponds to a column of data. The SELECT ... FROM ... statement selects data from the k1 and v1 fields in table t1 and table t2 respectively. The selected data is then combined using a UNION ALL statement, listing all matching records regardless of duplicate values, and placed into the k and v fields in table t. Here, t is an alias for a field in t1, and v is an alias for a field in v1. By injecting "AS k" after v1 and "AS v" after t1, the syntax tree is standardized, allowing tracing to focus only on the name-based relationships.
[0079] Step 202: Inject a reference alias at the source field location;
[0080] In this embodiment, in the data selection statement "SELECT <field1>, <field2>, ... FROM ", if each field belongs to another table, then the referenced table name needs to be injected at the source field position, i.e., an alias reference, for example:
[0081] Before injection
[0082] insert into t select k+v from t1 left join(select id from t2)tt ont1.id=tt.id
[0083] After injection
[0084] insert into t select t1.k+t1.v from t1 left join(select id from t2)tton t1.id=tt.id
[0085] The SQL statement before injection selects the fields "k" and "v" from table t1 using "select k+v from t1". Then, it uses a left join...on... statement to return all rows from the left table (t1) and adds them to the right table (tt). During the addition, the alignment principle for each row is t1.id = tt.id. Since tt only has the id field, k and v belong only to t1. Therefore, an alias t1 needs to be injected before k and v to indicate their referencing relationship.
[0086] Step 203: Inject the field name at the group by or order by position.
[0087] In this embodiment, the `GROUP BY` statement is used in conjunction with aggregate functions to group the result set based on one or more columns. The `ORDER BY` statement is used to sort the result set based on specified columns. By default, `ORDER BY` sorts records in ascending order. If you want to sort records in descending order, you can use the `DESC` keyword. Both `GROUP BY` and `ORDER BY` are used to filter specific fields. Sometimes, field names are not explicitly inserted into these statements. Injecting field names unifies the abstract syntax tree.
[0088] Optional, Figure 2 The steps also include:
[0089] Inject the main clause of the SQL statement into the WITH clause;
[0090] In this embodiment, the `WITH` statement is equivalent to creating a temporary virtual table, that is, using the `WITH` clause to treat the dataset of the subquery as a temporary in-memory table. Parsing in memory improves execution efficiency and enhances the readability of the SQL statement; it is destroyed after use. The `WITH` clause is equivalent to a subquery, and the subquery and the main clause can be linked by injecting the main clause of the SQL statement.
[0091] Inject asset metadata.
[0092] Figure 3 This is a flowchart illustrating a data lineage analysis method according to an exemplary embodiment, such as... Figure 3 As shown, Figure 1 The steps in step 103 are as follows:
[0093] Step 301: Convert the standardized source syntax tree into a query matrix, wherein each row of the query matrix is a subquery.
[0094] In this embodiment, the standardized source tracing syntax tree is a tree structure containing a root node and multiple child nodes. Each child node is treated as a subquery, and the query matrix is generated by arranging each subquery from top to bottom.
[0095] Step 302: Based on the subqueries in each row of the query matrix, trace the source and merge the same trace results to obtain the target field model and the source field model and generate the data lineage model.
[0096] Figure 4 This is a schematic diagram illustrating a data lineage model according to an exemplary embodiment. For example... Figure 4 As shown in the diagram, the upper model contains a data lineage model, including a target field model and a source field model. The lower model is related to the upper model and represents the table structure corresponding to the target field model, which includes the table model, field names, and the expressions corresponding to the fields.
[0097] Optionally, the method further includes:
[0098] If a subquery in the SQL statement is not related to other statements in the SQL statement, the subquery is traced separately to generate the data lineage model.
[0099] The method in this invention requires no additional storage, boasts superior performance, and can meet the needs of massive data warehouse script analysis with millisecond-level response. The standardization of SQL eliminates subjective SQL writing habits, avoids the significant inconvenience of having both field position and field name correspondences simultaneously, greatly reduces the complexity of the tracing algorithm, and solves the problem of excessive reference to database table metadata. Furthermore, the standardized tracing algorithm is simple, clear, easy to implement, and easy to maintain.
[0100] Figure 5 This is a block diagram illustrating a data lineage analysis apparatus according to an exemplary embodiment. (Refer to...) Figure 5 The device 500 includes:
[0101] The receiving module 510 is used to obtain a lineage resolution request, wherein the lineage resolution request contains at least one structured query language (SQL) statement;
[0102] The annotation module 520 is used to add annotations to the SQL statement in the lineage resolution request and generate a standardized tracing syntax tree based on the SQL statement;
[0103] The tracing module 530 is used to generate a corresponding data lineage model based on each subquery in the standardized tracing syntax tree.
[0104] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.
[0105] Figure 6 This is a block diagram illustrating an apparatus 800 according to an exemplary embodiment. For example, apparatus 800 may be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, etc.
[0106] Reference Figure 6 The device 800 may include one or more of the following components: a processing component 802, a memory 804, a power component 806, a multimedia component 808, an audio component 810, an input / output (I / O) interface 812, a sensor component 814, and a communication component 816.
[0107] Processing component 802 typically controls the overall operation of device 800, such as operations associated with display, telephone calls, data communication, camera operation, and recording. Processing component 802 may include one or more processors 820 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processing component 802 may include one or more modules to facilitate interaction between processing component 802 and other components. For example, processing component 802 may include a multimedia module to facilitate interaction between multimedia component 808 and processing component 802.
[0108] Memory 804 is configured to store various types of data to support the operation of device 800. Examples of this data include instructions for any application or method operating on device 800, contact data, phonebook data, messages, pictures, videos, etc. Memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.
[0109] Power supply component 806 provides power to various components of device 800. Power supply component 806 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power to device 800.
[0110] Multimedia component 808 includes a screen that provides an output interface between the device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 808 includes a front-facing camera and / or a rear-facing camera. When the device 800 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.
[0111] Audio component 810 is configured to output and / or input audio signals. For example, audio component 810 includes a microphone (MIC) configured to receive external audio signals when device 800 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 804 or transmitted via communication component 816. In some embodiments, audio component 810 also includes a speaker for outputting audio signals.
[0112] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.
[0113] Sensor assembly 814 includes one or more sensors for providing status assessments of various aspects of device 800. For example, sensor assembly 814 may detect the on / off state of device 800, the relative positioning of components such as the display and keypad of device 800, changes in the position of device 800 or a component of device 800, the presence or absence of user contact with device 800, the orientation or acceleration / deceleration of device 800, and temperature changes of device 800. Sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 814 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 814 may also include an accelerometer, a gyroscope, a magnetometer, a pressure sensor, or a temperature sensor.
[0114] Communication component 816 is configured to facilitate wired or wireless communication between device 800 and other devices. Device 800 can access wireless networks based on communication standards, such as WiFi, carrier networks (such as 2G, 3G, 4G, or 5G), or combinations thereof. In one exemplary embodiment, communication component 816 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 816 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.
[0115] In an exemplary embodiment, the apparatus 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.
[0116] In an exemplary embodiment, a storage medium including instructions is also provided, such as a memory 804 including instructions, which can be executed by a processor 820 of the device 800 to perform the above method. Optionally, the storage medium may be a non-transitory computer-readable storage medium, such as a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device.
[0117] Figure 7This is a block diagram illustrating an apparatus 900 for... according to an exemplary embodiment. For example, apparatus 900 may be provided as a server. (Refer to...) Figure 7 The apparatus 900 includes a processing component 922, which further includes one or more processors, and memory resources represented by memory 932 for storing instructions, such as application programs, that can be executed by the processing component 922. The application programs stored in memory 932 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 922 is configured to execute instructions to perform the methods described above.
[0118] The device 900 may also include a power supply component 926 configured to perform power management of the device 900, a wired or wireless network interface 950 configured to connect the device 900 to a network, and an input / output (I / O) interface 958. The device 900 can operate on an operating system stored in memory 932, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, or similar.
[0119] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.
[0120] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.
Claims
1. A data lineage analysis method, characterized in that, The method comprises: obtaining a blood relationship analysis request, wherein the blood relationship analysis request contains at least one structured query language (SQL) statement; adding annotations to the SQL statement in the blood relationship analysis request, and generating a standardized traceability syntax tree according to the SQL statement; generating a corresponding data blood relationship model according to each subquery in the standardized traceability syntax tree; the step of adding annotations to the SQL statement in the blood relationship analysis request specifically comprises: injecting a corresponding field alias at a source field position; injecting a reference alias at the source field position; injecting a field name at a group by or order by position; the step further comprises: injecting a main sentence of the SQL statement at a with clause; injecting asset metadata.
2. The method of claim 1, wherein, The step of generating a corresponding data blood relationship model according to each subquery in the standardized traceability syntax tree specifically comprises: converting the standardized traceability syntax tree into a query matrix, wherein each row of the query matrix is a subquery; performing traceability according to the subqueries in each row of the query matrix and merging the same traceability results to obtain a target field model and a source field model and generate the data blood relationship model.
3. The method of claim 1, wherein, The method further comprises: if a subquery in the SQL statement is not associated with other statements in the SQL statement, performing traceability on the subquery alone to generate the data blood relationship model.
4. A data lineage resolution apparatus, comprising: The method comprises: a receiving module configured to obtain a blood relationship analysis request, wherein the blood relationship analysis request contains at least one structured query language (SQL) statement; an annotation module configured to add annotations to the SQL statement in the blood relationship analysis request, and generate a standardized traceability syntax tree according to the SQL statement; a traceability module configured to generate a corresponding data blood relationship model according to each subquery in the standardized traceability syntax tree; the step of adding annotations to the SQL statement in the blood relationship analysis request specifically comprises: injecting a corresponding field alias at a source field position; injecting a reference alias at the source field position; injecting a field name at a group by or order by position; the step further comprises: injecting a main sentence of the SQL statement at a with clause; injecting asset metadata.
5. An electronic device, comprising: The method comprises: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement the method of any one of claims 1 to 3.
6. A computer-readable storage medium, when the instructions in the storage medium are executed by a processor of an electronic device, enabling the electronic device to perform the method of any one of claims 1 to 3.
Citation Information
Patent Citations
Column operator blood relationship construction method, server and computer readable storage medium
CN115757525A