SQL blood relationship obtaining method and device, storage medium and electronic equipment
By using pre-trained statement recognition model to identify SQL blood ties, the problem of low parsing efficiency in the prior art is solved, and more efficient blood ties acquisition is achieved.
Patent Information
- Application Number
- CN202311616651.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-11-29
- Publication Date
- 2025-05-30
AI Technical Summary
In the prior art, SQL blood relationship analysis is inefficient, and it is necessary to manually adjust the parsing methods and recompile the parsing tools.
The model recognizes the blood relationship in the target SQL statement through pre-trained statement recognition model, and uses sample SQL statements and their blood relationship for model training and adjustment.
It improves the efficiency of identifying and obtaining blood relationships in SQL and reduces the steps of artificial adjustment and compilation.
Smart Images
Figure CN120067125A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of computers, and in particular, to a method, apparatus, storage medium, and electronic device for obtaining SQL lineage relationships. Background Art
[0002] In the prior art, to obtain the lineage relationship in an SQL statement, an SQL parsing tool is required. The means in the SQL parsing tool are usually parsing means summarized manually. If an update is needed, the parsing means need to be readjusted and the parsing tool needs to be recompiled, resulting in low efficiency in parsing the lineage relationship of SQL statements. Summary of the Invention
[0003] The present application provides a method, apparatus, storage medium, and electronic device for obtaining SQL lineage relationships to solve the technical problem of low efficiency in parsing SQL lineage relationships.
[0004] In a first aspect, the present application provides a method for obtaining an SQL lineage relationship, including: obtaining a target SQL statement, where the target SQL statement is an SQL statement for which the lineage relationship is to be obtained; inputting the target SQL statement into a statement recognition model, where the statement recognition model is used to recognize the lineage relationship of the SQL statement, and the statement recognition model is a model pre-trained using sample SQL statements and the lineage relationships of the sample SQL statements; and taking the lineage relationship output by the statement recognition model as the lineage relationship of the target SQL statement.
[0005] In a second aspect, the present application provides an apparatus for obtaining an SQL lineage relationship, including: an obtaining module for obtaining a target SQL statement, where the target SQL statement is an SQL statement for which the lineage relationship is to be obtained; an input module for inputting the target SQL statement into a statement recognition model, where the statement recognition model is used to recognize the lineage relationship of the SQL statement, and the statement recognition model is a model pre-trained using sample SQL statements and the lineage relationships of the sample SQL statements; and a determination module for taking the lineage relationship output by the statement recognition model as the lineage relationship of the target SQL statement.
[0006] As an alternative example, the above-mentioned device further includes: an adjustment module, configured to, before inputting the target SQL statement into the statement recognition model, obtain a sample SQL statement and the expected lineage relationship of the sample SQL statement; input the sample SQL statement into the original recognition model; predict the lineage relationship of the sample SQL statement by the original recognition model; adjust the model parameters of the original recognition model according to the expected lineage relationship of the sample SQL statement and the predicted lineage relationship of the sample SQL statement, and obtain the statement recognition model after the adjustment is completed.
[0007] As an alternative example, the above-mentioned adjustment module includes: a determination unit, configured to, after obtaining the sample SQL statement, input the sample SQL statement into the parsing model, and parse the lineage relationship of the sample SQL statement by the parsing model; determine the lineage relationship parsed by the parsing model as the expected lineage relationship of the sample SQL statement.
[0008] As an alternative example, the above-mentioned determination unit includes: a processing subunit, configured to identify each word and each sentence in the sample SQL statement by the parsing model; in the case of identifying a preset keyword, determine the field after the preset keyword as the field to be recognized; in the case of identifying a special symbol in the field to be recognized, determine each sub-field obtained by splitting the field to be recognized by the special symbol as a database column field, and in the case of not identifying the special symbol in the field to be recognized, determine the field to be recognized as a database column field; search for the lineage relationship of the database column field from the lineage relationship table according to the database column field; determine the found lineage relationship as the expected lineage relationship of the sample SQL statement.
[0009] As an alternative example, the above-mentioned processing subunit is further configured to search for the database column field from the lineage relationship table; determine a string of column field - table field - library field starting with the database column field in the lineage relationship table as the lineage relationship of the database column field.
[0010] As an alternative example, the above-mentioned adjustment module includes: an adjustment unit, configured to, in the case where the expected lineage relationship of the input sample SQL statement is the same as the predicted lineage relationship of the sample SQL statement, keep the model parameters unchanged or keep the model parameters fine-tuned within a preset range; in the case where the expected lineage relationship of the input sample SQL statement is different from the predicted lineage relationship of the sample SQL statement, determine the adjustment range of the model parameters according to the degree of difference between the expected lineage relationship of the input sample SQL statement and the predicted lineage relationship of the sample SQL statement.
[0011] As an optional example, the above adjustment unit includes: a comparison subunit, configured to start from the first character of the expected blood relationship of the input sample SQL statement and the predicted blood relationship of the sample SQL statement, and sequentially compare each character of the expected blood relationship of the input sample SQL statement and the predicted blood relationship of the sample SQL statement; when the characters at the same position in the expected blood relationship of the input sample SQL statement and the predicted blood relationship of the sample SQL statement are different, increase the value of the difference degree.
[0012] In a third aspect, the present application provides an electronic device, including: at least one communication interface; at least one bus connected to the at least one communication interface; at least one processor connected to the at least one bus; at least one memory connected to the at least one bus, wherein the memory stores a computer program, and the processor is configured to implement the SQL blood relationship acquisition method described in any one of the above when executing the computer program.
[0013] In a fourth aspect, the present application further provides a computer storage medium storing computer-executable instructions, and the computer-executable instructions are used to execute the SQL blood relationship acquisition method described in any one of the above in the present application.
[0014] The above technical solution provided by the embodiments of the present application has the following advantages compared with the prior art: The solution provided by the embodiments of the present application can identify the SQL blood relationship in the target SQL statement through a pre-trained statement recognition model, thereby improving the recognition and acquisition efficiency of the SQL blood relationship. BRIEF DESCRIPTION OF THE DRAWINGS
[0015] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present invention, and are used together with the specification to explain the principles of the present invention.
[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, other drawings can also be obtained based on these drawings without creative efforts.
[0017] One or more embodiments are exemplarily illustrated by the pictures in the corresponding accompanying drawings. These exemplary illustrations do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings are represented as similar elements, unless otherwise stated, and the drawings in the drawings do not constitute a proportional limitation.
[0018] Figure 1Flowchart of a method for obtaining SQL lineage provided by an embodiment of the present application;
[0019] Figure 2 Statement recognition model diagram of a method for obtaining SQL lineage provided by an embodiment of the present application;
[0020] Figure 3 Flowchart of another method for obtaining SQL lineage provided by an embodiment of the present application;
[0021] Figure 4 Flowchart of yet another method for obtaining SQL lineage provided by an embodiment of the present application;
[0022] Figure 5 Flowchart of yet another method for obtaining SQL lineage provided by an embodiment of the present application;
[0023] Figure 6 Flowchart of a system for obtaining SQL lineage provided by an embodiment of the present application;
[0024] Figure 7 Structural schematic diagram of a device for obtaining SQL lineage provided by an embodiment of the present application;
[0025] Figure 8 Schematic diagram of an electronic device provided by an embodiment of the present application. Detailed implementation manners
[0026] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Apparently, the described embodiments are some but not all of the embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts shall fall within the scope of protection of the present application.
[0027] The following disclosure provides many different embodiments or examples for implementing different structures of the present invention. To simplify the disclosure of the present invention, the components and settings of specific examples are described below. Of course, they are only examples and are not intended to limit the present invention. In addition, the present invention may repeat reference numerals and / or letters in different examples. Such repetition is for the purpose of simplification and clarity, and does not in itself indicate the relationship between the various embodiments and / or settings discussed.
[0028] To solve the technical problem of low SQL lineage parsing efficiency in the prior art, the present application provides a method for obtaining SQL lineage, which can achieve the effect of improving the recognition and acquisition efficiency of SQL lineage.
[0029] Figure 1 This is a flowchart of a method for obtaining SQL lineage provided by an embodiment of the present application. As Figure 1 shown, the method for obtaining the above SQL lineage includes:
[0030] S102, obtain a target SQL statement, where the target SQL statement is the SQL statement for which the lineage is to be obtained;
[0031] S104, input the target SQL statement into a statement recognition model, where the statement recognition model is used to recognize the lineage of the SQL statement, and the statement recognition model is a model pre-trained using sample SQL statements and the lineages of the sample SQL statements;
[0032] S106, use the lineage output by the statement recognition model as the lineage of the target SQL statement.
[0033] A database statement (Structured Query Language, SQL) is a statement used to query data in a database. The statement can include data columns to be queried. In a database, a data column is located in a certain data table, and the data table is located in a certain database. The relationship is that the database contains the table, and the table contains the data column, the data table where the data column is located, and the database where the data table is located. For the data columns in an SQL statement, it is necessary to parse which table the data column is located in and which database the table is located in.
[0034] In the field of databases, the lineage of an SQL statement refers to the dependency relationship or source relationship between various data elements in the SQL statement. The lineage describes how data flows from one place to another in an SQL query or operation.
[0035] Specifically, the lineage can include the following aspects:
[0036] 1. Column Lineage: Describes the relationship between each column in the query and the original data. For example, if a certain column in a query is calculated from a certain column in Table A, then the lineage of this column will trace back to the corresponding column in Table A.
[0037] 2. Table Lineage: Describes the relationship between each table in the query. If a certain table in a query is derived from other tables, then the lineage of this table will establish a dependency relationship with the original table.
[0038] 3. Function and Operator Lineage: Describes the relationship between the functions and operators used in a query and the input data. If a function or operator is used in a query to transform or calculate data, then the lineage of this function or operator will be traced back to the input data.
[0039] Lineage is very important for data management and data analysis. It can help understand the source and changes of data during query and operation processes, trace the transformation path of data, ensure data accuracy and consistency, and support requirements in aspects such as data lineage tracing, data quality inspection, and data compliance. In the fields of big data and data governance, the management and tracing of lineage are key tasks. The lineage in this application at least includes the lineage of database column fields - table fields - library fields, and may also include at least one of the above lineages.
[0040] The statement recognition model in this embodiment can be a neural network model. The neural network model can be pre-trained, and the pre-trained neural network model is responsible for recognizing SQL statements and obtaining the lineage of SQL statements.
[0041] If you want to recognize the lineage of a target SQL statement, input the target SQL statement into the neural network model. When inputting, it can be input in a format, such as inputting the target SQL statement line by line, or filling the target SQL statement into a file and inputting the file into the statement recognition model. After the statement recognition model recognizes the target SQL statement, it outputs the recognition result. The recognition result can also be output in a format. For example, for each data column in the target SQL statement, the table name, table alias, library name, and library alias to which the data column belongs can be output in sequence. If data migration is to be performed, the source table name, source table alias, destination table name, destination table alias, source library name, source library alias, destination library name, destination library alias, etc. can be output, with one name per line. Each lineage relationship is enclosed in a pair of curly braces to distinguish the lineage relationships of different data columns. The lineage relationships are output in the order of the data columns in the SQL statement.
[0042] Figure 2 is a schematic diagram of the statement recognition model in this embodiment. The statement recognition model includes multiple convolutional layers and multiple fully connected layers. The target SQL statement passes through each layer in sequence to obtain the lineage relationship.
[0043] The solution provided in the embodiment of this application can recognize the SQL lineage relationship in the target SQL statement through a pre-trained statement recognition model, thereby improving the recognition and acquisition efficiency of the SQL lineage relationship.
[0044] As an optional example, such as Figure 3As shown, before inputting the target SQL statement into the statement recognition model, the above method further includes:
[0045] S302. Obtain a sample SQL statement and the expected lineage relationship of the sample SQL statement;
[0046] S304. Input the sample SQL statement into the original recognition model;
[0047] S306. Have the original recognition model predict the lineage relationship of the sample SQL statement;
[0048] S308. Adjust the model parameters of the original recognition model according to the expected lineage relationship of the sample SQL statement and the predicted lineage relationship of the sample SQL statement, and obtain the statement recognition model after the adjustment is completed.
[0049] In this embodiment, the statement recognition model is a pre-trained neural network model. During pre-training, sample SQL statements are used, and the lineage relationships of the sample SQL statements are known. By inputting the sample SQL statements, the original recognition model is allowed to predict the lineage relationships based on the sample SQL statements, and the input expected lineage relationships and the predicted lineage relationships are compared, and the model parameters of the original recognition model are adjusted according to the comparison results.
[0050] As an optional example, as Figure 4 shown, obtaining a sample SQL statement and the lineage relationship of the sample SQL statement includes:
[0051] S402. After obtaining the sample SQL statement, input the sample SQL statement into the parsing model, and have the parsing model parse the lineage relationship of the sample SQL statement;
[0052] S404. Determine the lineage relationship parsed by the parsing model as the expected lineage relationship of the sample SQL statement.
[0053] In this embodiment, the blood relationship of the sample SQL statement can be manually identified or parsed through a parsing model. The parsing model is a language model that can identify the expected blood relationship of the SQL statement. The parsing model is pre-configured with a parsing strategy and a parsing format, that is, the format of the SQL statement is identified, and a parsing strategy corresponding to the format is obtained, that is, the characters at which positions of the SQL statement are parsed to obtain the parsing result. Since the parsing model needs to be pre-configured, and different parsing models need to be configured according to different formats of the SQL statement, which is relatively time-consuming, therefore, the expected blood relationship of the sample SQL statement is identified through the parsing model, and the sample SQL statement and the identified expected blood relationship are used to train the statement recognition model, and the trained statement recognition model can be used to parse the blood relationship of the target SQL statement, thereby improving the efficiency of parsing the blood relationship.
[0054] As an alternative example, Figure 5 As shown, after obtaining the sample SQL statement, the sample SQL statement is input into the parsing model, and the sample SQL statement is parsed by the parsing model to obtain the following lineage relationships:
[0055] S502, identifying each word and each sentence in the sample SQL statement by the parsing model;
[0056] S504, when a preset keyword is identified, determining the field after the preset keyword as a field to be identified;
[0057] S506, when a special symbol is recognized in the field to be recognized, each of the multiple subfields obtained by dividing the field to be recognized by the special symbol is determined as a database column field; when the special symbol is not recognized in the field to be recognized, the field to be recognized is determined as a database column field;
[0058] S508, searching the blood relationship of the database column field in the blood relationship table according to the database column field;
[0059] S510: Determine the found blood relationship as the expected blood relationship of the sample SQL statement.
[0060] The above-mentioned preset keywords can be words with a symbolic function determined in advance from SQL statements, such as SELECT. In SQL statements, SELECT is generally followed by a database column field. Therefore, by searching SELECT, the field to be identified after the word can be found.
[0061] In this embodiment, for the parsing model, each word and each sentence of the sample SQL statement can be recognized. If a keyword or a key sentence is recognized, the field after the keyword or the key sentence is obtained, and this field is used as the field to be recognized. The field to be recognized may consist of one database column field or multiple database column fields. Therefore, it can be recognized whether the field to be recognized contains special symbols, and the special symbols are used to separate different database column fields. Therefore, by recognizing the special symbols, each database column field in the field to be recognized can be recognized. For each database column field, the blood relationship corresponding to the column field is searched in the blood relationship table. Alternatively, in this embodiment, the table can be traced back according to the first letter or the first word of the column field, and the library can be traced back through the first letter or the first character of the table, so as to obtain the blood relationship.
[0062] As an optional example, searching for the blood relationship of a database column field from the blood relationship table according to the database column field includes: searching for the database column field from the blood relationship table; and determining a string of column field-table field-library field starting with the database column field in the blood relationship table as the blood relationship of the database column field.
[0063] In this embodiment, the blood relationship can be stored in the blood relationship table in the form of database column field-table field-library field. After the database column field is obtained, the blood relationship table can be queried to obtain all possible cases of the blood relationship of the database column field.
[0064] As an optional example, adjusting the model parameters of the statement recognition model according to the blood relationship of the input sample SQL statement and the predicted blood relationship of the sample SQL statement includes: when the expected blood relationship of the input sample SQL statement is the same as the predicted blood relationship of the sample SQL statement, keeping the model parameters unchanged or slightly adjusting the model parameters within a preset range; when the expected blood relationship of the input sample SQL statement is different from the predicted blood relationship of the sample SQL statement, determining the adjustment range of the model parameters according to the degree of difference between the expected blood relationship of the input sample SQL statement and the predicted blood relationship of the sample SQL statement.
[0065] In this embodiment, when training a statement recognition model using a sample SQL statement and the expected lineage relationship of the sample SQL statement, the statement recognition model can recognize the sample SQL statement, predict the lineage relationship of the sample SQL statement, compare the predicted lineage relationship with the input expected lineage relationship. If the two are the same, it indicates that the prediction result of the statement recognition model is accurate. At this time, the model parameters can be left unchanged or fine-tuned. If the predicted lineage relationship is different from the input expected lineage relationship, the amplitude of adjusting the model parameters is determined according to the level of the difference between the two.
[0066] As an optional example, determining the difference between the lineage relationship of the input sample SQL statement and the predicted lineage relationship of the sample SQL statement includes: starting from the first character of the expected lineage relationship of the input sample SQL statement and the predicted lineage relationship of the sample SQL statement, comparing each character of the expected lineage relationship of the input sample SQL statement and the predicted lineage relationship of the sample SQL statement in sequence; when the characters at the same position in the expected lineage relationship of the input sample SQL statement and the predicted lineage relationship of the sample SQL statement are different, increase the value of the difference.
[0067] In this embodiment, in the process of determining the difference between the input expected lineage relationship and the predicted lineage relationship, the input expected lineage relationship and the predicted lineage relationship can be compared. The comparison dimension can be multi-dimensional. It can be compared character by character, word by word, or according to the dimension from bottom to top of the lineage relationship to determine the difference. For example, when comparing character by character, if the characters at the same position are different, increase the value of the difference. For example, when comparing according to the dimension from bottom to top, first compare the database column fields, then the table fields, and then the database fields. If there are different fields, increase the value of the difference.
[0068] Figure 6 It is a schematic diagram of the system in this embodiment. The input SQL statement is a sample SQL statement or a target SQL statement. The SLQ statement is parsed by a parsing model to obtain the expected lineage relationship. The parsing model outputs the expected lineage relationship and asynchronously sends the expected lineage relationship and the SQL statement to the statement recognition model. The statement recognition model recognizes the SQL statement, predicts the lineage relationship, compares the predicted lineage relationship with the received expected lineage relationship, and can adjust its own model parameters according to the comparison result, or send a feedback message to the parsing model.
[0069] An example is given for illustration. There are two libraries: d1 and d2. In d1, there is a table d1.product, and in d2, there is a table d2.order. The columns in the tables are as follows: d1.product: id, name, model, product_date, a total of 4 data columns; d2.order: id, name, model, order_time, user_name, address, a total of 6 data columns.
[0070] If the input sample SQL statement is:
[0071] SELECT o.id,o.name order_name,p.name product_name,p.model,o.order_time,o.user_name,o.address FROM d2.order o LEFT JOIN d1.product p
[0072] Then, first use the parsing model to parse the expected lineage. The parsing model parses the above SQL statement word by word. When it parses SELECT, it determines the string from SELECT to FROM as the fields to be recognized. Since there is a special symbol "," in the fields to be recognized, it is split by "," to obtain multiple sub-fields, such as "o.id", "o.name", "order_name", etc. Taking a sub-field "order_name" as an example, parse the expected lineage of this sub-field. Since the "order" in "order_name" belongs to the table d2.order in library 2, order_name is derived from the o.name column, the o.name column is derived from the o table, and the o table is an alias of the d2.order table, and the order table is derived from the d2 library. Therefore, the lineage "order_name->o.name->o->d2.order-d2" is obtained.
[0073] Input the SQL statement and the expected lineage into the statement recognition model, and the statement recognition model gives the recognition result:
[0074] {
[0075] "sourceCatalog":null,
[0076] "sourceDatabase":"d2",
[0077] "sourceTable":"order",
[0078] "sourceTableAlias":"o",
[0079] "sourceColumn": "id",
[0080] "targetCatalog": null,
[0081] "targetDatabase": null,
[0082] "targetTable": "RS-1",
[0083] "targetTableAlias": null,
[0084] "targetColumn": "id"
[0085] }
[0086] Compare the recognition result with the expected blood relationship of the input, so as to adjust the model parameters of the statement recognition model. When comparing, it is possible to compare whether the words at the corresponding library-table-column positions are the same. If they are not the same, increase the value of the difference degree.
[0087] Figure 7 This is a schematic structural diagram of an apparatus for obtaining SQL blood relationship provided by an embodiment of the present application. As Figure 7 shown, the above-mentioned apparatus for obtaining SQL blood relationship includes:
[0088] An obtaining module 702, configured to obtain a target SQL statement, where the target SQL statement is an SQL statement for which the blood relationship is to be obtained;
[0089] An input module 704, configured to input the target SQL statement into a statement recognition model, where the statement recognition model is used to recognize the blood relationship of the SQL statement, and the statement recognition model is a model pre-trained using sample SQL statements and the blood relationships of the sample SQL statements;
[0090] A determination module 706, configured to use the blood relationship output by the statement recognition model as the blood relationship of the target SQL statement.
[0091] A database statement (Structured Query Language, SQL) is a statement used to query data in a database. The statement may include data columns to be queried. In a database, a data column is located in a certain data table, and the data table is located in a certain database. The relationship is that the database contains the table, and the table contains the data column. For the data column in the SQL statement, it is necessary to parse which table the data column is located in and which database the table is located in.
[0092] In the field of databases, the lineage of SQL statements refers to the dependency or source relationship among various data elements in SQL statements. Lineage describes how data flows from one place to another in SQL queries or operations.
[0093] Specifically, the lineage can include the following aspects:
[0094] 1. Column Lineage: Describes the relationship between each column in a query and the original data. For example, if a column in a query is calculated from a column in Table A, then the lineage of this column will trace back to the corresponding column in Table A.
[0095] 2. Table Lineage: Describes the relationship between each table in a query. If a table in a query is derived from other tables, then the lineage of this table will establish a dependency relationship with the original tables.
[0096] 3. Function and Operator Lineage: Describes the relationship between the functions and operators used in a query and the input data. If a function or operator is used in a query to transform or calculate data, then the lineage of this function or operator will trace back to the input data.
[0097] Lineage is very important for data management and data analysis. It can help understand the source and changes of data during query and operation processes, trace the transformation path of data, ensure data accuracy and consistency, and support requirements in aspects such as data lineage tracing, data quality inspection, and data compliance. In the fields of big data and data governance, the management and tracing of lineage are key tasks. The lineage in this application at least includes the lineage of database column fields - table fields - library fields, and may also include at least one of the above lineages.
[0098] The statement recognition model in this embodiment can be a neural network model. The neural network model can be pre-trained, and the pre-trained neural network model is responsible for recognizing SQL statements and obtaining the lineage of SQL statements.
[0099] If you want to identify the lineage of the target SQL statement, input the target SQL statement into the neural network model. When inputting, you can input it in a format, such as inputting the target SQL statement line by line, or filling the target SQL statement into a file and inputting the file into the statement recognition model. After the statement recognition model recognizes the target SQL statement, it outputs the recognition result. The recognition result can also be output in a format. For example, for each data column in the target SQL statement, the table name, table alias, database name, and database alias to which the data column belongs can be output in sequence. If data migration is to be performed, the source table name, source table alias, destination table name, destination table alias, source database name, source database alias, destination database name, destination database alias, etc. can be output one name per line. Each lineage relationship is enclosed in a pair of curly braces to distinguish the lineage relationships of different data columns. The lineage relationships are output in the order of the data columns in the SQL statement.
[0100] Figure 2 It is a schematic diagram of the model of the statement recognition model in this embodiment. The statement recognition model includes multiple convolutional layers and multiple fully connected layers. The target SQL statement passes through each layer in sequence to obtain the lineage relationship.
[0101] The solution provided in the embodiment of the present application can identify the SQL lineage relationship in the target SQL statement through a pre-trained statement recognition model, thereby improving the recognition and acquisition efficiency of the SQL lineage relationship.
[0102] For other examples of this embodiment, please refer to the above examples and will not be elaborated here.
[0103] Such as Figure 8 As shown, the embodiment of the present application provides an electronic device, including a processor 111, a communication interface 112, a memory 113, and a communication bus 114. Among them, the processor 111, the communication interface 112, and the memory 113 complete communication with each other through the communication bus 114.
[0104] The memory 113 is used to store a computer program.
[0105] In an embodiment of the present application, when the processor 111 executes the program stored on the memory 113, it implements the method for obtaining the SQL lineage relationship provided in any one of the foregoing method embodiments.
[0106] The embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the steps of the method for obtaining the SQL lineage relationship provided in any one of the foregoing method embodiments.
[0107] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0108] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the essence of the above technical solution or the part that contributes to the relevant technology can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.
[0109] It should be understood that the terms used herein are for the purpose of describing specific example embodiments only and are not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms "a", "an", and "the" as used herein may also include the plural forms. The terms "comprising", "including", "containing", and "having" are inclusive and thus specify the presence of the stated features, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, steps, operations, elements, components, and / or combinations thereof. The method steps, processes, and operations described herein are not to be construed as necessarily requiring them to be executed in the particular order described or illustrated, unless the order of execution is explicitly stated. It should also be understood that additional or alternative steps may be used.
[0110] The above description is only the specific implementation manners of the present invention, enabling those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to these embodiments shown herein, but will be accorded the widest scope consistent with the principles and novel features claimed herein.
Claims
1. A method for obtaining SQL lineage, characterized in that, it includes: Obtain a target SQL statement, where the target SQL statement is the SQL statement for which the lineage is to be obtained; Input the target SQL statement into a statement recognition model, where the statement recognition model is used to recognize the lineage of the SQL statement, and the statement recognition model is a model pre-trained using sample SQL statements and the lineage of the sample SQL statements; Use the lineage output by the statement recognition model as the lineage of the target SQL statement.
2. The method according to claim 1, characterized in that, before inputting the target SQL statement into the statement recognition model, the method further includes: Obtain sample SQL statements and the expected lineage of the sample SQL statements; Input the sample SQL statements into the original recognition model; Predict the lineage of the sample SQL statements by the original recognition model; Adjust the model parameters of the original recognition model according to the expected lineage of the sample SQL statements and the predicted lineage of the sample SQL statements, and obtain the statement recognition model after the adjustment is completed.
3. The method according to claim 2, characterized in that, the obtaining of sample SQL statements and the expected lineage of the sample SQL statements includes: After obtaining the sample SQL statements, input the sample SQL statements into a parsing model, and the parsing model parses the lineage of the sample SQL statements; Determine the lineage parsed by the parsing model as the expected lineage of the sample SQL statements.
4. The method according to claim 3, characterized in that, after obtaining the sample SQL statements, input the sample SQL statements into a parsing model, and the parsing model parses the lineage of the sample SQL statements includes: The parsing model recognizes each word and each sentence in the sample SQL statements; When a preset keyword is recognized, determine the field after the preset keyword as the field to be recognized; When a special symbol is recognized in the field to be recognized, determine each sub-field obtained by splitting the field to be recognized by the special symbol as a database column field, and when the special symbol is not recognized in the field to be recognized, determine the field to be recognized as a database column field; Find the lineage of the database column field from the lineage table according to the database column field; Determine the found lineage as the expected lineage of the sample SQL statements.
5. The method according to claim 4, characterized in that, the finding of the lineage of the database column field from the lineage table according to the database column field includes: finding the database column field from the lineage table; Determine a string of column field - table field - library field starting with the database column field in the lineage table as the lineage of the database column field.
6. The method according to claim 2, It is characterized in that Adjusting the model parameters of the original recognition model according to the expected blood relationship of the sample SQL statement and the predicted blood relationship of the sample SQL statement, and obtaining the statement recognition model after the adjustment includes: When the expected blood relationship of the input sample SQL statement is the same as the predicted blood relationship of the sample SQL statement, keeping the model parameters unchanged or fine-tuning the model parameters within a preset range; When the expected blood relationship of the input sample SQL statement is different from the predicted blood relationship of the sample SQL statement, determining the adjustment range of the model parameters according to the degree of difference between the expected blood relationship of the input sample SQL statement and the predicted blood relationship of the sample SQL statement.
7. The method according to claim 6, It is characterized in that Determining the degree of difference between the expected blood relationship of the input sample SQL statement and the predicted blood relationship of the sample SQL statement includes: Starting from the first character of the expected blood relationship of the input sample SQL statement and the predicted blood relationship of the sample SQL statement, comparing each character of the expected blood relationship of the input sample SQL statement and the predicted blood relationship of the sample SQL statement in turn; When the characters at the same position in the expected blood relationship of the input sample SQL statement and the predicted blood relationship of the sample SQL statement are different, increasing the value of the degree of difference.
8. An apparatus for obtaining SQL blood relationship, It is characterized in that Including: An acquisition module, configured to acquire a target SQL statement, where the target SQL statement is an SQL statement for which the blood relationship is to be acquired; An input module, configured to input the target SQL statement into a statement recognition model, where the statement recognition model is used to recognize the blood relationship of the SQL statement, and the statement recognition model is a model pre-trained using a sample SQL statement and the blood relationship of the sample SQL statement; A determination module, configured to use the blood relationship output by the statement recognition model as the blood relationship of the target SQL statement.
9. An electronic device, It is characterized in that Including: At least one communication interface; At least one bus connected to the at least one communication interface; At least one processor connected to the at least one bus; At least one memory connected to the at least one bus, where a computer program is stored in the memory, and when the processor executes the computer program, the method described in any one of claims 1 to 7 is implemented.
10. A computer-readable storage medium, the storage medium stores computer-executable instructions, and the computer-executable instructions are used to execute the method described in any one of claims 1 to 7 of the present application.