Data lake query permission point extraction method and device, equipment and medium

Abstract

The present disclosure relates to the technical field of data security, and discloses a method and device for extracting data lake query permission points, equipment and a storage medium. The method comprises: obtaining an abstract syntax tree structure corresponding to a query statement; extracting a scan operator, a column selection operator and a filter condition operator in the abstract syntax tree structure; determining the library table permission points to be extracted based on the scan operator; determining the column permission points and the row permission points to be extracted based on the column selection operator and the filter condition operator; combining the library table permission points, the column permission points and the row permission points to determine the permission point query range. Through the technical solution of the present disclosure, the row and column information of the bottom layer table to be queried at this time can be accurately obtained, and the permission point query range of the query at this time is determined in combination with the extracted permission points, so as to perform fine-grained permission control on the extracted library, table, row and column.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of data security, in particular to an extraction method and device of query permission points based on a data lake, equipment and medium. BACKGROUND

[0002] In the data lake scenario, data is usually structured and organized in the form of a table according to certain rules. A table is a relationship, and a table is essentially a collection of rows and columns. The permission control for the data lake can be further divided into database permissions, table permissions, row permissions, and column permissions, and the like.

[0003] At present, the data in the table is usually analyzed and processed using the structured query language (SQL). However, the extraction of permission points for the data lake is based on fixed rules, and the SQL statements written by users are diverse. Therefore, it is difficult to match all types of SQL statements based on fixed rules, and it is difficult to extract the database / table / row / column permissions in detail and accurately, and it is difficult to perform fine-grained permission control. SUMMARY

[0004] Therefore, the embodiments of the present disclosure provide an extraction method and device of query permission points based on a data lake, equipment and medium, to solve the problem of fine-grained permission control for the data lake.

[0005] In a first aspect, the embodiments of the present disclosure provide an extraction method of query permission points based on a data lake, comprising: obtaining an abstract syntax tree structure corresponding to a query statement; extracting a scan operator, a column selection operator, and a filter condition operator in the abstract syntax tree structure; determining a database table permission point to be extracted based on the scan operator; determining a column permission point and a row permission point to be extracted based on the column selection operator and the filter condition operator; and combining the database table permission point, the column permission point, and the row permission point to determine a permission point query range.

[0006] The extraction method of query permission points based on a data lake provided by the embodiments of the present disclosure extracts the scan operator, the column selection operator, and the filter condition operator from the abstract syntax tree structure to obtain the database table permission point, the column permission point, and the row permission point to be extracted by the query statement. Therefore, only the scan operator, the column selection operator, and the filter condition operator need to be extracted from the abstract syntax tree structure, and are not limited to fixed rules, so that fine-grained permission extraction can be performed on the data lake, the row and column information of the bottom table to be queried at this time can be accurately obtained, and the permission point query range of the query at this time can be determined in combination with the extracted permission points, so that fine-grained permission control can be performed on the extracted database, table, row, and column.

[0007] In a second aspect, the embodiments of the present disclosure provide an extraction device based on a data lake query permission point, the device comprising: an acquisition module configured to acquire an abstract syntax tree structure corresponding to a query statement; an operator extraction module configured to extract a scan operator, a column selection operator, and a filter condition operator in the abstract syntax tree structure; a first permission point determination module configured to determine a library table permission point corresponding to the query statement based on the scan operator; a second permission point determination module configured to determine a column permission point and a row permission point corresponding to the query statement based on the column selection operator and the filter condition operator; and a query range determination module configured to combine the library table permission point, the column permission point, and the row permission point to determine a permission point query range.

[0008] In a third aspect, the embodiments of the present disclosure provide a computer device, comprising a memory and a processor, the memory and the processor being in communication connection with each other, the memory storing computer instructions, and the processor executing the computer instructions to perform the extraction method of the data lake query permission point based on the first aspect or any of the corresponding embodiments thereof.

[0009] In a fourth aspect, the embodiments of the present disclosure provide a computer readable storage medium, characterized in that the computer readable storage medium stores computer instructions, and the computer instructions are used to make a computer execute the extraction method of the data lake query permission point based on the first aspect or any of the corresponding embodiments thereof. BRIEF DESCRIPTION OF DRAWINGS

[0010] In order to more clearly illustrate the specific embodiments of the present disclosure or the technical solutions in the prior art, the drawings needed in the specific embodiments or prior art description will be briefly introduced as follows. Obviously, the drawings in the following description are some embodiments of the present disclosure, and those skilled in the art can also obtain other drawings according to these drawings without creative labor.

[0011] Figure 1 is a flowchart of the extraction method of the data lake query permission point according to some embodiments of the present disclosure;

[0012] Figure 2 is a flowchart of another extraction method of the data lake query permission point according to some embodiments of the present disclosure;

[0013] Figure 3 is a schematic diagram of an abstract syntax tree structure according to some embodiments of the present disclosure;

[0014] Figure 4 is a flowchart of still another extraction method of the data lake query permission point according to some embodiments of the present disclosure;

[0015] Figure 5is a structural block diagram of an extraction device for query permission points based on a data lake according to an embodiment of the present disclosure.

[0016] Figure 6 is a hardware structure schematic diagram of a computer device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0017] To make the objectives, technical solutions, and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be described clearly and completely below with reference to the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are some but not all of the embodiments of the present disclosure. Based on the embodiments in the present disclosure, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present disclosure.

[0018] In related technologies, the extraction of permission points is usually based on fixed rules. However, the SQL statements written by users are various, and it is difficult to match all types of SQL statements according to the fixed extraction rules. In extreme cases, the extraction of permission points fails, and the system falls back to extracting only the coarse-grained library table level permissions. However, the extraction of coarse-grained permissions is difficult to support the requirements of fine-grained permission control.

[0019] Based on this, the technical solutions of the present disclosure extract the permission points in the SQL query statement based on the data blood relationship by analyzing the SQL query statement of the user, obtain the row and column information of the bottom layer data table that the user really needs to query, and extract the permission points for the row and column information of the bottom layer data table, so as to realize fine-grained permission control.

[0020] According to the embodiments of the present disclosure, a method for extracting permission points based on a data lake query is provided. It should be noted that the steps shown in the flowchart can be executed in a computer system such as a group of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order from that shown here.

[0021] In the present embodiment, a method for extracting permission points based on a data lake query is provided, which can be used in computer devices such as computers, servers, etc. Figure 1 is a flowchart of a method for extracting permission points based on a data lake query according to an embodiment of the present disclosure, as shown in Figure 1 The flowchart includes the following steps:

[0022] In step S101, the abstract syntax tree structure corresponding to the query statement is obtained.

[0023] In a data lake scenario, in order to facilitate data analysis and processing, data is usually structured and organized in the form of tables according to certain rules. A data table is a relation, and a data table is essentially a collection of rows and columns. Each row in the data table represents a record, and each column represents a field of the record.

[0024] The abstract syntax tree structure is generated based on a structured query language (SQL). When data in the data lake needs to be queried, a user can write a corresponding query statement based on the SQL according to the current query requirement, and generate a corresponding abstract syntax tree structure by parsing the query statement.

[0025] In step S102, a scan operator, a column selection operator, and a filter condition operator in the abstract syntax tree structure are extracted.

[0026] All table structures in the SQL query statement are mapped to a scan operator structure in the abstract syntax tree structure, and the computer device can extract the scan operator from the abstract syntax tree structure.

[0027] The column selection operator is the outermost operator of the abstract syntax tree structure, and is used to determine all column information. The computer device can identify the outermost operator of the abstract syntax tree structure to extract the column selection operator from the abstract syntax tree structure.

[0028] The filter condition operator is used to determine the filter condition of the row and column information, and is represented by keywords such as Join and Filter. The computer device can identify the filter keyword corresponding to the filter condition operator in the abstract syntax tree structure, and extract the filter condition operator of the current query from the abstract syntax tree structure.

[0029] In step S103, based on the scan operator, a library table permission point to be extracted is determined.

[0030] The library table permission point represents a database in which a data table to be queried is located, and is used to represent a permission point of a library table dimension. Since all data table structures in the query statement are mapped to a scan operator TableScan in the abstract syntax tree structure, the corresponding scan operator TableScan is not the same if the same data table is queried multiple times. The computer device can obtain all data table information required for the current query by identifying all scan operators TableScan in the abstract syntax tree structure, and determine the library table permission point to be authenticated.

[0031] In step S104, based on the column selection operator and the filter condition operator, a column permission point and a row permission point to be extracted are determined.

[0032] The column permission point is the column-level permission point information required to be extracted, and the row permission point is the row-level permission point information required to be extracted.

[0033] Specifically, all column information held by the column selection operator is extracted from the abstract syntax tree structure according to the column selection operator. According to the data blood relationship, the column information of the underlying data table corresponding to the column selection operator is obtained one by one. The column information of these underlying data tables is the column-level permission point information required to be extracted.

[0034] According to the filter condition operator, all filter conditions actually used in the current query are determined. The filter conditions specifically include column and column equality, column and value equality, and value and value equality. For column and column equality, column information when column and column are equal can be further extracted according to the data blood relationship to further determine column-level permission point information. For column and value equality, row information corresponding to the column information can be further extracted according to the data blood relationship to further determine row-level permission point information. For value and value equality, no permission check is involved, and no permission point extraction is performed at this time.

[0035] In step S105, the library table permission point, the column permission point, and the row permission point are combined to determine the permission point query range.

[0036] Different data tables have different identification information, that is, the data table identification information is different. The data table identification information is determined based on the table name of the data table and the unique identification corresponding to the scan operator. The determined library table permission point, row permission point, and column permission point are matched and combined according to the data table identification information, so that the permission point query range required to be checked for each data table can be obtained.

[0037] The extraction method of the permission point based on the data lake query provided in this embodiment extracts the scan operator, the column selection operator, and the filter condition operator from the abstract syntax tree structure to obtain the library table permission point, the column permission point, and the row permission point required to be extracted by the query statement. Therefore, only the scan operator, the column selection operator, and the filter condition operator need to be extracted from the abstract syntax tree structure, and are not limited to fixed rules. Thus, fine-grained permission extraction can be performed on the data lake, and the row and column information of the most underlying table to be queried at this time can be accurately obtained. Therefore, the permission point query range of the current query is determined in combination with the extracted permission point, so that fine-grained permission control can be performed on the extracted library, table, row, and column.

[0038] In this embodiment, an extraction method of a permission point based on a data lake query is provided, which can be used for computer devices such as computers, servers, and the like. Figure 2 The flowchart of the extraction method of the permission point based on the data lake query according to the embodiments of the present disclosure is shown in FIG. 1. Figure 2 The flowchart of the extraction method of the permission point based on the data lake query according to the embodiments of the present disclosure is shown in FIG. 1.

[0039] Step S201: Obtain the abstract syntax tree structure corresponding to the query statement. For detailed explanation, please refer to the relevant descriptions in the above embodiments; they will not be repeated here.

[0040] Step S202: Extract the scan operator, column selection operator, and filtering condition operator from the abstract syntax tree structure. For detailed explanations, please refer to the relevant descriptions in the above embodiments; they will not be repeated here.

[0041] Step S203: Based on the scanning operator, determine the permission points of the database tables to be extracted.

[0042] Specifically, step S203 above may include:

[0043] Step S2031: Extract the data table to be queried in the current session and the operator identifier corresponding to the scan operator from the abstract syntax tree structure based on the scan operator.

[0044] Extract all the scan operators TableScan from the abstract syntax tree structure, and extract the table name of the data table to be queried and the unique identifier corresponding to the scan operator TableScan from each scan operator TableScan. The unique identifier can be represented by an entity number.

[0045] Step S2032: Determine the data table identifier information according to the data table and operator identifier.

[0046] The table name and operator identifier are concatenated according to a preset rule to generate table identifier information for each data table. This preset rule is a pre-defined concatenation rule, such as concatenating according to "table name - operator identifier". Figure 3 For example, if all the data tables are db.table1, db.table2, and db.table3, and the operator identifier corresponding to each scan operator TableScan is different, the data table identifier information corresponding to each data table formed in the end will be db.table1-1, db.table2-2, and db.table3-3, respectively.

[0047] Since each scan operator TableScan has a different operator identifier, the resulting data table identifier information is also different, thus effectively distinguishing different operator queries.

[0048] Step S2033: Determine the database table permission points to be identified in this instance based on the data table identifier information.

[0049] Based on the table identifiers corresponding to all the TableScan scanning operators, the permission points at the database / table level required for this query can be determined. Figure 3As an example, if the determined all data table identification information is "db.table1-1, db.table2-2, db.table3-3", then the library table level permission point to be identified in this time is db.table1-1, db.table2-2, db.table3-3, and thus the library table level permission point information required by the SQL query statement is obtained.

[0050] In step S204, the column permission point and the row permission point to be extracted are determined based on the column selection operator and the filter condition operator.

[0051] Specifically, the above step S204 can include:

[0052] In step S2041, the column information of the data table is extracted from the abstract syntax tree structure based on the column selection operator, and the column selection operator is located at the outermost layer of the abstract syntax tree structure.

[0053] The column permission point only pays attention to the column information of the operator at the outermost layer of the abstract syntax tree structure, and thus after the column selection operator at the outermost layer of the abstract syntax tree structure is identified, the column information to be extracted by the column selection operator can be extracted. Figure 3 As an example, the column selection operator is select, and the column information to be extracted in this time is t1.col2, t1.col3 and t2.col3.

[0054] In step S2042, the column information is traversed according to the data blood relationship of the abstract syntax tree structure, and the column permission point corresponding to the column information is determined.

[0055] All column information is traversed, and the data table column corresponding to each column information is located in the operator layer corresponding to each column information according to the data blood relationship of the abstract syntax tree structure, and the data table column and the corresponding data table identification information are recorded. All data table columns and the corresponding data table identification information are combined, and the required column permission point is obtained.

[0056] In some optional embodiment modes, the above step S2042 can include:

[0057] In step a1, the column information is traversed according to the data blood relationship of the abstract syntax tree structure, and the target query operator corresponding to each column is determined.

[0058] In step a2, the first blood column corresponding to each column in the column information is located based on the recursive search of the target query operator, and the column permission point corresponding to the first blood column is obtained.

[0059] The first blood relation column is a data table column having a blood relation with the column information, and the target query operator is a data table column query operator corresponding to each piece of column information. Recursive searching is performed in combination with the target query operator to determine the first blood relation column having a data blood relation with each piece of column information, and the plurality of first blood relation columns are combined to obtain the column permission point required by the current query.

[0060] As shown in the abstract syntax tree structure Figure 3 The column information is t1.col2, t1.col3 and t2.col3, and the search logic of the first blood relation column corresponding to each piece of column information is described one by one as follows.

[0061] (1) For t1.col2, it can be determined according to the data blood relation that t1.col2 is derived from the Union operator structure of the t1 subquery; recursive search is performed on the Union operator to further determine that the column is derived from the combination of the col2 columns in the upper and lower parts of the Union operator structure, which correspond to 2 Project operator structures; recursive search is performed on the 2 Project operators respectively to locate the first blood relation column having a data blood relation with t1.col2 as db.table1.a2 and db.table2.b2; the data table and data table identifier information in which the first blood relation column is located are recorded, and the specific records are as follows:

[0062] db.table1-1: a2;

[0063] db.table2-2: b2.

[0064] (2) For t1.col3, it can be determined according to the data blood relation that t1.col3 is derived from the Union operator structure of the t1 subquery; recursive search is performed on the Union operator to further determine that the column is derived from the combination of the col3 columns in the upper and lower parts of the Union operator structure, which correspond to 1 Project operator structure; recursive search is performed on the Project operator to locate the second blood relation column having a data blood relation with t1.col3 as: db.table1.a3, db.table1.a4, db.table2.b3 and db.table2.b4; the data table and data table identifier information in which the second blood relation column is located are recorded, and the specific records are as follows:

[0065] db.table1-1: a3;

[0066] db.table1-1: a4;

[0067] db.table2-2: b3;

[0068] db.table2-2: b4.

[0069] (3) For t2.col3, according to the data blood relationship, it can be determined that t2.col3 is derived from the col3 column in the t2 subquery, which corresponds to 1 Project operator structure; recursive search is performed on the Project operator, and the third blood column having a data blood relationship with t2.col3, i.e., db.table3.c3, can be located; the data table and the data table identifier information in which the third blood column is located are recorded, and the specific records are as follows:

[0070] db.table3.c3.

[0071] Combining the data tables and the data table identifier information in which all the blood columns are located obtained in the above (1), (2) and (3) can obtain the column permission points of the current query, and the specific records are as follows:

[0072] db.table1-1.a2;

[0073] db.table1-1.a3;

[0074] db.table1-1.a4;

[0075] db.table2-2.b2;

[0076] db.table2-2.b3;

[0077] db.table2-2.b4;

[0078] db.table3-3.c3.

[0079] In the above embodiment, the column information is traversed based on the data blood relationship, and all blood columns are recursively searched one by one by the target query operator, so that the column-level permission point information that needs to be extracted can be determined by combining the data table and the blood column, and the column granularity permission extraction is realized.

[0080] In step S2043, the row permission point is determined based on the filter condition corresponding to the filter condition operator.

[0081] By extracting the filter condition operator in the abstract syntax tree structure, all filter conditions used in the current query can be obtained. The corresponding row permission point is extracted from the column information combined with the filter condition, and the column permission point can be further extracted.

[0082] In some optional embodiments, the above step S2043 can include:

[0083] In step b1, the type of the filter condition is obtained.

[0084] Step b2: When the type of the filter condition is column and value equality, determine the second lineage column corresponding to the column in the filter condition based on the data lineage.

[0085] Step b3: Concatenate the second lineage column with the values ​​in the filter conditions to obtain the row permission point.

[0086] As mentioned above, filtering conditions include column-to-column equality, column-to-value equality, and value-to-value equality. When a filtering condition operator is extracted, its corresponding filtering condition is determined at the filtering condition operator layer. When the filtering condition type is column-to-value equality, for example, t1.id = 5, the lineage column corresponding to t1.id is obtained based on the data lineage, and the 5th row corresponding to this lineage column is finally determined. This row is used as the row permission point determined by t1.id = 5.

[0087] In the above implementation, by detecting the type of filtering conditions, relying on the data lineage traversal column information, the corresponding lineage column is determined, and the actual row-level permission point information to be extracted is determined by combining the lineage column and the value in the filtering conditions, thus realizing permission extraction at the row level.

[0088] In some alternative embodiments, the above method may further include:

[0089] Step b4: When the type of the filtering condition is column equality, determine the third lineage column corresponding to the column in the filtering condition based on the data lineage relationship, and obtain the column permission point corresponding to the third lineage column.

[0090] If the filtering condition is column equality, that is, when two columns are equal, such as t1.id = t2.id, then the third lineage column corresponding to t1.id and t2.id will be obtained according to the data lineage relationship, and the data table and data table identifier information of the third lineage column will be recorded, so as to obtain the column permission point determined by t1.id = t2.id.

[0091] In the above implementation, by relying on the data lineage to traverse column information, the lineage columns that are equal to each other are extracted, and the column-level permission point information that actually needs to be extracted is further determined, thus achieving accurate extraction of column-level permissions.

[0092] by Figure 3 Taking the abstract syntax tree structure shown as an example, by extracting the filter condition operators, we can determine that the filter conditions are b1=3, c1=10, t1.col2=t2.col2, and t1.col5=5. The following explains the lookup logic for the row or column permission points corresponding to each filter condition.

[0093] The types of the above filtering conditions are classified and processed, and it is determined that "t1.col2=t2.col2" is column-to-column equality, and "t1.col5=5, b1=3, c1=10" is column-to-value equality.

[0094] (1) t1.col2=t2.col2 satisfies the filtering condition of two columns being equal, at this time, according to the data blood relationship, it can be determined that t1.col2 is derived from the Union operator structure of the t1 subquery, and is recursively searched layer by layer to finally locate the corresponding blood column as db.table1.a2 and db.table2.b2; at the same time, according to the data blood relationship, it can be determined that t2.col2 is derived from the Project operator structure of the t2 subquery, and is recursively searched layer by layer to finally locate the corresponding blood column as db.table3.c2; record the data table and data table identifier information where the above extracted blood column is located, and obtain the column permission point determined by the filtering condition, as follows:

[0095] db.table1-1.a2;

[0096] db.table2-2.b2;

[0097] db.table3-3.c2.

[0098] (2) b1=3, c1=10, t1.col5=5 satisfy the filtering condition of column-to-value equality, at this time, according to the data blood relationship, it can be determined that b1 is derived from the Project operator structure, and is recursively searched layer by layer to finally locate the blood column as db.table2.b1; according to the data blood relationship, it can be determined that c1 is derived from the Project operator structure, and is recursively searched layer by layer to finally locate the blood column as db.table3.c1; according to the data blood relationship, it can be determined that t1.col5 is derived from the Union operator structure of the t1 subquery, and is recursively searched layer by layer to finally locate the blood column as db.table1.a5 and db.table2.b5; the above extracted blood column information is spliced with the value in the filtering condition, and the data table identifier information corresponding to the blood column is recorded to obtain the final row permission point as follows:

[0099] db.table2-2.b1=3;

[0100] db.table3-3.c1=10;

[0101] db.table1-1.a5=5;

[0102] db.table2-2.b5=5.

[0103] The all row-level and column-level permission points determined in the above process are divided according to the data table identification information, and the division result is as follows:

[0104] The column permission points are: db.table1-1.a2; db.table2-2.b2; db.table3-3.c2.

[0105] The row permission points are: db.table1-1.a5 = 5; db.table2-2.b1 = 3; db.table2-2.b5 = 5; db.table3-3.c1 = 10.

[0106] It should be noted that if the filter condition is that two values are equal, such as 5 = 5, the calculation result is actually a constant at this time, and it does not involve resource access of the data lake. At this time, the filter condition will be directly ignored, and no permission point will be extracted.

[0107] In step S205, the library table permission point, the column permission point and the row permission point are combined to determine the permission point query range. For details, refer to the related description of the above embodiment, which will not be repeated here.

[0108] The extraction method of the permission point based on the data lake query provided in this embodiment can determine all data tables to be queried by the user at this time by extracting the data table and the operator identification from the abstract syntax tree structure through the scan operator, so as to obtain the library table permission point, and realize the extraction of the library table granularity permission point. The column information and the row information are extracted from the abstract syntax tree structure through the column selection operator and the filter condition operator, and the column permission point and the row permission point are determined according to the data blood relationship, so as to realize the extraction of the row granularity and the column granularity permission point.

[0109] In this embodiment, an extraction method of a permission point based on a data lake query is provided, which can be used for computer devices such as computers, servers, etc. Figure 4 is a flowchart of the extraction method of the permission point based on the data lake query according to the embodiments of the present disclosure, as Figure 4 shown, the flowchart includes the following steps:

[0110] In step S301, an abstract syntax tree structure corresponding to a query statement is obtained.

[0111] Specifically, the above step S301 can include:

[0112] In step S3011, a query statement generated based on a structured query language is obtained.

[0113] The query statement is used to query a data table in the data lake.

[0114] As described above, the data in the data lake scenario is structured and organized in the form of data tables. The query statement is used to query the data table in the data lake. The query statement can be written by the user according to the data processing requirement, can be generated by the user calling a template, or can be imported from an external storage device (such as a U disk, a mobile hard disk, etc.). The way of obtaining the query statement is not limited here, and a person skilled in the art can determine it according to the actual requirement.

[0115] In step S3012, a preset parsing engine is used to parse the query statement, and an abstract syntax tree structure corresponding to the query statement is generated.

[0116] The preset parsing engine is an engine preset for parsing the SQL query statement. The preset parsing engine is deployed in the computer device and provides a query and connection of various data source basic framework to easily access various data sources. The obtained query statement is input into the preset parsing engine, and the query statement is structured by the preset parsing engine to convert the query statement into a structured abstract syntax tree structure.

[0117] In step S302, the scan operator, the column selection operator, and the filter condition operator in the abstract syntax tree structure are extracted. For details, refer to the related description of the corresponding embodiment in the above embodiment, which will not be repeated here.

[0118] In step S303, based on the scan operator, the library table permission point corresponding to the query statement is determined. For details, refer to the related description of the corresponding embodiment in the above embodiment, which will not be repeated here.

[0119] In step S304, based on the column selection operator and the filter condition operator, the column permission point and the row permission point corresponding to the query statement are determined. For details, refer to the related description of the corresponding embodiment in the above embodiment, which will not be repeated here.

[0120] In step S305, the library table permission point, the column permission point, and the row permission point are combined to determine the permission point query range.

[0121] Specifically, the above step S305 can include:

[0122] In step S3051, the first data table identification information corresponding to the library table permission point, the second data table identification information corresponding to the row permission point, and the third data table identification information corresponding to the column permission point are detected.

[0123] The first data table identification information is used to represent the data table where the library table is located, the second data table identification information is used to represent the data table where the row permission point is located, and the third data table identification information is used to represent the data table where the column permission point is located.

[0124] As described above, the corresponding data table identifier information has been recorded when the library table permission point, the row permission point and the column permission point are acquired, and thus the first data table identifier information corresponding to the library table permission point, the second data table identifier information corresponding to the row permission point and the third data table identifier information corresponding to the column permission point can be determined respectively.

[0125] In step S3052, when the first data table identifier information, the second data table identifier information and the third data table identifier information are the same, the library table permission point, the column permission point and the row permission point are combined according to the data table identifier information to obtain the permission point query range corresponding to each data table.

[0126] The first data table identifier information, the second data table identifier information and the third data table identifier information are compared to determine whether the first data table identifier information, the second data table identifier information and the third data table identifier information are the same. When the first data table identifier information, the second data table identifier information and the third data table identifier information are the same, it can be determined that the library table permission point, the column permission point and the row permission point are in the same subquery of the same data table. Then, the extracted library table permission point, column permission point and row permission point are combined and matched according to the data table identifier information, and the permission point query range required for checking each data table can be obtained.

[0127] Taking the example shown in FIG. 8, the library table permission point, the column permission point and the row permission point obtained above are combined and matched to obtain the following permission point query range: Figure 3

[0128] (1) For db.table1-1, the column permission point and the row permission point are:

[0129] db.table1-1.a2; db.table1-1.a3; db.table1-1.a4; db.table1-1.a5 = 5.

[0130] (2) For db.table2-2, the column permission point and the row permission point are:

[0131] db.table2-2.b1 = 3; db.table2-2.b2; db.table2-2.b3; db.table2-2.b4;

[0132] db.table2-2.b5 = 5.

[0133] (3) For db.table3-3, the column permission point and the row permission point are:

[0134] ​db.table3-3.c2; db.table3-3.c3; db.table3-3.c1 = 10.

[0135] That is, the final extracted permission point query range is:

[0136] (1) On the db.tablel table, extract the a2, a3, a4 column permissions, and the a5 = 5 row permission;

[0137] (2) On the db.table2 table, extract the b2, b3, b4 column permissions, and the b5 = 5 row permission;

[0138] (3) On the db.table3 table, extract the c2 column permission, and the cl = 10 row permission.

[0139] In some optional embodiments, the above method can further include:

[0140] Step cl, based on the permission point query range, detecting whether there are column permission points and row permission points that need to be verified in each data table.

[0141] Step c2, when there are column permission points and row permission points that need to be verified in the data table, performing permission checking on the column permission points and the row permission points.

[0142] Step c3, when there are no column permission points and row permission points that need to be verified in the data table, performing permission checking on the data table.

[0143] After determining the row / column permission points of the data table required for authentication according to the permission point query range, the row / column level permission points of the data table can be directly checked to determine whether access is allowed. If the data table does not have row / column level permission points that need to be authenticated, the data table permission corresponding to the row / column level permission points is checked. In this way, the most fine-grained row / column dimension permission control effect can be achieved.

[0144] Here, the column permission points and the row permission points in the data table are checked according to the permission query range to achieve the most fine-grained row / column dimension permission control effect. When there are no column permission points and row permission points that need to be verified in the data table, the permission of the data table is directly verified to achieve the library / table dimension permission control effect.

[0145] The method for extracting a query permission point based on a data lake provided in this embodiment converts a query statement into an abstract syntax tree structure through a parsing engine. Even if the query statement written by a user is various, the use rules of the structured query language are the same. Therefore, converting the query statement into the abstract syntax tree structure can be suitable for various query statements for fine-grained permission extraction of the data lake. The data table identifier information corresponding to the library table permission point, the row permission point and the column permission point is detected, so as to combine and match the library table permission point, the row permission point and the column permission point required for authenticating each data table according to the data table identifier information, obtain a real permission point query range, determine the most fine-grained data resource range to be really queried, and thus the fine-grained permission control can be achieved in combination with the permission point query range.

[0146] In this embodiment, an apparatus for extracting a query permission point based on a data lake is also provided, which is used to implement the above-described embodiments and preferred embodiments, and will not be described again. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, implementation of hardware, or a combination of software and hardware, is also possible and contemplated.

[0147] The apparatus for extracting a query permission point based on a data lake provided in this embodiment includes: Figure 5 As shown in the figure, the apparatus includes:

[0148] The obtaining module 401 is configured to obtain an abstract syntax tree structure corresponding to a query statement.

[0149] The operator extracting module 402 is configured to extract a scan operator, a column selection operator and a filter condition operator in the abstract syntax tree structure.

[0150] The first permission point determining module 403 is configured to determine a library table permission point to be extracted based on the scan operator.

[0151] The second permission point determining module 404 is configured to determine a column permission point and a row permission point to be extracted based on the column selection operator and the filter condition operator.

[0152] The query range determining module 405 is configured to combine the library table permission point, the column permission point and the row permission point to determine a permission point query range.

[0153] In some optional embodiments, the first permission point determining module 403 can include:

[0154] The first extracting unit is configured to extract a data table to be queried at this time and an operator identifier corresponding to the scan operator from the abstract syntax tree structure based on the scan operator.

[0155] The identifier determination unit is configured to determine the data table identifier information according to the data table and the operator identifier.

[0156] The library table permission point determination unit is configured to determine the library table permission point to be identified in the current time according to the data table identifier information.

[0157] In some optional embodiments, the second permission point determination module 404 can include:

[0158] The second extraction unit is configured to extract column information of the data table from the abstract syntax tree structure based on a column selection operator, the column selection operator being at the outermost layer of the abstract syntax tree structure.

[0159] The column permission point determination unit is configured to traverse the column information according to the data blood relationship of the abstract syntax tree structure, and determine a column permission point corresponding to the column information.

[0160] The row permission point determination unit is configured to determine a row permission point based on a filter condition corresponding to a filter condition operator.

[0161] In some optional embodiments, the column permission point determination unit can include:

[0162] The first traversal subunit is configured to traverse the column information according to the data blood relationship of the abstract syntax tree structure, and determine a target query operator corresponding to each column.

[0163] The first blood column determination subunit is configured to locate a first blood column corresponding to each column in the column information based on the target query operator, and obtain a column permission point corresponding to the first blood column through recursive searching.

[0164] In some optional embodiments, the row permission point determination unit can include:

[0165] The filter type acquisition subunit is configured to acquire a type of the filter condition.

[0166] The second blood column determination subunit is configured to determine a second blood column corresponding to a column in the filter condition based on the data blood relationship when the type of the filter condition is column and value equality.

[0167] The splicing subunit is configured to splice the second blood column and a value in the filter condition to obtain a row permission point.

[0168] In some optional embodiments, the column permission point determination unit can further include:

[0169] The third blood column determination subunit is configured to determine a third blood column corresponding to a column in the filter condition based on the data blood relationship when the type of the filter condition is column and column equality, and obtain a column permission point corresponding to the third blood column.

[0170] In some optional embodiments, the obtaining module 401 can include:

[0171] The query statement obtaining unit is configured to obtain a query statement generated based on a structured query language, where the query statement is used to query data tables in the data lake.

[0172] The parsing unit is configured to parse the query statement by using a preset parsing engine to generate an abstract syntax tree structure for the query statement.

[0173] In some optional embodiments, the query range determining module 405 can include:

[0174] The identification detecting unit is configured to detect first data table identification information corresponding to the library table permission point, second data table identification information corresponding to the row permission point, and third data table identification information corresponding to the column permission point.

[0175] The permission point combining unit is configured to combine the library table permission point, the column permission point, and the row permission point according to the data table identification information when the first data table identification information, the second data table identification information, and the third data table identification information are the same, to obtain a permission point query range corresponding to each data table.

[0176] In some optional embodiments, the apparatus can further include:

[0177] The permission point detecting module is configured to detect, based on the permission point query range, whether there is a column permission point and a row permission point that need to be verified in each data table.

[0178] The first permission checking module is configured to perform permission checking on the column permission point and the row permission point when there is a column permission point and a row permission point that need to be verified in the data table.

[0179] The second permission checking module is configured to perform permission checking on the data table when there is no column permission point and row permission point that need to be verified in the data table.

[0180] Further function descriptions of the above modules and units are the same as those of the corresponding embodiments, and will not be described here.

[0181] The extraction apparatus for the query permission point based on the data lake in this embodiment is presented in the form of a functional unit. The unit here refers to an ASIC circuit, a processor and a memory executing one or more software or fixed programs, and / or other devices that can provide the above functions.

[0182] The extraction device for query permission points based on a data lake provided by the embodiment extracts a scan operator, a column selection operator and a filter condition operator from an abstract syntax tree structure to obtain library table permission points, column permission points and row permission points to be extracted by a query statement. Thus, the scan operator, the column selection operator and the filter condition operator can be extracted from the abstract syntax tree structure, and are not limited to fixed rules, so that fine-grained permission extraction can be performed on the data lake, and the row and column information of the bottom layer table to be queried at this time can be accurately obtained, so that the permission point query range of the query at this time is determined in combination with the extracted permission points, and the extracted library, table, row and column are subjected to fine-grained permission control.

[0183] The embodiment of the present disclosure also provides a computer device having the extraction device for query permission points based on a data lake shown in the above Figure 5

[0184] Please refer to Figure 6 , Figure 6 is a structural schematic diagram of a computer device provided by an optional embodiment of the present disclosure, as shown in the figure, the computer device comprises one or more processors 10, a memory 20, and an interface for connecting various components, including a high-speed interface and a low-speed interface. Various components are communicatively connected to each other by different buses, and can be installed on a common motherboard or in other ways as needed. The processor can process instructions executed in the computer device, including instructions stored in the memory or on the memory to display graphical information of a GUI on an external input / output device such as a display device coupled to the interface. In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, each providing part of the necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 6 Figure 6 The processor 10 in the above

[0185] The processor 10 can be a central processor, a network processor or a combination thereof. The processor 10 can further include a hardware chip. The hardware chip can be an application specific integrated circuit, a programmable logic device or a combination thereof. The programmable logic device can be a complex programmable logic device, a field programmable logic gate array, a general array logic or any combination thereof.

[0186] The memory 20 stores instructions executable by the at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiments.

[0187] ​​The memory 20 can include a program storage area and a data storage area. The program storage area can store an operating system, application programs required for at least one function, and the like. The data storage area can store data created according to the use of the computer device, and the like. In addition, the memory 20 can include a high-speed random access memory, and can further include a non-transitory memory such as at least one of a magnetic disk storage device, a flash memory device, or other non-transitory solid state memory device. In some alternative embodiments, the memory 20 can optionally include a memory disposed remotely from the processor 10, which can be connected to the computer device through a network. Examples of the network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0188] The memory 20 can include a volatile memory such as a random access memory, and can further include a non-volatile memory such as a flash memory, a hard disk, or a solid state disk, and a combination thereof.

[0189] The computer device further includes a communication interface 30 for data communication of the computer device with other devices or communication networks.

[0190] The embodiments of the present disclosure further provide a computer readable storage medium, and the method according to the embodiments of the present disclosure can be implemented in hardware, firmware, or recorded in a storage medium, or implemented as computer code to be originally stored in a remote storage medium or a non-transitory machine readable storage medium downloaded through a network and stored in a local storage medium, so that the method described herein can be processed by such software on a storage medium using a general purpose computer, a special purpose processor, or programmable or special purpose hardware. The storage medium can be a magnetic disk, an optical disk, a read-only memory, a random access memory, a flash memory, a hard disk, or a solid state disk, and the like. Further, the storage medium can further include a combination of the above-mentioned kinds of memories. It can be understood that the computer, the processor, the microprocessor controller, or the programmable hardware includes a storage component that can store or receive software or computer code, when the software or computer code is accessed and executed by the computer, the processor, or the hardware, the method shown in the above embodiments is implemented.

[0191] Although the embodiments of the present disclosure are described with reference to the drawings, various modifications and changes can be made by those skilled in the art without departing from the spirit and scope of the present disclosure, and such modifications and changes fall within the scope defined by the appended claims.

Claims

1. A method for extracting query permission points based on a data lake, characterized in that, The method comprises the following steps: obtaining an abstract syntax tree structure corresponding to a query statement; extracting a scan operator, a column selection operator and a filter condition operator in the abstract syntax tree structure; based on the scan operator, determining a library table permission point to be extracted, wherein the library table permission point represents a database in which a data table to be queried is located, and is used to represent a permission point of a library table dimension; based on the column selection operator and the filter condition operator, determining a column permission point and a row permission point to be extracted; combining the library table permission point, the column permission point and the row permission point to determine a permission point query range; based on the column selection operator and the filter condition operator, determining a column permission point and a row permission point to be extracted, comprising: based on the column selection operator, extracting column information of a data table from the abstract syntax tree structure, wherein the column selection operator is located at the outermost layer of the abstract syntax tree structure; traversing the column information according to a data blood relationship of the abstract syntax tree structure to determine the column permission point corresponding to the column information; based on a filter condition corresponding to the filter condition operator, determining the row permission point.

2. The method of claim 1, wherein, The method comprises the following steps: based on the scan operator, extracting a data table to be queried at this time and an operator identifier corresponding to the scan operator from the abstract syntax tree structure; determining data table identifier information according to the data table and the operator identifier; determining the library table permission point to be identified at this time according to the data table identifier information.

3. The method of claim 1, wherein, The method comprises the following steps: traversing the column information according to a data blood relationship of the abstract syntax tree structure to determine a target query operator corresponding to each column; based on the target query operator, recursively finding and locating a first blood column corresponding to each column in the column information to obtain the column permission point corresponding to the first blood column.

4. The method according to claim 1 or 3, characterized in that, The method comprises the following steps: obtaining the type of the filter condition; when the type of the filter condition is column and value equality, determining a second blood column corresponding to a column in the filter condition based on the data blood relationship; splicing the second blood column and the value in the filter condition to obtain the row permission point.

5. The method according to claim 1 or 3, characterized in that, The method further comprises the following steps: when the type of the filter condition is column and column equality, determining a third blood column corresponding to a column in the filter condition based on the data blood relationship to obtain the column permission point corresponding to the third blood column.

6. The method of claim 1, wherein, The method comprises the following steps: obtaining a query statement generated based on a structured query language, wherein the query statement is used to query a data table in the data lake; using a preset parsing engine to parse the query statement to generate an abstract syntax tree structure corresponding to the query statement.

7. The method of claim 1, wherein, The method comprises the following steps: Detect the first data table identifier information corresponding to the library table permission point, the second data table identifier information corresponding to the row permission point, and the third data table identifier information corresponding to the column permission point; When the first data table identifier information, the second data table identifier information, and the third data table identifier information are the same, the library table permission point, the column permission point, and the row permission point are combined according to the data table identifier information to obtain the permission point query range corresponding to each data table.

8. The method according to claim 1 or 7, characterized in that, Also includes: Based on the permission point query range, detect whether the column permission point and the row permission point that need to be verified exist in each data table; When the column permission point and the row permission point that need to be verified exist in the data table, the column permission point and the row permission point are checked for permissions; When the column permission point and the row permission point that need to be verified do not exist in the data table, the data table is checked for permissions.

9. An apparatus for extracting query permission points based on a data lake, the apparatus comprising: a data lake query permission point extractor configured to extract query permission points from a data lake based on a query. The device includes: An acquisition module for acquiring an abstract syntax tree structure corresponding to a query statement; An operator extraction module for extracting a scan operator, a column selection operator, and a filter condition operator in the abstract syntax tree structure; A first permission point determination module for determining a library table permission point to be extracted based on the scan operator, wherein the library table permission point represents a database in which a data table to be queried is located, and is used to represent a permission point of a library table dimension; A second permission point determination module for determining a column permission point and a row permission point to be extracted based on the column selection operator and the filter condition operator; A query range determination module for combining the library table permission point, the column permission point, and the row permission point to determine a permission point query range; The second permission point determination module includes: A second extraction unit for extracting column information of a data table from the abstract syntax tree structure based on the column selection operator, the column selection operator being located at the outermost layer of the abstract syntax tree structure; A column permission point determination unit for traversing the column information according to the data blood relationship of the abstract syntax tree structure to determine a column permission point corresponding to the column information; A row permission point determination unit for determining a row permission point based on a filter condition corresponding to the filter condition operator.

10. A computer device, comprising: Includes: A memory and a processor, which are communicatively connected, the memory stores computer instructions, and the processor executes the computer instructions to perform the method of any one of claims 1-8.

11. A computer readable storage medium, characterized in that, The computer readable storage medium stores computer instructions for causing a computer to execute the method of any one of claims 1-8.

Images