Database-based access control method and device, medium, equipment and product
By obtaining the semantic information and total row count of the data table and calculating the number of unique values, the problem of quickly calculating the number of unique values in massive data is solved, and efficient data access control is achieved.
Patent Information
- Application Number
- CN202411808274.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-09
- Publication Date
- 2025-03-07
AI Technical Summary
When processing massive data, it is difficult to quickly and accurately calculate the number of unique values (NDV) of the data table. Existing methods such as full scanning and random sampling will lead to large data access overhead and take longer.
By obtaining semantic information representation of each column in the data table and determining the total number of rows of the data table, the number of unique values of the data table is calculated based on this information, and avoiding directly reading the original data.
This method can quickly calculate the number of unique values without reading the original data of the data table, reduce calculation time, and improve data access efficiency.
Smart Images
Figure CN119577808A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of data processing technology, and in particular, to a database-based access control method, device, medium, equipment and product. Background Art
[0002] In data analysis and database management scenarios, the estimation of the number of distinct values (NDV) is an important research topic. Accurately estimating the number of unique values is a necessary condition for optimizing queries in almost all relational databases. With the development of big data technology, it is particularly important to quickly and accurately calculate the number of unique values when processing massive amounts of data.
[0003] In related technologies, all data in a data table can be scanned, and then the number of unique values can be calculated through different hash value calculation and analysis methods, or a certain number of data samples can be randomly sampled from the original data, and then the sampled data samples can be analyzed through statistical methods or machine learning methods to calculate the number of unique values. However, both full scan and random sampling will bring large data access overhead, resulting in a long time to calculate the number of unique values. Summary of the invention
[0004] This summary is provided to introduce concepts in a brief form that will be described in detail in the detailed description below. This summary is not intended to identify key features or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.
[0005] In a first aspect, the present disclosure provides a database-based access control method, comprising: Obtaining semantic information representation of each column in a data table and determining the total number of rows in the data table, wherein the semantic information representation of each column includes an independent semantic message corresponding to each column and interactive semantic information between each column and other columns in the data table; The number of unique values in the data table is determined according to the semantic information representation of each column and the total number of rows in the data table, and the number of unique values is used to perform access control on the database.
[0006] In a second aspect, the present disclosure provides a database-based access control device, comprising: An acquisition module is configured to acquire the semantic information representation of each column in the data table and determine the total number of rows in the data table, wherein the semantic information representation of each column includes an independent semantic message corresponding to each column and interactive semantic information between each column and other columns in the data table; The determination module is configured to determine the number of unique values in the data table according to the semantic information representation of each column and the total number of rows in the data table, wherein the number of unique values is used for access control to the database.
[0007] In a third aspect, the present disclosure provides a computer-readable medium having a computer program stored thereon, which, when executed by a processing device, implements the steps of the method described in the first aspect.
[0008] In a fourth aspect, the present disclosure provides an electronic device, including: a storage device having a computer program stored thereon; A processing device is used to execute the computer program in the storage device to implement the steps of the method described in the first aspect.
[0009] In a fifth aspect, the present disclosure provides a computer program product, including a computer program, which implements the steps of the method described in the first aspect when executed by a processor.
[0010] Based on the above technical solution, by obtaining the semantic information representation of each column in the data table and determining the total number of rows in the data table, the number of unique values in the data table is determined according to the semantic information representation and the total number of rows, and the number of unique values in the data table can be calculated while relying on the original data in the data table, thereby reducing the time required to calculate the number of unique values. Moreover, by using the semantic information representation of the column to calculate the number of unique values, there is no need to use the original data of the data table to calculate the number of unique values, so there is no need to read the original data of the data table, thereby greatly reducing the calculation time of the number of unique values.
[0011] Other features and advantages of the present disclosure will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS
[0012] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale. In the drawings: Figure 1 is a flowchart of a database-based access control method according to some embodiments.
[0013] Figure 2 It is a schematic diagram showing the principles of a database-based access control method according to some embodiments.
[0014] Figure 3 It is a schematic diagram of the structure of a database-based access control device according to some embodiments.
[0015] Figure 4 is a schematic diagram of the structure of an electronic device according to some embodiments. DETAILED DESCRIPTION
[0016] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.
[0017] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.
[0018] The term "including" and its variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.
[0019] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.
[0020] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".
[0021] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.
[0022] Figure 1 FIG. 1 is a flowchart of a database-based access control method according to some embodiments. Figure 1 As shown, the embodiment of the present disclosure provides a database-based access control method, which can be executed by an electronic device, specifically by a database-based access control device, which can be implemented by software and / or hardware and configured in an electronic device. Figure 1As shown, the method may include the following steps.
[0023] In step 110, the semantic information representation of each column in the data table is obtained and the total number of rows in the data table is determined.
[0024] Here, a data table is the basic unit for storing data in a database, which consists of rows and columns. Each column represents an attribute or field and stores data of the same type. The semantic information of a column may include the independent semantic message corresponding to each column and the interactive semantic information between each column and other columns in the data table. Among them, the independent semantic message refers to the data meaning of the column itself, and the interactive semantic information refers to the data relationship between the column and other columns in the data table in the context. Through the semantic information representation of each column, the relationship and constraints of the data in the column and the relationship between the data in the column and the data in other columns can be described.
[0025] It is worth noting that the semantic information representation of each column in the data table can be obtained by analyzing the data table in response to a request for determining the number of unique values. Of course, the semantic information representation of each column in the data table can also be obtained from a database in response to a request for determining the number of unique values. The semantic information representation of each column in the data table is pre-stored in the database. In other words, the semantic information representation can be calculated in real time or in advance. When determining the number of unique values, the pre-calculated semantic information representation is directly used.
[0026] The total number of rows in a data table refers to the total number of all rows in the data table. Each row in the data table represents a data record and contains the values of all columns defined in the data table. The total number of rows can be used to measure the amount of data in the data table, reflecting the scale of data stored in the data table.
[0027] It should be understood that the total number of rows can be pre-calculated or obtained in real time when the number of unique values is determined. For example, the total number of rows in the data table can be obtained from a system table that maintains the total number of rows in the database without accessing the original data in the data table.
[0028] In step 120, the number of unique values in the data table is determined according to the semantic information representation of each column and the total number of rows in the data table.
[0029] Here, the number of unique values refers to the number of different values in a field in a data table. For example, if the data table contains the values 100, 200, and 300, then the number of unique values is 3.
[0030] The semantic information representation and total number of rows corresponding to each column in the data table can be input into the trained machine learning model to obtain the number of unique values output by the machine learning model.
[0031] Exemplarily, the trained machine learning model can be obtained by training an initial machine learning model using a historical data table carrying labels with a unique number of values.
[0032] It is worth noting that, through the semantic information representation of each column, the number of unique values in the data table can be calculated without using the original data in the data table.
[0033] Among them, the number of unique values can be used to control access to the database. In a database management system, the number of unique values is a statistic of the number of unique values in a table field, which is crucial for the database query optimizer (CBO) to estimate the cost of the execution plan. After determining the number of unique values in a data table, access control can be performed on the database based on the number of unique values. Access control can include index recommendations or estimated query plan costs.
[0034] Therefore, by obtaining the semantic information representation of each column in the data table and determining the total number of rows in the data table, the number of unique values in the data table is determined based on the semantic information representation and the total number of rows, and the number of unique values in the data table can be calculated while relying on the original data in the data table, thereby reducing the time required to calculate the number of unique values. Moreover, by using the semantic information representation of the column to calculate the number of unique values, there is no need to use the original data of the data table to calculate the number of unique values, so there is no need to read the original data of the data table, thereby greatly reducing the calculation time of the number of unique values.
[0035] In some feasible implementations, the column vector corresponding to each column in the data table can be obtained, and then a pre-trained interactive attention mechanism model can be called to perform semantic interaction between the columns in the data table using the column vector of each column as the model input to obtain the column interaction vector corresponding to each column. The column vector corresponding to each column and the column interaction vector corresponding to each column are then vector fused to obtain the semantic information representation of each column.
[0036] Here, the column vector represents the independent semantic information corresponding to each column, and the column interaction vector represents the interactive semantic information between each column and other columns.
[0037] In some embodiments, for each column in a data table, multidimensional attribute information of each column is extracted, and then the multidimensional attribute information extracted from each column is sequentially concatenated into text, and a pre-trained language model is called to encode the text corresponding to each column to generate a column vector corresponding to each column.
[0038] The multidimensional attribute information includes the name of the column, the data type of the column, the data constraint of the column, and the comment of the column. The data type of the column can refer to the full name of the type defined in the database schema (describing the structure and organization of the database). For variable-length data types such as varchar (variable-length string), the data type of the column needs to include the length. The data constraint of the column refers to the constraint conditions of the column. For example, for a certain column, it can be restricted that a certain column cannot be empty. The comment of the column refers to the explanation added for the column.
[0039] The multi-dimensional attribute information extracted from each column can be sequentially concatenated into a sentence (empty strings can be used to replace missing attribute information), and then the sentences in each column are input into the pre-trained language model to obtain the column vector of each column.
[0040] Interactive semantic information refers to the semantic information obtained by semantically interacting the column vector of a column with the column vectors of other columns in the data table. For each column in the data table, the column vector corresponding to the column and the column vectors corresponding to other columns in the data table are input into the interactive attention mechanism model to obtain the interactive semantic information of the column. For example, if the data table has 100 columns, for the 5th column, the column vector of the 5th column and the column vectors of the remaining 99 columns are input into the interactive attention mechanism model to obtain the interactive semantic information of the 5th column.
[0041] Exemplarily, the interactive attention mechanism model can be a multi-head attention model. Accordingly, for each column in the data table, the column vector corresponding to the column and the column vectors of other columns in the data table can be input into the multi-head attention model to obtain the interactive semantic information of the column output by the multi-head attention model.
[0042] The multi-head attention model is used to perform semantic interaction between the column vector corresponding to the column and the column vectors corresponding to other columns in the data table to obtain the interactive semantic information of the column.
[0043] The multi-head attention model captures and understands the semantic associations and features at different levels and angles in the input data through the multi-head self-attention mechanism. The multi-head attention model can learn different feature associations from different subspaces of the input sequence. In this way, the multi-head attention model can simultaneously focus on different aspects of the input sequence through multiple attention heads, such as grammatical structure, semantic role, topic transfer, etc. Through multiple attention heads, the multi-head attention model can learn richer contextual information. Each attention head may focus on different features of the input, and these features combined can more comprehensively understand and process the input sequence.
[0044] For each column in the data table, the column vector of the column and the corresponding interactive semantic information may be concatenated into a semantic information representation corresponding to the column.
[0045] Therefore, each semantic information representation includes both independent semantic information of the column itself and interactive semantic information between the column and other columns. That is, the semantic information representation includes the data relationship of the column itself and the data relationship between the column and other columns.
[0046] Therefore, through the above implementation, the semantic information representation of each column can be accurately obtained, so that the semantic content of each column can be fully understood, so that the number of unique values calculated subsequently can be more accurate.
[0047] In some feasible implementations, in step 110, the pre-maintained semantic information representation corresponding to each column may be read from the database.
[0048] Here, the semantic information representation in the database may be pre-calculated and stored in the database. It should be understood that the database storing the semantic information representation and the database storing the data table may not be the same database.
[0049] Among them, the semantic information representation can be obtained by vector fusion of the column vector corresponding to each column and the column interaction vector corresponding to each column. The column interaction vector corresponding to each column is obtained by calling the pre-trained interactive attention mechanism model and using the column vector of each column as the model input to perform semantic interaction between the columns in the data table. The column vector represents the independent semantic information corresponding to each column, and the column interaction vector represents the interactive semantic information between each column and other columns in the data table.
[0050] That is to say, the semantic information representation of each data table can be calculated in advance by providing steps for determining the semantic information representation through the above-mentioned embodiment, and then the semantic information representation is stored in a database. When a request for determining the number of unique values is received, the semantic information representation of each column in the data table is read from the database in response to the request.
[0051] Exemplarily, when resources are idle, the semantic information representation of the data table can be pre-calculated and stored in the database.
[0052] That is, the semantic information representation can be calculated in advance and stored in the database. That is, the semantic information representation calculation process and the unique value number calculation process can be separated, thereby reducing the time required for calculating the unique value number.
[0053] Therefore, through the above implementation, the time required for calculating the number of unique values can be reduced, and the calculation efficiency of the number of unique values can be greatly improved.
[0054] In some feasible implementations, in step 120, a pre-trained multi-layer perceptron model may be called to obtain the number of unique values in the data table.
[0055] Here, the Multilayer Perceptron (MLP) model is a feed-forward artificial neural network model that contains at least three layers of nodes: an input layer, one or more hidden layers, and an output layer. MLP is trained using a supervised learning technique called backpropagation to approximate a target function.
[0056] It is worth noting that the pre-trained multi-layer perceptron model can be obtained by training the multi-layer perceptron model with a historical data table carrying labels with unique value numbers.
[0057] By inputting the semantic information representation of each column in the data table and the total number of rows in the data table into the pre-trained multi-layer perceptron model, the number of unique values in the data table output by the multi-layer perceptron model is obtained.
[0058] Of course, the semantic information representation of each column in the data table, the total number of rows in the data table, and the data statistical information of each column can also be input into the pre-trained multi-layer perceptron model to obtain the number of unique values in the data table output by the multi-layer perceptron model.
[0059] Therefore, through the above implementation, since the information input into the multi-layer perceptron model contains sufficient semantic information, the multi-layer perceptron model can accurately identify the number of unique values in the data table.
[0060] In some feasible implementations, data statistics information of each column in the data table may also be obtained. Accordingly, in step 120, the number of unique values in the data table may be determined based on the semantic information representation of each column, the total number of rows in the data table, and the data statistics information of each column.
[0061] Here, the data statistical information of a column refers to the data statistical features corresponding to each column, and may include the occurrence frequency of the data under each column and the frequency of the occurrence frequency of the data under each column.
[0062] Exemplarily, the semantic information representation, the total number of rows and the data statistical information may be input into a trained multi-layer perceptron model to obtain the number of unique values in the data table output by the multi-layer perceptron model.
[0063] It is worth noting that by using data statistics, the data features in the data table can be added to the calculation of the number of unique values in the data table, so that the calculated number of unique values in the data table can be more accurate.
[0064] In some feasible implementations, the data content of the first data block in the data table can be read, and then for each column in the data content, the frequency of occurrence of different data values in the column and the frequency corresponding to each occurrence frequency in the column are counted, and then the data statistical information corresponding to each column is determined based on the occurrence frequency and the frequency corresponding to the occurrence frequency.
[0065] The data content of a data block (page) in the data table includes one or more rows of data, and each row includes data under all columns.
[0066] By reading the data content of the first data block, the data in the data table can be obtained with the least I / O cost. It should be noted that whether reading all the data in the data table or randomly sampling the data in the data table, the data in the data table may actually be read once, which will result in a longer data reading time. For example, for a data table with 10,000 data blocks, if the data content is read by random sampling, even if 1% of the data is sampled, 10,000 data blocks may be read, which will result in a longer data reading time.
[0067] In the disclosed embodiment, by reading the first data block in the data table and determining the data statistics of each column based on the data content of the first data block, the time required for data reading can be greatly reduced. It is worth noting that the data statistics of each column are actually represented by the data features of a local data block.
[0068] For each column in the data content, the occurrence frequencies of different data values in the column and the frequencies corresponding to each occurrence frequency in the column are counted. The occurrence frequencies of different data values refer to the number of occurrences of different data values in a column. For example, if a column includes data "10, 20, 11, 10, 10, 11", the occurrence frequency of 10 is 3, the occurrence frequency of 20 is 1, and the occurrence frequency of 11 is 2. The frequency corresponding to each occurrence frequency refers to the occurrence frequency of the corresponding value of the occurrence frequency. For example, the frequency of the occurrence frequency 3 is 1, the frequency of the occurrence frequency 1 is 1, and the frequency of the occurrence frequency 2 is 1.
[0069] For each column in the data content, the occurrence frequency and the frequency of the occurrence frequency corresponding to the column can be determined as the data statistics of the column. It is worth noting that by sequentially reading the data content of the first data block in the data table, the data sample required for calculating the data statistics can be obtained in one data read, which can greatly reduce the I / O cost.
[0070] Therefore, through the above implementation, by using data statistical information, the data features in the data table can be added to the calculation of the number of unique values in the data table, so that the calculated number of unique values in the data table can be more accurate. Moreover, by reading the data content of the first data block in the data table and obtaining data statistical information based on the data content, the time for data reading can be greatly reduced, thereby greatly reducing the time required for calculating the number of unique values.
[0071] In some feasible implementations, in response to an index recommendation from a database, the cost of the index can be estimated based on the number of unique values in a data table, and an index recommendation can be made based on the estimated cost of the index; or, in response to a query request from a database, the cost of a query plan can be estimated based on the number of unique values in a data table, and query processing can be performed based on the estimated cost of the query plan.
[0072] Here, calculating the number of unique values is a very basic requirement and function in databases and big data systems. For example, in an index recommendation scenario, calculating the number of unique values can be used to estimate the cost of the index. For another example, in a query optimization scenario, calculating the number of unique values can be used to estimate the cost of the query plan to improve query performance and efficiency. In these scenarios, there are high requirements for the calculation time of the number of unique values, and it is usually difficult to reserve sufficient time for accessing data. Through the database-based access control method provided by the embodiment of the present disclosure, the number of unique values can be quickly calculated while spending less data access time or even without relying on the original data.
[0073] When estimating the cost of an index based on the number of unique values in a data table, the cost of different indexes can be calculated based on the number of unique values and the total number of rows corresponding to different fields, and then index recommendations can be made based on the estimated cost of the index. It should be understood that the larger the number of unique values, the larger the amount of data that the index can cover, and accordingly, the larger the cost of the index, and the index with the highest cost can be recommended to the user to reduce the amount of data that needs to be scanned based on the index. When estimating the cost of a query plan based on the number of unique values in a data table, the cost of the query plan can be calculated based on the number of unique values and the total number of rows, and then the query plan with the lowest cost can be recommended to the user to perform query processing based on the query plan with the lowest cost, thereby optimizing query performance and improving query response time.
[0074] Therefore, through the above implementation, index recommendations or query plans can be optimized based on the number of unique values to improve query performance and query efficiency.
[0075] Figure 2 FIG. 1 is a schematic diagram showing a database-based access control method according to some embodiments. Figure 2As shown, the multi-dimensional attribute information of each column can be extracted from the data table, and the multi-dimensional attribute information can be sequentially concatenated into sentences. The sentences corresponding to each column are encoded by the pre-trained language model to obtain the column vector corresponding to the sentences corresponding to each column.
[0076] Then, for each column vector, the column vector of the column and the column vectors of other columns in the data table are input into the multi-head attention model, and the multi-head attention model performs semantic interaction between the column vector of the column and the vectors of other columns to obtain the interactive semantic information of the columns in the same data table. Then, for each column, the column vector of the column is concatenated with the interactive semantic information of the column to obtain the semantic information representation corresponding to the column.
[0077] The data content of the first data block is read from the data table, and the data statistics of each column are obtained based on the data content. Then, the data statistics are encoded into data features.
[0078] The semantic information representation, data features and total number of rows of each column are input into the pre-trained multi-layer perceptron model to obtain the number of unique values in the data table output by the multi-layer perceptron model.
[0079] It is worth noting that, based on the database-based access control method provided by the embodiment of the present disclosure, the semantic information representation of the column can be used to perform numerical feature estimation, so that the number of unique values in the data table can be accurately calculated without relying on the original data.
[0080] Based on the database-based access control method provided by the embodiment of the present disclosure, training and testing were performed on the open source table data set TabLib. On the test set containing more than one thousand data tables, the estimated error of the number of unique values calculated by the database-based access control method provided by the embodiment of the present disclosure is much lower than the estimated error of the technical solution of the related technology. Whether it is reading 100 rows of data sequentially or reading 100 rows of data randomly, the effect of the database-based access control method proposed by the embodiment of the present disclosure far exceeds all current related technologies.
[0081] Moreover, the time required for calculating the number of unique values by the database-based access control method provided by the embodiment of the present disclosure is also much shorter than the time required for calculating the number of unique values by the technical solutions of the related art. When using a GPU (Graphics Processing Unit), the number of unique values can be calculated within a dozen milliseconds through the database-based access control method provided by the present disclosure. Even without using a GPU, the number of unique values can be calculated within 2 seconds. In addition, since the calculation of the semantic information representation and the calculation of the number of unique values can be performed separately, that is, the semantic information representation can be pre-calculated when resources are idle, no matter how large the scale of the data table is, the data access cost of the database-based access control method provided by the present disclosure is particularly small and fixed.
[0082] Figure 3 FIG. 1 is a schematic diagram of a structure of a database-based access control device according to some embodiments. Figure 3 As shown, the embodiment of the present disclosure provides a database-based access control device 300, and the database-based access control device 300 includes: The acquisition module 301 is configured to acquire the semantic information representation of each column in the data table and determine the total number of rows in the data table, wherein the semantic information representation of each column includes an independent semantic message corresponding to each column and interactive semantic information between each column and other columns in the data table; The determination module 302 is configured to determine the number of unique values in the data table according to the semantic information representation of each column and the total number of rows in the data table, wherein the number of unique values is used for access control to the database.
[0083] Optionally, the acquisition module 301 includes: An acquisition unit is configured to acquire a column vector corresponding to each column in the data table, wherein the column vector represents independent semantic information corresponding to each column; An interaction unit is configured to call a pre-trained interactive attention mechanism model, use the column vector of each column as a model input to perform semantic interaction between the columns in the data table, and obtain a column interaction vector corresponding to each column, wherein the column interaction vector represents interactive semantic information between each column and other columns; The fusion unit is configured to perform vector fusion on the column vector corresponding to each column and the column interaction vector corresponding to each column to obtain the semantic information representation of each column.
[0084] Optionally, the acquiring unit is configured to: For each column in the data table, extract the multidimensional attribute information of each column, wherein the multidimensional attribute information includes the name of the column, the data type of the column, the data constraint of the column and the comment of the column; The multi-dimensional attribute information extracted from each column is sequentially concatenated into text, and the pre-trained language model is called to encode the text corresponding to each column to generate a column vector corresponding to each column.
[0085] Optionally, the acquisition module 301 is specifically configured to: The pre-maintained semantic information representation corresponding to each column is read from the database, wherein the semantic information representation is obtained by vector fusion of the column vector corresponding to each column and the column interaction vector corresponding to each column, and the column interaction vector corresponding to each column is obtained by calling a pre-trained interactive attention mechanism model and using the column vector of each column as a model input to perform semantic interaction between columns in the data table, the column vector represents independent semantic information corresponding to each column, and the column interaction vector represents interactive semantic information between each column and other columns in the data table.
[0086] Optionally, the acquisition module 301 is further configured to: Obtain data statistics information of each column in the data table; The determination module 302 is configured to: The number of unique values in the data table is determined according to the semantic information representation of each column, the total number of rows in the data table, and the data statistical information of each column.
[0087] Optionally, the acquisition module 301 is specifically configured to: Read the data content of the first data block in the data table; For each column in the data content, counting the occurrence frequencies of different data values in each column and counting the frequencies corresponding to the occurrence frequencies in the column; Data statistical information corresponding to each column is determined according to the occurrence frequency and the frequency corresponding to the occurrence frequency.
[0088] Optionally, the determining module 302 is configured to: The pre-trained multi-layer perceptron model is called to obtain the number of unique values of the data table.
[0089] Optionally, the database-based access control device 300 further includes: A first estimation module is configured to estimate the cost of the index according to the number of unique values of the data table in response to the index recommendation of the database, and make an index recommendation according to the estimated cost of the index; or The second estimation module is configured to respond to a query request of the database, estimate the cost of the query plan according to the number of unique values of the data table, and perform query processing according to the estimated cost of the query plan.
[0090] The logic of the method executed by each functional module in the above-mentioned database-based access control device 300 can refer to the part of the method related to the above-mentioned embodiment, and will not be repeated here.
[0091] Reference below Figure 4 , which shows a schematic diagram of the structure of an electronic device (such as a terminal device or a server) 400 suitable for implementing the embodiment of the present disclosure. The terminal device in the embodiment of the present disclosure may include but is not limited to mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 4 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.
[0092] like Figure 4 As shown, the electronic device 400 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 401, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 402 or a program loaded from a storage device 408 into a random access memory (RAM) 403. In the RAM 403, various programs and data required for the operation of the electronic device 400 are also stored. The processing device 401, the ROM 402, and the RAM 403 are connected to each other via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.
[0093] Typically, the following devices may be connected to the I / O interface 405: an input device 406 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 407 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 408 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 409. The communication device 409 may allow the electronic device 400 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 4 The electronic device 400 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.
[0094] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 409, or installed from the storage device 408, or installed from the ROM 402. When the computer program is executed by the processing device 401, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.
[0095] It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. Computer readable signal media may also be any computer readable medium other than computer readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.
[0096] In some embodiments, the electronic devices may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.
[0097] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.
[0098] The above-mentioned computer-readable medium carries one or more programs. When the above-mentioned one or more programs are executed by the electronic device, the electronic device: obtains the semantic information representation of each column in the data table and determines the total number of rows in the data table, wherein the semantic information representation of each column includes the independent semantic message corresponding to each column and the interactive semantic information between each column and other columns in the data table; determines the number of unique values of the data table based on the semantic information representation of each column and the total number of rows of the data table, and the number of unique values is used to perform access control on the database.
[0099] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages or a combination thereof, including, but not limited to, object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0100] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0101] The modules involved in the embodiments described in the present disclosure may be implemented by software or hardware, wherein the name of a module does not, in some cases, limit the module itself.
[0102] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.
[0103] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0104] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.
[0105] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.
[0106] Although the subject matter has been described in language specific to structural features and / or method logic actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims. Regarding the device in the above embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.
Claims
1. A database-based access control method, characterized in that: include: Obtaining semantic information representation of each column in a data table and determining the total number of rows in the data table, wherein the semantic information representation of each column includes an independent semantic message corresponding to each column and interactive semantic information between each column and other columns in the data table; The number of unique values in the data table is determined according to the semantic information representation of each column and the total number of rows in the data table, and the number of unique values is used to perform access control on the database.
2. The method according to claim 1, characterized in that The obtaining of the semantic information representation of each column in the data table includes: Obtain a column vector corresponding to each column in the data table, wherein the column vector represents independent semantic information corresponding to each column; Call the pre-trained interactive attention mechanism model, use the column vector of each column as the model input to perform semantic interaction between the columns in the data table, and obtain the column interaction vector corresponding to each column, wherein the column interaction vector represents the interactive semantic information between each column and other columns; The column vector corresponding to each column and the column interaction vector corresponding to each column are fused to obtain the semantic information representation of each column.
3. The method according to claim 2, characterized in that The obtaining of the column vector corresponding to each column in the data table includes: For each column in the data table, extract the multidimensional attribute information of each column, wherein the multidimensional attribute information includes the name of the column, the data type of the column, the data constraint of the column and the comment of the column; The multi-dimensional attribute information extracted from each column is sequentially concatenated into text, and the pre-trained language model is called to encode the text corresponding to each column to generate a column vector corresponding to each column.
4. The method according to claim 1, characterized in that The obtaining of the semantic information representation of each column in the data table includes: The pre-maintained semantic information representation corresponding to each column is read from the database, wherein the semantic information representation is obtained by vector fusion of the column vector corresponding to each column and the column interaction vector corresponding to each column, and the column interaction vector corresponding to each column is obtained by calling a pre-trained interactive attention mechanism model and using the column vector of each column as a model input to perform semantic interaction between columns in the data table, the column vector represents independent semantic information corresponding to each column, and the column interaction vector represents interactive semantic information between each column and other columns in the data table.
5. The method according to claim 1, characterized in that: The method further comprises: Obtain data statistics information of each column in the data table; The determining the number of unique values of the data table according to the semantic information representation of each column and the total number of rows of the data table includes: The number of unique values in the data table is determined according to the semantic information representation of each column, the total number of rows in the data table, and the data statistical information of each column.
6. The method according to claim 5, characterized in that: The obtaining of data statistics information of each column in the data table includes: Read the data content of the first data block in the data table; For each column in the data content, counting the occurrence frequencies of different data values in each column and counting the frequencies corresponding to the occurrence frequencies in the column; Data statistical information corresponding to each column is determined according to the occurrence frequency and the frequency corresponding to the occurrence frequency.
7. The method according to claim 1, characterized in that: The obtaining the number of unique values of the data table includes: The pre-trained multi-layer perceptron model is called to obtain the number of unique values of the data table.
8. The method according to claim 1, characterized in that: The method further comprises: In response to the index recommendation of the database, estimating the cost of the index according to the number of unique values of the data table, and making an index recommendation according to the estimated cost of the index; or, In response to a query request from a database, a cost of a query plan is estimated according to the number of unique values in the data table, and query processing is performed according to the estimated cost of the query plan.
9. A database-based access control device, characterized in that: include: An acquisition module is configured to acquire the semantic information representation of each column in the data table and determine the total number of rows in the data table, wherein the semantic information representation of each column includes an independent semantic message corresponding to each column and interactive semantic information between each column and other columns in the data table; The determination module is configured to determine the number of unique values in the data table according to the semantic information representation of each column and the total number of rows in the data table, wherein the number of unique values is used for access control to the database.
10. A computer readable medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processing device, the steps of the method according to any one of claims 1 to 8 are implemented.
11. An electronic device, characterized in that: include: a storage device having a computer program stored thereon; A processing device, configured to execute the computer program in the storage device to implement the steps of the method according to any one of claims 1 to 8.
12. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.