Data authority control method and device based on multimode heterogeneous data source
The method addresses the challenge of unified data permission control across heterogeneous data sources by employing a Jupiter-based system to abstract and validate metadata, ensuring secure and consistent access across diverse data storage formats and structures.
Patent Information
- Application Number
- CN202411636538.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-15
- Publication Date
- 2025-03-21
AI Technical Summary
The heterogeneity of data storage formats and structures across multiple data sources, such as relational databases, NoSQL databases, and file systems, makes unified data permission control challenging, as some systems lack schema or table concepts, complicating the management of access rights across diverse data sources.
A method and apparatus for unified data permission control across multiple heterogeneous data sources, utilizing a Jupiter-based SQL parser to abstract user queries into four-layer metadata, which is then validated and filtered through a data permission controller comprising meta-data controllers, strategy transformers, and operator transformers, ensuring data visibility conditions are applied uniformly across different data sources.
Enables comprehensive and uniform data permission management across diverse data sources by abstracting and validating metadata at multiple layers, ensuring secure and consistent data access control through adaptable filtering and transformation mechanisms.
Smart Images
Figure CN119670111A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of data processing technology, and in particular to a data authority control method and device based on multi-mode heterogeneous data sources. Background Art
[0002] Data elements are the smallest data units of data transactions. Depending on the structure and function of the data, they are stored in different storages of the data vault, such as relational databases such as MySQL and DAMO, data warehouse databases such as Hive, and even directly in file systems such as HDFS. However, the data storage formats and storage structures of these multi-mode heterogeneous data sources are diverse, resulting in different external logical abstractions. Some databases do not have schemas, such as elasticsearch, and some data sources do not have the concept of tables, such as file systems such as HDFS. This makes it difficult to unify the data permissions of these heterogeneous data sources.
[0003] In view of this, how to provide a data permission control method to achieve unified data permission control of multi-mode heterogeneous data sources has become a technical problem that needs to be solved urgently. Summary of the invention
[0004] The embodiments of the present application provide a data permission control method based on multi-mode heterogeneous data sources, a data permission control device based on multi-mode heterogeneous data sources, an electronic device and a computer storage medium, which are used to solve the problem of unified data permission control of multi-mode heterogeneous data sources.
[0005] In a first aspect of an embodiment of the present application, a data authority control method based on a multi-mode heterogeneous data source is provided, comprising:
[0006] The user query SQL statement is abstracted into four layers of metadata through Jupiter's SQL parser, and the four layers of metadata are sent to the data authority controller, where the four layers of metadata include datasource layer, schema layer, table layer and column layer; the data authority controller is designed based on the Jupiter computing engine, including metadata authority controller, filter, strategy converter and operator converter;
[0007] Use the four-level metadata verifier to verify the data permissions corresponding to the four-layer metadata. When the metadata at the datasource layer, schema layer, table layer, and column layer all pass the permission verification, use the pluggable filter to obtain the data strategy;
[0008] The data strategy is converted into data visibility conditions through the strategy converter, and the where child node of the jupiterSelect syntax tree is registered;
[0009] The operator converter is used to convert the data visibility conditions into Jupiter's filter operator, and the data visibility conditions are pushed to the multi-mode data source through the Jupiter optimizer to control data permissions.
[0010] In a second aspect of an embodiment of the present application, a data authority control device based on a multi-mode heterogeneous data source is provided, comprising:
[0011] The sending module is configured to abstract the user query SQL statement into four layers of metadata through Jupiter's SQL parser, and send the four layers of metadata to the data authority controller, wherein the four layers of metadata include datasource layer, schema layer, table layer and column layer; the data authority controller is designed based on the Jupiter computing engine, and includes a metadata authority controller, a filter, a strategy converter and an operator converter;
[0012] The acquisition module is configured to use the four-level metadata verifier to verify the data permissions corresponding to the obtained four-layer metadata. When the metadata at the datasource layer, schema layer, table layer, and column layer all pass the permission verification, the pluggable filter is used to obtain the data strategy;
[0013] The registration module is configured to convert the data strategy into the data visibility condition through the strategy converter and register the where child node of the jupiterSelect syntax tree;
[0014] The control module is configured to use the operator converter to convert the data visibility conditions into Jupiter's filter operator, and push the data visibility conditions to the multi-mode data source through the Jupiter optimizer to control data permissions.
[0015] According to a third aspect of an embodiment of the present application, a computing device is provided, including:
[0016] Memory and processor;
[0017] The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the above-mentioned data authority control method based on multi-mode heterogeneous data sources are implemented.
[0018] According to a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, which stores computer-executable instructions, and when the instructions are executed by a processor, the steps of the above-mentioned data permission control method based on multi-mode heterogeneous data sources are implemented.
[0019] The present application provides a data permission control method based on multi-mode heterogeneous data sources, including: abstracting a user query SQL statement into four layers of metadata through Jupiter's SQL parser, and sending the four layers of metadata to a data permission controller, wherein the four layers of metadata include a datasource layer, a schema layer, a table layer, and a column layer; the data permission controller is designed on the basis of the Jupiter computing engine, and includes a metadata permission controller, a filter, a strategy converter, and an operator converter; using a four-level metadata verifier to verify the data permissions corresponding to the obtained four layers of metadata, and when the metadata of the datasource layer, the schema layer, the table layer, and the column layer all pass the permission verification, using a pluggable filter to obtain the data policy; converting the data policy into a data visibility condition through a strategy converter, and registering a where child node of a JupiterSelect syntax tree; using an operator converter to convert the data visibility condition into a filter operator of Jupiter, and pushing the data visibility condition to the multi-mode data source through a Jupiter optimizer to control the data permissions.
[0020] The data permission control method based on multi-mode heterogeneous data sources provided in the embodiment of the present application is applied. On the basis of the Jupiter computing engine, a data permission controller is designed. The controller is composed of a metadata permission controller, a filter, a policy converter and an operator converter component. The Jupiter permission manager abstracts different data sources into four layers: datasource, schema, table and column, and uses a four-level permission controller to implement metadata permission verification to ensure unified permission verification of multi-mode queries. After the metadata permission is passed, a pluggable filter is used to obtain the data policy; the policy converter converts the policy into a data visibility condition and registers the where child node of the JupiterSelect syntax tree; finally, the operator converter is used to convert the data visibility condition into the Jupiter filter operator, and the Jupiter optimizer pushes the data visibility condition to the multi-mode data source to implement data permission control.
[0021] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, which can be implemented in accordance with the contents of the specification, and in order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS
[0022] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present application. Also, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:
[0023] Figure 1 A schematic diagram of the architecture of a data authority control system based on multi-mode heterogeneous data sources provided in an embodiment of the present application;
[0024] Figure 2 A flowchart of a data permission control method based on multi-mode heterogeneous data sources provided in an embodiment of the present application;
[0025] Figure 3 A schematic diagram of the process flow of the Jupiter verifier in a data authority control method based on multi-mode heterogeneous data sources provided in an embodiment of the present application;
[0026] Figure 4 A Jupiter permission check timing diagram in a data permission control method based on multi-mode heterogeneous data sources provided in an embodiment of the present application;
[0027] Figure 5 A flow chart of implementing dynamic data authority control by using filters in a data authority control method based on multi-mode heterogeneous data sources provided in an embodiment of the present application;
[0028] Figure 6 A schematic diagram of a syntax tree corresponding to jupiterSelect in a data permission control method based on multi-mode heterogeneous data sources provided in an embodiment of the present application;
[0029] Figure 7 A schematic diagram of the relationship between a filter manager, filters and policies in a data permission control method based on multi-mode heterogeneous data sources provided in an embodiment of the present application;
[0030] Figure 8 A schematic diagram of the conversion process of a policy converter in a data authority control method based on multi-mode heterogeneous data sources provided in an embodiment of the present application;
[0031] Fig. 9A schematic diagram of the conversion process of an operator converter in a data authority control method based on multi-mode heterogeneous data sources provided in an embodiment of the present application;
[0032] Fig.10 A schematic diagram of the structure of a data authority control device based on multi-mode heterogeneous data sources provided in an embodiment of the present application;
[0033] Fig.11 A structural block diagram of a computing device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0034] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.
[0035] See also Figure 1 , Figure 1 The following is a schematic diagram of the architecture of a data authority control system based on multi-mode heterogeneous data sources provided in an embodiment of the present application. Figure 1 As shown, the data permission control system of the multi-mode heterogeneous data source includes a user (user end), which can be an Actor actor; and a Jupiter distributed computing engine.
[0036] Specifically, the Jupiter distributed computing engine includes Jupiter's SQL parser, permission controller, and distributed executor.
[0037] The permission controller includes a four-level metadata verifier, a Jupiter policy converter, and a Jupiter operator converter.
[0038] In the embodiment of the present application, the flowchart of the data permission control method based on multi-mode heterogeneous data sources can be found in Figure 2 ,in, Figure 2 A flowchart of a data authority control method based on multi-mode heterogeneous data sources provided in an embodiment of the present application. Figure 2 As shown, the specific steps include the following steps.
[0039] Step S202: Abstract the user query SQL statement into four layers of metadata through Jupiter's SQL parser, and send the four layers of metadata to the data permission controller, where the four layers of metadata include datasource layer, schema layer, table layer and column layer; the data permission controller is designed based on the Jupiter computing engine, including a metadata permission controller, a filter, a policy converter and an operator converter.
[0040] Step S204: Use the four-level metadata verifier to verify the data permissions corresponding to the four layers of metadata. When the metadata at the datasource layer, schema layer, table layer and column layer all pass the permission verification, use the pluggable filter to obtain the data strategy.
[0041] In an embodiment of the present application, the use of a four-level metadata verifier to verify the data permissions corresponding to the obtained four-layer metadata includes: based on the four-level metadata verifier, encapsulating the obtained four-layer metadata into a Resource entity, wherein the Resource entity includes a datasource entity, a schema entity, a table entity, and a column entity; binding the user to the Resource, and using the ranger's storage strategy to verify the data permissions corresponding to the obtained four-layer metadata.
[0042] In actual applications, Jupiter, as the computing engine of the data vault, connects to a variety of different data sources such as relational databases, nosql databases, newsql databases, and data lakes. The table space of each data source is different. In order to achieve unified permission management of these different data sources, the embodiment of this application designs a four-level permission controller based on datasource, schema, table, and column. The controller accepts the results of Jupiter's SQL parser, and uses Jupiter's permission verifier to encapsulate the SQL parsing results into a Resource entity. The resource entity is composed of datasource, schema, table, and column component entities. The user and resource are bound, and the ranger storage strategy is used to implement metadata permission verification.
[0043] See also Figure 3 , Figure 3 A schematic diagram of the process flow of the Jupiter verifier in a data authority control method based on multi-mode heterogeneous data sources provided in an embodiment of the present application. Figure 3 As shown, the main verification process of the Jupiter verifier is as follows.
[0044] Perform permission check on datasource metadata. If datasource metadata passes permission check, then check schema metadata. Otherwise, the operation is not allowed. If schema metadata passes permission check, then check table metadata. Otherwise, the operation is not allowed. If table metadata passes permission check, then check column metadata. Otherwise, the operation is not allowed. If column metadata passes permission check, run the operation. Otherwise, the operation is not allowed.
[0045] In addition, the present application embodiment also provides a Jupiter permission check timing diagram, see Figure 4 .
[0046] In the embodiment of the present application, the strategy of obtaining data by using a pluggable filter includes:
[0047] The received user query SQL statement is processed by the Jupiter syntax parser, and the corresponding SQL string is converted into a jupiterSelect object. Based on the jupiterSelect object, the jupiter syntax parser communicates with the Filter manager to check whether the user has registered a filter. If the user has registered a filter, the filter policy corresponding to the filter is obtained, and the filter policy is added to the jupiter policy converter, wherein the JupiterSelect object carries a condition attribute for controlling the user's data authority;
[0048] The filtering strategy corresponding to the filter is registered to the child node of the where node of the result jupiterSelect tree of the Jupiter syntax parser through the Jupiter strategy converter to form a new jupiterSelect parsing tree;
[0049] The Jupiter operator converter converts the child nodes of the where node into the Jupiter filter operator to generate a Jupiter logical plan. The Jupiter optimizer pushes the filter operator down to the data source to generate a physical plan for execution. When querying data, the data source filters out the data that meets the filter policy, making the data invisible to users and achieving the purpose of controlling data permissions.
[0050] Jupiter uses pluggable filters to implement dynamic data permission control and implements different data views for different users through user registration filter strategies.
[0051] See also Figure 5 , Figure 5A flow chart of implementing dynamic data permission control using filters in a data permission control method based on multi-mode heterogeneous data sources provided in an embodiment of the present application.
[0052] First, the Jupiter parser is used to parse the user SQL and convert it into a Jupiter Select syntax tree.
[0053] Then, query the filters registered by the user through the filter manager;
[0054] Secondly, determine whether there is a filter plug-in;
[0055] Secondly, get the user filter policy and register it to the Jupiter converter;
[0056] Secondly, the converter accepts the sqlNode of the sql parser, registers the policy to the child node of the JupiterSelect tree, and converts it into a new JupiterSelect;
[0057] Secondly, use the Jupiter operator converter to convert JupiterSelect into a physical operator and the filter into Jupiter's filter operator;
[0058] Secondly, through the Jupiter optimizer, the filter operator is pushed down to the data source for storage;
[0059] Finally, determine whether the data source meets the filtering conditions. If so, the data is not visible; if not, the data is visible.
[0060] In the embodiment of the present application, Jupiter's parser mainly converts the user's query SQL statement into a jupiterSelect object. The jupiterSelect object is an abstraction of the SQL syntax parsing tree. JupiterSelect has a condition attribute for controlling the user's data permissions. The syntax tree corresponding to jupiterSelect can be found in Figure 6 .
[0061] The Filter manager is responsible for managing the operations of pluggable filters, including registration, updating, obtaining and deleting filters. In the embodiment of the present application, the same filter can be registered for different users, or different filters can be registered for different tenants and users.
[0062] Filters are dynamic and pluggable as the key components of Jupiter's permission control. Users can register different filters according to their permission requirements. Filters consist of policies and user information. Jupiter has built-in ranger and memory-based filters, and also supports user-defined filters. For the relationship between filter manager, filters, and policies, see Figure 7 .
[0063] In an embodiment of the present application, the filtering policy corresponding to the filter is registered in the child node of the where node of the jupiterSelect tree of the result of the Jupiter syntax parser through the Jupiter strategy converter to form a new jupiterSelect parse tree, including: based on the communication between the strategy converter and the filter manager, according to the object information, the filter registered by the user and the corresponding filtering policy are obtained, and a user policy list is constructed, wherein the object information carries tenant information or user information; the sqlNode of the jupiter parser is received through the strategy converter, the Condition attribute of the jupiterSelect is reconstructed using the strategy list, and the obtained Condition attribute is correspondingly registered in the child node of the where node of the jupiterSelect tree of the result of the Jupiter syntax parser to form a new jupiterSelect parse tree.
[0064] The policy converter is the core component of the permission control. It is responsible for registering the filter policy to the condition of jupiterSelect and building a new jupiterSelect as the input of the jupiter operator converter. Figure 8 The conversion process is as follows: The policy converter communicates with the filter manager, obtains the user-registered filters and corresponding policies based on the tenant or user information, and builds the user policy list List <policy>lp; strategy Converter, accepts sqlNode of Jupiter's parser, using strategy list List <policy>lp, rebuild the Condition property of jupiterSelect.
[0065] In the embodiment of the present application, the filter operator is pushed down to the data source through the Jupiter optimizer to generate a physical plan for execution. When querying data, the data source filters out the data that meets the filter strategy, including: pushing the filter conditions of the filter operator to different data sources through the Jupiter optimizer, and converting the filter into corresponding conditional filtering expressions for different heterogeneous data sources; when querying data, the data source filters out the data that meets the filter strategy.
[0066] The operator converter is responsible for converting jupiterSelect into the corresponding execution plan. The filter strategy will be converted into the filter operator of jupiter. For specific conversion, see Fig. 9 .
[0067] Jupiter's optimizer pushes the filter conditions of the filter operator to different data sources, and converts the filter to the corresponding conditional filter expression for different heterogeneous data sources. For example, traditional relational databases will be converted into corresponding where conditions, and nosql such as hbase will be converted into hbase corresponding filters. Finally, the data that meets the filter conditions will be filtered out in the data source, so as to ensure that the data is not visible to users and achieve the purpose of permission control.
[0068] Step S206: convert the data policy into the data visibility condition through the policy converter, and register the where child node of the jupiterSelect syntax tree.
[0069] Step S208: Use the operator converter to convert the data visibility conditions into Jupiter's filter operator, and push the data visibility conditions to the multi-mode data source through the Jupiter optimizer to control data permissions.
[0070] The data permission control method based on multi-mode heterogeneous data sources provided in the embodiment of the present application is applied. On the basis of the Jupiter computing engine, a data permission controller is designed. The controller is composed of a metadata permission controller, a filter, a policy converter and an operator converter component. The Jupiter permission manager abstracts different data sources into four layers: datasource, schema, table and column, and uses a four-level permission controller to implement metadata permission verification to ensure unified permission verification of multi-mode queries. After the metadata permission is passed, a pluggable filter is used to obtain the data policy; the policy converter converts the policy into a data visibility condition and registers the where child node of the JupiterSelect syntax tree; finally, the operator converter is used to convert the data visibility condition into the Jupiter filter operator, and the Jupiter optimizer pushes the data visibility condition to the multi-mode data source to implement data permission control.
[0071] Corresponding to the above method embodiment, this specification also provides an embodiment of a data authority control device based on multi-mode heterogeneous data sources, Fig.10 The present invention provides a schematic diagram of a data authority control device based on a multi-mode heterogeneous data source. Fig.10 As shown, the device comprises:
[0072] The sending module 1002 is configured to abstract the user query SQL statement into four layers of metadata through the Jupiter SQL parser, and send the four layers of metadata to the data authority controller, wherein the four layers of metadata include datasource layer, schema layer, table layer and column layer; the data authority controller is designed based on the Jupiter computing engine, and includes a metadata authority controller, a filter, a strategy converter and an operator converter;
[0073] The acquisition module 1004 is configured to use a four-level metadata checker to verify the data permissions corresponding to the obtained four-layer metadata, and use a pluggable filter to obtain the data strategy when the metadata at the datasource layer, schema layer, table layer and column layer all pass the permission verification;
[0074] A registration module 1006 is configured to convert the data policy into a data visibility condition through a policy converter, and register a where child node of the jupiterSelect syntax tree;
[0075] The control module 1008 is configured to use an operator converter to convert the data visibility conditions into Jupiter's filter operator, and push the data visibility conditions to the multi-mode data source through the Jupiter optimizer to control data permissions.
[0076] In an optional embodiment, the acquisition module 1004 is further configured to:
[0077] Based on the four-level metadata verifier, the obtained four-level metadata is encapsulated into a Resource entity, where the Resource entity includes datasource entity, schema entity, table entity and column entity;
[0078] Bind the user to the resource and use the ranger's storage strategy to verify the data permissions corresponding to the obtained four-layer metadata.
[0079] In an optional embodiment, the acquisition module 1004 is further configured to:
[0080] Perform permission check on datasource metadata. If datasource metadata passes permission check, then perform verification on schema metadata. Otherwise, the operation is not allowed.
[0081] If the schema metadata passes the permission check, the table metadata is checked. Otherwise, the operation is not allowed.
[0082] If the table metadata passes the permission check, the column metadata is checked, otherwise the operation is not allowed;
[0083] If the column metadata passes the permission check, the operation is performed; otherwise, the operation is not allowed.
[0084] In an optional embodiment, the acquisition module 1004 is further configured to:
[0085] The received user query SQL statement is processed by the Jupiter syntax parser, and the corresponding SQL string is converted into a jupiterSelect object. Based on the jupiterSelect object, the jupiter syntax parser communicates with the Filter manager to check whether the user has registered a filter. If the user has registered a filter, the filter policy corresponding to the filter is obtained, and the filter policy is added to the jupiter policy converter, wherein the JupiterSelect object carries a condition attribute for controlling the user's data authority;
[0086] The filtering strategy corresponding to the filter is registered to the child node of the where node of the result jupiterSelect tree of the Jupiter syntax parser through the Jupiter strategy converter to form a new jupiterSelect parsing tree;
[0087] The Jupiter operator converter converts the child nodes of the where node into the Jupiter filter operator to generate a Jupiter logical plan. The Jupiter optimizer pushes the filter operator down to the data source to generate a physical plan for execution. When querying data, the data source filters out the data that meets the filter strategy.
[0088] In an optional embodiment, the acquisition module 1004 is further configured to:
[0089] Based on the communication between the policy converter and the filter manager, the filter registered by the user and the corresponding filtering policy are obtained according to the object information, and a user policy list is constructed, wherein the object information carries tenant information or user information;
[0090] Receive the sqlNode of the jupiter parser through the strategy converter, use the strategy list to rebuild the Condition attribute of jupiterSelect, and register the obtained Condition attribute to the child node of the where node of the result jupiterSelect tree of the Jupiter syntax parser accordingly to form a new jupiterSelect parsing tree.
[0091] In an optional embodiment, the acquisition module 1004 is further configured to:
[0092] Through the Jupiter optimizer, the filter operator's filter conditions are pushed down to different data sources, and the filter is converted into corresponding conditional filter expressions for different heterogeneous data sources;
[0093] When querying data, the data source filters out the data that meets the filtering strategy.
[0094] The data permission control method based on multi-mode heterogeneous data sources provided in the embodiment of the present application is applied. On the basis of the Jupiter computing engine, a data permission controller is designed. The controller is composed of a metadata permission controller, a filter, a policy converter and an operator converter component. The Jupiter permission manager abstracts different data sources into four layers: datasource, schema, table and column, and uses a four-level permission controller to implement metadata permission verification to ensure unified permission verification of multi-mode queries. After the metadata permission is passed, a pluggable filter is used to obtain the data policy; the policy converter converts the policy into a data visibility condition and registers the where child node of the JupiterSelect syntax tree; finally, the operator converter is used to convert the data visibility condition into the Jupiter filter operator, and the Jupiter optimizer pushes the data visibility condition to the multi-mode data source to implement data permission control.
[0095] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the data permission control device based on a multi-mode heterogeneous data source, since it is basically similar to the data permission control method embodiment based on a multi-mode heterogeneous data source, the description is relatively simple, and the relevant parts can be referred to the partial description of the data permission control method embodiment based on a multi-mode heterogeneous data source.
[0096] Fig.11 This is a block diagram of a computing device provided in an embodiment of the present application. The components of the computing device 1100 include but are not limited to a memory 1110 and a processor 1120. The processor 1120 is connected to the memory 1110 via a bus 1130, and the database 1150 is used to store data.
[0097] The computing device 1100 also includes an access device 1140, which enables the computing device 1100 to communicate via one or more networks 1160. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 1140 may include one or more of any type of network interface (e.g., a network interface card (NIC)) of wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a world-wide interoperability for microwave access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, and a near field communication (NFC).
[0098] In one embodiment of the present specification, the above components of the computing device 1100 and Fig.11 Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Fig.11 The computing device structure block diagram shown is only for the purpose of illustration, and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.
[0099] The computing device 1100 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or a personal computer (PC). The computing device 1100 may also be a mobile or stationary server.
[0100] Among them, the processor 1120 is used to execute the following computer executable instructions, which, when executed by the processor, implement the steps of the above-mentioned data authority control method based on multi-mode heterogeneous data sources.
[0101] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the computing device embodiment, since it is basically similar to the data permission control method embodiment based on multi-mode heterogeneous data sources, the description is relatively simple, and the relevant parts can be referred to the partial description of the data permission control method embodiment based on multi-mode heterogeneous data sources.
[0102] An embodiment of the present specification also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the steps of the above-mentioned data permission control method based on multi-mode heterogeneous data sources.
[0103] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the computer-readable storage medium embodiment, since it is basically similar to the data permission control method embodiment based on multi-mode heterogeneous data sources, the description is relatively simple, and the relevant parts can be referred to the partial description of the data permission control method embodiment based on multi-mode heterogeneous data sources.
[0104] An embodiment of the present specification also provides a computer program, wherein when the computer program is executed in a computer, the computer is caused to execute the steps of the above-mentioned data authority control method based on multi-mode heterogeneous data sources.
[0105] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the computer program embodiment, since it is basically similar to the data permission control method embodiment based on multi-mode heterogeneous data sources, the description is relatively simple, and the relevant parts can be referred to the partial description of the data permission control method embodiment based on multi-mode heterogeneous data sources.
[0106] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
[0107] The computer instructions include computer program codes, which may be in source code form, object code form, executable files or some intermediate forms, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.
[0108] It should be noted that the above is a description of a specific embodiment of the present specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the embodiments of the present specification.
[0109] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
[0110] The preferred embodiments of this specification disclosed above are only used to help explain this specification. The optional embodiments do not describe all the details in detail, nor do they limit the invention to only the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of the embodiments of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the embodiments of this specification, so that technicians in the relevant technical field can well understand and use this specification. This specification is only limited by the claims and their full scope and equivalents.< / policy> < / policy>
Claims
1. A data authority control method based on multi-mode heterogeneous data sources, characterized in that: include: The user query SQL statement is abstracted into four layers of metadata through Jupiter's SQL parser, and the four layers of metadata are sent to the data authority controller, where the four layers of metadata include datasource layer, schema layer, table layer and column layer; the data authority controller is designed based on the Jupiter computing engine, including metadata authority controller, filter, strategy converter and operator converter; Use the four-level metadata verifier to verify the data permissions corresponding to the four-layer metadata. When the metadata at the datasource layer, schema layer, table layer, and column layer all pass the permission verification, use the pluggable filter to obtain the data strategy; The data strategy is converted into data visibility conditions through the strategy converter, and the where child node of the jupiterSelect syntax tree is registered; The operator converter is used to convert the data visibility conditions into Jupiter's filter operator, and the data visibility conditions are pushed to the multi-mode data source through the Jupiter optimizer to control data permissions.
2. The method according to claim 1, characterized in that The method of using the four-level metadata checker to check the data permissions corresponding to the obtained four-level metadata includes: Based on the four-level metadata verifier, the obtained four-level metadata is encapsulated into a Resource entity, where the Resource entity includes datasource entity, schema entity, table entity and column entity; Bind the user to the resource and use the ranger's storage strategy to verify the data permissions corresponding to the obtained four-layer metadata.
3. The method according to claim 2, characterized in that The binding user and resource uses the ranger storage strategy to verify the data permissions corresponding to the obtained four-layer metadata, including: Perform permission check on datasource metadata. If datasource metadata passes permission check, then perform verification on schema metadata. Otherwise, the operation is not allowed. If the schema metadata passes the permission check, the table metadata is checked. Otherwise, the operation is not allowed. If the table metadata passes the permission check, the column metadata is checked, otherwise the operation is not allowed; If the column metadata passes the permission check, the operation is performed; otherwise, the operation is not allowed.
4. The method according to claim 1, characterized in that: The strategy of obtaining data by using pluggable filters includes: The received user query SQL statement is processed by the Jupiter syntax parser, and the corresponding SQL string is converted into a jupiterSelect object. Based on the jupiterSelect object, the jupiter syntax parser communicates with the Filter manager to check whether the user has registered a filter. If the user has registered a filter, the filter policy corresponding to the filter is obtained, and the filter policy is added to the jupiter policy converter, wherein the JupiterSelect object carries a condition attribute for controlling the user's data authority; The filtering strategy corresponding to the filter is registered to the child node of the where node of the result jupiterSelect tree of the Jupiter syntax parser through the Jupiter strategy converter to form a new jupiterSelect parsing tree; The Jupiter operator converter converts the child nodes of the where node into the Jupiter filter operator to generate a Jupiter logical plan. The Jupiter optimizer pushes the filter operator down to the data source to generate a physical plan for execution. When querying data, the data source filters out the data that meets the filter strategy.
5. The method according to claim 4, characterized in that The filtering strategy corresponding to the filter is registered to the child node of the where node of the jupiterSelect tree of the result of the Jupiter syntax parser through the Jupiter strategy converter to form a new jupiterSelect parsing tree, including: Based on the communication between the policy converter and the filter manager, the filter registered by the user and the corresponding filtering policy are obtained according to the object information, and a user policy list is constructed, wherein the object information carries tenant information or user information; Receive the sqlNode of the jupiter parser through the strategy converter, use the strategy list to rebuild the Condition attribute of jupiterSelect, and register the obtained Condition attribute to the child node of the where node of the result jupiterSelect tree of the Jupiter syntax parser accordingly to form a new jupiterSelect parsing tree.
6. The method according to claim 4, characterized in that The filter operator is pushed down to the data source through the Jupiter optimizer to generate a physical plan for execution. When querying data, the data source filters out data that meets the filter strategy, including: Through the Jupiter optimizer, the filter operator's filter conditions are pushed down to different data sources, and the filter is converted into corresponding conditional filter expressions for different heterogeneous data sources; When querying data, the data source filters out the data that meets the filtering strategy.
7. A data authority control device based on multi-mode heterogeneous data sources, characterized in that: include: The sending module is configured to abstract the user query SQL statement into four layers of metadata through Jupiter's SQL parser, and send the four layers of metadata to the data authority controller, wherein the four layers of metadata include datasource layer, schema layer, table layer and column layer; the data authority controller is designed based on the Jupiter computing engine, and includes a metadata authority controller, a filter, a strategy converter and an operator converter; The acquisition module is configured to use the four-level metadata verifier to verify the data permissions corresponding to the obtained four-layer metadata. When the metadata at the datasource layer, schema layer, table layer, and column layer all pass the permission verification, the pluggable filter is used to obtain the data strategy; The registration module is configured to convert the data strategy into the data visibility condition through the strategy converter and register the where child node of the jupiterSelect syntax tree; The control module is configured to use the operator converter to convert the data visibility conditions into Jupiter's filter operator, and push the data visibility conditions to the multi-mode data source through the Jupiter optimizer to control the data permissions.
8. A computing device comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the method described in any one of claims 1 to 6 are implemented.
9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores an implementation program for information transmission, and when the program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.