A Hive permission control method, device and equipment and readable storage medium

Abstract

The application discloses a Hive permission control method, which extracts the library name, table name and column name of a data object requested to be accessed in Hive storage after receiving a Hive operation request initiated by a client, splices a request operation type as an access dimension, and realizes Hive permission verification according to whether the dimension value corresponding to the access dimension contains a current user group. The dimension value verified in the verification method is accurate to the library name, table name and column name, can realize table field level authentication, and completes fine-grained control of data permission. Meanwhile, the method directly manages permission through operation dimensions after receiving the client operation, and does not need to limit the operation of the client, so that the integrity of the user operation can be guaranteed. In addition, the method can support reuse and expansion of Hive. The application also discloses a Hive permission control device, equipment and readable storage medium, which have corresponding technical effects.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of security assurance, in particular to a Hive permission control method, device and equipment and readable storage medium. BACKGROUND

[0002] Hive is a data warehouse tool based on Hadoop (a kind of distributed system infrastructure), which can map structured data files into a database table and provide complete SQL (Structured Query Language) query function, and can convert SQL statements into MapReduce (a kind of programming model) tasks for running. Because of its low learning cost, simple MapReduce statistics can be quickly realized through SQL-like statements, and special MapReduce applications do not have to be developed, so it is very suitable for statistical analysis of data warehouse.

[0003] At present, there are various schemes that can be used for Hive table permission management, such as Hive default authorization, Hive authorization based on SQL standard, Range & Sentry, etc. However, various schemes have different degrees of defects and problems in actual application, such as Hive default authorization provides a control system based on table and table field permissions, but cannot realize comprehensive permission control, which may cause missed detection; the Hive authorization scheme based on SQL standard only provides a control system based on table permission level, which cannot realize fine-grained permission control, and at the same time, the scheme limits user operations and cannot guarantee the integrity of user operations; Range & Sentry is a third-party open source software, which provides policy-based permission management, although it can realize column-level permission management, but it is not compatible with a large number of security components and cannot be used directly, in addition, it cannot support Spark (a kind of computing engine) reuse of Hive table authentication, and is difficult to reuse and extend.

[0004] In summary, how to control fine-grained data permissions while fully guaranteeing the integrity of user operations, realizing comprehensive and effective permission control, and facilitating reuse and extension, is a technical problem that needs to be solved by the technical personnel in the field at present. SUMMARY

[0005] The purpose of the present application is to provide a Hive permission control method, device, equipment and readable storage medium, so as to control fine-grained data permissions while fully guaranteeing the integrity of user operations, realizing comprehensive and effective permission control, and facilitating reuse and extension.

[0006] To solve the above technical problems, the present application provides the following technical solutions:

[0007] A Hive permission control method, comprising:

[0008] After a server receives a Hive operation request initiated by a client, determining a user group initiating the operation request, a data object requested to operate, and an operation type;

[0009] Extracting a library name, a table name, and a column name of the data object, and concatenating the operation type as an access dimension;

[0010] Querying a dimension value corresponding to the access dimension; wherein the dimension value indicates a user group with permission;

[0011] Determining whether the dimension value includes the user group;

[0012] If yes, determining that the user group has table permission; if no, determining that the permission is abnormal.

[0013] Optionally, after the determination that the user group has table permission, further comprising:

[0014] Determining whether the operation request includes path information;

[0015] If yes, performing path permission verification on the path information to generate a path verification result;

[0016] If the path verification result shows that the verification fails, performing the step of determining that the permission is abnormal;

[0017] If the path verification result shows that the verification passes, submitting a task corresponding to the operation request so as to execute the operation request.

[0018] Optionally, after the task corresponding to the operation request is submitted to Hadoop according to the operation request, further comprising:

[0019] If the operation request is a storage operation, after the operation request is executed successfully, adding a new data access dimension, and adding a corresponding user group with permission in a dimension value of the new data access dimension;

[0020] If the operation request is a deletion operation, traversing and deleting access dimensions and dimension values corresponding to data to be deleted;

[0021] If the operation request is a modification operation, adding a data access dimension after modification, and shifting a dimension value corresponding to a data access dimension before modification to the data access dimension after modification, and traversing and deleting the dimension value corresponding to the data access dimension before modification.

[0022] Optionally, after the new data access dimension is added, and the corresponding user group with permission is added in the dimension value of the new data access dimension, further comprising:

[0023] publish the new data access dimension and new data information to a data market, so as to receive a new data operation application.

[0024] Optionally, the adding of the new data access dimension and the adding of the corresponding authorized user group to the dimension value of the new data access dimension comprise:

[0025] creating a full table read-write dimension or a field read dimension according to a storage location of the new data, as the new data access dimension;

[0026] adding the user group as a data owner to a full table write dimension in the new data access dimension as a dimension value;

[0027] after receiving an authorization application of a first user group, if the authorization application is approved by the data owner, adding the first user group as a dimension value to the new data access dimension; wherein the first user group is not the data owner.

[0028] Optionally, after the adding of the first user group as a dimension value to the new data access dimension, the method further comprises:

[0029] recording an expiration time of the authorization of the first user group;

[0030] checking whether the expiration time is reached in a timely manner;

[0031] if the expiration time is reached, deleting the first user group from the dimension value of the new data access dimension.

[0032] A Hive permission control device applied to a Hive server side, the device comprises:

[0033] a request determination unit configured to determine a user group initiating a Hive operation request, a data object requested to be operated and an operation type after receiving the Hive operation request initiated by a client;

[0034] a dimension extraction unit configured to extract a library name, a table name and a column name of the data object, and splice the operation type as an access dimension;

[0035] a dimension value query unit configured to query a dimension value corresponding to the access dimension; wherein the dimension value indicates an authorized user group;

[0036] a dimension value authentication unit configured to determine whether the user group is included in the dimension value; if the user group is included, it is determined that the user group has a table permission; if the user group is not included, it is determined that the authorization is abnormal.

[0037] Optionally, the Hive permission control device further comprises a path authentication unit.

[0038] The path authentication unit is connected to the output end of the dimension value authentication unit, for judging whether path information is included in the operation request after judging that the user group exists table permission; if yes, carrying out path permission check on the path information, and generating a path check result; if the path check result shows that the check fails, executing the step of judging that the permission is abnormal; if the path check result shows that the check passes, submitting a task corresponding to the operation request, so as to execute the operation request.

[0039] A computer device comprises:

[0040] A memory for storing a computer program;

[0041] A processor for executing the computer program to realize the steps of the Hive permission control method.

[0042] A readable storage medium, wherein the readable storage medium stores a computer program, and the computer program is executed by a processor to realize the steps of the Hive permission control method.

[0043] The method provided by the embodiment of the present application extracts the library name, table name and column name of the data object accessed by the request in the Hive storage after receiving the Hive operation request initiated by the client, splices the request operation type as the access dimension, and realizes the Hive permission check according to whether the dimension value corresponding to the access dimension contains the current user group. In the check method, the dimension value checked is accurate to the library name, table name, column name and operation type, so that the table field level authentication can be realized, the fine-grained control of the data permission is completed, and the comprehensiveness of the permission control is ensured. Meanwhile, in the method, the permission is managed through the operation dimension directly after receiving the client operation, and the operation of the client does not need to be limited, so that the integrity of the user operation can be ensured. In addition, the method realizes the Hive permission control by calling the internal component of the Hive service end, so that when the Hive is reused, the permission control method realized by calling the internal component of the Hive can be compatible, and the reuse and expansion of the Hive can be supported. Therefore, the method can ensure the integrity of the user operation while realizing the fine-grained data permission control, and realizes the comprehensive and effective permission control.

[0044] Correspondingly, the embodiment of the present application also provides a Hive permission control device, a computer device and a readable storage medium corresponding to the Hive permission control method, which have the above technical effects, and details are not repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0045] In order to make the technical scheme of the present application or the related art clearer, the accompanying drawings needed in the embodiment or the related art description will be briefly introduced. Obviously, the accompanying drawings described are only some embodiments of the present application, and all other embodiments obtained by those of ordinary skill in the art without any creative work on the premise of the embodiments in the present application are within the scope of the present application.

[0046] Figure 1 An implementation flowchart of the Hive permission control method in the embodiment of the present application is shown in the figure.

[0047] Figure 2 A dimension diagram in the embodiment of the present application is shown in the figure.

[0048] Figure 3 A relationship diagram between the dimension and the dimension value in the embodiment of the present application is shown in the figure.

[0049] Figure 4 A service mode diagram in the embodiment of the present application is shown in the figure.

[0050] Figure 5 A dimension maintenance diagram in the embodiment of the present application is shown in the figure.

[0051] Figure 6 A structural diagram of the Hive permission control device in the embodiment of the present application is shown in the figure.

[0052] Figure 7 A diagram of the Hive permission control device calling each component to realize permission control in the embodiment of the present application is shown in the figure.

[0053] Figure 8 A structural diagram of the computer device in the embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0054] The core of the present application is to provide a Hive permission control method, which can guarantee the integrity of user operation while controlling data permission in fine granularity, realize comprehensive and effective permission control, and facilitate reuse and expansion.

[0055] In order to make the technical scheme of the present application or the related art clearer, the accompanying drawings needed in the embodiment or the related art description will be briefly introduced. Obviously, the accompanying drawings described are only some embodiments of the present application, and all other embodiments obtained by those of ordinary skill in the art without any creative work on the premise of the embodiments in the present application are within the scope of the present application.

[0056] Although there are many solutions available for Hive table permission management, there are different degrees of defects and problems in practical application. The following three kinds of Hive table permission management methods are introduced.

[0057] (1) Default Hive Authorization (Legacy Mode)

[0058] This model is based on the authorization mode similar to the SQL standard, which provides grant, revoke statement access control. By creating a view and authorizing the view (rather than authorizing the tables on which the view depends), it provides fine-grained access control, i.e. column-level permission control. This scheme provides a control system based on table and table field permissions.

[0059] Legacy mode is the most basic authorization scheme, and the design purpose is not to prevent malicious users from accessing and operating unauthorized data, but to help users avoid some unexpected operation behaviors. It does not have a complete access control model, so many security vulnerabilities are not addressed. For example, authorization confirmation is not performed in many instructions; users are allowed to execute dfs instructions, user-defined functions, and shell instructions, but these special operations may bypass the client's security mechanisms, so this method cannot achieve comprehensive permission control and may cause missed detection, making it difficult to ensure the security of Hive data.

[0060] (2) SQL Standards Based Hive Authorization

[0061] Like the default authorization mechanism of Hive, authorization confirmation in the SQL Standards Based Hive Authorization occurs during the compilation phase of the SQL statement. This authorization method is fully compatible with the authorization model of SQL and will not cause backward compatibility problems for current users, so it is widely used. This scheme provides a control system based on table permission level.

[0062] To ensure that the authorization model has a security effect, the client also needs security guarantees, so this mode is limited, which restricts some client instructions that may have unsafe factors, such as: user access must be and only through HiveServer2, and user code and non-SQL instructions are restricted from being executed, so this method restricts user operations and cannot guarantee the integrity of user operations; moreover, this method does not support table field level-based authorization, but only supports table level, and cannot achieve fine-grained permission control.

[0063] (3) Range & Sentry

[0064] Range & Sentry is an open source software for third party, which provides policy-based permission management, that is, adding Hive Service, then adding custom policy for the service, such as access granularity Database, Table, Column, and then adding group or user access permission such as Select, Create, Drop, etc.

[0065] Although Ranger & Sentry can realize column-level permission management, it is incompatible with a large number of security components and cannot be directly used, and it also cannot support Spark reuse of authentication for Hive tables, and is difficult to reuse and extend.

[0066] Therefore, the purpose of the present application is to provide a Hive permission control method to meet the following needs:

[0067] (1) Realize Hive table field-level authentication, and realize the most fine-grained data permission control.

[0068] (2) The authentication does not limit user operations, such as allowing users to execute dfs instructions, user-defined functions and shell instructions, and fully guarantees the integrity of user operations.

[0069] (3) Facilitate reuse and extension, such as Spark can reuse the permission model to complete the authentication process.

[0070] Please refer to Figure 1 , Figure 1 The flowchart of a Hive permission control method in the embodiment of the present application, the method comprises the following steps:

[0071] S101, after the service end receives the Hive operation request initiated by the client, the user group initiating the operation request, the data object of the request operation and the operation type are determined;

[0072] The user logs in to the Hive service end (in this embodiment, HiveServer2 is taken as an example) through the client (such as Beeline, JDBC (Java DataBase Connectivity, java database connection) and the like), and initiates a Hive operation request (SQL).

[0073] The user roles in the method mainly include: user and user group (RCC). The user is a subject of accessing or operating resource entities such as a business system. If the use scenarios of some users are consistent, the users can be packaged into a group, which is referred to as an RCC group. There can be multiple users in a group, and a user can join multiple groups. The permission management of the Hive table is limited in the dimension of the RCC group. When the user logs in to the Hive server through the client, the user first connects to the Zookeeper service to obtain one of the highly available HiveServer2 nodes. The client replaces the user group (RCC) with the user information in the current user Kerberos cache to connect the HiveServer2 in the identity of the RCC group.

[0074] The HiveServer2 parses the operation request after receiving the operation request. The method mainly checks the fields including the user group (RCC group) initiating the operation request, the data object (data of a column of a table in a library) of the request operation, and the operation type. The data object of the request operation specifically includes the library name, table name, and column name of the data object.

[0075] The fields checked in the method are refined from the library name, table name to the column name. The extraction of the library name and table name can realize the table-level authentication, and the refinement to the column name can realize the field-level authentication. Therefore, the method can realize the Hive table field-level authentication and realize the most fine-grained control of data permissions. Of course, other information such as operation time can be further extracted for verification, which is not limited herein.

[0076] S102, the library name, table name, and column name of the data object are extracted, and the operation type is spliced to serve as an access dimension;

[0077] The fields (library name, table name, column name, and operation type) involved in the verification are spliced into a dimension, that is, an access dimension, so as to determine the user group (dimension value) having the permission corresponding to the access dimension to verify the current user group according to the access dimension.

[0078] The method uses "dimension" to describe the data access control. The dimension refers to the table and its field. As follows Figure 2As described in the foregoing embodiment, the dimensions of a Hive table can be classified into full table dimensions and column dimensions, and the full table dimensions can be further classified into full table read-write dimensions "database name.table name.-.rw" and full table read dimensions "database name.table name.-.ro", and the column dimensions are only read-enabled expressions "database name.table name.column name.ro". In the present embodiment, the splicing rules of the dimensions are not limited, and the dimensions are spliced according to the rule "database name.table name.column name.operation type" in the present embodiment, and other splicing rules can be referred to the description of the present embodiment, and will not be described herein.

[0079] S103, querying the dimension value corresponding to the access dimension;

[0080] After the Hive table access dimension that the current user wants to operate is determined, the access dimension and the maintenance component of the dimension value are queried to determine the dimension value corresponding to the access dimension.

[0081] The relationship between the dimensions and the dimension values can be referred to the following table Figure 3 In the present method, the "dimension value" is used to represent the user group that has the dimension permission, one dimension can have multiple dimension values, and one user group (i.e., the dimension value in the figure) can apply for the permission of multiple dimensions, but the write dimension of the full table has only one dimension value, which is the user who creates the table, i.e., the table owner.

[0082] S104, judging whether the dimension value includes the user group, if yes, performing S105, and if not, performing S106;

[0083] The dimension value corresponding to the current access dimension indicates the user group that has the corresponding operation permission on the current operation object, and it is judged whether the user group that initiates the operation request currently includes the user group. If yes, it proves that the current user belongs to the user group that has the permission of the operation object, and S105 is performed to determine whether the user has the table permission; if not, it proves that the current user does not belong to the user group that has the permission of the operation object, and S106 is performed to determine whether the permission is abnormal.

[0084] The data object of the request operation can be one or multiple, for example, the user submits the operation request "select col1, col2, col3 from db.tb1;", and the dimension values under the three column dimensions involved are queried in the authentication, and the single access object is mainly introduced in the present embodiment, and the authentication rules of the single access object can be referred to for the authentication of each access object in the case of multiple access objects, and if one or more do not have the permission, the permission exception information is thrown, and it is determined that the permission is abnormal.

[0085] S105, determining whether the user has the table permission;

[0086] S106, determining whether the permission is abnormal.

[0087] It should be noted that the Hive permission control method provided in the embodiment can be realized by calling a Hive internal component or a third-party component compatible with Hive, such as calling a Hive internal Authorization Validator (Hive authorization validator) component to realize, etc. The dimension values under each access dimension can be maintained by Hive or a third-party component compatible with Hive, and the response of the step S103 to the saving and querying of the dimension values can also be realized by calling a Hive internal component or a third-party component (such as FACS) compatible with Hive, and the selection of the specific component relied on by the method implementation is not limited here, and can be measured according to the actual application scene, and will not be described here.

[0088] It should be noted that for the data warehouse of the big data platform, it is often necessary to support users of different products to execute SQL, and different products use different Kerberos users. By using the proxy feature of HiveServer2, different users can use the same HiveServer2 and achieve data and permission isolation between each other. Therefore, in the embodiment, HiveServer2 is taken as an example to be introduced, as shown in Figure 4 When starting HiveServer2, a Keytab file (a Kerberos permission system credential file used for identity authentication) needs to be configured, and its user is specified as a proxyable user on Hadoop, and has super permission. After the operation request (SQL) of the ordinary user is authenticated by the Hive permission control method provided in the application, HiveServer2 will access HDFS and submit MR tasks in a proxy manner by using the super permission. The implementation mode of other types of Hive server can refer to the introduction of the embodiment, and will not be described here.

[0089] Based on the above introduction, the technical scheme provided by the embodiment of the application, after receiving the Hive operation request initiated by the client, extracts the library name, table name and column name of the data object accessed by the request in the Hive storage, splices the request operation type as the access dimension, and implements the Hive permission check according to whether the dimension value corresponding to the access dimension contains the current user group. In the check method, the dimension value checked is accurate to the library name, table name, column name and operation type, so that table field level authentication can be realized, fine-grained control of data permission can be completed, and the comprehensiveness of permission control is ensured. At the same time, in the method, the operation dimension is directly used to manage the permission after receiving the client operation, and the client does not need to be limited in operation, so that the integrity of user operation can be guaranteed. In addition, the method realizes Hive permission control by calling internal components on the Hive server side, so that when Hive is reused, the permission control method realized by calling internal components of Hive can be compatible, and reuse and expansion of Hive can be supported. Therefore, the method can guarantee the integrity of user operation while realizing fine-grained data permission control, and realize comprehensive and effective permission control.

[0090] It should be noted that based on the above embodiment, the application embodiment also provides a corresponding improvement scheme. In the preferred / improved embodiment, the steps involved in the above embodiment can be mutually referred to, and the corresponding beneficial effects can also be mutually referred to. In the preferred / improved embodiment herein, it will not be described one by one.

[0091] On the basis of the above embodiment, after it is determined in step S105 that the user has table permission, the following steps can be further executed:

[0092] S107, determine whether the path information is included in the operation request; if yes, execute S108;

[0093] After the table permission is checked, if the location (path) information is included in the operation request of the user, such as specifying a path to build a table, the user needs to have the permission of the path, and then (the DGS-Manager is called) to check whether the user has the permission of the path. If the check fails, the permission exception information is also thrown.

[0094] S108, perform path permission check on the path information to generate a path check result;

[0095] The path permission can be uniformly applied on the FCR cloud resource management platform, and the background can call the DGS (a kind of file virtual management system) to realize the authorization operation on the big data storage path. In the embodiment, the execution subject of the path permission check is not limited, and related components (such as AuthorizationValidator) can be called to realize the path permission check function according to actual needs, which will not be described here.

[0096] S109, if the path verification result shows that the verification fails, a step of determining an abnormal permission is performed;

[0097] If the path verification result shows that the verification fails, it proves that the current user does not have the Hive access path permission, and then the step of determining an abnormal permission is performed to determine the abnormal permission.

[0098] S110, if the path verification result shows that the verification passes, a task corresponding to the operation request is submitted so as to execute the operation request.

[0099] If the path verification result shows that the verification passes, it proves that the current user not only has the table field level access permission, but also has the path access permission,

[0100] Then, the HiveServer2 connects the MetaStore service as an agent (with super user permission) to query the metadata and submit the related task to the Hadoop so as to execute the operation request.

[0101] If the operation request does not include the path information, the processing manner for this case in the embodiment is not limited, and S110 can be directly triggered, or further authentication and the like in other aspects can be performed.

[0102] In addition to the table field authentication, the embodiment further performs the path authentication, which can comprehensively authenticate the operation behavior and guarantee the reliability of the authentication behavior.

[0103] Further, after the above path authentication step, if the verification passes, a task corresponding to the operation request is submitted, in order to guarantee the real-time update of the Hive table data and the permission data to automatically respond to the changing use demand at any time, after the corresponding task is submitted to the Hadoop according to the operation request, the following steps can be further performed:

[0104] S111, if the operation request is a storage operation, after the operation request is executed successfully, a new data access dimension is added, and a corresponding authorized user group is added to the dimension value of the new data access dimension;

[0105] If the content of the SQL is a storage operation (DDL operation), the HiveServer2 will make the addition, deletion and modification operation on the table dimension and the dimension value information after the statement execution is successful (calling the FACS, a unified permission management system).

[0106] Optionally, the process of adding the new data access dimension and adding the corresponding authorized user group to the dimension value of the new data access dimension can specifically include:

[0107] (1) Create a full table read / write dimension or a field read dimension based on the storage location of the new data, as the new data access dimension;

[0108] (2) Add the user group as the data owner and the data owner as the dimension value to the full table write dimension in the new data access dimension;

[0109] (3) After receiving the permission request from the first user group, if the permission request is approved by the data owner, the first user group will be added as a dimension value to the new data access dimension.

[0110] Depend on Figure 5 As described, table creation simultaneously creates the table's dimension information, including full table read / write dimensions and read dimensions for all fields. The user who created the table is designated as the table owner and granted full table read / write permissions (i.e., the RCC group is added as a dimension value to the full table write dimension). Other RCC groups (i.e., the first user group that is not the data owner) request permissions as needed. After approval by the Owner RCC group, the platform backend calls FACS for authorization, adding the RCC group as a dimension value to the dimension corresponding to the requested field.

[0111] S112. If the operation request is a deletion operation, iterate through the access dimensions and dimension values ​​corresponding to the data to be deleted.

[0112] If the operation request is a delete operation, it will iterate through and delete all dimensions and dimension value information of the table.

[0113] S113. If the operation request is a modification operation, add the modified data access dimension, and shift the corresponding dimension value of the data access dimension before modification to the modified data access dimension. Then, iterate through and delete the corresponding dimension value of the data access dimension before modification.

[0114] If the operation request is a modification operation, first add the dimension values ​​of the new table structure, move the corresponding dimension values ​​to the new dimension, and then iterate through and delete the dimension information.

[0115] This embodiment introduces three permission control feedback schemes under different operation types, which can realize the automatic adjustment of dimensional permission information under different operation types and improve the maintenance efficiency of permission control.

[0116] Based on the above embodiments, after the storage operation is successfully executed, a new data access dimension is added, and a corresponding authorized user group is added to the dimension value of the new data access dimension. After the execution is completed, in order to facilitate users' operation on the new data, the following can be further executed: the new data access dimension and the new data information are published to the data marketplace in order to receive new data operation requests.

[0117] The data owner can publish table information to a data market, so as to receive and respond to user operation requests for new data, and can publish all tables or selected fields, and can be set according to actual publishing needs.

[0118] In order to further guarantee the security of the permission control and avoid illegal use of the permission, after the first user group is added as a dimension value to the new data access dimension, the expiration time of the permission of the first user group can be further recorded, and whether the expiration time of the permission is reached is checked regularly, and if the expiration time of the permission is reached, the first user group is deleted from the dimension value of the new data access dimension.

[0119] When the user group (RCC group) is added as a dimension value to the dimension corresponding to the application storage field, the expiration time of the permission (or the start time of the permission and the valid period of the permission) is recorded. The external program regularly checks whether the application permission is expired, and clears the expired dimension value information after the expiration. The method can avoid illegal use of the permission and guarantee the security of the data through the expiration clearing mechanism of the permission.

[0120] Corresponding to the method embodiments above, the embodiments of the application further provide a Hive permission control device applied to a Hive server side, and the Hive permission control device described below can be correspondingly referred to the Hive permission control method described above.

[0121] Referring to Figure 6 The device includes the following modules:

[0122] The request determination unit 110 is mainly used for determining the user group initiating the operation request, the data object requested to be operated and the operation type after receiving the Hive operation request initiated by the client;

[0123] The dimension extraction unit 120 is mainly used for extracting the library name, the table name and the column name of the data object, and splicing the operation type as an access dimension;

[0124] The dimension value query unit 130 is mainly used for querying the dimension value corresponding to the access dimension; wherein the dimension value indicates a user group with permission;

[0125] The dimension value authentication unit 140 is mainly used for judging whether the user group is included in the dimension value; if yes, it is determined that the user has table permission; if not, it is determined that the permission is abnormal.

[0126] The application scenario is to perform fine-grained management on Hive data permissions in a Hadoop ecological system, and belongs to the technical field of database access permission control of big data. Specifically, as Figure 7As shown is a schematic view of a Hive permission control device calling each component to realize permission control, the Hive permission control device can be HiveServer2, and table permission verification is completed by AuthorizationValidator, thereby forming a complete Hive authentication system. The flowchart in the dashed box is a process of completing table permission verification by AuthorizationValidator, and the returned result specifically includes authentication passing and authentication failing. Among them, the dimensions can be uniformly managed by the FDW (a data map) platform, such as applying for permissions, changing permissions, etc., and HiveServer2 can call the FACS system to implement the adding, deleting and modifying operations of specific dimensions, so as to achieve the purpose of precisely controlling the permissions of the Hive table. The dimensions can be created by HiveServer2 module calling FACS at the same time of table building, and when the table structure is changed, the dimensions will also be changed accordingly, for details, see Figure 5 .

[0127] In one specific embodiment of the application, the Hive permission control device further comprises a path authentication unit.

[0128] Among them, the path authentication unit is connected to the output end of the dimension value authentication unit, for judging whether the path information is included in the operation request after judging that the user has the table permission; if yes, performing path permission verification on the path information to generate a path verification result; if the path verification result shows that the verification fails, executing the step of judging the permission exception; if the path verification result shows that the verification passes, submitting the task corresponding to the operation request so as to execute the operation request.

[0129] The granting of path permission can be provided by the FCR (a cloud resource management system) cloud resource management platform. If the user's SQL command contains path information, such as specifying a path to build a table, the user needs to have the permission of the path, and the path permission can be uniformly applied in the FCR cloud resource management platform, and the background is to call DGS to realize the authorization operation on the big data storage path.

[0130] Further, the Kerberos service can be connected before the Hive permission control device, so as to identify the identity of the user.

[0131] The Hive permission control device provided by the embodiment provides a fine-grained table dimension system and a complete application process of an authorization platform, can realize on-demand application and on-demand query in work, achieves the most accurate permission control, and guarantees data security to the maximum extent; meanwhile, the device does not have the limitations of the prior art, and will not limit the operation of the user, such as a custom function and a dfs instruction. Moreover, the scheme is convenient for reuse and extension, for example, Spark can reuse the permission model to complete the authentication process. Therefore, the Hive permission control device provided by the scheme has a complete right authorization process, a highly available Hive service, and safe identity authentication, realizes accurate column-level data permission management, effectively solves the data permission problem in the Hadoop system, and has good practical application ability.

[0132] Corresponding to the method embodiment above, the embodiment of the application further provides a computer device, and the computer device described below can be correspondingly referred to the Hive permission control method described above.

[0133] The computer device comprises:

[0134] a memory for storing a computer program;

[0135] a processor for executing the computer program to realize the steps of the Hive permission control method of the method embodiment.

[0136] Specifically, refer to Figure 8 A specific structural diagram of the computer device provided by the embodiment is shown in the figure, and the computer device can have great differences due to different configurations or performances, and can comprise one or more than one processor (central processing unit, CPU) 322 (for example, one or more than one processor) and a memory 332, wherein the memory 332 stores one or more than one computer application program 342 or data 344. The memory 332 can be temporary storage or persistent storage. The program stored in the memory 332 can comprise one or more than one module (not shown in the figure), and each module can comprise a series of instruction operations in the data processing device. Further, the central processing unit 322 can be arranged to communicate with the memory 332 and execute a series of instruction operations in the memory 332 on the computer device 301.

[0137] The computer device 301 can further comprise one or more than one power supply 326, one or more than one wired or wireless network interface 350, one or more than one input and output interface 358, and / or one or more than one operating system 341.

[0138] The steps in the Hive permission control method described above can be realized by the structure of the computer device.

[0139] Corresponding to the above method embodiments, the embodiments of the present application also provide a readable storage medium. The readable storage medium described below can be referred to the Hive permission control method described above.

[0140] A readable storage medium, the readable storage medium storing a computer program, the computer program being executed by a processor to implement the steps of the Hive permission control method of the above method embodiments.

[0141] The readable storage medium can be a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, or various readable storage media that can store program codes.

[0142] Those skilled in the art can further realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized in electronic hardware, computer software or a combination of both. In order to clearly illustrate the interchangeability of hardware and software, the components and steps of each example have been described in a general manner in the above description. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

Claims

1. A Hive permission control method, characterized in that, The method comprises the following steps: After the server receives the Hive operation request initiated by the client, the user group initiating the operation request, the data object requested by the operation and the operation type are determined; The library name, table name and column name of the data object are extracted, and the operation type is spliced as an access dimension; wherein the access dimension is used for data access control; The dimension value corresponding to the access dimension is queried; wherein the dimension value indicates the user group with permission; It is judged whether the user group is included in the dimension value; If yes, it is determined that the user group has table permission; if no, it is determined that the permission is abnormal; after determining that the user group has table permission, the following steps are further included: It is judged whether the path information is included in the operation request; If yes, the path permission check is performed on the path information, and the path check result is generated; If the path check result shows that the check fails, the step of determining that the permission is abnormal is executed; If the path check result shows that the check passes, the task corresponding to the operation request is submitted so as to execute the operation request.

2. The Hive permission control method of claim 1, wherein, After the corresponding task of the operation request is submitted to Hadoop, the following steps are further included: If the operation request is a storage operation, after the operation request is executed successfully, a new data access dimension is added, and the corresponding user group with permission is added to the dimension value of the new data access dimension; If the operation request is a deletion operation, the access dimension and the dimension value corresponding to the data to be deleted are iteratively deleted; If the operation request is a modification operation, a modified data access dimension is added, and the dimension value corresponding to the unmodified data access dimension is shifted to the modified data access dimension, and the dimension value corresponding to the unmodified data access dimension is iteratively deleted.

3. The Hive permission control method of claim 2, wherein, After the new data access dimension is added, and the corresponding user group with permission is added to the dimension value of the new data access dimension, the following steps are further included: The new data access dimension and the new data information are published to the data market so as to receive the new data operation application.

4. The Hive permission control method of claim 2, wherein, The step of adding the new data access dimension and adding the corresponding user group with permission to the dimension value of the new data access dimension comprises the following steps: According to the storage location of the new data, a full table read-write dimension or a field read dimension is created as a new data access dimension; The user group is taken as a data owner, and the data owner is taken as a dimension value and added to the full table write dimension in the new data access dimension; After receiving the permission application of the first user group, if the permission application is approved by the data owner, the first user group is added as a dimension value to the new data access dimension; wherein the first user group is not the data owner.

5. The Hive permission control method of claim 4, wherein, After the first user group is added as a dimension value to the new data access dimension, the following steps are further included: The expiration time of the permission of the first user group is recorded; Whether the expiration time is reached is checked regularly; If yes, the first user group is deleted from the dimension value of the new data access dimension.

6. A Hive permission control apparatus, characterized by, The device is applied to the Hive server side, and comprises the following steps: The request determining unit is configured to determine a user group initiating the operation request, a data object requested to be operated, and an operation type after receiving the Hive operation request initiated by the client; The dimension extracting unit is configured to extract a library name, a table name, and a column name of the data object, and splice the operation type as an access dimension; the access dimension is used for data access control. The dimension value querying unit is configured to query a dimension value corresponding to the access dimension; the dimension value indicates a user group having a permission; The dimension value authenticating unit is configured to determine whether the dimension value includes the user group; if yes, it is determined that the user group has a table permission; if no, it is determined that a permission is abnormal; the path authenticating unit is further included. The path authenticating unit is connected to an output end of the dimension value authenticating unit, and is configured to determine whether path information is included in the operation request after the determination that the user group has the table permission; if yes, the path information is subjected to path permission verification to generate a path verification result; if the path verification result shows that the verification fails, the step of determining that the permission is abnormal is executed; if the path verification result shows that the verification passes, a task corresponding to the operation request is submitted to execute the operation request.

7. A computer device, comprising: The memory is configured to store a computer program; The processor is configured to execute the computer program to implement the steps of the Hive permission control method according to any one of claims 1 to 5. The computer program is stored on the readable storage medium, and the computer program is executed by the processor to implement the steps of the Hive permission control method according to any one of claims 1 to 5.

8. A readable storage medium, characterized by, ​

Images