A method, server, and media for access control of big data components running on the Kubernetes platform.

Abstract

This invention discloses a method, server, and medium for access control of big data components running on a Kubernetes platform. The method involves deploying ranger-admin on the Kubernetes platform, downloading or compiling a ranger plugin compatible with the version of the big data components in the user's environment, decompressing the ranger plugin package, copying the decompressed directory of the ranger plugin to the big data component image using a Dockerfile, building a new big data component image, configuring the big data component's charts package file, integrating the big data component with the ranger plugin, and starting both the ranger plugin and the big data component. After the big data component is running, the user enters the ranger-admin web interface, first creating an external user, and then configuring access control policies for big data components such as HDFS, YARN, Hive, and HBase. This invention allows users to easily and quickly integrate Hadoop, Hive, HBase, and other big data components running on their Kubernetes platform with Ranger, enabling access control for the big data components, improving the security of the big data components, and has significant application value.

Description

Technical Field

[0001] This invention belongs to the fields of container technology and big data technology. Specifically, it involves implementing access control policies for big data components such as Hadoop (HDFS, YARN), HBase, and Hive running on the Kubernetes (k8s) container management platform through the Ranger permission management framework. Background Technology

[0002] With the rapid development of container technology, Kubernetes (k8s) technology has emerged as a leading technology, and more and more applications are being deployed on k8s. Technologies such as Hadoop, HBase, and Hive, as commonly used components in big data, are also gradually running on the k8s platform. In the traditional model, big data components run directly on physical machines or virtual machine environments. Integrating big data components with the Ranger permission management framework to implement access control is relatively easy. However, integrating Ranger to implement access control for big data components running on the k8s platform is quite difficult. Therefore, a simple and easy-to-use integration mode has been invented to quickly enable access control for big data components running on the k8s platform. Summary of the Invention

[0003] To address the problems existing in the prior art, this invention provides a method for access control of big data components running on the Kubernetes platform. This method allows users to easily and quickly integrate big data components such as Hadoop, Hive, and HBase running on their Kubernetes platform with Ranger, enabling access control for the big data components and thus improving their security.

[0004] To achieve the above-mentioned objectives, a method for access control of big data components running on the Kubernetes platform was designed, comprising the following steps.

[0005] Step 1: Install ranger-admin in Kubernetes; ranger-admin is the server-side component of ranger. Users configure permission policies for the big data component through its web UI. Plugins in the big data component interact with ranger-admin via REST API. The plugins of the big data component periodically pull policies from ranger-admin. When a client makes a request to the big data component, the ranger plugin will match the policy against the user in the request and determine whether to accept or reject the request.

[0006] Step 2: Download or compile a Ranger plugin compatible with the big data component version; you can download the corresponding version of the Ranger plugin directly from the Internet, or you can generate the corresponding version of the Ranger plugin by compiling Ranger.

[0007] Step 3: Place the ranger plugin directly into the container image of the big data component; building the ranger plugin directly into the container image of the big data component will save a lot of trouble.

[0008] Step 4: Configure the charts package file for the big data component to integrate the big data component with the ranger plugin and start the ranger plugin and the big data component.

[0009] S1. Use the ranger plugin's configuration file content as the Kubernetes configmap. This method has significant advantages over directly modifying the ranger plugin's configuration file and putting it into the big data component container. For example, when the ranger plugin's configuration file content needs to be updated, users will not need to rebuild the big data component image; they only need to modify the configmap on Kubernetes.

[0010] S2. Add the identifier for integrating Ranger to the values.yaml file of the charts package of the big data component; add the identifier for enabling Ranger to the values.yaml file of the charts package of big data components such as Hadoop, HBase, and Hive, for example, enabled: true. This configuration will give users some flexibility. When users do not need to integrate Ranger into their big data components on the Kubernetes platform, they only need to configure enabled: false here.

[0011] S3. In the startup script or command of the big data component's charts package, check the identifier in step S2 to integrate Ranger. For example, if the value of `enabled` in `values.yaml` is `true`, copy the configmap for the Ranger plugin from Kubernetes to the big data component container, replacing the default Ranger plugin configuration file. Then, start the Ranger plugin first, and then run the big data component startup script to run the big data component. The startup order cannot be reversed. If Ranger integration is not required in the `values.yaml` file, for example, if the value of `enabled` is `false`, do not copy the configmap from S1, and directly start the big data component startup script.

[0012] Step 5: Configure access control permissions and users for the big data components in the ranger-admin web UI. There is no need to synchronize users inside the container; you can directly use the users on ranger-admin.

[0013] The present invention also provides a proxy server, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method described above.

[0014] The present invention also provides a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the steps of the method described above.

[0015] The beneficial effects of this invention are as follows: This invention provides a method for quickly integrating big data components running on the Kubernetes platform with Ranger. It includes designing how the Ranger plugin combines with the big data component container and how to dynamically insert the Ranger plugin's configuration file into the big data component container. This allows users to quickly implement access control for big data components running on the Kubernetes platform, improving the security of big data components while saving significant time. It eliminates the problem of users not knowing how to enable access control for big data components running on the Kubernetes platform. Furthermore, users can enable or disable access control for big data components at any time through big data group configuration, increasing the flexibility of use on the Kubernetes platform.

[0016] This invention integrates the ranger plugin with the big data component container and configures the big data component charts package file to achieve the integration of the big data component with the ranger plugin. This allows users to quickly implement access control for big data components running on the Kubernetes platform, improving the security of big data components while saving a significant amount of time. It eliminates the problem of users not knowing where to start when they want to enable access control for big data components running on the Kubernetes platform. At the same time, users can enable or disable access control for big data components at any time through the configuration of the big data component charts package file, increasing the flexibility of using big data components on the Kubernetes platform. This invention has great potential for widespread application. Attached Figure Description

[0017] Figure 1 This is the system architecture diagram of the present invention.

[0018] Figure 2 This is a system flowchart of the present invention. Implementation

[0019] Example 1: A method for quickly integrating various data components on the Kubernetes platform with the ranger, comprising the following steps (see...). Figure 1 ).

[0020] 1. Deploy ranger-admin on the Kubernetes platform. This means installing ranger-admin in Kubernetes. Ranger-admin is the server-side component of Ranger. Users configure permission policies for the Big Data component through its web UI. Plugins in the Big Data component interact with ranger-admin via a REST API. The plugins periodically retrieve policies from ranger-admin. When a client makes a request to the Big Data component, the ranger plugin matches the policy against the user in the request and determines whether to accept or reject the request.

[0021] 2. Download or compile a Ranger plugin compatible with the version of the big data components in your environment: for example, ranger-2.0.0-hdfs-plugin.tar.gz, ranger-2.0.0-yarn-plugin.tar.gz, ranger-2.0.0-hbase-plugin.tar.gz, and ranger-2.0.0-hive-plugin.tar.gz. These plugins can be downloaded directly from the internet or generated by compiling Ranger.

[0022] 3. Unzip the ranger plugin package, and copy the unzipped directory of the ranger plugin to the big data component image by writing a Dockerfile, and build a new big data component image.

[0023] 4. Configure the charts package file for the big data component to integrate the big data component with the ranger plugin and start the ranger plugin and the big data component, including the following steps.

[0024] S1 modifies the charts file of the big data component, storing the ranger plugin configuration file as a configmap in Kubernetes. This approach has significant advantages over directly modifying the ranger plugin configuration file and placing it into the big data component container. For example, when the ranger plugin configuration file needs to be updated, users do not need to rebuild the big data component image; they only need to modify the configmap on Kubernetes.

[0025] S2 adds the `enabled: true` flag to the `values.yaml` file within the `charts` folder of the big data component. Similarly, add the `enabled: true` flag to the `values.yaml` file of the `charts` package for big data components such as Hadoop, HBase, and Hive. This configuration provides users with some flexibility; if a user does not need to integrate Ranger into their Kubernetes platform's big data components, they only need to configure `enabled: false` here.

[0026] S3 uses the `{{- if .Values.enabled}}` setting to check if the `enabled` value in the `values.yaml` file of the big data component's charts file is true. If so, it copies the ranger plugin configuration from the Kubernetes configmap to the big data component container, replacing the ranger plugin's default configuration file. It then starts the ranger plugin first, followed by the big data component application. If the `enabled` value in the `values.yaml` file of the big data component's charts file is false, it neither copies the Kubernetes configmap nor starts the ranger plugin; it starts the big data component directly.

[0027] 5. After the big data components are running, go to the ranger-admin web interface, first create an external user, and then configure the access control policies for big data components such as hdfs, yarn, hive, and hbase.

[0028] 6. After the policy is configured, wait approximately 15-30 seconds. The plugin in the big data component container will obtain the permission policy configured on ranger-admin. In the big data component container or the big data component client, use the same account as the external user to verify. The access control permissions have taken effect according to the policy configured on ranger-admin.

[0029] Example 2: This example provides a proxy server, showing only the parts relevant to this example.

[0030] The proxy server in this embodiment includes a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps in the method embodiment 1 described above. In this embodiment, when the processor of the proxy server executes the computer program, it deploys ranger-admin on the Kubernetes platform, downloads or compiles a ranger plugin compatible with the version of the big data component in its environment, decompresses the ranger plugin package, copies the decompressed directory of the ranger plugin to the big data component image using a Dockerfile, builds a new big data component image, and then configures the big data component charts package file to integrate the big data component with the ranger plugin and start the ranger plugin and the big data component. This process includes modifying the big data component charts file, saving the ranger plugin configuration file as a configmap in Kubernetes, adding the `enabled: true` flag to the values.yaml file in the big data component charts file, and determining the above flag in the big data component charts package's startup script or command to integrate ranger. After the big data components are running, access the ranger-admin web interface. First, create an external user, then configure access control policies for the big data components such as HDFS, YARN, Hive, and HBase. This allows users to easily and quickly integrate their Hadoop, Hive, HBase, and other big data components running on their Kubernetes platform with Ranger, enabling access control and improving the security of the big data components. Example

[0031] This embodiment provides a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the steps in Embodiment 1 of the data access control method described above. In this embodiment, when the computer program is executed by the processor, it deploys ranger-admin on the Kubernetes platform. It downloads or compiles a ranger plugin compatible with the version of the big data component in its environment, decompresses the ranger plugin package, and copies the decompressed directory of the ranger plugin to the big data component image using a Dockerfile. A new big data component image is then built. The big data component charts package file is then configured to integrate the big data component with the ranger plugin and start both the ranger plugin and the big data component. This process includes modifying the big data component's charts file, saving the ranger plugin configuration file as a configmap in Kubernetes, adding the `enabled: true` flag to the `values.yaml` file in the big data component's charts file, and determining the above flag in the big data component's charts package startup script or command to integrate ranger. After the big data components are running, access the ranger-admin web interface. First, create an external user, then configure access control policies for the big data components such as HDFS, YARN, Hive, and HBase. This allows users to easily and quickly integrate their Hadoop, Hive, HBase, and other big data components running on their Kubernetes platform with Ranger, enabling access control and improving the security of the big data components.

[0032] The specific embodiments described above are merely illustrative or explanatory of the principles of the present invention and do not constitute a limitation thereof. Therefore, any modifications, equivalent substitutions, improvements, etc., made without departing from the spirit and scope of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for access control of big data components running on a Kubernetes platform, characterized in that, Includes the following steps: Step 1: Install the ranger server (ranger-admin) in the Kubernetes environment; Step 2: Download or compile a version of the ranger plugin that is compatible with big data components; Step 3: Place the ranger plugin directly into the container image of the big data component; Step 4: Configure the charts package file for the big data component to integrate the big data component with the ranger plugin and start both the ranger plugin and the big data component. This includes the following steps: S1: Use the configuration file content of the ranger plugin as the Kubernetes configmap; S2: Add the identifier for integrating Ranger to the values.yaml file of the charts package in the big data component; S3: In the big data program startup script or command of the charts package of the big data component, determine the identifier in step S2 to integrate the ranger; Step 5: Configure access control permissions and users for the big data component in the ranger-admin web UI.

2. The method for access control of big data components running on a Kubernetes platform according to claim 1, characterized in that, The underlying operating environment of the big data component is Kubernetes, or various platforms used to manage and run Kubernetes.

3. The method for access control of big data components running on a Kubernetes platform according to claim 1, characterized in that, Big data components include HDFS, YARN, Hive, and HBase.

4. The method for access control of big data components running on a Kubernetes platform according to claim 1, characterized in that, This includes creating user data for implementing access control for big data components.

5. The method for access control of big data components running on a Kubernetes platform according to claim 1, characterized in that, This includes configuring control permissions for big data components.

6. The method for access control of big data components running on a Kubernetes platform according to claim 1, characterized in that, In step S2, add the enable flag for ranger to the values.yaml file in the charts package of the Hadoop, HBase, or Hive big data component. When users do not need to integrate ranger into their big data components on their Kubernetes platform, they only need to configure enabled: false in the values.yaml file of the charts package of the corresponding big data component.

7. The method for access control of big data components running on a Kubernetes platform according to claim 1, characterized in that, When the value of enabled in values.yaml is true, copy the configmp file for the ranger plugin from Kubernetes to the big data component container, replacing the default ranger plugin configuration file. Then, start the ranger plugin first, and then run the big data component startup script to run the big data component.

8. The method for access control of big data components running on a Kubernetes platform according to claim 1, characterized in that, If the ranger is not required in the values.yaml file and the enabled value is false, then the configmap described in step S1 of step 4 will not be copied, and the startup script of the big data component will be started directly.

9. A proxy server, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 8.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 8.

Images