Implementation method of dynamic connection of Flink and Kerberos authentication components

Abstract

The invention relates to the field of computing, and particularly provides an implementation method of dynamic connection of Flink and Kerberos authentication components, which comprises the following steps of: S1, preparing an Flink cluster and a plurality of sets of different Kerberos authentication components; S2, mounting NFS shared storage; S3, dynamically integrating Kerberos authentication files by a Kerberos management center, and storing the Kerberos authentication files in a shared storage directory; S4, changing the flink-conf.yaml configuration file of the Flink cluster; S5, in a Job related code of the Flink, refreshing Krb5.conf related configuration in real time; and S6, operating the Job of the Flink so that the Flink cluster is successfully connected with the plurality of sets of Kerberos authentication components. Compared with the prior art, the method has the advantages that the problems that conflicts occur when Flink is connected with a plurality of different Kerberos authentication components, a cluster needs to be restarted when the Kerberos authentication components are changed and the like are solved.

Description

technical field [0001] The invention relates to the computing field, and specifically provides an implementation method for Flink to dynamically connect Kerberos authentication components. Background technique [0002] The predecessor of Apache Flink was a research project of the Technical University of Berlin. It was accepted by the Apache Incubator in 2014, and then quickly became one of the top projects of the Apache Software Foundation. It is an open source for distributed data stream processing and batch data processing. Computing platform that provides functions to support two types of applications, stream processing and batch processing. [0003] Kerberos is an authentication system based on a trusted third party proposed by the Massachusetts Institute of Technology. Kerberos provides a method of identity authentication in an open network environment, which enables users on the network to prove their identities to each other. [0004] Currently, the official kerbero...

AI Technical Summary

Problems solved by technology

However, in actual business scenarios, we need Flink clusters to access multiple different input and output sources, or the same type of input and output sources to access multiple instances, so when the input and output sources involve Kerberos authentication components (such as Kafka, Hbase, Hdfs, etc.), it is necessary to support the dynamic configuration of Kerberos components; at the same time, because the number of jobs submitted to Flink is not fixed, multiple Kerberos configurations need to be implemented at the same time; and because the input and output sources are dynamically changed during online operation, they cannot be passed through the cluster. The service restarts to update the configuration file, which must take effect immediately

Images