An authentication management system and method based on Kerberos mechanism

Abstract

The application discloses a kind of authentication management systems and methods based on Kerberos mechanism, it is related to identity authentication technical field, including unified authentication management module, dynamic arrangement engine, task scheduling middleware and real-time feedback module;Unified authentication management module centralized management Hive, Zookeeper, Flink and Spark component's Kerberos authentication configuration, including user creation, Principal subject management and keytab file distribution;Dynamic arrangement engine is matched authentication mode according to task type and target component and generates authentication script and configuration file;Task scheduling middleware distributes authentication task and maintains ticket state;Real-time feedback module monitors authentication state, collects log and triggers abnormal alarm.The system realizes automated Kerberos configuration by unified authentication management module and dynamic arrangement engine, simplifies Kerberos authentication process, reduces the cumbersome operation of being separately configured for each component in traditional mode, and user creation, Principal subject management and keytab file distribution are completed by unified interface, without manual intervention.

Description

Technical Field

[0001] The present invention relates to the technical field of identity authentication, in particular to an authentication management system and method based on the Kerberos mechanism. Background Art

[0002] Data security and integrity are paramount in big data platforms. Kerberos, a network authentication protocol, uses "tickets" to verify user identities and ensure secure data transmission. Components such as Hive and Zookeeper are widely used in the big data ecosystem, while Flink and Spark, as data processing frameworks, also require running in a secure environment. Therefore, implementing secure and efficient authentication and task submission between these components has become a pressing issue.

[0003] Currently, it is common practice to configure Kerberos authentication for each component separately. For example, when configuring Kerberos for Hive, you need to create a Hive user and set a group in the Hadoop cluster, create a Principal for the Hive service, and write the service principal to the keytab file. Similar Kerberos configuration is required for Zookeeper. Kerberos authentication is also required when submitting Flink and Spark tasks.

[0004] The above method of configuring Kerberos authentication for each component separately has some shortcomings: it is necessary to create users, principals, and configure keytab files for each component separately, which increases the workload of administrators and the probability of errors; the Kerberos configurations of each component are independent of each other, making centralized management and monitoring difficult, which is not conducive to the unified implementation of security policies; when new components or services need to be added, Kerberos configuration needs to be reconfigured, which limits the scalability of the system.

[0005] Based on this, a Kerberos-based authentication management system and method are now provided to eliminate the drawbacks of existing technical solutions. Summary of the Invention

[0006] The purpose of the present invention is to provide an authentication management system and method based on the Kerberos mechanism to solve the problem that the method of configuring Kerberos authentication for each component separately in the background technology is insufficient, resulting in the inability to achieve secure and effective identity authentication and task submission between components.

[0007] To achieve the above object, the present invention provides the following technical solutions:

[0008] An authentication management system and method based on the Kerberos mechanism, comprising:

[0009] A unified authentication management module is used to communicate with the key distribution center server over the network to centrally manage the Kerberos authentication configuration of target components in the big data platform. The target components include big data components that communicate with the unified authentication management module, including Hive, Zookeeper, Flink, and Spark. The Kerberos authentication configuration includes user creation, Principal subject management, and keytab file distribution operations;

[0010] The dynamic orchestration engine automatically selects the appropriate Kerberos authentication method from the authentication method library when submitting a task based on the target component and task type, and generates the corresponding authentication configuration file and execution script. The task types include Flink computing tasks, Spark job tasks, and Hive query tasks. The authentication method library is used to store the Kerberos authentication methods corresponding to different task types.

[0011] Task scheduling middleware, which is used to securely distribute Kerberos-authenticated tasks to corresponding components for execution and maintain the authentication status and ticket validity period during the task lifecycle;

[0012] A real-time feedback module monitors the Kerberos authentication status and results during task execution, collects security log information, and triggers an alert mechanism when anomalies are detected, including authentication failures, ticket expiration, unauthorized access, and service anomalies.

[0013] The unified authentication management module, dynamic orchestration engine, task scheduling middleware and real-time feedback module communicate with each other through a network.

[0014] Preferably, the unified authentication management module is used to create a Principal subject and generate an encrypted keytab file through a key distribution center server, wherein the key distribution center server includes a primary key distribution center server and a backup key distribution center server. The unified authentication management module includes:

[0015] The user management submodule is used to create and manage user accounts in the big data platform;

[0016] The principal management submodule is used to create, manage and maintain the Kerberos Principal principals corresponding to each component;

[0017] The keytab file management submodule is used to generate, store, and distribute keytab files, and to encrypt and store keytab files and control access permissions.

[0018] Preferably, the dynamic orchestration engine includes:

[0019] A task type identification unit, used to identify the task type to which the submitted task belongs;

[0020] An authentication method matching unit, configured to perform matching based on a preset authentication method library, wherein the authentication method library contains task types and corresponding Kerberos authentication methods, wherein the authentication parameters of the Kerberos authentication method include the Principal format and the validity period of the ticket;

[0021] The script generation unit is used to generate an authentication script containing a kinit command based on a matching preset Kerberos authentication method and associate it with a corresponding keytab file path.

[0022] Preferably, the task scheduling middleware includes:

[0023] The authentication environment injection unit is used to obtain the Kerberos ticket before submitting the task and inject the authentication information into the task running environment variables;

[0024] A switching unit, configured to automatically switch to a backup key distribution center server for authentication when the master key distribution center server fails authentication;

[0025] The bill renewal management unit is used to automatically trigger a renewal command when it detects that the remaining validity period of the bill is lower than a preset threshold, and the preset threshold is set to 5 minutes.

[0026] Preferably, the real-time feedback module includes:

[0027] The monitoring submodule is used to monitor the Kerberos authentication status and results of tasks in real time;

[0028] The log collection submodule is used to collect security logs and information related to Kerberos authentication, including authentication request records, ticket usage records, and abnormal event records;

[0029] The exception handling submodule is used to trigger an alarm and notify management personnel after detecting an abnormal situation.

[0030] A Kerberos-based authentication management method is applied to the Kerberos-based authentication management system. The specific steps are as follows:

[0031] S1. The task submitter submits the task to the big data platform. The task submitter includes the user and the system.

[0032] S2. The unified authentication management module performs configuration management related to user authentication for the task submitted in step S1, including creating users, managing principals, and distributing keytab files.

[0033] S3. When receiving a task submission request, the dynamic orchestration engine selects the Kerberos authentication method based on the target component and task type, and generates an authentication script and configuration file.

[0034] S4. The task scheduling middleware obtains the Kerberos ticket, distributes the task authenticated in step S3 to the corresponding target component for execution, and manages the ticket status during the execution process;

[0035] S5. The real-time feedback module monitors the authentication status, collects log information, triggers alarms in case of anomalies, and records the processing process.

[0036] Preferably, the step S1 specifically includes: taking the user and the system as the task submission subject, initiating a task execution request to the big data platform, and submitting the task information to the big data platform. The task submission method includes submitting through the graphical operation interface provided by the big data platform, submitting by entering specific instructions using the command line interface, and submitting the task through the configuration interface. The task information includes the task type, target component, task specific execution logic and parameters.

[0037] Preferably, the step S2 specifically includes: the unified authentication management module creates a corresponding user account for the task submission subject in the big data platform, the user account includes a unique user identifier and user attributes, creates a corresponding service Principal subject for task execution for the target component, and generates a keytab file corresponding to each service Principal subject, and securely distributes the keytab file to the corresponding component and task execution environment.

[0038] Preferably, step S3 specifically includes: when the dynamic orchestration engine receives the task submission request, analyzing the target component and task type of the task, selecting an authentication method suitable for the task from preset Kerberos authentication methods based on the analysis result, and generating a corresponding authentication script and configuration file based on the selected authentication method, wherein the authentication script includes a kinit command for obtaining a Kerberos ticket, and the configuration file includes Kerberos-related parameters in the task execution environment, including a key distribution center address and domain.

[0039] Preferably, step S4 specifically includes: the task scheduling middleware obtains the Kerberos ticket related to the task from the unified authentication management module, distributes the authenticated task to the corresponding target component for execution, and during the task execution process, the task scheduling middleware continuously monitors the Kerberos ticket status and validity period. When it detects that the ticket is about to expire, the task scheduling middleware automatically triggers the ticket update process.

[0040] Compared with the prior art, the present invention has the following beneficial effects:

[0041] This authentication management system based on the Kerberos mechanism realizes automatic Kerberos configuration through a unified authentication management module and a dynamic orchestration engine, reducing the tedious operation of configuring each component separately in the traditional method. Operations such as user creation, Principal subject management and keytab file distribution are all completed through a unified interface, reducing the workload and error rate of management personnel. The dynamic orchestration engine can automatically match the Kerberos authentication method according to the task type and generate the corresponding authentication script and configuration file without manual intervention. It can support the rapid expansion of new components or task types by simply adding the corresponding configuration to the authentication method library, thereby improving the flexibility and scalability of the system. The real-time feedback module realizes transparent management of the entire life cycle of Kerberos authentication by monitoring the authentication status, collecting security logs and analyzing abnormal events, quickly locating and resolving security-related issues, and reducing the time and cost of troubleshooting. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 Schematic diagram of the steps of the authentication management method based on the Kerberos mechanism of the present invention.

[0043] Figure 2 Schematic diagram of the overall structure of the system of the present invention.

[0044] Figure 3 Schematic diagram of the structure of the module of the present invention.

[0045] Figure 4 Schematic diagram of the interaction between the modules of the present invention.

[0046] Notes on the accompanying drawings: unified authentication management module 10, user management sub-module 11, subject management sub-module 12, keytab file management sub-module 13, dynamic orchestration engine 20, task type identification unit 21, authentication method matching unit 22, script generation unit 23, task scheduling middleware 30, authentication environment injection unit 31, switching unit 32, ticket renewal management unit 33, real-time feedback module 40, monitoring sub-module 41, log collection sub-module 42, exception handling sub-module 43. DETAILED DESCRIPTION

[0047] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments.

[0048] In this embodiment, if Figures 1-4 The figure shows an authentication management system and method based on the Kerberos mechanism. Kerberos is a network identity authentication protocol that verifies user identities by using tickets to ensure data transmission security. It is often used in big data platforms to ensure the security of identity authentication and data transmission between components. Big data platforms refer to systems that include components such as Hive, Zookeeper, Flink, and Spark, and are used for data storage, distributed coordination, stream computing, and batch processing.

[0049] Specifically, the basic principle of Kerberos authentication is as follows: the Kerberos protocol is based on a trusted third-party key distribution center and adopts a client or server mode, such as a key distribution center server (KDC). User identity authentication is achieved through tickets. The third-party key distribution center includes two main components: an authentication server (AS) and a ticket granting server (TGS). The user sends an authentication request to the authentication server (AS). After the authentication server (AS) verifies the user's identity, it issues a ticket (TGT) to the user. The ticket contains user identity information and a session key, and is encrypted using the master key of the third-party key distribution center. The user uses the initial ticket (TGT) to request a task ticket to access a specific service from the ticket granting server (TGS). After the ticket granting server (TGS) verifies the validity of the initial ticket (TGT), it generates a corresponding task ticket. The task ticket contains the user identity, service identifier, and session key, and is encrypted using the service's key. The user uses the task ticket to request a service from the server. After the server verifies the validity of the task ticket, it provides services to the user.

[0050] The authentication management system based on the Kerberos mechanism includes a unified authentication management module 10, a dynamic orchestration engine 20, a task scheduling middleware 30 and a real-time feedback module 40;

[0051] The unified authentication management module 10 is used to centrally manage the Kerberos authentication configuration of target components in the big data platform. The target components include big data components that communicate with the unified authentication management module 10, such as Hive, Zookeeper, Flink, and Spark. The Kerberos authentication configuration includes user creation, principal subject management, and keytab file distribution operations. The keytab file distribution operation is based on the AES-256 encryption algorithm. The principal subject naming rule is "component / server host name@Kerberos domain name".

[0052] like Figure 3 and Figure 4 As shown, the unified authentication management module 10 communicates with the key distribution center server via a network, and is used to create a principal subject and generate an encrypted keytab file through the key distribution center server. The key distribution center server and the unified authentication management module 10 use AES-256-CTS mode for encrypted communication. The key distribution center server includes a primary key distribution center server and a backup key distribution center server. The unified authentication management module 10 specifically includes:

[0053] The user management submodule 11 is used to create and manage user accounts in the big data platform. Accounts are created for all users of the big data platform in a unified Kerberos domain to achieve centralized management of user information. Through a unified user management interface or configuration interface (API interface), administrators can quickly create, delete, and modify user information to ensure the uniqueness and legitimacy of user identities.

[0054] The principal management submodule 12 is used to create, manage, and maintain the Kerberos principals corresponding to each component. It creates corresponding Kerberos principals for components such as Hive, Zookeeper, Flink, and Spark. It can also include other big data components such as Hadoop and Kafka based on actual usage requirements. Each principal represents a service instance. Through unified naming rules and management methods, the standardization and manageability of the principals of each component are ensured. For example, a principal such as "hive / _HOST@Kerberos_DOMAIN" is created for the Hive component, where "_HOST" is the server host name and "Kerberos_DOMAIN" is the specifically named Kerberos domain name in the authentication realm.

[0055] The keytab file management submodule 13 is used to generate, store, and distribute keytab files. It uses encrypted storage and access control operations on keytab files. The keytab file contains the principal subject and its corresponding key, which is used for automatic authentication of the server. The unified authentication management module 10 automatically generates the keytab files required by each component and distributes them to the corresponding components or nodes in a secure manner. During the distribution process, encrypted transmission and access control mechanisms are used to ensure the security of the keytab files and prevent unauthorized access.

[0056] Specifically, it also includes a visual configuration interface, which is used to provide a graphical interface and configuration interface to achieve unified setting and batch modification of authentication policies. Each sub-module interacts through an internal interface to achieve centralized management of Kerberos authentication configuration;

[0057] Specifically, the Principal subject is an identity identifier used to identify a user, service, or other security entity, such as the "Principal subject of the Hive service" or "hive / _HOST@Kerberos_DOMAIN". The keytab file is a file containing the Kerberos principal and its corresponding key, which is used to perform Kerberos authentication operations without user interaction. The authentication server (AS) is used to receive the user's initial authentication request and issue the initial ticket (TGT) after verifying the user's identity. The ticket granting server (TGS) is used to receive service access requests initiated by the user using the initial ticket (TGT). Hive is a big data warehouse tool used to store and query structured data. Zookeeper is a distributed coordination service used to maintain configuration information and distributed synchronization. Flink and Spark are both data processing frameworks used for stream computing and batch processing tasks, respectively.

[0058] In the authentication management system based on the Kerberos mechanism, the unified authentication management module 10 works in conjunction with other modules to form a clear hierarchical structure and task processing flow. The unified authentication management module 10 internally adopts a three-layer sub-module structure:

[0059] User management submodule 11: located at the top layer, responsible for the creation and management of user accounts;

[0060] Subject management submodule 12: Located in the middle layer, responsible for the creation and maintenance of Kerberos Principal subjects;

[0061] Keytab file management submodule 13: Located at the bottom layer, it is responsible for the generation, storage, and distribution of keytab files, task flows, and data flows;

[0062] The Zookeeper component is a bottom-level support component that provides a unified service registration and discovery mechanism for other components (Hive, Flink, and Spark). Hive belongs to the data storage layer, receives authenticated query tasks (such as Hive SQL tasks), and returns processing results. Flink and Spark belong to the computing processing engine, receives authentication scripts and tasks generated by the dynamic orchestration engine 20, performs calculations under the coordination of the task scheduling middleware 30, and relies on Zookeeper for cluster resource scheduling and state synchronization. Flink focuses on real-time data processing, and Spark focuses on offline data processing. Both need to access the data storage layer (such as Hive) to obtain data. Taking "user submitting Spark jobs to query Hive data" as an example, the task flow and data flow are as follows: the user submits a task through the graphical interface, command line, or API interface. The task information (including task type, target component, etc.) is passed to the unified authentication management module 10. The user management submodule 11 verifies the user identity or creates a new user account. The subject management submodule 12 creates or matches the corresponding Kerberos for the task. Principal subject, the keytab file management submodule 13 generates and distributes the encrypted keytab file to the relevant components. The task of completing the authentication configuration is passed to the dynamic orchestration engine 20 for the next step of processing. The authentication information (Principal, keytab path, etc.) is injected into the task environment variables. Components such as Hive, Zookeeper, Flink and Spark interact with the authentication system through a unified interface, so that each component no longer needs to maintain the authentication configuration separately. Instead, it obtains the task from the unified authentication management module 10 and smoothly passes it to the dynamic orchestration engine 20. All components interact with the authentication system through a unified standard interface, and the authentication configuration is automatically generated and injected into the task environment. The modules maintain real-time communication through the network to ensure consistent status.

[0063] The dynamic orchestration engine 20 is used to automatically select the appropriate Kerberos authentication method from the authentication method library when submitting a task based on the target component and task type, and to generate the corresponding authentication configuration file and execution script. Task types include Flink computing tasks, Spark job tasks, and Hive query tasks.

[0064] like Figure 3 As shown, the dynamic orchestration engine 20 specifically includes:

[0065] The task type identification unit 21 is used to identify the task type of the submitted task. By analyzing the metadata, submission parameters or task description information of the task, the task type is identified, such as Flink task, Spark task, Hive query task, etc. Different types of tasks require different Kerberos authentication methods and configurations. For example, if a task contains a spark command, it is identified as a Spark task; if it contains a hive query statement, it is identified as a Hive task.

[0066] For mixed tasks that contain both Spark commands and Hive statements, the system uses the following mechanism to identify and authenticate them: the type identification unit 21 will simultaneously check the Spark commands (such as spark-submit) and Hive statements (such as SELECT) in the task, identify mixed features through grammatical analysis, and determine the dominant task type based on the main execution environment of the task. For example, a task that starts with spark-submit will be authenticated even if it contains Hive statements. QL will also be preferentially identified as a Spark task, and the user can explicitly specify the task type when submitting the task. The system will record the type characteristics of historical tasks for auxiliary identification. The authentication method matching unit 22 will match the composite authentication strategy for the hybrid task, which also includes the authentication configuration required by the Spark and Hive components. Multiple Principal subjects (such as Spark and Hive’s respective Principals) are created or associated for the task. Each Principal corresponds to different service access permissions. The script generation unit 23 will generate a composite authentication script containing multiple kinit commands to ensure that both Spark and Hive components can obtain valid Kerberos tickets. The environment variables required by different components are then integrated into a unified task execution environment to avoid environment variable conflicts. If some authentications fail but other components succeed, the system will mark the partial success status and decide whether to continue execution based on the task dependencies. Through this design, the system can effectively handle complex hybrid task authentication scenarios and ensure the smooth execution and secure access of different types of tasks in the big data platform.

[0067] The authentication method matching unit 22 is used to match based on a preset authentication method library. The authentication method library contains task types and corresponding Kerberos authentication methods. The authentication parameters of the Kerberos authentication method include the Principal format and the validity period of the ticket. It is used to establish a mapping relationship between the task type and the Kerberos authentication method. For example, for Flink tasks, it is necessary to use a specific Flink Kerberos authentication plug-in and obtain the corresponding ticket. For Hive tasks, it is necessary to authenticate through the Hive Kerberos authentication interface. The authentication method matching unit 22 can select the most appropriate authentication method from the preset authentication method library. The authentication method library can store Kerberos authentication policy tasks corresponding to different task types;

[0068] The script generation unit 23 is configured to generate an authentication script containing a kinit command based on a preset Kerberos authentication method and associate it with a corresponding keytab file path. The kinit command is a ticket acquisition command in the Kerberos protocol and is used to apply for an initial ticket (TGT) from the key distribution center. The configuration file includes Kerberos client configuration and Kerberos configuration parameters for each component. The execution script includes commands for obtaining tickets and setting environment variables. The generated configuration file and script can automatically adapt to the operating environment of the target component to ensure a smooth authentication process.

[0069] Specifically, the traditional method requires separate configuration of different components, while the dynamic orchestration engine 20 can automatically match authentication methods, dynamically generate scripts, and adapt to multiple components. The dynamic orchestration engine 20 solves the complex configuration problems of traditional Kerberos authentication through an automated process of task type identification, authentication method matching, and script generation, thereby improving security and management efficiency.

[0070] Task scheduling middleware 30, used to securely distribute Kerberos-authenticated tasks to corresponding components for execution and maintain the authentication status and ticket validity period during the task lifecycle;

[0071] like Figure 3 As shown, specifically including:

[0072] The authentication environment injection unit 31 is used to obtain a Kerberos ticket before submitting a task and inject the authentication information into the task execution environment variables. It obtains user authentication information and a keytab file from the unified authentication management module 10, obtains the generated authentication configuration file and script from the dynamic orchestration engine 20, integrates the authentication information with the task to form a complete task information submission package. The task information submission package includes the task code, authentication configuration, and execution script. It also sets the task execution environment variables so that the target component can recognize and use the Kerberos authentication information. The authentication environment variables refer to the system environment variables related to Kerberos authentication required for task execution and are used to configure and transmit authentication information to ensure that the task can correctly interact with the Key Distribution Center (KDC).

[0073] The switching unit 32 is used to automatically switch to the backup key distribution center server for authentication when the authentication of the primary key distribution center server fails. The heartbeat monitoring mechanism can be used to monitor the availability of the primary key distribution center server. If the response timeout of the primary key distribution center server exceeds 5 seconds, the backup key distribution center server is switched to. When the primary key distribution center server recovers, the server can be automatically or manually switched back to the primary server.

[0074] The ticket renewal management unit 33 is used to automatically trigger a renewal command when it detects that the remaining validity period of the ticket is lower than a preset threshold. The preset threshold is set to 5 minutes. The 5-minute threshold is determined based on the default ticket validity period of the Kerberos protocol (usually 10 hours) and the task execution time to avoid frequent renewals affecting performance. The threshold can also be adjusted according to actual needs to ensure the validity and security of the Kerberos ticket during task submission and execution. The ticket renewal management unit 33 triggers a renewal command when the remaining validity period of the ticket is lower than 5 minutes to avoid task interruption due to ticket expiration. The remaining validity period of the ticket can be checked in real time, such as once every 1 minute. If the renewal fails, an alarm is triggered and re-authentication is attempted;

[0075] Specifically, the task scheduling middleware 30 can ensure the secure execution of tasks in a Kerberos authentication environment. Through the switchable design of the master key distribution center server and the backup server, it improves system reliability, reduces manual intervention, and improves the task scheduling efficiency of the big data platform.

[0076] A real-time feedback module 40 is used to monitor the Kerberos authentication status and results during task execution, collect security log information, and trigger an alarm mechanism when an abnormality is detected, such as authentication failure, ticket expiration, unauthorized access, and service anomalies;

[0077] like Figure 3 As shown, specifically including:

[0078] The monitoring submodule 41 is used to monitor the Kerberos authentication status and results of tasks in real time. By connecting to the log system and monitoring interface of each component, it obtains Kerberos authentication-related information during task execution in real time. The monitoring content includes the validity status and renewal records of Kerberos tickets, the number of authentication failures and failure reason logs of each component, the source IP address, time and operation records of unauthorized access attempts, and the matching status of authentication policies and security compliance during task execution. The monitoring data is analyzed and processed in real time.

[0079] The log collection submodule 42 is used to collect security logs and information related to Kerberos authentication, including authentication request records, ticket usage records, and abnormal event records. It is used to parse and analyze the collected Kerberos authentication logs to extract key indicators and abnormal characteristics, such as the number, frequency, and cause of authentication failures, and to detect whether there are abnormal authentication request patterns. By establishing a log analysis model, it can realize the identification of security events;

[0080] The exception handling submodule 43 is used to trigger an alarm and notify management personnel after detecting an abnormal situation, set abnormal detection rules and thresholds, and when the monitoring data or log analysis results exceed the threshold or meet the abnormal rules, it is judged as an abnormal situation. The exception handling submodule 43 immediately triggers the alarm mechanism and notifies the management personnel through email, SMS or system notification, and records the detailed information of the abnormal event to provide a basis for subsequent troubleshooting and security optimization.

[0081] Specifically, if Figure 4 As shown in the figure, the unified authentication management module, dynamic orchestration engine, task scheduling middleware, and real-time feedback module communicate with each other through the network. The authentication management system based on the Kerberos mechanism provides a centralized Kerberos authentication configuration management center, breaking the traditional model of independent configuration of each component and establishing a unified authentication management mechanism. By connecting Hive and Zookeeper and submitting Flink and Spark tasks based on Kerberos tickets, the workload of each component to configure Kerberos authentication separately is reduced. The AES-256-CTS encryption algorithm can be used to encrypt tickets and keys. The AES-256-CTS encryption algorithm is an existing technology that prevents ticket tampering and key leakage. The network time protocol ensures the clock synchronization of the key distribution center server (KDC), client, and server to avoid ticket invalidation due to time difference.

[0082] Among them Figure 1 As shown, a Kerberos-based authentication management method is applied to an authentication management system based on the Kerberos mechanism. The specific steps are as follows:

[0083] Before the authentication management system based on the Kerberos mechanism is started to be used, a key distribution center server (KDC) is deployed, an authentication domain (realm) and a key distribution center server (KDC) master key are configured, a big data platform user account is created, a basic Principal principal is created for components such as Hive, Zookeeper, and an initial keytab file is generated and securely stored, connection parameters of the unified authentication management module 10 and the key distribution center server (KDC) are configured, including the key distribution center server (KDC) address, the task authentication mode of the dynamic orchestration engine 20 is initialized, the component connection information and distribution strategy of the task scheduling middleware 30 are set;

[0084] S1, the task submission principal submits a task to the big data platform, and the task submission principal includes a user and a system;

[0085] Specifically, the user and the system are taken as the task submission principal, the task is submitted through a client tool, the task scheduling middleware 30 performs format verification and permission pre-checking on the task, initiates a task execution request to the big data platform, and submits task information to the big data platform, the submission mode of the task includes submission through a graphical operation interface provided by the big data platform, submission by inputting a specific instruction through a command line interface, and task submission through a configuration interface, and the task information includes a task type, a target component, a task specific execution logic and parameters;

[0086] S2, the unified authentication management module 10 performs configuration management related to user authentication on the task submitted in step S1, including creating a user, managing a Principal principal, and distributing a keytab file;

[0087] Specifically, the unified authentication management module 10 first queries the user library in the key distribution center server (KDC), and if the user does not exist, automatically creates a corresponding user account in the big data platform for the task submission principal, the user account includes a unique user identifier and user attributes, generates a corresponding Principal principal including a server host name for target components such as Hive, Zookeeper, Flink and Spark, and registers in the key distribution center server (KDC), generates a keytab file corresponding to each service Principal principal, and securely distributes the keytab file to the corresponding components and task execution environment, transmits it to the task execution location through an encrypted channel, and automatically deletes the keytab file after the task is executed;

[0088] S3, when receiving a task submission request, the dynamic orchestration engine 20 selects a Kerberos authentication mode according to the target component and the task type, generates an authentication script and a configuration file;

[0089] Specifically, upon receiving a task submission request, the dynamic orchestration engine 20 analyzes the target component and task type of the task, selects an authentication method suitable for the task from preset Kerberos authentication methods based on the analysis results, and generates a corresponding authentication script and configuration file based on the selected authentication method. The authentication script includes a kinit command for obtaining a Kerberos ticket, and the configuration file includes Kerberos-related parameters in the task execution environment, including the key distribution center address and realm. The kinit command is a command in the Kerberos tool used to obtain a Kerberos ticket.

[0090] S4. The task scheduling middleware 30 obtains the Kerberos ticket, distributes the task authenticated in step S3 to the corresponding target component for execution, and manages the ticket status during the execution process;

[0091] Specifically, the task scheduling middleware 30 obtains the Kerberos ticket related to the task from the unified authentication management module 10, distributes the authenticated task to the corresponding target component for execution, and during the task execution process, the task scheduling middleware 30 continuously monitors the Kerberos ticket status and validity period. When it detects that the ticket is about to expire, the task scheduling middleware 30 automatically triggers the ticket renewal process;

[0092] S5, real-time feedback module 40 monitors the authentication status, collects log information, triggers an alarm in case of anomalies and records the processing process;

[0093] Specifically, it collects key distribution center (KDC) server logs and component authentication logs, analyzes authentication requests, ticket status, and other information, such as the validity period of the initial ticket (TGT) and the number of authentication failures. When an authentication failure, ticket expiration, or key distribution center (KDC) server service anomaly is detected, an alarm is triggered and the anomaly details are recorded. For anomalies that can be handled automatically, such as ticket expiration, the dynamic orchestration engine 20 is automatically called to regenerate the ticket to ensure continuous task operation. For anomalies that cannot be handled automatically, such as authentication failure, an alarm is sent to the administrator via email, system notification, etc., containing the anomaly type, time, component, details, impact range, and handling suggestions. The threshold in the real-time feedback module 40 can be dynamically adjusted through the management interface.

[0094] Based on historical data or industry standards, specific exception details include: the ability to set authentication failure thresholds, such as triggering a warning if a single component fails authentication more than 5 times within 10 minutes, and a severe alarm if it fails more than 10 times; monitoring the validity period of the initial ticket (TGT), automatically triggering a ticket update when the remaining time is less than 5 minutes; and triggering a severe alarm when the same IP address initiates more than 20 authentication requests from different users within 5 minutes.

[0095] Example 1

[0096] Hive query task authentication process:

[0097] User A submits a Hive query task through a graphical interface. The task information includes the SQL statement and the target database. The unified authentication management module 10 creates a Kerberos Principal for user A, such as userA@REALM, and generates a Principal and a corresponding keytab file for the Hive service, such as hive / serverA@REALM. The dynamic orchestration engine 20 identifies the task type as a Hive query, matches the preset Hive Kerberos authentication method, and generates an authentication script. The task scheduling middleware 30 injects authentication environment variables and submits the task to the Hive server. The real-time feedback module 40 monitors the authentication status. If the Hive server returns a "ticket expired" error, it automatically triggers ticket renewal and retries the task.

[0098] Example 2

[0099] Fault tolerance for Spark jobs:

[0100] Assuming that the abnormal scenario is that the master key distribution center server cannot respond due to a network failure, the switching unit 32 detects that the master key distribution center server has timed out and automatically switches to the backup key distribution center server to complete the authentication. The log collection submodule 42 records the switching event and analyzes the cause of the master key distribution center server failure. The exception handling submodule 43 sends an alarm to notify the management personnel while continuing to ensure the normal execution of the Spark job;

[0101] Example 3

[0102] Dynamic authentication and monitoring of Flink computing tasks:

[0103] Assume that user B submits a Flink computing task through the command line. The task needs to process encrypted data streams in Kafka in real time and requires access to Kafka and HDFS through Kerberos authentication.

[0104] When user B submits a task, the dynamic orchestration engine 20 identifies the task type as a Flink streaming task, and the target components are Kafka and HDFS. The unified authentication management module 10 generates service principals for Kafka and HDFS, and distributes the encrypted keytab file to the task node. The dynamic orchestration engine 20 matches the Kerberos authentication method of the Flink task, generates a script and configuration containing the kinit command, and the task scheduling middleware 30 injects the authentication environment before the task starts to ensure that the Flink task can automatically obtain the ticket and access Kafka and HDFS. When the ticket renewal management unit 33 detects that the remaining validity period of the ticket is less than 5 minutes, it automatically calls the kinit-R command to renew it. The real-time feedback module 40 monitors the Kafka authentication failure, immediately triggers an alarm and records the error log, and the exception handling submodule 43 automatically retries the authentication. If multiple failures occur, the management personnel are notified and the task is paused to avoid data loss.

[0105] In summary, this system solves the fragmentation problem of multi-component Kerberos authentication in big data platforms through automated Kerberos configuration and dynamic orchestration, reducing the tedious operation of configuring each component separately in the traditional method. The dynamic orchestration engine can automatically match the Kerberos authentication method according to the task type and generate the corresponding authentication script and configuration file without manual intervention, which has good application prospects.

[0106] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. An authentication management method based on the Kerberos mechanism, characterized in that: The specific steps are as follows: S1. The task submitter submits the task to the big data platform. The task submitter includes the user and the system. S2, the unified authentication management module (10) performs configuration management related to user authentication for the task submitted in step S1, including creating users, managing principals, and distributing keytab files; S3. When receiving a task submission request, the dynamic orchestration engine (20) selects the Kerberos authentication method according to the target component and task type, and generates an authentication script and configuration file; S4, the task scheduling middleware (30) obtains the Kerberos ticket, distributes the task authenticated in step S3 to the corresponding target component to perform the operation, and manages the ticket status during the execution process; S5, real-time feedback module (40) monitors the authentication status, collects log information, triggers alarms in case of abnormalities and records the processing process; The step S2 specifically includes: the unified authentication management module (10) creates a corresponding user account for the task submitting subject in the big data platform, the user account includes a unique user identifier and user attributes, creates a corresponding service principal subject for task execution for the target component, and generates a keytab file corresponding to each service principal subject, and securely distributes the keytab file to the corresponding component and task execution environment; The step S3 specifically includes: when the dynamic orchestration engine (20) receives the task submission request, it analyzes the target component and task type of the task, selects an authentication method suitable for the task from the preset Kerberos authentication methods according to the analysis result, and generates a corresponding authentication script and configuration file based on the selected authentication method, wherein the authentication script includes a kinit command for obtaining a Kerberos ticket, and the configuration file includes Kerberos related parameters in the task execution environment, including the key distribution center address and domain.

2. The Kerberos-based authentication management method according to claim 1, wherein: The step S1 specifically includes: using the user and the system as the task submission subject, initiating a task execution request to the big data platform, and submitting the task information to the big data platform. The task submission methods include submitting through the graphical operation interface provided by the big data platform, submitting by entering specific instructions using the command line interface, and submitting the task through the configuration interface. The task information includes the task type, target component, specific task execution logic and parameters.

3. The Kerberos-based authentication management method according to claim 1, wherein: The step S4 specifically includes: the task scheduling middleware (30) obtains the Kerberos ticket related to the task from the unified authentication management module (10), distributes the authenticated task to the corresponding target component for execution, and during the task execution process, the task scheduling middleware (30) continuously monitors the Kerberos ticket status and validity period. When it is detected that the ticket is about to expire, the task scheduling middleware (30) automatically triggers the ticket renewal process.

4. An authentication management system based on the Kerberos mechanism, applied to the authentication management method based on the Kerberos mechanism according to claim 1, characterized in that: include: A unified authentication management module (10) is used to communicate with a key distribution center server via a network and centrally manage Kerberos authentication configurations of target components in a big data platform, wherein the target components include big data components that communicate with the unified authentication management module (10), and the big data components include Hive, Zookeeper, Flink, and Spark, and the Kerberos authentication configuration includes user creation, Principal subject management, and keytab file distribution operations; A dynamic orchestration engine (20), based on target components and task types, is used to automatically select an adapted Kerberos authentication method from an authentication method library when a task is submitted, and to generate a corresponding authentication configuration file and an execution script. The task types include Flink computing tasks, Spark job tasks, and Hive query tasks. The authentication method library is used to store Kerberos authentication methods corresponding to different task types. Task scheduling middleware (30), which is used to securely distribute Kerberos-authenticated tasks to corresponding components for execution and maintain the authentication status and ticket validity period during the task life cycle; A real-time feedback module (40) is used to monitor the Kerberos authentication status and results during task execution, collect security log information, and trigger an alarm mechanism when abnormal conditions are detected, such as authentication failure, ticket expiration, unauthorized access, and service abnormality; The unified authentication management module (10), the dynamic orchestration engine (20), the task scheduling middleware (30) and the real-time feedback module (40) communicate with each other via a network.

5. The Kerberos-based authentication management system according to claim 4, characterized in that: The unified authentication management module (10) is used to create a Principal subject and generate an encrypted keytab file through a key distribution center server, wherein the key distribution center server includes a primary key distribution center server and a backup key distribution center server. The unified authentication management module (10) includes: User management submodule (11), used to create and manage user accounts in the big data platform; The principal management submodule (12) is used to create, manage and maintain the Kerberos Principal principals corresponding to each component; The keytab file management submodule (13) is used to generate, store and distribute keytab files, and adopts encryption storage and access permission control operations for the keytab files.

6. The Kerberos-based authentication management system according to claim 4, characterized in that: The dynamic orchestration engine (20) includes: A task type identification unit (21), used for identifying the task type to which the submitted task belongs; An authentication method matching unit (22) is used for matching based on a preset authentication method library, wherein the authentication method library contains task types and corresponding Kerberos authentication methods, and the authentication parameters of the Kerberos authentication method include Principal format and ticket validity period; The script generation unit (23) is used to generate an authentication script containing a kinit command based on a matching preset Kerberos authentication method, and associate it with a corresponding keytab file path.

7. The Kerberos-based authentication management system according to claim 5, characterized in that: The task scheduling middleware (30) includes: The authentication environment injection unit (31) is used to obtain the Kerberos ticket before the task is submitted and inject the authentication information into the task running environment variables; A switching unit (32) is used to automatically switch to a backup key distribution center server for authentication when the master key distribution center server fails to authenticate; The bill renewal management unit (33) is used to automatically trigger a renewal command when it is detected that the remaining validity period of the bill is lower than a preset threshold, and the preset threshold is set to 5 minutes.

8. The Kerberos-based authentication management system according to claim 4, characterized in that: The real-time feedback module (40) includes: The monitoring submodule (41) is used to monitor the Kerberos authentication status and results of the task in real time; A log collection submodule (42) is used to collect security logs and information related to Kerberos authentication, including authentication request records, ticket usage records, and abnormal event records; The exception handling submodule (43) is used to trigger an alarm and notify management personnel after detecting an abnormal situation.

Images