A multi-data-source authentication method and device, electronic equipment and storage medium
By caching authentication information obtained remotely locally, the limitation of multiple data source authentication in a single JVM process in the Kerberos authentication mechanism is resolved, enabling efficient access and authentication of multiple data sources and improving the system's flexibility and performance.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- BEIJING BAIDU NETCOM SCI & TECH CO LTD
- Filing Date
- 2022-11-16
- Publication Date
- 2026-07-07
AI Technical Summary
In existing technologies, the Kerberos authentication mechanism does not support the simultaneous use of multiple Kerberos authentications in a single JVM process. This results in the inability to function properly when accessing multiple Hive clusters with Kerberos authentication enabled at the same time. KDC becomes a performance bottleneck, and Java native authentication is limited, failing to effectively support security authentication for multiple data sources.
By determining whether to cache the authentication information of the data source to be authenticated, if no cache is used, a connection is established with the target remote server, the target remote server is called to obtain the authentication information, and it is cached locally for use in subsequent authentication processes. Authentication of multiple data sources is supported.
It improves the efficiency of data source access, reduces business interaction costs, supports authentication of multiple data sources, overcomes the limitations of the Kerberos authentication mechanism, and enhances the flexibility and performance of the system.
Smart Images
Figure CN115906118B_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of artificial intelligence, specifically cloud computing and big data technologies, and can be applied in scenarios such as intelligent cloud. Background Technology
[0002] In the field of big data technology and data processing, most data sources use Kerberos (a computer network authorization protocol used to securely authenticate personal communications in insecure networks) for security authentication. Data sources that enable Kerberos authentication require Kerberos authentication every time they are accessed. Summary of the Invention
[0003] This disclosure provides a multi-data source authentication method, apparatus, electronic device, and storage medium.
[0004] According to one aspect of this disclosure, a multi-data source authentication method is provided, comprising:
[0005] Determine whether to cache authentication information from the data source to be authenticated;
[0006] Without caching the authentication information of the data source to be authenticated, a connection is established with the target remote server;
[0007] The target remote server is invoked to connect to the data source to be authenticated, and the authentication information is obtained from the data source to be authenticated.
[0008] The authentication information is cached, and the data source to be authenticated is authenticated based on the authentication information.
[0009] According to another aspect of this disclosure, a multi-data source authentication device is provided, comprising:
[0010] The information determination module is used to determine whether to cache authentication information from the data source to be authenticated;
[0011] The connection creation module creates a connection with the target remote server if the information determination module determines that there is no cached authentication information of the data source to be authenticated.
[0012] The information acquisition module is used to call the target remote server to connect to the data source to be authenticated, and to obtain the authentication information from the data source to be authenticated;
[0013] The first authentication module is used to cache the authentication information and authenticate the data source to be authenticated based on the authentication information.
[0014] According to another aspect of this disclosure, an electronic device is provided, comprising:
[0015] At least one processor; and
[0016] A memory communicatively connected to the at least one processor; wherein,
[0017] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform any of the methods described in this disclosure.
[0018] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause the computer to perform any of the methods described in this disclosure.
[0019] According to another aspect of this disclosure, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the method described in any one of this disclosure.
[0020] This disclosure embodiment can support authentication from multiple data sources.
[0021] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description
[0022] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:
[0023] Figure 1 This is a schematic diagram of a multi-data source authentication method according to this disclosure;
[0024] Figure 2 This is another schematic diagram of the multi-data source authentication method according to this disclosure;
[0025] Figure 3 This is a schematic diagram illustrating the process of obtaining authentication information from a data source to be authenticated, in accordance with this disclosure.
[0026] Figure 4 This is another schematic diagram illustrating the acquisition of authentication information from a data source to be authenticated, based on this disclosure.
[0027] Figure 5 This is a schematic diagram illustrating the effective updating of authentication information based on this disclosure;
[0028] Figure 6 This is another schematic diagram of the multi-data source authentication method disclosed herein;
[0029] Figure 7This is another schematic diagram illustrating the acquisition of authentication information from a data source to be authenticated according to this disclosure;
[0030] Figure 8 This is a schematic diagram of a multi-data source authentication device according to the present disclosure;
[0031] Figure 9 This is a block diagram of an electronic device used to implement the multi-data source authentication method of the embodiments of this disclosure. Detailed Implementation
[0032] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.
[0033] In the field of big data technology and data processing, in order to ensure data security, Hive clusters (a data warehouse tool based on Hadoop) usually use Kerberos authentication to ensure that sensitive data will not be leaked.
[0034] After enabling Kerberos authentication, services must obtain authentication tickets from the KDC (Key Distribution Center) before interacting with each other. YARN (Yet Another Resource Negotiator) is a general-purpose resource management system that provides unified resource management and scheduling for upper-layer applications. During the execution of a YARN task, many task containers may be generated. Each task container may access HDFS (Hadoop Distributed File System). Since authentication tickets are required before accessing Hadoop, this puts significant pressure on the KDC, making it easy for the KDC to become a performance bottleneck.
[0035] Kerberos also supports authorizing external applications to access internal cluster data via principle (user security authentication information) authentication. In actual business or production environments, depending on business needs, it may be necessary to access multiple Hive clusters with Kerberos authentication enabled simultaneously. Due to Kerberos' authentication mechanism, it is not supported for external applications to hold multiple principles for security authentication at the same time, which causes services that need to access multiple Kerberos-authenticated clusters simultaneously to fail to operate normally.
[0036] In related technologies, Java's native support for Kerberos authentication technology has limitations. These limitations mean that only one type of Kerberos authentication can be used (supported) at a time within a single JVM (Java Virtual Machine) process. In other words, more than two types of Kerberos authentication cannot be used (supported) simultaneously within a single JVM process.
[0037] To support authentication from multiple data sources, this disclosure provides a method for authentication from multiple data sources. First, it is determined whether the authentication information of the data source to be authenticated is cached. If the authentication information of the data source to be authenticated is not cached, a connection is established with the target remote server, the target remote server is invoked to connect to the data source to be authenticated, and the authentication information is obtained from the data source to be authenticated. Furthermore, the authentication information is cached, and the data source to be authenticated is authenticated based on the authentication information.
[0038] In this embodiment of the disclosure, when there is no local cache of authentication information of the data source to be authenticated, a connection is established with the target remote server. The connection to the data source to be authenticated is made by calling the target remote server, and then the authentication information is obtained from the data source to be authenticated. Furthermore, the obtained authentication information is cached locally so that the cached authentication information can be directly used to authenticate the data source to be authenticated in the subsequent authentication process. This can support the authentication of multiple data sources and improve the access efficiency of data sources.
[0039] The multi-data source authentication method provided in the embodiments of this disclosure will be described in detail below.
[0040] The multi-data source authentication method provided in this disclosure can be applied to electronic devices, such as server devices, smart terminal devices, etc. For example, it can be specifically applied to Java applications in electronic devices. The multi-data source authentication method provided in this disclosure can also be applied in scenarios such as smart clouds.
[0041] See Figure 1 , Figure 1 A flowchart illustrating a multi-data source authentication method provided in this disclosure includes the following steps:
[0042] S101, determine whether to cache authentication information of the data source to be authenticated.
[0043] Local services (such as Java applications) check their local cache to determine if the authentication information for the data source to be authenticated is cached there. The data source to be authenticated can be any data source that requires authentication.
[0044] In one possible implementation, any data source to be authenticated is a Hive data source that is authenticated using Kerberos.
[0045] In one possible implementation, the authentication information of the data source to be authenticated may include: the identification information of the data source to be authenticated, port information, and a delegation token.
[0046] Delegation tokens are a lightweight authentication method in Hadoop that can supplement Kerberos authentication. When authenticating data sources with Kerberos authentication enabled, a Delegation token can be used for authorization only once, and subsequent access requests can continue to use the same Delegation token without requiring re-authorization, thus improving the efficiency of data source access.
[0047] In one example, the identification information of the data source to be authenticated can be the name, address, etc. of the data source to be authenticated, which can be represented by the host. The port information represents the port number of the data source to be authenticated, which can be represented by the port.
[0048] In this embodiment of the disclosure, the identification information and port information of the data source to be authenticated can be used to accurately locate the data source to be authenticated, and the Delegation token can be used to authenticate the data source to be authenticated.
[0049] In one possible implementation, if the authentication information of the data source to be authenticated is cached, the data source to be authenticated is authenticated based on the authentication information.
[0050] In one example, the authentication information of the data source to be authenticated is cached locally in the form of a Map, where Map is a commonly used data structure in Java programming.
[0051] In this embodiment of the disclosure, when the local service determines that there is authentication information of the data source to be authenticated in the local cache, it directly authenticates the data source to be authenticated based on the cached authentication information, so as to support the authentication of multiple data sources and improve the access efficiency of the data source.
[0052] S102, without caching the authentication information of the data source to be authenticated, establish a connection with the target remote server.
[0053] In one example, if the local service determines that it does not have the authentication information for the data source to be authenticated cached locally, it establishes a JSch Session connection with the target remote server. If the connection with the target remote server is successful, the local service can invoke the target remote server to connect to the data source to be authenticated; if the connection with the target remote server fails, a connection failure message is displayed.
[0054] Jsch is a pure Java implementation of SSH2 (Secure Shell), allowing connections to an sshd (secure shell) server. The sshd service enables remote shell access to other hosts using the SSH protocol. For example, the target remote server could be an sshd server.
[0055] For example, if the local service determines that there is no cached authentication information of the data source to be authenticated locally, it connects to the sshd server (i.e., the target remote server) by creating a Jsch Session.
[0056] S103, calls the target remote server to connect to the data source to be authenticated, and obtains authentication information from the data source to be authenticated.
[0057] In one example, after establishing a connection with the target remote server, a new process is started on the target remote server by sending a command. This process is then used to connect to the data source to be authenticated and obtain authentication information from the data source.
[0058] For example, after establishing a connection with the target remote server, the local service sends a process creation command to the target remote server to create a new process on the target remote server. This new process then connects to the data source to be authenticated. Further, after connecting to the data source, the authentication information of the data source is printed to the local service's log file via JSch, and the authentication information of the data source can then be obtained from the log file. The process of the new process connecting to the data source can involve obtaining an authentication ticket from the KDC, or performing authentication using pre-stored principles, etc.
[0059] S104 caches the authentication information and performs authentication on the data source to be authenticated based on the authentication information.
[0060] After obtaining the authentication information of the data source to be authenticated, the local service caches the authentication information in a Map and then authenticates the data source based on the authentication information.
[0061] In this embodiment of the disclosure, when there is no local cache of authentication information of the data source to be authenticated, a connection is established with the target remote server. The target remote server is called to start a new process to connect to the data source to be authenticated, and then the authentication information is obtained from the data source to be authenticated. Furthermore, the obtained authentication information is cached locally so that the cached authentication information can be directly used to authenticate the data source to be authenticated in subsequent authentication processes. This can support authentication of multiple data sources, improve the access efficiency of data sources, and reduce business interaction costs.
[0062] See Figure 2 , Figure 2 A flowchart illustrating another multi-data source authentication method provided in this disclosure embodiment includes the following steps:
[0063] S201, determine whether to cache authentication information for the data source to be authenticated.
[0064] S202: Establish a connection with the target remote server without caching authentication information from the data source to be authenticated.
[0065] The implementation process of steps S201-S202 can be referred to the implementation process of steps S101-S102 above, and will not be repeated here in this embodiment.
[0066] S203 calls the target remote server and uses the Secure File Transfer Protocol (SFTP) channel to detect whether the target application exists in the target cluster.
[0067] In one example, after establishing a connection with the target remote server, the local service sends a process creation command to the target remote server, creating a new process on the target remote server and opening an SFTP (SSH File Transfer Protocol) channel (SFTPChannel). The SFTP channel is used to detect whether the target application exists in the target cluster. The target application is used to print the Delegation token, which can be represented as DelegationTokenPrinter. The target cluster is the cluster where the data source to be authenticated resides. For example, the target cluster can be the cluster where the data source to be authenticated resides, or any node or sub-cluster within that cluster that needs to be accessed.
[0068] S204: If the target application exists in the target cluster, execute the target application to obtain the authentication information of the data source to be authenticated.
[0069] If the target application is detected in the target cluster, a task channel (execChannel) is opened to remotely execute the target application to print the Delegation token, thereby obtaining the authentication information of the data source to be authenticated.
[0070] S205: If the target application does not exist in the target cluster, copy the target application to the target cluster, execute the target application, and obtain the authentication information of the data source to be authenticated.
[0071] If the target application is not detected in the target cluster, the target application is copied to the target cluster via the SFTP channel. Then, the execChannel is enabled to remotely execute the target application to print the Delegationtoken, thereby obtaining the authentication information of the data source to be authenticated.
[0072] S206 caches authentication information and performs authentication on the data source to be authenticated based on the authentication information.
[0073] After obtaining the authentication information of the data source to be authenticated, the local service caches the authentication information in a Map and then authenticates the data source based on the authentication information.
[0074] In this embodiment, when there is no local cache of authentication information for the data source to be authenticated, a connection is established with the target remote server, and the target remote server is invoked to detect whether the target application exists in the target cluster via the SFTP channel. If it exists, the target application is remotely executed to obtain the authentication information for the data source to be authenticated. If it does not exist, the target application is copied to the target cluster and remotely executed to obtain the authentication information for the data source to be authenticated. Furthermore, the obtained authentication information is cached locally so that it can be directly used to authenticate the data source to be authenticated in subsequent authentication processes. This supports authentication of multiple data sources and improves the access efficiency of data sources.
[0075] In one possible implementation, see Figure 3 The process of executing the target application and obtaining authentication information from the data source to be authenticated in steps S204 and S205 above may include:
[0076] S301, when the target application is in HA format, accesses the registration directory of hiveserver2 based on the target cluster identifier and port information contained in the target application.
[0077] Different target clusters require different target application formats. In the case of a target application in HA format, for example, the target application could be a `hive jdbcUrl`. In a HiveServer2 HA high-availability deployment, the `host:port` specified in `hive jdbcUrl` actually represents the address and port of the target cluster. Here, HA is a file type, and HiveServer2 is a service that enables clients and other electronic devices to execute Hive queries.
[0078] For example, the hive jdbcUrl is represented as: jdbc:hive2: / / master-d21f64f-2:2181,master-d21f64f-1:2181 / default;serviceDiscoveryMode=zookeeper;zooKeeperNamespace=hiveserver2. In the hive jdbcUrl, "master-d21f64f-2:2181,master-d21f64f-1:2181 / default" represents the path to the target cluster ZooKeeper (a distributed application coordination service). The specified host is the identifier of ZooKeeper (such as an address), and the specified port is hiveserver2. Thus, it is possible to use the host and port of ZooKeeper to create a ZooKeeper connection and access the registered directory of hiveserver2.
[0079] S302: Obtain the address information of the currently active hiveserver2 from the hiveserver2 registry.
[0080] Access the registry directory of hiveserver2, retrieve the address information of the currently active hiveserver2 from the registry directory, and obtain the real address information of the Hive query service. This address information includes the identification information (host) and port information of the data source to be authenticated.
[0081] S303 connects to the data source to be authenticated based on address information.
[0082] Using the obtained host and port, create a Hive connection to connect to the data source to be authenticated.
[0083] S304: Obtain authentication information from the data source to be authenticated.
[0084] After connecting to the data source to be authenticated, the authentication information is obtained from the data source and written into the response information (host, port, and Delegation token) and returned to the local service.
[0085] In this embodiment of the disclosure, when the target application is in HA format, the registration directory of hiveserver2 is accessed based on the target cluster identifier and port information contained in the target application. The host and port of the currently active hiveserver2 are obtained from the registration directory of hiveserver2. A Hive connection is created using the obtained host and port to connect to the data source to be authenticated, and then authentication information is obtained from the data source to be authenticated, thus realizing the remote printing of authentication information.
[0086] In one possible implementation, see Figure 4 The process of executing the target application and obtaining authentication information from the data source to be authenticated in steps S204 and S205 above may further include:
[0087] S401, if the target application is not in HA format, create user group information (UGI) based on the location of hdfsPricipal and keytab contained in the target application.
[0088] In one example, the target application's variable parameters specify the location of the hdfsPricipal (HDFS user security authentication information) and the keytab (keystore). If the target application is not in HA format, the location of hdfsPricipal and keytab can be used to create Hadoop's UGI (UserGroupInformation).
[0089] The created UGI is a Kerberos-certified ticket, which allows access to the Hadoop system.
[0090] S402, based on UGI, connects to the data source to be authenticated.
[0091] For example, after successfully creating a UGI, you can create a HiveConnection under the UGI using jdbc:hive2: / / host:port / ;principal=hive / _HOST@xxx to establish a connection to the data source to be authenticated. Here, HiveConnection represents a Hive connection object, which can be used to query Hive.
[0092] S403: Obtain authentication information from the data source to be authenticated.
[0093] For example, after connecting to the data source to be authenticated, you can use getDelegationToken(hdfsPricipal) to obtain the Delegation token of the current proxy user hdfsPricipal from the data source to be authenticated, obtain the authentication information, and write the obtained authentication information (host, port and Delegation token) into the response information and return it to the local service.
[0094] In this embodiment of the disclosure, when the target application is not in the HA format, a UGI is created based on the location of the hdfsPricipal and the keytab contained in the target application. The UGI is then used to connect to the data source to be authenticated, and authentication information is obtained from the data source to be authenticated, thereby enabling remote printing of the authentication information.
[0095] Authentication information typically has a limited validity period. Within this period, the authentication information is valid; after the period expires, it becomes invalid and can no longer be used for data source authentication. For example, the validity period of the Delegation token in a Hive data source is controlled by the attribute hive.cluster.delegation.token.renew-interval in hiveserver2.xml, with a default validity period of 1 day.
[0096] In one possible implementation, see Figure 5 Based on the above embodiments, the following steps can also be performed:
[0097] S501 adds timestamp information to the cached authentication information.
[0098] After obtaining the authentication information, the authentication information is cached and timestamped to facilitate subsequent determination of whether the authentication information is within its validity period.
[0099] S502 determines whether the authentication information has expired based on the timestamp information and the preset validity period.
[0100] For example, at certain time intervals, the system checks whether the time interval between the timestamp corresponding to the cached authentication information and the detection time exceeds a preset validity period. If it exceeds the preset validity period, the authentication information has expired and can no longer be used for data source authentication; therefore, the authentication information is deleted from the cache. If it has not exceeded the preset validity period, the authentication information is still valid and can continue to be used for data source authentication. The time interval and preset validity period can be set according to requirements. For example, the time interval can be set to 5 minutes, 10 minutes, or 15 minutes, and the preset validity period can be set to 1 hour, 2 hours, or 3 hours, etc.
[0101] S503: Delete authentication information if it has expired.
[0102] In this embodiment of the disclosure, the cached authentication information is marked with timestamp information, and then the authentication information is determined to be expired based on the timestamp information and the preset validity period, so as to delete the authentication information and obtain new authentication information in a timely manner when the authentication information expires.
[0103] For example, such as Figure 6 As shown, Figure 6 The document demonstrates the implementation process of a multi-data source authentication method, which includes:
[0104] Step 1: The local service determines whether the local cache caches authentication information of the data source to be authenticated;
[0105] Step 2: If the local service determines that the authentication information of the data source to be authenticated is cached in the local cache, it directly obtains the authentication information of the data source to be authenticated and performs authentication based on the authentication information.
[0106] Step 3: If the local service determines that the authentication information of the data source to be authenticated is not cached in the local cache, it will create a Jsch session to establish a connection with the target remote server.
[0107] Step 4: The local service calls the target remote server and checks whether the target application exists in the target cluster through the SFTP channel. The target application (DelegationTokenPrinter) is used to print the Delegation token, and the target cluster is the cluster where the data source to be authenticated is located.
[0108] Step 5: If the target application is not detected in the target cluster, copy the target application to the target cluster;
[0109] Step 6: Open the task channel, remotely execute the target application, and obtain the authentication information of the data source to be authenticated;
[0110] Step 7: Cache the authentication information and authenticate the data source to be authenticated based on the authentication information.
[0111] For example, such as Figure 7 As shown, Figure 7 The document illustrates an implementation process for obtaining authentication information from a data source to be authenticated (i.e., the process of remotely executing a target application to obtain authentication information from the data source to be authenticated, which is executed by a local service calling a target remote server within the target cluster). This process includes:
[0112] 1) If the target application is in HA format, access the registration directory of hiveserver2 based on the target cluster identifier and port information contained in the target application, where the target cluster identifier and port information contained in the target application correspond to the target node in the target cluster.
[0113] 2) Obtain the address information of the currently active HiveServer2 from the registration directory of HiveServer2. This address information includes the identification information (host) and port information of the data source to be authenticated.
[0114] 3) Connect to the data source to be authenticated based on address information (host, port);
[0115] 4) Obtain authentication information from the data source to be authenticated.
[0116] 5) If the target application is not in HA format, create user group information UGI based on the location of hdfsPricipal and keytab contained in the target application;
[0117] 6) Connect to the data source to be authenticated based on UGI;
[0118] 7) Obtain authentication information from the data source to be authenticated.
[0119] In this embodiment of the disclosure, when there is no local cache of authentication information of the data source to be authenticated, a connection is established with the target remote server. The target remote server is called to start a new process to connect to the data source to be authenticated, and then the authentication information is obtained from the data source to be authenticated. Furthermore, the obtained authentication information is cached locally so that the cached authentication information can be directly used to authenticate the data source to be authenticated in subsequent authentication processes. This can support authentication of multiple data sources and improve the access efficiency of data sources.
[0120] For example, the DelegationToken of the data source to be authenticated can be obtained using the following code:
[0121]
[0122]
[0123] This disclosure also provides a multi-data source authentication device, see [link to documentation]. Figure 8 The device includes:
[0124] The information determination module 801 is used to determine whether to cache the authentication information of the data source to be authenticated;
[0125] The connection creation module 802 creates a connection with the target remote server if the information determination module determines that there is no cached authentication information of the data source to be authenticated.
[0126] The information acquisition module 803 is used to call the target remote server to connect to the data source to be authenticated and obtain authentication information from the data source to be authenticated;
[0127] The first authentication module 804 is used to cache authentication information and authenticate the data source to be authenticated based on the authentication information.
[0128] In this embodiment of the disclosure, when there is no local cache of authentication information of the data source to be authenticated, a connection is established with the target remote server. The target remote server is called to start a new process to connect to the data source to be authenticated, and then the authentication information is obtained from the data source to be authenticated. Furthermore, the obtained authentication information is cached locally so that the cached authentication information can be directly used to authenticate the data source to be authenticated in subsequent authentication processes. This can support authentication of multiple data sources and improve the access efficiency of data sources.
[0129] In one possible implementation, the above-described apparatus further includes:
[0130] The second authentication module is used to authenticate the data source to be authenticated based on the authentication information when the information determination module determines that there is authentication information of the data source to be authenticated in the cache.
[0131] In one possible implementation, the information acquisition module 803 includes:
[0132] The program detection submodule is used to call the target remote server and detect whether the target application exists in the target cluster through the secure file transfer protocol SFTP channel. The target application is used to print the delegation token, and the target cluster is the cluster where the data source to be authenticated is located.
[0133] The first information acquisition submodule is used to execute the target application and obtain the authentication information of the data source to be authenticated when the program detection submodule detects that the target application exists in the target cluster.
[0134] The second information acquisition submodule is used to copy the target application to the target cluster and execute the target application to obtain the authentication information of the data source to be authenticated when the program detection submodule detects that the target application does not exist in the target cluster.
[0135] In one possible implementation, the data source to be authenticated is a Hive data source, and the execution of the target application to obtain the authentication information of the data source to be authenticated includes:
[0136] If the target application is in HA format, access the registration directory of hiveserver2 based on the target cluster identifier and port information contained in the target application;
[0137] Retrieve the address information of the currently active HiveServer2 from the HiveServer2 registry. The address information includes the identifier information of the data source to be authenticated and the port information.
[0138] Connect to the data source to be authenticated based on the address information;
[0139] Obtain authentication information from the data source to be authenticated.
[0140] In one possible implementation, the above-mentioned execution of the target application to obtain authentication information from the data source to be authenticated further includes:
[0141] If the target application is not in HA format, create user group information UGI based on the location of hdfsPricipal and keytab contained in the target application;
[0142] Connect to the data source to be authenticated based on UGI;
[0143] Retrieve authentication information from the data source to be authenticated.
[0144] In one possible implementation, the authentication information includes: the identification information of the data source to be authenticated, port information, and a delegation token.
[0145] In one possible implementation, the above-described apparatus further includes:
[0146] The timestamp module is used to add timestamp information to the cached authentication information;
[0147] The expiration determination module is used to determine whether the authentication information has expired based on the timestamp information and the preset validity period.
[0148] The information deletion module is used to delete authentication information if the expiration judgment module determines that the authentication information has expired.
[0149] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information in this technical solution comply with relevant laws and regulations and do not violate public order and good morals. It should be noted that the head model in this embodiment is not a head model specific to any particular user and does not reflect the personal information of any particular user. It should also be noted that the two-dimensional face images in this embodiment are from publicly available datasets.
[0150] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.
[0151] This disclosure provides an electronic device, comprising:
[0152] At least one processor; and
[0153] A memory that is communicatively connected to at least one processor; wherein,
[0154] The memory stores instructions that can be executed by at least one processor to enable the at least one processor to perform any of the methods of this disclosure.
[0155] This disclosure provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to perform any of the methods described in this disclosure.
[0156] This disclosure provides a computer program product, including a computer program that, when executed by a processor, implements any of the methods described in this disclosure.
[0157] Figure 9 A schematic block diagram of an example electronic device 900 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.
[0158] like Figure 9As shown, device 900 includes a computing unit 901, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 902 or a computer program loaded from storage unit 908 into random access memory (RAM) 903. RAM 903 may also store various programs and data required for the operation of device 900. The computing unit 901, ROM 902, and RAM 903 are interconnected via bus 904. Input / output (I / O) interface 905 is also connected to bus 904.
[0159] Multiple components in device 900 are connected to I / O interface 905, including: input unit 906, such as keyboard, mouse, etc.; output unit 907, such as various types of monitors, speakers, etc.; storage unit 908, such as disk, optical disk, etc.; and communication unit 909, such as network card, modem, wireless transceiver, etc. Communication unit 909 allows device 900 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.
[0160] The computing unit 901 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 901 performs the various methods and processes described above, such as the multi-data source authentication method. For example, in some embodiments, the multi-data source authentication method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 908. In some embodiments, part or all of the computer program may be loaded and / or installed on device 900 via ROM 902 and / or communication unit 909. When the computer program is loaded into RAM 903 and executed by the computing unit 901, one or more steps of the multi-data source authentication method described above may be performed. Alternatively, in other embodiments, the computing unit 901 may be configured to perform the multi-data source authentication method by any other suitable means (e.g., by means of firmware).
[0161] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.
[0162] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.
[0163] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.
[0164] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).
[0165] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.
[0166] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.
[0167] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.
[0168] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.
Claims
1. A multi-data source authentication method, comprising: Determine whether to cache authentication information from the data source to be authenticated; The authentication information includes: the identification information, port information, and delegation token of the data source to be authenticated; Without caching the authentication information of the data source to be authenticated, a connection is established with the target remote server; The target remote server is invoked to detect the presence of a target application in the target cluster via the Secure File Transfer Protocol (SFTP) channel. The target application is used to print a delegation token. The target cluster is the cluster where the data source to be authenticated is located. If the target application exists in the target cluster, execute the target application to obtain the authentication information of the data source to be authenticated; If the target application does not exist in the target cluster, the target application is copied to the target cluster and executed to obtain the authentication information of the data source to be authenticated; The authentication information is cached, and the data source to be authenticated is authenticated based on the authentication information.
2. The method according to claim 1, further comprising: If the authentication information of the data source to be authenticated is cached, the data source to be authenticated is authenticated based on the authentication information.
3. The method according to claim 1, wherein, The data source to be authenticated is a Hive data source. The process of executing the target application and obtaining the authentication information of the data source to be authenticated includes: If the target application is in HA format, access the registration directory of hiveserver2 based on the target cluster identifier and port information contained in the target application; Obtain the address information of the currently active HiveServer2 from the registration directory of HiveServer2. The address information includes the identification information and port information of the data source to be authenticated. Based on the address information, connect to the data source to be authenticated; The authentication information is obtained from the data source to be authenticated.
4. The method according to claim 1 or 3, wherein, The step of executing the target application and obtaining the authentication information of the data source to be authenticated further includes: If the target application is not in Ha format, create user group information UGI based on the location of hdfsPricipal and keytab contained in the target application; Based on the UGI, connect to the data source to be authenticated; The authentication information is obtained from the data source to be authenticated.
5. The method according to claim 1, further comprising: The cached authentication information is timestamped. Based on the timestamp information and the preset validity period, determine whether the authentication information has expired; If the authentication information expires, the authentication information will be deleted.
6. A multi-data source authentication device, comprising: The information determination module is used to determine whether to cache authentication information from the data source to be authenticated; The authentication information includes: the identification information, port information, and delegation token of the data source to be authenticated; The connection creation module creates a connection with the target remote server if the information determination module determines that there is no cached authentication information of the data source to be authenticated. The program detection submodule is used to call the target remote server and detect whether the target application exists in the target cluster through the secure file transfer protocol SFTP channel. The target application is used to print the delegation token, and the target cluster is the cluster where the data source to be authenticated is located. The first information acquisition submodule is used to execute the target application and obtain the authentication information of the data source to be authenticated when the program detection submodule detects that the target application exists in the target cluster. The second information acquisition submodule is used to copy the target application to the target cluster and execute the target application to obtain the authentication information of the data source to be authenticated when the program detection submodule detects that the target application does not exist in the target cluster. The first authentication module is used to cache the authentication information and authenticate the data source to be authenticated based on the authentication information.
7. The apparatus according to claim 6, further comprising: The second authentication module is used to authenticate the data source to be authenticated based on the authentication information when the information determination module determines that the authentication information of the data source to be authenticated is cached.
8. The apparatus according to claim 6, wherein, The data source to be authenticated is a Hive data source. The process of executing the target application and obtaining the authentication information of the data source to be authenticated includes: If the target application is in HA format, access the registration directory of hiveserver2 based on the target cluster identifier and port information contained in the target application; Obtain the address information of the currently active HiveServer2 from the registration directory of HiveServer2. The address information includes the identification information and port information of the data source to be authenticated. Based on the address information, connect to the data source to be authenticated; The authentication information is obtained from the data source to be authenticated.
9. The apparatus according to claim 6 or 8, wherein, The step of executing the target application and obtaining the authentication information of the data source to be authenticated further includes: If the target application is not in Ha format, create user group information UGI based on the location of hdfsPricipal and keytab contained in the target application; Based on the UGI, connect to the data source to be authenticated; The authentication information is obtained from the data source to be authenticated.
10. The apparatus according to claim 6, further comprising: The timestamp module is used to annotate the cached authentication information with timestamp information; The expiration determination module is used to determine whether the authentication information has expired based on the timestamp information and the preset validity period. The information deletion module is used to delete the authentication information when the expiration determination module determines that the authentication information has expired.
11. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-5.
12. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-5.
13. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-5.