Data sharing implementation method of asymmetric encryption in Hadoop multi-tenant environment

Abstract

The invention discloses a data sharing implementation method of asymmetric encryption in a Hadoop multi-tenant environment. The method comprises the following steps: in a cluster with tenant-level symmetric encryption isolation, generating an RSA key pair for each tenant, storing a public key in a plaintext, and performing secondary encryption protection on a private key by a tenant master key; when a source tenant shares, the Ranger KMS internally decrypts the to-be-shared EZ-Key or DEK, a public key of a target tenant is used for encryption to generate a digital envelope, a unique identifier is distributed, and the NameNode extension attribute of a shared object is written; when a target tenant accesses the envelope, after the Ranger KMS verifies the effectiveness of the envelope, the private key of the envelope is used for decryption to obtain a symmetric key plaintext, and transparent decryption is realized; revocation only needs to mark envelope invalidation and clear extended attributes, and re-encryption is not needed. According to the method, the problems that symmetric keys cannot be revoked and plaintexts are leaked in cross-tenant sharing are solved, and the method has the advantages of key security, lossless performance, transparent access and full-link auditing.

Description

Technical Field

[0001] This application belongs to the field of multi-tenant data sharing technology, and specifically relates to a method, apparatus, computer-readable storage medium and electronic device for implementing data sharing in a Hadoop multi-tenant environment using asymmetric encryption. Background Technology

[0002] The Hadoop Encryption Zone (EZ), as the core mechanism of HDFS file-level transparent encryption, achieves zero-intrusion protection by strongly binding directories to symmetric keys, enabling automatic encryption during data writing and automatic decryption during data reading. Its technical architecture possesses the following core characteristics: First, the strong association between the key and the directory ensures manageability of the encryption scope; second, the NameNode only stores the Encrypted Data Encryption Key (EDEK), reducing the risk of metadata leakage; third, encryption and decryption operations are performed locally on the client side, reducing server-side computational pressure; fourth, it supports integration with Ranger / KMS for fine-grained access control; and fifth, it is compatible with multiple encryption algorithms such as AES-256 and AES-128 to meet diverse security needs.

[0003] However, in actual deployments of user-exclusive encrypted zones, the above mechanism has gradually revealed the following key shortcomings: 1. Low efficiency of sharing and collaboration: Cross-tenant data sharing requires manual export / import of keys, which is cumbersome and key transmission is prone to leakage risks; when migrating data across encrypted zones, it is necessary to decrypt and then re-encrypt, and the plaintext exposure window is significant during the process, making it impossible to achieve secure migration through conventional HDFS commands.

[0004] 2. High resource and operation and maintenance costs: The number of keys increases linearly with the number of tenants. KMS needs to perform key generation, rotation, backup and revocation operations one by one, which is difficult to manage in batches. Duplicate data in each encryption zone is encrypted independently, resulting in increased storage space. Byte-by-byte re-encryption during migration further aggravates CPU resource consumption.

[0005] 3. High complexity of cleanup and management: When a tenant goes offline, data needs to be cleaned up manually and keys need to be revoked. The operation is easy to be missed, resulting in "ghost encryption zones" and wasting storage resources.

[0006] 4. Vulnerabilities exist in identity control: exported files are not bound to the target tenant's identity information and may be arbitrarily forwarded; the existing framework only verifies export permissions and does not restrict the final user, causing the principle of least privilege to fail.

[0007] To address the aforementioned issues, existing technologies have built a tenant-level symmetric encryption isolation mechanism on top of the native Ranger-KMS: This is achieved by inserting an independent ciphertext row for each tenant in the ranger_masterkey table, adding a tenant_id field to the ranger_keystore to implement physical key isolation, and enforcing verification in the key provider, encryption zone manager, and Ranger plugin based on thread-level TenantContext. <tenant>Add a "tenant prefix" whitelist check to the kmsadmin.sh command (rejecting non-tenant prefixes). <tenantid> _ <suffix>(Formatting operations). This mechanism effectively blocks illegal cross-tenant key acquisition paths, but it still fails to solve the core challenge of "standardization, revocability, and auditability of cross-tenant key delivery in legitimate scenarios"—specifically, the lack of technical revocation capabilities for symmetric keys, the persistent risk of plaintext data forwarding, and the difficulty in meeting compliant cross-tenant data sharing requirements. Summary of the Invention

[0008] To address the aforementioned issues, this application proposes a novel asymmetric encryption method for data sharing in a Hadoop multi-tenant environment. Building upon a highly available multi-tenant cluster with existing tenant isolation capabilities, this method introduces asymmetric digital envelope technology to specifically fill the technical gap in "controlled sharing after isolation." It focuses on resolving the pain points of cross-tenant sharing, such as the inability to technically revoke symmetric keys, the ease of plaintext leakage, and the lack of auditability. Ultimately, it achieves compliant cross-tenant data sharing capabilities with "one-time sharing, one-click retrieval," rather than redundantly building upon existing isolation functions.

[0009] This invention proposes a multi-tenant data sharing method based on asymmetric digital envelopes. Building upon "tenant-level symmetric encryption isolation," it introduces asymmetric (RSA / ECC) digital envelopes to achieve cross-tenant data sharing, secure key transfer, and revocable control. A hybrid encryption architecture is adopted, with symmetric keys still used for the storage layer and asymmetric keys used for the sharing process, balancing performance and compliance. The entire process involves zero plaintext data storage, and envelopes can automatically expire or be manually revoked, achieving controlled sharing that is "authorizable, retrievable, and auditable." All shared areas remain standard HDFS encrypted areas, reusing the existing transparent encryption mechanism; Only two lightweight checks are added on the NameNode plugin side (share.envelope= <envelopeid>xAttr) enables tenant binding through Ranger strategy + xAttr, achieving zero intrusion into the HDFS backbone.

[0010] To achieve the above objectives, the present invention employs the following technical strategies: 1. Tenant-level RSA key pair hosting The public key is stored in plaintext, and the private key is encrypted twice with the tenant's symmetric master key before being stored in the database, forming a two-layer key protection system (the private key must be encrypted again with the tenant's symmetric master key before it can be stored in the database to prevent the private key from being exposed due to database leakage; this encryption operation must be completed within KMS and is prohibited from being processed or assembled in plaintext in any external link); even if the entire database is leaked, the private key cannot be recovered, ensuring key security.

[0011] 2. Digital Envelope Generation and Parsing The source tenant encrypts the symmetric key using the target public key; the decryption process of the target tenant's private key is completed internally by KMS and is not exposed to the application layer.

[0012] 3. Envelope lifecycle management The table records include status (ACTIVE / REVOKED) and expiration time, and support batch revocation and automatic cleanup; envelope revocation means technical failure, without the need for the target tenant's cooperation. By deleting or marking envelope records and using the synchronous cleanup Ranger strategy, the target tenant cannot decrypt the data, forming a "one-click recovery" hard isolation mechanism.

[0013] 4. Zero plaintext write to disk and memory clearing The symmetric key plaintext only exists briefly in the JVM heap and is immediately erased by Unsafe.setMemory() after use. The finally block ensures that it also applies to abnormal paths.

[0014] Specifically, this application provides the following technical solutions: The first aspect of this application provides a method for implementing asymmetric encryption in a Hadoop multi-tenant environment for data sharing, the method comprising: S1. In a multi-tenant Hadoop cluster that already has tenant-level symmetric encryption isolation capabilities, generate and host an asymmetric key pair for each tenant. The public key is stored in plaintext, and the private key is stored after being encrypted and protected by the tenant's own symmetric master key, forming a two-layer key protection system. S2. In response to the cross-tenant sharing request initiated by the source tenant, decrypt the symmetric key to be shared by the source tenant within Ranger KMS to obtain its plaintext; the symmetric key to be shared is either the encryption zone key EZ-Key or the file-level data encryption key DEK. S3. Encrypt the plaintext of the symmetric key to be shared using the public key of the target tenant, generate the ciphertext of the digital envelope, and assign it a unique envelope identifier; S4. Bind the envelope identifier to the shared object, where the shared object is an encrypted directory or a single file, and the binding method is to write the envelope identifier into the metadata extended attribute of the shared object; S5. When the target tenant accesses the shared object, the validity of the corresponding digital envelope is verified. After successful verification, Ranger KMS uses the target tenant's private key to decrypt the digital envelope and recover the plaintext of the symmetric key to be shared, thereby completing the transparent decryption access of the data.

[0015] Furthermore, the method of this application also includes: A digital envelope revocation mechanism is provided, which marks the digital envelope status as invalid and clears the extended attributes, so that the target tenant immediately loses the decryption ability, thereby realizing instant permission revocation without data re-encryption; The digital envelope revocation mechanism is triggered by the source tenant administrator, which updates the envelope record status to REVOKED and asynchronously notifies the NameNode to clear the extended attribute. Ranger KMS subsequently rejects all decryption requests based on the envelope identifier, achieving technical-level hard isolation of the target tenant's decryption capabilities.

[0016] Furthermore, in the method of this application, the escrow of the asymmetric key pair further includes: The Ranger KMS generates RSA-3072 or ECC-strength asymmetric key pairs for tenants. The public key is stored in plaintext in PEM format in the database. The private key is encrypted by the tenant's symmetric master key within Ranger KMS to form the private key ciphertext and is stored thereon. It is forbidden to process the private key in plaintext form in any external process.

[0017] Furthermore, in the method of this application, the method for obtaining the symmetric key to be shared includes: When the sharing granularity is at the encrypted zone level, Ranger KMS uses the source tenant's master key to decrypt the EZ-Key corresponding to the encrypted zone; When the sharing granularity is at the file level, Ranger KMS reads the EDEK encrypted with EZ-Key from the encrypted information of the file and decrypts it to obtain the DEK unique to the file.

[0018] Furthermore, the method of this application also includes: For encrypted zone-level sharing, the system uses the acquired EZ-Key to decrypt the file EDEK to obtain the DEK plaintext; for file-level sharing, the system directly obtains the DEK plaintext of the target file; finally, the client uses the DEK plaintext to decrypt the data block. The entire process is completed in memory and the key trace is immediately cleared, achieving completely transparent access to the shared party.

[0019] Furthermore, in the method of this application, after the digital envelope is generated, the envelope record is persisted to the envelope management table, which includes at least: envelope identifier, source tenant identifier, target tenant identifier, alias of the symmetric key to be shared, sharing granularity, sharing object path, envelope ciphertext, status field, creation time, revocation time and expiration management field; The status field of the envelope management table is an enumeration value, including: ACTIVE indicating that sharing is effective, REVOKED indicating that the administrator actively revokes it, and EXPIRED indicating that it automatically expires according to the preset expiration time; The expiration management field supports either a relative expiration period based on creation time or a fixed expiration time based on an absolute timestamp; The envelope management table establishes a combined unique key to constrain the source tenant, target tenant, alias of the symmetric key to be shared, sharing granularity, and shared object path to be unique five-tuples, preventing the repeated generation of digital envelopes with the same authorization relationship.

[0020] Furthermore, in the method of this application, when the target tenant accesses the shared object, verifying the validity of the corresponding digital envelope includes: Before decryption, Ranger KMS verifies whether the target tenant has access to the envelope identifier and queries the status and expiration date of the envelope record. Decryption is only performed if the status is ACTIVE and the envelope has not expired. Ranger KMS automatically records all digital envelope decryption operation logs. Each log entry includes at least the access type ENVELOPE_DECRYPT, source tenant, target tenant, envelope identifier, operation timestamp, and client IP, forming a full-chain audit chain and supporting one-click export of compliance reports.

[0021] Furthermore, in the method of this application, the plaintext of the symmetric key to be shared only exists momentarily in the KMS memory and the JVM heap memory of the target tenant client. After use, the Unsafe class immediately performs memory zeroing and erasure, ensuring that the finally code block also performs the cleanup operation in the abnormal path, so that no plaintext key is left in the disk, log and backup.

[0022] Furthermore, in the method of this application, the extended attribute is xattr in the NameNode, the attribute name is share.envelope, and the attribute value is the envelope identifier; The Ranger KMS retrieves the corresponding digital envelope using the envelope identifier and performs permission verification and decryption operations.

[0023] A second aspect of this application provides an apparatus for implementing asymmetric encryption in a Hadoop multi-tenant environment for data sharing, the apparatus comprising: The tenant key management module is used to generate and host asymmetric key pairs for each tenant in a multi-tenant Hadoop cluster that already has tenant-level symmetric encryption isolation capabilities. The public key is stored in plaintext, and the private key is stored after being encrypted and protected by the tenant's own symmetric master key, forming a two-layer key protection system. The KMS decryption module is used to respond to cross-tenant sharing requests initiated by the source tenant, and decrypt the source tenant's symmetric key to be shared within Ranger KMS to obtain its plaintext; the symmetric key to be shared is either the encryption zone key EZ-Key or the file-level data encryption key DEK. The envelope generation module is used to encrypt the plaintext of the symmetric key to be shared using the target tenant's public key, generate digital envelope ciphertext, and assign a unique envelope identifier to it; The envelope identifier binding module is used to bind an envelope identifier to a shared object, which is an encrypted directory or a single file. The binding method is to write the envelope identifier into the metadata extended attribute of the shared object. The access control module is used to verify the validity of the corresponding digital envelope when a target tenant accesses a shared object. After successful verification, Ranger KMS uses the target tenant's private key to decrypt the digital envelope and recover the plaintext of the symmetric key to be shared, thereby completing the transparent decryption access to the data.

[0024] The device implements the steps of the aforementioned asymmetric encryption data sharing method in a Hadoop multi-tenant environment during operation.

[0025] A third aspect of this application provides an electronic device, including: a memory and a processor; Memory: Used to store computer programs; Processor: Used to execute the computer program to implement the steps of the aforementioned method for data sharing in a Hadoop multi-tenant environment using asymmetric encryption.

[0026] A fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, it implements the steps of the aforementioned method for implementing data sharing in a Hadoop multi-tenant environment using asymmetric encryption.

[0027] In summary, this invention achieves secure, efficient, and compliant cross-tenant data sharing in a multi-tenant Hadoop environment through asymmetric digital envelope technology. Its technical advantages and beneficial effects are as follows: (1) Absolutely secure key lifecycle, eliminating the risk of plaintext leakage: The symmetric key (EZ-Key / DEK) exists only momentarily in the JVM heap memory of the source / target tenant. After use, it is immediately cleared by the Unsafe interface by filling it with zeros. There is no retention in disk, log and backup. It strictly meets the requirements of "key lifecycle ends and is destroyed" in FIPS-140 and the national cryptographic standard GM / T0054, blocking the possibility of key plaintext leakage from the physical level.

[0028] (2) Efficient and controllable cross-tenant sharing and revocation: Supports the "one-time sharing, one-click revocation" mechanism. The revocation operation only requires deleting the digital envelope record and clearing the Ranger policy. The target tenant immediately loses the decryption ability. The technical level realizes the true revocation of permissions without the need to re-encrypt the data, which greatly simplifies the cross-tenant collaboration process.

[0029] (3) Full-chain compliance audit to meet regulatory requirements: The audit chain records key information such as envelope ID, source tenant, target tenant, and timestamp throughout the process, and supports one-click export of standardized reports; it also has a dual mode of automatic expiration and active cancellation to ensure that the data sharing life cycle is traceable and controllable, and is suitable for strict compliance audit scenarios.

[0030] (4) Lossless performance and maintain native encryption efficiency: Asymmetric encryption is only used to protect the 256-bit symmetric key. Large data blocks are still processed by AES streaming decryption, maintaining the throughput on par with native HDFS transparent encryption and avoiding performance loss caused by encryption mechanism upgrades.

[0031] (5) Strong adaptability of cloud-native multi-tenancy: Multiple sets of cross-tenant shared operations can be executed concurrently within the same physical cluster, and each process is independent and without interference; old tenants do not need to be modified, and new tenants can quickly enable the sharing function through standard HDFS commands, reducing the complexity of deployment and operation and maintenance.

[0032] (6) Deep integration of transparent access and flexible management: target tenants can seamlessly access shared data through standard HDFS commands without being aware of the underlying envelope decryption logic; private keys are always protected by the tenant's key system to eliminate the risk of leakage; it supports flexible sharing granularity of single files or whole encrypted areas, and the revocation operation is made effective immediately by setting the status to REVOKED; based on the "master key number" design of the directory-envelope-tenant triplet, it builds full-link audit capabilities, supports second-level location and revocation, and further improves security management efficiency.

[0033] In summary, this invention offers systematic advantages in ensuring key security, improving sharing efficiency, adapting to cloud-native environments, and meeting compliance requirements, effectively addressing the core pain points of cross-tenant collaboration in traditional multi-tenant encrypted zones.

[0034] Other features and advantages of this application will be set forth in detail in the following description, or will become apparent through the implementation of the relevant technical solutions of this application. The objectives and other advantages of this application can be achieved through the technical features and means explicitly pointed out in the description, claims, and drawings, and will be obtained through the implementation of these technical contents. Attached Figure Description

[0035] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings involved in the description of the embodiments will be briefly introduced below. It should be noted that the accompanying drawings only show some embodiments of this application. For those skilled in the art, other related drawings can be derived from these drawings without creative effort.

[0036] Figure 1 This is a flowchart illustrating the overall implementation process of the asymmetric encryption data sharing method in a Hadoop multi-tenant environment.

[0037] Figure 2 This is a flowchart of the multi-tenant key process in an embodiment of this application.

[0038] Figure 3 This is a flowchart illustrating the encryption and decryption process for writing and reading encrypted files in this embodiment.

[0039] Figure 4 This is a flowchart of the encrypted file reading process in an embodiment of this application (Tenant B - the shared party).

[0040] Figure 5 This is a diagram illustrating the envelope state transition mechanism in the embodiments of this application.

[0041] Figure 6 This is a diagram showing the sequence of envelope status transitions in an embodiment of this application.

[0042] Figure 7 This is a structural diagram of the data sharing implementation device for asymmetric encryption in a Hadoop multi-tenant environment according to this application.

[0043] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0044] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be noted that the described embodiments are only some embodiments of this application, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the protection scope of this application.

[0045] In this document, the term "comprising" and any variations thereof (such as "including," "including," etc.) are open-ended expressions and should be understood as "including but not limited to," meaning that the listed content is not exhaustive and may include other content not explicitly mentioned. The term "based on" should be understood as "at least partially based on," meaning that the basis or condition referred to may not be the only factor and may involve other relevant factors. The term "one embodiment" should be understood as "at least one embodiment," meaning that the described embodiment is not the only possible implementation, and other similar embodiments may exist.

[0046] In this application, the terms "a" and "a plurality of" are used to modify related elements or features, and their expression is illustrative rather than restrictive. Unless otherwise expressly stated in the context, "a" should be understood as "at least one," and "a plurality of" should be understood as "at least two." Those skilled in the art should reasonably interpret these terms based on the semantic and logical relationships of the context to ensure that they cover the possibility of "one or more."

[0047] Figure 1 The diagram shows the overall implementation flow of the asymmetric encryption data sharing method provided in this application in a Hadoop multi-tenant environment, including the following steps: S1. In a multi-tenant Hadoop cluster that already has tenant-level symmetric encryption isolation capabilities, generate and host an asymmetric key pair for each tenant. The public key is stored in plaintext, and the private key is stored after being encrypted and protected by the tenant's own symmetric master key, forming a two-layer key protection system. S2. In response to the cross-tenant sharing request initiated by the source tenant, decrypt the symmetric key to be shared by the source tenant within Ranger KMS to obtain its plaintext; the symmetric key to be shared is either the encryption zone key EZ-Key or the file-level data encryption key DEK. S3. Encrypt the plaintext of the symmetric key to be shared using the public key of the target tenant, generate the ciphertext of the digital envelope, and assign it a unique envelope identifier; S4. Bind the envelope identifier to the shared object, where the shared object is an encrypted directory or a single file, and the binding method is to write the envelope identifier into the metadata extended attribute of the shared object; S5. When the target tenant accesses the shared object, the validity of the corresponding digital envelope is verified. After successful verification, Ranger KMS uses the target tenant's private key to decrypt the digital envelope and recover the plaintext of the symmetric key to be shared, thereby completing the transparent decryption access of the data.

[0048] To more clearly illustrate the technical solution of this application, the following will provide further explanation through specific scenario embodiments.

[0049] This invention is applicable to multi-tenant Hadoop clusters that already have tenant-level symmetric encrypted isolation, wherein: Each tenant has an independent symmetric master key; The key table has been updated to include a tenant_id column for storage. Strategy resources have been forcibly prefixed.

[0050] Based on this, the present invention achieves cross-tenant data sharing through asymmetric digital envelopes, solving the problem of irretrievable symmetric keys that are "shared once and held for life".

[0051] I. Key Management System

[0052] II. Comparison of Shared Granularity

[0053] III. Core Database Tables ranger_tenant_keypair: Stores the RSA key pair for each tenant; the public key is in plaintext, and the private key is encrypted using the tenant's master key.

[0054] The `ranger_share_envelope` table stores all shared envelopes; it is compatible with both single-file and entire encrypted zone granularities, and supports automatic expiration and active revocation.

[0055] IV. Operation Process and Internal Implementation (Administrator Triggered, Zero Modification to Client) tenantA (source tenant): data sharing party; tenantB (target tenant): data user.

[0056] share.envelope= <envelopeid>It is the "master key number" attached to a shared directory or file, and it only does the following three things: (1) The message indicates that HDFS is using a cross-tenant shared path; (2) Prompt Ranger KMS to decrypt the EZ-Key / DEK using the corresponding envelope; (3) Provide one-click traceability of audit triplets (directory-envelope-tenant).

[0057] Data sharing process (supports encryption zone level and file level): 1. The administrator calls the digital envelope interface. Encryption zone level: Decrypt the source tenant's encryption zone key (EZ-Key) to generate plaintext.

[0058] File level: Decrypt the source file data encryption key (DEK) to generate plaintext.

[0059] 2. Encrypt the plaintext key using the target tenant's public key. Forming "envelope ciphertext": an encrypted EZ-Key (encryption zone level) or DEK (file level).

[0060] 3. Save the envelope to the shared table And create share.envelope= on the shared object (encrypted zone directory or single file). <envelopeid>Mark and associate the envelope.

[0061] 4. The target tenant decrypts and uses the symmetric key. The target tenant uses their private key to unlock the envelope and obtain the plaintext EZ-Key or DEK, which is used for HDFS read / write encryption and decryption; the memory is cleared after use.

[0062] Figure 2 This invention demonstrates the asymmetric digital envelope implementation process for cross-tenant data sharing key layered protection and envelope encryption. Figure 3 This demonstrates the encryption and decryption process for writing and reading encrypted files, such as... Figure 2 and Figure 3 As shown: (1) Encrypted file writing process When tenant A writes a new file to the encrypted zone, the client first detects that the target path is within the encrypted zone, and then requests the Ranger KMS to generate a data encryption key. After generating a random DEK, the Ranger KMS encrypts it using tenant A's EZ-Key to obtain an EDEK, and returns the plaintext DEK and EDEK to the client. The client encrypts the file data in memory using the plaintext DEK, then submits the FileEncryptionInfo containing the EDEK to the NameNode for storage as file metadata, and finally writes the encrypted data block to the DataNode. After the write is completed, the plaintext DEK in memory is immediately cleared.

[0063] (2) Encrypted file reading process (owner) When tenant A reads its own encrypted file, the client obtains the EDEK from the NameNode's FileEncryptionInfo and then sends a decryption request to the Ranger KMS. The Ranger KMS uses tenant A's EZ-Key to decrypt the EDEK, recover the plaintext DEK, and return it to the client. After the client reads the encrypted data block from the DataNode, it decrypts the original data locally using the plaintext DEK. The plaintext DEK is cleared from memory immediately after the entire process is completed.

[0064] Figure 4 The encrypted file reading process (Tenant B - Shared Party) is demonstrated: When the shared party reads a shared encrypted file, the system automatically triggers a secure key-by-key decryption process. Tenant B accesses the file using standard HDFS commands. When the client obtains file metadata from the NameNode, it discovers the shared attribute and submits a digital envelope identifier to Ranger KMS. After verifying permissions, Ranger KMS uses Tenant B's RSA private key to decrypt the digital envelope, recovering Tenant A's encryption zone key or file data key. For encryption zone-level sharing, the system also needs to use the obtained EZ-Key to decrypt the file's EDEK to obtain the DEK; for file-level sharing, it directly obtains the plaintext DEK of the target file. Finally, the client uses the DEK to decrypt the data block. The entire process is completed in memory and the key traces are immediately cleared, achieving completely transparent access to the shared party.

[0065] Figure 5 and Figure 6 The document demonstrates the envelope state transition mechanism: Envelope state management employs a simple and efficient three-state transition model to ensure controllable lifecycles of shared permissions. The initial state is ACTIVE, indicating that the sharing relationship is effective and the target tenant can access the data normally. The state transition has two termination paths: the administrator manually triggers the REVOKED state using the revoke command, immediately severing access permissions; the system automatically switches to the EXPIRED state based on a preset absolute or relative expiration time, achieving automatic permission revocation. Both termination states retain audit logs, with the EXPIRED state supporting an automatic cleanup mechanism. The entire state transition is a unidirectional and irreversible process, and all state changes are recorded in a complete audit log, forming a closed-loop security management system.

[0066] Figure 7 The diagram shows a data sharing implementation apparatus for asymmetric encryption in a Hadoop multi-tenant environment proposed in this application. The apparatus includes: The tenant key management module is used to generate and host asymmetric key pairs for each tenant in a multi-tenant Hadoop cluster that already has tenant-level symmetric encryption isolation capabilities. The public key is stored in plaintext, and the private key is stored after being encrypted and protected by the tenant's own symmetric master key, forming a two-layer key protection system. The KMS decryption module is used to respond to cross-tenant sharing requests initiated by the source tenant, and decrypt the source tenant's symmetric key to be shared within Ranger KMS to obtain its plaintext; the symmetric key to be shared is either the encryption zone key EZ-Key or the file-level data encryption key DEK. The envelope generation module is used to encrypt the plaintext of the symmetric key to be shared using the target tenant's public key, generate digital envelope ciphertext, and assign a unique envelope identifier to it; The envelope identifier binding module is used to bind an envelope identifier to a shared object, which is an encrypted directory or a single file. The binding method is to write the envelope identifier into the metadata extended attribute of the shared object. The access control module is used to verify the validity of the corresponding digital envelope when a target tenant accesses a shared object. After successful verification, KMS uses the target tenant's private key to decrypt the digital envelope and recover the plaintext of the symmetric key to be shared, thereby completing the transparent decryption access to the data.

[0067] The above-mentioned device implements the steps of the data sharing method for asymmetric encryption in a Hadoop multi-tenant environment disclosed in this application when it is in operation.

[0068] The flowcharts and block diagrams in the accompanying drawings illustrate possible implementations of apparatus, methods, and computer program products according to various embodiments of this application, including architecture, functionality, and operation. In these figures, each block may represent a module, program segment, or portion of code containing one or more executable instructions for implementing a specified logical function. Each block in the block diagrams and / or flowcharts, and combinations thereof, can be implemented using either a dedicated hardware-based system or a combination of dedicated hardware and computer instructions to achieve the specified function or operation.

[0069] like Figure 8 As shown in the illustration, an embodiment of this application also discloses an electronic device, including: a processor 310, a communication interface 320, a memory 330 for storing a processor-executable computer program, and a communication bus 340. The processor 310, communication interface 320, and memory 330 communicate with each other via the communication bus 340. The processor 310 executes the executable computer program to implement the steps of the above-described asymmetric encryption data sharing method in a Hadoop multi-tenant environment.

[0070] It is understood that, in addition to memory and a processor, this electronic device may also include input devices (such as a keyboard), output devices (such as a display), and other communication modules. These input devices, output devices, and other communication modules all communicate with the processor through I / O interfaces (i.e., input / output interfaces).

[0071] The operations described in this application can be implemented by writing computer program code using one or more programming languages ​​or a combination thereof. The programming languages ​​include, but are not limited to, the following types: Object-oriented programming languages, such as Java, Smalltalk, C++, etc. Conventional procedural programming languages, such as "C" or similar programming languages.

[0072] The execution methods of program code include, but are not limited to: It runs entirely on the user's computer; Part of it executes on the user's computer, and part of it executes on a remote computer; Execute as a standalone software package; It is executed entirely on a remote computer or server.

[0073] In scenarios involving remote computers, the remote computer can connect to the user's computer via any type of network, including but not limited to local area networks (LANs) or wide area networks (WANs). Furthermore, the remote computer can also connect to external computers via an internet service provider, for example, by utilizing the internet.

[0074] Furthermore, this application also discloses a computer-readable storage medium, wherein when the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is able to perform the various steps of the data sharing implementation method of asymmetric encryption in a Hadoop multi-tenant environment disclosed in this application.

[0075] In the context of this application, a computer-readable storage medium refers to a tangible medium capable of storing computer program code and related data. Specific examples include, but are not limited to, the following: (1) Portable computer disk: such as floppy disks and other removable magnetic storage media.

[0076] (2) Hard disk: including mechanical hard disks and solid-state hard disks and other fixed storage devices.

[0077] (3) Random Access Memory (RAM): A volatile storage medium used for temporary storage of data and program code.

[0078] (4) Read-only memory (ROM): a non-volatile storage medium used to store fixed programs and data.

[0079] (5) Erasable programmable read-only memory (EPROM) or flash memory: non-volatile storage media that supports multiple erasures and reprogrammings.

[0080] (6) Fiber optic storage devices: storage media based on fiber optic technology.

[0081] (7) Portable compact disc read-only memory (CD-ROM): a read-only medium that stores data in the form of an optical disc.

[0082] (8) Optical storage devices: such as DVDs, Blu-ray discs and other storage media based on optical principles.

[0083] (9) Magnetic storage devices: such as magnetic tapes, disks and other storage media based on magnetic principles.

[0084] (10) Any suitable combination of the above: for example, combining multiple storage media to meet different storage needs.

[0085] These computer-readable storage media can be used to store the program code and related data described in this application to support program execution and persistent data storage.

[0086] Specifically, according to embodiments of this application, the processes described in the flowcharts can be implemented as computer software programs. For example, embodiments of this application relate to a computer program product comprising a computer program carried on a non-transitory computer-readable medium. This computer program contains program code for executing the data sharing implementation method for asymmetric encryption in a Hadoop multi-tenant environment disclosed in this application. When the computer program is executed by a processing device, it can achieve the functions defined in the embodiments of this application.

[0087] While the foregoing discussion contains several specific implementation details, these details should not be construed as limiting the scope of this application. The above description is merely a preferred embodiment of this application and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this application is not limited to technical solutions formed by specific combinations of the above-described technical features. Furthermore, this application should also cover other technical solutions formed by any combination of the above-described technical features or their equivalents without departing from the foregoing disclosed concept.

[0088] Those skilled in the art should also understand that modifications can be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features, without departing from the spirit and scope of the technical solutions of the embodiments of this application. These modifications or substitutions will not cause the essence of the corresponding technical solutions to deviate from the core spirit and scope of the technical solutions of the embodiments of this application.< / envelopeid> < / envelopeid> < / envelopeid> < / suffix> < / tenantid> < / tenant>

Claims

1. A method for implementing asymmetric encryption in a Hadoop multi-tenant environment for data sharing, characterized in that, The method includes: S1. In a multi-tenant Hadoop cluster that already has tenant-level symmetric encryption isolation capabilities, generate and host an asymmetric key pair for each tenant. The public key is stored in plaintext, and the private key is stored after being encrypted and protected by the tenant's own symmetric master key, forming a two-layer key protection system. S2. In response to the cross-tenant sharing request initiated by the source tenant, decrypt the symmetric key to be shared by the source tenant within Ranger KMS to obtain its plaintext; the symmetric key to be shared is either the encryption zone key EZ-Key or the file-level data encryption key DEK. S3. Encrypt the plaintext of the symmetric key to be shared using the public key of the target tenant, generate the ciphertext of the digital envelope, and assign it a unique envelope identifier; S4. Bind the envelope identifier to the shared object, where the shared object is an encrypted directory or a single file, and the binding method is to write the envelope identifier into the metadata extended attribute of the shared object; S5. When the target tenant accesses the shared object, the validity of the corresponding digital envelope is verified. After successful verification, Ranger KMS uses the target tenant's private key to decrypt the digital envelope and recover the plaintext of the symmetric key to be shared, thereby completing the transparent decryption access of the data.

2. The method according to claim 1, characterized in that, The method further includes: A digital envelope revocation mechanism is provided, which marks the digital envelope status as invalid and clears the extended attributes, so that the target tenant immediately loses the decryption ability, thereby realizing instant permission revocation without data re-encryption; The digital envelope revocation mechanism is triggered by the source tenant administrator, which updates the envelope record status to REVOKED and asynchronously notifies the NameNode to clear the extended attribute. Ranger KMS subsequently rejects all decryption requests based on the envelope identifier, achieving technical-level hard isolation of the target tenant's decryption capabilities.

3. The method according to claim 1, characterized in that, The escrow of the asymmetric key pair further includes: The Ranger KMS generates RSA-3072 or ECC-strength asymmetric key pairs for tenants. The public key is stored in plaintext in PEM format in the database. The private key is encrypted by the tenant's symmetric master key within Ranger KMS to form the private key ciphertext and is stored thereon. It is forbidden to process the private key in plaintext form in any external process.

4. The method according to claim 1, characterized in that, The methods for obtaining the symmetric key to be shared include: When the sharing granularity is at the encrypted zone level, Ranger KMS uses the source tenant's master key to decrypt the EZ-Key corresponding to the encrypted zone; When the sharing granularity is at the file level, Ranger KMS reads the EDEK encrypted with EZ-Key from the encrypted information of the file and decrypts it to obtain the DEK unique to the file.

5. The method according to claim 1, characterized in that, The method further includes: For encrypted zone-level sharing, the system uses the acquired EZ-Key to decrypt the file EDEK to obtain the DEK plaintext; for file-level sharing, the system directly obtains the DEK plaintext of the target file; finally, the client uses the DEK plaintext to decrypt the data block. The entire process is completed in memory and the key trace is immediately cleared, achieving completely transparent access to the shared party.

6. The method according to claim 1, characterized in that, After the digital envelope is generated, the envelope record is persisted to the envelope management table. The envelope management table includes at least: envelope identifier, source tenant identifier, target tenant identifier, alias of the symmetric key to be shared, sharing granularity, sharing object path, envelope ciphertext, status field, creation time, revocation time, and expiration management field. The status field of the envelope management table is an enumeration value, including: ACTIVE indicating that sharing is effective, REVOKED indicating that the administrator actively revokes it, and EXPIRED indicating that it automatically expires according to the preset expiration time; The expiration management field supports either a relative expiration period based on creation time or a fixed expiration time based on an absolute timestamp; The envelope management table establishes a combined unique key to constrain the source tenant, target tenant, alias of the symmetric key to be shared, sharing granularity, and shared object path to be unique five-tuples, preventing the repeated generation of digital envelopes with the same authorization relationship.

7. The method according to claim 6, characterized in that, When the target tenant accesses the shared object, the validity of the corresponding digital envelope is verified, including: Before decryption, Ranger KMS verifies whether the target tenant has access to the envelope identifier and queries the status and expiration date of the envelope record. Decryption is only performed if the status is ACTIVE and the envelope has not expired. Ranger KMS automatically records all digital envelope decryption operation logs. Each log entry includes at least the access type ENVELOPE_DECRYPT, source tenant, target tenant, envelope identifier, operation timestamp, and client IP, forming a full-chain audit chain and supporting one-click export of compliance reports.

8. The method according to claim 1, characterized in that, The plaintext of the symmetric key to be shared exists only momentarily in the KMS memory and the JVM heap memory of the target tenant client. After use, the Unsafe class immediately performs memory zeroing and erasure, ensuring that the finally block also performs the cleanup operation in abnormal paths, so that no plaintext key is left in the disk, logs and backups.

9. The method according to claim 1, characterized in that, The extended attribute is xattr in the NameNode, with the attribute name being share.envelope and the attribute value being the envelope identifier; The Ranger KMS retrieves the corresponding digital envelope using the envelope identifier and performs permission verification and decryption operations.

10. A data sharing implementation device for asymmetric encryption in a Hadoop multi-tenant environment, characterized in that, The device includes: The tenant key management module is used to generate and host asymmetric key pairs for each tenant in a multi-tenant Hadoop cluster that already has tenant-level symmetric encryption isolation capabilities. The public key is stored in plaintext, and the private key is stored after being encrypted and protected by the tenant's own symmetric master key, forming a two-layer key protection system. The KMS decryption module is used to respond to cross-tenant sharing requests initiated by the source tenant, and decrypt the source tenant's symmetric key to be shared within Ranger KMS to obtain its plaintext; the symmetric key to be shared is either the encryption zone key EZ-Key or the file-level data encryption key DEK. The envelope generation module is used to encrypt the plaintext of the symmetric key to be shared using the target tenant's public key, generate digital envelope ciphertext, and assign a unique envelope identifier to it; The envelope identifier binding module is used to bind an envelope identifier to a shared object, which is an encrypted directory or a single file. The binding method is to write the envelope identifier into the metadata extended attribute of the shared object. The access control module is used to verify the validity of the corresponding digital envelope when a target tenant accesses a shared object. After successful verification, KMS uses the target tenant's private key to decrypt the digital envelope and recover the plaintext of the symmetric key to be shared, thereby completing the transparent decryption access to the data.

Images