Distributed key management and encryption communication method oriented to big data environment

Abstract

The invention relates to the technical field of network security, in particular to a distributed key management and encryption communication method oriented to a big data environment. The method comprises the following steps: establishing a system architecture and configuring safety parameters; generating a master key and a session key through the random number; segmenting the master key into a plurality of segments and storing the segments to different nodes; encrypting the session key by using the public key of the receiver, distributing the session key through the secure channel, and decrypting the session key by the receiver to obtain the session key; constructing a Merkle tree through an AES-GCM function, and introducing a network adaptation factor into an encryption process by considering that security requirements and network conditions of data encryption in different environments are different; and regularly updating a secret key, implementing an access control strategy, and recording a log to perform system audit so as to guarantee long-term security. According to the invention, the efficient AES-GCM encryption algorithm is used, so that the speed and security of data encryption are improved. By implementing the access control strategy, only the authorized user can access and operate the key, and the security of the system is improved.

Description

Technical Field

[0001] The present invention relates to the field of network security technology, and more specifically, to a distributed key management and encrypted communication method for a big data environment. Background Art

[0002] The distributed key management and encrypted communication method for a big data environment aims to construct an efficient, secure, and scalable key management and communication protection mechanism by combining modern cryptography, distributed computing, and big data processing technologies. By using distributed key generation technology, the secure generation and distribution of keys are realized in a decentralized or partially centralized architecture to ensure the security and availability of keys throughout their life cycle. At the same time, to meet the massive data transmission requirements in the big data environment, this method adopts efficient encryption algorithms and optimized communication protocols to reduce the computational and communication overhead while ensuring data confidentiality and integrity. In addition, this method also introduces technologies such as load balancing, resource scheduling, and data sharding to address the challenges brought by high concurrency, large-scale storage, and complex network topologies in the big data environment. By deeply integrating key management and encrypted communication and combining advanced security mechanisms, this method can not only meet the dual requirements of high performance and high security in big data scenarios but also effectively resist various potential attacks, thus providing reliable security guarantees for emerging fields such as cloud computing, the Internet of Things, and blockchain.

[0003] In an existing distributed key management and encrypted communication method for a big data environment, system initialization and configuration are insufficient and lack unified management; network conditions vary in different environments; key generation is insecure, and communication between nodes is insecure; there are risks of single-point failure and data loss; and key distribution is insecure, and there may be cases of man-in-the-middle attacks; data encryption efficiency is low, environmental adaptability is poor, and data integrity protection is insufficient. In summary, a distributed key management and encrypted communication method for a big data environment is provided. Summary of the Invention

[0004] The purpose of the present invention is to provide a distributed key management and encrypted communication method for a big data environment to solve the problems raised in the above background art, including insufficient system initialization and configuration and lack of unified management; insecure key generation and insecure communication between nodes; risks of single-point failure and data loss; insecure key distribution, and there may be cases of man-in-the-middle attacks; low data encryption efficiency, poor environmental adaptability, and insufficient data integrity protection.

[0005] To achieve the above object, the present invention provides a distributed key management and encrypted communication method for a big data environment, including the following steps:

[0006] S1. Build the system architecture, configure security parameters, generate the initial seed value, and deploy the key management center;

[0007] S2. Generate the master key and session key through random numbers, and generate the public-private key pair for asymmetric encryption for each node;

[0008] S3. Split the master key into multiple segments and store them in different nodes, and create redundant backups to ensure fault tolerance;

[0009] S4. Encrypt the session key using the recipient's public key and distribute it through a secure channel. The recipient decrypts it to obtain the session key;

[0010] S5. Construct a Merkle tree through the AES-GCM function, and considering that the security requirements and network conditions of data encryption are different in different environments, introduce the network adaptation factor into the encryption process to optimize the data encryption and the process of storing it in chunks in distributed nodes;

[0011] S6. Regularly update the keys, implement access control policies, and record logs for system auditing to ensure long-term security.

[0012] As a further improvement of this technical solution, in S1, the specific steps of building the system architecture, configuring security parameters, generating the initial seed value, and deploying the key management center are as follows:

[0013] S1.1. Construct a set G of distributed network nodes containing N nodes and design a hierarchical architecture;

[0014] S1.2. Based on the hierarchical architecture, configure cryptographic parameters through elliptic curves, hash functions, and symmetric encryption algorithms;

[0015] S1.3. Collect the high-entropy seed s from the hardware entropy source and derive the root key K through HKDF root ;

[0016] S1.4. Based on the PBFT consensus algorithm, allocate proposal nodes and verification nodes in the hierarchical architecture and set the fault tolerance conditions.

[0017] As a further improvement of this technical solution, in S1.2, the specific process of configuring cryptographic parameters through elliptic curves, hash functions, and symmetric encryption algorithms based on the hierarchical architecture is as follows:

[0018] S1.21. Select an elliptic curve for secure encryption;

[0019] S1.22. Map a binary string of any length to a binary string with a fixed length of 256 bits through a hash function;

[0020] S1.23. The 256-bit input key K generates 15 rounds of keys through the Rijndael key scheduling algorithm, and the plaintext is encrypted and the authentication tag is calculated using the AES-256 encryption algorithm in combination with the counter mode and the GHASH polynomial hash function.

[0021] As a further improvement of this technical solution, in S1.21, the elliptic curve is specifically:

[0022] E: y 2 = x 3 + ax + b mod p;

[0023] In the formula, E represents the elliptic curve equation; y represents the coordinate variable of the ordinate on the curve; x represents the coordinate variable of the abscissa on the curve; a represents the linear coefficient; b represents the constant term coefficient; p represents the elliptic curve modulus; mod represents the modulo operation;

[0024] Considering that environmental risks can affect the confidentiality of secure encryption, an environmental risk variable is introduced to optimize the elliptic curve equation:

[0025] E′: y 2 = x 3 + ax + b mod p′;

[0026] In the formula, E′ represents the optimized elliptic curve equation; p′ represents the elliptic curve modulus after introducing the environmental risk variable;

[0027] As a further improvement of this technical solution, in S2, the specific steps for generating the master key and the session key through random numbers and generating the public-private key pair for asymmetric encryption for each node are:

[0028] S2.1. Use a cryptographically secure random number generator to generate a random number r;

[0029] S2.2. Derive the session key K through HKDF in combination with the timestamp t and the random number r session ;

[0030] S2.3. Generate an ECDSA public-private key pair Q based on the elliptic curve for each node N n i .

[0031] As a further improvement of this technical solution, in S3, the specific steps for splitting the master key into multiple segments and storing them in different nodes and creating redundant backups to ensure fault tolerance are:

[0032] S3.1. Construct a polynomial f(X) to split the master key and distribute it to multiple nodes according to the threshold;

[0033] Among them, considering that environmental risks will affect the construction of the polynomial sharding master key, an environmental risk variable is introduced to optimize the polynomial:

[0034]

[0035] In the formula, f(X)' represents the polynomial after introducing the environmental risk variable; K master represents the master key; represents the randomly generated coefficient;

[0036] Sharding generation:

[0037] S s =(X s , f(X s )');

[0038] In the formula, S s represents the original shard; X s represents different non-zero integers; f(X s )' represents the value obtained through polynomial calculation; s represents the number of original shards;

[0039] S3.2. Use Reed-Solomon coding to generate redundant shards to ensure fault tolerance and recoverability;

[0040] S3.3. Attach a MAC verification tag to each shard to prevent storage tampering.

[0041] As a further improvement of this technical solution, in S4, the specific steps for encrypting the session key with the recipient's public key and distributing it through a secure channel, and the recipient decrypting to obtain the session key are as follows:

[0042] S4.1. Generate a temporary elliptic curve key pair based on the recipient's public key Q temp to encrypt the session key K session ;

[0043] S4.2. Establish a forward secrecy channel through TLS1.3 to transmit the encrypted session key;

[0044] S4.3. The recipient uses the private key d recv to decrypt and obtain the session key K session , completing the key negotiation.

[0045] As a further improvement of this technical solution, in S5, the specific steps for encrypting the data with the session key and storing it in chunks on distributed nodes are as follows:

[0046] S5.1. Split the original data into blocks of a fixed size;

[0047] Among them, in view of the problem of differences in network conditions for data encryption in different environments, a network adjustment variable is introduced to dynamically adjust the block size and encryption parameters to cope with different network conditions. Therefore, a network adaptation factor α is introduced to optimize data encryption and store it in chunks on distributed nodes;

[0048] S5.2. Encrypt the data block using the AES-GCM function and generate a GHASH tag, and construct a Merkle tree to verify the integrity;

[0049] S5.3. Map the encrypted data block to a distributed node for storage through the consistent hashing algorithm;

[0050] Among them, the process of mapping the encrypted data block to a distributed node for storage through the consistent hashing algorithm is as follows:

[0051] For each encrypted data block C j Calculate the consistent hash value h′(C j )

[0052] Use the consistent hash value h′(C j ) to map the encrypted data block C j to a node N n in the distributed node set G = {N1, N2, …, N m}

[0053] As a further improvement of this technical solution, in the S5.2, the specific steps of encrypting the data block using the AES-GCM function and generating a GHASH tag, and constructing a Merkle tree to verify the integrity are as follows:

[0054] Use the session key and the initial counter to encrypt each data block using the AES-GCM function to generate a ciphertext and a GHASH authentication tag;

[0055] Use the GHASH tag T j of each encrypted data block as a leaf node to construct a Merkle tree;

[0056] Calculate the hash value of each internal node until the root node H root .

[0057] As a further improvement of this technical solution, in the S6, the specific steps of periodically updating the key, implementing an access control policy, and recording logs for system auditing to ensure long-term security are as follows:

[0058] S6.1. Periodically update the master key through HKDF and re-shard it to ensure the freshness of the key;

[0059] S6.2. Control the shard access permission based on the attribute encryption policy;

[0060] S6.3. Record the operation logs using a blockchain structure to ensure the irreversibility of the audit chain.

[0061] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0062] 1. In the distributed key management and encrypted communication method for the big data environment, by deploying a key management center, a mechanism for centralized key management and distribution is provided, simplifying the key management process. Generate an asymmetric encryption public-private key pair for each node to ensure that communication between nodes can be carried out securely and prevent man-in-the-middle attacks; create redundant backups to ensure that even if some nodes fail, the master key can still be restored through other nodes, enhancing the fault tolerance of the system.

[0063] 2. In the distributed key management and encrypted communication method for the big data environment, by encrypting the session key with the public key of the recipient, it is ensured that only the legitimate recipient can decrypt and use the key, improving the security of key distribution. By using the efficient AES-GCM encryption algorithm, the speed and security of data encryption are improved. By implementing access control policies, it is ensured that only authorized users can access and operate the keys, improving the security of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 It is the overall method flowchart of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0065] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0066] Please refer to Figure 1 As shown, this embodiment provides a distributed key management and encrypted communication method for the big data environment, including the following steps:

[0067] S1. Build a system architecture and configure security parameters, generate an initial seed value, and deploy a key management center;

[0068] In this example, the specific steps for building a system architecture and configuring security parameters, generating an initial seed value, and deploying a key management center are as follows:

[0069] S1.1. Construct a distributed network node set G containing N nodes and design a hierarchical architecture;

[0070] Among them, it is assumed that the system contains N nodes, forming a distributed network:

[0071] G = {N1, N2, …, N n};

[0072] Wherein, G represents the set of distributed network nodes; N n represents the Nth node; n represents the number of nodes;

[0073] S1.2. Configure cryptographic parameters based on the hierarchical architecture through elliptic curves, hash functions, and symmetric encryption algorithms;

[0074] In this example, the specific process of configuring cryptographic parameters based on the hierarchical architecture through elliptic curves, hash functions, and symmetric encryption algorithms is as follows:

[0075] S1.21. Select an elliptic curve for secure encryption;

[0076] In this example, the elliptic curve is specifically:

[0077] E: y 2 = x 3 + ax + b mod p;

[0078] Wherein, E represents the elliptic curve equation; y represents the coordinate variable of the ordinate on the curve; x represents the coordinate variable of the abscissa on the curve; a represents the linear coefficient; b represents the constant term coefficient; p represents the elliptic curve modulus; mod represents the modulo operation;

[0079] Considering that environmental risks will affect the confidentiality of secure encryption, an environmental risk variable is introduced to optimize the elliptic curve equation:

[0080] E′: y 2 = x 3 + ax + b mod p′;

[0081] Wherein, E′ represents the optimized elliptic curve equation; p′ represents the elliptic curve modulus after introducing the environmental risk variable;

[0082] Among them, p′ is:

[0083] p ′ = p0 + Δ p · R env ;

[0084] Wherein, p0 represents the initial modulus; Δ p represents the maximum growth of the modulus; R env represents the environmental risk variable;

[0085] Among them, R env is:

[0086] R env ∈ [0, 1];

[0087] Among them, R env = 0 indicates no security threat;

[0088] R env = 1 indicates the existence of a security threat;

[0089] Specifically, the dynamic evaluation method of R env is as follows:

[0090]

[0091] In the formula, N attack represents the number of network attacks detected per unit time; N total represents the maximum tolerable attack count threshold set by the system; F node represents the current node failure rate; F max represents the maximum node failure rate allowed by the system; λ represents the balance between attack and failure weights;

[0092] Specific application examples are as follows:

[0093] The initial elliptic curve parameters are:

[0094] The initial modulus p0 = 2 256 - 2 224 + 2 192 + 2 96 - 1 (NIST P-256 curve standard modulus);

[0095] The maximum growth increment of the modulus Δ p = 10 6 ;

[0096] The environmental risk variable R env = 0.7 (dynamically calculated by the security threat detection model);

[0097] The optimized modulus p ′ is calculated as:

[0098] p ′ = p0 + Δ p ·R env = (2 256 - 2 224 + 2 192 + 2 96 - 1)+10 6 ×0.7;

[0099] During actual calculation, p ′ is adjusted to a prime number to meet the security requirements of the elliptic curve.

[0100] For example:

[0101] p0 = 115792089210356248762697446949407573530086143415290314195533631308867097853951;

[0102] Then:

[0103] p' ≈ 115792089210356248762697446949407573530086143415290314195533631308867097853951 + 700000;

[0104] Finally, select the prime number closest to p through the prime number detection algorithm ′ .

[0105] S1.22. Map a binary string of any length to a binary string of a fixed length of 256 bits through a hash function;

[0106] Among them, the hash function is:

[0107] H: {0, 1} * → {0, 1} 256 ;

[0108] In the formula, H represents the hash function;

[0109] S1.23. Input the 256-bit key K to generate 15 rounds of keys through the Rijndael key scheduling algorithm, and use the AES-256 encryption algorithm combined with the counter mode and the GHASH polynomial hash function to encrypt the plaintext and calculate the authentication tag;

[0110] The specific process is as follows:

[0111] Input the 256-bit key K to generate 15 rounds of keys through the Rijndael key scheduling algorithm:

[0112] {RK0, RK1,..., RK 14};

[0113] The initial counter IV ∈ {0, 1} 96 , the counter block CTR i = IV || Nonce || i;

[0114] Encrypt the i-th plaintext block P i :

[0115]

[0116] In the formula, K represents the 256-bit symmetric key; IV represents the 96-bit initialization vector; C iDenote the i-th ciphertext block; P i Denote the i-th plaintext block; CTR i Denote the counter block; AES denotes block encryption; Denote the bitwise XOR operation; i denotes the index variable;

[0117] Authentication key:

[0118] H = AES-256 K (0 128 )

[0119] Calculate the authentication tag T:

[0120]

[0121] Wherein, A represents additional data; C represents ciphertext; GHASH represents polynomial hashing;

[0122] Among them, the polynomial hashing is:

[0123]

[0124] Wherein, X i Represents the value after the data block is grouped by 128 bits; m represents the number of samples; X represents the formal variable.

[0125] Specifically, by selecting an elliptic curve equation suitable for environmental risks (S1.21), defining a powerful hash function (S1.22), and configuring an efficient symmetric encryption algorithm (S1.23), the system can dynamically adjust its encryption parameters in a complex and changing big data environment to ensure the security of the key and the confidentiality and integrity of the data. The optimized elliptic curve enhances the anti-attack ability, the strong hash function ensures the uniqueness and immutability of the data, and the symmetric encryption algorithm provides a fast and secure data encryption and decryption process, thus providing a solid security foundation and technical guarantee for key management and encrypted communication in the entire distributed network.

[0126] S1.3. Collect a high-entropy seed s from the hardware entropy source and derive the root key through HKDF;

[0127] Among them, the process of collecting the high-entropy seed s from the hardware entropy source and deriving the root key through HKDF is:

[0128] Collect the seed s ∈ {0, 1} from the hardware entropy source 512 , satisfying min-entropy(s) ≥ 256;

[0129] Wherein, min-entropy(s) represents the minimum entropy of the seed value s;

[0130] Derive the root key K through s→HKDF(s, info = "KMC - Seed") root ;

[0131] Wherein, HKDF represents the HMAC - based key derivation function; info represents the context information parameter; K root represents the derived root key;

[0132] S1.4. Allocate the proposer nodes and verifier nodes in the hierarchical architecture based on the PBFT consensus algorithm, and set the fault - tolerance condition.

[0133] Specifically, the specific steps for allocating the proposer nodes and verifier nodes in the hierarchical architecture based on the PBFT consensus algorithm and setting the fault - tolerance condition are as follows:

[0134] Calculate the maximum number of fault - tolerant nodes f:

[0135]

[0136] Wherein, N represents the total number of nodes;

[0137] Check whether N>3f is satisfied;

[0138] When it is not satisfied, the system cannot run properly;

[0139] For the v - th view, calculate the proposer node P v :

[0140] P v =(v mod N)+1;

[0141] Wherein, v represents the view number;

[0142] The set of verifier nodes is:

[0143] V v ={N1,N2,...,N n}\{P v};

[0144] Wherein, V v represents the set of verifier nodes in the v - th view;

[0145] Calculate the consensus threshold:

[0146] T = 2f + 1;

[0147] Wherein, T represents the minimum number of nodes required to reach a consensus.

[0148] Specifically, by calculating the maximum number of fault-tolerant nodes and ensuring that the total number of nodes satisfies N > 3f, the system can tolerate up to f faulty or malicious nodes without affecting normal operation; by dynamically allocating proposal nodes and determining the set of verification nodes, the proposal and verification processes under each view are ensured to proceed in an orderly manner; setting the consensus threshold T = 2f + 1 guarantees the minimum number of nodes required to reach a consensus, thus ensuring the reliability and consistency of the system and providing a solid foundation and technical guarantee for distributed key management and encrypted communication.

[0149] Furthermore, through the design of a distributed network and a hierarchical architecture, combined with strong cryptographic parameter configuration, a high-quality seed generation and key derivation mechanism, and an efficient consensus algorithm, the secure storage and transmission of data and keys are achieved, and the fault tolerance and consistency of the system are guaranteed, providing a solid foundation for subsequent key management and encrypted communication.

[0150] S2. Generate the master key and session key through random numbers, and generate the public-private key pair for asymmetric encryption for each node;

[0151] In this example, the specific steps for generating the master key and session key through random numbers and generating the public-private key pair for asymmetric encryption for each node are as follows:

[0152] S2.1. Use a cryptographically secure random number generator to generate a random number r;

[0153] Among them, the specific process of generating the random number is:

[0154] Using the high-entropy seed generated in S1.3 as the input, combined with the CSPRNG to generate a random number:

[0155] r = CSPRNG(s root )

[0156] r ∈ {0, 1} 256 ;

[0157] In the formula, r represents the random number;

[0158] S2.2. Derive the session key K through HKDF in combination with the timestamp t and the random number r session ;

[0159] Among them, the process of deriving the session key is:

[0160] Taking K root as one of the inputs, combined with the timestamp t and the random number r, further derive the session key K through HKDF session :

[0161] K session = HKDF(K root , r||t||info);

[0162] Wherein, K session represents the session key;

[0163] S2.3. For each node N n generate an ECDSA public-private key pair Q based on the elliptic curve i ;

[0164] Among them, the process of generating the ECDSA public-private key pair based on the elliptic curve is as follows:

[0165] For each node N n ∈ J, select the optimized elliptic curve E′ and its base point J;

[0166] Generate a random private key:

[0167] d i ∈ [1, q - 1];

[0168] Wherein, q represents the order of the base point J;

[0169] Calculate the corresponding public key:

[0170] Q i = d i · J;

[0171] Wherein, Q i represents the corresponding public key.

[0172] Specifically, by using a high-quality random number generator to generate random numbers and combining the timestamp and HKDF to derive the session key, the unpredictability and security of the key are ensured; at the same time, an ECDSA public-private key pair based on the optimized elliptic curve is generated for each node, enhancing the security of communication between nodes, preventing man-in-the-middle attacks and other security threats, thus providing a solid security foundation and technical guarantee for key management and encrypted communication in the entire distributed network.

[0173] S3. Split the master key into multiple fragments and store them in different nodes, and create redundant backups to ensure fault tolerance;

[0174] In this example, the specific steps of splitting the master key into multiple fragments and storing them in different nodes and creating redundant backups to ensure fault tolerance are as follows:

[0175] S3.1. Construct a polynomial f(X) to fragment the master key and distribute it to multiple nodes according to the threshold;

[0176] Among them, the specific process of constructing a polynomial f(X) to fragment the master key and distribute it to multiple nodes according to the threshold is as follows:

[0177] Select the threshold t s , the threshold ts To determine how many shards are needed to reconstruct the master key;

[0178] Construct a polynomial:

[0179]

[0180] In the formula, f(X) represents the polynomial; K master represents the master key; represents randomly generated coefficients;

[0181] Similarly, considering that environmental risks can affect the construction of the polynomial shard master key, an environmental risk variable is introduced to optimize the polynomial:

[0182]

[0183] In the formula, f(X)' represents the polynomial after introducing the environmental risk variable;

[0184] Shard generation:

[0185] S s =(X s , f(X s ));

[0186] In the formula, S s represents the original shard; X s represents different non-zero integers; f(X s ) represents the value obtained through polynomial calculation; s represents the number of original shards;

[0187] S3.2. Use Reed-Solomon coding to generate redundant shards to ensure fault tolerance and recoverability;

[0188] Among them, use Reed-Solomon coding to generate redundant shards:

[0189] {C1, C2,..., C s+k}=ReedSolomonEncode({S1, S2,..., S s});

[0190] In the formula, ReedSolomonEncode() represents the encoding function; k represents the number of redundant shards; C s+k represents the original shards plus redundant shards;

[0191] S3.3. Attach a MAC verification tag to each shard to prevent storage tampering;

[0192] Among them, attaching a MAC verification tag to each shard is:

[0193] Generate MAC tag:

[0194] MAC i = HMAC(K auth , C i );

[0195] Wherein, MAC i represents the i-th MAC tag; HMAC represents a hash-based message authentication code function; K auth represents the key for generating MAC; C i represents the i-th master key shard;

[0196] Additional MAC tag:

[0197] D i = (C i , MAC i )

[0198] Wherein, D i represents a data packet containing the master key shard and the MAC tag.

[0199] Specifically, by constructing polynomial shards, distributing the master key to multiple nodes, and introducing environmental risk variables to optimize the polynomial, the security and adaptability of the key shards are ensured; using Reed-Solomon coding to generate redundant shards enhances the fault tolerance and data recovery capabilities of the system; attaching MAC verification tags to each shard further prevents storage tampering, thus providing a highly secure, reliable, and strongly fault-tolerant foundation and technical guarantee for key management and encrypted communication in the entire distributed network.

[0200] Specific application embodiments are as follows:

[0201] Master key shard and redundant backup:

[0202] Master key K master = 0x5A3E...D9F1 (256 bits), shard threshold t s = 3, total number of shards s = 5;

[0203] Construct an optimized polynomial:

[0204] The polynomial after introducing environmental risk variables is:

[0205] f(X)' = K master + a1X + a2X 2 mod p';

[0206] Randomly generate coefficients:

[0207] a1 = 0x8C2F…B4A7;

[0208] a2 = 0xE19D...6F3B;

[0209] Generate shards:

[0210] Select a non-zero integer X s = {1, 2, 3, 4, 5}, and calculate the shard values:

[0211]

[0212] Attach a MAC tag (using HMAC-SHA256) to each shard:

[0213] MAC i = HMAC(K auth , S i );

[0214] Redundant shard generation:

[0215] Use Reed-Solomon coding (parameters n = 7, k = 2 to generate 2 redundant shards):

[0216] {C6, C7} = ReedSolomonEncode(S1, S2, S3, S4, S5);

[0217] The redundant shards allow the system to recover the master key even if any 2 shards are lost at most.

[0218] S4. Encrypt the session key using the recipient's public key and distribute it through a secure channel. After decryption by the recipient, the session key is obtained;

[0219] In this example, the specific steps for encrypting the session key using the recipient's public key and distributing it through a secure channel, and obtaining the session key after decryption by the recipient are as follows:

[0220] S4.1. Based on the recipient's public key Q temp Generate a temporary elliptic curve key pair and encrypt the session key K session ;

[0221] Among them, the process of generating a temporary elliptic curve key pair based on the recipient's public key Q temp and encrypting the session key K session is as follows:

[0222] Generate a temporary elliptic curve key pair:

[0223] Select the same elliptic curve E' as the recipient and its base point J;

[0224] Generate a temporary private key d temp :

[0225] d temp ∈ [1, q - 1];

[0226] Calculate the corresponding temporary public key Q temp :

[0227] Q temp = d temp ·J;

[0228] Calculate the shared key:

[0229] K shared = d temp ·Q recv ;

[0230] In the formula, K shared represents the shared key; Q recv represents the public key of the recipient;

[0231] Use the shared key K shared to encrypt the session key K session :

[0232]

[0233] In the formula, Encrypt AES represents encryption using the AES algorithm; represents the encrypted session key;

[0234] Specifically, AES (Advanced Encryption Standard) is a symmetric encryption algorithm widely used to protect the security of electronic data. It ensures data confidentiality and integrity by using the same key for both encryption and decryption operations. AES supports key lengths of 128, 192, and 256 bits, corresponding to the AES-128, AES-192, and AES-256 variants respectively, with AES-256 providing the highest level of security. The algorithm processes data in blocks of 128 bits each and transforms the data through multiple rounds of complex substitution, permutation, and mixing operations to resist various attack methods.

[0235] Send the temporary public key Q temp and the encrypted session key to the recipient;

[0236] S4.2. Establish a forward secrecy channel through TLS1.3 to transmit the encrypted session key;

[0237] S4.3. The recipient uses the private key d recv to decrypt and obtain the session key K session , completing the key negotiation;

[0238] Among them, the recipient uses the private key d recvDecrypt to obtain the session key K session , the process of completing the key negotiation is as follows:

[0239] The receiver receives the ephemeral public key Q temp and the encrypted session key

[0240] Calculate the shared key:

[0241] K shared = d recv ·Q temp ;

[0242] In the formula, K shared represents the shared key; d recv represents the private key of the receiver;

[0243] Use the shared key K shared to decrypt the encrypted session key :

[0244]

[0245] In the formula, Decrypt AES represents decryption using the AES algorithm.

[0246] Specifically, by generating an ephemeral elliptic curve key pair and encrypting the session key, the security and unpredictability in the key distribution process are ensured; using TLS1.3 to establish a forward secrecy channel enhances the security and forward secrecy of the transmission; the receiver uses the private key to decrypt and obtain the session key, ensuring the security and reliability of the key negotiation process, thereby providing a highly secure, reliable and strongly forward-secret basis and technical guarantee for key management and encrypted communication in the entire distributed network.

[0247] Specific application embodiments are as follows:

[0248] Session key encryption and secure transmission:

[0249] Generate an ephemeral key pair:

[0250] Receiver's public key Q recv = d recv ·J (based on the optimized elliptic curve E′);

[0251] The sender generates an ephemeral private key d temp = 0x3D9A...F2E1, and calculates the ephemeral public key:

[0252] Q temp = d temp ·J;

[0253] Calculate the shared key:

[0254] K shared = d temp ·Q recv = d temp ·d recv ·J;

[0255] Encrypted session key:

[0256] Encrypt K using AES-256-GCM session = 0xA7B2...4C9D:

[0257]

[0258] Output ciphertext and authentication tag T;

[0259] Secure transmission:

[0260] Send through a TLS1.3 forward secrecy channel

[0261] Receiver decryption:

[0262] The receiver calculates the shared key K shared = d recv ·Q temp , and decrypts to obtain:

[0263]

[0264] Data verification:

[0265] The elliptic curve parameters need to meet the NIST or SECG standards to ensure resistance to quantum attacks;

[0266] Fragment recovery: Any 3 fragments can be recovered to f(X) through Lagrange interpolation ′ , and then obtain K master ;

[0267] Redundant fragments: The Reed-Solomon coding parameters need to meet n ≥ t s + k to ensure fault tolerance;

[0268] Encryption strength: AES-256 and ECDH key exchange comply with the NIST SP 800-56A standard.

[0269] S5. Construct a Merkle tree through the AES-GCM function, and considering that the security requirements and network conditions of data encryption vary in different environments, introduce a network adaptation factor into the encryption process to optimize data encryption and the process of storing it in chunks on distributed nodes;

[0270] In this example, the specific steps of encrypting data with a session key and storing it in chunks on distributed nodes are as follows:

[0271] S5.1. Split the original data into chunks of a fixed size;

[0272] Among them, the process of splitting the original data into chunks of a fixed size is as follows:

[0273] Split the original data into multiple chunks of a fixed size;

[0274] The number of data chunks is:

[0275]

[0276] In the formula, w represents the number of data chunks; |A| represents the length of the original data; Z represents the chunk size;

[0277] The size of each data chunk is:

[0278]

[0279] In the formula, B j represents the j-th data chunk; j represents the chunk number index variable;

[0280] Regarding the problem of differences in network conditions for data encryption in different environments, a network adjustment variable is introduced to dynamically adjust the chunk size and encryption parameters to cope with different network conditions. Therefore, a network adaptation factor α is introduced to optimize data encryption and store it in chunks on distributed nodes;

[0281] Then, the number of data chunks after introducing the network adaptation factor is:

[0282]

[0283] In the formula, w′ represents the number of data chunks after introducing the network adaptation factor; Z′ represents the chunk size after introducing the network adaptation factor;

[0284] Among them, Z′ is:

[0285] Z′ = Z0 + Δ Z ·α;

[0286] In the formula, Z0 represents the initial chunk size; Δ Z represents the maximum increase in chunk size; α represents the network adaptation factor;

[0287] Specifically, α is:

[0288]

[0289] In the formula, L net represents the current network latency; L maxRepresents the maximum network latency threshold allowed by the system;

[0290] Then the size of each data block after introducing the network adaptation factor is:

[0291]

[0292] In the formula, B j ′ represents the j-th data block after introducing the network adaptation factor;

[0293] S5.2. Encrypt the data block using the AES-GCM function and generate the GHASH tag, and construct a Merkle tree to verify the integrity;

[0294] In this example, the specific steps of encrypting the data block using the AES-GCM function and generating the GHASH tag and constructing a Merkle tree to verify the integrity are as follows:

[0295] Use the session key and the initial counter to encrypt each data block using the AES-GCM function to generate the ciphertext and the GHASH authentication tag:

[0296] (C j , T j ) = AES-GCM(K session , IV, B j ′);

[0297] In the formula, C j represents the ciphertext; T j represents the GHASH authentication tag;

[0298] Specifically, AES-GCM (Galois / Counter Mode) is an encryption mode that combines symmetric encryption and authentication, used to ensure the confidentiality, integrity, and authenticity of data. It is based on the AES (Advanced Encryption Standard) algorithm, encrypts data through the counter mode (CTR mode), and generates an authentication tag (GHASH) through multiplication calculations in the Galois field to verify the integrity of the data. Specifically, AES-GCM uses a symmetric key and an initialization vector (IV) to encrypt the plaintext data, generating the ciphertext and an additional authentication tag. The receiver can use the same key and IV to decrypt the ciphertext and confirm that the data has not been tampered with by verifying the authentication tag. Due to its high efficiency and security, AES-GCM is widely used in communication systems and data storage applications that require high security.

[0299] Use the GHASH tag T j of each encrypted data block as a leaf node to construct a Merkle tree;

[0300] Calculate the hash value of each internal node until the root node Hroot :

[0301]

[0302] In the formula, H k represents the hash value of an internal node in the Merkle tree; represents the hash value of the left child node; represents the hash value of the right child node.

[0303] Specifically, by using the AES-GCM function to encrypt each data block and generate a GHASH authentication tag, the confidentiality and integrity of the data are ensured; constructing a Merkle tree with each GHASH tag as a leaf node and calculating the hash values of internal nodes up to the root node provides an efficient and reliable integrity verification mechanism to prevent data tampering and forgery, thus providing a highly secure, reliable, and strongly consistent foundation and technical guarantee for data encryption and integrity verification in the entire distributed network.

[0304] S5.3. Map the encrypted data blocks to distributed nodes for storage through the consistent hashing algorithm;

[0305] Among them, the process of mapping the encrypted data blocks to distributed nodes through the consistent hashing algorithm is as follows:

[0306] For each encrypted data block C j calculate the consistent hash value h′(C j ):

[0307] h ′ (C j ) = ConsistentHash(C j );

[0308] In the formula, ConsistentHash represents the consistent hashing function.

[0309] Specifically, the consistent hashing function (Consistent Hashing) is a distributed hashing technology designed to solve the problem of data redistribution when nodes are added or removed in a distributed system. It maps data and nodes to a virtual circular space in a special way, thereby minimizing the impact of node changes on the entire system.

[0310] Use the consistent hash value n ′ (C j ) to map the encrypted data block C j to a node N in the distributed node set G = {N1, N2, …, N n}: m :

[0311] Nm = MapToNode(h ′ (C j ), G);

[0312] In the formula, MapToNode represents a function that maps a data block to a node according to the consistent hash value; N m represents the m-th distributed node; m represents the number of nodes included in n.

[0313] Specifically, MapToNode is a function used to map data blocks to specific nodes in a distributed system. It is usually used in combination with the consistent hashing algorithm to ensure the balanced distribution and efficient access of data in a distributed storage system.

[0314] Furthermore, by dynamically adjusting the data block size and encryption parameters to adapt to different security requirements and network conditions, the flexibility and security of data encryption are ensured; the AES-GCM function is used to encrypt each data block and generate a GHASH authentication tag, and the Merkle tree structure is combined for integrity verification to prevent data tampering; the encrypted data blocks are efficiently and evenly distributed to distributed nodes through the consistent hashing algorithm, ensuring the reliability and scalability of data storage, thus providing a highly secure, reliable and strongly consistent foundation and technical guarantee for data encryption, storage and integrity verification in the entire distributed network.

[0315] S6. Regularly update the key, implement the access control policy, and record logs for system auditing to ensure long-term security.

[0316] In this example, the specific steps for regularly updating the key, implementing the access control policy, and recording logs for system auditing to ensure long-term security are as follows:

[0317] S6.1 Periodically update the master key through HKDF and re-shard to ensure key freshness;

[0318] Specifically, HKDF (HMAC-based Key Derivation Function) is a standard method for generating encryption keys from initial key material such as passwords or random seeds. It uses the Hashing Message Authentication Code (HMAC) algorithm to ensure that the generated keys have sufficient randomness and security. The HKDF update process generally includes two main stages: Extract and Expand. In the Extract stage, HKDF combines the input high-entropy seed value (such as a random number or password) with a salt value to generate a fixed, high-quality pseudo-random key. In the Expand stage, this pseudo-random key is further processed to generate the final key material of the required length. By regularly updating the master key using HKDF, the freshness of the key can be ensured, thereby enhancing the security of the system and preventing long-term used keys from being cracked or leaked. This method is widely used in systems that require high security, such as encrypted communication and data storage.

[0319] Furthermore, to ensure the security of the system, the master key needs to be updated regularly to prevent long-term used keys from being cracked or leaked. First, set a fixed update period (for example, every 30 days). At the end of each update period, collect a new high-entropy seed value from the hardware entropy source. This seed value should have sufficient randomness and unpredictability to increase the security of the key. Next, use HKDF (HMAC-based Key Derivation Function) to combine the current master key and the newly generated seed value to generate a new master key. The generated new master key will replace the old master key, and to further enhance security, the new master key can be split into multiple segments, which will be stored on different nodes. This sharding technique not only increases the fault tolerance of the system but also reduces the risk of single points of failure.

[0320] S6.2. Control the access rights of shards based on the attribute encryption policy;

[0321] Specifically, to effectively manage the access rights to key shards, an Attribute-Based Encryption (ABE) policy is adopted. First, define a set of attributes for each node in the system. These attributes can be roles, permission levels, or other identifiers used to describe the characteristics or permissions of the node. Then, formulate an access policy according to security requirements, specifying which combinations of attributes can access specific data blocks or key shards. For example, some critical operations may require both the "administrator" and "auditor" roles to be executed. Use the attribute-based encryption algorithm to encrypt the key shards. Only when the node has the attribute combination that meets the access policy can it decrypt and access these key shards. This method can not only flexibly control access rights but also ensure that only authorized users can access sensitive information.

[0322] S6.3. Use blockchain structure to record operation logs to ensure the irreversibility of the audit chain.

[0323] Specifically, in order to ensure the transparency and traceability of system operations, blockchain technology is used to record all key operation logs. First, an initial block is created, which contains the basic information and initial status of the system. Each time a key operation (such as key update, access control, etc.) is performed, the relevant information is recorded in a new block. Each new block contains not only the log of this operation, but also the hash value of the previous block. In this way, all blocks form a chain structure, and any tampering with historical data will destroy the consistency of the chain, which can be easily detected. In order to verify the integrity of the blockchain, the system regularly checks whether the hash value of each block is correct. If the hash value of a block is found to be mismatched, it means that the block or its subsequent blocks may have been tampered with, and further investigation is required. This blockchain-based audit mechanism not only ensures the irreversibility of the operation log, but also improves the overall security of the system.

[0324] Furthermore, by regularly updating the master key and re-sharding, the freshness and security of the key are ensured; strict access control is implemented based on attribute encryption strategies to prevent unauthorized access; blockchain technology is used to record operation logs to ensure the immutability and transparency of the audit chain, thereby providing a highly secure, reliable and audit-capable foundation and technical guarantee for key management and encrypted communications in the entire distributed network.

[0325] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments, and the above embodiments and descriptions are only preferred examples of the present invention, and are not intended to limit the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, and these changes and improvements all fall within the scope of the present invention to be protected.

Claims

1. A distributed key management and encrypted communication method for a big data environment, characterized in that, It includes the following steps: S1. Build the system architecture and configure security parameters, generate the initial seed value and deploy the key management center; S2. Generate the master key and session keys through random numbers, and generate public-private key pairs for asymmetric encryption for each node; S3. Split the master key into multiple segments and store them in different nodes, and create redundant backups to ensure fault tolerance; S4. Encrypt the session key using the recipient's public key and distribute it through a secure channel. The recipient decrypts it to obtain the session key; S5. Construct a Merkle tree through the AES-GCM function, and considering that the security requirements and network conditions of data encryption vary in different environments, introduce a network adaptation factor into the encryption process to optimize the process of data encryption and block storage to distributed nodes; S6. Regularly update the keys, implement access control policies, and record logs for system auditing to ensure long-term security.

2. The distributed key management and encrypted communication method for the big data environment according to claim 1, characterized in that: In S1, the specific steps of building the system architecture and configuring security parameters, generating the initial seed value and deploying the key management center are as follows: S1.

1. Construct a set G of distributed network nodes containing N nodes and design a hierarchical architecture; S1.

2. Based on the hierarchical architecture, configure cryptographic parameters through elliptic curves, hash functions, and symmetric encryption algorithms; S1.

3. Collect the high-entropy seed s from the hardware entropy source and derive the root key through HKDF; S1.

4. Based on the PBFT consensus algorithm, allocate proposal nodes and verification nodes in the hierarchical architecture and set the fault tolerance conditions.

3. The distributed key management and encrypted communication method for the big data environment according to claim 2, wherein: In S1.2, the specific process of configuring cryptographic parameters through elliptic curves, hash functions, and symmetric encryption algorithms based on the hierarchical architecture is as follows: S1.

21. Select an elliptic curve for secure encryption; S1.

22. Map binary strings of any length to binary strings with a fixed length of 256 bits through a hash function; S1.

23. Input the 256-bit key K to generate 15-round keys through the Rijndael key scheduling algorithm, and use the AES-256 encryption algorithm combined with the counter mode and the GHASH polynomial hash function to encrypt the plaintext and calculate the authentication tag.

4. The distributed key management and encrypted communication method for a big data environment according to claim 3, characterized in that: In S1.21, the elliptic curve is specifically: E: y 2 = x 3 + ax + b mod p; In the formula, E represents the elliptic curve equation; y represents the coordinate variable of the ordinate on the curve; x represents the coordinate variable of the abscissa on the curve; a represents the linear coefficient; b represents the constant term coefficient; p represents the elliptic curve modulus; mod represents the modulo operation; Considering that environmental risks will affect the confidentiality of secure encryption, an environmental risk variable is introduced to optimize the elliptic curve equation: E′: y 2 = x 3 + ax + b mod p′; In the formula, E′ represents the optimized elliptic curve equation; p′ represents the elliptic curve modulus after introducing the environmental risk variable.

5. The distributed key management and encrypted communication method for the big data environment according to claim 4, characterized in that: In S2, the specific steps of generating the master key and session keys through random numbers, and generating public-private key pairs for asymmetric encryption for each node are as follows: S2.

1. Use a cryptographically secure random number generator to generate a random number r; S2.2 Derive the session key K by combining the timestamp t and the random number r through HKDF session ; S2.

3. Generate an ECDSA public-private key pair Q based on the elliptic curve for each node N n i .​ 6. The distributed key management and encrypted communication method for the big data environment according to claim 5, characterized in that: In S3, the specific steps of splitting the master key into multiple segments and storing them in different nodes, and creating redundant backups to ensure fault tolerance are as follows: S3.

1. Construct a polynomial f(X) for fragmenting the master key and distribute it to multiple nodes according to the threshold; Among them, considering that environmental risks will affect the construction of the polynomial sharding master key, an environmental risk variable is introduced to optimize the polynomial f(X): In the formula, f(X)′ represents the polynomial after introducing the environmental risk variable; K master represents the master key; represents the randomly generated coefficient; Sharding generation: S s = (X s , f(X s ))'; Where, S s represents the original shard; X s represents different non-zero integers; f(X s )′ represents the value obtained through polynomial calculation; s represents the number of original shards; S3.

2. Use Reed-Solomon coding to generate redundant shards to ensure fault tolerance and recoverability; S3.

3. Attach a MAC verification tag to each shard to prevent storage tampering.

7. The distributed key management and encrypted communication method for the big data environment according to claim 6, characterized in that: In the above-mentioned S4, the specific steps for encrypting the session key with the recipient's public key and distributing it through a secure channel, and the recipient obtaining the session key after decryption are: S4.

1. Generate a temporary elliptic curve key pair based on the recipient's public key Q temp and encrypt the session key K session ; S4.

2. Establish a forward secrecy channel through TLS1.3 to transmit the encrypted session key; S4.

3. The recipient uses the private key d recv to decrypt and obtain the session key K session , completing the key negotiation.

8. The distributed key management and encrypted communication method for the big data environment according to claim 7, characterized in that: In the above-mentioned S5, the specific steps for encrypting data with the session key and storing it in chunks on distributed nodes are: S5.

1. Split the original data into blocks of a fixed size; Among them, in view of the problem of network condition differences in data encryption under different environments, a network adjustment variable is introduced to dynamically adjust the block size and encryption parameters to cope with different network conditions. Therefore, a network adaptation factor ɑ is introduced to optimize data encryption and store it in chunks on distributed nodes; S5.

2. Use the AES-GCM function to encrypt the data block and generate a GHASH tag, and construct a Merkle tree to verify integrity; S5.

3. Map the encrypted data blocks to distributed nodes for storage through the consistent hashing algorithm; Among them, the process of mapping the encrypted data blocks to distributed nodes for storage through the consistent hashing algorithm is: For each encrypted data block C j compute the consistent hash value h′(C j ); Use the consistent hash value h′(C j ) to map the encrypted data block C j to a node N in the distributed node set G = {N1, N2, …, N n}. m .

9. The distributed key management and encrypted communication method for the big data environment according to claim 8, wherein: In the above-mentioned S5.2, the specific steps for using the AES-GCM function to encrypt the data block and generate a GHASH tag, and constructing a Merkle tree to verify integrity are: Use the session key and the initial counter to encrypt each data block with the AES-GCM function to generate ciphertext and a GHASH authentication tag; Take the GHASH tag T of each encrypted data block j As leaf nodes, construct a Merkle tree; Calculate the hash value of each internal node until the root node H root 。 10. The distributed key management and encrypted communication method for the big data environment according to claim 9, characterized in that: In the above-mentioned S6, the specific steps for periodically updating the key, implementing an access control policy, and recording logs for system auditing to ensure long-term security are: S6.

1. Periodically update the master key through HKDF and re-shard to ensure key freshness; S6.

2. Control the access permissions of shards based on the attribute encryption policy; S6.

3. Record the operation logs with a blockchain structure to ensure the irreversibility of the audit chain.

Images