A distributed key management and encrypted communication method for a big data environment

Abstract

The application relates to the technical field of network security, in particular to a distributed key management and encrypted communication method for a big data environment. The method comprises the following steps: building a system architecture and configuring security parameters; generating a master key and a session key through a random number; storing the master key into different nodes in the form of multiple segments; encrypting the session key by using a receiver public key and distributing the session key through a secure channel, and obtaining the session key after decryption by the receiver; constructing a Merkle tree through an AES-GCM function, and introducing a network adaptation factor into the encryption process by considering the different security requirements and network conditions of data encryption in different environments; regularly updating the key, implementing an access control strategy, and recording logs for system auditing to guarantee long-term security. The application improves the speed and security of data encryption by using an efficient AES-GCM encryption algorithm. The application ensures that only authorized users can access and operate the key by implementing the access control strategy, thereby improving the security of the system.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of network security, in particular to a distributed key management and encrypted communication method for big data environment. BACKGROUND

[0002] The distributed key management and encrypted communication method for big data environment aims to build an efficient, secure and scalable key management and communication protection mechanism by combining modern cryptography, distributed computing and big data processing technology. Using distributed key generation technology, the security of key generation and distribution is realized in a decentralized or partially centralized architecture, ensuring the security and availability of the key throughout its life cycle. At the same time, to adapt to the massive data transmission demand in the big data environment, this method uses efficient encryption algorithms and optimized communication protocols to reduce the computational and communication overhead while ensuring data confidentiality and integrity. In addition, this method also introduces load balancing, resource scheduling and data sharding technologies to cope with the challenges brought by high concurrency, large-scale storage and complex network topology in the big data environment. By deeply integrating key management and encrypted communication and combining advanced security mechanisms, this method not only meets the dual needs of high performance and high security in the big data scenario, but also effectively resists various potential attacks, thus providing reliable security protection for emerging fields such as cloud computing, Internet of Things and blockchain.

[0003] In an existing distributed key management and encrypted communication method for big data environment, system initialization and configuration are insufficient and lack unified management; network conditions differ in different environments; key generation is insecure, and inter-node communication is insecure; there is a single point of failure risk and data loss risk; and key distribution is insecure, which may have a man-in-the-middle attack; data encryption efficiency is low, environmental adaptability is poor, and data integrity protection is insufficient. In summary, a distributed key management and encrypted communication method for big data environment is provided. SUMMARY

[0004] The present application aims to provide a distributed key management and encrypted communication method for big data environment to solve the problems of insufficient system initialization and configuration and lack of unified management; insecure key generation, insecure inter-node communication; single point of failure risk and data loss risk; and insecure key distribution, which may have a man-in-the-middle attack; low data encryption efficiency, poor environmental adaptability and insufficient data integrity protection in the existing distributed key management and encrypted communication method for big data environment.

[0005] To achieve the above-mentioned purpose, the present application aims to provide a distributed key management and encrypted communication method for big data environment, comprising the following steps:

[0006] S1. Build the system architecture and configure security parameters, generate initial seed values ​​and deploy the key management center;

[0007] S2. Generate a master key and session key using random numbers, and generate asymmetric encryption public-private key pairs for each node;

[0008] S3. Divide the master key into multiple fragments and store them on different nodes, and create redundant backups to ensure fault tolerance;

[0009] S4. Encrypt the session key using the recipient's public key and distribute it through a secure channel. The recipient then decrypts the key to obtain the session key.

[0010] S5. Construct a Merkle tree using the AES-GCM function, and consider that the security requirements and network conditions of data encryption vary in different environments. Introduce a network adaptation factor into the encryption process to optimize the process of data encryption and block storage to distributed nodes.

[0011] S6. Regularly update keys, implement access control policies, and log system audits to ensure long-term security.

[0012] As a further improvement to this technical solution, the specific steps in S1 for building the system architecture, configuring security parameters, generating the initial seed value, and deploying the key management center are as follows:

[0013] S1.1 Construct a distributed network node set G containing N nodes and design a layered architecture;

[0014] S1.2 Based on a layered architecture, cryptographic parameters are configured using elliptic curves, hash functions, and symmetric encryption algorithms;

[0015] S1.3. Collect a high-entropy seed s from the hardware entropy source and derive the root key K using HKDF. root ;

[0016] S1.4. Based on the PBFT consensus algorithm, allocate proposal nodes and verification nodes in the layered architecture and set fault tolerance conditions.

[0017] As a further improvement to this technical solution, in S1.2, the specific process of configuring cryptographic parameters based on a layered architecture using elliptic curves, hash functions, and symmetric encryption algorithms is as follows:

[0018] S1.21. Select elliptic curves for security encryption;

[0019] S1.22. Map a binary string of arbitrary length to a binary string of fixed length of 256 bits using a hash function;

[0020] S1.23. Input a 256-bit key K and generate 15 rounds of keys using the Rijndael key scheduling algorithm. Then, use the AES-256 encryption algorithm combined with the counter mode and the GHASH polynomial hash function to encrypt the plaintext and calculate the authentication tag.

[0021] As a further improvement to this technical solution, in S1.21, the elliptic curve is specifically:

[0022] E:y 2 =x 3 +ax+bmodp;

[0023] In the formula, E represents the equation of the elliptic curve; y represents the coordinate variable of the ordinate on the curve; x represents the coordinate variable of the abscissa on the curve; a represents the linear coefficient; b represents the constant term coefficient; p represents the elliptic curve modulus; mod represents the modulo operation.

[0024] Considering that environmental risks can affect the confidentiality of secure encryption, an environmental risk variable is introduced to optimize the elliptic curve equation:

[0025] E′:y 2 =x 3 +ax+bmodp′;

[0026] In the formula, E′ represents the optimized elliptic curve equation; p′ represents the elliptic curve modulus after incorporating environmental risk variables;

[0027] As a further improvement to this technical solution, the specific steps in S2 for generating the master key and session key using random numbers, and for generating asymmetric encryption public-private key pairs for each node, are as follows:

[0028] S2.1. Generate a random number r using a cryptographically secure random number generator;

[0029] S2.2. Derive the session key K by combining the timestamp t and the random number r using HKDF. session ;

[0030] S2.3, for each node N n Generate ECDSA public-private key pair Q based on elliptic curves. i .

[0031] As a further improvement to this technical solution, in step S3, the specific steps for dividing the master key into multiple fragments and storing them on different nodes, and creating redundant backups to ensure fault tolerance are as follows:

[0032] S3.1 Construct a polynomial f(X) fragmentation master key and distribute it to multiple nodes according to a threshold.

[0033] In this regard, considering that environmental risks can affect the construction of the polynomial sharding master key, an environmental risk variable is introduced to optimize the polynomial:

[0034]

[0035] In the formula, f(X)′ represents the polynomial after introducing environmental risk variables; K master Indicates the master key; Indicates the coefficients generated randomly;

[0036] Fragment generation:

[0037] S s =(X s ,f(X s )′);

[0038] In the formula, S s Indicates the original fragment; X s f(X) represents distinct non-zero integers; s )′ represents the value obtained through polynomial calculation; s represents the number of original fragments;

[0039] S3.2 Use Reed-Solomon coding to generate redundant fragments to ensure fault tolerance and recoverability;

[0040] S3.3 Attach a MAC verification tag to each shard to prevent storage tampering.

[0041] As a further improvement to this technical solution, in step S4, the specific steps for encrypting the session key using the receiver's public key and distributing it through a secure channel, and for the receiver to decrypt and obtain the session key, are as follows:

[0042] S4.1, Based on the receiver's public key Q temp Generate temporary elliptic curve key pairs and encrypt session key K. session ;

[0043] S4.2. Establish a forward confidential channel via TLS 1.3 to transmit the encrypted session key;

[0044] S4.3, The receiver uses the private key d recv Decrypt to obtain session key K session Complete key negotiation.

[0045] As a further improvement to this technical solution, the specific steps in S5 of encrypting the data using the session key and storing it in blocks to the distributed nodes are as follows:

[0046] S5.1 Divide the original data into blocks of fixed size;

[0047] To address the issue of varying network conditions in different environments, a network adjustment variable is introduced to dynamically adjust the block size and encryption parameters to cope with different network conditions. Therefore, a network adaptation factor α is introduced to optimize data encryption and store it in blocks on distributed nodes.

[0048] S5.2. Use the AES-GCM function to encrypt the data block and generate a GHASH tag, then construct a Merkle tree to verify integrity;

[0049] S5.3. Map encrypted data blocks to distributed node storage using a consistent hashing algorithm;

[0050] The process of mapping encrypted data blocks to distributed node storage using the consistent hashing algorithm is as follows:

[0051] For each encrypted data block C j Calculate the consistent hash value h′(C) j );

[0052] Use consistent hash value h′(C) j ) Encrypt data block C j Mapped to a distributed node set G = {N1, N2, ..., N} n A node N in} m .

[0053] As a further improvement to this technical solution, in S5.2, the specific steps for encrypting data blocks using the AES-GCM function and generating GHASH tags, and constructing a Merkle tree to verify integrity are as follows:

[0054] Each data block is encrypted using the AES-GCM function with a session key and an initial counter, generating ciphertext and a GHASH authentication tag.

[0055] GHASH tag T for each encrypted data block j As leaf nodes, construct a Merkle tree;

[0056] Calculate the hash value of each internal node, up to the root node H. root .

[0057] As a further improvement to this technical solution, the specific steps in S6 for periodically updating keys, implementing access control policies, and recording logs for system auditing to ensure long-term security are as follows:

[0058] S6.1 Periodically update the master key and re-shard it via HKDF to ensure key freshness;

[0059] S6.2 Control fragment access permissions based on attribute encryption strategy;

[0060] S6.3 Use a blockchain structure to record operation logs to ensure the irreversibility of the audit chain.

[0061] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0062] 1. This distributed key management and encrypted communication method for big data environments provides a centralized key management and distribution mechanism by deploying a key management center, simplifying the key management process. It generates asymmetric encrypted public-private key pairs for each node, ensuring secure communication between nodes and preventing man-in-the-middle attacks. Redundant backups ensure that even if some nodes fail, the master key can still be recovered from other nodes, enhancing the system's fault tolerance.

[0063] 2. This distributed key management and encrypted communication method for big data environments improves key distribution security by encrypting the session key using the recipient's public key, ensuring that only authorized recipients can decrypt and use the key. The use of the efficient AES-GCM encryption algorithm enhances both the speed and security of data encryption. Furthermore, implementing access control policies ensures that only authorized users can access and manipulate the keys, thus improving system security. Attached Figure Description

[0064] Figure 1 This is a flowchart of the overall method of the present invention. Detailed Implementation

[0065] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0066] Please see Figure 1 As shown, this embodiment provides a distributed key management and encrypted communication method for big data environments, including the following steps:

[0067] S1. Build the system architecture and configure security parameters, generate initial seed values ​​and deploy the key management center;

[0068] In this example, the specific steps for building the system architecture, configuring security parameters, generating initial seed values, and deploying the key management center are as follows:

[0069] S1.1 Construct a distributed network node set G containing N nodes and design a layered architecture;

[0070] The system consists of N nodes forming a distributed network.

[0071] G = {N1, N2, ..., N} n};

[0072] In the formula, G represents the set of distributed network nodes; N n This represents the Nth node; n represents the number of nodes.

[0073] S1.2 Based on a layered architecture, cryptographic parameters are configured using elliptic curves, hash functions, and symmetric encryption algorithms;

[0074] In this example, the specific process of configuring cryptographic parameters based on a layered architecture using elliptic curves, hash functions, and symmetric encryption algorithms is as follows:

[0075] S1.21. Select elliptic curves for security encryption;

[0076] In this example, the elliptic curve is specifically:

[0077] E:y 2 =x 3 +ax+b mod p;

[0078] In the formula, E represents the equation of the elliptic curve; y represents the coordinate variable of the ordinate on the curve; x represents the coordinate variable of the abscissa on the curve; a represents the linear coefficient; b represents the constant term coefficient; p represents the elliptic curve modulus; mod represents the modulo operation.

[0079] Considering that environmental risks can affect the confidentiality of secure encryption, an environmental risk variable is introduced to optimize the elliptic curve equation:

[0080] E′:y 2 =x 3 +ax+bmodp′;

[0081] In the formula, E′ represents the optimized elliptic curve equation; p′ represents the elliptic curve modulus after incorporating environmental risk variables;

[0082] Where p′ is:

[0083] p ′ =p0+Δ p ·R env ;

[0084] In the formula, p0 represents the initial modulus; Δ p R represents the maximum increase in the modulus. env Indicates environmental risk variables;

[0085] Among them, R env for:

[0086] R env ∈[0,1];

[0087] Among them, R env =0 indicates no security threat;

[0088] R env =1 indicates that a security threat exists;

[0089] Specifically, R env The dynamic evaluation method is as follows:

[0090]

[0091] In the formula, N attack N represents the number of network attacks detected per unit of time. total This indicates the maximum tolerable number of attacks set by the system; F node Indicates the current node failure rate; F max The maximum allowed node failure rate of the system is represented by λ; λ represents the weighting of attacks and failures.

[0092] Specific application examples are as follows:

[0093] The initial elliptic curve parameters are:

[0094] Initial modulus p0 = 2 256 -2 224 +2 192 +2 96 -1 (NIST P-256 curve standard modulus);

[0095] Maximum modulus growth Δ p =10 6 ;

[0096] Environmental risk variable R env =0.7 (dynamically calculated by the security threat detection model);

[0097] Optimized modulus p ′ The calculation is as follows:

[0098] p ′ =p0+Δ p ·R env =(2 256 -2 224 +2 192 +2 96 -1)+10 6 ×0.7;

[0099] In actual calculations, p ′ It was adjusted to a prime number to meet the elliptic curve safety requirements.

[0100] For example:

[0101] p0=115792089210356248762697446949407573530086143415290314195533631308867097853951;

[0102] but:

[0103] p′≈115792089210356248762697446949407573530086143415290314195533631308867097853951+700000;

[0104] Finally, the closest prime number is selected as p using a prime number detection algorithm. ′ .

[0105] S1.22. Map a binary string of arbitrary length to a binary string of fixed length of 256 bits using a hash function;

[0106] The hash function is:

[0107] H:{0,1} * →{0,1} 256 ;

[0108] In the formula, H represents the hash function;

[0109] S1.23. Input a 256-bit key K and generate 15 rounds of keys using the Rijndael key scheduling algorithm. Then, use the AES-256 encryption algorithm combined with the counter mode and the GHASH polynomial hash function to encrypt the plaintext and calculate the authentication tag.

[0110] The specific process is as follows:

[0111] Input a 256-bit key K, and generate 15 rounds of keys using the Rijndael key scheduling algorithm:

[0112] {RK0,RK1,...,RK 14};

[0113] Initial counter IV∈{0,1} 96 Counter block CTR i =IV||Nonce||i;

[0114] Encrypt the i-th plaintext block P i :

[0115]

[0116] In the formula, K represents a 256-bit symmetric key; IV represents a 96-bit initialization vector; C iP represents the i-th ciphertext block; i Represents the i-th plaintext block; CTR i Indicates a counter block; AES indicates block encryption; This indicates a bitwise XOR operation; i represents the index variable;

[0117] Authentication key:

[0118] H = AES - 256 K (0 128 );

[0119] Calculate the authentication label T:

[0120]

[0121] In the formula, A represents additional data; C represents ciphertext; GHASH represents polynomial hash;

[0122] The polynomial hash is:

[0123]

[0124] In the formula, X i This represents the value of the data block after being grouped into 128-bit groups; m represents the number of samples; and X represents the formal variable.

[0125] Specifically, by selecting an elliptic curve equation suitable for the environmental risks (S1.21), defining a strong hash function (S1.22), and configuring an efficient symmetric encryption algorithm (S1.23), the system can dynamically adjust its encryption parameters in complex and ever-changing big data environments, ensuring key security and data confidentiality and integrity. The optimized elliptic curve enhances resistance to attacks, the strong hash function guarantees the uniqueness and immutability of data, and the symmetric encryption algorithm provides a fast and secure data encryption and decryption process, thus providing a solid security foundation and technical guarantee for key management and encrypted communication throughout the distributed network.

[0126] S1.3. Collect a high-entropy seed s from the hardware entropy source and derive the root key using HKDF;

[0127] The process of acquiring a high-entropy seed s from a hardware entropy source and deriving the root key from HKDF is as follows:

[0128] Seeds s∈{0,1} are collected from a hardware entropy source. 512 The condition is that min-entropy(s) ≥ 256.

[0129] In the formula, min-entropy(s) represents the minimum entropy of the seed value s;

[0130] The root key K is derived from s→HKDF(s,info="KMC-Seed") root ;

[0131] In the formula, HKDF represents the HMAC-based key derivation function; info represents the context information parameter; K root This represents the derived root key;

[0132] S1.4. Based on the PBFT consensus algorithm, allocate proposal nodes and verification nodes in the layered architecture and set fault tolerance conditions.

[0133] Specifically, the steps for allocating proposal nodes and verification nodes in a layered architecture based on the PBFT consensus algorithm and setting fault tolerance conditions are as follows:

[0134] Calculate the maximum number of fault-tolerant nodes f:

[0135]

[0136] In the formula, N represents the total number of nodes;

[0137] Check if N > 3f;

[0138] If these conditions are not met, the system will not function properly.

[0139] For the v-th view, compute the proposal node P. v :

[0140] P v = (v mod N) + 1;

[0141] In the formula, v represents the number of views;

[0142] The set of verification nodes is:

[0143] V v ={N1,N2,...,N n}\{P v};

[0144] In the formula, V v This represents the set of verification nodes in the v-th view;

[0145] Calculate the consensus threshold:

[0146] T = 2f + 1;

[0147] In the formula, T represents the minimum number of nodes required to reach a consensus.

[0148] Specifically, by calculating the maximum number of fault-tolerant nodes and ensuring that the total number of nodes satisfies N>3f, the system can tolerate up to f faulty or malicious nodes without affecting normal operation; by dynamically allocating proposal nodes and determining the set of verification nodes, the system ensures that the proposal and verification processes under each view proceed in an orderly manner; by setting a consensus threshold T=2f+1, the system guarantees the minimum number of nodes required to reach consensus, thereby ensuring the reliability and consistency of the system and providing a solid foundation and technical guarantee for distributed key management and encrypted communication.

[0149] Furthermore, through the design of a distributed network and a layered architecture, combined with strong cryptographic parameter configuration, high-quality seed generation and key derivation mechanisms, and efficient consensus algorithms, secure storage and transmission of data and keys are achieved, and the system's fault tolerance and consistency are guaranteed, providing a solid foundation for subsequent key management and encrypted communication.

[0150] S2. Generate a master key and session key using random numbers, and generate asymmetric encryption public-private key pairs for each node;

[0151] In this example, the specific steps for generating the master key and session key using random numbers, and for generating asymmetric encryption public-private key pairs for each node, are as follows:

[0152] S2.1. Generate a random number r using a cryptographically secure random number generator;

[0153] The specific process for generating random numbers is as follows:

[0154] Using the high-entropy seed generated in S1.3 as input, random numbers are generated by combining it with CSPRNG:

[0155] r = CSPRNG(s) root );

[0156] r∈{0,1} 256 ;

[0157] In the formula, r represents a random number;

[0158] S2.2. Derive the session key K by combining the timestamp t and the random number r using HKDF. session ;

[0159] The process of deriving the session key is as follows:

[0160] K root As one of the inputs, the session key K is further derived through HKDF by combining the timestamp t and the random number r. session :

[0161] K session =HKDF(K root ,r||t||info);

[0162] In the formula, K session Indicates the session key;

[0163] S2.3, for each node N n Generate ECDSA public-private key pair Q based on elliptic curves. i ;

[0164] The process of generating ECDSA public-private key pairs based on elliptic curves is as follows:

[0165] For each node N n ∈J, select the optimized elliptic curve E′ and its base point J;

[0166] Generate a random private key:

[0167] d i ∈[1,q-1];

[0168] In the formula, q represents the order of the base point J;

[0169] Calculate the corresponding public key:

[0170] Q i =d i ·J;

[0171] In the formula, Q i This represents the corresponding public key.

[0172] Specifically, by using a high-quality random number generator to generate random numbers and combining them with timestamps and HKDF to derive session keys, the unpredictability and security of the keys are ensured. At the same time, by generating ECDSA public-private key pairs based on optimized elliptic curves for each node, the security of inter-node communication is enhanced, preventing man-in-the-middle attacks and other security threats. This provides a solid security foundation and technical guarantee for key management and encrypted communication in the entire distributed network.

[0173] S3. Divide the master key into multiple fragments and store them on different nodes, and create redundant backups to ensure fault tolerance;

[0174] In this example, the specific steps for splitting the master key into multiple fragments and storing them on different nodes, and creating redundant backups to ensure fault tolerance are as follows:

[0175] S3.1 Construct a polynomial f(X) fragmentation master key and distribute it to multiple nodes according to a threshold.

[0176] The specific process of constructing the polynomial f(X) fragmented master key and distributing it to multiple nodes according to a threshold is as follows:

[0177] Select threshold t s Threshold ts Used to determine how many fragments are needed to rebuild the master key;

[0178] Construct polynomials:

[0179]

[0180] In the formula, f(X) represents a polynomial; K master Indicates the master key; Indicates the coefficients generated randomly;

[0181] Similarly, considering that environmental risks can affect the construction of the polynomial sharding master key, an environmental risk variable is introduced to optimize the polynomial:

[0182]

[0183] In the formula, f(X)′ represents the polynomial after introducing environmental risk variables;

[0184] Fragment generation:

[0185] S s =(X s ,f(X s )′);

[0186] In the formula, S s Indicates the original fragment; X s f(X) represents distinct non-zero integers; s )′ represents the value obtained through polynomial calculation; s represents the number of original fragments;

[0187] S3.2 Use Reed-Solomon coding to generate redundant fragments to ensure fault tolerance and recoverability;

[0188] Among them, Reed-Solomon encoding is used to generate redundant fragments:

[0189] {C1,C2,…,C s+k}=ReedSolomonEncode({S1,S2,…,S s});

[0190] In the formula, ReedSolomonEncode() represents the encoding function; k represents the number of redundant fragments; C s+k This represents the original fragment plus redundant fragments;

[0191] S3.3 Attach a MAC verification tag to each shard to prevent storage tampering;

[0192] The MAC verification tag attached to each shard is as follows:

[0193] Generate MAC tags:

[0194] MAC i =HMAC(K auth C i );

[0195] In the formula, MAC i Represents the i-th MAC tag; HMAC represents the hash-based message authentication code function; K auth Indicates the key used to generate the MAC; C i This represents the i-th master key fragment;

[0196] Additional MAC tag:

[0197] D i =(C i MAC i )

[0198] In the formula, D i This indicates a data packet containing master key fragments and MAC tags.

[0199] Specifically, by constructing polynomial sharding and distributing the master key to multiple nodes, and by introducing environmental risk variables to optimize the polynomial, the security and adaptability of key sharding are ensured; Reed-Solomon encoding is used to generate redundant shards, enhancing the system's fault tolerance and data recovery capabilities; and a MAC verification tag is attached to each shard to further prevent storage tampering, thus providing a highly secure, reliable, and fault-tolerant foundation and technical guarantee for key management and encrypted communication in the entire distributed network.

[0200] Specific application examples are as follows:

[0201] Master key fragmentation and redundancy backup:

[0202] Master Key K master =0x5A3E...D9F1 (256 bits), fragmentation threshold t s =3, total number of fragments s = 5;

[0203] Construct an optimized polynomial:

[0204] The polynomial after introducing environmental risk variables is:

[0205] f(X)′=K master +a1X+a2X 2 modp′;

[0206] Randomly generated coefficients:

[0207] a1 = 0x8C2F…B4A7;

[0208] a2 = 0xE19D...6F3B;

[0209] Generate shards:

[0210] Choose a non-zero integer X s ={1,2,3,4,5}, calculate the fragment value:

[0211]

[0212] Each shard is appended with a MAC tag (using HMAC-SHA256):

[0213] MAC i =HMAC(K auth ,S i );

[0214] Redundant fragment generation:

[0215] Using Reed-Solomon encoding (parameters n=7, k=2 to generate 2 redundant fragments):

[0216] {C6,C7}=ReedSolomonEncode(S1,S2,S3,S4,S5);

[0217] Redundant fragmentation allows the system to recover the master key even if at most two fragments are lost.

[0218] S4. Encrypt the session key using the recipient's public key and distribute it through a secure channel. The recipient then decrypts the key to obtain the session key.

[0219] In this example, the session key is encrypted using the recipient's public key and distributed through a secure channel. The specific steps for the recipient to decrypt and obtain the session key are as follows:

[0220] S4.1, Based on the receiver's public key Q temp Generate temporary elliptic curve key pairs and encrypt session key K. session ;

[0221] Among them, based on the receiver's public key Q temp Generate temporary elliptic curve key pairs and encrypt session key K. session The process is as follows:

[0222] Generate temporary elliptic curve key pairs:

[0223] Choose the same elliptic curve E′ and its base point J as the receiver;

[0224] Generate a temporary private key d temp :

[0225] d temp ∈[1,q-1];

[0226] Calculate the corresponding temporary public key Q temp :

[0227] Q temp =d temp ·J;

[0228] Calculate the shared key:

[0229] K shared =d temp ·Q recv ;

[0230] In the formula, K shared Indicates a shared key; Q recv This represents the recipient's public key;

[0231] Using shared key K shared Session key K session Encrypt:

[0232]

[0233] In the formula, Encrypt AES This indicates that the AES algorithm is used for encryption; This represents the encrypted session key;

[0234] Specifically, AES (Advanced Encryption Standard) is a symmetric encryption algorithm widely used to protect the security of electronic data. It ensures data confidentiality and integrity by using the same key for both encryption and decryption. AES supports key lengths of 128, 192, and 256 bits, corresponding to AES-128, AES-192, and AES-256 variants respectively, with AES-256 offering the highest security. The algorithm processes data in blocks of 128 bits each, transforming the data through multiple rounds of complex substitution, permutation, and mixing operations to resist various attack methods.

[0235] Use the temporary public key Q temp and the encrypted session key Send to the recipient;

[0236] S4.2. Establish a forward confidential channel via TLS 1.3 to transmit the encrypted session key;

[0237] S4.3, The receiver uses the private key d recv Decrypt to obtain session key K session Complete key negotiation;

[0238] The recipient uses the private key d recvDecrypt to obtain session key K session The process of completing key negotiation is as follows:

[0239] The recipient receives the temporary public key Q temp and the encrypted session key

[0240] Calculate the shared key:

[0241] K shared =d recv ·Q temp ;

[0242] In the formula, K shared Indicates a shared key; d recv This represents the recipient's private key;

[0243] Using shared key K shared The encrypted session key Decryption:

[0244]

[0245] In the formula, Decrypt AES This indicates that the AES algorithm is used for decryption.

[0246] Specifically, by generating temporary elliptic curve key pairs and encrypting session keys, the security and unpredictability of the key distribution process are ensured; the use of TLS 1.3 to establish a forward confidentiality channel enhances the security and forward confidentiality of transmission; and the receiver uses its private key to decrypt and obtain the session key, ensuring the security and reliability of the key negotiation process. This provides a highly secure, reliable foundation and technical guarantee with strong forward confidentiality for key management and encrypted communication throughout the distributed network.

[0247] Specific application examples are as follows:

[0248] Session key encryption and secure transmission:

[0249] Generate temporary key pairs:

[0250] Recipient's public key Q recv =d recv ·J (based on optimized elliptic curve E′);

[0251] The sender generates a temporary private key d temp =0x3D9A...F2E1, calculate the temporary public key:

[0252] Q temp =d temp ·J;

[0253] Calculate the shared key:

[0254] K shared =d temp ·Q recv =d temp ·d recv ·J;

[0255] Encrypted session key:

[0256] Encrypt K using AES-256-GCM session =0xA7B2...4C9D:

[0257]

[0258] Output ciphertext and certification label T;

[0259] Secure transmission:

[0260] Send via TLS 1.3 forward secret channel

[0261] Recipient decryption:

[0262] The receiver calculates the shared key K shared =d recv ·Q temp Decryption yields:

[0263]

[0264] Data validation:

[0265] Elliptic curve parameters must meet NIST or SECG standards to ensure resistance to quantum attacks;

[0266] Piecewise recovery: f(X) can be recovered from any three pieces using Lagrange interpolation. ′ Thus, K is obtained. master ;

[0267] Redundant fragmentation: Reed-Solomon encoding parameters must satisfy n≥t s +k to ensure fault tolerance;

[0268] Encryption strength: AES-256 and ECDH key exchange conform to NIST SP 800-56A standard.

[0269] S5. Construct a Merkle tree using the AES-GCM function, and consider that the security requirements and network conditions of data encryption vary in different environments. Introduce a network adaptation factor into the encryption process to optimize the process of data encryption and block storage to distributed nodes.

[0270] In this example, the specific steps for encrypting data using the session key and storing it in blocks on the distributed nodes are as follows:

[0271] S5.1 Divide the original data into blocks of fixed size;

[0272] The process of dividing the original data into fixed-size blocks is as follows:

[0273] The original data is divided into multiple fixed-size blocks;

[0274] The number of data blocks is:

[0275]

[0276] In the formula, w represents the number of data blocks; |A| represents the original data length; and Z represents the block size.

[0277] The size of each data block is:

[0278]

[0279] In the formula, B j This represents the j-th data block; j represents the block number index variable.

[0280] To address the issue of varying network conditions in different environments, a network adjustment variable is introduced to dynamically adjust the block size and encryption parameters to cope with different network conditions. Therefore, a network adaptation factor α is introduced to optimize data encryption and store it in blocks on distributed nodes.

[0281] The number of data blocks after introducing the network adaptation factor is:

[0282]

[0283] In the formula, w′ represents the number of data blocks after introducing the network adaptation factor; Z′ represents the block size after introducing the network adaptation factor.

[0284] Where Z′ is:

[0285] Z′=Z0+Δ Z ·α;

[0286] In the formula, Z0 represents the initial block size; Δ Z This represents the maximum increase in block size; α represents the network adaptation factor.

[0287] Specifically, α is:

[0288]

[0289] In the formula, L net Indicates the current network latency; L maxThis indicates the maximum network latency threshold allowed by the system.

[0290] Then, after introducing the network adaptation factor, the size of each data block is:

[0291]

[0292] In the formula, B j ′ represents the j-th data block after the introduction of the network adaptation factor;

[0293] S5.2. Use the AES-GCM function to encrypt the data block and generate a GHASH tag, then construct a Merkle tree to verify integrity;

[0294] In this example, the specific steps for encrypting data blocks using the AES-GCM function, generating GHASH tags, and constructing a Merkle tree to verify integrity are as follows:

[0295] Each data block is encrypted using the AES-GCM function with the session key and an initial counter, generating ciphertext and a GHASH authentication tag:

[0296] (C j ,T j ) = AES - GCM(K session IV,B j ′);

[0297] In the formula, C j Indicates ciphertext; T j Indicates GHASH certification label;

[0298] Specifically, AES-GCM (Galois / Counter Mode) is an encryption mode that combines symmetric encryption and authentication to ensure the confidentiality, integrity, and authenticity of data. It is based on the AES (Advanced Encryption Standard) algorithm and encrypts data using Counter Mode (CTR mode), while simultaneously using multiplication over the Galois field to generate an authentication tag (GHASH) to verify data integrity. In detail, AES-GCM uses a symmetric key and an initialization vector (IV) to encrypt plaintext data, generating ciphertext and an additional authentication tag. The recipient can decrypt the ciphertext using the same key and IV and verify the authentication tag to confirm that the data has not been tampered with. Due to its efficiency and security, AES-GCM is widely used in communication systems and data storage applications requiring high security.

[0299] GHASH tag T for each encrypted data block j As leaf nodes, construct a Merkle tree;

[0300] Calculate the hash value of each internal node, up to the root node H.root :

[0301]

[0302] In the formula, H k This represents the hash value of an internal node in a Merkle tree; Represents the hash value of the left child node; This represents the hash value of the right child node.

[0303] Specifically, by using the AES-GCM function to encrypt each data block and generate a GHASH authentication tag, the confidentiality and integrity of the data are ensured. Each GHASH tag is used as a leaf node to construct a Merkle tree, and the hash value of the internal nodes is calculated up to the root node, providing an efficient and reliable integrity verification mechanism to prevent data tampering and forgery. This provides a highly secure, reliable and strongly consistent foundation and technical guarantee for data encryption and integrity verification in the entire distributed network.

[0304] S5.3. Map encrypted data blocks to distributed node storage using a consistent hashing algorithm;

[0305] The process of mapping encrypted data blocks to distributed node storage using the consistent hashing algorithm is as follows:

[0306] For each encrypted data block C j Calculate the consistent hash value h′(C) j ):

[0307] h ′ (C j = ConsistentHash(C j );

[0308] In the formula, ConsistentHash represents the consistent hash function.

[0309] Specifically, consistent hashing is a distributed hashing technique designed to solve the problem of data redistribution when nodes are added or removed in a distributed system. It maps data and nodes to a virtual ring space in a special way, thereby minimizing the impact of node changes on the entire system.

[0310] Use consistent hash value n ′ (C j ) Encrypt data block C j Mapped to a distributed node set G = {N1, N2, ..., N} n A node N in} m :

[0311] Nm =MapToNode(h ′ (C j ),G);

[0312] In the formula, MapToNode represents a function that maps data blocks to nodes based on consistent hash values; N m This represents the m-th distributed node; m represents the number of nodes included in n.

[0313] Specifically, MapToNode is a function used to map data blocks to specific nodes in a distributed system. It is often used in conjunction with a consistent hashing algorithm to ensure balanced distribution and efficient access to data in a distributed storage system.

[0314] Furthermore, by dynamically adjusting the data block size and encryption parameters to adapt to different security requirements and network conditions, the flexibility and security of data encryption are ensured; each data block is encrypted using the AES-GCM function and a GHASH authentication tag is generated, combined with a Merkle tree structure for integrity verification to prevent data tampering; and the encrypted data blocks are efficiently and evenly distributed to distributed nodes through a consistent hashing algorithm, ensuring the reliability and scalability of data storage. This provides a highly secure, reliable, and strongly consistent foundation and technical guarantee for data encryption, storage, and integrity verification throughout the distributed network.

[0315] S6. Regularly update keys, implement access control policies, and log system audits to ensure long-term security.

[0316] In this example, the specific steps for regularly updating keys, implementing access control policies, and logging for system auditing to ensure long-term security are as follows:

[0317] S6.1 Periodically update the master key and re-shard it via HKDF to ensure key freshness;

[0318] Specifically, HKDF (HMAC-based Key Derivation Function) is a standard method for generating cryptographic keys from initial key material (such as a cipher or random seed). It ensures sufficient randomness and security of the generated key by using the Hash Message Authentication Code (HMAC) algorithm. The HKDF update process typically involves two main phases: Extract and Expand. In the Extract phase, HKDF combines the input high-entropy seed value (such as a random number or cipher) with a salt value to generate a fixed, high-quality pseudo-random key. In the Expand phase, this pseudo-random key is further processed to generate the final key material of the desired length. By periodically updating the master key using HKDF, key freshness can be ensured, thereby enhancing system security and preventing long-used keys from being compromised or leaked. This method is widely used in systems requiring high security, such as encrypted communication and data storage.

[0319] Furthermore, to ensure system security, the master key needs to be updated periodically to prevent keys used for extended periods from being cracked or leaked. First, a fixed update cycle is set (e.g., every 30 days). At the end of each update cycle, a new high-entropy seed value is collected from a hardware entropy source. This seed value should have sufficient randomness and unpredictability to increase key security. Next, a new master key is generated using HKDF (an HMAC-based key derivation function) combined with the current master key and the newly generated seed value. The generated new master key replaces the old master key, and to further enhance security, the new master key can be segmented into multiple fragments, which are stored on different nodes. This sharding technique not only increases the system's fault tolerance but also reduces the risk of single points of failure.

[0320] S6.2 Control fragment access permissions based on attribute encryption strategy;

[0321] Specifically, to effectively manage access permissions to key fragments, an attribute-based encryption (ABE) strategy is employed. First, a set of attributes is defined for each node in the system. These attributes can be roles, permission levels, or other identifiers used to describe the characteristics or permissions of the node. Then, access policies are formulated based on security requirements, specifying which attribute combinations can access specific data blocks or key fragments. For example, certain critical operations may require both "administrator" and "auditor" roles to execute. Key fragments are encrypted using attribute-based encryption algorithms; only nodes possessing the attribute combinations that match the access policy can decrypt and access these key fragments. This approach not only provides flexible access control but also ensures that only authorized users can access sensitive information.

[0322] S6.3 Use a blockchain structure to record operation logs to ensure the irreversibility of the audit chain.

[0323] Specifically, to ensure the transparency and traceability of system operations, blockchain technology is used to record all critical operation logs. First, an initial block is created, containing basic system information and its initial state. Each time a critical operation is performed (such as key updates or access control), the relevant information is recorded in a new block. Each new block contains not only the log of the current operation but also the hash value of the previous block. In this way, all blocks form a chain structure, and any tampering with historical data will disrupt the consistency of the chain, making it easily detectable. To verify the integrity of the blockchain, the system periodically checks the hash value of each block for correctness. If a block's hash value does not match, it indicates that the block or its subsequent blocks may have been tampered with, requiring further investigation. This blockchain-based auditing mechanism not only guarantees the irreversibility of operation logs but also improves the overall security of the system.

[0324] Furthermore, by regularly updating the master key and re-sharding it, the freshness and security of the key are ensured; strict access control is implemented based on attribute encryption policies to prevent unauthorized access; and operation logs are recorded using blockchain technology to ensure the immutability and transparency of the audit chain. This provides a highly secure, reliable, and audit-capable foundation and technical guarantee for key management and encrypted communication throughout the distributed network.

[0325] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely preferred examples and are not intended to limit the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the claimed invention.

Claims

1. A distributed key management and encrypted communication method for big data environment, characterized in that, The method comprises the following steps: S1, building a system architecture and configuring security parameters, generating an initial seed value and deploying a key management center; S2, generating a master key and a session key through a random number, and generating an asymmetrically encrypted public-private key pair for each node; S3, dividing the master key into multiple segments and storing them in different nodes, and creating a redundant backup to ensure fault tolerance; S4, encrypting the session key using the receiver's public key and distributing it through a secure channel, and the receiver decrypts to obtain the session key; S5, constructing a Merkle tree through an AES-GCM function, and considering that data encryption has different security requirements and network conditions in different environments, introducing a network adaptation factor into the encryption process to optimize data encryption and block storage to distributed nodes; S6, regularly updating the key, implementing access control policies, and recording logs for system auditing to ensure long-term security; In S3, the specific steps for dividing the master key into multiple segments and storing them in different nodes, and creating a redundant backup to ensure fault tolerance are: S3.1, Constructing a polynomial For a shard master key, distributed to multiple nodes by threshold Among them, considering that environmental risk will affect the construction of polynomial fragmentation master key, therefore, the environmental risk variable is introduced to optimize the polynomial : ； wherein represents a polynomial after introduction of an environmental risk variable; represents a master key; represents a randomly generated coefficient; Fragment generation: ； wherein represents the original fragments; represents different non-zero integers; represents a value calculated by a polynomial; represents the number of original fragments; S3.2, generate redundant fragments using Reed-Solomon encoding to ensure fault tolerance and recoverability; S3.3, append to each shard authentication tag, to prevent storage tampering.

2. The method for distributed key management and encrypted communication in big data environment according to claim 1, characterized in that: In S1, the specific steps for building a system architecture and configuring security parameters, generating an initial seed value and deploying a key management center are: S1.1, constructing a set of distributed network nodes comprising nodes , designing a hierarchical architecture; S1.2, based on the hierarchical architecture, configure the cryptography parameters through the elliptic curve and hash function, and symmetric encryption algorithm; S1.3, harvesting a high-entropy seed from a hardware entropy source , by deriving a root key; S1.4, based on the PBFT consensus algorithm, allocate proposal nodes and verification nodes in the hierarchical architecture, and set the fault tolerance condition.

3. The method for distributed key management and encrypted communication in big data environment according to claim 2, characterized in that: In S1.2, the specific process for configuring cryptography parameters based on the hierarchical architecture through the elliptic curve and hash function, and symmetric encryption algorithm is: S1.21, select an elliptic curve for secure encryption; S1.22, map an arbitrary length binary string to a fixed length binary string of 256 bits through a hash function; S1.23, input 256-bit key The 15 round keys are generated by the Rijndael key schedule algorithm and the plaintext is encrypted and the authentication tag is computed using the AES-256 encryption algorithm in combination with the counter mode and the GHASH polynomial hash function.

4. The method for distributed key management and encrypted communication in big data environment according to claim 3, characterized in that: In S1.21, the elliptic curve is specifically: ； wherein represents an elliptic curve equation; represents a coordinate variable for the ordinate on the curve; represents a coordinate variable for the abscissa on the curve; represents a linear coefficient; represents a constant term coefficient; represents an elliptic curve modulus; represents a modulo operation; Considering that environmental risks can affect the confidentiality of secure encryption, an environmental risk variable is introduced to optimize the elliptic curve equation: ； In the formula, represents the optimized elliptic curve equation; represents the elliptic curve modulus after importing the environmental risk variable.

5. The method for distributed key management and encrypted communication in big data environment according to claim 4, characterized in that: In S2, the specific steps for generating a master key and a session key through a random number, and generating an asymmetrically encrypted public-private key pair for each node are: S2.1, generating a random number using a cryptographically secure random number generator ; S2.2, by combining timestamps and random numbers deriving a session key ; S2.3, for each node Generating an elliptic curve based ECDSA public-private key pair .

6. The method for distributed key management and encrypted communication in big data environment according to claim 5, characterized in that: In S4, the specific steps for encrypting the session key using the receiver's public key and distributing it through a secure channel, and the receiver decrypting to obtain the session key are: S4.1, based on the receiver public key Generating a temporary elliptic curve key pair, encrypting the session key ; S4.2, establish a forward secrecy channel through TLS 1.3 to transmit the encrypted session key; S4.3, receiver uses private key Decryption yields session key Key agreement is complete.

7. The method for distributed key management and encrypted communication in big data environment according to claim 6, characterized in that: In S5, the specific steps for encrypting data using the session key and storing it in blocks in distributed nodes are: S5.1, divide the original data into fixed-size blocks; Among them, for the problem of network condition difference under different environments for data encryption, network adjustment variable is introduced, and block size and encryption parameter are dynamically adjusted to cope with different network conditions, so network adaptation factor is introduced Optimize data encryption and block storage to distributed nodes; S5.2, encrypt the data blocks using the AES-GCM function and generate a GHASH tag to construct a Merkle tree to verify integrity; S5.3, map the encrypted data blocks to distributed node storage through a consistent hashing algorithm; The process of mapping encrypted data blocks to distributed node storage through a consistent hashing algorithm is: for each encrypted data block computing a consistent hash value ; Using consistent hash values Mapping encrypted data blocks To one of a set of distributed nodes .​ 8. The method for distributed key management and encrypted communication in big data environment according to claim 7, characterized in that: In S5.2, the data block is encrypted using the AES-GCM function and the GHASH tag is generated, and the specific steps for constructing the Merkle tree to verify the integrity are as follows: In S5.2, the data block is encrypted using the AES-GCM function and the GHASH tag is generated, and the specific steps for constructing the Merkle tree to verify the integrity are as follows: calculating a GHASH tag for each encrypted data block constructing a Merkle tree as leaf nodes; Compute the hash value of each internal node until the root node .

9. The distributed key management and encrypted communication method for big data environment of claim 8, wherein: In S6, the specific steps for periodically updating the key, implementing access control strategy, and recording the log for system audit to ensure long-term security are as follows: S6.1, periodically update the master key and re-sharding through HKDF to ensure key freshness; S6.2, control the access permission of the shard based on the attribute encryption policy; S6.3, record the operation log with the block chain structure to ensure the irreversibility of the audit chain.

Images