A distributed encryption storage method based on dynamic key and data fragmentation

Abstract

The application discloses a kind of distributed encryption storage methods based on dynamic key and data fragment, it is related to the field of distributed encryption storage, including obtaining original data, dynamic fragmentation is carried out according to attribute information and current topological state of system;Based on dynamic key generation mechanism, the corresponding dynamic encryption key is respectively generated for each data fragment;Corresponding data fragment is respectively encrypted using each dynamic encryption key;Each encrypted data fragment is stored to multiple storage nodes according to preset strategy, and storage location mapping table is generated;Threshold secret sharing algorithm is used to carry out fragmentation processing to each dynamic encryption key, and each key fragment is stored in the node different from corresponding encrypted data fragment;When reading, the corresponding key is restored by obtaining threshold number of key fragments, and the original data is reorganized and restored after decryption according to fragment identification and mapping table.The application improves system security and adaptability through the cooperation of key update and topological change linkage and the physical isolation storage of key and data.

Description

Technical Field

[0001] This application relates to the field of distributed encrypted storage technology, and in particular to a distributed encrypted storage method based on dynamic keys and data fragmentation. Background Technology

[0002] With the rapid development of cloud computing and big data technologies, distributed storage systems have become a core infrastructure for managing massive amounts of data. In a distributed storage architecture, data is typically distributed across multiple physical nodes to improve storage capacity, read / write performance, and system availability. To ensure data security, existing commercial distributed storage solutions generally employ static encryption mechanisms, which encrypt the entire data using a fixed key before writing it, and then store the encrypted data in chunks of fixed size and quantity across different nodes. Furthermore, technologies such as consistent hashing and redundant backups have also been widely applied in the field of data distribution and fault tolerance management in distributed storage.

[0003] However, the aforementioned existing technologies still have the following shortcomings: First, the encryption keys are static keys and are centrally stored on a single server. Once this server is attacked or malfunctions, it will lead to large-scale data loss or service interruption. Second, the storage location of the encryption keys and the encrypted data shards lack effective isolation constraints. The two can coexist in the same set of nodes. If an attacker controls some nodes, they can simultaneously obtain the data shards and their decryption keys, resulting in a systemic vulnerability in security protection. Third, the data sharding strategy and key update strategy are both statically configured. The number and size of shards do not change with data attributes. Key updates are only triggered based on fixed time periods and are independent of the real-time topology changes and load status of storage nodes. It is impossible to achieve adaptive dynamic adjustment of keys and shards when the node status changes. Summary of the Invention

[0004] The purpose of this application is to provide a distributed encrypted storage method based on dynamic keys and data fragmentation to solve the above-mentioned problems in the existing technology.

[0005] To achieve the above objectives, this application provides a distributed encrypted storage method based on dynamic keys and data fragmentation, comprising the following steps: S1: Obtain the original data to be stored, and perform dynamic sharding on the original data according to the attribute information of the original data and the current topology status information of the distributed storage system, generate N data shards, N≥2, and generate a unique shard identifier for each data shard. S2: Based on the dynamic key generation mechanism, a one-to-one dynamic encryption key is generated for each of the N data shards. The dynamic encryption key is dynamically updated according to the time dimension and the node topology change status of the distributed storage system. S3: Use each dynamic encryption key to encrypt the corresponding data fragments to generate N encrypted data fragments; S4: Store the N encrypted data fragments into multiple storage nodes of the distributed storage system according to the preset distributed storage strategy, and generate a storage location mapping table for each encrypted data fragment. S5: Perform key fragmentation processing on each dynamic encryption key. Each dynamic encryption key generates M key fragments, M≥2. Store the M key fragments in storage nodes that are different from the corresponding encrypted data fragments. The same storage node does not store the complete data fragment and all its corresponding key fragments at the same time. S6: In response to a data read request, obtain the key fragments of a preset threshold number corresponding to each encrypted data fragment from the distributed storage system to restore the corresponding dynamic encryption key, use the restored dynamic encryption key to decrypt each encrypted data fragment, and reassemble and restore the original data according to the fragment identifier and storage location mapping table.

[0006] Preferably, the attribute information includes the data size and data sensitivity level of the original data, and the current topology status information includes the number of currently available storage nodes and the real-time load status of each storage node; the dynamic sharding process dynamically determines the number of shards N and the size of each data shard based on the data size, data sensitivity level and real-time load status.

[0007] Preferably, the dynamic key generation mechanism in step S2 includes: S21: Obtain the current system timestamp T, user ID UID, system random number R, and the list of currently active nodes in the distributed storage system L; S22: Concatenate the combined hash values ​​of the timestamp T, user ID (UID), random number R, and active node list L to obtain the combination factor. ; S23: Utilize the key derivation function to operate on the combination factor F to generate the dynamic encryption key corresponding to each data fragment. ; S24: Monitor the node topology changes of the distributed storage system. When a storage node is detected to join, leave, or undergo a permanent state change, automatically update the active node list L and re-execute steps S21 to S23 to generate a new generation of dynamic encryption keys, while marking the old key as a historical state.

[0008] Preferably, the encryption process in step S3 employs a hybrid encryption mechanism combining symmetric and asymmetric encryption algorithms, including: S31: Use a dynamic encryption key to perform symmetric encryption on the corresponding data fragments to generate encrypted data fragments; S32: Use the public key in the asymmetric encryption algorithm to encrypt the dynamic encryption key to generate key ciphertext; S33: Associate the key ciphertext with the encrypted data fragments for storage.

[0009] Preferred, pre-defined distributed storage strategies include: The target storage node for each encrypted data shard is determined based on the consistent hashing algorithm. Each encrypted data shard is stored on a different physical storage node, and the data redundancy is dynamically adjusted according to the real-time status of the storage node. The number of redundant replicas is increased for encrypted data shards that are accessed frequently.

[0010] Preferably, in step S5, the preset threshold number of the threshold secret sharing algorithm is t, where t < M. Any t key fragments can restore the dynamic encryption key, and any number of fewer than t key fragments cannot obtain any information about the dynamic encryption key. The M key fragments are stored on M different storage nodes, and the storage nodes storing the key fragments and the storage nodes storing the corresponding encrypted data fragments satisfy the following: the intersection of their node sets is an empty set or the number of intersection nodes is less than the preset threshold number t.

[0011] Preferably, the data read request in step S6 includes a user authentication step, specifically including: Receive a data read request and extract the user identity credentials and the identifier of the data to be read from the request; The user's identity credentials are compared with a preset access control list to verify whether the user has the access rights to the data corresponding to the data identifier to be read. The key fragment acquisition and dynamic encryption key restoration operations are only performed after both authentication and authorization verification are passed.

[0012] Preferably, it also includes a data integrity verification step: After performing encryption, the hash value of each encrypted data fragment is calculated, and the hash value is associated with the corresponding fragment identifier and stored in an immutable distributed storage system to form an integrity verification record. Before performing decryption, the hash value of the corresponding encrypted data fragment is obtained from the immutable distributed storage system. The hash value of the current encrypted data fragment is recalculated and compared. If the comparison results match, the decryption operation continues. If they do not match, an automatic error correction or data recovery mechanism is triggered.

[0013] Preferably, it also includes key lifecycle management steps: Set an expiration date for each dynamic encryption key. Before the expiration date, automatically trigger the key update process to generate a new generation of dynamic encryption keys. The corresponding data fragments are re-encrypted using a new generation of dynamic encryption keys, and the old dynamic encryption keys and their corresponding key fragments are destroyed or archived. Key update operation records are written to the tamper-proof audit log system to form an immutable key update audit record.

[0014] Preferably, it also includes an anomaly recovery step: When a storage node in a distributed storage system is detected to be permanently offline or its data is corrupted, redundant copies of the corresponding encrypted data fragments are obtained from other storage nodes to recover the data, based on the redundancy configuration in the preset distributed storage strategy. If the dynamic encryption key cannot be restored due to the loss of key fragments, the system will obtain a sufficient number of key fragments from other storage nodes and restore the dynamic encryption key based on the redundancy characteristics of the threshold secret sharing algorithm for key fragments. This ensures that the system can still decrypt data normally when no more than a preset threshold number of storage nodes fail.

[0015] Therefore, the distributed encrypted storage method based on dynamic keys and data fragmentation described above has the following advantages: (1) This application avoids the risk of historical data loss caused by static keys remaining unchanged for a long time by setting the encryption key to be dynamically updated based on time dimension and node topology changes. When the key is leaked, it only affects the data within a limited time window, reducing the scope of damage caused by single point attacks.

[0016] (2) This application improves the anti-attack capability of the distributed storage system by using threshold secret sharing and fragmented storage of dynamic encryption keys and physically isolating the storage node sets of key fragments and data fragments. When any storage node is compromised, the data fragments and their corresponding decryption keys are not leaked at the same time.

[0017] (3) This application dynamically segments the original data and encrypts each data segment independently, so that each data segment corresponds to an independent dynamic encryption key. When the key is updated, only the current segment needs to be processed, and there is no need to re-encrypt the entire data, which reduces the computational overhead and network transmission cost when updating the key.

[0018] The technical solution of this application will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0019] Figure 1 This is a flowchart illustrating a distributed encrypted storage method based on dynamic keys and data fragmentation in this application. Detailed Implementation

[0020] The following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0021] Unless otherwise defined, the technical or scientific terms used in this application shall have the ordinary meaning as understood by a person of ordinary skill in the art to which this application pertains.

[0022] The terms "comprising" or "including," as used in this application, mean that the element preceding the term encompasses the element listed after the term, and do not exclude the possibility of encompassing other elements as well. The terms "inner," "outer," "upper," and "lower," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application. When the absolute position of the described object changes, the relative positional relationship may also change accordingly. In this application, unless otherwise expressly specified and limited, the term "attached," etc., should be interpreted broadly. For example, it can refer to a fixed connection, a detachable connection, or an integral part; it can refer to a direct connection or an indirect connection through an intermediate medium; it can refer to the internal communication of two elements or the interaction relationship between two elements. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.

[0023] Example 1: A distributed encrypted storage method based on dynamic keys and data fragmentation, such as Figure 1 As shown, it includes the following steps: S1: Obtain the original data to be stored, and perform dynamic sharding on the original data according to the attribute information of the original data and the current topology status information of the distributed storage system, generate N data shards, N≥2, and generate a unique shard identifier for each data shard. The attribute information includes the data size and data sensitivity level of the original data. The current topology status information includes the number of currently available storage nodes and the real-time load status of each storage node. The dynamic sharding process dynamically determines the number of shards N and the size of each data shard based on the data size, data sensitivity level, and real-time load status.

[0024] S2: Based on the dynamic key generation mechanism, a one-to-one dynamic encryption key is generated for each of the N data shards. The dynamic encryption key is dynamically updated according to the time dimension and the node topology change status of the distributed storage system. Dynamic key generation mechanisms include: S21: Obtain the current system timestamp T, user ID (UID), system random number R, and the list of currently active nodes in the distributed storage system L; specifically, the timestamp T (with millisecond precision, e.g., August 5, 2026, 14:30:25.123), the user ID (e.g., user ID "user_123"), the 128-bit random number R generated by the system's secure random number generator (e.g., "0x7F3A92B1C4D5E6F7890ABCDEF1234567"), and the list of currently active nodes in the distributed storage system L (e.g., ... There are 12 storage nodes in total. The active node list contains the identification information of all currently online storage nodes and is dynamically updated as storage nodes are added or removed: each storage node reports its online status via heartbeat. If it does not respond within a timeout period, it is considered offline and removed from the list. New nodes are added to the list after completing registration.

[0025] S22: Concatenate the combined hash values ​​of the timestamp T, user ID (UID), random number R, and active node list L to obtain the combination factor. ; S23: Utilize the key derivation function to operate on the combination factor F to generate the dynamic encryption key corresponding to each data fragment. ; S24: Monitor node topology changes in the distributed storage system. When a storage node is detected to join, leave, or undergo a permanent state change, automatically update the active node list L and re-execute steps S21 to S23 to generate a new generation of dynamic encryption keys, while marking the old key as a historical state. Specifically, monitoring is performed through the heartbeat connection between each storage node and the central management node. When a storage node is detected to join (first heartbeat report), leave (heartbeat timeout with 3 consecutive no response, timeout threshold 30 seconds), or undergo a state change (such as a change in node load triggering topology reconfiguration), the storage copy of the active node list L is automatically updated.

[0026] Through the aforementioned dynamic key generation mechanism, different data fragments correspond to different dynamic encryption keys at any given time, and the key for the same fragment continues to evolve with time and system topology changes. The validity period of any key is only the time window from the time of generation to the time when the next topology change triggers the update, which greatly reduces the risk of batch loss of historical data due to single key leakage.

[0027] S3: Use each dynamic encryption key to encrypt the corresponding data fragments to generate N encrypted data fragments; The encryption process employs a hybrid encryption mechanism that combines symmetric and asymmetric encryption algorithms, including: S31: Use a dynamic encryption key to perform symmetric encryption on the corresponding data fragments to generate encrypted data fragments; Specifically, using dynamic encryption keys The corresponding data fragments are subjected to symmetric encryption. The system treats each data fragment as an independent encryption unit and encrypts it using a symmetric encryption algorithm (such as AES-256-GCM). Specifically, the data fragments are... and dynamic encryption key Input is fed into the AES-256-GCM encryption engine to generate encrypted data fragments. During the encryption process, the system independently generates a 96-bit random initialization vector (IV) for each data fragment, for example... =Random number generator(), to ensure that the same key produces different ciphertext outputs when encrypting the same data fragment at different times, effectively resisting replay attacks; at the same time, AES-256-GCM mode synchronously generates the authentication tag of the data fragment during the encryption process. This is used for integrity verification during decryption. After encryption, the system fragments the encrypted data. , corresponding initial vector Certification Label Together with the fragment identifier, they are encapsulated into an encrypted fragment data packet, which serves as the final storage unit for the fragment. The initialization vector and authentication tag are stored in plaintext as metadata associated with the encrypted data fragment, without the need for separate encryption.

[0028] S32: Use the public key in the asymmetric encryption algorithm to encrypt the dynamic encryption key to generate key ciphertext; Specifically, the system pre-generates an asymmetric key pair for each user or each storage space, including a public key PK and a private key SK. The public key PK is used for encryption, and the private key SK is kept by the user and used for decryption. For each data fragment i, a dynamic encryption key is generated. (256 bits in length) The system uses an asymmetric encryption algorithm (such as RSA-2048 or SM2 elliptic curve public-key cryptography) to encrypt the dynamic encryption key, generating the corresponding key ciphertext. The encryption operation is performed in secure memory. After encryption is complete, the plaintext of the dynamic encryption key K_i is immediately cleared from memory, leaving only the ciphertext of the key. .

[0029] S33: Associate the key ciphertext with the encrypted data fragments for storage.

[0030] Specifically, the key ciphertext As part of the fragment metadata, it is encapsulated together with the encrypted data fragment in the same encrypted fragment data packet, or stored separately in the key metadata table associated with the encrypted data fragment, and associated with it through the fragment identifier. Because it uses asymmetric encryption, even if the user's public key PK is exposed, an attacker cannot recover the dynamic encryption key from the key ciphertext. Only authorized users holding the corresponding private key SK can decrypt the ciphertext of the key and recover the dynamic encryption key during the data reading phase. The plaintext is then used to decrypt the corresponding encrypted data fragments. Through this method, the dynamic encryption key is stored in ciphertext form, achieving dual protection for both the key and the data.

[0031] S4: Store the N encrypted data fragments into multiple storage nodes of the distributed storage system according to the preset distributed storage strategy, and generate a storage location mapping table for each encrypted data fragment. The preset distributed storage strategies include: The target storage node for each encrypted data shard is determined based on the consistent hashing algorithm. Each encrypted data shard is stored on a different physical storage node, and the data redundancy is dynamically adjusted according to the real-time status of the storage node. The number of redundant replicas is increased for encrypted data shards that are accessed frequently.

[0032] Specifically, the system maps the IP addresses or node identifiers of each storage node in the distributed storage system to a contiguous hash ring using hash calculation. Simultaneously, the fragment identifiers of each encrypted data fragment are also mapped to this hash ring after being calculated using the same hash function. The system then searches clockwise for the first storage node whose hash value is greater than or equal to the fragment's hash value, and designates that node as the target storage node for that encrypted data fragment, thus achieving a uniform distribution of encrypted data fragments among storage nodes. To ensure that each encrypted data fragment is stored on a different physical storage node, the system performs node deduplication during allocation. If a storage node has already been allocated to store an encrypted data fragment of the same original data, the system continues searching clockwise along the hash ring for the next available storage node until a physical node different from the already allocated node is found. Meanwhile, the system monitors the load status of each storage node in real time (including CPU utilization, disk space remaining, network bandwidth utilization, etc.). When it detects that the load of a storage node exceeds a preset threshold (e.g., CPU utilization exceeds 80% or disk space remaining is less than 10%), it dynamically adjusts the allocation strategy of subsequent shards and allocates them to storage nodes with lower loads to avoid single-node overload.

[0033] The system dynamically adjusts data redundancy based on data access frequency. Specifically, the system maintains an access counter and a time window counter for each encrypted data shard, counting the number of times the shard is accessed within a preset period (e.g., the past 24 hours). When this number exceeds a preset high-frequency threshold (e.g., 1000 times), the system automatically triggers a redundant replica addition operation, creating an additional replica for the encrypted data shard and storing it on a new storage node (preferably located on a different rack or in a different availability zone than the original replica). The access records of this additional replica are also included in the load balancing strategy. When the access frequency of the shard falls below the high-frequency threshold and remains below it for a preset time (e.g., 72 hours), the system automatically reclaims the additional replica to release storage resources. Through this method, the system achieves load-aware differentiated redundancy protection while ensuring physical isolation of each encrypted data shard, guaranteeing high availability of frequently accessed data and avoiding excessive waste of storage resources.

[0034] S5: Perform key fragmentation processing on each dynamic encryption key. Each dynamic encryption key generates M key fragments, M≥2. Store the M key fragments in storage nodes that are different from the corresponding encrypted data fragments. The same storage node does not store the complete data fragment and all its corresponding key fragments at the same time. The key fragmentation process employs a threshold secret sharing algorithm with a preset threshold number of fragments, t < M. Any t key fragments can be used to reconstruct the dynamic encryption key, and any number of fragments less than t cannot obtain any information about the dynamic encryption key. The M key fragments are stored on M different storage nodes, and the storage nodes storing the key fragments and the storage nodes storing the corresponding encrypted data fragments satisfy the following: the intersection of their node sets is an empty set or the number of intersection nodes is less than the preset threshold number t.

[0035] In this embodiment, for the key fragmentation processing of dynamic encryption keys, the Shamir threshold secret sharing algorithm is used to fragment the dynamic encryption key corresponding to each data fragment separately. Specifically, for any dynamic encryption key... The system uses this as a secret value, selects a large prime number p (p > K and p > M), and randomly generates a polynomial of degree t-1 over the finite field GF(p). , where the coefficient , ,…, The key is randomly selected by the system's secure random number generator. The system then assigns a unique non-zero number to each key fragment. Calculate the polynomial value corresponding to each number. Generate M key fragments This involves constructing each key fragment as a coordinate point on a two-dimensional plane. Based on the mathematical principle of Shamir threshold secret sharing, any t key fragments (i.e., t coordinate points) can be uniquely reconstructed from the polynomial using Lagrange interpolation. The constant term K is used to realize the restoration of the dynamic encryption key; however, any number of key fragments less than t cannot reconstruct this polynomial, and therefore cannot obtain any information about the dynamic encryption key K, thus ensuring the absolute security of the key from the information theory level.

[0036] Regarding the storage and allocation of key fragments, the system stores the M key fragments on M different storage nodes, with each storage node storing only one key fragment. Simultaneously, the system imposes strict physical isolation constraints on the storage nodes for the key fragments and the corresponding encrypted data fragments: the system obtains a first set of nodes P storing the M key fragments and a second set of nodes Q storing the corresponding encrypted data fragments. When allocating key fragment storage nodes, priority is given to allocating key fragments to nodes in the first set of nodes P that have no intersection with the second set of nodes Q; that is, the intersection of the two set of nodes is an empty set. When the number of available nodes in the distributed storage system is insufficient to achieve completely intersectionless allocation, the system controls the number of intersection nodes to be less than a preset threshold t. This isolation constraint ensures that even if an attacker compromises all nodes in the intersection of the storage node sets P and Q (i.e., nodes that simultaneously store key fragments and encrypted data fragments), the number of key fragments obtained will still be less than the threshold value t, making it impossible to reconstruct the complete dynamic encryption key and thus unable to decrypt the corresponding encrypted data fragments. Through this combination of threshold secret sharing and physical isolation, the system achieves a balance between high key security and high availability.

[0037] S6: In response to a data read request, obtain the key fragments of a preset threshold number corresponding to each encrypted data fragment from the distributed storage system to restore the corresponding dynamic encryption key, use the restored dynamic encryption key to decrypt each encrypted data fragment, and reassemble and restore the original data according to the fragment identifier and storage location mapping table.

[0038] Data read requests include user authentication steps, specifically: Receive a data read request and extract the user identity credentials and the identifier of the data to be read from the request; The user's identity credentials are compared with a preset access control list to verify whether the user has the access rights to the data corresponding to the data identifier to be read. The key fragment acquisition and dynamic encryption key restoration operations can only be performed after both authentication and authorization verification have passed.

[0039] In an optional embodiment, a data integrity verification step is also included: After encryption, the hash value of each encrypted data fragment is calculated using the SHA-256 hash algorithm. The hash value is associated with the corresponding fragment identifier to construct an integrity verification record, which is then stored in an immutable distributed storage system to form an integrity verification record. This immutable distributed storage system is a blockchain network (such as Hyperledger Fabric or Ethereum). The integrity verification record is written as transaction data into the distributed ledger of the blockchain. Once written, it cannot be tampered with or deleted, forming a traceable and verifiable integrity verification ledger.

[0040] Before decryption, the hash value of the corresponding encrypted data fragment is obtained from the immutable distributed storage system. The hash value of the current encrypted data fragment is recalculated and compared. If the comparison results match, the decryption operation continues; otherwise, an automatic error correction or data recovery mechanism is triggered. The automatic error correction or data recovery mechanism includes: according to the redundant storage strategy in step S4, obtaining a redundant copy of the encrypted data fragment from other storage nodes, recalculating the hash value of the redundant copy, and comparing it again with the original hash value stored in the blockchain until an undisturbed valid copy is found. If all copies fail the integrity verification, the system returns a data corruption anomaly report to the upper-layer application and freezes all associated versions of the fragment to prevent further propagation. Through the above mechanism combining blockchain notarization and real-time verification, the system achieves tamper-proof verification and automatic recovery of encrypted data fragments, ensuring the integrity and trustworthiness of the stored data.

[0041] In an optional embodiment, a key lifecycle management step is also included: Set an expiration date for each dynamic encryption key. Before the expiration date, automatically trigger the key update process to generate a new generation of dynamic encryption keys. The corresponding data fragments are re-encrypted using a new generation of dynamic encryption keys, and the old dynamic encryption keys and their corresponding key fragments are destroyed or archived. Key update operation records are written to the tamper-proof audit log system to form an immutable key update audit record.

[0042] Specifically, the system independently sets an expiration period for each dynamic encryption key. The validity period is determined based on the sensitivity level of the data fragment (e.g., 24 hours, 7 days, or 30 days). The system records the generation time of each key and uses the sum of the generation time and the validity period as the expiration time of that key. The system background polls the key metadata table at preset time intervals. When it detects that the current time has exceeded the key's expiration time or that the time remaining before the expiration time is less than the preset update threshold, it automatically triggers the key update process: re-executes steps S21 to S23 to generate a new generation of dynamic encryption keys; re-encrypts the corresponding data fragment using the new generation of dynamic encryption keys (decrypting the data fragment with the old key and then re-encrypting it with the new key), and replaces the original storage content with the re-encrypted data fragment, while the number and identifier of the fragments remain unchanged. After re-encryption, the system performs destruction processing (completely erasing it from the physical storage medium by overwrite deletion) or archiving processing (transferring it to the archive storage area after encryption and setting access permissions). The system writes key update operation records to an anti-tampering audit log system, which is a blockchain network or WORM storage system. Each record contains the old key identifier, the new key identifier, the update time, the shard identifier involved, and the operation status. Once written, it cannot be modified or deleted.

[0043] In an optional embodiment, an anomaly recovery step is also included: When a storage node in a distributed storage system is detected to be permanently offline or its data is corrupted, the system queries the storage location mapping table for all replica locations of the encrypted data fragment according to the redundancy configuration in the preset distributed storage strategy (i.e., the number of redundant replicas and storage locations of each encrypted data fragment), selects the redundant replicas on other online storage nodes, copies them, and replaces them with the new online nodes to restore the number of redundant replicas of the fragment. If the dynamic encryption key cannot be recovered due to the loss of key fragments, the number of currently available key fragments is counted based on the redundancy characteristics of the threshold secret sharing algorithm. If the number of available fragments is still greater than or equal to the preset threshold t, the dynamic encryption key is recovered from any t available key fragments, and the decryption operation continues. If the number of available fragments is less than t, the key cannot be recovered, and an error report is returned. Through this mechanism, the system can still decrypt data normally even when no more than the preset threshold t of storage nodes fail simultaneously.

[0044] Therefore, this application adopts the above-mentioned distributed encrypted storage method based on dynamic keys and data sharding. By dynamically sharding the original data and encrypting each shard independently, dynamically updating the encryption key corresponding to each shard based on time and topology, and storing the dynamic encryption key in sharded secrets with thresholds and physically isolating it from the data shards, this method achieves the technical effects of only affecting the data within a limited time window when the key is leaked, not leaking the data shards and their corresponding keys when any storage node is compromised, and not needing to re-encrypt the entire data when the key is updated.

[0045] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application and not to limit them. Although this application has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of this application, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of this application.

Claims

1. A distributed encrypted storage method based on dynamic keys and data fragmentation, characterized in that, Includes the following steps: S1: Obtain the original data to be stored, and perform dynamic sharding on the original data according to the attribute information of the original data and the current topology status information of the distributed storage system, generate N data shards, N≥2, and generate a unique shard identifier for each data shard. S2: Based on the dynamic key generation mechanism, a one-to-one dynamic encryption key is generated for each of the N data shards. The dynamic encryption key is dynamically updated according to the time dimension and the node topology change status of the distributed storage system. S3: Use each dynamic encryption key to encrypt the corresponding data fragments to generate N encrypted data fragments; S4: Store the N encrypted data fragments into multiple storage nodes of the distributed storage system according to the preset distributed storage strategy, and generate a storage location mapping table for each encrypted data fragment. S5: The threshold secret sharing algorithm is used to perform key fragmentation processing on each dynamic encryption key. Each dynamic encryption key generates M key fragments, M≥2. The M key fragments are stored in storage nodes that are different from the corresponding encrypted data fragments. The same storage node does not store the complete data fragment and its corresponding key fragments at the same time. S6: In response to a data read request, obtain the key fragments of a preset threshold number corresponding to each encrypted data fragment from the distributed storage system to restore the corresponding dynamic encryption key, use the restored dynamic encryption key to decrypt each encrypted data fragment, and reassemble and restore the original data according to the fragment identifier and storage location mapping table.

2. The distributed encrypted storage method based on dynamic keys and data fragmentation as described in claim 1, characterized in that, The attribute information includes the data size and data sensitivity level of the original data, and the current topology status information includes the number of currently available storage nodes and the real-time load status of each storage node; the dynamic sharding process dynamically determines the number of shards N and the size of each data shard based on the data size, data sensitivity level and real-time load status.

3. The distributed encrypted storage method based on dynamic keys and data fragmentation as described in claim 1, characterized in that, The dynamic key generation mechanism in step S2 includes: S21: Obtain the current system timestamp T, user ID UID, system random number R, and the list of currently active nodes in the distributed storage system L; S22: Concatenate the combined hash values ​​of the timestamp T, user ID (UID), random number R, and active node list L to obtain the combination factor. ; S23: Utilize the key derivation function to operate on the combination factor F to generate the dynamic encryption key corresponding to each data fragment. ; S24: Monitor the node topology changes of the distributed storage system. When a storage node is detected to join, leave, or undergo a permanent state change, automatically update the active node list L and re-execute steps S21 to S23 to generate a new generation of dynamic encryption keys, while marking the old key as a historical state.

4. The distributed encrypted storage method based on dynamic keys and data fragmentation as described in claim 1, characterized in that, Step S3 employs a hybrid encryption mechanism combining symmetric and asymmetric encryption algorithms, including: S31: Use a dynamic encryption key to perform symmetric encryption on the corresponding data fragments to generate encrypted data fragments; S32: Use the public key in the asymmetric encryption algorithm to encrypt the dynamic encryption key to generate key ciphertext; S33: Associate the key ciphertext with the encrypted data fragments for storage.

5. The distributed encrypted storage method based on dynamic keys and data fragmentation as described in claim 1, characterized in that, The preset distributed storage strategies include: The target storage node for each encrypted data shard is determined based on the consistent hashing algorithm. Each encrypted data shard is stored on a different physical storage node, and the data redundancy is dynamically adjusted according to the real-time status of the storage node. The number of redundant replicas is increased for encrypted data shards that are accessed frequently.

6. The distributed encrypted storage method based on dynamic keys and data fragmentation as described in claim 1, characterized in that, In step S5, the preset threshold number of the threshold secret sharing algorithm is t, where t < M. Any t key fragments can restore the dynamic encryption key, and any number of fewer than t key fragments cannot obtain any information about the dynamic encryption key. The M key fragments are stored on M different storage nodes, and the storage nodes storing the key fragments and the storage nodes storing the corresponding encrypted data fragments satisfy the following: the intersection of their node sets is an empty set or the number of intersection nodes is less than the preset threshold number t.

7. The distributed encrypted storage method based on dynamic key and data fragmentation as described in claim 1, characterized in that, The data read request in step S6 includes a user authentication step, specifically including: Receive a data read request and extract the user identity credentials and the identifier of the data to be read from the request; The user's identity credentials are compared with a preset access control list to verify whether the user has the access rights to the data corresponding to the data identifier to be read. The key fragment acquisition and dynamic encryption key restoration operations are only performed after both authentication and authorization verification are passed.

8. The distributed encrypted storage method based on dynamic key and data fragmentation as described in claim 1, characterized in that, It also includes a data integrity verification step: After performing encryption, the hash value of each encrypted data fragment is calculated, and the hash value is associated with the corresponding fragment identifier and stored in an immutable distributed storage system to form an integrity verification record. Before performing decryption, the hash value of the corresponding encrypted data fragment is obtained from the immutable distributed storage system. The hash value of the current encrypted data fragment is recalculated and compared. If the comparison results match, the decryption operation continues. If they do not match, an automatic error correction or data recovery mechanism is triggered.

9. A distributed encrypted storage method based on dynamic keys and data fragmentation as described in claim 3, characterized in that, It also includes key lifecycle management steps: Set an expiration date for each dynamic encryption key. Before the expiration date, automatically trigger the key update process to generate a new generation of dynamic encryption keys. The corresponding data fragments are re-encrypted using a new generation of dynamic encryption keys, and the old dynamic encryption keys and their corresponding key fragments are destroyed or archived. Key update operation records are written to the tamper-proof audit log system to form an immutable key update audit record.

10. A distributed encrypted storage method based on dynamic keys and data fragmentation as described in claim 1, characterized in that, It also includes anomaly recovery steps: When a storage node in a distributed storage system is detected to be permanently offline or its data is corrupted, redundant copies of the corresponding encrypted data fragments are obtained from other storage nodes to recover the data, based on the redundancy configuration in the preset distributed storage strategy. If the dynamic encryption key cannot be restored due to the loss of key fragments, the system will obtain a sufficient number of key fragments from other storage nodes and restore the dynamic encryption key based on the redundancy characteristics of the threshold secret sharing algorithm for key fragments. This ensures that the system can still decrypt data normally when no more than a preset threshold number of storage nodes fail.

Images