An encryption deduplication method supporting file key continuous update

Abstract

The application discloses a random key encryption deduplication method supporting continuous updating. In order to enable a client to safely outsource private data to a server, the method comprises the following steps: after system parameters are initialized, the client selects a random key to encrypt data and sends the encrypted data to the server. The server checks and deduplicates the data of the user. The server verifies the ownership of the file of each uploader through a challenge-response mechanism to realize effective ownership management control. The application supports the replacement of the file key of the user and can guarantee that all legal users obtain the key updated for multiple times, thereby reducing the risk of private data leakage after the key leakage. Due to the randomness of the key selected by the client, the application can resist the brute force attack introduced by the deterministic convergent encryption. Compared with the encryption deduplication method assisted by a third-party independent server, the method is simpler in deployment. The application can provide data privacy in the field of encryption deduplication.

Description

Technical Field

[0001] This invention relates to the field of encrypted deduplication, and in particular to an encrypted deduplication method that supports continuous updating of file keys. Background Technology

[0002] The rapid development of internet technology has led to increasingly widespread internet access and a surge in per capita data volume. Statista predicts that global data volume will reach 2142 Zb by 2035, posing a significant challenge to data storage. Individuals and SMEs, facing limited storage and maintenance capabilities, are increasingly opting for cloud storage. To reduce costs, cloud service providers employ deduplication technology. This technology detects and deletes identical files from outsourced data from different users, retaining only a single encrypted backup, thereby reducing storage space usage, improving storage resource utilization, and providing users with a more economical and efficient data storage solution. To prevent privacy breaches, users encrypt data before outsourcing. However, traditional encryption technologies often result in different users encrypting the same file into different ciphertexts, increasing the difficulty of deduplication. To address this, Douceur et al. [Douceur JR, Adya A, Boloky WJ, et al. Reclaiming space from duplicate files in a serverless distributed filesystem[C]. Proceedings 22nd International conference on distributed computing systems. 2002. 617-624.] proposed the Convergent Encryption (CE) algorithm, which uses file fingerprints as encryption keys. Different users can encrypt the same file using CE to obtain the same ciphertext. However, the deterministic encryption nature of the CE method makes it vulnerable to offline brute-force attacks. Although cloud storage servers can detect duplicate data in ciphertext, security vulnerabilities remain. This has prompted continuous exploration of more secure and efficient encryption deduplication methods to meet the requirements of data storage security and resource utilization efficiency.

[0003] In 2013, Bellare et al. [Keelveedhi S, Bellare M, Ristenpart T. DupLESS: server-aided encryption for deduplicated storage[C]. Proceedings of the 22nd USENIX conference on Security. 2013. 179-194.] proposed the DupLESS algorithm. Clients obtain plaintext-based keys by interacting with a key server via an oblivious PRF protocol, thus resisting offline brute-force attacks that may exist in CE schemes. Simultaneously, DupLESS resists online brute-force attacks by limiting the frequency of user access. In this way, clients can use existing services to store encrypted data, while the server performs deduplication, achieving strong confidentiality guarantees. For deduplicated storage encryption, DupLESS achieves nearly the same performance and space savings as using plaintext data storage services.

[0004] In 2015, Liu Jian et al. [Liu J, Asokan N, Pinkas B. Secure deduplication of encrypted data without additional independent servers [C]. Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security. 2015. 874-885.] proposed a cross-user encrypted data deduplication scheme. This scheme does not require an additional independent server; the data owner of the previously uploaded file needs to assist the subsequent uploader in encrypting the data. Therefore, there is no need to consider the single point of failure problem of the key server.

[0005] In 2017, Jay Dave et al. [Dave J, Saharan S, Faruki P, et al. Secure random encryption for deduplicated storage[C] Information Systems Security: 13th International Conference. 2017. 164-176.] pointed out that deterministic key-based encryption mechanisms (such as CE and MLE) are vulnerable to brute-force attacks and dictionary attacks against predictable files, and proposed a secure encryption method for deduplicated storage based on a random key. To allow deduplication, the random key is encrypted using a set of hash values ​​calculated from the plaintext file. Therefore, subsequent uploaders of the file can also decrypt the random key. This method provides both file confidentiality and key security without requiring any additional key server.

[0006] In 2020, Tian et al. [Tian G, Ma H, Xie Y, et al. Randomized deduplication with ownership management and data sharing in cloud storage[J]. Journal of Information Security and Applications, 2020, 51: 102432.] proposed a client-side deduplication protocol that provides more secure ownership authentication while resisting brute-force attacks and deduplication forgery attacks. Furthermore, they proposed a dynamic KEK tree to achieve more efficient ownership management in deduplication systems and designed a novel data sharing technology based on the dynamic KEK tree, enriching cloud services and reducing the business pressure on cloud servers.

[0007] In 2024, Wu et al. [Wu X, Li J, Wang H, et al. A randomized encryption deduplication method against frequency attack[J]. Journal of Information Security and Applications, 2024, 83: 103774.] proposed a randomized encryption deduplication method against frequency attacks (REFA), which provides high randomness in the ciphertext. The core idea of ​​REFA is that after obtaining a convergence key generated by a key server, the user randomizes the convergence key using a random value of the same length. Then, REFA encrypts the plaintext block with the random key. REFA hides the random value in the ciphertext and encrypts the convergence key for user ownership verification. REFA perfectly masks the frequency of the plaintext block, achieving high storage efficiency and data security. Subsequent ciphertext uploads by the user are updated by the CSP. REFA can effectively defend against frequency analysis attacks.

[0008] Furthermore, Chinese patent application CN108400970A discloses a method for detecting and deleting similar data messages in a cloud environment. This method employs a similarity-preserving hash function, a key extraction method based on error-correcting codes, and a secure symmetric encryption algorithm based on a pseudo-random generator, further improving the deduplication efficiency of existing schemes. Additionally, this method utilizes Hamming distance reduction and label segmentation optimization methods to improve the efficiency of label queries on cloud servers.

[0009] Chinese patent application CN118748589A discloses a multifunctional cloud-based collaborative data auditing method. This method integrates cryptographic technologies such as encrypted deduplication, remote data integrity auditing and protocol RDC, and heterogeneous ring signatures, achieving efficient, secure, privacy-preserving, and flexible device authentication and data integrity verification.

[0010] Chinese patent application CN115225409A discloses a cloud data security deduplication method based on multi-backup joint verification. This method generates a public and private key pair based on the user's identity and the selected number of backups; it hides the encryption key within tags in the data to obtain the ciphertext block's tag; and simultaneously, it verifies the data's integrity before decryption. Summary of the Invention

[0011] The purpose of this invention is to address the aforementioned shortcomings by providing an encrypted deduplication method that supports continuous file key updates, efficiently achieving data encryption and deduplication, and providing users with outsourced storage services for data confidentiality and integrity. This invention uses a random key to encrypt files, which resists both offline and online brute-force attacks and improves key generation efficiency. Furthermore, this invention supports continuous updates of the file key / ciphertext, enabling effective access control.

[0012] This invention is achieved through the following scheme:

[0013] An encrypted deduplication method that supports continuous updating of file keys includes the following steps:

[0014] Step 1: Set system parameters;

[0015] Step 2: The client encrypts file F;

[0016] Step 3: File upload deduplication;

[0017] Step 4: Ciphertext Update;

[0018] If the key or ciphertext of file F is leaked, the client can choose to re-encrypt the file with a different key.

[0019] Step 5: The client downloads file F;

[0020] Step 501: The client sends T to the server;

[0021] Step 502: The server queries whether the client is in T's ownership list;

[0022] Step 503: After receiving the ciphertext, the client first calculates the file key key = H(e(g) r Then decrypt the ciphertext F = D(C, key);

[0023] Step 504: After receiving the ciphertext, the client first calculates... File key Then decrypt the ciphertext F = D(C, key) to recover file F; then calculate T. * =H(F), compare T * =T indicates whether the condition is true. If it is true, the file is complete; otherwise, it is discarded.

[0024] In step 2, specifically step 201: the client calculates the file label T = H(F).

[0025] In step 1, specifically, It is a bilinear mapping; H:{0,1} * →Z p; The output is a fixed-length hash function; H1: {0, 1} * → G1, mapping an arbitrarily long input to an element in group G1; H2: G1 → Z p ; p is a large prime number, the server's public key y = g x , the server's private key x (1 < x < p - 1); The client i 's private key a i (1 < a i < p - 1, 1 < i < s), s is the number of clients, and the public key is Public system parameters; G1: A multiplicative cyclic group of order p with the generator g; G2: Also a multiplicative cyclic group of order p.

[0026] In step 3, it specifically includes:

[0027] Step 301: The client sends T to the server;

[0028] Step 302: The server checks whether it has already stored T;

[0029] Step 303: After receiving the tag sent by the server, the client randomly selects r ∈ Z*, and calculates the file key Key = H1(e(g r , y)); The client encrypts the file F with the file key: C = E(Key, F);

[0030] Step 304: The client randomly selects an n-dimensional vector V = [r1, r2, r3,..., r n , divides the file F into n chunks {f1, f2, f3,..., f n}, and calculates h i = H(r i ||f i ); The client uses {h i}1≤i≤n as leaf nodes to calculate the Merkle hash tree root h root , and calculates n is the number of data block shards; || represents string concatenation;

[0031] Step 305: The client sends the ciphertext C, the file tag T f , Passkey, and the vector V to the server, and keeps g r and T locally;

[0032] Step 306: After receiving the Passkey and the vector V sent by the server, the client divides the file F into n chunks {f1, f2, f3,..., f n}, calculates h i = H(r i ||fi ); The client uses {h i}1≤i≤n are used as leaf nodes to calculate the Merkle hash tree root h, and then the result is calculated.

[0033]

[0034] The client calculates C = E(F, key); finally, the ciphertext C is divided into n equal-sized blocks {C i}1≤i≤n, calculate {hc i =H(C i ||u i As leaf nodes, construct a Merkle hash tree to obtain the root hc. root The client sends hc root Give it to the server and save it (g) r And T is local;

[0035] Step 307: The server receives hc root Construct a Merkle hash tree based on the ciphertext C and vector U that you have saved, and then connect the resulting root to hc. root Compare and judge.

[0036] In step 302: If there is no storage, the server sends tag=0 to the client; then proceeds to step 303;

[0037] If it has already been stored, the server randomly selects an n-dimensional vector U = [u1, u2, ..., u...]. n The Passkey containing part of the file key, vectors V and U, are sent to the client; then proceed to step 306.

[0038] Step 307: (i) If they are the same, add the client to the owner list of file F; (ii) If they are not the same, reject the client's request.

[0039] In step 4, specifically step 401: The client randomly selects r1. ′ ∈G1, calculate Then calculate r1 = H2(r1 ′ ), new file key Use the new key new Encrypted files

[0040] C new =E(key) new ,F)

[0041] Then the new ciphertext and token are uploaded to the server;

[0042] Step 402: After receiving the ciphertext and token, the server first saves the ciphertext C.new Then calculate

[0043] r1=H2(r1 ′ ).

[0044] In step 503, if the client has not updated the ciphertext after joining the ownership list, the server sends the ciphertext C to the client; then step 503 continues. If the client has updated the key k times after joining the ownership list, and the random value chosen by the user for each update is r... i ′ The server will encrypt C and update Send to the client; continue to step 504.

[0045] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:

[0046] 1. The encryption and deduplication method of this invention can complete data encryption and deduplication without the assistance of a third-party independent server, and can effectively resist offline / online brute-force attacks and duplicate forgery attacks. Secondly, when a user revokes their encryption key after it has been leaked, this invention supports the data owner in updating the encryption key and re-encrypting the file. Meanwhile, previously legitimate users can still download and decrypt the ciphertext. Attached Figure Description

[0047] Figure 1 This is a flowchart illustrating the encryption process of a specific embodiment of the present invention;

[0048] Figure 2 This is a construction diagram of the Merkle hash tree. Detailed Implementation

[0049] All features disclosed in this specification, or all steps in all disclosed methods or processes, may be combined in any way, except for mutually exclusive features and / or steps.

[0050] Any feature disclosed in this specification (including any appended claims and abstract) may be replaced by other equivalent or similar features, unless specifically stated otherwise. That is, unless specifically stated otherwise, each feature is merely one example of a series of equivalent or similar features.

[0051] In the description of this invention, it should be understood that the terms "upper", "lower", "left", "right", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0052] In addition, terms such as "first", "second", etc. are only used for descriptive purposes and should not be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, features defined with "first", "second", etc. may explicitly or implicitly include one or more such features.

[0053] Example 1

[0054] As Figures 1-2 shown, the present invention provides a technical solution:

[0055] An encryption deduplication method supporting continuous update of file keys, comprising the following steps:

[0056] Step 1: Set system parameters;

[0057] λ is a security parameter, n is the number of data block shards; || represents string concatenation hereinafter; G1: a multiplicative cyclic group of order p with generator g; G2: also a multiplicative cyclic group of order p;

[0058] is a bilinear mapping; H: {0,1} * →Z p ; an output fixed-length hash function; H1: {0,1} * →G1, mapping an arbitrary-length input to an element in group G1; H2: G1→Z p ; p is a large prime number, the public key of the server y = g x , the private key of the server x (1 < x < p - 1); the private key a of the client Client i (1 < a i (1 < a i < p - 1, 1 < i < s), s is the number of clients, and the public key is Disclose system parameters;

[0059] Step 2: The client performs encryption processing on the file F;

[0060] Step 201: The client calculates the file tag T = H(F);

[0061] Step 3: File upload deduplication processing;

[0062] Step 301: The client transmits T to the server;

[0063] Step 302: The server checks whether T has been stored;

[0064] If not stored, the server sends tag = 0 to the client; then proceeds to Step 303;

[0065] If it has already been stored, the server randomly selects an n-dimensional vector U = [u1, u2, ..., u...]. n The Passkey containing part of the file key, vectors V and U, are sent to the client; then proceed to step 306.

[0066] Step 303: After receiving the tag sent by the server, the client randomly selects r∈Z* and calculates the file key Key=H1(e(g) r The client uses a file key to encrypt file F: C = E(Key, F);

[0067] Step 304: The client randomly selects an n-dimensional vector V = [r1, r2, r3, ..., r n Divide the file F into n blocks {f1, f2, f3, ..., fn}. n}, calculate h i =H(r) i ||f i ); The client uses {h i}1≤i≤n are used as leaf nodes to compute the Merkle hash tree root h root and calculate

[0068] Step 305: The client sends the ciphertext C and the file tag T. f The Passkey and vector V are sent to the server together, and g is retained. r And T is local;

[0069] Step 306: After receiving the Passkey and vector V from the server, the client divides the file F into n blocks {f1, f2, f3, ..., f...}. n}, calculate h i =H(r) i ||f i ); The client uses {h i}1≤i≤n are used as leaf nodes to calculate the Merkle hash tree root h, and then the result is calculated.

[0070]

[0071] The client calculates C = E(F, key); finally, the ciphertext C is divided into n equal-sized blocks {C i}1≤i≤n, calculate {hc i =H(C i ||u i As leaf nodes, construct a Merkle hash tree to obtain the root hc. root The client sends hc root Give it to the server and save it (g) r And T is local;

[0072] Step 307: The server receives hc root Construct a Merkle hash tree based on the ciphertext C and vector U that you have saved, and then connect the resulting root to hc. root The comparison is performed. (i) If they are the same, the client is added to the owner list of file F; (ii) if they are different, the client's request is rejected.

[0073] Step 4: Ciphertext Update;

[0074] If the key or ciphertext of file F is leaked, the client can choose to re-encrypt the file with a different key.

[0075] Step 401: The client randomly selects r1 ′ ∈G1, calculate Then calculate r1 = H2(r1 ′ ), new file key Use the new key new Encrypted files

[0076] C new =E(key) new ,F)

[0077] Then the new ciphertext and token are uploaded to the server;

[0078] Step 402: After receiving the ciphertext and token, the server first saves the ciphertext C. new Then calculate

[0079] r1=H2(r1 ′ );

[0080] Step 5: The client downloads file F;

[0081] Step 501: The client sends T to the server;

[0082] Step 502: The server checks if the client is in the ownership list of T; if the client has not updated the ciphertext after being added to the ownership list; the server sends the ciphertext C to the client; proceed to step 503; if the client has updated the key k times after being added to the ownership list, and the random value chosen by the user for each update is r. i ′ The server will encrypt C and update Send to the client. Proceed to step 504;

[0083] Step 503: After receiving the ciphertext, the client first calculates the file key key = H(e(g) r Then decrypt the ciphertext F = D(C, key);

[0084] Step 504: After receiving the ciphertext, the client first calculates... File key Then decrypt the ciphertext F = D(C, key) to recover file F; then calculate T. * =H(F), compare T * =T indicates whether the condition is true. If it is true, the file is complete; otherwise, it is discarded.

[0085] This invention discloses a continuously updated random key encryption and deduplication method. To enable clients to securely outsource their private data to the server, the method includes initializing system parameters, allowing the client to select a random key to encrypt the data before sending it to the server. The server performs deduplication checks on the user's data. The server verifies the ownership of the file by each uploader through a challenge-response mechanism, achieving effective ownership management and control. This invention supports users changing file keys and ensures that all legitimate users obtain multiple updated keys, reducing the risk of privacy data leakage due to key leaks. Due to the randomness of the client's key selection, this invention can resist brute-force attacks introduced by deterministic convergence encryption. Compared to encryption and deduplication methods assisted by a third-party independent server, this method is simpler to deploy. This invention provides data privacy, integrity, and key update services in the field of encryption and deduplication.

[0086] Example 2

[0087] like Figure 1 As shown, the present invention provides a technical solution:

[0088] An encrypted deduplication method that supports continuous updating of file keys includes the following steps:

[0089] Configure system parameters to generate client and server private keys S ID PK with public key ID And the system parameters required for interaction between the client and the server;

[0090] During the file upload phase: The first client to upload a file generates a tag, an encryption key, a vector V for key transmission, and a Passkey for the transmission key. It then sends the file tag, ciphertext, vector V, and Passkey to the storage server, which stores them. Subsequent clients uploading files use the file tag to download vector V, Passkey, and a challenge vector U proving the user's ownership of the file from the storage server. They then extract the encryption key from these to encrypt the file. Finally, they generate a response to the ownership challenge using the ciphertext and vector U. After the server verifies and approves the response, ownership of file F is granted to the user.

[0091] During the key update phase, the client can update the key while ensuring that previous clients can still correctly decrypt the plaintext.

[0092] During the file download phase, the client uses file tags to download the encrypted text from the server and then encrypts it using the saved key.

[0093] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An encrypted deduplication method supporting continuous updating of file keys, characterized in that, Includes the following steps: Step 1: Set system parameters; In step 1, specifically, It is a bilinear mapping; ; The output is a fixed-length hash function; , map any length of input to elements in group G1; p is a large prime number, and the server's public key. The server's private key ; Client private key s is the number of clients, and the public key is Publicly disclose system parameters; G1: A p-order multiplicative loop with generator g; G2: Also a p-order multiplicative loop; Step 2: The client encrypts file F; In step 2, specifically step 201: The client calculates the file tag T=H( ; Step 3: File upload deduplication; Step 4: Ciphertext Update; If the key or ciphertext of file F is leaked, the client can choose to re-encrypt the file with a different key. In step 4, specifically step 401: The client randomly selects... ,calculate Then calculate New file key and using a new key Encrypted files Then the new ciphertext and token are uploaded to the server; Step 402: After receiving the ciphertext and token, the server first saves the ciphertext. Then calculate ; Step 5: The client downloads file F; Step 501: The client sends T to the server; Step 502: The server queries whether the client is in T's ownership list; Step 503: After receiving the ciphertext, the client first calculates the file key. Then decrypt the ciphertext. ; In step 503, if the client does not update the ciphertext after adding it to the ownership list, the server sends ciphertext C to the client and continues with step 503. If the client has updated the key k times since adding it to the ownership list, and the random value chosen by the user for each update is... The server will encrypt C and update Send to the client; continue to step 504; Step 504: After receiving the ciphertext, the client first calculates... File key Then decrypt the ciphertext. Recover file F; then calculate ,Compare Check if the statement is true; if so, the file is complete; otherwise, discard it.

2. The encryption deduplication method supporting continuous file key updates as described in claim 1, characterized in that: Step 3 specifically includes: Step 301: The client sends T to the server; Step 302: The server checks whether T has already been stored; Step 303: After receiving the tag sent by the server, the client randomly selects r∈Z* and calculates the file key. The client uses a file key to encrypt file F: C=E(Key,F); Step 304: The client randomly selects an n-dimensional vector V=[r1,r2,r3,…,r n Divide file F into n blocks {f1, f2, f3, ..., fn}. n }, calculate h i =H(r i ||f i ); The client uses {h i }1≤i≤n are used as leaf nodes to compute the Merkle hash tree root h root and calculate n is the number of data block fragments; || represents string concatenation; Step 305: The client sends the ciphertext C, file tag T, passkey, and vector V to the server, while keeping g as a record. r And T is local; Step 306: After receiving the Passkey and vector V from the server, the client divides the file F into n blocks {f1, f2, f3, ..., f...} n }, calculate h i =H(r i ||f i ); The client uses {h i }1≤i≤n are used as leaf nodes to calculate the Merkle hash tree root h, and then the result is calculated. The client calculates C=E(F,key); finally, the ciphertext C is divided into n equal-sized blocks {C i }1≤i≤n, calculate {hc i =H(C i ||u i As leaf nodes, construct a Merkle hash tree to obtain the root hc. root The client sends hc root Give it to the server and save it (g) r And T is local; Step 307: The server receives hc root Construct a Merkle hash tree based on the ciphertext C and vector U that you have saved, and then connect the resulting root to hc. root Compare and judge.

3. The encryption deduplication method supporting continuous file key updates as described in claim 2, characterized in that: In step 302: If there is no storage, the server sends tag=0 to the client; then proceeds to step 303; If it has already been stored, the server randomly selects an n-dimensional vector U=[u1,u2,…,u…]. n The Passkey containing part of the file key, vectors V and U, are sent to the client; then proceed to step 306.

4. The encryption deduplication method supporting continuous file key updates as described in claim 3, characterized in that: Step 307: (i) If they are the same, add the client to the owner list of file F; (ii) If they are not the same, reject the client's request.

Images