Certificateless signature cross-file block-level update deduplication and hybrid auditing method

Abstract

The application discloses a cross-file block-level update deduplication and hybrid auditing method of a certificateless signature, and comprises the following steps: system initialization, identity key generation, first data file processing, audit generation and verification, subsequent file uploading processing, data block-level dynamic, and identity key update. The application combines a newly designed deduplication and dynamic index table, utilizes a certificateless signature and a message locking encryption strategy, reduces certificate overhead, avoids a key escrow problem, and utilizes a partial encryption strategy in the initial file processing and subsequent file processing stages to realize privacy protection of user data.

Description

Technical Field

[0001] This invention relates to the field of information security technology, and in particular to a method for deduplication and hybrid auditing of cross-file block-level updates without certificate signatures. Background Technology

[0002] Cloud storage, a popular data hosting service favored by businesses and individuals, offers a convenient way for users to continuously upload data to cloud servers and access it on an on-demand basis. Despite the existence of various popular cloud storage services, data integrity remains a serious issue. This is because users typically do not retain copies of their data locally, and data corruption or loss on the cloud server can lead to irreparable damage. Therefore, data integrity auditing has emerged, allowing users or authorized third parties to verify the integrity of data in cloud storage at any time. By analyzing the audit evidence returned by the server, it can be determined whether the data in cloud storage is truly and completely held by the server. Currently, auditing research mainly focuses on the public auditing of single-file storage, covering aspects such as data sharing, privacy protection, and dynamic data management.

[0003] Meanwhile, in order to reduce users' payment for storage space and the storage overhead of cloud servers, deduplication-supporting auditing schemes have been proposed. Most schemes mainly rely on data hash as encryption key, allowing the same data blocks to be encrypted into the same ciphertext to achieve deduplication. Although there are some auditing schemes that achieve deduplication, they have the following drawbacks: (1) The construction of most schemes either relies on traditional PKI or identity-based signatures. Therefore, either the performance of the scheme is limited due to excessive certificate overhead or the application is difficult due to key custody issues. (2) Auditing schemes that partially rely on blockchain to achieve deduplication require all relevant tags to be uploaded to the cloud server and blockchain at the same time. Multiple interactions in the data processing phase lead to high communication overhead. Moreover, the security of the blockchain itself will also pose a threat to the scheme. In addition, auditing schemes based on blockchain to achieve deduplication are almost impossible to support dynamic data operations, because once stored on the blockchain, it will be difficult to modify the data. (3) Current deduplication methods always default to cross-user and cross-file deduplication, and fail to provide a reasonable solution for block deduplication within a single file with duplicate content. These solutions often opt for a simple method of deduplicating all file blocks for all users on a cloud server. This approach fails to meet data isolation requirements and cannot satisfy the maintenance needs of all users for their own data blocks. It also cannot prevent users from repeatedly uploading duplicate data blocks and tags. Furthermore, existing deduplication schemes assume that the initially uploaded single file does not contain duplicate blocks, or they simply remove duplicates, leading to difficulties for users in subsequent access and maintenance. Summary of the Invention

[0004] To address the aforementioned technical issues, this invention proposes a certificate-free, cross-file block-level update deduplication and hybrid auditing method, which achieves functions such as integrity protection, data privacy protection, cross-file data block-level dynamic updates, cross-file block-level deduplication, plaintext-ciphertext hybrid auditing, and user key updates.

[0005] To achieve the above objectives, the technical solution of the present invention is as follows:

[0006] A method for deduplication and hybrid auditing of cross-file block-level updates without certificate signatures includes the following steps:

[0007] System initialization: The key generation center generates the system's public parameters paras and the master private key msk;

[0008] Identity Key Generation: Users interact with the key generation center to obtain their identity private key (dk). U And select secret parameters. As part of the private key, calculate the public key pk U With private key sk U ;

[0009] First data file processing: User's processing of the first file F 1 The processing includes checking for duplicates, dividing into blocks, encryption, calculating block tags, block tags and file tags, and creating a deduplication and dynamic index table. The deduplication and dynamic index table stores the block tag, the file tag of the file containing the data block, and the index set of the data block in the file for each unique data block. After the cloud server checks the validity of the file signature and block tags, it stores the data set and the deduplication and dynamic index table. Only the deduplication and dynamic index table is stored locally.

[0010] Audit generation and verification: A third-party auditor initiates an audit challenge on behalf of the user, sending a random challenge to the cloud server. The cloud server calculates the corresponding integrity audit evidence and returns it to the third-party auditor, who then verifies the audit evidence.

[0011] Subsequent file upload processing: The user uploads subsequent file F. 2 The system divides non-repeating data blocks into blocks, encrypts them, updates the deduplication and dynamic index table, calculates block tags and file tags, and uploads the data set to the cloud server; for duplicate data blocks, it inserts a new sequence number set into the row corresponding to the block tag in the deduplication and dynamic index table, and uploads the set corresponding to the non-repeating blocks and the updated deduplication and dynamic index table to the cloud server.

[0012] Data block-level dynamics: Users process data locally and interact with the cloud server to update block tags, file tags, block tags, deduplication, and dynamic index tables;

[0013] Identity Key Update: Users send their identity identifier and current system timestamp to the key generation center to obtain a new identity key. It also calculates the block label update factor and sends the block label update factor to the cloud server for authenticator update.

[0014] Preferably, the system initialization specifically includes the following steps:

[0015] ① Choose two multiplicative cyclic groups G1 and G2 of order q, and choose a computable bilinear pair e such that e: G1×G1→G2, where q is a large prime number;

[0016] ② Select one generator g from G1, and choose four hash functions, H1: {0,1} * →G1, H2: H3: {0,1} * →{0,1} * H4: {0,1} * →G1, where

[0017] ③ The key generation center selects secret parameters. And calculate the system public key Y = g γ ∈G1;

[0018] ④ The output system's public parameters are paras={G1,G2,e,q,g,H1,H2,H3,H4,Y}, and the master private key msk=γ is secretly held by the key generation center.

[0019] Preferably, the identity key generation specifically includes the following steps:

[0020] ①Assume the user's identity is id U ∈{0,1} * , will id U and the current system timestamp TS1∈{0,1} * Send it to the key generation center to obtain the identity key;

[0021] ②When {id} is received U After TS1, the key generation center calculates the identity key dk. U =H1(id) U ||TS1) γ The key generation center will generate the user identity key dk U Send to users via a private channel;

[0022] ③ Users calculate Verify DK U If the validity of the verification is successful, then U accepts the identity private key dk. UAnd keep it secret; if it fails, the user will resend the request to the key generation center.

[0023] ④ The user randomly selects a secret parameter. As part of the private key, and calculate your own public key. U saves its own private key. And publicly disclose its public key W U .

[0024] Preferably, the processing of the first data file specifically includes the following steps:

[0025] ① The user will send the first file F 1 Divided into N 1 Data blocks Each block has an equal length, i.e., s|q|, where s is the number of partitions in each block, and each partition is |q| bits long; check N 1 If there are duplicate blocks in the block, remove the duplicates and then reorder them to form a block of M. 1 Data block set For Λ 1 Each data block Record it in the original file F 1 The corresponding block sequence number set The corresponding number of repetitions are as follows: Keep in the original file F 1 The data block appearing for the first time in the process is then renumbered. in At the same time M 1 ≤N 1 as well as

[0026] ②The user will put file Λ 1 Divided into two sets For Λ 1 The set of data blocks that can be made public in the middle For Λ 1 The set of data blocks that need to be encrypted in the file Λ 1 Partial encryption is performed to obtain the data block set as follows Where E is the symmetric encryption algorithm AES;

[0027] ③ The user divides the encrypted block into s regions, i.e. in And calculate the block label as and file tags

[0028] ④ The user creates a deduplication and dynamic index table DDIT. DDIT records all unique data blocks corresponding to the original file F. 1 The index position in the data is for each unique data block. Record a corresponding tuple in Represents data block Corresponding to the original file F 1 The set of index numbers in the middle;

[0029] ⑤ The user randomly selects s ​​secret parameters This is used to aggregate the area data corresponding to the block when the block label is generated, and to calculate s public values. Used for subsequent audit evidence verification, followed by calculation of block labels. Get the complete set of block tags

[0030] ⑥ The user will put the data collection {id U ,TS1,C 1 ,Φ 1 The DDIT and DDIT are uploaded to the cloud server together. Only the DDIT is stored locally. When the cloud server uses the DDIT for verification... If the test fails, the cloud server will refuse to store the data and notify the user. If the test succeeds, the consistency between the data block and the tag will be further verified. If the condition is met, the cloud server stores the corresponding data set; otherwise, the cloud server refuses to store the data and notifies the user.

[0031] Preferably, the audit generation and verification specifically includes the following steps:

[0032] ① A third-party auditor generates a random challenge Q = {j} c ,θ j}, where j c ∈J c J c For the set of integers [1, M] 1 Randomly select a set of c integers. From The user selects a random value and sends the random challenge Q to the cloud server.

[0033] ② The cloud server generates audit evidence P = {{μ} k} 1≤k≤s ,σ}, where data evidence Label evidence The cloud server sends audit evidence P to the third-party auditor;

[0034] ③ Third-party auditors verify the integrity of data on the cloud server by checking whether the formula holds true: If the equation holds true, it proves that the data integrity on the cloud server has passed the inspection.

[0035] Preferably, the subsequent file upload process specifically includes the following steps:

[0036] ① The user will then transfer the subsequent file F 2 Divided into N 2 Data blocks Each block has s regions. After removing duplicates, they are reordered to form a block of M regions. 2 Data block set For Λ 2 Each data block in the file F records its data in the original file F. 2 The corresponding block sequence number set Right now in At the same time M 2 ≤N 2 as well as

[0037] ②The user will put file Λ 2 Divided into two sets For Λ 2 The publicly available collection of data blocks in China For Λ 2 The set of data blocks that need to be encrypted in the file Λ 2 Partial encryption is performed to obtain the data block set as follows

[0038] ③ The user divides the encrypted block into s regions, i.e. in And calculate the block label as and file tags

[0039] ④ Users utilize tf 2 as well as Search DDIT

[0040] 1) If a duplicate file tag is detected, it means that the subsequent file is completely duplicated from the user's previous file, and no further processing will be performed;

[0041] 2)tf 2 Non-repeating and for Indicates subsequent file F 2 For any block in the array, j is the block number. If none of the values ​​in DDIT are duplicates, then insert M at the end of the table. 2 line record in

[0042] Representation block Corresponding to the original subsequent file F 2 The set of index numbers in the middle;

[0043] 3)tf 2 No repetition There are partial duplicates, which are handled in two ways: for duplicate blocks, set...

[0044] Then in Add to the corresponding row For non-repeating blocks, insert at the end of the table. Travelogue

[0045] record Indicates the number of unique blocks;

[0046] ⑤ For step ④1), the user does not perform any further processing; for step ④2), the user calculates... The user will set {C 2 ,Φ 2 The updated DDIT is uploaded to the cloud server. After the cloud server verifies the consistency between the data block and the tag, it stores the corresponding data set and updates the DDIT. Otherwise, the cloud server refuses to store the data and notifies the user. For step ④3), the user needs to calculate the authenticator for the non-repeating block and upload it, along with the ciphertext set and the updated DDIT, to the cloud server. After the cloud server verifies the consistency between the data block and the tag, it stores the corresponding data set and updates the DDIT. Otherwise, the cloud server refuses to store the data and notifies the user.

[0047] Preferably, the data block-level dynamics specifically include the following steps:

[0048] ① Data modification: Suppose a user modifies the content of a data block n to... Let χ be the index of n in the set of unique data Λ. If n belongs to a publicly available data block, no processing is performed, which is represented as: If it belongs to a data block that needs to be encrypted, then encrypt it: Here, it is used uniformly to refer to partially encrypted file data blocks. For publicly available data blocks... Directly equivalent to the original data block For data blocks that need to be encrypted With the original data block The relationship is Continue to divide into s regions, that is User calculates old block labels and new block labels The user uses t to obtain the corresponding set of serial numbers X from DDIT.χ And the file tag tf, which will be divided into two cases:

[0049] 1) If |X χ If | = 1, then calculate the file signature update factor. And new file tags use as well as Replace tf and t, and the user calculates the block tag update factor due to changes in block content. Users will dataset Upload them together to the cloud server, and when the cloud server utilizes... verify Correctness: If the request fails, the cloud server will reject the request and notify the user; if the request succeeds, the cloud server will perform the calculation. Simultaneous use Replace 'c' with 'c'. Replace t, use Replace tf;

[0050] 2) If |X χ |≠1, meaning the modified block forms a new block, i.e., |X ν | = 1, υ is the modified value Calculate the file signature update factor based on the corresponding sequence number. And new file tags use Replace tf and add a tuple at the end of the table. and the set X corresponding to t χ Updated to X χ {χ}, User-calculated block label update factor Users will dataset Upload them together to the cloud server, and when the cloud server utilizes... verify Correctness: If the request fails, the cloud server will reject the request and notify the user; if the request succeeds, the cloud server will perform the calculation. Store separately as well as use Replace tf and add a tuple at the end of the table. and the set X corresponding to t χ Updated to X χ \{χ};

[0051] ③ Data block insertion: The block number is appended to the end of the unique data block set. Data block deletion: All block numbers remain unchanged. The operation process is similar to data block update. Fine-grained updates are supported, allowing users to modify the content of each area of ​​each block. The update process is equivalent to data modification (1).

[0052] Preferably, the identity key update specifically includes the following steps:

[0053] User will identify ID U and the current system timestamp Send it to the key generation center to obtain the identity key. When the key generation center receives it... Next, calculate the new identity key. The key generation center will give the new identity key Send it to the user through a private channel, and the user calculates... verify If the verification is successful, the user accepts the identity private key. And keep it secret; if it fails, the user will resend the request to the key generation center.

[0054] User calculation block label update factor And The file is sent to the cloud server, which then updates the block tags of the corresponding user's file. This indicates that the block label update factor is caused by the update of the user identity key.

[0055] Based on the above technical solution, the beneficial effects of the present invention are:

[0056] 1) Data integrity guarantee: By launching integrity challenges to the cloud server during the audit, it is possible to detect whether the cloud server actually holds a copy of the data without holding a copy of the local data block, so as to achieve data integrity protection;

[0057] 2) Plaintext-ciphertext hybrid auditing: Unlike the traditional method of completely encrypting the entire file, the proposed solution allows encryption of some data blocks containing privacy information and combines encryption technology with certificateless signing to achieve plaintext-ciphertext hybrid auditing for multiple files;

[0058] 3) Data privacy protection: Encrypt data blocks containing privacy using a message locking encryption strategy. Without proof of possession of the original data block, no entity other than the group user can deduce the plaintext information.

[0059] 4) Dynamic updates across file data blocks: Through the design of deduplication and dynamic index table, combined with certificateless signing and authenticator generation mechanism, users can perform dynamic operations on multiple file blocks, and the computational overhead of the local end and cloud server is relatively small.

[0060] 5) Local cross-file block-level deduplication: The hash value generated by the content of the block itself is used as the block-level deduplication tag, and the file tag is calculated using the block tag. This enables users to perform block-level deduplication between multiple files on the local end. By recording the index position of unique blocks, users are prevented from uploading duplicate data one after another, reducing the communication between users and the cloud server and the storage overhead of the cloud server. At the same time, it does not affect the user's later maintenance of blocks with the same content but different indexes.

[0061] 6) Supports efficient user key updates: When the identity key expires or the user wants to update it, the cloud server can quickly convert the block tag by sending a block tag update factor to the cloud server. That is, the block signature signed by the previous private key is converted into the block signature corresponding to the new private key. The local computing and communication overhead is a very small constant. Attached Figure Description

[0062] Figure 1 This is a schematic diagram of the interactions between entities in a cross-file block-level update deduplication and hybrid auditing method without certificate signing in one embodiment. Detailed Implementation

[0063] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0064] This embodiment provides a method for deduplication and hybrid auditing of cross-file block-level updates without certificate signing. This method involves four entities, see [link to documentation]. Figure 1 As shown below:

[0065] User (U) needs to outsource multiple data files to a cloud server for storage. During the first file upload phase, a deduplication and dynamic index table needs to be created, and during the update phase, the table needs to be maintained.

[0066] Third-party auditor (TPA): Represents user U in initiating an audit challenge to the cloud server during the audit phase and verifies the returned audit evidence;

[0067] Cloud Server (CS): It has sufficient storage space and computing power, and is responsible for checking and storing uploaded data. It is required to respond to challenges initiated by TPA, and during the data update phase, it is responsible for maintaining deduplication and dynamic index tables according to user requirements, and is responsible for converting the tags of cloud storage during the key update phase.

[0068] Key Generation Center (KGC): During the system initialization phase, it generates public keys and identity keys for users, and generates new identity keys for users when user keys are updated.

[0069] The interaction process between the four entities in this embodiment is as follows: (1) U sends the identity identifier to KGC to obtain the identity key dk U And select secret parameters. As part of the private key, calculate your own public key pk U With private key sk U (2) U for the first data document F 1 The processing includes dividing the data into blocks, partially encrypting, calculating block labels and file labels, creating a deduplication and dynamic index table (DDIT), and uploading the data to the CS. The CS checks the file signature and the legality of the block labels before storing the data. (3) In the subsequent file upload stage, U divides the non-duplicate data blocks into blocks, encrypts them, updates the deduplication and dynamic index table, calculates block labels and file labels, and uploads the data set to the CS. For duplicate data blocks, a new sequence set is inserted into the corresponding row of the block label in the table. Then, the set corresponding to the non-duplicate blocks and the latest deduplication and dynamic index table are uploaded to the cloud server. (4) In the audit interaction stage, TPA initiates an audit challenge on behalf of user U and sends a random challenge Q to the cloud server CS. The CS then calculates the corresponding integrity audit evidence P and returns it to TPA. The TPA verifies the audit evidence P. (5) In the data dynamic stage, U processes the data locally and then interacts with the CS to update the block labels, file labels, block labels, and deduplication and dynamic index table. (6) In the key update stage, U sends the identity identifier and the current system timestamp to the KGC to obtain a new identity key. And calculate the authenticator update factor It is then sent to CS for authenticator update. The process is detailed below:

[0070] (1) System initialization: KGC generates the system public parameter paras and the master private key msk.

[0071] ① Choose two multiplicative cyclic groups G1 and G2 of order q, and select a computable bilinear pair e: G1×G1→G2, where q is a large prime number.

[0072] ② Select one generator g from G1, and choose four hash functions, H1: {0,1} * →G1,H2: H3: {0,1} * →{0,1} * H4: {0,1} * →G1, where

[0073] ③KGC selects secret parameters And calculate the system public key Y = g γ ∈G1;

[0074] ④ The publicly available parameters of the output system are paras={G1,G2,e,q,g,H1,H2,H3,H4,Y}, and msk=γ are secretly held by KGC.

[0075] (2) Identity key generation: User U interacts with KGC to obtain the identity private key and calculates their own private key and public key.

[0076] ①Assume user U's identity is id U ∈{0,1} * , will id U and the current system timestamp TS1∈{0,1} * Send it to KGC to obtain the identity key.

[0077] ②When {id} is received U After TS1, KGC calculates the identity key dk. U =H1(id) U ||TS1) γ KGC will store the user's identity key (dk). U Send it to U via a private channel.

[0078] ③U is calculated Verify DK U If the validity of the verification is successful, then U accepts the identity private key dk. U And it is kept secret. If it fails, U will resend the request to KGC.

[0079] ④U randomly selects secret parameters As part of your private key, and calculate the public key. U saves its own private key. And publicly disclose its public key W U .

[0080] (3) Processing the first data file: User U processes the first data file F 1 Processing (superscript 1 indicates the first data file that user U needs to outsource, followed by subsequent uploaded files F) 2 To make a distinction, for ease of understanding, F 1 The labels involved will be marked with 1 in the upper right corner, which includes processes such as checking for duplicates, dividing blocks, encryption, calculating block tags, block tags and file tags, and uploading to CS.

[0081] ① User U first sends the first file F 1 Divided into N 1Data blocks Each block has an equal length, s|q|, where s is the number of partitions in each block, and each partition is |q| bits long. Then check N. 1 If there are duplicate blocks in the block, remove the duplicates and then reorder them to form a block of M. 1 Data block set For Λ 1 Each data block Record it in the original file F 1 The corresponding block sequence number set The corresponding number of repetitions are as follows: This is how it is preserved in the original file F. 1 The data block appearing for the first time in the process is then renumbered. in At the same time M 1 ≤N 1 as well as

[0082] ②U will transfer the file Λ 1 Divided into two sets For Λ 1 The set of data blocks that can be made public in the middle For Λ 1 The set of data blocks that need to be encrypted. U to file Λ 1 Partial encryption is performed to obtain the data block set as follows E uses the popular symmetric encryption algorithm AES.

[0083] ③U divides the encrypted block into s regions, i.e. in And calculate the block label as and file tags When U downloads any block If the data is meaningful, it is read directly; otherwise, it is decrypted using algorithm E to obtain the plaintext. For unauthorized users, even if they obtain the block... (This block is encrypted.) Because it lacks a hash value calculated from the original plaintext, its original plaintext cannot be obtained, thus protecting data privacy. Compared to existing methods of encrypting the entire file, it also reduces the overhead of encryption and decryption.

[0084] ④U creates a Deduplication and Dynamic Index Table (DDIT). DDIT records all unique data blocks corresponding to the original file F. 1 The index position in the data, note that it is for each unique data block. Record a corresponding tuple in Representation block Corresponding to the original file F 1 A set of index numbers in the middle. For ease of understanding, this is illustrated using an original file F with 8 blocks. 1 As an example (its corresponding file tag is tf) 1 Assuming the first block duplicates the third block, the second block duplicates the fourth and fifth blocks, the sixth block duplicates the seventh block, and the eighth block is unique, the correspondence between blocks before and after processing is given. Note that although the blocks are further divided into regions to reduce the number of data blocks, the subsequent operations in this paper still use the block level as the basic unit of data processing to reduce computational and communication overhead. Furthermore, DDIT is stored locally, allowing users to directly achieve file-level and block-level deduplication using this table, eliminating the need for existing deduplication methods that require uploading file tags and block tags to a cloud server for deduplication. This simplifies user data before uploading, preventing users from uploading duplicate data to the client-side and saving communication overhead. Simultaneously, relying on this table, this paper allows users to dynamically maintain the data, which will be described in subsequent steps. Note that the first column in Table 1 is not part of the DDIT table; it is presented here for clarity. The DDIT content only includes the last two columns.

[0085] Table 1

[0086]

[0087] ⑤U randomly selects s ​​secret parameters And calculate s public values. Then calculate the block label. Get the complete tag set That is, for N 1 The original files of each block are eventually generated into M. 1 Tags. For files with high repetition, M 1 It will be much smaller than N 1 At this stage, the proposed solution significantly improves performance by reducing the number of data blocks and labels, while utilizing the DDIT solution designed in this paper to ensure block-level updates and data deduplication.

[0088] ⑥U will set {id U ,TS1,C 1 ,Φ 1 The DDIT and DDIT are uploaded to the cloud server CS together, and only the DDIT is saved locally. When CS uses the DDIT for verification... If the test fails, CS rejects the storage and notifies U. If it succeeds, the consistency between the data block and the tag is further verified: If true, CS stores the corresponding data set. Otherwise, CS refuses to store and notifies U.

[0089] (4) Audit generation and verification: TPA generates a random challenge Q and sends it to CS. CS generates the corresponding audit evidence P and returns it to TPA. TPA verifies the legality of P to determine the integrity of the data stored on CS.

[0090] ①TPA generates random challenges Q={j c ,θ j}. Where j c ∈J c J c For the set of integers [1, M] 1 Randomly select a set of c integers. From The random value selected from the options. U sends the random challenge Q to CS.

[0091] ②CS generates audit evidence P={{μ k} 1≤k≤s ,σ}, where CS sends evidence P to TPA.

[0092] ③ TPA verifies the integrity of data on CS by checking whether the formula holds true: If the equation is true, output "TRUE", indicating that the data integrity on CS has passed the check; otherwise, output "FALSE". This application supports public auditing (i.e., the audit verification process does not require the user's confidential information).

[0093] (5) Subsequent file upload processing: User U uploads subsequent document F 2 Let's process it. Here, we assume F... 2 For documents that need to be uploaded later, so as to match the first document F 1 To make distinctions.

[0094] ①U first saves the data file F 2 Divided into N 2 Data blocks Each block has s regions. After removing duplicates, they are reordered to form a block of M regions. 2 Data block set For Λ 2 Each data block in the file F records its data in the original file F. 2 The corresponding block sequence number set Right now in At the same time M 2 ≤N 2 as well as

[0095] ②U will transfer the file Λ 2 Divided into two sets For Λ 2 The set of data blocks that can be made public in the middle For Λ 2 The set of data blocks that need to be encrypted. U to file Λ 2 Partial encryption is performed to obtain the data block set as follows

[0096] ③U divides the encrypted block into s regions, i.e. in And calculate the block label as and file tags

[0097] ④U utilizes tf 2 as well as Search DDIT

[0098] 1) If a duplicate file tag is detected, it means that the file is completely duplicated from a previous file of the user, and no further processing will be performed.

[0099] 2)tf 2 Non-repeating and for Indicates subsequent file F 2 For any block in the array, j is the block number. If none of the values ​​in DDIT are duplicates, then insert M at the end of the table. 2 line record in Representation block Corresponding to the original file F 2 The set of index numbers in the middle. For ease of understanding, we will still use an original file F with 8 blocks as an example. 1 As an example, suppose F 2 Contains four data blocks Assuming the first piece is a duplicate of the fourth piece, and the second piece is a duplicate of the third piece. Referring to Table 2, the table would be updated as follows:

[0100] Table 2

[0101]

[0102]

[0103] 3)tf 2 No repetition and but There are partial duplicates, which are handled in two cases: for duplicate blocks (assuming...) Then in Add to the corresponding row Like this In addition to storing the original It also stored For non-repeating blocks, insert at the end of the table. line record ( (This represents the number of unique blocks). For ease of understanding, we will still use an original file with 8 blocks. As an example, suppose subsequent files Assume that only the first and fourth blocks are repeated, and as well as

[0104] Referring to Table 3, the table is then updated as follows:

[0105] Table 3

[0106]

[0107] ⑤ For ④1), U is not processed further; for ④2), U is calculated. U will set {C 2 ,Φ 2 The updated DDIT is uploaded to the CS. After the CS verifies the consistency between the data block and the tag, it stores the corresponding data set and updates the DDIT. Otherwise, the CS refuses to store and notifies U. For ④3), U needs to calculate the authenticator for the non-repeating block and upload it, along with the ciphertext set and the updated DDIT, to the CS. After the CS verifies the consistency between the data block and the tag, it stores the corresponding data set and updates the DDIT. Otherwise, the CS refuses to store and notifies U. Since the tag generation part is highly similar to step (3), it will not be described in detail here.

[0108] (6) Data block level dynamic: U sends a data update request req to CS, CS performs dynamic operations on the specified block and updates DDIT, and at the same time calculates the new file signature. Return to U, U according to and the file signature TF previously stored locally. 1 The process involves checking the data. If the verification passes, DDIT is updated; otherwise, CS is required to respond again. Because the scheme supports block-level operations across files, the update is not limited to the first file, so the "1" is not marked in the upper right corner of this section.

[0109] ① Data modification: Suppose U needs to modify the content of one of its data blocks n to... (Assume that the index of n in the unique set Λ is χ). If n belongs to a publicly accessible data block, no processing is performed, i.e., it is represented as: If it belongs to a data block that needs to be encrypted, then encrypt it: Here, it is used uniformly to refer to partially encrypted file data blocks. For publicly available data blocks... Directly equivalent to the original data block For data blocks that need to be encrypted With the original data block The relationship is Continue to divide into s regions, that is U calculates old block labels and new block labels U uses t to obtain the corresponding set of indices X from DDIT. χ And the file tag tf. Next, we'll divide it into two cases:

[0110] 1) If |X χ If |=1, then calculate the update factor. And new file tags use as well as Replace tf and t. Next, U calculates the block tag update factor due to changes in block content. Finally, U will use the dataset Upload them together to the cloud server CS. When CS uses... verify Correctness: If the condition fails, CS rejects the request and notifies U. If the condition is met, CS calculates... Simultaneous use Replace 'c' with 'c'. Replace t, use Replace tf.

[0111] 2) If |X χ |≠1, meaning the modified block forms a new block, i.e., |X ν |=1(υ is the modified value) (Corresponding sequence number). Calculate the file block tag update factor. And new file tags use Replace tf and add a tuple at the end of the table. and the set X corresponding to t χ Updated to X χ Next, U calculates the block label update factor. Finally, U will use the dataset Upload them together to CS. When CS uses...

[0112] verify Correctness: If the condition fails, CS rejects the request and notifies U. If the condition is met, CS calculates... Store separately as well as use Replace tf and add a tuple at the end of the table. and the set X corresponding to t χ Updated to X χ \{χ}.

[0113] ② The data block insertion process (note that only the block sequence number is added to the end of the unique data block set) and the data block deletion process (all block sequence numbers remain unchanged) are basically the same as the data update operation, so they are ignored here. In addition, the solution in this paper also supports fine-grained updates, that is, users can modify the content of each area of ​​each block. This process is equivalent to case 1) in data modification.

[0114] (7) Identity Key Update: If user U's identity key expires or is exposed, U can update the corresponding identity private key dk with the assistance of KGC. U And calculate the identity private key update factor. And Send it to CS to update the block label.

[0115] Assuming the identifier is id U User U wants to update their identity key. U will send their ID. U and the current system timestamp Send to KGC to obtain the identity key. Upon receiving... Then, KGC calculates the new identity key. KGC will use the new identity key Send it to U via a private channel. U then performs calculations. verify If the verification is successful, then U accepts the identity private key. And it is secretly stored. If it fails, U will resend the request to KGC. U calculates the block tag update factor. And Send to CS, CS updates signature here This refers to the block label update factor caused by the update of the user identity key, which is different from the block label update factor caused by changes in the block content. In other words, the computation and communication overhead on the user's local end during the key update phase is a negligible constant, while the time-consuming block tag conversion operation is performed by the CS. Therefore, the proposed scheme is suitable for performance-constrained local ends.

[0116] The above description is merely a preferred embodiment of the certificate-free, cross-file block-level update deduplication and hybrid auditing method disclosed in this invention, and is not intended to limit the scope of protection of the embodiments in this specification. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the embodiments in this specification should be included within the scope of protection of the embodiments in this specification.

Claims

1. A method for deduplication and hybrid auditing of cross-file block-level updates without certificate signatures, characterized in that: Includes the following steps: System initialization: The key generation center generates the system's public parameters paras and the master private key msk; Identity Key Generation: Users interact with the key generation center to obtain their private identity key. And select secret parameters. As part of the private key, the public key is calculated. With private key ; First data file processing: User's processing of the first file The processing includes checking for duplicates, dividing into blocks, encryption, calculating block tags, block tags and file tags, and creating a deduplication and dynamic index table. The deduplication and dynamic index table stores the block tag, the file tag of the file containing the data block, and the index set of the data block in the file for each unique data block. After the cloud server checks the validity of the file signature and block tags, it stores the data set and the deduplication and dynamic index table. Only the deduplication and dynamic index table is stored locally. Audit generation and verification: A third-party auditor initiates an audit challenge on behalf of the user, sending a random challenge to the cloud server. The cloud server calculates the corresponding integrity audit evidence and returns it to the third-party auditor, who then verifies the audit evidence. Subsequent file upload processing: User uploads subsequent files The system divides non-repeating data blocks into blocks, encrypts them, updates the deduplication and dynamic index table, calculates block tags and file tags, and uploads the data set to the cloud server; for duplicate data blocks, it inserts a new sequence number set into the row corresponding to the block tag in the deduplication and dynamic index table, and uploads the set corresponding to the non-repeating blocks and the updated deduplication and dynamic index table to the cloud server. Data block-level dynamics: Users process data locally and interact with the cloud server to update block tags, file tags, block tags, deduplication, and dynamic index tables; Identity Key Update: Users send their identity identifier and current system timestamp to the key generation center to obtain a new identity key. It calculates the block label update factor and sends the block label update factor to the cloud server for authenticator update.

2. The method for deduplication and hybrid auditing of cross-file block-level updates without certificate signatures as described in claim 1, characterized in that, The system initialization specifically includes the following steps: ① Choose two multiplicative cyclic groups of order q. and And select a computable bilinear pair e that satisfies , where q is a large prime number; ② Select One generator is And select four hash functions, H1: {0,1} * → H2: →{0,1} * H3: {0,1} * → {0,1} * H4: {0,1} * → ,in =[1,2,...,q-1]; ③ The key generation center selects secret parameters. And calculate the system public key. ; ④ The output system's public parameters are paras={ , , e, q, H1, H2, H3, H4, Y}, Master private key msk= It is secretly held by the key generation center.

3. The method for deduplication and hybrid auditing of cross-file block-level updates without certificate signatures as described in claim 1, characterized in that, The identity key generation specifically includes the following steps: ① Assume the user's identity is ,Will and the current system timestamp Send it to the key generation center to obtain the identity key; ② When receiving { , After that, the key generation center calculates the identity key. The key generation center will use the user's identity key. Send to users via a private channel; ③ Users calculate verify The validity of the test, if passed, then Accept identity private key And keep it secret; if it fails, the user will resend the request to the key generation center. ④ The user randomly selects a secret parameter. As part of the private key, and calculate your own public key. , Keep your private key and publish their public key. .

4. The method for deduplication and hybrid auditing of cross-file block-level updates without certificate signatures as described in claim 1, characterized in that, The processing of the first data file specifically includes the following steps: ① The user will submit the first file Divided into Data blocks Each block has the same length. ,in The number of partitions for each block, and the length of each partition is... bits; check If there are duplicate blocks in the block, remove the duplicates and then reorder them to form a block with a number of... Data block set ,for Each data block Record it in the original file The corresponding block sequence number set The corresponding number of repetitions were obtained as follows: ; Keep in the original file The data block appearing for the first time in the process is then renumbered. ,in ,at the same time as well as ; ② The user will put the file Divided into two sets , for The publicly available collection of data blocks in China for The set of data blocks that need to be encrypted in the file, the user's file Partial encryption is performed to obtain the data block set as follows Where E is the symmetric encryption algorithm AES; ③ The user divides the encrypted block into Each district ,in And calculate the block label as and file tags ; ④ The user creates a deduplication and dynamic index table DDIT, which records all unique data blocks corresponding to the original file. The index position in the data is for each unique data block. Record a corresponding tuple ,in Represents data block Corresponding in the original file The set of index numbers in the middle; ⑤ The user randomly selects s ​​secret parameters This is used to aggregate the area data corresponding to the block when the block label is generated, and to calculate s public values. Used for subsequent audit evidence verification, followed by calculation of block labels. This yields the complete set of block tags. ; ⑥ The user will collect the data set Upload the DDIT file along with the cloud server; only the DDIT file is stored locally. The cloud server then uses the DDIT file for verification. If the test fails, the cloud server will refuse to store the data and notify the user. If the test succeeds, the consistency between the data block and the tag will be further verified. If the condition is met, the cloud server stores the corresponding data set; otherwise, the cloud server refuses to store the data and notifies the user.

5. The method for deduplication and hybrid auditing of cross-file block-level updates without certificate signatures as described in claim 1, characterized in that, The audit generation and verification specifically includes the following steps: ① Third-party auditors generate random challenges ,in J c For the set of integers [1, Randomly select a set of c integers. It is from The user selects a random value and sends the random challenge Q to the cloud server. ② Cloud server generates audit evidence Among them, data evidence Label evidence The cloud server sends audit evidence P to the third-party auditor; ③ Third-party auditors verify the integrity of data on the cloud server by checking whether the formula holds true: If the equation holds true, it proves that the data integrity on the cloud server has passed the inspection.

6. The method for deduplication and hybrid auditing of cross-file block-level updates without certificate signatures as described in claim 1, characterized in that, The subsequent file upload process specifically includes the following steps: ① The user will submit subsequent files Divided into Data blocks Each block has s regions. After removing duplicates, they are reordered to form a block of s regions. Data block set ,for Each data block in the file records its data in the original file. The corresponding block sequence number set ,Right now ,in ,at the same time as well as ; ② The user will put the file Divided into two sets , for The publicly available collection of data blocks in China for The set of data blocks that need to be encrypted in the file, the user's file Partial encryption is performed to obtain the data block set as follows ; ③ The user divides the encrypted block into Each district ,in And calculate the block label as and file tags ; ④ User utilization as well as Search DDIT, 1) If a duplicate file tag is detected, it means that the subsequent file is completely duplicated from the user's previous file, and no further processing will be performed; 2) Non-repeating and for , Indicates subsequent file F 2 For any block in the array, j is the block number. If none of the values ​​in DDIT are duplicates, then insert a new value at the end of the table. line record ,in Representation block Corresponding to the original subsequent file The set of index numbers in the middle; 3) No repetition There are partial duplicates, which are handled in two ways: for duplicate blocks, set... = Then in Add to the corresponding row ; For non-repeating blocks, insert at the end of the table. line record , Indicates the number of unique blocks; ⑤ For step ④ 1), the user does not perform any further processing; for step ④ 2), the user calculates... Users will gather The updated DDIT is uploaded to the cloud server. Once the cloud server verifies the consistency between the data block and the tag, it stores the corresponding data set and updates the DDIT. Otherwise, the cloud server refuses to store the data and notifies the user. For step ④ 3), the user needs to calculate the authenticator for the unique block and upload it, along with the ciphertext set and the updated DDIT, to the cloud server. Once the cloud server verifies the consistency between the data block and the tag, it stores the corresponding data set and updates the DDIT. Otherwise, the cloud server refuses to store the data and notifies the user. .

7. The method for deduplication and hybrid auditing of cross-file block-level updates without certificate signatures according to claim 1, characterized in that, The data block-level dynamics specifically include the following steps: ① Data modification: Suppose a user modifies a data block Content modified to ,set up In a set of non-repeating The serial number in is ,if If the data block is publicly available, no processing is performed, i.e., it is represented as: ; If it belongs to a data block that needs to be encrypted, then encrypt it: , Here, it is used uniformly to refer to partially encrypted file data blocks. For publicly available data blocks... Directly equivalent to the original data block For data blocks that need to be encrypted, With the original data block The relationship is Further divided into Each district, namely User calculates old block labels and new block labels Users utilize Obtain the corresponding set of serial numbers from DDIT. and file tags Next, we will divide the situation into two cases: 1) If Then calculate the file signature update factor. And new file tags ,use as well as replace as well as The user calculates the block tag update factor due to changes in block content. The user will put the dataset { , , , , Upload them together to the cloud server, and when the cloud server uses... verify Correctness: If the request fails, the cloud server will reject the request and notify the user; if it succeeds, the cloud server will calculate... Simultaneously use replace ,use replace ,use replace ; 2) If In other words, the modified block forms a new block, i.e. , For the revised Calculate the file signature update factor based on the corresponding sequence number. And new file tags ,use replace And add a tuple { to the end of the table. }, and will corresponding set Updated to User-calculated block label update factor The user will put the dataset { , , , , Upload them together to the cloud server, and when the cloud server uses... verify Correctness: If the request fails, the cloud server will reject the request and notify the user; if it succeeds, the cloud server will calculate... Store separately as well as ,use replace And add a tuple { to the end of the table. }, and will corresponding set Updated to ; ② Data block insertion: The block number is appended to the end of the unique data block set. Data block deletion: All block numbers remain unchanged. The operation process is similar to data block update. It supports fine-grained updates, allowing users to modify the content of each area in every block. The update process is equivalent to data modification (1).

8. The method for deduplication and hybrid auditing of cross-file block-level updates without certificate signatures according to claim 1, characterized in that, The identity key update specifically includes the following steps: User will identify and the current system timestamp Send it to the key generation center to obtain the identity key. When the key generation center receives { , After that, calculate the new identity key. The key generation center will generate new identity keys. Send it to the user through a private channel, and the user calculates... verify If the verification is successful, the user accepts the identity private key. And keep it secret; if it fails, the user will resend the request to the key generation center. User calculation block label update factor and will The file is sent to the cloud server, which then updates the block tags of the corresponding user's file. , This indicates that the block label update factor is caused by the update of the user identity key.

Images