A label clustering based secure deduplication method

Abstract

The application relates to a label clustering-based security deduplication method, relates to the technical field of deep learning and information security, and provides a label clustering-based security deduplication method, which introduces a label clustering method on the basis of a deduplication method based on a self-encoding model, realizes rapid screening of fingerprint labels, and further improves deduplication efficiency. A user uploads data fingerprint labels and summary labels to a CSP, issues a deduplication application, and decides whether to upload encrypted data and labels according to a deduplication detection result; the CSP periodically clusters the summary labels, and identifies a category where the summary labels are located to carry out repeated data retrieval and deletion; the method can start comparison of the fingerprint labels after returning certain labels, and can omit distance calculation and sorting steps.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of deep learning and information security, and particularly relates to a secure deduplication method based on label clustering, which is used for reducing the calculation overhead of encrypted data deduplication in cloud storage. BACKGROUND

[0002] Due to the advantages of low cost, massive storage, scalability, convenient data access and the like, more and more individuals, institutions and organizations choose to store data on cloud servers. However, different people or organizations may store the same data on the cloud server, resulting in repeated storage and waste of storage resources, which requires deleting the repeated data. However, in order to protect their own data from being leaked, users usually store the data in the cloud server after encrypting the data. Different encryption methods and different encryption keys make the same data produce different ciphertexts, so that the cloud server cannot effectively deduplicate the encrypted data.

[0003] In view of this, a method for deleting repeated data for encrypted data is proposed. The user first needs to extract the hash value of the data, and encrypt the hash value as a fingerprint label for deduplication. In order to ensure the confidentiality of the fingerprint label, some scholars have proposed a secure data deduplication method based on random message lock encryption (R-MLE). This method encrypts the hash value into a fingerprint label that is difficult to be brute-forced based on the elliptic curve encryption method. However, the comparison of the label is based on the bilinear mapping theory, and the calculation amount is much larger than that of the traditional encryption.

[0004] Therefore, some scholars have proposed to introduce a self-encoding model into the R-MLE method. This method requires the user to train a self-encoding model independently, and generates a summary label for the data through the model. The summary label can be used to calculate the similarity between the data. When deduplicating, the cloud service provider (CSP) calculates the similarity between the summary label of the user and the label in the label library, selects the most similar summary label corresponding to the fingerprint label, and then performs repeated data comparison. Through the filtering of the summary label, the number of fingerprint labels that need to be compared is significantly reduced. SUMMARY

[0005] The present application provides a secure deduplication method based on label clustering. The label clustering method is introduced on the basis of the deduplication method based on the self-encoding model, and the rapid screening of the fingerprint label is realized, so as to further improve the deduplication efficiency.

[0006] A security deduplication method based on label clustering, a user uploads a data fingerprint label and an abstract label to a CSP, issues a deduplication application, and decides whether to upload encrypted data and labels according to a deduplication detection result; the CSP periodically clusters the abstract labels, and identifies a category in which the abstract labels are located to perform repeated data retrieval and deletion; a user-trained auto-encoding model is set; and a specific process of the method is as follows:

[0007] Step one, a user generates a hash value H(m1) for data m1, selects a random number a, and constructs a fingerprint label of the data

[0008] Step two, an abstract label AT(m1) of the data is generated by using an auto-encoding model;

[0009] Step three, the user uploads AT(m1) and to the CSP;

[0010] Step four, the CSP traverses each cluster center in a c_centroid table, calculates a distance from the cluster center to AT(m1), and selects a nearest category i;

[0011] Step five, all labels under the category i are queried in a c_tag table according to an id of the category i to form a label set;

[0012] Step six, the label set in step five is traversed, a distance between each label and AT(m1) is calculated, and the distance is stored in a binary sorting tree;

[0013] Step seven, a new label set is formed by selecting a minimum of n labels in the binary sorting tree in ascending order of distance;

[0014] Step eight, the new label set is traversed, and each fingerprint label b is selected as a random number by another user, and and are calculated, respectively. If the two are equal, it indicates that m1=m2, that is, the user uploads repeated data, the CSP informs the user that the data does not need to be uploaded, and the process is ended; otherwise, step nine is performed.

[0015] Step nine, the CSP informs the user to upload encrypted data, stores AT(m1) and in the c_tag table, and marks the category of AT(m1) and as i;

[0016] Step ten, the CSP calculates a variance of the number of labels, if the variance is greater than a threshold value, the CSP starts abstract label clustering, otherwise, the process is ended.

[0017] The method has the following beneficial effects:

[0018] In the method, after returning a certain type of label, the comparison of the fingerprint label can be started, and therefore, the method can omit the steps of distance calculation and sorting in terms of filtering of the summary label. In this case, the time complexity of the label filtering is O(m), where m is the number of cluster centers. In the security deduplication method based on the auto-encoding model, the time complexity of the summary label filtering is O(nlogn), where n is the total number of summary labels. From the perspective of time complexity, the linear time complexity of the present application is better than the linear-logarithmic time complexity of the security deduplication method based on the auto-encoding model. Meanwhile, the number of clusters can be set as needed, and is theoretically irrelevant to the number of summary labels, which is a constant much smaller than n.

[0019] The efficiency of the method in label filtering is much higher than that of the security deduplication method based on the auto-encoding model. Assuming that the number of each type of label is consistent and is n / m, and the filtering ratio of the security deduplication method based on the auto-encoding model is 1 / m, the time complexity of the subsequent fingerprint label comparison of the two methods is Since the fingerprint label comparison is the most computationally intensive in the entire deduplication process, the execution efficiency of the two methods is actually not much different. Considering that the time complexity is in the worst case, when the user uploads the data, the sorted label set is more likely to be retrieved than the label set without sorting.

[0020] After filtering out a type of summary label, the method sorts them according to the distance calculation, and then filters out a smaller range of summary labels according to a certain ratio. Assuming that the ratio is still 1 / m, the time complexity of the fingerprint label comparison can be further reduced to Even without further narrowing the range, the efficiency of the sorted fingerprint label comparison is comparable to that of the security deduplication method based on the auto-encoding model. At this time, the total time complexity of the summary label filtering and sorting is which is still better than the O(nlogn) of the security deduplication method based on the auto-encoding model. BRIEF DESCRIPTION OF DRAWINGS

[0021] Figure 1 a timing diagram for the security deduplication based on the auto-encoding model;

[0022] Figure 2 a timing diagram for the security deduplication method based on label clustering according to the present application. DETAILED DESCRIPTION

[0023] Combining Figure 1 and Figure 2To illustrate the present embodiment, a security deduplication method based on label clustering is described. The method periodically clusters the digest labels in the CSP using a k-means clustering algorithm and stores the cluster centers. For this purpose, a special data structure is designed to store the digest labels and their cluster centers, i.e., a data table c_centroid for storing the cluster centers and a data table c_tag for storing the labels (digest labels and fingerprint labels).

[0024] The c_centroid table contains two fields, namely an identifier id and a coordinate of the cluster center. The coordinate field connects the coordinate values of each dimension into a string in scientific notation. The c_tag table contains four fields, namely an identifier id, a fingerprint label fp, a digest label as_t, and a cluster identifier centroid_id. The centroid_id is used to associate the c_centroid table and indicates which category the label belongs to.

[0025] In the present embodiment, two entities are included: a cloud service provider (CSP) and a user.

[0026] CSP: provides data storage services, has powerful computing capabilities and huge storage resources, periodically clusters the digest labels, and quickly identifies the category of the uploaded digest labels by the user to perform deduplication detection and deletion.

[0027] User: uploads the data fingerprint label and the digest label to the cloud server, issues a deduplication application, and decides whether to upload the encrypted data and the label according to the deduplication detection result.

[0028] The data uploading process is as follows. It is assumed that the user has trained a self-encoding model.

[0029] Step 1: The user generates a hash value H(m1) for the data m1, selects a random number a, and constructs the fingerprint label of the data

[0030] Step 2: The self-encoding model is used to generate the digest label AT(m1) of the data.

[0031] Step 3: The user uploads AT(m1) and to the CSP.

[0032] Step 4: The CSP traverses each cluster center in the c_centroid table, calculates the distance between the cluster center and AT(m1), and selects the closest category i.

[0033] Step 5: According to the id of the category i, all labels in the category are queried in the c_tag table to form a label set.

[0034] Step 6: Traverse the label set of the last step, calculate the distance of each label to AT(m1) and store it in a binary sort tree.

[0035] Step 7: Select the smallest n labels in the sort tree according to the distance from small to large to form a new label set.

[0036] Step 8: Traverse the new label set, take out each fingerprint label Where b is a random number selected by other users, and then calculate And If they are equal, it means that m1=m2, that is, the user uploads duplicate data, and the CSP tells the user not to upload data, and the process ends. Otherwise, step 9 is executed.

[0037] Step 9: The CSP informs the user to upload encrypted data, and then stores AT(m1) and In the c_tag table, and marks its classification as i.

[0038] Step 10: The CSP calculates the variance of the number of labels, and if the variance is greater than a certain threshold, the CSP starts the summary label clustering process.

[0039] In this embodiment, the summary label clustering process of the CSP is as follows:

[0040] Step 1: Initialize variables K=0, V=+∞

[0041] Step 2: Select an initial cluster number k=5.

[0042] Step 3: Use the k-means clustering algorithm to cluster all summary labels, and calculate the number of labels in each class.

[0043] Step 4: Calculate the variance v of the number of labels, if v

[0044] Step 5: If k<=50, then k=k+1, continue step 3, otherwise end.

[0045] Step 6, according to the value of K, apply the k-means clustering algorithm to cluster all summary labels again, store the cluster centers in the c_centroid table, and modify the classification of each label in the c_tag table.

[0046] As shown in Figure 1 , the interaction process between each entity of the secure deduplication method based on the auto-encoding model. The interaction between each entity in this embodiment is shown in Figure 2 . Compared with Figure 1 and Figure 2It can be seen that the tag library has an additional clustering process, and the process of directly obtaining certain type of summary tags through clustering centers between CSP and the tag library.

[0047] The embodiment can start the comparison of the fingerprint tags after returning certain type of tags, so the present application can omit the steps of distance calculation and sorting in terms of the filtering of summary tags. In this case, the time complexity of the tag filtering is O(m), where m is the number of clustering centers. In the security deduplication method based on the self-encoding model, the time complexity of the summary tag filtering is O(nlogn), where n is the total number of summary tags. From the perspective of time complexity, the present application is linear time complexity, and the security deduplication method based on the self-encoding model is linear logarithmic time complexity, and the present application is better. At the same time, since the number of clusters can be set on demand, it is theoretically independent of the number of summary tags, which is a constant much smaller than n.

[0048] The efficiency of the present application in tag filtering is much higher than that of the security deduplication method based on the self-encoding model. Assuming that the number of each type of tags is consistent, and is n / m, and the filtering ratio of the security deduplication method based on the self-encoding model is 1 / m, the time complexity of the subsequent fingerprint tag comparison of the two methods is Since the fingerprint tag comparison is the most computationally intensive in the entire deduplication process, the execution efficiency of the two methods is actually not much different. Considering that this time complexity is in the worst case, when the user uploads the data, the sorted tag set will be easier to retrieve than the tag set without sorting.

[0049] After filtering out a type of summary tags, the present application calculates the distance and sorts them, and then filters out a smaller range of summary tags according to a certain ratio. Assuming that this ratio is still 1 / m, the time complexity of the fingerprint tag comparison can be further reduced to Even without further narrowing the range, the efficiency of the comparison of the sorted fingerprint tags is comparable to that of the security deduplication method based on the self-encoding model. At this time, the total time complexity of the summary tag filtering and sorting is Still better than the O(nlogn) of the security deduplication method based on the self-encoding model.

Claims

1. A secure deduplication method based on label clustering, characterized by: The data owner uploads the data fingerprint label and the digest label to the cloud service provider (CSP), sends a deduplication request, and decides whether to upload the encrypted data and labels according to the deduplication detection result; the CSP periodically clusters the digest labels and identifies the categories where the digest labels are located for duplicate data retrieval and deletion; a pre-trained autoencoder model of the user is set; the specific process of this method is as follows: Step 1: The user generates a hash value H(m1) for the data m1, and then selects a random number a to construct the fingerprint tag of the data. Step 2: Use the autoencoder model to generate the digest label AT(m1) of the data; Step 3: User uploads AT(m1) and To CSP; Step 4: The CSP traverses each cluster center in the c_centroid table, calculates the distance from the cluster center to AT(m1), and selects the nearest category i; Step 5: Query all the labels under this category in the c_tag table according to the id of category i to form a label set; Step 6: Traverse the label set in Step 5, calculate the distance between each label and AT(m1), and store it in the binary sort tree; Step 7: Select the smallest first n labels from the binary sort tree in ascending order of distance to form a new label set; Step 8: Traverse the new tag set and retrieve each fingerprint tag. b is a random number selected by other users, and then calculated separately. and If the two are equal, it means that m1 = m2, that is, the user uploaded duplicate data. The CSP tells the user not to upload data and the process ends. Otherwise, execute Step 9; Step 9: The CSP instructs the user to upload encrypted data, and then sends AT(m1) and... Store it in the c_tag table and label its category as i; Step 10: The CSP calculates the variance of the number of labels. If the variance is greater than a certain threshold, the CSP starts clustering the digest labels, otherwise, it ends.

2. The secure deduplication method based on tag clustering according to claim 1, characterized in that: In Step 2, the digest label clustering of the CSP is as follows: Step (2.1): Initialize the variables K = 0, V = +∞; Step (2.2): Select an initial number of clusters k = 5; Step (2.3): Use the k-means clustering algorithm to cluster all the digest labels and calculate the number of labels in each category; Step (2.4): Calculate the variance v of the number of labels. If v < V, then let V = v, K = k; execute Step (2.5); otherwise, end; Step (2.5): If k <= 50, then let k = k + 1, return to execute Step (2.3), otherwise end; Step (2.6): Apply the k-means clustering algorithm again to all the digest labels according to the K value, store the cluster centers in the c_centroid table, and modify the category to which each label in the c_tag table belongs.

3. A secure deduplication method based on label clustering according to claim 1, characterized in that: The CSP uses the k-means clustering algorithm to periodically cluster the digest labels and stores the cluster centers; that is: store the cluster centers in the data table c_centroid, and store the labels in the data table c_tag; The c_centroid table contains 2 fields, namely the identifier id and the coordinates coordinate of the cluster center. The coordinate field concatenates the coordinate values of each dimension into a string in scientific notation; the c_tag table contains 4 fields, namely the identifier id, the fingerprint label fp, the digest label as_t, and the cluster identifier centroid_id. The centroid_id is used to associate with the c_centroid table to indicate which category this label belongs to.

Images