Cloud data deduplication method, device and cloud server based on request merging

Abstract

The present invention discloses a cloud data deduplication method, device and cloud server based on request merging, the method comprising: receiving a deduplication query request sent by a user, wherein the deduplication query request comprises an index value of a target file; dividing the index value into a plurality of index blocks, the plurality of index blocks comprising a common block, a sensitive block and an additional block; determining a corresponding target file according to the common block; when the number of deduplication query requests pointing to the same target file within a preset time reaches a preset threshold, determining a plurality of target index values ​​pointing to the same target file; determining the number of additional blocks corresponding to each target index value, and generating an index value set for all target index values ​​containing the same number of additional blocks; generating a same additional block deduplication index value set for the target index value in the index value set containing the same additional block; and generating a deduplication response according to the XOR value of the additional block, the common block and the sensitive block of the target index value in the same additional block deduplication index value set.

Description

Technical Field

[0001] The present invention relates to the field of cloud service technology, and in particular to a cloud data deduplication method based on request merging, a cloud data deduplication device based on request merging, a cloud server, an electronic device and a non-temporary computer-readable storage medium. Background Art

[0002] In the era of big data, more and more enterprises and users choose cloud service providers to store and manage massive data, which leads to explosive growth of data in cloud storage servers. This inevitably generates a large amount of redundant data, resulting in serious consumption of cloud storage space. In order to solve this problem, cross-user cloud data deduplication technology has been widely studied and applied, so that cloud servers only need to keep one copy of user data.

[0003] According to the different granularities of deduplication data, deduplication technology can be divided into file-level deduplication and chunk-level deduplication. Since chunk-level deduplication technology has more advantages in saving storage and computing overhead, chunk-level cloud data deduplication technology is more favored by researchers. In block-level cross-user cloud data deduplication, before outsourcing the data, the user first divides the target file into data blocks of a specified size and uploads the label information of the corresponding data blocks. The cloud uses the labels to search locally for identical copies. If identical copies are found, the cloud issues a blocking response, and the user no longer uploads the data block. The user only needs to upload ownership verification information; otherwise, the user needs to upload the data block normally.

[0004] However, in actual application scenarios, cross-user deduplication technology puts user privacy data at risk of being stolen. Currently, there are two threat models in cloud data deduplication: side channel attack and additional block attack. In the process of data upload, the cloud's deterministic response to the label builds a side channel for malicious attackers. The attacker can determine the real existence of the cloud data based on whether the data block in the side channel is uploaded normally. Once the attacker determines that a data block exists in the cloud, it will lead to user privacy leakage. Summary of the invention

[0005] The present invention aims to solve one of the technical problems in the above-mentioned technology at least to some extent.

[0006] To this end, the first purpose of the present invention is to propose a cloud data deduplication method based on request merging, which can resist side channel attacks and more complex additional block attacks, and can achieve the security of the cloud data deduplication process.

[0007] The second object of the present invention is to provide a cloud data deduplication device based on request merging.

[0008] The third objective of the present invention is to provide a cloud server.

[0009] A fourth objective of the present invention is to provide an electronic device.

[0010] A fifth object of the present invention is to provide a non-transitory computer-readable storage medium.

[0011] To achieve the above-mentioned purpose, the first aspect of the present invention proposes a cloud data deduplication method based on request merging, comprising the following steps: receiving a deduplication query request sent by a user, wherein the deduplication query request includes an index value of a target file; dividing the index value into multiple index blocks, wherein the multiple index blocks include common blocks, sensitive blocks and additional blocks; determining the corresponding target file according to the common blocks; when the number of deduplication query requests pointing to the same target file within a preset time reaches a preset threshold, determining the multiple target index values ​​pointing to the same target file; determining the number of additional blocks corresponding to each of the target index values, and generating an index value set for all target index values ​​containing the same number of additional blocks. Combine to generate at least one index value set corresponding to the target index value; generate an identical additional block de-duplication index value set from the target index value in the index value set containing the identical additional block, to generate at least one identical additional block de-duplication index value set corresponding to the target index value in the index value set; determine the sensitive block position corresponding to each target index value in the identical additional block de-duplication index value set according to a preset first sensitive block position judgment strategy; and extract the sensitive block of the target index value in the identical additional block de-duplication index value set according to the sensitive block position, and generate a de-duplication response according to the XOR value of the additional block, common block and sensitive block of the target index value in the identical additional block de-duplication index value set.

[0012] According to a cloud data deduplication method based on request merging according to an embodiment of the present invention, a deduplication query request sent by a user is first received, wherein the deduplication query request includes an index value of a target file, and the index value is divided into multiple index blocks, wherein the multiple index blocks include common blocks, sensitive blocks and additional blocks, and the corresponding target file is determined according to the common blocks, when the number of deduplication query requests pointing to the same target file within a preset time reaches a preset threshold, multiple target index values ​​pointing to the same target file are determined, and then the number of additional blocks corresponding to each target index value is determined, and an index value set containing all target index values ​​of the same number is generated, and a same additional block deduplication index value set is generated for the target index values ​​in the index value set containing the same additional block, finally, according to a preset first sensitive block position judgment strategy, the sensitive block position corresponding to each target index value in the same additional block deduplication index value set is determined, and the sensitive block of the target index value in the same additional block deduplication index value set is extracted according to the sensitive block position, and a deduplication response is generated according to the XOR value of the additional block, the common block and the sensitive block of the target index value in the same additional block deduplication index value set. Therefore, it is possible to resist side channel attacks and more complex additional block attacks, and the security of the cloud data deduplication process can be achieved.

[0013] In addition, the cloud data deduplication method based on request merging proposed in the above embodiment of the present invention may also have the following additional technical features:

[0014] In one embodiment of the present invention, the above-mentioned cloud data deduplication method based on request merging also includes: generating a different additional block deduplication index value set from the target index value in the index value set containing different additional blocks; determining the sensitive block position corresponding to each target index value in the different additional block deduplication index value set according to a preset second sensitive block position judgment strategy; obtaining the sensitive block of the target index value in the different additional block deduplication index value set according to the sensitive block position, and generating a deduplication response according to the XOR value of the additional block of the target index value in the different additional block deduplication index value set, the common block and the sensitive block.

[0015] In one embodiment of the present invention, determining the corresponding target file based on the common block specifically includes: obtaining a correspondence table between the target file and the target file index value, wherein the target file index value is divided into multiple target file index blocks; querying the correspondence table between the target file and the target file index value based on the common block to determine the target file corresponding to the common block.

[0016] In one embodiment of the present invention, determining the number of additional blocks corresponding to each of the target index values ​​specifically includes: determining duplicate blocks between multiple index blocks of the target index value and multiple target file index blocks of the target file index value; calculating the XOR value of the duplicate blocks of the target index value, and obtaining the global XOR value of the multiple target file index blocks of the target file index value; comparing the XOR value with the global XOR value, if the XOR value is equal to the global XOR value, then the number of additional blocks corresponding to the target index value is the number of remaining index blocks in the multiple index blocks of the target index value excluding duplicate blocks; if the XOR value is not equal to the global XOR value, then the number of additional blocks corresponding to the target index value is the number of remaining index blocks in the multiple index blocks of the target index value excluding duplicate blocks minus 1.

[0017] To achieve the above-mentioned purpose, the second aspect of the present invention proposes a cloud data deduplication device based on request merging, comprising: a receiving module, used to receive a deduplication query request sent by a user, wherein the deduplication query request includes an index value of a target file; a dividing module, used to divide the index value into multiple index blocks, wherein the multiple index blocks include common blocks, sensitive blocks and additional blocks; a first determining module, used to determine the corresponding target file according to the common blocks; a second determining module, used to determine the multiple target index values ​​pointing to the same target file when the number of deduplication query requests pointing to the same target file within a preset time reaches a preset threshold; a first generating module, used to determine the number of additional blocks corresponding to each of the target index values, and generate a group of all target index values ​​containing the same number of additional blocks. An index value set is provided to generate at least one index value set corresponding to the target index value; a second generating module is used to generate an identical additional block de-duplication index value set from the target index value in the index value set containing the identical additional block, to generate at least one identical additional block de-duplication index value set corresponding to the target index value in the index value set; a third determining module is used to determine the sensitive block position corresponding to each target index value in the identical additional block de-duplication index value set according to a preset first sensitive block position judgment strategy; and a third generating module is used to extract the sensitive block of the target index value in the identical additional block de-duplication index value set according to the sensitive block position, and generate a de-duplication response according to the XOR value of the additional block, common block and sensitive block of the target index value in the identical additional block de-duplication index value set.

[0018] The cloud data deduplication device based on request merging of the embodiment of the present invention first receives a deduplication query request sent by a user through a receiving module, wherein the deduplication query request includes an index value of a target file, and divides the index value into multiple index blocks through a dividing module, wherein the multiple index blocks include common blocks, sensitive blocks and additional blocks, and determines the corresponding target file according to the common blocks through a first determining module, when the number of deduplication query requests pointing to the same target file within a preset time reaches a preset threshold, determines multiple target index values ​​pointing to the same target file through a second determining module, and determines the number of additional blocks corresponding to each target index value through a first generating module, And generate an index value set containing all target index values ​​of the same number, and generate an identical additional block deduplication index value set from the target index values ​​in the index value set containing the identical additional block through the second generation module, and then determine the sensitive block position corresponding to each target index value in the identical additional block deduplication index value set according to the preset first sensitive block position judgment strategy through the third determination module, and finally extract the sensitive block of the target index value in the identical additional block deduplication index value set according to the sensitive block position through the third generation module, and generate a deduplication response according to the additional block of the target index value in the identical additional block deduplication index value set, the XOR value of the common block and the sensitive block. In this way, it is possible to resist side channel attacks and more complex additional block attacks, and achieve the security of the cloud data deduplication process.

[0019] In addition, the cloud data deduplication device based on request merging proposed in the above embodiment of the present invention may also have the following additional technical features:

[0020] In one embodiment of the present invention, the above-mentioned cloud data deduplication device based on request merging also includes: a fourth generation module, used to generate a different additional block deduplication index value set from the target index value in the index value set containing different additional blocks; a fourth determination module, used to determine the sensitive block position corresponding to each target index value in the different additional block deduplication index value set according to a preset second sensitive block position judgment strategy; a fifth generation module, used to obtain the sensitive block of the target index value in the different additional block deduplication index value set according to the sensitive block position, and generate a deduplication response according to the additional block of the target index value in the different additional block deduplication index value set, the XOR value of the common block and the sensitive block.

[0021] In order to achieve the above-mentioned purpose, a cloud server proposed in an embodiment of the third aspect of the present invention includes: a cloud data deduplication device based on request merging in an embodiment of the second aspect of the present invention.

[0022] The cloud server of the embodiment of the present invention, through the above-mentioned cloud data deduplication device based on request merging, can resist side channel attacks and more complex additional block attacks, and can achieve the security of the cloud data deduplication process.

[0023] To achieve the above-mentioned purpose, the fourth aspect of the present invention proposes an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the cloud data deduplication method based on request merging as described in the first aspect of the present invention.

[0024] The electronic device of the embodiment of the present invention can resist side-channel attacks and more complex additional block attacks by executing a computer program stored in a memory through a processor, and can achieve the security of the cloud data deduplication process.

[0025] To achieve the above-mentioned purpose, the fifth aspect embodiment of the present invention proposes a non-temporary computer-readable storage medium on which a computer program is stored, characterized in that the program is executed by a processor to implement the cloud data deduplication method based on request merging as described in the first aspect embodiment of the present invention.

[0026] The non-temporary computer-readable storage medium of the embodiment of the present invention can resist side-channel attacks and more complex additional block attacks by executing the computer program stored therein, and can achieve the security of the cloud data deduplication process.

[0027] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0029] Figure 1 is a schematic diagram of a model of a cloud data deduplication system according to an embodiment of the present invention;

[0030] Figure 2 is a flow chart of a cloud data deduplication method based on request merging according to an embodiment of the present invention;

[0031] Figure 3 is a schematic diagram of a classification model of target index values ​​according to a specific embodiment of the present invention;

[0032] Figure 4 is a schematic diagram of a model of a target index value according to a specific embodiment of the present invention;

[0033] Figure 5 is a schematic diagram of a model of a target index value according to another specific embodiment of the present invention;

[0034] Figure 6 is a schematic diagram of a model of a target index value according to another specific embodiment of the present invention;

[0035] Figure 7 is a block diagram of a cloud data deduplication device based on request merging according to an embodiment of the present invention; and

[0036] Figure 8 4 is a block diagram of a cloud data deduplication device based on request merging according to another embodiment of the present invention. DETAILED DESCRIPTION

[0037] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0038] The following describes a cloud data deduplication method based on request merging, a cloud data deduplication device based on request merging, a cloud server, an electronic device, and a non-transitory computer-readable storage medium according to an embodiment of the present invention in conjunction with the accompanying drawings.

[0039] In an embodiment of the present invention, Figure 1 As shown in FIG, a general cross-user deduplication model may include two entities, a cloud server and a client, wherein the cloud server may be provided by a cloud service provider (CSP). In the deduplication scenario, the client first sends a deduplication query request containing predictable sensitive information to the CSP for inspection, and uploads the non-duplicate part according to the result of the deduplication response.

[0040] Figure 2 The present invention is a flowchart of a cloud data deduplication method based on request merging according to an embodiment of the present invention.

[0041] like Figure 2 As shown, the cloud data deduplication method based on request merging according to an embodiment of the present invention may include the following steps:

[0042] S1, receiving a deduplication query request sent by a user, wherein the deduplication query request includes an index value of a target file.

[0043] Specifically, the user can send a deduplication query request for the file to be uploaded to the cloud server through the client, and the cloud server receives the deduplication query request sent by the user through the client, and analyzes the deduplication query request to obtain the index value of the target file. It should be noted that the deduplication query request described in this embodiment can be a string generated by the client according to the file to be uploaded, or a file, and no limitation is made here.

[0044] S2, dividing the index value into multiple index blocks, where the multiple index blocks include common blocks, sensitive blocks and additional blocks.

[0045] Specifically, after obtaining the index value in the deduplication query request, the cloud server may divide the index value into multiple index blocks according to a preset division algorithm, wherein each index block may have the same size, and the multiple index blocks may include common blocks, sensitive blocks, and additional blocks. The preset division algorithm may be calibrated according to actual conditions.

[0046] S3, determining the corresponding target file according to the common block.

[0047] In one embodiment of the present invention, determining the corresponding target file based on the common block may specifically include obtaining a correspondence table between the target file and the target file index value, wherein the target file index value is divided into multiple target file index blocks, and querying the correspondence table between the target file and the target file index value based on the common block to determine the target file corresponding to the common block.

[0048] It should be noted that the correspondence table between the target files and the target file index values ​​described in this embodiment can be pre-stored in the cloud server, wherein the multiple target file index blocks divided by the target file index values ​​can also store corresponding global XOR values, and the size of the global XOR value can be equal to the size of a single target file index block.

[0049] Specifically, when a large number of deduplication query requests arrive at the cloud server, they are first stored in the cache or hard memory of the cloud server in order. Since the common blocks of the index values ​​in the deduplication query requests should be exactly the same as the common blocks of the target file index values ​​stored in the cloud server, the corresponding target file can be determined by the common blocks of the index values. The cloud server can obtain the common blocks of the index values ​​in each deduplication query request in order of storage, and call out the correspondence table between the target file and the target file index value pre-stored in the cloud server, and then compare the common blocks of the index values ​​in each deduplication query request with the common blocks of the target file index value in the correspondence table in order to determine the corresponding target file. Among them, when the common blocks of the index values ​​are the same as the common blocks of the target file index values ​​in the correspondence table, the target file corresponding to the target file index value is the target file corresponding to the index value.

[0050] S4, when the number of deduplication query requests pointing to the same target file within a preset time reaches a preset threshold, multiple target index values ​​pointing to the same target file are determined. The preset time and the preset threshold can be calibrated according to actual conditions.

[0051] Specifically, after the cloud server determines a target file for each deduplication query request, the real-time counter of the number of requests pointing to the target file increases by 1. Within a preset time, once the value in the counter reaches a preset threshold, the target file can be considered a hot file, and multiple target index values ​​pointing to the target file can be determined. Otherwise, the target file can be considered a cold file, and the client can be required to upload the complete file.

[0052] S5, determining the number of additional blocks corresponding to each target index value, and generating an index value set for all target index values ​​containing the same number of additional blocks, so as to generate at least one index value set corresponding to the target index value.

[0053] In one embodiment of the present invention, determining the number of additional blocks corresponding to each target index value may specifically include determining duplicate blocks between multiple index blocks of the target index value and multiple target file index blocks of the target file index value, calculating the XOR value of the duplicate blocks of the target index value, obtaining the global XOR value of the multiple target file index blocks of the target file index value, and comparing the XOR value with the global XOR value. If the XOR value is equal to the global XOR value, the number of additional blocks corresponding to the target index value is the number of remaining index blocks in the multiple index blocks of the target index value after removing the duplicate blocks. If the XOR value is not equal to the global XOR value, the number of additional blocks corresponding to the target index value is the number of remaining index blocks in the multiple index blocks of the target index value after removing the duplicate blocks minus 1.

[0054] Specifically, for a deduplication query request corresponding to a hot file that has been determined, the cloud server may first determine the duplicate blocks between the multiple index blocks of the target index value corresponding to the hot file and the multiple target file index blocks of the target file index value, and calculate the XOR value of the duplicate blocks of the target index value, and then obtain the global XOR value of the multiple target file index blocks of the target file index value from the storage space of the cloud server (e.g., a database), and compare the XOR value with the global XOR value. If the XOR value is equal to the global XOR value, it indicates that the sensitive block in the target index value is the same as the sensitive block of the target file index value stored in the cloud server, that is, the sensitive block hits, and at this time the number of additional blocks corresponding to the target index value may be the number of remaining index blocks in the multiple index blocks of the target index value excluding duplicate blocks. If the XOR value is not equal to the global XOR value, it indicates that the sensitive block misses, and at this time the number of additional blocks corresponding to the target index value may be the number of remaining index blocks in the multiple index blocks of the target index value excluding duplicate blocks minus 1.

[0055] After determining the number of additional blocks corresponding to each target index value, the cloud server can classify the target index values ​​for the same target file and containing the same number of additional blocks into one category, that is, generate an index value set containing all target index values ​​of the same number to generate at least one index value set corresponding to the target index value, so as to prepare for the subsequent request merging. To simplify the explanation, assume that the number of additional blocks = 2 as an example. For a deduplication query request with two blocks attached, once the number of deduplication query requests in the cloud server reaches a certain random number, the range of the random number can be [2m, 2m + φm], where φ∈[0,1], and the value of m can be calibrated according to the actual situation. Specifically, assuming that the corresponding sensitive blocks in the deduplication query request are evenly distributed, after reaching the left boundary value 2m of the interval, it can be ensured that multiple deduplication query requests are directed to the same sensitive data block. In addition, due to the randomness of the triggering conditions, potential collusion attacks are limited to a minimum.

[0056] S6, generating an identical additional block deduplicated index value set from the target index value in the index value set containing the identical additional block, so as to generate at least one identical additional block deduplicated index value set corresponding to the target index value in the index value set.

[0057] S7, according to a preset first sensitive block position determination strategy, determine the sensitive block position corresponding to each target index value in the same additional block deduplication index value set. The preset first sensitive block position determination strategy can be calibrated according to actual conditions.

[0058] S8, extracting the sensitive block of the target index value in the same additional block deduplication index value set according to the sensitive block position, and generating a deduplication response according to the XOR value of the additional block of the target index value in the same additional block deduplication index value set, the common block and the sensitive block.

[0059] Furthermore, in one embodiment of the present invention, the above-mentioned cloud data deduplication method based on request merging may also include: generating a different additional block deduplication index value set from the target index value in the index value set containing different additional blocks, and determining the sensitive block position corresponding to each target index value in the different additional block deduplication index value set according to the preset second sensitive block position judgment strategy, and obtaining the sensitive block of the target index value in the different additional block deduplication index value set according to the sensitive block position, and generating a deduplication response according to the XOR value of the additional block, common block and sensitive block of the target index value in the different additional block deduplication index value set. Among them, the preset second sensitive block position judgment strategy can be calibrated according to actual conditions.

[0060] Specifically, Figure 3As shown in the figure, in the cloud server, according to whether the additional blocks are the same, the deduplication query requests can be divided into two categories. For the deduplication query requests corresponding to the target index value in the at least one index value set, one category is the deduplication query requests from the same attacker, which are divided into S groups (S 1 ,S 2 ,...,S δ ), where (S 1 ,S 2 ,...,S δ ) represent the groups of target index values ​​in multiple deduplication query requests corresponding to different attackers. Figure 3 Medium S 1 Group, f S1,1 ,f S1,2 , ...represent the target index values ​​in different deduplication query requests of the same attacker, where the public block P1 and the additional block of each target index value are the same, and the sensitive block (C 1 ,C 2 ,...,C n ) are different from each other. 2 The target index values ​​of multiple deduplication query requests initiated by another attacker for the same target file in the cloud. Their public blocks and sensitive blocks are still the same, but the additional blocks may be different. To simplify the representation, different colors can be used to represent the differences between the additional blocks. The other type is the target index values ​​of deduplication query requests from different attackers. Regardless of whether the sensitive blocks in these target index values ​​are the same, they are all uniformly assigned to group D because their additional blocks are different. The data block C marked with bright colors 1 It can represent the situation of sensitive block hit. It is worth noting that the cloud server groups the deduplication query requests according to their content rather than their sources, so it can effectively resist identity hiding and collusion attacks by multiple attackers.

[0061] The preset first sensitive block position determination strategy and the preset second sensitive block position determination strategy are described in detail below to determine the specific position of the sensitive block in the target index value:

[0062] According to the preset first sensitive block location judgment strategy, when the deduplication query request comes from the same attacker, the cloud server can simply identify the additional block by comparing the non-repeated data blocks. Figure 4 As shown, for simplicity, S 1 Take the target index values ​​of the three deduplication query requests in the group as an example. Figure 4 Medium S1,1 ,f S1,2 ,f S1,3 The additional blocks and common blocks of the target index values ​​of the three deduplication query requests are the same, only the sensitive blocks are different, and only f S1,1 Sensitive block C1 is the same as the sensitive block in the cloud server, that is, a hit. In this case, once the target index value f of the deduplication query request is S1,1 The block containing the hit sensitive block, then compare f S1,1 and f S1,2 , we can find that C 1 and C 2 are different, so C 2 is the sensitive block that was missed. After that, compare f S1,2 and f S1,3 , only C 2 With C 3 are different from each other, so the sensitive block C in 3 Therefore, in the group where the request of the same attacker is located, regardless of whether the sensitive block in the request is hit or not, the specific position of the sensitive block in the target index value of the deduplication query request and the target file can be determined by comparing with each other, so that the sensitive block can be effectively included in the deduplication response.

[0063] For the preset second sensitive block position judgment strategy, such as Figure 5 As shown, for group D containing sensitive block C 1 The target index value of a single deduplication query request, when the sensitive block C 1 When a hit occurs, the location of the sensitive block can be determined by comparing the XOR values ​​of all repeated data blocks with the global XOR value. Since the comparison results are consistent, by comparing the XOR values, it can be determined whether the sensitive block is hit. If it is hit, the specific location of the sensitive block can be obtained.

[0064] For the case where the sensitive blocks in group D are not hit, the cloud server can determine the specific location of the sensitive blocks in different situations through two comparison methods. Figure 6 As shown in (a), the target index value f of the deduplication query request D,2 and f D,5 The sensitive block C in 2 The same, but both miss. In this case, the cloud server first determines that the sensitive block misses by calculating the XOR value, and then compares the same data blocks in the request to determine the specific location of the sensitive block. Figure 6 As shown in (b), the target index value f of the deduplication query request D,4 Due to its additional block and S 1 There are differences between the groups and they cannot be merged into S 1 group, and the target index value of the deduplication query request with the same XOR value cannot be matched in group D. In this case, by comparing the target index value f of the deduplication query request S1,4 and f D,4The location of sensitive blocks can be determined.

[0065] Furthermore, after determining the location of the sensitive block, the cloud server can generate a deduplication response based on the target index value in the deduplication query request, and the XOR value of the common block of the target index value in the deduplication query request and its sensitive block. This can resist side-channel attacks and more complex additional block attacks, and can achieve the security of the cloud data deduplication process.

[0066] Through the cloud data deduplication method based on request merging of the present invention, the cloud server can not only determine the number of additional blocks, but also determine the content of the sensitive block and its specific location. In order to resist the additional block attack and enable the deduplication response of the cloud server to confuse the attacker, the cloud server generates a deduplication response of N+XOR. Among them, N is the additional block of the target index value, that is, the index value corresponding to the additional block in the target index value, and XOR is the XOR value of the common block and its sensitive block in the target index value, and the size of the XOR value is equal to the size of a data block.

[0067] It should be noted that after receiving the above deduplication response, the client can obtain the non-duplicate data block in the file to be uploaded according to the index of the additional block in the deduplication response, and upload it to the cloud server to complete this storage operation.

[0068] In an embodiment of the present invention, the design of the cloud data deduplication method based on request merging of the present invention includes considerations of both security and efficiency. Among them, security means that the proposed method must effectively prevent attackers from stealing the existential privacy of cloud data by establishing a side channel in the scenario of additional block attacks. That is, consider that the attacker attaches a random number of non-hit blocks to the file to be detected and uploads it to the cloud server. The cloud server generates a response according to the proposed method, and includes a certain amount of redundant block information in the response to achieve fuzzification, thereby achieving the purpose of confusing the attacker. When talking about security, the cost of achieving security must be considered. Since the proposed method confuses attackers by adding redundant block information to the response, and the data blocks included in the response are all blocks required to be uploaded by the user, the overhead considered by the present invention is mainly traffic overhead. The efficiency goal of the proposed method is to control the traffic overhead to the lowest level in the existing scheme.

[0069] Furthermore, in terms of security, the method of the present invention can resist side channel attacks and more complex additional block attacks. Since the method of the present invention can determine the specific location of the sensitive block in the deduplication request file in the merge module, the deduplication response returned by the method always contains the sensitive block regardless of whether the sensitive block is hit or not. This fuzzy response can effectively confuse the attacker, thereby maintaining the true existence of the detected file. Therefore, no matter whether the malicious attacker uses a side channel attack or an additional block attack, it is impossible to steal the user's privacy through the deduplication response, thereby achieving security in the deduplication process.

[0070] In terms of efficiency, the bandwidth overhead of the method of the present invention can be reduced to a minimum value compared to the method in the prior art. The result of the deduplication response of the method of the present invention is finally returned as N+XOR, where N is the index value corresponding to the additional block, and XOR is the XOR value of the common block and the sensitive block. Since the size of XOR is the same as the size of a data block, the bandwidth overhead of XOR is always N+1. Therefore, the method of the present invention can achieve the lowest bandwidth overhead.

[0071] In summary, according to the cloud data deduplication method based on request merging of an embodiment of the present invention, first, a deduplication query request sent by a user is received, wherein the deduplication query request includes an index value of a target file, and the index value is divided into multiple index blocks, wherein the multiple index blocks include common blocks, sensitive blocks and additional blocks, and the corresponding target file is determined according to the common blocks. When the number of deduplication query requests pointing to the same target file within a preset time reaches a preset threshold, multiple target index values ​​pointing to the same target file are determined, and then the number of additional blocks corresponding to each target index value is determined, and an index value set containing all target index values ​​of the same number is generated, and a same additional block deduplication index value set is generated for the target index values ​​in the index value set containing the same additional block. Finally, according to a preset first sensitive block position judgment strategy, the sensitive block position corresponding to each target index value in the same additional block deduplication index value set is determined, and the sensitive block of the target index value in the same additional block deduplication index value set is extracted according to the sensitive block position, and a deduplication response is generated according to the XOR value of the additional block, the common block and the sensitive block of the target index value in the same additional block deduplication index value set. Therefore, it is possible to resist side channel attacks and more complex additional block attacks, and achieve the security of the cloud data deduplication process.

[0072] Figure 7 4 is a block diagram of a cloud data deduplication device based on request merging according to an embodiment of the present invention.

[0073] like Figure 7 As shown, the cloud data deduplication device 1000 based on request merging of an embodiment of the present invention may include: a receiving module 100, a dividing module 200, a first determination module 300, a second determination module 400, a first generation module 500, a second generation module 600, a third determination module 700 and a third generation module 800.

[0074] The receiving module 100 is used to receive a deduplication query request sent by a user, wherein the deduplication query request includes an index value of a target file.

[0075] The partitioning module 200 is used to partition the index value into a plurality of index blocks, wherein the plurality of index blocks include a common block, a sensitive block and an additional block.

[0076] The first determination module 300 is used to determine the corresponding target file according to the common block.

[0077] The second determination module 400 is used to determine multiple target index values ​​pointing to the same target file when the number of deduplication query requests pointing to the same target file within a preset time reaches a preset threshold.

[0078] The first generating module 500 is used to determine the number of additional blocks corresponding to each target index value, and generate an index value set for all target index values ​​containing the same number of additional blocks, so as to generate at least one index value set corresponding to the target index value.

[0079] The second generating module 600 is used to generate an identical additional block deduplicated index value set from the target index value in the index value set containing the identical additional block, so as to generate at least one identical additional block deduplicated index value set corresponding to the target index value in the index value set.

[0080] The third determination module 700 is used to determine the sensitive block position corresponding to each target index value in the same additional block deduplication index value set according to a preset first sensitive block position determination strategy.

[0081] The third generation module 800 is used to extract the sensitive block of the target index value in the same additional block deduplication index value set according to the sensitive block position, and generate a deduplication response according to the additional block of the target index value in the same additional block deduplication index value set, the XOR value of the common block and the sensitive block.

[0082] In one embodiment of the present invention, Figure 8 As shown, the above-mentioned cloud data deduplication device 1000 based on request merging also includes: a fourth generating module 900 , a fourth determining module 10 and a fifth generating module 20 .

[0083] The fourth generating module 900 is used to generate a different additional block deduplication index value set from the target index value in the index value set containing different additional blocks.

[0084] The fourth determination module 10 is used to determine the sensitive block position corresponding to each target index value in different additional block deduplication index value sets according to a preset second sensitive block position judgment strategy.

[0085] The fifth generation module 20 is used to obtain the sensitive block of the target index value in different additional block deduplication index value sets according to the sensitive block position, and generate a deduplication response according to the XOR value of the additional block of the target index value in different additional block deduplication index value sets, the common block and the sensitive block.

[0086] It should be noted that for details not disclosed in the cloud data deduplication device based on request merging in an embodiment of the present invention, please refer to the details disclosed in the cloud data deduplication method based on request merging in an embodiment of the present invention, and the details will not be repeated here.

[0087] In summary, the cloud data deduplication device based on request merging in the embodiment of the present invention first receives a deduplication query request sent by a user through a receiving module, wherein the deduplication query request includes an index value of a target file, and divides the index value into multiple index blocks through a dividing module, wherein the multiple index blocks include common blocks, sensitive blocks and additional blocks, and determines the corresponding target file according to the common blocks through a first determining module, and when the number of deduplication query requests pointing to the same target file within a preset time reaches a preset threshold, determines multiple target index values ​​pointing to the same target file through a second determining module, and determines the number of additional blocks corresponding to each target index value through a first generating module. The method comprises the following steps: first, determining the sensitive block position corresponding to each target index value in the same additional block deduplication index value set according to the preset first sensitive block position judgment strategy, and finally extracting the sensitive block of the target index value in the same additional block deduplication index value set according to the sensitive block position through the third generation module, and generating a deduplication response according to the additional block of the target index value in the same additional block deduplication index value set according to the XOR value of the additional block, the common block and the sensitive block of the target index value in the same additional block deduplication index value set. Thus, it is possible to resist side channel attacks and more complex additional block attacks, and realize the security of the cloud data deduplication process.

[0088] In order to achieve the above-mentioned purpose, a cloud server proposed in an embodiment of the third aspect of the present invention includes: a cloud data deduplication device based on request merging in an embodiment of the second aspect of the present invention.

[0089] The cloud server of the embodiment of the present invention, through the above-mentioned cloud data deduplication device based on request merging, can resist side channel attacks and more complex additional block attacks, and can achieve the security of the cloud data deduplication process.

[0090] In order to implement the above embodiments, the present invention also proposes an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the cloud data deduplication method based on request merging of the above embodiments.

[0091] The electronic device of the embodiment of the present invention can resist side-channel attacks and more complex additional block attacks by executing a computer program stored in a memory through a processor, and can achieve the security of the cloud data deduplication process.

[0092] In order to implement the above embodiments, the present invention also proposes a non-temporary computer-readable storage medium on which a computer program is stored, characterized in that the program is executed by a processor to implement the cloud data deduplication method based on request merging of the above embodiments.

[0093] The non-temporary computer-readable storage medium of the embodiment of the present invention can resist side-channel attacks and more complex additional block attacks by executing the computer program stored therein, and can achieve the security of the cloud data deduplication process.

[0094] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", "axial", "radial", "circumferential" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the referred device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as limiting the present invention.

[0095] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0096] In the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0097] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0098] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. A cloud data deduplication method based on request merging, characterized in that: The following steps are involved: Receiving a deduplication query request sent by a user, wherein the deduplication query request includes an index value of a target file; Dividing the index value into a plurality of index blocks, wherein the plurality of index blocks include a common block, a sensitive block and an additional block; Determine the corresponding target file according to the common block; When the number of deduplication query requests pointing to the same target file within a preset time reaches a preset threshold, determining the multiple target index values ​​pointing to the same target file; Determine the number of additional blocks corresponding to each of the target index values, and generate an index value set for all target index values ​​containing the same number of additional blocks, so as to generate at least one index value set corresponding to the target index value; Generating a same additional block de-duplication index value set from the target index value in the index value set containing the same additional block, so as to generate at least one same additional block de-duplication index value set corresponding to the target index value in the index value set; Determine the sensitive block position corresponding to each target index value in the same additional block deduplication index value set according to a preset first sensitive block position judgment strategy; and The sensitive block of the target index value in the same additional block deduplication index value set is extracted according to the sensitive block position, and a deduplication response is generated according to the XOR value of the additional block of the target index value in the same additional block deduplication index value set, the common block and the sensitive block.

2. The cloud data deduplication method based on request merging according to claim 1, characterized in that: Also includes: Generate a different additional block deduplication index value set from the target index value in the index value set containing different additional blocks; Determine the sensitive block position corresponding to each target index value in the different additional block deduplication index value sets according to a preset second sensitive block position judgment strategy; The sensitive block of the target index value in the different additional block deduplication index value sets is obtained according to the sensitive block position, and a deduplication response is generated according to the XOR value of the additional block of the target index value in the different additional block deduplication index value sets, the common block and the sensitive block.

3. The cloud data deduplication method based on request merging according to claim 1, characterized in that: Determining the corresponding target file according to the common block specifically includes: Obtaining a correspondence table between target files and target file index values, wherein the target file index values ​​are divided into a plurality of target file index blocks; The corresponding relationship table between the target file and the target file index value is queried according to the common block to determine the target file corresponding to the common block.

4. The cloud data deduplication method based on request merging according to claim 3, characterized in that: The determining the number of additional blocks corresponding to each target index value specifically includes: Determine duplicate blocks between the multiple index blocks of the target index value and the multiple target file index blocks of the target file index value; Calculating the XOR value of the repeated blocks of the target index value, and obtaining the global XOR value of multiple target file index blocks of the target file index value; Compare the XOR value with the global XOR value, if the XOR value is equal to the global XOR value, the number of additional blocks corresponding to the target index value is the number of remaining index blocks in the multiple index blocks of the target index value excluding duplicate blocks; If the XOR value is not equal to the global XOR value, the number of additional blocks corresponding to the target index value is the number of remaining index blocks in the multiple index blocks of the target index value excluding duplicate blocks minus 1.

5. A cloud data deduplication device based on request merging, characterized in that: include: A receiving module, configured to receive a deduplication query request sent by a user, wherein the deduplication query request includes an index value of a target file; A partitioning module, used for partitioning the index value into a plurality of index blocks, wherein the plurality of index blocks include a common block, a sensitive block and an additional block; A first determining module, used to determine a corresponding target file according to the common block; A second determination module is used to determine the multiple target index values ​​pointing to the same target file when the number of deduplication query requests pointing to the same target file reaches a preset threshold within a preset time; A first generating module, configured to determine the number of additional blocks corresponding to each of the target index values, and generate an index value set for all target index values ​​containing the same number of additional blocks, so as to generate at least one index value set corresponding to the target index value; A second generating module, configured to generate an identical additional block de-duplicated index value set from a target index value in the index value set containing the identical additional block, so as to generate at least one identical additional block de-duplicated index value set corresponding to the target index value in the index value set; A third determination module is used to determine the sensitive block position corresponding to each target index value in the same additional block deduplication index value set according to a preset first sensitive block position judgment strategy; and The third generating module is used to extract the sensitive block of the target index value in the same additional block de-duplication index value set according to the sensitive block position, and generate a de-duplication response according to the additional block of the target index value in the same additional block de-duplication index value set, the XOR value of the common block and the sensitive block.

6. The cloud data deduplication device based on request merging according to claim 5, characterized in that: Also includes: A fourth generating module, configured to generate a different additional block deduplication index value set from the target index value in the index value set containing different additional blocks; A fourth determination module, configured to determine the sensitive block position corresponding to each target index value in the different additional block deduplication index value sets according to a preset second sensitive block position determination strategy; The fifth generation module is used to obtain the sensitive block of the target index value in the different additional block deduplication index value sets according to the sensitive block position, and generate a deduplication response according to the additional block of the target index value in the different additional block deduplication index value sets, the XOR value of the common block and the sensitive block.

7. A cloud server, characterized in that: It comprises a cloud data deduplication device based on request merging as described in any one of claims 5-6.

8. An electronic device, characterized in that: The invention comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the cloud data deduplication method based on request merging as described in any one of claims 1 to 4.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement the cloud data deduplication method based on request merging as described in any one of claims 1 to 4.

Images