A data integrity verification method and device based on independent copies

Abstract

The embodiment of the present application provides a kind of data integrity verification method and device based on independent copy, method includes: when user side responds data integrity verification instruction in cloud storage, the metadata of at least two cloud storage copies is obtained;According to the metadata of each cloud storage copy, send computing instruction to the cloud storage server of cloud storage copy, the computing instruction includes: the metadata of corresponding cloud storage copy, specified data digest algorithm, the information of the hash value of specified data section in data copy calculated using specified data digest algorithm, the data of cloud storage copy is the data after original storage order according to metadata restoration;Receive the hash value returned by cloud storage server according to computing instruction, judge whether the hash value returned by all cloud storage servers is consistent;If consistent, determine that the data in cloud storage copy is complete.The above method is simple and convenient, and solves the problem that PDP of prior art is complex and dependent on third party, and there is reliability proof.

Description

Technical Field

[0001] This invention relates to the field of cloud storage data verification technology, and in particular to a data integrity verification method based on independent copies, a data integrity verification device based on independent copies, an electronic device, and a computer-readable medium in cloud storage. Background Technology

[0002] In cloud storage scenarios, customers store data on untrusted cloud storage services. How to efficiently verify the integrity of the customer's stored data without local data copies is a real need for users.

[0003] For example, using a third-party auditing firm can replace the user in verifying the integrity of data stored on remote nodes, reducing the computational burden on the user. Auditing firms typically use various types of PDP (Proof of Data Possession) algorithms for reliability verification. They input the data owner's public key, send a challenge request and evidence, and receive a response indicating whether the verification is correct or incorrect. Auditing verifies data integrity with a certain probability, so repeatedly initiating challenge requests increases the probability of confirming data integrity.

[0004] Third-party auditing firms using the PDP protocol for verification require clients to perform complex preprocessing of data and subsequently send audit requests proactively or periodically to analyze the audit results. This reliance on third-party verification objectively raises issues regarding data security and the reliability of proving forgery. Summary of the Invention

[0005] This invention provides a data integrity verification method, apparatus, electronic device, and computer-readable storage medium based on independent copies, to solve the problems of existing PDPs being complex, dependent on third parties, and lacking reliability proof.

[0006] This invention discloses a data integrity verification method based on independent copies, comprising:

[0007] When the user responds to a data integrity verification command in cloud storage, it obtains metadata from at least two cloud storage copies.

[0008] Based on the metadata of each cloud storage replica, a calculation instruction is sent to the cloud storage server of the cloud storage replica. The calculation instruction includes: the metadata of the corresponding cloud storage replica, a specified data digest algorithm, and information on calculating the hash value of a specified data segment in the data replica using the specified data digest algorithm. The data replica is the data of the cloud storage replica after restoring the original storage order according to the metadata.

[0009] Receive the hash value returned by the cloud storage server based on the calculation instructions, and determine whether the hash values ​​returned by all cloud storage servers are consistent;

[0010] If they match, then the data in the cloud storage copy is confirmed to be complete.

[0011] Optionally, the method further includes:

[0012] If at least two cloud storage copies are provided by the same cloud storage server, and the user's local storage has a copy of the original data, then the hash value of the specified data segment in the original data copy is obtained using the specified data digest algorithm.

[0013] Determine whether the hash values ​​returned by each cloud storage server are consistent with the hash values ​​corresponding to the specified data segments on the user side;

[0014] If they match, then the data in the cloud storage copy is confirmed to be complete.

[0015] Optionally, if there is no original data copy on the user's local machine, then metadata from at least two cloud storage copies is obtained, including:

[0016] Obtain metadata corresponding to cloud storage replicas on different cloud storage servers.

[0017] Optionally, if only one cloud storage copy exists, the calculation instructions include: metadata corresponding to the cloud storage copy, a specified data digest algorithm, and information on calculating the data summary hash value in the data copy using the specified data digest algorithm, wherein the data copy is the data in the cloud storage copy after restoring the original storage order according to the metadata;

[0018] Accordingly, the method further includes:

[0019] The hash value of the user-side local raw data summary is calculated using a specified data digest algorithm;

[0020] Determine whether the hash value returned by the cloud storage server is consistent with the hash value calculated by the user.

[0021] If they match, then the data in the cloud storage copy is confirmed to be complete.

[0022] Optionally, the method further includes:

[0023] On the user side, the storage order of the original data to be stored in the cloud is transformed based on the storage order transformation method to obtain a cloud storage copy and the metadata of the cloud storage copy.

[0024] The cloud storage copy is sent to the cloud storage server, and the metadata is stored on the user side;

[0025] Any two cloud storage copies are different;

[0026] Accordingly, obtain metadata for at least two cloud storage replicas, including:

[0027] Obtain metadata for at least two cloud storage copies of the user-side storage.

[0028] Optionally, the method further includes:

[0029] On the user side, the storage order of the original data to be stored in the cloud is transformed based on the storage order transformation method to obtain a cloud storage copy and the metadata of the cloud storage copy.

[0030] The cloud storage copy and its metadata are combined according to predefined rules to form the storage text to be sent to the cloud storage.

[0031] Send the stored text to the cloud storage server;

[0032] Any two cloud storage copies are different;

[0033] Accordingly, obtain metadata for at least two cloud storage replicas, including:

[0034] Send a metadata retrieval request to the cloud storage server. The metadata retrieval request includes: the data storage location of the metadata in the storage file, determined according to predefined rules, and the metadata download information in the data storage location.

[0035] Receives metadata of cloud storage copies returned by each cloud storage server based on the metadata retrieval request.

[0036] Optionally, the method further includes:

[0037] If inconsistencies exist, obtain the hash values ​​of each data segment to which all cloud storage replicas belong, and based on the hash values ​​of each data segment to which all cloud storage replicas belong, parse out the data area of ​​the cloud storage replica with the error.

[0038] The data in the data area is recovered based on the data area of ​​the cloud storage copy with the storage error and the data copy converted from the cloud storage copy without the error.

[0039] This invention also provides a data integrity verification device based on independent copies, comprising:

[0040] The metadata acquisition unit is used to acquire metadata from at least two cloud storage replicas when responding to a data integrity verification command in cloud storage.

[0041] The hash value acquisition unit is used to send a calculation instruction to the cloud storage server of each cloud storage replica based on the metadata of each cloud storage replica. The calculation instruction includes: the metadata of the corresponding cloud storage replica, a specified data digest algorithm, and information on calculating the hash value of a specified data segment in the data replica using the specified data digest algorithm. The data replica is the data of the cloud storage replica after restoring the original storage order according to the metadata.

[0042] The comparison unit is used to receive the hash value returned by the cloud storage server according to the calculation instruction, and to determine whether the hash values ​​returned by all cloud storage servers are consistent; if they are consistent, it is determined that the data in the cloud storage replica is complete.

[0043] This invention also discloses an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0044] The memory is used to store computer programs;

[0045] When the processor executes a program stored in the memory, it implements the method described in the embodiments of the present invention.

[0046] This invention also discloses one or more computer-readable media storing instructions that, when executed by one or more processors, cause the processors to perform the methods described in this invention.

[0047] The embodiments of the present invention have the following advantages:

[0048] The method of this invention can verify the integrity of cloud storage copies in scenarios where the user holds the original data copy, by using two or more cloud storage copies and an independent data copy in the cloud storage. It restores the data copy of the cloud storage copy, then uses a specified data digest algorithm to obtain the hash value of a specified data segment, and compares the hash values ​​to achieve data integrity verification of the cloud storage copy. It is simple and fast, and improves the reliability of cloud storage copy data integrity verification based on the performance of the cloud storage server.

[0049] The device in this invention is relatively simple and efficient. By using an independent verification method that makes it impossible to forge audit results, it avoids the complexity of PDP and the shortcomings of algorithms in the prior art, and provides a data integrity verification process that is easier to implement. Attached Figure Description

[0050] Figure 1 This is a flowchart of the steps of a data integrity verification method based on independent copies provided in an embodiment of the present invention;

[0051] Figure 2This is a block diagram of a data integrity verification method based on independent copies provided in an embodiment of the present invention;

[0052] Figure 3 This is a schematic diagram illustrating the acquisition of a cloud storage copy based on a data storage transformation method provided in an embodiment of the present invention;

[0053] Figure 4 This is a structural block diagram of a data integrity verification device based on an independent copy provided in an embodiment of the present invention;

[0054] Figure 5 This is a block diagram of an electronic device provided in an embodiment of the present invention;

[0055] Figure 6 This is a schematic diagram of a computer-readable medium provided in an embodiment of the present invention. Detailed Implementation

[0056] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, some terms are explained as follows:

[0057] Message Digest: It is a fixed-length value that uniquely corresponds to a message or text. It is generated by applying a one-way hash encryption function to the message.

[0058] PDP (Proof of Data Possession): Used to verify whether the data owner is actually using the data stored in the cloud or distributed storage system.

[0059] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0060] Users store data on cloud servers, losing absolute control over their data. Data loss is sometimes unavoidable due to reasons such as server downtime or storage media damage. Cloud storage is the foundation of cloud computing; if data integrity cannot be guaranteed, all data analysis and computation become meaningless. Therefore, this invention provides a data integrity verification method based on independent copies.

[0061] Reference Figure 1 and Figure 2 , Figure 1 The following is a flowchart illustrating the steps of a data integrity verification method based on independent copies provided in an embodiment of the present invention, which may specifically include the following steps:

[0062] 101. When the user responds to the data integrity verification command in the cloud storage, it obtains the metadata of at least two cloud storage copies.

[0063] The metadata in this embodiment can be understood as data that describes the data copy / document definition data transformation rules / order, which can be stored locally on the user side or stored on a cloud storage server according to the cloud storage copy.

[0064] In practical applications, metadata corresponding to cloud storage replicas on different cloud storage servers can be obtained. Of course, metadata corresponding to multiple cloud storage replicas on the same cloud storage server can also be obtained. In this embodiment, a cloud storage replica can be a file composed of random paragraphs. Typically, different cloud storage replicas display different data externally.

[0065] 102. Based on the metadata of each cloud storage replica, send a calculation instruction to the cloud storage server of the cloud storage replica. The calculation instruction includes: the metadata of the corresponding cloud storage replica, a specified data digest algorithm, and information on calculating the hash value of a specified data segment in the data replica using the specified data digest algorithm. The data replica is the data of the cloud storage replica after restoring the original storage order according to the metadata.

[0066] For example, the specified data digest algorithm in this embodiment can be a message digest algorithm, which is a fixed-length value that uniquely corresponds to a message or text. It is generated by applying a one-way hash encryption function to the message. As long as the input message is different, the digest message generated after digesting it will also be different; however, the same input will produce the same output. Therefore, the message digest algorithm guarantees the integrity of the message, such as the MD5 algorithm or the SHA-1 algorithm.

[0067] The specified data segment in the above steps can be a continuous data segment or a separate data segment.

[0068] 103. Receive the hash value returned by the cloud storage server based on the calculation instruction, and determine whether the hash values ​​returned by all cloud storage servers are consistent.

[0069] In this embodiment, each cloud storage server uses the same data digest algorithm to calculate the hash value of the same data segment to obtain the hash value to be returned.

[0070] Since the content of the original data copy to which the cloud storage copy belongs is identical, the hash value of the specified data segment can be calculated and compared after conversion. The cloud storage server in this embodiment cannot indirectly rely on other data copies to generate counterfeit hash values ​​for verification.

[0071] 104. If they match, then the data in the cloud storage copy is confirmed to be complete.

[0072] Of course, in practical applications, if the hash values ​​are inconsistent, and two of the three cloud storage servers return the same hash value, it can be determined that the cloud storage replicas of the inconsistent cloud storage servers have data incompleteness.

[0073] If there are only two cloud storage copies and their hash values ​​are inconsistent, the hash value of the user-side data summary will be used for further judgment.

[0074] For example, a specified data digest algorithm is used to obtain the hash value of the data summary on the user side, and new calculation instructions are sent to each cloud storage server so that the cloud storage server can calculate the hash value of the data summary in the data copy based on the new calculation instructions and using the specified data digest algorithm.

[0075] The user compares the hash value calculated locally with the hash value returned by the cloud storage server. If there is an inconsistency, it can be confirmed that the data of the non-local cloud storage copy is incomplete.

[0076] In this embodiment, the hash values ​​of the converted data copies from multiple cloud storage copies are obtained and compared to verify the data integrity, which has high reliability.

[0077] Specifically, the method in this embodiment may further include: when the hash values ​​are inconsistent, obtaining the hash values ​​of each data segment to which all cloud storage replicas belong, parsing out the data area of ​​the cloud storage replica with errors based on the hash values ​​of each data segment to which all cloud storage replicas belong; and then restoring the data in the data area based on the data area of ​​the cloud storage replica with errors and the data replica converted from the cloud storage replica without errors.

[0078] The method in this embodiment can verify the integrity of cloud storage copies in scenarios where the user holds a copy of the original data, by using two or more cloud storage copies and an independent data copy in the cloud storage. It restores the data copy of the cloud storage copy, then uses a specified data digest algorithm to obtain the hash value of a specified data segment, and compares the hash values ​​to achieve data integrity verification of the cloud storage copy. It is simple and fast, and improves the reliability of cloud storage copy data integrity verification based on the performance of the cloud storage server.

[0079] In addition, to better verify data integrity, the above Figure 1 The method shown may also include;

[0080] In step 101 above, at least two cloud storage copies are provided by the same cloud storage server. At this time, if the user's local storage has a copy of the original data, the hash value of the specified data segment in the original data copy is obtained by using a specified data digest algorithm.

[0081] Then it can be determined whether the hash value returned by each cloud storage server is consistent with the hash value corresponding to the specified data segment on the user side; if they are consistent, it is determined that the data in the cloud storage replica is complete.

[0082] In this implementation, the hash value of the corresponding data segment can be obtained by traversing each data segment in the data replica. Then, the hash value of the cloud storage server is compared with the hash value of the corresponding data segment on the user's local side one by one to determine the integrity of the data in the cloud storage replica.

[0083] In the third possible implementation, there is only one cloud storage copy in step 101 above. In this case, the calculation instructions in step 102 may include: metadata of the corresponding cloud storage copy, a specified data digest algorithm, and information on calculating the data summary hash value in the data copy using the specified data digest algorithm. The data copy is the data of the cloud storage copy after restoring the original storage order according to the metadata.

[0084] at this time, Figure 1 The method also includes: calculating the hash value of the user-side local raw data summary using a specified data digest algorithm;

[0085] Determine whether the hash value returned by the cloud storage server is consistent with the hash value calculated by the user; if they are consistent, then the data in the cloud storage replica is confirmed to be complete.

[0086] In other words, any document that needs to be stored in the cloud on the user side has a data summary information on the user side.

[0087] Combination Figure 3 As shown, typically, the user can convert the original data into multiple cloud storage copies (cloud storage data copy 1, cloud storage data copy 2, cloud storage copy ..., cloud storage copy n, etc.) and store them in the cloud, i.e., a cloud storage server. Figure 3 One of the cloud storage copies shown (such as cloud storage data copy 1) can be a data copy in normal data order and stored in plaintext. Other cloud storage copies can be cloud storage files whose storage order has been changed. This way, if data in other cloud storage copies is lost, it can be recovered or rebuilt using the data copies in normal data order. In this embodiment, the logic for segmenting the original data can also be adjusted to reduce the number and length of segments, or different segments can be selected for different batches, with the remaining segments selected successively in subsequent batches. This facilitates effectively reducing the computational load during integrity verification, thus reducing the load on the cloud storage server and enabling rapid and comprehensive verification of data reliability.

[0088] Figure 3The segmented storage shown is merely illustrative and is not intended to limit the scope of this embodiment. Adjustments and variations can be made according to actual needs. Of course, in other embodiments, a copy of the original data in normal data order can be stored locally on the user's side.

[0089] like Figure 3 As shown, before performing step 101 above, the method of this embodiment further includes step 100, which is not shown in the figures below:

[0090] 100. On the user side, the storage order of the original data to be stored in the cloud is transformed based on the storage order transformation method to obtain a cloud storage copy for cloud storage and the metadata of the cloud storage copy; the cloud storage copy is sent to the cloud storage server, and the metadata is stored on the user side.

[0091] In this embodiment, any two cloud storage copies are different;

[0092] Accordingly, obtaining the metadata of at least two cloud storage copies in step 101 above may include: obtaining the metadata of at least two cloud storage copies stored on the user side.

[0093] In this embodiment, the metadata is retained on the user side and cannot be accessed by the cloud storage server, thus making it impossible to forge audit results that resemble data through summary parameters or other means.

[0094] exist Figure 3 Multiple independent cloud storage data copies are created, which can be stored on different cloud storage servers, ensuring service independence. Figure 3 The first data copy uses the original data order, while the second cloud storage copy can be obtained by transforming the storage order. This involves segmenting the data and changing the segment positions to make the data segmentation and storage order as random as possible. Simultaneously, metadata corresponding to this cloud storage copy is created. This metadata describes the segmentation and order information of the cloud storage copy. Therefore, during subsequent restoration, the cloud storage copy can be restored based on the information recorded in the metadata to obtain a data copy with the same segmentation and order as the original data.

[0095] The storage order transformation method in this embodiment also includes: [the ability to transform the storage order]. Figure 3 The data in the second data copy shown is "processed", such as adding random length data at fixed positions and recording this random data description in the metadata for use when restoring the original data segments, which can prevent the segmentation order logic of the cloud storage copy from being analyzed.

[0096] By inserting data of random length into the original data, the logical order of the data can be disrupted, increasing the difficulty of data parsing and analysis and improving data confidentiality. During the verification process, the metadata is decrypted first, and then the data copy is restored based on the metadata, ensuring data integrity and correctness. Storing the metadata and data copy together and using encryption protection effectively prevents unauthorized access and tampering. Decrypting the metadata before each data verification increases the security level. This method further enhances data storage security and is suitable for scenarios with high requirements for data confidentiality and integrity.

[0097] Of course, the aforementioned metadata can also be placed in the data copy of the cloud storage service provider. For example, the metadata and the cloud storage copy can be stored together, and the metadata location and data location can be defined using a fixed length method (with sufficient length reserved). The metadata part can be stored in encryption and decrypted before each verification.

[0098] That is, in another optional implementation, combined with Figure 3 As shown, step 100 of this embodiment may include: the user side performs storage order transformation on the original data to be stored in the cloud based on the storage order transformation method to obtain a cloud storage copy for cloud storage and the metadata of the cloud storage copy; the cloud storage copy and the metadata of the cloud storage copy are combined according to predefined rules to form a storage text to be sent to the cloud storage; and the storage text is sent to the cloud storage server.

[0099] Of course, in this embodiment, any two cloud storage copies are different.

[0100] At this point, obtaining the metadata of at least two cloud storage replicas in step 101 above may include:

[0101] The user sends a metadata retrieval request to the cloud storage server. The metadata retrieval request includes: the data storage location of the metadata in the storage file determined according to predefined rules, and the metadata download information in the data storage location; and receiving the metadata of the cloud storage copy returned by each cloud storage server based on the metadata retrieval request.

[0102] This means that the user side does not need to store metadata, and the metadata of each cloud storage copy and the cloud storage copy itself can be stored on the cloud storage server.

[0103] The method described in this invention can solve the problem of verifying the integrity and reliability of cloud storage copies when the user side lacks a local copy of the original data. It can also solve the problem of verifying the data integrity and reliability of cloud storage copies when the user side lacks metadata.

[0104] The method in this embodiment is simple, can be implemented quickly, and has high reliability.

[0105] To better understand the "metadata" in the above embodiments, the metadata is explained below.

[0106] The metadata in this embodiment may include: global metadata and replica 1-n metadata; the global metadata includes: the overall length of the data and the overall data digest hash value; the replica 1-n metadata includes: metadata segments 1-n (starting position and continuous length) and metadata segment index (segment number and actual order).

[0107] The cloud storage servers in the above embodiments use the same data digest algorithm, i.e., a unified standard algorithm, when obtaining hash values. This data digest algorithm supports calculating digest hash values ​​based on specified data segments (continuous data segments starting from a specified position).

[0108] The data digest algorithm takes a two-dimensional array of type "int[n][2]" as its parameter. One dimension (n) represents multiple data segments, and each two-dimensional array ({start position, length}) represents the specific starting position and continuous length of the data segment. The data digest algorithm needs to concatenate all the data in the input parameter data segment order to calculate the digest hash value.

[0109] Example parameters:

[0110] Suppose we want to calculate the parameters of three consecutive data segments as follows:

[0111]

[0112] Parameter meaning: Read 5000 consecutive data units from the starting position 100, then read 20000 consecutive data units from the starting position 6000, and then read 100 consecutive data units from the starting position 90000. Calculate the digest hash value using all the above data and return it.

[0113] The process by which each cloud storage server obtains the hash value of a specified data segment according to the calculation instructions may include: referring to the cloud storage replica metadata, calculating the actual corresponding segment group of the data replica after the cloud storage replica is restored, forming a two-dimensional array of type "int[n][2]", determining the start and end positions of the specified data segment, and using a data digest algorithm to obtain the digest hash value of the specified data segment, i.e., the hash value.

[0114] The above-mentioned specified data segment can be defined by randomly dividing the original data into n segments based on the original data length, thereby determining the start and end positions of the specified data segment.

[0115] If hash values ​​need to be calculated for all data segments of the restored cloud storage copy, a data digest algorithm is used to traverse all data segments that make up the complete data copy, obtaining the hash value of each segment. The hash values ​​of each segment from multiple cloud storage copies are compared. If the hash values ​​of each segment are the same, the overall hash value should also be the same. This full-scale approach proves the consistency of the entire copy data. By obtaining hash values ​​by data segment for comparison, the damaged areas of incomplete copies can be quickly and accurately analyzed, facilitating rapid and accurate recovery.

[0116] Based on the method described in the above embodiments, the user side only needs to rely on the cloud storage copy to "convert metadata documents" and restore the data storage location of the cloud storage copy. There is no need to save the original data. The digest hash calculation of the audit data segment is also performed in the cloud, i.e., on the cloud storage server, with the user side only responsible for a small amount of computation. Data integrity verification can be performed by cross-checking the digest hash values ​​of multiple independent data copies.

[0117] It should be noted that, for the sake of simplicity, the method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of the present invention are not limited to the described order of actions, because according to the embodiments of the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily essential to the embodiments of the present invention.

[0118] Reference Figure 4 The diagram illustrates a structural block diagram of a data integrity verification device based on an independent copy, provided in an embodiment of the present invention. Specifically, it may include the following units:

[0119] The metadata acquisition unit is used to acquire metadata from at least two cloud storage replicas when responding to a data integrity verification command in cloud storage.

[0120] The hash value acquisition unit is used to send calculation instructions to the cloud storage server of each cloud storage replica based on the metadata of that replica. These instructions include: the metadata of the corresponding cloud storage replica, a specified data digest algorithm, and information on calculating the hash value of a specified data segment in the data replica using the specified data digest algorithm. The data replica is the data in the cloud storage replica restored to its original storage order according to the metadata. The data digest algorithm, also known as a hash function, is an algorithm that converts data of arbitrary length into a fixed-length output, typically called a hash value or message digest. Furthermore, since a message digest algorithm uniquely corresponds to a fixed-length value for a message or text, and is generated by applying a one-way hash encryption function to the message, different input messages will produce different digest messages; however, the same input will always produce the same output. In this embodiment, the hash value acquisition unit can use MD5, SHA-1, SHA-2, SHA-3, or RIPEMD-160 as the specified data digest algorithm. However, during data integrity verification, SHA-256 (Secure Hash Algorithm 256-bit) is typically chosen. SHA-256 is a variant of SHA-2, offering higher security and better collision resistance. Compared to older algorithms like MD5 and SHA-1, SHA-256 provides a longer output length and a more complex calculation method, making it difficult to crack. When the original data acquisition unit responds to the data integrity verification command in cloud storage, obtains the metadata of two cloud storage copies, and sends it to the hash value acquisition unit, it calculates the hash value of the data copies using the SHA-256 algorithm, generating a 256-bit digest for verifying data integrity. By calculating the SHA-256 hash value of the data tokens from different cloud storage copies according to their original storage order, these hash values ​​can be compared to confirm data consistency, thus effectively verifying data integrity. Of course, other types of data digest algorithms can also be selected based on security, performance, compatibility, and usage scenarios.

[0121] The comparison unit receives the hash value returned by the cloud storage server according to the calculation instructions and determines whether the hash values ​​returned by all cloud storage servers are consistent. If they are consistent, it is determined that the data in the cloud storage replica is complete. If the hash values ​​are inconsistent, when there are three cloud storage replicas, if two cloud storage servers return the same hash value, it can be determined that the cloud storage replicas of the inconsistent cloud storage servers have data incompleteness.

[0122] When there are only two cloud storage replicas and the hash values ​​of the two cloud storage replicas are inconsistent, the hash value of the data summary on the user side is used for further judgment.

[0123] At this time, the hash value acquisition unit is also used to directly calculate the hash value of the user-side data summary based on the metadata of the user-side data summary sent by the metadata acquisition unit; and send new calculation instructions to each cloud storage server respectively, so that the cloud storage server calculates the hash value of the data summary in the data copy based on the new calculation instructions and using the specified data digest algorithm.

[0124] The comparison unit is also used to receive the hash value calculated locally by the user and the hash value returned by the cloud storage server, and compare them. If there is an inconsistency, it can be confirmed that the data of the non-local cloud storage copy is incomplete.

[0125] The metadata acquisition unit in this embodiment may include metadata stored locally or acquired from a cloud storage server, as described in the above method embodiment.

[0126] Furthermore, the hash value acquisition unit in this embodiment mainly sends calculation instructions to each cloud storage server. Each cloud storage server calculates the same data segment in its own data copy based on the same data digest algorithm in the calculation instructions to obtain the hash value, thereby enabling subsequent comparison and thus realizing the integrity and reliability verification of the data copy.

[0127] Furthermore, the apparatus of this embodiment also includes a parsing and recovery unit, which is used to obtain the hash values ​​of each data segment to which all cloud storage replicas belong when the comparison unit determines that there is an inconsistency in the hash values ​​returned by all cloud storage servers, and to parse out the data area of ​​the cloud storage replica with errors based on the hash values ​​of each data segment to which all cloud storage replicas belong; and then to recover the data in the data area based on the data area of ​​the cloud storage replica with errors and the data replica converted from the cloud storage replica without errors.

[0128] Furthermore, the hash value acquisition unit is also used to determine whether all cloud storage replicas are provided by the same cloud storage server based on the metadata of each cloud storage replica; if they are provided by the same cloud storage server, it receives the original data replica sent by the metadata acquisition unit, uses a specified data digest algorithm to obtain the hash value of a specified data segment in the original data replica; at the same time, it sends a calculation instruction to the cloud storage server so that the cloud storage server calculates the hash value of the data summary in the data replica based on the calculation instruction and using the specified data digest algorithm.

[0129] The comparison unit is also used to determine whether the hash value returned by each cloud server is consistent with the hash value corresponding to the specified data segment on the user side; if they are consistent, it is determined that the data in the cloud storage replica is complete.

[0130] Furthermore, when the metadata acquisition unit acquires only one cloud storage copy, the hash value acquisition unit sends a calculation instruction to the cloud storage server of the cloud storage copy based on the metadata of the cloud storage copy. The calculation instruction at this time includes: the metadata of the corresponding cloud storage copy, the specified data digest algorithm, and the information of calculating the data summary hash value in the data copy using the specified data digest algorithm. The data copy is the data of the cloud storage copy after restoring the original storage order according to the metadata.

[0131] Then, the hash value acquisition unit is also used to calculate the hash value of the local original data summary sent by the metadata acquisition unit using a specified data digest algorithm; at the same time, it sends a calculation instruction to the cloud storage server of the cloud storage copy, so that the cloud storage server calculates the hash value of the data summary in the data copy based on the calculation instruction and using the specified data digest algorithm.

[0132] The comparison unit is also used to determine whether the hash value returned by the cloud storage server is consistent with the hash value of the calculated local original data summary; if they are consistent, it is determined that the data in the cloud storage copy is complete.

[0133] In other words, any document that needs to be stored in the cloud on the user side has a data summary information on the user side.

[0134] The device in this embodiment verifies the validity of data by cross-referencing data copies. The data copies are converted and stored, and the auditing method adopts a random data segment full coverage approach, which can verify the integrity 100%.

[0135] Based on the apparatus of this embodiment, the audited cloud storage server cannot predict the data content to be audited, nor can it directly refer to a certain cloud storage copy to calculate the audit data segment digest hash data, thus ensuring that the data audit results cannot be forged or predicted.

[0136] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0137] In addition, embodiments of the present invention also provide an electronic device, such as... Figure 5 As shown, it includes a processor 1301, a communication interface 1302, a memory 1303, and a communication bus 1304. The processor 1301, the communication interface 1302, and the memory 1303 communicate with each other through the communication bus 1304.

[0138] Memory 1303 is used to store computer programs;

[0139] When processor 1301 executes a program stored in memory 1303, it performs the following steps:

[0140] When the user responds to a data integrity verification command in cloud storage, it obtains metadata from at least two cloud storage copies.

[0141] Based on the metadata of each cloud storage replica, a calculation instruction is sent to the cloud storage server of the cloud storage replica. The calculation instruction includes: the metadata of the corresponding cloud storage replica, a specified data digest algorithm, and information on calculating the hash value of a specified data segment in the data replica using the specified data digest algorithm. The data replica is the data of the cloud storage replica after restoring the original storage order according to the metadata.

[0142] The system receives the hash value returned by the cloud storage server based on the calculation instructions and determines whether the hash values ​​returned by all cloud storage servers are consistent. If they are consistent, the system determines that the data in the cloud storage replica is complete.

[0143] The communication bus mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0144] The communication interface is used for communication between the aforementioned terminal and other devices.

[0145] The memory may include random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0146] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0147] like Figure 6 As shown, in another embodiment of the present invention, a computer-readable storage medium 1401 is also provided, which stores instructions that, when executed on a computer, cause the computer to perform the following steps:

[0148] When the user responds to a data integrity verification command in cloud storage, it obtains metadata from at least two cloud storage copies.

[0149] Based on the metadata of each cloud storage replica, a calculation instruction is sent to the cloud storage server of the cloud storage replica. The calculation instruction includes: the metadata of the corresponding cloud storage replica, a specified data digest algorithm, and information on calculating the hash value of a specified data segment in the data replica using the specified data digest algorithm. The data replica is the data of the cloud storage replica after restoring the original storage order according to the metadata.

[0150] The system receives the hash value returned by the cloud storage server based on the calculation instructions and determines whether the hash values ​​returned by all cloud storage servers are consistent. If they are consistent, the system determines that the data in the cloud storage replica is complete.

[0151] As the computer-readable storage medium 1401 embodiment is basically similar to the method embodiment, the description is relatively simple, and the relevant details can be found in the method embodiment section.

[0152] In another embodiment of the present invention, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to perform the following steps:

[0153] When the user responds to a data integrity verification command in cloud storage, it obtains metadata from at least two cloud storage copies.

[0154] Based on the metadata of each cloud storage replica, a calculation instruction is sent to the cloud storage server of the cloud storage replica. The calculation instruction includes: the metadata of the corresponding cloud storage replica, a specified data digest algorithm, and information on calculating the hash value of a specified data segment in the data replica using the specified data digest algorithm. The data replica is the data of the cloud storage replica after restoring the original storage order according to the metadata.

[0155] The system receives the hash value returned by the cloud storage server based on the calculation instructions and determines whether the hash values ​​returned by all cloud storage servers are consistent. If they are consistent, the system determines that the data in the cloud storage replica is complete.

[0156] For computer program product embodiments containing instructions, since they are basically similar to method embodiments, the description is relatively simple, and the relevant expenses can be found in the method embodiment section.

[0157] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0158] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0159] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0160] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A data integrity verification method based on independent copies, characterized in that, include: When the user responds to a data integrity verification command in cloud storage, it obtains metadata from at least two cloud storage copies. Based on the metadata of each cloud storage replica, a calculation instruction is sent to the cloud storage server of the cloud storage replica. The calculation instruction includes: the metadata of the corresponding cloud storage replica, a specified data digest algorithm, and information on calculating the hash value of a specified data segment in the data replica using the specified data digest algorithm. The data replica is the data of the cloud storage replica after restoring the original storage order according to the metadata. The metadata can be any of the following: data describing data copies, data transformation rules defined in documents, or data in sequence; Receive the hash value returned by the cloud storage server based on the calculation instructions, and determine whether the hash values ​​returned by all cloud storage servers are consistent; If they match, then the data in the cloud storage copy is confirmed to be complete; On the user side, the storage order of the original data to be stored in the cloud is transformed based on the storage order transformation method to obtain a cloud storage copy and the metadata of the cloud storage copy.

2. The method according to claim 1, characterized in that, The method further includes: If at least two cloud storage copies are provided by the same cloud storage server, and the user's local storage has a copy of the original data, then the hash value of the specified data segment in the original data copy is obtained using the specified data digest algorithm. Determine whether the hash values ​​returned by each cloud storage server are consistent with the hash values ​​corresponding to the specified data segments on the user side; If they match, then the data in the cloud storage copy is confirmed to be complete.

3. The method according to claim 1, characterized in that, If the user does not have a local copy of the original data, then obtain metadata from at least two cloud storage copies, including: Obtain metadata corresponding to cloud storage replicas on different cloud storage servers.

4. The method according to claim 1, characterized in that, If only one cloud storage copy exists, the calculation instructions include: the metadata of the corresponding cloud storage copy, the specified data digest algorithm, and information on calculating the data summary hash value in the data copy using the specified data digest algorithm. The data copy is the data in the cloud storage copy after restoring the original storage order according to the metadata. Accordingly, the method further includes: The hash value of the user-side local raw data summary is calculated using a specified data digest algorithm; Determine whether the hash value returned by the cloud storage server is consistent with the hash value calculated by the user. If they match, then the data in the cloud storage copy is confirmed to be complete.

5. The method according to claim 1, characterized in that, The method further includes: The cloud storage copy is sent to the cloud storage server, and the metadata is stored on the user side; Any two cloud storage copies are different; Accordingly, obtain metadata for at least two cloud storage replicas, including: Obtain metadata for at least two cloud storage copies of the user-side storage.

6. The method according to claim 1, characterized in that, The method further includes: The cloud storage copy and its metadata are combined according to predefined rules to form the storage text to be sent to the cloud storage. Send the stored text to the cloud storage server; Any two cloud storage copies are different; Accordingly, obtain metadata for at least two cloud storage replicas, including: Send a metadata retrieval request to the cloud storage server. The metadata retrieval request includes: the data storage location of the metadata in the storage file, determined according to predefined rules, and the metadata download information in the data storage location. Receives metadata of cloud storage copies returned by each cloud storage server based on the metadata retrieval request.

7. The method according to any one of claims 1 to 6, characterized in that, The method further includes: If inconsistencies exist, obtain the hash values ​​of each data segment to which all cloud storage replicas belong, and based on the hash values ​​of each data segment to which all cloud storage replicas belong, parse out the data area of ​​the cloud storage replica with the error. The data in the data area is recovered based on the data area of ​​the cloud storage copy with the storage error and the data copy converted from the cloud storage copy without the error.

8. A data integrity verification device based on independent copies, characterized in that, include: The metadata acquisition unit is used to acquire metadata from at least two cloud storage replicas when responding to a data integrity verification command in cloud storage. The hash value acquisition unit is used to send a calculation instruction to the cloud storage server of each cloud storage replica based on the metadata of each cloud storage replica. The calculation instruction includes: the metadata of the corresponding cloud storage replica, a specified data digest algorithm, and information on calculating the hash value of a specified data segment in the data replica using the specified data digest algorithm. The data replica is the data of the cloud storage replica after restoring the original storage order according to the metadata. The metadata is any one of the following: data describing the data replica, document defining data transformation rules, and order. The comparison unit is used to receive the hash value returned by the cloud storage server according to the calculation instruction, and to determine whether the hash values ​​returned by all cloud storage servers are consistent; if they are consistent, it is determined that the data in the cloud storage replica is complete. The device is also used to perform storage order transformation on the user side based on the storage order transformation method on the original data to be stored in the cloud, so as to obtain a cloud storage copy for cloud storage and the metadata of the cloud storage copy.

9. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; The memory is used to store computer programs; When the processor executes a program stored in the memory, it implements the method as described in any one of claims 1-7.

10. A computer-readable medium having instructions stored thereon that, when executed by one or more processors, cause the processors to perform the method as described in any one of claims 1-7.

Images