Data verification method and device

Abstract

The embodiment of the invention discloses a data verification method and device. The data verification method and device are used for improving the flexibility of data verification. The method provided by the embodiment of the invention comprises the following steps: the server receives a data downloading request sent by a client, wherein the data downloading request is used for requesting to download data in a specified range of the server; and the server generates a first check code based on the data in the specified range, wherein the first check code is used for checking the data consistency between the sending data of the server and the receiving data of the client. And the server sends the data in the specified range and the first check code to the client, so that the client checks the data in the specified range based on the first check code.

Description

[0001] This application claims the priority of the Chinese patent application filed with the State Intellectual Property Office on November 22, 2023, with application number 202311572639.2 and invention name “A method, device and other equipment for data processing”, all contents of which are incorporated by reference in this application. Technical Field

[0002] The embodiments of the present application relate to the field of cloud computing, and in particular to a data verification method and device. Background Art

[0003] With the development of cloud computing technology, cloud storage, as an important part of cloud computing technology, has been more and more widely used. In the application of cloud storage, when the client downloads data from the server, due to network hijacking or data caching, the data downloaded by the client may be inconsistent with the original data on the server. Therefore, the client needs to perform consistency verification to ensure the correctness of the downloaded data.

[0004] At present, when the client performs consistency check on the downloaded data, the server generates a check code for the original data based on the original data, and sends the check code for the original data to the client, and the client performs consistency check on the received downloaded data according to the check code.

[0005] However, when downloading the original data from the server, the client often specifies the download range of the original data instead of downloading the complete original data. Therefore, the client cannot perform consistency verification on the downloaded data in the specified range based on the verification code of the original data, resulting in poor data verification flexibility of the client. Summary of the invention

[0006] The embodiment of the present application provides a data verification method, in which the server can generate a verification code for the data within the specified range of the client, thereby improving the verification flexibility during the data verification process. The embodiment of the present application also provides a data verification device, a computing device, a computing device cluster, a computer-readable storage medium, and a computer program product corresponding to the data verification method.

[0007] In the first aspect, an embodiment of the present application provides a data verification method, which can be executed by a server of a cloud storage system, or by a component of the server of the cloud storage system, such as a processor, chip or chip system of the server of the cloud storage system, or can be implemented by a logic module or software that can implement all or part of the server functions of the cloud storage system. The method provided in the first aspect is applied to a server of a cloud storage system, and the cloud storage system also includes a client. The method provided in the first aspect includes: the server receives a data download request sent by the client, and the data download request is used to request downloading of data within a specified range of the server. The server generates a first verification code based on the data within the specified range, and the first verification code is used to verify the data consistency between the data sent by the server and the data received by the client. The server sends the data within the specified range and the first verification code to the client, so that the client verifies the data within the specified range based on the first verification code.

[0008] In the embodiment of the present application, the client of the cloud storage system can download data within a specified range from the server. At the same time, the server can generate a verification code for the data within the specified range separately and send it to the client, and the client verifies the downloaded data. Compared with the current solution in which the server only generates a verification code for the complete original data, in the embodiment of the present application, the server can generate a verification code for the data within the specified range, thereby improving the data verification flexibility of the client and server in the cloud storage system during data transmission.

[0009] In a possible implementation, after the server sends data within a specified range and a first verification code to the client, the client generates a second verification code based on the data within the specified range, and the data within the specified range is the data received by the client. The client verifies the consistency of the data within the specified range based on the first verification code and the second verification code. If the first verification code is consistent with the second verification code, it is determined that the data received by the client is consistent with the data sent by the server.

[0010] After the client of the cloud storage system in the embodiment of the present application receives the data sent by the server, it can regenerate a verification code based on the received data, and compare the generated verification code with the verification code sent by the server to verify the data within the specified range sent by the server, thereby improving the accuracy of the client's data verification.

[0011] In one possible implementation, when the server sends data within a specified range and a first verification code to a client, the server splits the data within the specified range into multiple data blocks, and sends the multiple data blocks and the first verification code to the client, wherein the first verification code is located in the tail data block among the multiple data blocks.

[0012] In the embodiment of the present application, when the server sends data within a specified range and a first verification code to the client, the data within the specified range and the first verification code can be sent in blocks based on the block transfer protocol, thereby eliminating the need to store the entire data within the specified range into the memory before sending it, thereby reducing the memory utilization of the server and reducing the response delay during data download.

[0013] In one possible implementation, the data block sent by the server to the client includes two parts, wherein the first part is a message header field, which is used to indicate the valid data length of the data block, or to indicate that the data block contains a check code, and the second part is a data part, i.e., valid data or a check code carried by the data block, and the valid data is the data in the specified range requested to be downloaded by the client. The first check code is located in the data part of the tail block, and since the message header field of the data block carrying the valid data is different from the message header field of the data block carrying the first check code, the client can identify the tail data block based on the message header field of the data block.

[0014] In the embodiment of the present application, when the server transmits data within a specified range to the client in blocks, the content of the data block can be indicated by the message header field of the data block, so that the client can identify the content of the data block based on the message header field of the data block, thereby improving the data download efficiency of the cloud storage system.

[0015] In a possible implementation, the server sends data within a specified range and a first checksum to the client based on a block transfer encoding mechanism of the hypertext transfer protocol HTTP / 1.1. In the block transfer encoding mechanism, the server divides the data into a series of blocks of indefinite length, each of which is separated by a specific mark so that it can be transmitted in real time during the data transmission process.

[0016] In the embodiment of the present application, the server can send data within a specified range and a first check code to the client based on the block transfer encoding mechanism of the hypertext transfer protocol HTTP / 1.1, thereby improving the real-time performance of data transmission.

[0017] In one possible implementation, during the process of the server generating a first verification code based on data within a specified range, when the server determines that the data within the specified range has been modified, the server generates a first verification code in real time based on the sent data blocks before sending the tail data blocks, and the sent data blocks are the modified data within the specified range.

[0018] In the embodiment of the present application, the server can generate a check code in real time during the process of sending data blocks. Since the check code is carried in the tail data block, the server can generate the check code in real time based on the check of the sent data blocks, thereby avoiding check code errors caused by modifications to the data within the specified range. The server generates the check code in real time based on the check of the sent data blocks, which improves the accuracy of the check code and further improves the accuracy of the data verification process.

[0019] In one possible implementation, during the process of the server generating a first verification code based on data within a specified range, the server first reads the data within the specified range, then stores the complete data within the specified range into a memory, and calculates the first verification code based on the data within the specified range in the memory, and the server sends the complete data within the specified range and the first verification code to the client.

[0020] In the embodiment of the present application, the server can store the complete data within the specified range into the memory, and calculate the first verification code based on the data within the specified range in the memory, so as to avoid repeatedly reading the data in the specified range when calculating the first verification code and sending the data in the specified range to the client, thereby reducing the number of times the server reads the data in the specified range and improving the data verification efficiency.

[0021] In one possible implementation, after the server reads the data within a specified range, if the server's memory cannot buffer the complete data within the specified range, the server needs to first read the data within the complete instruction range to calculate a first verification code. After sending the first verification code to the client, the server re-reads the complete data within the specified range and sends it to the client.

[0022] In the embodiment of the present application, the server can directly read the data in the specified range and generate a first verification code without first storing the data in the specified range into the memory, thereby reducing the memory consumption of the server.

[0023] In a possible implementation, the server receives a data upload request sent by the client, the data upload request includes the uploaded data and a third verification code. The server generates a fourth verification code based on the uploaded data, and if the fourth verification code is consistent with the third verification code, the data received by the server is consistent with the data sent by the client.

[0024] In the embodiment of the present application, the server may also verify the received client uploaded data during the data upload process, thereby improving the applicability of the data verification method in the embodiment of the present application.

[0025] In one possible implementation, the storage type of data within the specified range includes one or more of the following: object storage and file storage. Object storage refers to the storage of data in the form of objects, which is suitable for large-capacity, unstructured data storage. File storage refers to the storage of data in the form of hierarchical folders and files, which is suitable for the storage of structured data with relatively fixed access patterns and for scenarios that require frequent reading and writing.

[0026] The data verification method provided in the embodiment of the present application is applicable to various types of data storage scenarios, thereby improving the applicability of the data verification method provided in the embodiment of the present application.

[0027] In a possible implementation, the first check code and the second check code both include one or more of the following: a message digest algorithm version 5 MD5 check code, a cyclic redundancy CRC check code, and a secure hash algorithm SHA check code. The check algorithm of the first check code is the same as the check algorithm of the second check code. Among them, the message digest algorithm version 5 MD5 algorithm is a check code check algorithm generated by a hash function, the cyclic redundancy CRC algorithm is an algorithm that generates a check code by performing polynomial division on a data block, and the secure hash algorithm SHA algorithm includes a series of algorithms that generate check codes based on cryptographic hash functions, including SHA-1, SHA-256, and SHA-512.

[0028] In the embodiment of the present application, the server and the client can generate verification codes based on a variety of different types of verification algorithms, thereby improving the richness of the verification codes generated by the client and the server in the embodiment of the present application.

[0029] In one possible implementation, before the server sends multiple data blocks to the client, the server sends a response message to the client for the data download request, and the message header field of the response message includes a block transfer identification field, and the block transfer identification field is used to indicate to the client that the server uses the block transfer protocol to transmit a specified range of data and a first check code.

[0030] In an embodiment of the present application, before the server transmits data in blocks to the client, it can send a response message corresponding to the data download request to the client, and instruct the server through the response message to send data within a specified range and a first verification code, thereby improving the feasibility of the server transmitting data within the specified range and the first verification code in blocks.

[0031] In a second aspect, an embodiment of the present application provides a data verification device, which includes a transceiver unit and a processing unit. The transceiver unit is used to receive a data download request sent by a client, and the data download request is used to request to download data within a specified range of a server. The processing unit is used to generate a first verification code based on the data within the specified range, and the first verification code is used to verify the data consistency between the data sent by the server and the data received by the client. The transceiver unit is also used to send the data within the specified range and the first verification code to the client, so that the client verifies the data within the specified range based on the first verification code.

[0032] In a possible implementation, the processing unit is specifically configured to split the data within a specified range into multiple data blocks, and send the multiple data blocks and a first check code to the client, wherein the first check code is located in a tail data block among the multiple data blocks.

[0033] In a possible implementation, each data block includes a message header field and a data portion, the first check code is located in the data portion of the tail data block, and the message header field of the tail data block is used by the client to identify the tail data block.

[0034] In a possible implementation, the processing unit is specifically configured to determine whether data within a specified range has been modified, and the server generates a first check code based on the sent data blocks before sending the tail data blocks.

[0035] In a possible implementation, data within a specified range is used by the client to generate a second verification code. If the second verification code is consistent with the first verification code, the data received by the client is consistent with the data sent by the server.

[0036] In a possible implementation manner, the transceiver unit is specifically configured to send the data within a specified range and the first check code to the client based on a block transfer encoding mechanism of the hypertext transfer protocol HTTP / 1.1.

[0037] In a third aspect, an embodiment of the present application provides a computing device, comprising a processor, the processor being coupled to a memory, the processor being used to store instructions, and when the instructions are executed by the processor, the computing device executes the method described in the first aspect or any possible implementation manner of the first aspect.

[0038] In a fourth aspect, an embodiment of the present application provides a computing device cluster, the computing device cluster includes one or more computing devices, the computing device includes a processor, the processor is coupled to a memory, the processor is used to store instructions, when the instructions are executed by the processor, the computing device cluster executes the method described in the first aspect or any possible implementation method of the first aspect.

[0039] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium having instructions stored thereon. When the instructions are executed, the computer executes the method described in the first aspect or any possible implementation manner of the first aspect.

[0040] In a sixth aspect, an embodiment of the present application provides a computer program product, which includes instructions. When the instructions are executed, the computer implements the method described in the first aspect or any possible implementation method of the first aspect.

[0041] It can be understood that the beneficial effects that can be achieved by any of the data verification devices, computing devices, computing device clusters, computer-readable media or computer program products provided above can refer to the beneficial effects in the corresponding methods and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1a A schematic diagram of the system architecture of a cloud storage system provided in an embodiment of the present application;

[0043] Figure 1b A schematic diagram of a flow chart of another data verification method provided in an embodiment of the present application;

[0044] Figure 1c A schematic diagram of an object storage bucket provided in an embodiment of the present application;

[0045] Figure 2 A schematic diagram of a data verification method provided in an embodiment of the present application;

[0046] Figure 3 A schematic diagram of a flow chart of another data verification method provided in an embodiment of the present application;

[0047] Figure 4 A schematic diagram of a flow chart of another data verification method provided in an embodiment of the present application;

[0048] Figure 5 A schematic diagram of the structure of a data verification device provided in an embodiment of the present application;

[0049] Figure 6 A schematic diagram of the structure of a computing device provided in an embodiment of the present application;

[0050] Figure 7 A schematic diagram of the structure of a computing device cluster provided in an embodiment of the present application;

[0051] Figure 8 A schematic diagram of the structure of another computing device cluster provided in an embodiment of the present application. DETAILED DESCRIPTION

[0052] The embodiments of the present application provide a data verification method and device for improving the flexibility of data verification.

[0053] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0054] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0055] First, some terms involved in the embodiments of the present application are introduced to facilitate technical personnel in this field to understand the technical solution.

[0056] Object storage refers to storing data in the form of objects. Each object has a unique identifier that can be accessed and operated through the identifier. Object storage is usually used to store unstructured data, such as pictures, videos, audio, documents, etc.

[0057] File storage refers to managing data through the file system. Data is stored in the form of files on the cloud server. Files can be modified, deleted, and queried anytime and anywhere.

[0058] Range download refers to the support in the field of object and file storage that allows the client to download data within a specified range when downloading specified data, such as from the 5th byte to the 100th byte.

[0059] Metadata refers to descriptive information about an object in the object storage field, such as the object's name, size, creation time, access permissions, etc.

[0060] A checksum is usually a specific code or number used to verify data integrity and accuracy. The checksum is transmitted or stored together with the data. Checksums include checksums, cyclic redundancy checks, and hash values.

[0061] Consistency check refers to the consistency check between the client and the server based on the check code when the data is transmitted. For example, when the server receives the file uploaded by the client, it will compare the received check code with the check code set by the client. The upload is considered successful only when the two are consistent. Or, when the client downloads the file, it will compare the check code sent by the server with the check code calculated by the client. The download is considered successful only when the two are consistent, thereby ensuring data consistency.

[0062] In order to make the technical solution of the present application clearer and easier to understand, the system architecture of the present application is introduced below with reference to the accompanying drawings.

[0063] See also Figure 1a , Figure 1a A schematic diagram of the system architecture of a cloud storage system provided as an example of this application. Figure 1a In the example shown, the cloud storage system 10 includes a client 101 and a server 102, wherein the server 102 includes a storage server 1021 and a metadata server 1022. The specific functions of each part of the cloud storage system 10 are described below.

[0064] The client 101 is used to provide the user with an interface for accessing the cloud storage system 10, and the user can interact with the server 102 through the client 101. For example, the user can upload, download, query and modify data on the server 102 through the client 101. When the user uploads data to the server 102 through the client 101, the client 101 is also used to slice the uploaded data and send the data slices to the server 102.

[0065] The client 101 is also used to perform data consistency check on the download data sent by the server 102. The client 101 calculates a check code based on the received download data, and compares the calculated check code with the check code sent by the server 102. When the two check codes are consistent, the client 101 determines that the data download is successful.

[0066] The client 101 is also used to implement functions such as identity authentication, access control, and data encryption, thereby ensuring that users can safely access and operate data stored in the server 102. For example, the client 101 can define and manage the access control policy of the server 102, specifying access rights for different data objects, such as read rights, write rights, and delete rights, so that authorized users or applications can access specific data.

[0067] It is understandable that the client 101 in the embodiment of the present application can be a terminal device, such as a personal computer, a smart phone or a tablet computer, or a software client on the terminal device, such as an application, without specific limitation.

[0068] The server 102 includes a storage server 1021 and a metadata server 1022. The storage server 1021 is used to implement functions such as data storage and optimized data distribution. The metadata server 1022 is used to control the interaction between the client 101 and the storage server 1021, including processing requests from the client 101, maintaining metadata information and directory structure of storage objects, etc.

[0069] The storage server 1021 is used to implement data storage and management. For example, the storage server 1021 actually stores the data uploaded by the client 101 and manages the stored data. When the client 101 requests to upload data, the storage server 1021 can receive the data and store it on a persistent storage medium, such as a disk array. The storage server 1021 is also used to distribute data to different nodes to achieve storage scalability and load balancing. Data distribution can be based on a consistent hashing algorithm or other distributed algorithms to ensure that data is evenly distributed in the cluster.

[0070] The metadata server 1022 is used to store and maintain metadata information of the stored objects, such as the name, size, creation time, access rights, etc. of the object. The metadata information is usually stored in the form of structured data to facilitate indexing and querying. The metadata server 1022 is also used to create indexes and provide metadata search functions. For example, the client 101 can search metadata based on the attributes or keywords of the object to locate and access a specific object.

[0071] The metadata server 1022 is also used to perform access control and authority management of metadata information. For example, the metadata server 1022 can control the client 101's access to metadata information through an access control list and authority management mechanism, allowing authorized users or applications to read, modify or delete metadata information.

[0072] It should be noted that the cloud storage system 10 in the embodiment of the present application can be applied to the object storage field and the file storage field. In the object storage field, the data in the server 102 is stored in the form of objects, each object contains the data itself, metadata and a unique identifier, and the server 102 accesses and manages the data through a unique identifier. The object storage field is suitable for storing large-capacity, unstructured data and has high scalability and persistence.

[0073] In the field of file storage, data is stored in the form of hierarchical folders and files in the server 102. The file storage field is suitable for storing structured data with relatively fixed access patterns, and can provide low-latency file access speed, which is suitable for scenes that require frequent reading and writing. It is understandable that when the cloud storage system 1021 is applied to the field of file storage, the server 102 in the cloud storage system 10 may only include the storage server 1021.

[0074] See also Figure 1b , Figure 1b A schematic diagram of a process for verifying downloaded data by a client provided in this application. Figure 1b In the example shown, the client 101 sends a data download request to the server 102, and the data download request is used to request to download the object data in the object storage bucket. Since the downloaded data is object data, the client 101 first needs to obtain the object information of the download object before downloading the data, including determining the unique identifier or path of the download object and related information such as the object size, and then the client 101 sends a data download request to the server based on the object information, thereby obtaining the data content of the object.

[0075] If the local MD5 value is consistent with the MD5 value sent by the server 102, the client 101 Figure 1b In the example shown, the server 102 generates an MD5 value corresponding to the download data based on the download data, and the MD5 value is the verification code of the verification object data. The server 102 sends the MD5 value to the terminal device 101. After receiving the download data, the client 101 generates a local MD5 value based on the download object. The client 101 compares the generated local MD5 value with the MD5 value sent by the server 102. If the received download object is consistent with the download object sent by the server 102, the download is successful. If they are inconsistent, the download fails, and the client 101 can resend a data download request to the server 102.

[0076] In the current technology, in the data download request sent by the client 101 to the server 102, the server 102 can only provide the MD5 value of the complete data, but cannot send the MD5 value of the data within the range specified by the client 101 to the client 101, resulting in the client 101 being unable to complete the data consistency check. At the same time, since the downloaded data may be modified, the MD5 value of the actual data is inconsistent with the MD5 value sent by the server 102 to the client 101, resulting in the failure of the download verification.

[0077] See also Figure 1c , Figure 1c A schematic diagram of an object storage bucket provided in an embodiment of the present application. Figure 1cIn the example shown, service restart 102 includes one or more data storage buckets, each object storage bucket contains one or more objects, an object is a basic data unit stored in an object storage bucket, and an object can be any type of unstructured data such as files, documents, pictures, videos, etc.

[0078] Each object includes data, metadata, and an identifier. Data refers to the actual content of the object, that is, the file or data itself that the user wants to store and retrieve. Metadata is descriptive information associated with the object, including the object's size, creation time, last modification time, file type, and storage category. The identifier is used to uniquely identify the object in the bucket, and each object has a unique identifier.

[0079] based on Figure 1a The cloud storage system 10 shown in the present application also provides a data verification method. The data verification method provided by the embodiment of the present application is introduced below in conjunction with the embodiment.

[0080] See also Figure 2 , Figure 2 A flow chart of a data verification method provided in an embodiment of the present application. Figure 2 In the example shown, the method includes the following steps:

[0081] Step 201: The client sends a data download request to the server, where the data download request is used to request downloading of data within a specified range of the server.

[0082] The client 101 sends a data download request to the server 102. The data download request is used to request to download data within a specified range of the server 102. The specified range is the starting position and the ending position of the data that the client 101 wants to download. The specified range is, for example, the 5th byte to the 100th byte. The data in the specified range includes the download object in the object storage field and the file data in the file storage field.

[0083] In a possible implementation, in the field of object storage, the client 101 sends a data download request to the server 102, and the data download request also includes an identifier of the download object, and the identifier is used to indicate the object to be downloaded by the client 101, and the identifier is, for example, the object name and ID. After receiving the data download request sent by the client 101, the server 102 parses the data download request to obtain the identifier and the specified range, and determines the location and size of the download data according to the identifier and the specified range in the data download request.

[0084] Specifically, the server 102 queries the metadata server 1022 for metadata corresponding to the downloaded data according to the identifier in the data download request, wherein the storage location of the downloaded data in the storage server 1021 is determined according to the metadata, including the storage node or storage device where the downloaded data is located. The server 102 further determines the actual data to be read according to the specified range in the data download request, the specified range includes an offset and a data size, and the data determined by the server 102 according to the specified range is the data that the server 102 is to send to the client 101.

[0085] See also Figure 3 , Figure 3 A flow chart of another data verification method provided in an embodiment of the present application. Figure 3 In step a of the illustrated example, the client 101 sends a data download request to the server 102 , where the data download request is used to request the server 102 to download data within a specified range, such as data from the 5th byte to the 500th byte.

[0086] Step 202: The server generates a first verification code based on data within a specified range, where the first verification code is used to verify data consistency between data sent by the server and data received by the client.

[0087] After the server 102 determines the data within the specified range, it generates a first verification code based on the data within the specified range. The first verification code is used to verify the data consistency between the data sent by the server 102 and the data actually received by the client 101. The first verification code can be, for example, a character string generated by processing the data within the specified range based on a verification algorithm.

[0088] In the embodiment of the present application, the first verification code can be a character string generated based on multiple verification algorithms, such as the message digest algorithm 5 (MD5) algorithm, the cyclic redundancy check (CRC) algorithm and the secure hash algorithm (SHA) algorithm, without specific limitation.

[0089] Among them, the fifth version of the message digest algorithm MD5 algorithm is a verification algorithm that uses a hash function to generate a check code, the cyclic redundancy CRC algorithm is an algorithm that generates a check code by performing polynomial division on a data block, and the secure hash algorithm SHA algorithm includes a series of algorithms that generate check codes based on cryptographic hash functions, including SHA-1, SHA-256 and SHA-512.

[0090] In one possible implementation, during the process of the server 102 generating the first verification code based on the data within the specified range, the server 102 first reads the data within the specified range, then stores the complete data within the specified range into the memory, and calculates the first verification code based on the data within the specified range in the memory. In this implementation, the server 102 may also directly transmit the complete data within the specified range to the client 101.

[0091] In the embodiment of the present application, the server 102 can store the data within the complete specified range to the memory, calculate the first verification code based on the data within the specified range in the memory, and directly read the data within the specified range from the memory and send it to the client 101, which can improve the efficiency of the server 101 in calculating the verification code and the data transmission efficiency.

[0092] In one possible implementation, if the memory of server 102 cannot buffer the complete data within the specified range, server 102 needs to first read the data within the complete instruction range to calculate the first verification code. After sending the first verification code to client 101, server 102 re-reads the data within the complete specified range and sends it to client 101. Since server 102 can directly read the data within the specified range and generate the first verification code without storing the data within the specified range in the memory first, the server's memory consumption is reduced. However, since repeated reading of data will cause the response delay of server 102 to increase, this implementation is not a preferred implementation.

[0093] It should be noted that the timing when the server 102 generates the first check code based on the data within the specified range can also be during the process of sending the data within the specified range, or after the data within the specified range is sent, and there is no specific limitation. For example, the server 102 can also split the data within the specified range and send the data within the specified range in blocks, and when the server 102 sends the data within the specified range in blocks, the server 102 generates the first check code based on the data within the specified range in real time, and places the first check code in the last data block to be sent.

[0094] Step 203: The server sends the data within the specified range and the first verification code to the client, so that the client verifies the data within the specified range based on the first verification code.

[0095] After the server 102 generates a first verification code based on the data in the specified range, the server 102 sends the data in the specified range and the first verification code to the client 101, so that the client 101 can verify the data in the specified range based on the first verification code. The data in the specified range that needs to be verified is the downloaded data received by the client 101.

[0096] Specifically, when sending data within a specified range and a first check code to the client 101, the server 102 splits the data within the specified range into multiple data blocks, and sends the multiple data blocks and a first check code to the client 101, wherein the first check code is located in the tail data block.

[0097] The data block in the embodiment of the present application includes two parts, wherein the first part is a message header field, which can be used to indicate the valid data length of the data block. The message header field can also be used to indicate that the data block contains a check code. For example, the message header field of the tail data block is "0". At this time, the message header field indicates that the data block carries a check code. The second part is the data part, that is, the valid data or check code carried by the data block. The valid data is the data in the specified range requested to be downloaded by the client 101.

[0098] In the embodiment of the present application, the server 102 sends data within a specified range and a first checksum to the client based on the block transfer encoding mechanism of the hypertext transfer protocol HTTP / 1.1. In the block transfer encoding mechanism, the server 102 divides the data into a series of data blocks of indefinite length, each of which is separated by a specific flag, and the message header field of each data block is a hexadecimal number, indicating the effective data length of the data block. When the message header field of the data block is "0", it indicates that the data block transmission is completed.

[0099] Please continue reading Figure 3 ,exist Figure 3 In step b of the example shown, when the server 102 sends data within a specified range to the client 101, the server 102 splits the data within the specified range and the first check code into multiple data blocks, and the server 102 sends the data blocks to the client 101. For example, the server 102 splits the data within the specified range and the first check code into x chunks, namely chunk1, ..., chunkx, wherein the first check code is carried in the last data block chunkx, and the data within the specified range is carried in multiple data blocks sent before chunkx.

[0100] exist Figure 3 In the example shown, each data block chunk includes a message header field and a data part, wherein the message header field of each chunk except the last chunk indicates the length of the data part in hexadecimal. The message header field of the last chunk is 0, indicating that the data part of the chunk is a checksum.

[0101] In a possible implementation, before the server 102 transmits the data blocks to the client 101, it may send a response message corresponding to the data download request to the client 101, and instruct the server to send the data within the specified range and the first check code through the response message. The message header field of the response message includes a block transmission identification field, and the block transmission identification field is used to indicate to the client 101 that the server 102 uses the block transmission protocol to transmit the data within the specified range and the first check code.

[0102] For example, before server 102 sends multiple data chunks to client 101, server 102 sends a response message of the data download request to client 101. Server 102 can add a "Transfer-Encoding: chunked" field in the response message header, and use the "chunked" keyword to indicate to client 101 that server 102 uses the chunked transfer protocol to transmit a specified range of data and a first checksum.

[0103] In one possible implementation, if the data within a specified range in the server 102 is modified, while the server 102 is sending the data within the specified range in blocks, before the server 102 sends the tail data block, since the first check code is in the tail data block, the server 102 can generate the first check code in real time based on the sent data block, and the sent data block is the data within the modified specified range.

[0104] See also Figure 4 , Figure 4 This is an example diagram of another data verification method provided in an embodiment of the present application. Figure 4 The example shown is a file storage scenario. Figure 4 In steps a to f of the illustrated example, after the client 101 sends a data download request to the server 102, the data download request is used to request the download of the file data uploaded by the client 103. Before the server 102 sends the file data requested for download to the client 101, the client 103 may modify the data in the server 102. For example, the client 103 may perform an additional write to the file data in the server 102, or perform a write modification operation on the file data in the server 102, so that the data within the specified range requested for download by the client 101 is modified.

[0105] exist Figure 4 In the example shown, during the process of server 102 sending data within a specified range and a first check code in blocks, since the data within the specified range may be modified, server 102 calculates the first check code in real time based on the sent data blocks, and carries the first check code in the last data block to be sent.

[0106] In the embodiment of the present application, since the check code is carried in the tail data block, the server can generate the check code in real time based on the sent data block check, thereby avoiding the error of the generated check code caused by the modification of the data within the specified range after the check code is generated. The server generates the check code in real time based on the sent data block check, which improves the accuracy of the check code and further improves the accuracy of the data verification process.

[0107] Step 204: The client generates a second verification code based on the data within the specified range.

[0108] After receiving the number within the specified range sent by the server 102, the client 101 generates a second check code based on the data within the specified range. The check algorithm used by the client 101 to generate the second check code is the same as the check algorithm used by the server 102 to generate the first check code.

[0109] Specifically, when the server 102 transmits data within a specified range and a first check code to the client 101 in blocks, after the client 101 receives the data blocks sent by the server 102, the client 101 parses the message header field in the data blocks to identify the data within the specified range and the first check code. If the message header field of the data block is a hexadecimal data length, the data block carries the data in the specified range. If the message header field of the data block is 0, the data block is a tail data block, and the data block carries the first check code.

[0110] After receiving the data blocks sent by the server 102, the client 101 reads the data part in the data blocks according to the block transmission protocol to obtain the data within the specified range. The client 101 generates a second check code based on the data within the specified range.

[0111] Please continue reading Figure 3 ,exist Figure 3 In the illustrated example, in step c to step d, the client 101 receives data blocks sent by the server 102, the data blocks include chunk1, ..., chunkx, and when the client 101 receives a data length in hexadecimal in the message header field, the data block carries data in a specified range. When the client 101 receives a data block with a message header field of 0, the data block is a tail data block, at which point the client 101 completes data reception and reads the first checksum sent by the server 102 from the tail data block.

[0112] exist Figure 3In step c and step d of the example shown, after the client 101 completes data reception, it splices the data parts of multiple data blocks to obtain download data, which is the data within the specified range requested by the client 101 to download. The client 101 generates a second verification code based on the download data, for example, the client generates an MD5 verification code based on the download data.

[0113] In the embodiment of the present application, the server 102 can indicate the content of the data block through the message header field of the data block, so that the client 101 can identify the content of the data block based on the message header field of the data block, and determine the tail data block and the first check code.

[0114] Step 205: The client verifies the consistency of the data in the specified range based on the first verification code and the second verification code. If the first verification code is consistent with the second verification code, it is determined that the data received by the client is consistent with the data sent by the server.

[0115] After the client 101 generates a second verification code based on the received download data, it verifies the data consistency between the downloaded data and the data within the specified range of the request based on the first verification code and the second verification code. If the first verification code is consistent with the second verification code, it is determined that the data received by the client 101 is consistent with the data sent by the server 102.

[0116] In the embodiment of the present application, after the client 101 receives the data sent by the server 102, it can generate a verification code again based on the received data, and compare the generated verification code with the verification sent by the server 102, thereby verifying the data within the specified range sent by the server 102, thereby improving the accuracy of the client's data verification.

[0117] Please continue reading Figure 3 ,exist Figure 3 In step e of the example shown, the client 101 performs verification calculations locally based on the received downloaded data, and after generating a second verification code, compares the second verification code with the first verification code sent by the server 102. If the second verification code is consistent with the first verification code, it means that the data within the specified range downloaded by the client 101 is the same as the data within the specified range sent by the server 102, and the data verification of the client 101 is successful. Otherwise, the data verification of the client 101 fails.

[0118] exist Figure 3 In the example shown, for example, the client 101 generates a verification code of MD5 value 2 based on the downloaded data, and the verification code sent by the server 102 is MD5 value 1. If MD5 value 1 is equal to MD5 value 2, the client 101 data download is successful; if MD5 value 1 is not equal to MD5 value 2, the client 101 data download fails.

[0119] It can be seen from the above embodiments that the client of the cloud storage system in the embodiments of the present application can download data within a specified range. At the same time, the server can generate a verification code for the data within the specified range separately, and send the data and verification code within the specified range to the client through block transmission, thereby improving the data verification flexibility between the client and the server in the cloud storage system during data transmission.

[0120] It is understandable that the data verification method provided in the embodiment of the present application can be applied to other data verification scenarios. For example, the server can also verify the data uploaded by the client based on the data verification method. For example, the server receives a data upload request sent by the client, and the data upload request includes the uploaded data and the third verification code. The server generates a fourth verification code based on the uploaded data. If the fourth verification code is consistent with the third verification code, the data received by the server is consistent with the data sent by the client.

[0121] Based on the above method embodiment, the embodiment of the present application also provides a data verification device. The data verification device provided by the embodiment of the present application is described in detail below.

[0122] See also Figure 5 , Figure 5 A schematic diagram of the structure of a data verification device provided in an embodiment of the present application. Figure 5 In the example shown, the data verification device 500 is used to implement the various steps executed by the server or client of the cloud storage system in the above embodiments. The data verification device 500 includes a transceiver unit 501 and a processing unit 502 .

[0123] The transceiver unit 501 is used to receive a data download request sent by a client, and the data download request is used to request to download data within a specified range of the server. The processing unit 502 is used to generate a first verification code based on the data within the specified range, and the first verification code is used to verify the data consistency between the data sent by the server and the data received by the client. The transceiver unit 501 is also used to send the data within the specified range and the first verification code to the client, so that the client verifies the data within the specified range based on the first verification code.

[0124] In a possible implementation, the processing unit 502 is specifically configured to split the data within a specified range into multiple data blocks, and send the multiple data blocks and a first check code to the client, wherein the first check code is located in a tail data block among the multiple data blocks.

[0125] In a possible implementation, each data block includes a message header field and a data portion, the first check code is located in the data portion of the tail data block, and the message header field of the tail data block is used by the client to identify the tail data block.

[0126] In a possible implementation, the processing unit 502 is specifically configured to determine that data within a specified range is modified, and the server generates a first check code based on the sent data blocks before sending the tail data blocks.

[0127] In a possible implementation, data within a specified range is used by the client to generate a second verification code. If the second verification code is consistent with the first verification code, the data received by the client is consistent with the data sent by the server.

[0128] In a possible implementation manner, the transceiver unit 501 is specifically configured to send data within a specified range and a first check code to the client based on a block transfer encoding mechanism of the hypertext transfer protocol HTTP / 1.1.

[0129] It is understandable that the transceiver unit 501 and the processing unit 502 in the data verification device 500 can be used as functional modules. Figure 1a There is a mapping between each module in the cloud storage system 10, so as to realize the function of each module in the cloud storage system 10.

[0130] It should be understood that the division of the units in the above device is only a division of logical functions. In actual implementation, they can be fully or partially integrated into one physical entity, or they can be physically separated. And the units in the device can all be implemented in the form of software calling through processing elements; they can also be all implemented in the form of hardware; some units can also be implemented in the form of software calling through processing elements, and some units can be implemented in the form of hardware. For example, each unit can be a separately established processing element, or it can be integrated in a certain chip of the device. In addition, it can also be stored in the memory in the form of a program, and called and executed by a certain processing element of the device. The function of the unit. In addition, all or part of these units can be integrated together, or they can be implemented independently. The processing element described here can also be a processor, which can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each unit above can be implemented by an integrated logic circuit of hardware in the processor element or in the form of software calling through a processing element.

[0131] It is worth noting that, for the above method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited to the described order of actions. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required for the present application.

[0132] Other reasonable step combinations that can be thought of by those skilled in the art based on the above description also fall within the scope of protection of this application. Secondly, those skilled in the art should also be familiar with the fact that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by this application.

[0133] See also Figure 6 , Figure 6 A schematic diagram of the structure of a computing device provided in an embodiment of the present application. Figure 6 As shown, the computing device 600 includes: a processor 601, a memory 602, a communication interface 603 and a bus 604. The processor 601, the memory 602 and the communication interface 603 are coupled via a bus (not marked in the figure). The memory 602 stores instructions. When the execution instructions in the memory 602 are executed, the computing device 600 executes the method executed by the server or the client in the above method embodiment.

[0134] The computing device 600 may be one or more integrated circuits configured to implement the above method, such as one or more application specific integrated circuits (ASIC), or one or more microprocessors (digital signal processors, DSP), or one or more field programmable gate arrays (FPGA), or a combination of at least two of these integrated circuit forms. For another example, when the unit in the device can be implemented in the form of a processing element scheduler, the processing element can be a general-purpose processor, such as a central processing unit (CPU) or other processor that can call a program. For another example, these units can be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0135] The processor 601 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. The general-purpose processor may be a microprocessor or any conventional processor.

[0136] The memory 602 may be a volatile memory or a nonvolatile memory, or may include both volatile and nonvolatile memories. Among them, the nonvolatile memory may be a read-only memory (ROM), a programmable ROM (PROM), an erasable programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

[0137] The memory 602 stores executable program codes, and the processor 601 executes the executable program codes to respectively implement the functions of the aforementioned units or modules, thereby implementing the aforementioned data verification method. That is, the memory 602 stores instructions for executing the aforementioned data verification method.

[0138] The communication interface 603 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 600 and other devices or a communication network.

[0139] In addition to the data bus, the bus 604 may also include a power bus, a control bus, a status signal bus, etc. The bus may be a peripheral component interconnect express (PCIe) bus, or an extended industry standard architecture (EISA) bus, a unified bus (Ubus or UB), a compute express link (CXL), a cache coherent interconnect for accelerators (CCIX), etc. The bus may be divided into an address bus, a data bus, a control bus, etc.

[0140] See also Figure 7 , Figure 7 A schematic diagram of a computing device cluster provided in an embodiment of the present application. Figure 7 As shown, the computing device cluster 700 includes at least one computing device 600 .

[0141] like Figure 7 As shown, the computing device cluster 700 includes at least one computing device 600. The memory 602 in one or more computing devices 600 in the computing device cluster 700 may store the same instructions for executing the above data verification method.

[0142] In some possible implementations, the memory 602 of one or more computing devices 600 in the computing device cluster 700 may also store some instructions for executing the above data verification method. In other words, the combination of one or more computing devices 600 can jointly execute the instructions for executing the above data verification method.

[0143] It should be noted that the memory 602 in different computing devices 600 in the computing device cluster 700 can store different instructions, which are respectively used to execute part of the functions of the above-mentioned data verification device. That is, the instructions stored in the memory 602 in different computing devices 600 can realize the functions of one or more modules in the processing unit and the transceiver unit.

[0144] In some possible implementations, one or more computing devices 600 in the computing device cluster 700 may be connected via a network, which may be a wide area network or a local area network.

[0145] See also Figure 8 , Figure 8A schematic diagram of computer devices in a computer cluster connected via a network provided in an embodiment of the present application. Figure 8 As shown, two computing devices 600A and 600B are connected via a network. Specifically, they are connected to the network via a communication interface in each computing device.

[0146] In a possible implementation, the memory in the computing device 600A stores instructions for executing the functions of the transceiver unit, and the memory in the computing device 600B stores instructions for executing the functions of the processing unit.

[0147] It should be understood that Figure 8 The functions of the computing device 600A shown in FIG. 6A may also be completed by multiple computing devices. Similarly, the functions of the computing device 600B may also be completed by multiple computing devices.

[0148] In another embodiment of the present application, a computer-readable storage medium is provided, in which computer-executable instructions are stored. When the processor of the device executes the computer-executable instructions, the device executes the method executed by the cloud storage system in the above method embodiment.

[0149] In another embodiment of the present application, a computer program product is provided, the computer program product includes computer executable instructions, the computer executable instructions are stored in a computer readable storage medium. When the processor of the device executes the computer executable instructions, the device executes the method executed by the cloud storage system in the above method embodiment.

[0150] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0151] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0152] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0153] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0154] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, read-only memory), random access memory (RAM, random access memory), disk or optical disk and other media that can store program code.

Claims

1. A data verification method, characterized in that: A server applied to a cloud storage system, wherein the cloud storage system further comprises a client, and the method comprises: The server receives a data download request sent by the client, wherein the data download request is used to request downloading of data within a specified range of the server; The server generates a first verification code based on the data within the specified range, where the first verification code is used to verify data consistency between the data sent by the server and the data received by the client; The server sends the data within the specified range and the first verification code to the client, so that the client verifies the data within the specified range based on the first verification code.

2. The method according to claim 1, characterized in that: The server sending the data within the specified range and the first verification code to the client includes: The server divides the data within the specified range into multiple data blocks, and sends the multiple data blocks and the first check code to the client, wherein the first check code is located in the tail data block of the multiple data blocks.

3. The method according to claim 2, characterized in that Each data block includes a message header field and a data part. The first check code is located in the data part of the tail data block. The message header field of the tail data block is used by the client to identify the tail data block.

4. The method according to claim 2 or 3, characterized in that: The server generating a first verification code based on the data within the specified range includes: The server determines that the data within the specified range is modified, and before sending the tail data block, the server generates the first check code based on the sent data block.

5. The method according to any one of claims 1 to 4, characterized in that The data within the specified range is used by the client to generate a second verification code. If the second verification code is consistent with the first verification code, the data received by the client is consistent with the data sent by the server.

6. The method according to any one of claims 1 to 5, characterized in that The server sending the data within the specified range and the first verification code to the client includes: The server sends the data within the specified range and the first check code to the client based on the block transfer encoding mechanism of the Hypertext Transfer Protocol HTTP / 1.

1.

7. A data verification device, characterized in that: The device comprises: A transceiver unit, configured to receive a data download request sent by the client, wherein the data download request is used to request downloading of data within a specified range of the server; A processing unit, configured to generate a first verification code based on the data within the specified range, wherein the first verification code is used to verify data consistency between the data sent by the server and the data received by the client; The transceiver unit is further configured to send the data within the specified range and the first verification code to the client, so that the client verifies the data within the specified range based on the first verification code.

8. The device according to claim 7, characterized in that The processing unit is specifically used for: The data within the specified range is split into a plurality of data blocks, and the plurality of data blocks and the first check code are sent to the client, wherein the first check code is located in the tail data block of the plurality of data blocks.

9. The device according to claim 8, characterized in that Each data block includes a message header field and a data part. The first check code is located in the data part of the tail data block. The message header field of the tail data block is used by the client to identify the tail data block.

10. The device according to claim 8 or 9, characterized in that The processing unit is specifically used for: It is determined that the data within the specified range is modified, and the server generates the first check code based on the sent data blocks before sending the tail data blocks.

11. The device according to any one of claims 7 to 10, characterized in that The data within the specified range is used by the client to generate a second verification code. If the second verification code is consistent with the first verification code, the data received by the client is consistent with the data sent by the server.

12. The device according to any one of claims 7 to 11, characterized in that The transceiver unit is specifically used for: The data within the specified range and the first check code are sent to the client based on the block transfer encoding mechanism of the Hypertext Transfer Protocol HTTP / 1.

1.

13. A computing device, characterized in that: The device comprises a processor coupled to a memory, wherein the processor is used to store instructions. When the instructions are executed by the processor, the computing device performs the method according to any one of claims 1 to 6.

14. A computing device cluster, characterized in that: The system comprises at least one computing device, wherein the computing device comprises a processor, wherein the processor is coupled to a memory, and the processor is used to store instructions. When the instructions are executed by the processor, the computing device cluster executes the method according to any one of claims 1 to 6.

15. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instructions are executed, the computer is caused to perform the method according to any one of claims 1 to 6.

16. A computer program product, comprising instructions, characterized in that: When the instructions are executed, the computer implements the method according to any one of claims 1 to 6.

Images