Data verification method and apparatus

Abstract

Embodiments of the present application disclose a data verification method and apparatus, which are used for improving the flexibility of data verification. The method provided by the embodiments of the present application comprises: a server receives a data download request sent by a client, the data download request being used to request to download data in a specified range of the server; the server generates a first check code on the basis of the data in the specified range, the first check code being used to verify data consistency between sent data of the server and received data of the client; and the server sends to the client the data in the specified range and the first check code, so that the client verifies the data in the specified range on the basis of the first check code.

Description

Data verification method and device

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office of China on November 22, 2023, with application number 202311572639.2, entitled “A method, device and other equipment for data processing”, and the Chinese patent application filed with the State Intellectual Property Office of China on April 12, 2024, with application number 202410444204.8, entitled “A method and device for data verification”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The embodiments of the present application relate to the field of cloud computing, and in particular to a data verification method and device. Background Art

[0003] With the development of cloud computing technology, cloud storage, as an important component of cloud computing, has become increasingly widely used. In cloud storage applications, when clients download data from servers, network hijacking or data caching can cause inconsistencies between the data downloaded by the client and the original data on the server. Therefore, clients need to perform consistency checks to ensure the correctness of the downloaded data.

[0004] Currently, when the client performs consistency check on downloaded data, the server generates a check code for the original data based on the original data and sends the check code to the client, and the client performs consistency check on the received downloaded data based on the check code.

[0005] However, when the client downloads the original data from the server, it often specifies the download range of the original data instead of downloading the complete original data. Therefore, the client cannot perform consistency verification on the downloaded data in the specified range based on the verification code of the original data, resulting in poor data verification flexibility of the client.

[0006] Summary of the Invention

[0007] The present invention provides a data verification method in which a server can generate a verification code for data within a specified range of a client, thereby improving the verification flexibility during the data verification process. The present invention also provides a data verification device, a computing device, a computing device cluster, a computer-readable storage medium, and a computer program product corresponding to the data verification method.

[0008] In the first aspect, an embodiment of the present application provides a data verification method, which can be executed by a server of a cloud storage system, or by a component of the server of the cloud storage system, such as a processor, chip or chip system of the server of the cloud storage system, or can be implemented by a logic module or software that can realize all or part of the server functions of the cloud storage system. The method provided in the first aspect is applied to a server of a cloud storage system, and the cloud storage system also includes a client. The method provided in the first aspect includes: the server receives a data download request sent by the client, and the data download request is used to request downloading of data within a specified range of the server. The server generates a first verification code based on the data within the specified range, and the first verification code is used to verify the data consistency between the data sent by the server and the data received by the client. The server sends the data within the specified range and the first verification code to the client, so that the client verifies the data within the specified range based on the first verification code.

[0009] In the embodiment of the present application, the client of the cloud storage system can download data within a specified range from the server. At the same time, the server can generate a verification code for the data within the specified range separately and send it to the client, and the client verifies the downloaded data. Compared with the current solution in which the server only generates a verification code for the complete original data, in the embodiment of the present application, the server can generate a verification code for the data within the specified range, thereby improving the data verification flexibility between the client and the server in the cloud storage system during the data transmission process.

[0010] In one possible implementation, after the server sends data within a specified range and a first verification code to the client, the client generates a second verification code based on the data within the specified range, where the data within the specified range is the data received by the client. The client verifies the consistency of the data within the specified range based on the first and second verification codes. If the first and second verification codes are consistent, the client determines that the data received is consistent with the data sent by the server.

[0011] After the client of the cloud storage system in the embodiment of the present application receives the data sent by the server, it can regenerate a verification code based on the received data, and compare the generated verification code with the verification code sent by the server to verify the data within the specified range sent by the server, thereby improving the accuracy of the client's data verification.

[0012] In one possible implementation, when the server sends data within a specified range and a first check code to the client, the server splits the data within the specified range into multiple data blocks, and sends the multiple data blocks and the first check code to the client, wherein the first check code is located in the tail data block among the multiple data blocks.

[0013] In an embodiment of the present application, when the server sends data within a specified range and a first verification code to the client, the data within the specified range and the first verification code can be sent in blocks based on the block transmission protocol, so there is no need to store the entire data within the specified range into the memory before sending it, thereby reducing the memory utilization of the server and reducing the response delay during the data download process.

[0014] In one possible implementation, the data block sent by the server to the client includes two parts, wherein the first part is a message header field, which is used to indicate the valid data length of the data block, or to indicate that the data block contains a check code, and the second part is the data portion, i.e., the valid data or check code carried by the data block, where the valid data is the data in the specified range requested by the client for download. The first check code is located in the data portion of the tail block. Since the message header field of the data block carrying the valid data is different from the message header field of the data block carrying the first check code, the client can identify the tail data block based on the message header field of the data block.

[0015] In an embodiment of the present application, when the server transmits data within a specified range to the client in blocks, the content of the data block can be indicated through the message header field of the data block, so that the client can identify the content of the data block based on the message header field of the data block, thereby improving the data download efficiency of the cloud storage system.

[0016] In one possible implementation, the server sends data within a specified range and a first checksum to the client based on the chunked transfer encoding mechanism of the Hypertext Transfer Protocol (HTTP / 1.1). In the chunked transfer encoding mechanism, the server divides the data into a series of blocks of variable length, each separated by a specific marker, so that the data can be transmitted in real time during the data transmission process.

[0017] In the embodiment of the present application, the server can send data within a specified range and a first check code to the client based on the block transfer encoding mechanism of the hypertext transfer protocol HTTP / 1.1, thereby improving the real-time performance of data transmission.

[0018] In one possible implementation, when the server generates a first verification code based on data within a specified range, when the server determines that the data within the specified range has been modified, the server generates a first verification code in real time based on the sent data block before sending the tail data block, and the sent data block is the modified data within the specified range.

[0019] In the embodiment of the present application, the server can generate a check code in real time during the process of sending data blocks. Since the check code is carried in the tail data block, the server can generate the check code in real time based on the check of the sent data blocks, thereby avoiding check code errors caused by modifications to the data within the specified range. The server generates the check code in real time based on the check of the sent data blocks, which improves the accuracy of the check code and further improves the accuracy of the data verification process.

[0020] In one possible implementation, during the process of the server generating a first verification code based on data within a specified range, the server first reads the data within the specified range, then stores the complete data within the specified range into the memory, and calculates the first verification code based on the data within the specified range in the memory. The server sends the complete data within the specified range and the first verification code to the client.

[0021] In the embodiment of the present application, the server can store the complete data within the specified range into the memory, and calculate the first verification code based on the data within the specified range in the memory, thereby avoiding repeated reading of the data in the specified range when calculating the first verification code and sending the data in the specified range to the client, reducing the number of times the server reads the data in the specified range and improving data verification efficiency.

[0022] In one possible implementation, after the server reads the data within a specified range, if the server's memory cannot buffer the complete data within the specified range, the server needs to first read the data within the complete instruction range to calculate a first check code. After sending the first check code to the client, the server re-reads the data within the complete specified range and sends it to the client.

[0023] In the embodiment of the present application, the server can directly read the data in the specified range and generate a first verification code without first storing the data in the specified range into the memory, thereby reducing the memory consumption of the server.

[0024] In one possible implementation, a server receives a data upload request from a client, the data upload request including uploaded data and a third verification code. The server generates a fourth verification code based on the uploaded data. If the fourth verification code is consistent with the third verification code, the data received by the server is consistent with the data sent by the client.

[0025] In the embodiment of the present application, the server can also verify the received client uploaded data during the data upload process, thereby improving the applicability of the data verification method in the embodiment of the present application.

[0026] In one possible implementation, the storage type of data within a specified range includes one or more of the following: object storage and file storage. Object storage refers to the storage of data in the form of objects and is suitable for storing large-capacity, unstructured data. File storage refers to the storage of data in the form of hierarchical folders and files. File storage is suitable for storing structured data with relatively fixed access patterns and is suitable for scenarios requiring frequent read and write operations.

[0027] The data verification method provided in the embodiment of the present application is applicable to various types of data storage scenarios, thereby improving the applicability of the data verification method provided in the embodiment of the present application.

[0028] In one possible implementation, both the first checksum and the second checksum include one or more of the following: a Message Digest Algorithm Version 5 (MD5) checksum, a Cyclic Redundancy Checksum (CRC) checksum, and a Secure Hash Algorithm (SHA) checksum. The checksum algorithm for the first checksum is the same as the checksum algorithm for the second checksum. The Message Digest Algorithm Version 5 (MD5) algorithm is a checksum algorithm that uses a hash function to generate a checksum, the Cyclic Redundancy Checksum (CRC) algorithm generates a checksum by performing polynomial division on a data block, and the Secure Hash Algorithm (SHA) algorithm includes a series of algorithms that generate checksums based on cryptographic hash functions, including SHA-1, SHA-256, and SHA-512.

[0029] In the embodiment of the present application, the server and the client can generate verification codes based on a variety of different types of verification algorithms, thereby improving the richness of the verification codes generated by the client and the server in the embodiment of the present application.

[0030] In one possible implementation, before the server sends multiple data blocks to the client, the server sends a response message to the client for the data download request. The message header field of the response message includes a block transmission identification field. The block transmission identification field is used to indicate to the client that the server uses the block transmission protocol to transmit a specified range of data and a first check code.

[0031] In an embodiment of the present application, before the server transmits data in blocks to the client, it can send a response message corresponding to the data download request to the client, and instruct the server through the response message to send data within a specified range and a first verification code, thereby improving the feasibility of the server transmitting data within the specified range and the first verification code in blocks.

[0032] In a second aspect, an embodiment of the present application provides a data verification device, comprising a transceiver unit and a processing unit. The transceiver unit is configured to receive a data download request sent by a client, wherein the data download request is used to request downloading of data within a specified range from a server. The processing unit is configured to generate a first verification code based on the data within the specified range, wherein the first verification code is used to verify the data consistency between the data sent by the server and the data received by the client. The transceiver unit is further configured to send the data within the specified range and the first verification code to the client, so that the client verifies the data within the specified range based on the first verification code.

[0033] In a possible implementation, the processing unit is specifically configured to split data within a specified range into multiple data blocks, and send the multiple data blocks and a first check code to the client, wherein the first check code is located in a tail data block among the multiple data blocks.

[0034] In a possible implementation, each data block includes a message header field and a data portion, the first check code is located in the data portion of the tail data block, and the message header field of the tail data block is used by the client to identify the tail data block.

[0035] In a possible implementation, the processing unit is specifically configured to determine that data within a specified range has been modified, and the server generates a first check code based on the sent data blocks before sending the tail data blocks.

[0036] In one possible implementation, data within a specified range is used by the client to generate a second verification code. If the second verification code is consistent with the first verification code, the data received by the client is consistent with the data sent by the server.

[0037] In a possible implementation, the transceiver unit is specifically configured to send data within a specified range and the first check code to the client based on a block transfer encoding mechanism of the Hypertext Transfer Protocol HTTP / 1.1.

[0038] In a third aspect, an embodiment of the present application provides a computing device, comprising a processor coupled to a memory, the processor being used to store instructions. When the instructions are executed by the processor, the computing device executes the method described in the first aspect or any possible implementation of the first aspect.

[0039] In a fourth aspect, an embodiment of the present application provides a computing device cluster, which includes one or more computing devices, each of which includes a processor coupled to a memory, and the processor is used to store instructions. When the instructions are executed by the processor, the computing device cluster executes the method described in the first aspect or any possible implementation method of the first aspect.

[0040] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium having instructions stored thereon. When the instructions are executed, the computer executes the method described in the first aspect or any possible implementation method of the first aspect.

[0041] In a sixth aspect, an embodiment of the present application provides a computer program product, which includes instructions. When the instructions are executed, the computer implements the method described in the first aspect or any possible implementation method of the first aspect.

[0042] It can be understood that the beneficial effects that can be achieved by any of the data verification devices, computing devices, computing device clusters, computer-readable media or computer program products provided above can refer to the beneficial effects in the corresponding methods and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] FIG1a is a schematic diagram of the system architecture of a cloud storage system provided in an embodiment of the present application;

[0044] FIG1b is a schematic diagram of a flow chart of another data verification method provided in an embodiment of the present application;

[0045] FIG1c is a schematic diagram of an object storage bucket provided in an embodiment of the present application;

[0046] FIG2 is a flow chart of a data verification method provided in an embodiment of the present application;

[0047] FIG3 is a flow chart of another data verification method provided in an embodiment of the present application;

[0048] FIG4 is a flow chart of another data verification method provided in an embodiment of the present application;

[0049] FIG5 is a schematic structural diagram of a data verification device provided in an embodiment of the present application;

[0050] FIG6 is a schematic diagram of the structure of a computing device provided in an embodiment of the present application;

[0051] FIG7 is a schematic diagram of the structure of a computing device cluster provided in an embodiment of the present application;

[0052] FIG8 is a schematic diagram of the structure of another computing device cluster provided in an embodiment of the present application. DETAILED DESCRIPTION

[0053] The embodiments of the present application provide a data verification method and device for improving the flexibility of data verification.

[0054] The terms "first," "second," "third," "fourth," and the like (if any) in the specification and claims of this application and in the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or sequential sequence. It should be understood that the terms used in this manner are interchangeable where appropriate so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" and "having," and any variations thereof, are intended to cover non-exclusive inclusions, e.g., a process, method, system, product, or apparatus comprising a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0055] In the embodiments of this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of this application should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0056] First, some terms involved in the embodiments of the present application are introduced to facilitate those skilled in the art to understand the technical solutions.

[0057] Object storage refers to the storage of data as objects. Each object has a unique identifier that can be used to access and manipulate it. Object storage is commonly used to store unstructured data, such as images, videos, audio, and documents.

[0058] File storage refers to managing data through a file system. Data is stored in the form of files on a cloud server. Files can be modified, deleted, and queried anytime and anywhere.

[0059] Range download refers to the support in the object and file storage field that allows the client to download data within a specified range when downloading specified data, such as from the 5th byte to the 100th byte.

[0060] Metadata refers to descriptive information about an object in the object storage field, such as the object's name, size, creation time, and access permissions.

[0061] A checksum is typically a specific code or number used to verify data integrity and accuracy. It is transmitted or stored along with the data. Examples of checksums include checksums, cyclic redundancy checks, and hash values.

[0062] Consistency checking refers to the process by which the client or server verifies the consistency of sent and received data based on a checksum during data transmission between the client and server. For example, when receiving a file uploaded by a client, the server compares the received checksum with the checksum set by the client. The upload is considered successful only if the two match. Similarly, when downloading a file, the client compares the checksum sent by the server with the checksum calculated by the client. The download is considered successful only if the two match, thus ensuring data consistency.

[0063] In order to make the technical solution of the present application clearer and easier to understand, the system architecture of the present application is introduced below with reference to the accompanying drawings.

[0064] Please refer to Figure 1a, which is a schematic diagram of the system architecture of a cloud storage system provided as an example in this application. In the example shown in Figure 1a, cloud storage system 10 includes client 101 and server 102, where server 102 includes storage server 1021 and metadata server 1022. The following describes the specific functions of each component of cloud storage system 10.

[0065] The client 101 is used to provide users with an interface for accessing the cloud storage system 10. Users can interact with the server 102 through the client 101. For example, users can use the client 101 to upload, download, query, and modify data on the server 102. When users upload data to the server 102 through the client 101, the client 101 is also used to segment the uploaded data and send the data segments to the server 102.

[0066] The client 101 is also used to perform data consistency check on the download data sent by the server 102. The client 101 calculates a check code based on the received download data and compares the calculated check code with the check code sent by the server 102. When the two check codes are consistent, the client 101 determines that the data download is successful.

[0067] The client 101 is also used to implement functions such as identity authentication, access control, and data encryption, thereby ensuring that users can securely access and operate data stored in the server 102. For example, the client 101 can define and manage the access control policy of the server 102, specifying access rights for different data objects, such as read permission, write permission, and delete permission, so that authorized users or applications can access specific data.

[0068] It is understandable that the client 101 in the embodiment of the present application can be a terminal device, such as a personal computer, a smart phone or a tablet computer, or a software client on the terminal device, such as an application, without specific limitation.

[0069] Server 102 includes a storage server 1021 and a metadata server 1022. Storage server 1021 is used to implement functions such as data storage and optimized data distribution. Metadata server 1022 is used to control the interaction between client 101 and storage server 1021, including processing requests from client 101 and maintaining metadata information and directory structures for storage objects.

[0070] Storage server 1021 is used to implement data storage and management. For example, storage server 1021 actually stores data uploaded by client 101 and manages the stored data. When client 101 requests to upload data, storage server 1021 can receive the data and store it on a persistent storage medium, such as a disk array. Storage server 1021 is also used to distribute data to different nodes to achieve storage scalability and load balancing. Data distribution can be based on consistent hashing algorithms or other distributed algorithms to ensure even data distribution within the cluster.

[0071] The metadata server 1022 is used to store and maintain metadata information for stored objects, such as the object's name, size, creation time, and access permissions. Metadata information is typically stored as structured data to facilitate indexing and querying. The metadata server 1022 is also used to create indexes and provide metadata search capabilities. For example, the client 101 can search metadata based on object attributes or keywords to locate and access specific objects.

[0072] The metadata server 1022 is also used to perform access control and rights management on metadata information. For example, the metadata server 1022 can control the client 101's access to metadata information through an access control list and rights management mechanism, allowing authorized users or applications to read, modify or delete metadata information.

[0073] It should be noted that the cloud storage system 10 in the embodiment of the present application can be applied to the object storage field and the file storage field. In the object storage field, the data in the server 102 is stored in the form of objects. Each object contains the data itself, metadata and a unique identifier. The server 102 accesses and manages the data through the unique identifier. The object storage field is suitable for storing large-capacity, unstructured data and has high scalability and persistence.

[0074] In the file storage field, data is stored in the form of hierarchical folders and files on server 102. File storage is suitable for storing structured data with relatively fixed access patterns, providing low-latency file access speeds and suitable for scenarios requiring frequent read and write operations. It is understood that when cloud storage system 1021 is applied to the file storage field, server 102 in cloud storage system 10 may only include storage server 1021.

[0075] Please refer to Figure 1b, which is a schematic diagram of a process for verifying downloaded data by a client, as provided in this application. In the example shown in Figure 1b, client 101 sends a data download request to server 102. This data download request requests the download of object data in an object storage bucket. Because the downloaded data is object data, client 101 must first obtain the object information of the download object before downloading the data, including determining the unique identifier or path of the download object and related information such as the object size. Client 101 then sends a data download request to the server based on the object information to obtain the data content of the object.

[0076] If the local MD5 value is consistent with the MD5 value sent by server 102, then in the example shown in FIG1b , server 102 generates an MD5 value corresponding to the downloaded data based on the downloaded data. This MD5 value is the verification code of the verification object data, and server 102 sends this MD5 value to terminal device 101. After receiving the downloaded data, client 101 generates a local MD5 value based on the downloaded object. Client 101 compares the generated local MD5 value with the MD5 value sent by server 102. If the received downloaded object is consistent with the downloaded object sent by server 102, the download is successful. If they are inconsistent, the download fails, and client 101 can resend the data download request to server 102.

[0077] In current technology, when client 101 sends a data download request to server 102, server 102 can only provide the MD5 value of the complete data, but cannot send the MD5 value of the data within the range specified by client 101 to client 101. As a result, client 101 cannot complete the data consistency check. At the same time, because the downloaded data may have been modified, the MD5 value of the actual data may be inconsistent with the MD5 value sent by server 102 to client 101, resulting in download verification failure.

[0078] Please refer to Figure 1c, which is a schematic diagram of an object storage bucket provided in an embodiment of the present application. In the example shown in Figure 1c, service restart 102 includes one or more data storage buckets, each of which contains one or more objects. An object is the basic unit of data stored in the object storage bucket and can be any type of unstructured data, such as files, documents, images, and videos.

[0079] Each object consists of data, metadata, and an identifier (key). Data refers to the actual content of the object, namely the file or data itself that users want to store and retrieve. Metadata is descriptive information associated with the object, including its size, creation time, last modification time, file type, and storage category. The identifier is used to uniquely identify the object in the bucket, and each object has a unique identifier (key).

[0080] Based on the cloud storage system 10 shown in Figure 1a, the present application further provides a data verification method. The data verification method provided by the embodiment of the present application is described below in conjunction with an embodiment.

[0081] Please refer to Figure 2, which is a flow chart of a data verification method provided in an embodiment of the present application. In the example shown in Figure 2, the method includes the following steps:

[0082] Step 201: The client sends a data download request to the server. The data download request is used to request downloading of data within a specified range from the server.

[0083] Client 101 sends a data download request to server 102. This data download request is used to request downloading data within a specified range from server 102. The specified range refers to the starting and ending positions of the data that client 101 is requesting to download, for example, bytes 5 to 100. The data in the specified range includes download objects in the object storage area and file data in the file storage area.

[0084] In one possible implementation, in the object storage field, client 101 sends a data download request to server 102. The request also includes an identifier for the download object. This identifier indicates the object that client 101 wants to download, such as the object name and ID. After receiving the data download request from client 101, server 102 parses the request to obtain the identifier and a specified range. Server 102 then determines the location and size of the download data based on the identifier and specified range in the request.

[0085] Specifically, server 102 queries metadata server 1022 for metadata corresponding to the downloaded data based on the identifier in the data download request. Based on the metadata, server 1022 determines the storage location of the downloaded data in storage server 1021, including the storage node or storage device where the downloaded data is located. Server 102 further determines the actual data to be read based on the specified range in the data download request. The specified range includes an offset and data size. The data determined by server 102 based on the specified range is the data that server 102 will send to client 101.

[0086] Please refer to Figure 3, which is a flowchart of another data verification method provided in an embodiment of the present application. In step a of the example shown in Figure 3, client 101 sends a data download request to server 102. The data download request is used to request server 102 to download data within a specified range, such as data from the 5th byte to the 500th byte.

[0087] Step 202: The server generates a first verification code based on data within a specified range. The first verification code is used to verify data consistency between data sent by the server and data received by the client.

[0088] After the server 102 determines the data within the specified range, it generates a first verification code based on the data within the specified range. The first verification code is used to verify the data consistency between the data sent by the server 102 and the data actually received by the client 101. The first verification code can be, for example, a character string generated by processing the data within the specified range based on a verification algorithm.

[0089] In the embodiment of the present application, the first verification code can be a character string generated based on multiple verification algorithms, such as the message digest algorithm 5 (MD5), the cyclic redundancy check (CRC) and the secure hash algorithm (SHA), without specific limitation.

[0090] Among them, the Message Digest Algorithm Version 5 MD5 algorithm is a verification algorithm that uses a hash function to generate a check code, the cyclic redundancy CRC algorithm is an algorithm that generates a check code by performing polynomial division on a data block, and the Secure Hash Algorithm SHA algorithm includes a series of algorithms that generate check codes based on cryptographic hash functions, including SHA-1, SHA-256, and SHA-512.

[0091] In one possible implementation, when server 102 generates the first verification code based on data within a specified range, server 102 first reads the data within the specified range, then stores the entire data within the specified range in memory, and calculates the first verification code based on the data within the specified range in memory. In this implementation, server 102 may also directly transmit the entire data within the specified range to client 101.

[0092] In an embodiment of the present application, the server 102 can store the data within the entire specified range into the memory, calculate the first verification code based on the data within the specified range in the memory, and directly read the data within the specified range from the memory and send it to the client 101, which can improve the efficiency of the server 101 in calculating the verification code and the data transmission efficiency.

[0093] In one possible implementation, if the memory of server 102 cannot buffer the complete data within the specified range, server 102 needs to first read the data within the complete instruction range to calculate the first verification code. After sending the first verification code to client 101, server 102 re-reads the data within the complete specified range and sends it to client 101. Since server 102 can directly read the data within the specified range and generate the first verification code without storing the data within the specified range in memory first, the server's memory consumption is reduced. However, since repeated reading of data will cause the response delay of server 102 to increase, this implementation is not a preferred implementation.

[0094] It should be noted that the server 102 may generate the first check code based on the data within the specified range during the process of sending the data within the specified range, or after the data within the specified range has been sent, without specific limitation. For example, the server 102 may split the data within the specified range and send the data within the specified range in blocks. Furthermore, while the server 102 is sending the data within the specified range in blocks, the server 102 generates the first check code based on the data within the specified range in real time, and places the first check code in the last data block to be sent.

[0095] Step 203: The server sends the data within the specified range and the first verification code to the client, so that the client verifies the data within the specified range based on the first verification code.

[0096] After the server 102 generates a first verification code based on the data within the specified range, the server 102 sends the data within the specified range and the first verification code to the client 101, so that the client 101 can verify the data within the specified range based on the first verification code. The data within the specified range that needs to be verified is the downloaded data received by the client 101.

[0097] Specifically, when the server 102 sends data within a specified range and a first check code to the client 101, the server 102 splits the data within the specified range into multiple data blocks, and sends multiple data blocks and a first check code to the client 101, wherein the first check code is located at the tail data block.

[0098] The data block in the embodiment of the present application includes two parts, wherein the first part is a message header field, which can be used to indicate the valid data length of the data block. The message header field can also be used to indicate that the data block contains a check code. For example, the message header field of the tail data block is "0". At this time, the message header field indicates that the data block carries a check code. The second part is the data part, that is, the valid data or check code carried by the data block. The valid data is the data in the specified range requested to be downloaded by the client 101.

[0099] In the embodiment of the present application, server 102 sends data within a specified range and a first checksum to the client based on the chunked transfer encoding mechanism of the Hypertext Transfer Protocol (HTTP / 1.1). In the chunked transfer encoding mechanism, server 102 segments the data into a series of data chunks of varying lengths, each separated by a specific marker. The message header field of each data chunk contains a hexadecimal number representing the effective data length of the data chunk. A data chunk header field containing "0" indicates the end of the data chunk transmission.

[0100] Please continue to refer to Figure 3. In step b of the example shown in Figure 3, when the server 102 sends data within the specified range to the client 101, the server 102 splits the data within the specified range and the first check code into multiple data blocks, and the server 102 sends the data blocks to the client 101. For example, the server 102 splits the data within the specified range and the first check code into x chunks, namely chunk1,..., chunkx, where the first check code is carried in the last data block chunkx, and the data within the specified range is carried in multiple data blocks sent before chunkx.

[0101] In the example shown in FIG3 , each data block chunk includes a message header field and a data portion, wherein the message header field of each chunk except the last chunk indicates the length of the data portion, expressed in hexadecimal. The message header field of the last chunk is 0, indicating that the data portion of the chunk is a checksum.

[0102] In one possible implementation, before server 102 transmits the data blocks to client 101, server 102 may send a response message corresponding to the data download request to client 101, and instruct the server, via the response message, on how to transmit the data within a specified range and the first verification code. The message header field of the response message includes a block transfer identification field, which is used to instruct client 101 that server 102 transmit the data within the specified range and the first verification code using the block transfer protocol.

[0103] For example, before the server 102 sends multiple data chunks to the client 101, the server 102 sends a response message of the data download request to the client 101. The server 102 can add a "Transfer-Encoding: chunked" field in the response message header, and use the "chunked" keyword to indicate to the client 101 that the server 102 uses the chunked transfer protocol to transmit the specified range of data and the first check code.

[0104] In one possible implementation, if the data within the specified range in the server 102 is modified, while the server 102 is sending the data within the specified range in blocks, before the server 102 sends the tail data block, since the first check code is in the tail data block, the server 102 can generate the first check code in real time based on the sent data block, and the sent data block is the data within the modified specified range.

[0105] Please refer to Figure 4, which is an example diagram of another data verification method provided by an embodiment of the present application. The example shown in Figure 4 is a file storage scenario. In steps a to f of the example shown in Figure 4, after the client 101 sends a data download request to the server 102, the data download request is used to request the download of the file data uploaded by the client 103. Before the server 102 sends the requested file data to the client 101, the client 103 can modify the data in the server 102. For example, the client 103 can append to the file data in the server 102, or modify the write operation on the file data in the server 102, thereby modifying the data within the specified range requested to be downloaded by the client 101.

[0106] In the example shown in Figure 4, during the process of server 102 sending data within a specified range and a first check code in blocks, since the data within the specified range may be modified, the server 102 calculates the first check code in real time based on the sent data blocks, and carries the first check code in the last data block to be sent.

[0107] In the embodiment of the present application, since the check code is carried in the tail data block, the server can generate the check code in real time based on the check of the sent data block, thereby avoiding the error of the generated check code caused by the modification of the data within the specified range after the check code is generated. The server generates the check code in real time based on the check of the sent data block, which improves the accuracy of the check code and further improves the accuracy of the data verification process.

[0108] Step 204: The client generates a second verification code based on the data within the specified range.

[0109] After receiving the number within the specified range sent by the server 102, the client 101 generates a second check code based on the data within the specified range. The check algorithm used by the client 101 to generate the second check code is the same as the check algorithm used by the server 102 to generate the first check code.

[0110] Specifically, when the server 102 transmits data within a specified range and a first check code to the client 101 in blocks, after the client 101 receives the data blocks sent by the server 102, the client 101 parses the message header field in the data blocks to identify the data within the specified range and the first check code. If the message header field of the data block is a hexadecimal data length, the data block carries the data within the specified range. If the message header field of the data block is 0, the data block is a tail data block, and the data block carries the first check code.

[0111] After receiving the data blocks sent by the server 102, the client 101 reads the data portion of the data blocks according to the block transmission protocol to obtain data within a specified range. The client 101 generates a second check code based on the data within the specified range.

[0112] Continuing with FIG3 , in steps c through d of the example shown in FIG3 , client 101 receives data chunks sent by server 102. The data chunks include chunk1, ..., chunkx. When client 101 receives a data chunk with a hexadecimal data length in the message header field, the data chunk carries a specified range of data. When client 101 receives a data chunk with a message header field of 0, the data chunk is a tail data chunk. At this point, client 101 completes data reception and reads the first checksum sent by server 102 from the tail data chunk.

[0113] In steps c and d of the example shown in FIG3 , after the client 101 completes data reception, it splices the data portions of the multiple data blocks to obtain downloaded data, which is the data within the specified range requested to be downloaded by the client 101. The client 101 generates a second verification code based on the downloaded data. For example, the client generates an MD5 verification code based on the downloaded data.

[0114] In the embodiment of the present application, the server 102 can indicate the content of the data block through the message header field of the data block, so that the client 101 can identify the content of the data block based on the message header field of the data block and determine the tail data block and the first check code.

[0115] Step 205: The client verifies the consistency of the data in the specified range based on the first verification code and the second verification code. If the first verification code is consistent with the second verification code, it is determined that the data received by the client is consistent with the data sent by the server.

[0116] After the client 101 generates a second verification code based on the received downloaded data, it verifies the data consistency between the downloaded data and the data within the specified range of the request based on the first verification code and the second verification code. If the first verification code is consistent with the second verification code, it is determined that the data received by the client 101 is consistent with the data sent by the server 102.

[0117] In the embodiment of the present application, after the client 101 receives the data sent by the server 102, it can generate a verification code again based on the received data, and compare the generated verification code with the verification sent by the server 102, thereby verifying the data within the specified range sent by the server 102, thereby improving the accuracy of the client's data verification.

[0118] Please continue to refer to Figure 3. In step e of the example shown in Figure 3, the client 101 performs verification calculations locally based on the received downloaded data, and after generating a second verification code, compares the second verification code with the first verification code sent by the server 102. If the second verification code is consistent with the first verification code, it means that the data within the specified range downloaded by the client 101 is the same as the data within the specified range sent by the server 102, and the data verification of the client 101 is successful. Otherwise, the data verification of the client 101 fails.

[0119] In the example shown in Figure 3, for example, the client 101 generates a verification code based on the downloaded data as MD5 value 2, and the verification code sent by the server 102 is MD5 value 1. If MD5 value 1 is equal to MD5 value 2, the client 101 data download is successful; if MD5 value 1 is not equal to MD5 value 2, the client 101 data download fails.

[0120] It can be seen from the above embodiments that the client of the cloud storage system in the embodiments of the present application can download data within a specified range. At the same time, the server can generate a verification code for the data within the specified range separately, and send the data and verification code within the specified range to the client through block transmission, thereby improving the data verification flexibility of the client and server in the cloud storage system during data transmission.

[0121] It is understandable that the data verification method provided in the embodiment of the present application can be applied to other data verification scenarios. For example, the server can also verify the data uploaded by the client based on the data verification method. Specifically, for example, the server receives a data upload request sent by the client, and the data upload request includes the uploaded data and a third verification code. The server generates a fourth verification code based on the uploaded data. If the fourth verification code is consistent with the third verification code, the data received by the server is consistent with the data sent by the client.

[0122] Based on the above method embodiment, the embodiment of the present application also provides a data verification device. The data verification device provided by the embodiment of the present application is described in detail below.

[0123] Please refer to Figure 5, which is a schematic diagram of the structure of a data verification device provided in an embodiment of the present application. In the example shown in Figure 5, the data verification device 500 is used to implement the various steps performed by the server or client of the cloud storage system in the above embodiments. The data verification device 500 includes a transceiver unit 501 and a processing unit 502.

[0124] The transceiver unit 501 is configured to receive a data download request from a client, requesting download of data within a specified range from the server. The processing unit 502 is configured to generate a first verification code based on the data within the specified range, the first verification code being used to verify data consistency between the data sent by the server and the data received by the client. The transceiver unit 501 is further configured to send the data within the specified range and the first verification code to the client, so that the client verifies the data within the specified range based on the first verification code.

[0125] In a possible implementation, the processing unit 502 is specifically configured to split data within a specified range into multiple data blocks, and send the multiple data blocks and a first check code to the client, wherein the first check code is located in the tail data block of the multiple data blocks.

[0126] In a possible implementation, each data block includes a message header field and a data portion, the first check code is located in the data portion of the tail data block, and the message header field of the tail data block is used by the client to identify the tail data block.

[0127] In a possible implementation, the processing unit 502 is specifically configured to determine whether data within a specified range has been modified, and the server generates a first check code based on the sent data blocks before sending the tail data blocks.

[0128] In one possible implementation, data within a specified range is used by the client to generate a second verification code. If the second verification code is consistent with the first verification code, the data received by the client is consistent with the data sent by the server.

[0129] In a possible implementation, the transceiver unit 501 is specifically configured to send data within a specified range and a first check code to the client based on a block transfer encoding mechanism of the Hypertext Transfer Protocol HTTP / 1.1.

[0130] It is understandable that the transceiver unit 501 and the processing unit 502 in the data verification device 500 can be mapped as functional modules to the various modules in the cloud storage system 10 in Figure 1a, thereby realizing the functions of the various modules in the cloud storage system 10.

[0131] It should be understood that the division of units in the above device is merely a division of logical functions. In actual implementation, they can be fully or partially integrated into one physical entity, or they can be physically separated. Moreover, the units in the device can all be implemented in the form of software called through processing elements; or they can all be implemented in the form of hardware; or some units can be implemented in the form of software called through processing elements, and some units can be implemented in the form of hardware. For example, each unit can be a separately established processing element, or it can be integrated into a certain chip of the device. In addition, it can also be stored in the memory in the form of a program, called by a certain processing element of the device and perform the function of the unit. In addition, all or part of these units can be integrated together, or they can be implemented independently. The processing element described here can also be a processor, which can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each unit above can be implemented by the integrated logic circuit of the hardware in the processor element or in the form of software called through the processing element.

[0132] It is worth noting that, for the sake of simplicity of description, the above method embodiments are all expressed as a series of action combinations. However, those skilled in the art should know that this application is not limited to the order of the actions described. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required for this application.

[0133] Other reasonable step combinations that can be thought of by those skilled in the art based on the above description also fall within the scope of protection of this application. Secondly, those skilled in the art should also be familiar with that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by this application.

[0134] Please refer to Figure 6, which is a schematic diagram of the structure of a computing device provided in an embodiment of the present application. As shown in Figure 6, computing device 600 includes: a processor 601, a memory 602, a communication interface 603, and a bus 604. Processor 601, memory 602, and communication interface 603 are coupled via a bus (not labeled in the figure). Memory 602 stores instructions. When the execution instructions in memory 602 are executed, computing device 600 performs the method executed by the server or client in the above method embodiments.

[0135] The computing device 600 may be one or more integrated circuits configured to implement the above method, such as one or more application specific integrated circuits (ASICs), one or more digital signal processors (DSPs), one or more field programmable gate arrays (FPGAs), or a combination of at least two of these integrated circuit forms. For example, when a unit in the apparatus can be implemented in the form of a processing element scheduler, the processing element may be a general-purpose processor, such as a central processing unit (CPU) or other processor that can call a program. For example, these units may be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0136] The processor 601 may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA), other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. The general-purpose processor may be a microprocessor or any conventional processor.

[0137] Memory 602 may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. Non-volatile memory may be read-only memory (ROM), programmable ROM (PROM), erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may be random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0138] The memory 602 stores executable program codes, and the processor 601 executes the executable program codes to respectively implement the functions of the aforementioned units or modules, thereby implementing the aforementioned data verification method. That is, the memory 602 stores instructions for executing the aforementioned data verification method.

[0139] The communication interface 603 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 600 and other devices or a communication network.

[0140] In addition to the data bus, bus 604 may also include a power bus, a control bus, and a status signal bus. The bus may be a Peripheral Component Interconnect Express (PCIe) bus, an Extended Industry Standard Architecture (EISA) bus, a unified bus (Ubus or UB), a Compute Express Link (CXL), or a Cache Coherent Interconnect for Accelerators (CCIX). Buses can be categorized as address buses, data buses, and control buses.

[0141] Please refer to FIG7 , which is a schematic diagram of a computing device cluster provided in an embodiment of the present application. As shown in FIG7 , the computing device cluster 700 includes at least one computing device 600 .

[0142] As shown in Figure 7, the computing device cluster 700 includes at least one computing device 600. The memory 602 in one or more computing devices 600 in the computing device cluster 700 may store the same instructions for executing the above data verification method.

[0143] In some possible implementations, the memory 602 of one or more computing devices 600 in the computing device cluster 700 may also store some instructions for executing the above-mentioned data verification method. In other words, the combination of one or more computing devices 600 can jointly execute the instructions for executing the above-mentioned data verification method.

[0144] It should be noted that the memory 602 in different computing devices 600 in the computing device cluster 700 can store different instructions, each for executing part of the functions of the above-mentioned data verification device. In other words, the instructions stored in the memory 602 in different computing devices 600 can implement the functions of one or more modules in the processing unit and the transceiver unit.

[0145] In some possible implementations, one or more computing devices 600 in the computing device cluster 700 may be connected via a network, which may be a wide area network or a local area network.

[0146] Please refer to Figure 8, which is a schematic diagram of computer devices in a computer cluster provided by an embodiment of the present application connected via a network. As shown in Figure 8, two computing devices 600A and 600B are connected via a network. Specifically, the connection to the network is through a communication interface in each computing device.

[0147] In one possible implementation, the memory of the computing device 600A stores instructions for executing the functions of the transceiver unit, while the memory of the computing device 600B stores instructions for executing the functions of the processing unit.

[0148] It should be understood that the functions of the computing device 600A shown in Figure 8 may also be completed by multiple computing devices. Similarly, the functions of the computing device 600B may also be completed by multiple computing devices.

[0149] In another embodiment of the present application, a computer-readable storage medium is provided, in which computer-executable instructions are stored. When the processor of the device executes the computer-executable instructions, the device executes the method executed by the cloud storage system in the above method embodiment.

[0150] In another embodiment of the present application, a computer program product is provided, comprising computer-executable instructions stored in a computer-readable storage medium. When a processor of a device executes the computer-executable instructions, the device performs the method performed by the cloud storage system in the above method embodiment.

[0151] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0152] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0153] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0154] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0155] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

Claims

1. A data verification method, characterized in that: A server applied to a cloud storage system, wherein the cloud storage system further comprises a client, and the method comprises: The server receives a data download request sent by the client, wherein the data download request is used to request downloading of data within a specified range of the server; The server generates a first verification code based on the data within the specified range, where the first verification code is used to verify data consistency between the data sent by the server and the data received by the client; The server sends the data within the specified range and the first verification code to the client, so that the client verifies the data within the specified range based on the first verification code.

2. The method according to claim 1, characterized in that The server sending the data within the specified range and the first verification code to the client includes: The server splits the data within the specified range into multiple data blocks, and sends the multiple data blocks and the first check code to the client, wherein the first check code is located in the tail data block of the multiple data blocks.

3. The method according to claim 2, characterized in that Each data block includes a message header field and a data part. The first check code is located in the data part of the tail data block. The message header field of the tail data block is used by the client to identify the tail data block.

4. The method according to claim 2 or 3, characterized in that: The server generating a first verification code based on the data within the specified range includes: The server determines that the data within the specified range is modified, and before sending the tail data block, the server generates the first check code based on the sent data block.

5. The method according to any one of claims 1 to 4, characterized in that The data within the specified range is used by the client to generate a second verification code. If the second verification code is consistent with the first verification code, the data received by the client is consistent with the data sent by the server.

6. The method according to any one of claims 1 to 5, characterized in that The server sending the data within the specified range and the first verification code to the client includes: The server sends the data within the specified range and the first check code to the client based on the block transfer encoding mechanism of the Hypertext Transfer Protocol HTTP / 1.

1.

7. A data verification device, characterized in that: The device comprises: A transceiver unit, configured to receive a data download request sent by the client, wherein the data download request is used to request downloading of data within a specified range of the server; A processing unit, configured to generate a first verification code based on the data within the specified range, wherein the first verification code is used to verify data consistency between the data sent by the server and the data received by the client; The transceiver unit is further configured to send the data within the specified range and the first verification code to the client, so that the client verifies the data within the specified range based on the first verification code.

8. The device according to claim 7, characterized in that The processing unit is specifically used for: The data within the specified range is split into a plurality of data blocks, and the plurality of data blocks and the first check code are sent to the client, wherein the first check code is located in the tail data block of the plurality of data blocks.

9. The device according to claim 8, characterized in that Each data block includes a message header field and a data part. The first check code is located in the data part of the tail data block. The message header field of the tail data block is used by the client to identify the tail data block.

10. The device according to claim 8 or 9, characterized in that The processing unit is specifically used for: It is determined that the data within the specified range is modified, and the server generates the first check code based on the sent data blocks before sending the tail data blocks.

11. The device according to any one of claims 7 to 10, characterized in that The data within the specified range is used by the client to generate a second verification code. If the second verification code is consistent with the first verification code, the data received by the client is consistent with the data sent by the server.

12. The device according to any one of claims 7 to 11, characterized in that The transceiver unit is specifically used for: The data within the specified range and the first check code are sent to the client based on the block transfer encoding mechanism of the Hypertext Transfer Protocol HTTP / 1.

1.

13. A computing device, characterized in that: The device comprises a processor coupled to a memory, wherein the processor is used to store instructions. When the instructions are executed by the processor, the computing device performs the method according to any one of claims 1 to 6.

14. A computing device cluster, characterized in that: The system comprises at least one computing device, wherein the computing device comprises a processor, wherein the processor is coupled to a memory, and the processor is used to store instructions. When the instructions are executed by the processor, the computing device cluster executes the method according to any one of claims 1 to 6.

15. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instructions are executed, the computer is caused to perform the method according to any one of claims 1 to 6.

16. A computer program product, comprising instructions, characterized in that: When the instructions are executed, the computer implements the method according to any one of claims 1 to 6.

Images