Data operation method and key value storage service device based on cloud management platform

Abstract

A data operation method and a key value (KV) storage service device based on a cloud management platform, which can improve the processing efficiency of KV data. The described method can be applied to the cloud management platform, wherein the cloud management platform is used for managing an infrastructure providing a KV storage service, and the infrastructure comprises a computing node. The cloud management platform receives a data scanning request sent by a first client, and then returns one or more pieces of KV data and a hash value of a key thereof to the first client on the basis of a target hash value range in the data scanning request. The cloud management platform further receives a data operation request sent by the first client, the data operation request comprising a hash value of a key of at least one piece of KV data and a first data operation, wherein the one or more pieces of KV data comprise the at least one piece of KV data; and then the cloud management platform notifies the computing node in the infrastructure to perform the first data operation on the at least one piece of KV data on the basis of the data operation request.

Description

Data operation method and key-value storage service device based on cloud management platform

[0001] The present application claims priority to the Chinese patent application No. 202410719616.8, filed on June 5, 2024, entitled "Data operation method, storage system and computer readable storage medium", and the Chinese patent application No. 202410979303.6, filed on July 19, 2024, entitled "Data operation method and key-value storage service device based on cloud management platform", the contents of which are incorporated herein by reference in their entirety. TECHNICAL FIELD

[0002] The present application relates to the field of cloud computing technology, and in particular to a data operation method and a key-value storage service device based on a cloud management platform. BACKGROUND

[0003] Key-value storage is a way of managing data in the form of key-value pairs. In key-value storage, a shard key usually needs to be set for key value (KV) data to be written, and based on the shard key, KV data in the same data table can be stored in different storage nodes. Since the shard key of the KV data is invisible to the user, when the user wants to process specific KV data in the data table, the user first needs to traverse all the KV data in the table to obtain the KV data to be processed, and then can process the obtained KV data accordingly. It can be understood that the traversal operation involved in the above method will affect the overall data processing efficiency. SUMMARY

[0004] The present application provides a data operation method and a key-value storage service device based on a cloud management platform, which can improve the KV data processing efficiency.

[0005] In a first aspect, the present application provides a data operation method. The method can be applied to a cloud management platform, which is used to provide an infrastructure for a key-value (KV) storage service. The infrastructure includes computing nodes, and the infrastructure includes a plurality of data centers in different regions, and each data center includes a plurality of servers. Specifically, the cloud management platform receives a data scan request sent by a first client, the data scan request including a target hash value range, and then returns one or more pieces of KV data and the hash values of the keys of the KV data to the first client according to the target hash value range. The cloud management platform also receives a first data operation request sent by the first client, the first data operation request including the hash values of the keys of at least one piece of KV data and a first data operation, wherein the one or more pieces of KV data include the at least one piece of KV data. Then, the cloud management platform instructs the computing nodes to perform the first data operation on the at least one piece of KV data according to the first data operation request.

[0006] In the technical solution provided by the present application, since the data scan request of the first client carries the target hash value range, the computing nodes in the infrastructure can directly determine the corresponding KV data based on the target hash value range after receiving the data scan request through the cloud management platform, so as to return the required KV data to the first client. Moreover, since the first data operation request carries the hash values of the keys of the KV data to be processed, the computing nodes in the infrastructure can not need to repeatedly perform hash calculation on the keys of the KV data to be processed after receiving the first data operation request through the cloud management platform. In general, the technical solution provided by the present application can improve the processing efficiency of the KV data.

[0007] In a possible implementation, the cloud management platform is further configured to receive a plurality of data scan requests in parallel, wherein each data scan request carries a hash value range, and the plurality of data scan requests are used to instruct to perform data scan operations on different KV data in a same data table. In this way, a user can use a plurality of clients to perform parallel scanning on the same data table, thereby improving the efficiency of data scanning, and further enabling the plurality of clients to obtain the KV data to be processed and the hash values of the keys of the KV data more quickly.

[0008] In a possible implementation, the cloud management platform is further configured to receive a plurality of data operation requests in parallel, and instruct the computing nodes to process the KV data in the plurality of data operation requests in parallel. Each data operation request carries at least one piece of KV data obtained by scanning and the hash values of the keys of the KV data. In this way, parallel processing of different KV data in a same data table can be implemented, thereby improving the data processing efficiency.

[0009] In a possible implementation, the target hash value range can include one hash value. Then, the cloud management platform further receives a second data operation request of a second client, where the second client can be the first client or another client, and the second data operation request includes a hash value of a key of a piece of KV data and a second data operation. Then, the cloud management platform instructs the computing node to perform the second data operation on the piece of KV data according to the hash value of the key of the piece of KV data. In this way, the cloud management platform can support performing data operations on a single piece of KV data.

[0010] In a possible implementation, the infrastructure managed by the cloud management platform further includes a plurality of storage nodes. The cloud management platform further receives a write data request, where the write data request includes a data table, and the data table includes one or more pieces of KV data. Then, the cloud management platform instructs the computing node to store the pieces of KV data in the data table to the plurality of storage nodes according to the partition keys of the pieces of KV data in the data table, where the partition keys of the pieces of KV data include the keys of the pieces of KV data and hash values of the keys. In this way, the cloud management platform can store the pieces of KV data in the data table to the plurality of storage nodes according to the partition keys of the pieces of KV data.

[0011] It should be understood that the distribution of the hash values calculated by using the hash algorithm in the value range is uniform, and therefore, by using the above implementation, the pieces of KV data in the data table can be uniformly distributed to the plurality of partitions, thereby avoiding that a specific partition carries too many data operation requests.

[0012] In a possible implementation, in the partition keys of the pieces of KV data, the hash value of the key of the piece of KV data is a prefix of the key.

[0013] In a possible implementation, before receiving the first data scan request sent by the first client, the cloud management platform further provides the first client with a hash value range of the keys of the pieces of KV data in the data table. In this way, the user can know the total range of the hash values of the keys of the pieces of KV data in the data table through the first client. Accordingly, when the user wants to process a specific piece of KV data in the data table, the corresponding data scan request can be configured for the client based on the range.

[0014] In a possible implementation, the multiple hash value ranges to which the hash values of the keys of the KV data in the data table belong are determined based on a partition rule. The first data operation request further includes the keys of the one or more pieces of KV data. The cloud management platform further instructs the computing node to determine that the hash value of the key of the KV data carried in the first data operation request does not match the multiple hash value ranges in the partition rule, and to calculate a new hash value based on the key of the KV data carried in the first data operation request. In this way, the hash value carried in the first data operation request can be checked, and data operation failure caused by an error in the hash value carried in the first data operation request can be avoided.

[0015] In a second aspect, the present application provides a cloud management platform-based key-value storage service apparatus. The cloud management platform is configured to manage infrastructure providing a key-value storage service, the infrastructure including a computing node, and the infrastructure including multiple data centers in different regions, each data center including multiple servers. The apparatus includes a receiving module, a sending module, and a processing module. The receiving module is configured to receive a data scan request sent by a first client, the data scan request including a target hash value range. The sending module is configured to send one or more pieces of KV data and hash values of keys of the KV data to the first client according to the target hash value range. The receiving module is further configured to receive a first data operation request sent by the first client, the first data operation request including a hash value of a key of at least one piece of KV data and a first data operation, wherein the one or more pieces of KV data include the at least one piece of KV data. The processing module is configured to instruct the computing node to perform the first data operation on the at least one piece of KV data according to the first data operation request.

[0016] In a possible implementation, the receiving module is further configured to receive multiple data scan requests in parallel, wherein each data scan request carries a hash value range, and the multiple data scan requests are configured to indicate performing data scan operations on different pieces of KV data in a same data table.

[0017] In a possible implementation, the receiving module is further configured to receive multiple data operation requests in parallel. The processing module is further configured to instruct the computing node to process the KV data in the multiple data operation requests in parallel, wherein each data operation request carries at least one piece of KV data obtained through scanning and a hash value of a key of the KV data.

[0018] In a possible implementation, the target hash value range can include one hash value. The receiving module is further configured to receive a second data operation request of a second client, the second data operation request including a hash value of a key of one piece of KV data and a second data operation. The processing module is further configured to instruct the computing node to perform the second data operation on the one piece of KV data according to the hash value of the key of the one piece of KV data.

[0019] In a possible implementation, the infrastructure provided by the cloud management platform further includes a plurality of storage nodes. The receiving module is further configured to receive a write data request, the write data request including a data table, the data table including the one or more pieces of KV data. The processing module is further configured to instruct the computing node to obtain, according to keys of the KV data in the data table, partition keys of the KV data in the data table, and store the KV data in the data table in the plurality of storage nodes according to the partition keys of the KV data in the data table. The partition key of the KV data includes the key of the KV data and a hash value of the key.

[0020] In a possible implementation, in the partition key of the KV data, the hash value of the key of the KV data is a prefix of the key.

[0021] In a possible implementation, the sending module is further configured to provide, to the first client, a hash value range of the key of the KV data in the data table.

[0022] In a possible implementation, the plurality of hash value ranges to which the hash value of the key of the KV data in the data table belongs are determined based on a partition rule. The first data operation request further includes the key of the one or more pieces of KV data. The processing module is further configured to instruct the computing node to determine that the hash value of the key of the KV data carried in the first data operation request does not match any of the plurality of hash value ranges in the partition rule, and then calculate a new hash value according to the key of the KV data carried in the first data operation request.

[0023] In a third aspect, a computing device is provided. The computing device includes a processor and a memory. The processor is configured to execute instructions stored in the memory to cause the computing device to perform some or all of the methods described in the first aspect and any possible implementation thereof.

[0024] In a fourth aspect, a computing device cluster is provided. The computing device cluster includes at least one computing device, and each computing device includes a processor and a memory. The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device to cause the computing device cluster to perform some or all of the methods described in the first aspect and any possible implementation thereof.

[0025] In a fifth aspect, a computer program product including instructions is provided. The computer program product can be software or program product including instructions, which can run on a computing device or be stored in any available medium. When the computer program product runs on the computing device, the computing device is caused to perform some or all of the methods described in the first aspect and any possible implementation thereof.

[0026] In a sixth aspect, the present application provides a computer readable storage medium. The computer storage medium includes computer program instructions, when the computer program instructions are executed by a computing device, cause the computing device to perform part or all of the method described in the first aspect and any implementation form thereof. BRIEF DESCRIPTION OF DRAWINGS

[0027] FIG. 1 is a schematic diagram of a key-value storage scenario provided by the present application;

[0028] FIG. 2 is a flowchart of a data operation method provided by the present application;

[0029] FIG. 3 is a flowchart of obtaining a hash value of KV data and its key provided by the present application;

[0030] FIG. 4 is a flowchart of performing a data operation on KV data provided by the present application;

[0031] FIG. 5 is a flowchart of storing a target data table provided by the present application;

[0032] FIG. 6 is a schematic diagram of the structure of a cloud management platform provided by the present application;

[0033] FIG. 7 is a schematic diagram of the structure of a computing device provided by the present application;

[0034] FIG. 8 is a schematic diagram of the structure of a computing device cluster provided by the present application;

[0035] FIG. 9 is a schematic diagram of another structure of a computing device cluster provided by the present application. DETAILED DESCRIPTION

[0036] In order to improve the processing efficiency of KV data, the present application provides a data operation method. The method can be applied to a cloud management platform, wherein the cloud management platform is used to manage an infrastructure providing a key-value storage service, and the infrastructure includes a plurality of computing nodes and a plurality of storage nodes. When a user obtains the key-value storage service, the user can obtain the total range of the hash value of the key of the KV data in the data table, so that the user can configure a data scanning request based on the total range of the hash value obtained above, so that it carries the hash value range of the KV data to be scanned, and sends the data scanning request to the cloud management platform through the client. Correspondingly, the cloud management platform can instruct the computing nodes in the infrastructure to obtain the required KV data from the storage nodes and return it to the client, so that the client can further perform data operations such as data update and deletion on the returned KV data. It should be understood that in the technical solution provided by the present application, since the user can obtain the distribution rule and distribution range of the KV data in the data table, it is convenient for the user to configure the data operation task for the client, and the processing efficiency of the KV data is improved.

[0037] The technical solutions provided by the present application will be described below with reference to the drawings.

[0038] Referring to FIG. 1, FIG. 1 exemplarily shows a schematic diagram of a key-value storage scenario to which the present application is applicable. As shown in FIG. 1, the scenario includes a client 100, a cloud management platform 200, and an infrastructure 300 managed by the cloud management platform 200 and used to provide a key-value storage service. The client 100, the cloud management platform 200, and the infrastructure 300 can be connected through a network, which can be a wide area network or a local area network.

[0039] The client 100 can be a software or an application deployed on a terminal device, for example, a browser, an application (APP), etc. The terminal device here can be a desktop computer, a notebook computer, a tablet computer, a smart phone, a wearable device, a smart conference device, a vehicle-mounted device, etc., which is not limited in the present application. It should be understood that the present application does not limit the number of clients 100, i.e., the number of clients 100 can be one or more. For the convenience of description, FIG. 1 takes two clients 100 as an example, which are respectively shown as a client 100A and a client 100B in the figure.

[0040] The cloud management platform 200 can be deployed on a single computing device or a computing device cluster composed of multiple computing devices, and the computing device here can be a server, for example, a server in a data center. The cloud management platform 200 can provide an interface related to the key-value storage service, for example, a graphical user interface (GUI) or an application programming interface (API). A user can register an account and a password on the cloud management platform 200 through the above interface, and log in to the cloud management platform 200 through the registered account and password. After successful login, the user can also select and purchase the key-value storage service on the cloud management platform 200 through the above interface. After successful purchase, the user can also send a request (for example, a data scanning request, a data operation request, a data writing request, etc.) to the cloud management platform 200 through the above interface, and then the cloud management platform 200 invokes the infrastructure 300 to provide the key-value storage service for the user.

[0041] The infrastructure 300 includes a plurality of computing nodes, which can include one or more routing nodes 310 and a plurality of indexing nodes 320. Among them, the routing nodes 310 and the indexing nodes 320 can be devices with computing capabilities, for example, servers, virtual machines or containers. The infrastructure 300 further includes a plurality of storage nodes 330. Among them, the storage nodes 330 can be devices with persistent storage capabilities, for example, the storage nodes 330 can include disks or other types of storage media, and can optionally include network cards, central processing units (CPUs) or data processing units (DPUs) and the like control components.

[0042] The routing nodes 310 and the indexing nodes 320 can be deployed on the same computing device or on different computing devices. The routing nodes 310, the indexing nodes 320 and the storage nodes 330 can be deployed in the same region or distributed in different regions. Further, the routing nodes 310, the indexing nodes 320 and the storage nodes 330 can be distributed in the same availability zone (AZ) or distributed in different AZs, each AZ including a data center or a plurality of data centers with close geographical locations. Among them, usually one region can include a plurality of AZs. Similarly, the routing nodes 310, the indexing nodes 320 and the storage nodes 330 can be distributed in the same virtual private cloud (VPC) or in a plurality of VPCs. Among them, usually one VPC is set in one region, and a communication gateway needs to be set in each VPC for cross-region communication between two VPCs in the same region and between VPCs in different regions, and the interconnection between VPCs is realized through the communication gateway.

[0043] It is worth noting that the key-value storage service provided by the infrastructure 300 can be implemented according to the actual business needs in addition to the storage-computing separation architecture shown in FIG. 1. For ease of description, the following embodiments will be described taking the storage-computing separation architecture of the key-value storage service as an example.

[0044] In the key-value storage scenario shown in FIG. 1, when a user wants to process KV data in a data table, the user can send a data scanning request to the cloud management platform 200 through the client 100, where the data scanning request carries a hash value range, which is a range to which a hash value of a key of the KV data required by the user in the data table belongs. After receiving the data scanning request sent by the client 100, the cloud management platform 200 can inform the infrastructure 300 to determine the KV data required by the user and the hash value of the key thereof according to the hash value range in the request, and return the determined KV data and the hash value of the key thereof to the client 100, so that the client 100 obtains the KV data to be processed and the hash value of the key thereof. Then, the client 100 can also send a data operation request to the cloud management platform 200, where the data operation request carries the obtained KV data and the hash value of the key thereof, and a data operation to be performed on the KV data. Correspondingly, the cloud management platform 200 can inform the infrastructure 300 to perform the corresponding data operation on the KV data carried in the data operation request according to the data operation request sent by the client 100. In this way, the user can obtain the key-value storage service provided by the cloud management platform 200.

[0045] It should be noted that in actual application, the user can request the key-value storage service from the cloud management platform 200 through a single client 100, or through multiple clients 100. When the user requests the key-value storage service from the cloud management platform 200 through multiple clients 100, the cloud management platform 200 provides the key-value storage service for each client 100 in a similar manner, and therefore, for the sake of description, the following embodiments will be described by taking multiple clients 100 (specifically, the client 100A and the client 100B) as an example.

[0046] Next, the cloud management platform 200 how to provide the key-value storage service for the user will be described in detail in combination with the flowchart of the data operation method shown in FIG. 2.

[0047] S101, the client 100A sends a first data scanning request to the cloud management platform 200, and the client 100B sends a second data scanning request to the cloud management platform 200. Correspondingly, the cloud management platform 200 receives the first data scanning request and the second data scanning request in parallel.

[0048] First of all, it should be noted that the storage node 330 pre-stores a target data table, and the target data table includes multiple pieces of KV data, each piece of KV data having a partition key. For the sake of brevity, the KV data, the partition key of the KV data, and the storage process of the target data table will not be described here, and can be referred to S201 to S203 in the following.

[0049] The first data scan request and the second data scan request are used to instruct the data scan operation on different KV data in the target data table. In a specific implementation, the first data scan request can include a first hash value range, and the first hash value range is used to indicate the KV data (hereinafter referred to as "first KV data") requested to be scanned by the client 100A. The number of the first KV data can be one or more, and when the first hash value range includes only one hash value, the first KV data indicated by the first hash value range can be one. For ease of understanding, an example is used for description here, assuming that the first hash value range is (0, 1], and the hash value is an integer, then the first hash value range includes only one hash value "1". If there is only one KV data with the hash value 1 in the data table, the first hash value range indicates one first KV data. Similarly, the second data scan request can include a second hash value range, and the second hash value range is used to indicate the KV data (hereinafter referred to as "second KV data") requested to be scanned by the client 100B. The number of the second KV data can be one or more, which will not be described here.

[0050] The first hash value range and the second hash value range can be configured by the user. Specifically, as known later (specifically, S2022), the hash value of the key of the KV data is obtained by the routing node 310 based on the hash algorithm to perform hash calculation on the key of the KV data, and therefore the value range of the hash value of the key of the KV data in the target data table is fixed, for example, the hash value obtained by using the MurmurHash algorithm is in the range of [0, uint32_max], where uint32_max is a constant, that is, "FFFFFFFF". The routing node 310 can provide the user with the hash value range of the key of all KV data in the target data table through the cloud management platform 200. That is, the user can know the hash value range of the key of all KV data in the target data table. Then, when the user wants to operate the target data table through multiple clients 100 in parallel, the user can divide the determined total range uniformly into multiple hash value ranges according to the number of clients 100 accessed in parallel, and one hash value range is used to indicate the KV data required to be operated by one client 100.

[0051] Exemplarily, still taking the MurmurHash algorithm as an example, the hash value range of the key of the KV data in the target data table calculated by using the algorithm is [0, uint32_max]. When the parallel operating clients 100 are the client 100A and the client 100B, the [0, uint32_max] can be evenly divided into two hash value ranges, i.e., a first hash value range and a second hash value range. The first hash value range is [0, uint32_max / 2], and the second hash value range is (uint32_max / 2, uint32_max].

[0052] It should be understood that the distribution of the hash value calculated by using the hash algorithm in the value range is uniform. Therefore, by the above implementation manner, i.e., the hash value range of the key of the KV data in the target data table is evenly divided into multiple hash value ranges according to the number of the parallel operating clients 100, and the data scanning request of the client 100 is configured according to the hash value range obtained by the above division, the data amount scanned by the multiple clients 100 can be uniform, and thus the multiple clients 100 can operate in parallel without interference, and the efficiency of performing data scanning on the target data table can be improved.

[0053] In addition, it should be further noted that the "parallel" in "the cloud management platform 200 receives the first data scanning request and the second data scanning request in parallel" in this step refers to that the difference between the time when the cloud management platform 200 receives the first data scanning request and the time when the cloud management platform 200 receives the second data scanning request is less than a first threshold. The first threshold can be set by a user or by the cloud management platform 200, and the application does not limit this.

[0054] S102, the cloud management platform 200 returns the first KV data and the hash value of the key thereof to the client 100A according to the first data scanning request, and returns the second KV data and the hash value of the key thereof to the client 100B according to the second data scanning request. Correspondingly, the client 100A receives the first KV data and the hash value of the key thereof, and the client 100B receives the second KV data and the hash value of the key thereof.

[0055] Specifically, the cloud management platform 200 instructs the infrastructure 300 to obtain the first KV data and the hash value of the key thereof from the target data table according to the first hash value range in the first data scan request, and to obtain the second KV data and the hash value of the key thereof from the target data table according to the second hash value range in the second data scan request. Then, the cloud management platform 200 returns the obtained first KV data and the hash value of the key thereof to the client 100A, and returns the obtained second KV data and the hash value of the key thereof to the client 100B. The first KV data returned to the client 100A can be the complete first KV data, or can be partial information of the first KV data, for example, only the key of the first KV data and part of the value of the first KV data can be returned. Similarly, the second KV data returned to the client 100B can be the complete second KV data, or can be partial information of the second KV data, for example, only the key of the second KV data and part of the value of the second KV data can be returned.

[0056] More specifically, as shown in FIG. 3, the infrastructure 300 can obtain the first KV data and the hash value of the key thereof, and the second KV data and the hash value of the key thereof from the target data table by the following steps:

[0057] S1021, the routing node 310 receives the first data scan request and the second data scan request sent by the cloud management platform 200.

[0058] As a possible implementation, the routing node 310 receiving the first data scan request and the routing node 310 receiving the second data scan request can be the same routing node 310 or different routing nodes 310. When the routing node 310 receiving the first data scan request and the routing node 310 receiving the second data scan request are different routing nodes 310 (for example, can be routing node 310A and routing node 310B respectively), the routing node 310A receiving the first data scan request and the routing node 310B receiving the second data scan request can be implemented in the following manner: the network between the cloud management platform 200 and the infrastructure 300 includes a network device, the network device pre-stores a load balancing policy, and the load balancing policy is used to indicate that a routing node 310 that can receive and process an external request is selected from a plurality of routing nodes 310 in the infrastructure 300. Therefore, the first data scan request sent by the client 100A first reaches the above-mentioned network device, and then the network device forwards the first data scan request to the routing node 310A according to the load balancing policy to select the routing node 310A from the plurality of routing nodes 310. Thus, the routing node 310A receives the first data scan request. Similarly, the second data scan request sent by the client 100B first reaches the above-mentioned network device, and then the network device forwards the second data scan request to the routing node 310B according to the load balancing policy to select the routing node 310B from the plurality of routing nodes 310. Thus, the routing node 310B receives the second data scan request.

[0059] S1022, the routing node 310 determines a first index node according to the first hash value range in the first data scan request and determines a second index node according to the second hash value range in the second data scan request.

[0060] Specifically, the routing node 310 is pre-provisioned with a partition rule, the partition rule includes the identification of the M index nodes 320 and the hash value range corresponding to each index node 320 indicated by the identification, wherein the M index nodes 320 include the first index node and the second index node, M is an integer greater than 2, and the hash value range in the partition rule contains the hash value range of the key of the KV data in the target data table. Therefore, the routing node 310 can determine the first index node according to the matching relationship between the first hash value range in the first data scan request and the hash value range in the partition rule, wherein the first index node is the index node 320 indicated by the identification corresponding to the matched hash value range. Similarly, the routing node 310 can determine the second index node according to the matching relationship between the second hash value range in the second data scan request and the hash value range in the partition rule, wherein the second index node is the index node 320 indicated by the identification corresponding to the matched hash value range.

[0061] Taking the first hash value range and the hash value range Q in the partition rule as an example, the routing node 310 can determine the matching relationship between the two in the following manner: the routing node 310 obtains the start value R1, the end value R2 and the interval type of the first hash value range, and the start value Q1, the end value Q2 and the interval type of the hash value range Q. Then, the routing node 310 respectively compares the start value R1 of the first hash value range and the start value Q1 of the hash value range Q, the end value R2 of the first hash value range and the end value Q2 of the hash value range Q, the interval type of the first hash value range and the interval type of the hash value range Q, and determines whether the first hash value range and the hash value range Q have an intersection according to the comparison result. When the first hash value range and the hash value range Q have an intersection, it means that the first hash value range and the hash value range Q match; otherwise, when the first hash value range and the hash value range Q do not have an intersection, it means that the first hash value range and the hash value range Q do not match.

[0062] It should be understood that, although the above process only describes how the routing node 310 determines the matching relationship between the first hash value range and the hash value range Q, in actual application, the matching relationship between the first hash value range and other hash value ranges in the partition rule can also be determined according to the above process, and similarly, the matching relationship between the second hash value range and the hash value ranges in the partition rule can also be determined according to the above process. Thus, the routing node 310 can determine the first index node and the second index node.

[0063] In addition, in actual application, the first hash value range can match one or more hash value ranges in the partition rule, and the second hash value range can also match one or more hash value ranges in the partition rule. Therefore, the number of the first index node can be one or more, and the number of the second index node can also be one or more. In addition, one first index node and one second index node can be the same index node 320, or can be different index nodes 320, for example, when a certain index node 320 corresponds to a hash value range including part of the first hash value range and part of the second hash value range, the index node 320 is both the first index node and the second index node.

[0064] S1023, the routing node 310 sends a first data scanning request to the first index node and a second data scanning request to the second index node. Correspondingly, the first index node receives the first data scanning request sent by the routing node 310, and the second index node receives the second data scanning request sent by the routing node 310.

[0065] S1024, the first index node acquires the first KV data and the hash value of the key thereof from the first storage node according to the first data scanning request, and the second index node acquires the second KV data and the hash value of the key thereof from the second storage node according to the second data scanning request.

[0066] Specifically, the first data scanning request further includes the identification of the target data table. Therefore, the first index node can retrieve the first KV data and the hash value of the key thereof in the target data table from the first storage node according to the identification of the target data table in the first data scanning request and the first hash value range. Similarly, the second data scanning request further includes the identification of the target data table. Therefore, the second index node can retrieve the second KV data and the hash value of the key thereof in the target data table from the second storage node according to the identification of the target data table in the second data scanning request and the second hash value range. It should be noted that the first storage node and the second storage node can be the same storage node 330, or can be different storage nodes 330.

[0067] It should be understood that through the above S102, the user can obtain the key of the first KV data and the key of the second KV data, and further obtain the distribution rule and distribution range of the first KV data and the second KV data. However, it should be noted that the cloud management platform 200 only supports the user to view the hash value of the key of the first KV data and the hash value of the key of the second KV data, but does not support the user to modify the hash value of the key of the first KV data and the hash value of the key of the second KV data.

[0068] S103, the client 100A sends a first data operation request to the cloud management platform 200, and the client 100B sends a second data operation request to the cloud management platform 200. Correspondingly, the cloud management platform 200 receives the first data operation request and the second data operation request in parallel.

[0069] The first data operation request includes the first KV data and the hash value of the key thereof, and the second data operation request includes the second KV data and the hash value of the key thereof. The first KV data in the first data operation request can be part or all of the first KV data returned by the cloud management platform 200 in the above S102. Similarly, the second KV data in the second data operation request can be part or all of the second KV data returned by the cloud management platform 200 in the above S102, which can be determined according to the business needs of the user. That is, the first KV data returned by the cloud management platform 200 in the above S102 includes the first KV data carried in the first data operation request, and the second KV data returned by the cloud management platform 200 includes the second KV data carried in the second data operation request.

[0070] The first data operation request further includes a data operation requested by the client 100A to perform on the first KV data, and the second data operation request further includes a data operation requested by the client 100B to perform on the second KV data. The data operation indicated by the first data operation request and the data operation indicated by the second data operation request can be the same type of data operation, or can be different types of data operation. The data operation can be a data update operation, a data deletion operation, or a data query operation. The data update operation is used to indicate modification of an attribute of the KV data, for example, a partition key or a sort key of the KV data. The data deletion operation is used to indicate deletion of the KV data. The data query operation is used to indicate querying of the KV data. For example, when the first KV data returned by the cloud management platform 200 to the client 100A in the above S102 is not complete first KV data, the complete first KV data can be obtained through the data query operation.

[0071] In addition, it should be noted that, in the step, the "in parallel" in "the cloud management platform 200 receives the first data operation request and the second data operation request in parallel" means that the difference between the time when the cloud management platform 200 receives the first data operation request and the time when the cloud management platform 200 receives the second data operation request is less than a second threshold value. The second threshold value can be set by a user or by the cloud management platform 200, and the present application does not limit this.

[0072] S104, the cloud management platform 200 returns a data operation result of the first KV data to the client 100A according to the first data operation request, and returns a data operation result of the second KV data to the client 100B according to the second data operation request.

[0073] Specifically, the cloud management platform 200 instructs the infrastructure 300 to perform a data operation on the first KV data according to the first data operation request, and to perform a data operation on the second KV data according to the second data operation request. Then, the cloud management platform 200 returns a data operation result of the first KV data to the client 100A, and returns a data operation result of the second KV data to the client 100B.

[0074] More specifically, as shown in FIG. 4, the infrastructure 300 can perform data operations on the first KV data and the second KV data through the following steps:

[0075] S1041, the routing node 310 receives the first data operation request and the second data operation request sent by the cloud management platform 200.

[0076] It should be understood that, similar to the above S1021, the routing node 310 that receives the first data operation request and the routing node 310 that receives the second data operation request in this step can be the same routing node 310, or can be different routing nodes 310, and the present application does not limit this.

[0077] S1042, the routing node 310 determines a first index node according to the hash value of the key of the first KV data in the first data operation request, and determines a second index node according to the hash value of the key of the second KV data in the second data operation request.

[0078] Specifically, the routing node 310 determines the first index node according to the matching relationship between the hash value of the key of the first KV data in the first data operation request and the hash value range in the partition rule, where the hash value of the key of the first KV data belongs to the hash value range corresponding to the first index node. Similarly, the routing node 310 can determine the second index node according to the matching relationship between the hash value of the key of the second KV data in the second data operation request and the hash value range in the partition rule, where the hash value of the key of the second KV data belongs to the hash value range corresponding to the second index node.

[0079] Taking a hash value (hereinafter referred to as "hash value P") of a key of a first KV data and a hash value range Q in the partition rule as an example, the routing node 310 can determine the matching relationship between the two in the following manner: the routing node 310 determines the start value Q1 and the end value Q2 of the hash value range Q, and the interval type of the hash value range Q. When any one of the following conditions is met, it means that the hash value P belongs to the hash value range Q, that is, the hash value P matches the hash value range Q. On the contrary, when all the following conditions are not met, it means that the hash value P does not belong to the hash value range Q, that is, the hash value P does not match the hash value range Q. The above conditions include: the interval type of the hash value range Q is open interval (i.e. the hash value range Q is (Q1, Q2)), the start value Q1 < the hash value P < the end value Q2; the interval type of the hash value range Q is closed interval (i.e. the hash value range Q is [Q1, Q2]), the start value Q1 ≤ the hash value P ≤ the end value Q2; the interval type of the hash value range Q is left open right closed (i.e. the hash value range Q is (Q1, Q2]), the start value Q1 < the hash value P ≤ the end value Q2; the interval type of the hash value range Q is left closed right open (i.e. the hash value range Q is [Q1, Q2)), the start value Q1 ≤ the hash value P < the end value Q2.

[0080] It can be understood that although the above process only describes how the routing node 310 determines the matching relationship between the hash value P and the hash value range Q, in actual application, the matching relationship between the hash value P and other hash value ranges in the partition rule can also be determined with reference to the above process, and similarly, the matching relationship between the hash value of the key of other first KV data or second KV data and the hash value range in the partition rule can also be determined with reference to the above process. Thus, the routing node 310 can determine the first index node and the second index node.

[0081] It is worth noting that in actual application, there can be a case that the hash value of the key of a certain piece of first KV data (or second KV data) in the first data operation request (or second data operation request) does not match all hash value ranges in the partition rule. In this case, the routing node 310 can perform hash calculation on the key of the KV data using a hash algorithm to obtain a new hash value, and then determine the corresponding index node 320 according to the matching relationship between the new hash value and the hash value ranges in the partition rule.

[0082] It should be understood that when the first data operation request includes the hash value of the key of the first KV data, the routing node 310 will not perform hash calculation on the key of the first KV data carried in the request after receiving the first data operation request. Similarly, when the second data operation request includes the hash value of the key of the second KV data, the routing node 310 will not perform hash calculation on the key of the second KV data carried in the request after receiving the second data operation request. That is, through the above implementation manner, the routing node 310 can be avoided to repeatedly perform hash calculation on the keys of the KV data carried in the first data operation request and the second data operation request.

[0083] S1043, the routing node 310 sends the first data operation request to the first index node and sends the second data operation request to the second index node. Correspondingly, the first index node receives the first data operation request, and the second index node receives the second data operation request.

[0084] S1044, the first index node performs data operation on the first KV data according to the first data operation request, and the second index node performs data operation on the second KV data according to the second data operation request.

[0085] Specifically, the first index node performs data operation on the first KV data in the first storage node according to the first KV data in the first data operation request. Similarly, the second index node performs data operation on the second KV data in the second storage node according to the second KV data in the second data operation request.

[0086] As known from the foregoing, the data operation indicated by the first data operation request and the second data operation request can be a data update operation, a data deletion operation or a data query operation. The present step will be further introduced in combination with these types of data operations:

[0087] When the data operation indicated by the first data operation request and the second data operation request is a data update operation, after the first index node retrieves the first KV data in the target data table, the first KV data in the table is updated according to the specified update content. When the first KV data is updated successfully, the first index node returns a notification message of successful update to the cloud management platform 200, so that the cloud management platform 200 returns a notification message of successful update to the client 100A. When the first KV data is updated unsuccessfully, the first index node returns a notification message of unsuccessful update to the cloud management platform 200, so that the cloud management platform 200 returns a notification message of unsuccessful update to the client 100A. Similarly, after the second index node retrieves the second KV data in the target data table, the second KV data in the table is updated according to the specified update content. When the second KV data is updated successfully, the second index node returns a notification message of successful update to the cloud management platform 200, so that the cloud management platform 200 returns a notification message of successful update to the client 100B. When the second KV data is updated unsuccessfully, the second index node returns a notification message of unsuccessful update to the cloud management platform 200, so that the cloud management platform 200 returns a notification message of unsuccessful update to the client 100B.

[0088] When the data operation indicated by the first data operation request and the second data operation request is a data deletion operation, after the first index node retrieves the first KV data in the target data table, the first KV data and its related information (including the hash value of the key of the first KV data) in the target data table are deleted. Correspondingly, the first storage node will not include the first KV data and its related information. When the first KV data is deleted successfully, the first index node returns a notification message of successful deletion to the cloud management platform 200, so that the cloud management platform 200 returns a notification message of successful deletion to the client 100A. When the first KV data is deleted unsuccessfully, the first index node returns a notification message of unsuccessful deletion to the cloud management platform 200, so that the cloud management platform 200 returns a notification message of unsuccessful deletion to the client 100A. Similarly, after the second index node retrieves the second KV data in the target data table, the second KV data and its related information (including the hash value of the key of the second KV data) in the target data table are deleted. Correspondingly, the second storage node will not include the second KV data and its related information. When the second KV data is deleted successfully, the second index node returns a notification message of successful deletion to the cloud management platform 200, so that the cloud management platform 200 returns a notification message of successful deletion to the client 100B. When the second KV data is deleted unsuccessfully, the second index node returns a notification message of unsuccessful deletion to the cloud management platform 200, so that the cloud management platform 200 returns a notification message of unsuccessful deletion to the client 100B.

[0089] When the data operations indicated by the first data operation request and the second data operation request are data query operations, if the first index node retrieves the first KV data, the complete first KV data is returned to the cloud management platform 200, so that the cloud management platform 200 returns the complete first KV data to the client 100A; if the first index node does not retrieve the first KV data, a query failure notification message is returned to the cloud management platform 200, so that the cloud management platform 200 returns the query failure notification message to the client 100A. Similarly, if the second index node retrieves the second KV data, the complete second KV data is returned to the cloud management platform 200, so that the cloud management platform 200 returns the complete second KV data to the client 100B; if the second index node retrieves the second KV data, a query failure notification message is returned to the cloud management platform 200, so that the cloud management platform 200 returns the query failure notification message to the client 100B.

[0090] As can be seen from the data operation method described in S101 to S104 above, the key value storage service provided by the cloud management platform 200 supports users to view the hash values of the keys of the KV data in the target data table, so that the users can know the distribution rule and distribution range of the KV data in the table, and then facilitate the users to divide batch processing tasks and use multiple clients to operate the KV data of the target data table in parallel, which not only can fully utilize the storage performance of the storage node 330, but also can improve the processing efficiency of the KV data. In general, the technical solution provided in the present application can be applied to businesses such as timing archiving and behavior analysis. Such businesses usually need to trigger scanning of the data table at regular intervals, filter out the KV data in the table that meets the conditions, and perform batch processing on these data.

[0091] The storage process of the target data table involved in S101 above will be described in detail below in conjunction with FIG. 5.

[0092] S201, the client 100 sends a write data request to the cloud management platform 200. Correspondingly, the cloud management platform 200 receives the write data request.

[0093] The client 100 can be the client 100A or the client 100B in FIG. 1, or other clients not shown in FIG. 1, which are not limited in the present application. The related description of the write data request can be referred to S202 below, which will not be introduced here.

[0094] S202, the cloud management platform 200 notifies the infrastructure 300 to store the target data table according to the write data request.

[0095] The write data request comprises a target data table, the target data table comprises a plurality of pieces of KV data, each piece of KV data comprises a key and a value associated with the key, the key is used to uniquely identify the value, and the key can be composed of values of one or more fields in the KV data. For ease of understanding, the following is described in conjunction with an employee information table shown in Table 1.

[0096] Table 1: Employee information table

[0097] The employee information table shown in Table 1 comprises four pieces of KV data, each piece of KV data represents information of an employee. Each piece of KV data comprises a user ID, a name, an age, a gender, a department to which the employee belongs, and a position of the employee. The key in the KV data is composed of the user ID and the name of the employee, and the value in the KV data comprises the age, the gender, the department to which the employee belongs, and the position of the employee.

[0098] It should be understood that the employee information table shown in Table 1 is only an example of the target data table, and in actual applications, the target data table can also be other types of data tables. In addition, the key in the KV data shown in Table 1 can include more or fewer values of fields, for example, can only include the user ID of the employee, or can include the user ID, the name, and the department to which the employee belongs, and the specific determination can be made according to the management requirements of the enterprise, which is not limited in the present application.

[0099] Specifically, the infrastructure 300 can store the target data table by the following steps:

[0100] S2021, the routing node 310 receives the write data request sent by the cloud management platform 200.

[0101] S2022, the routing node 310 calculates the hash value of the key of the KV data in the target data table according to the write data request.

[0102] Specifically, the routing node 310 obtains the key of the KV data in the target data table according to the write data request, and then performs a hash operation on the key of the KV data using a hash algorithm to obtain the hash value of the key of the KV data. Here, the hash algorithm used can be a secure hash algorithm (SHA) algorithm, for example, can be any one of a message digest algorithm (MD5), SHA-1, SHA-224, SHA-256, SHA-384, SHA-512, MurmurHash, and the like, which is not limited in the present application.

[0103] S2023, the routing node 310 obtains the partition key of the KV data in the target data table according to the hash value of the key of the KV data in the target data table.

[0104] The partition key of the KV data in the target data table includes the key of the KV data and the hash value of the key of the KV data. Optionally, the partition key of the KV data in the target data table further includes a checksum, and the checksum is used to verify the integrity of the key of the KV data and the hash value of the key of the KV data to determine whether the key of the KV data and the hash value of the key of the KV data have errors in the storage or transmission process.

[0105] As a possible implementation manner, in the partition key of the KV data, the hash value of the key of the KV data can be a prefix of the key of the KV data.

[0106] S2024, the routing node 310 stores the KV data in the target data table to the plurality of storage nodes 330 according to the partition key of the KV data in the target data table and through the index node 320.

[0107] Specifically, the routing node 310 determines the corresponding index node 320 according to the matching relationship between the hash value of the key of the KV data in the target data table and the hash value range in the partition rule, wherein the determined index node 320 is the index node 320 indicated by the identifier corresponding to the matching hash value range. Then, the routing node 310 sends the KV data and the hash value of the key of the KV data to the determined index node 320. Then, the index node 320 encodes the received plurality of KV data according to the dictionary order to obtain one or more pages, wherein one page includes one or more KV data and the hash value of the key of the KV data. Then, the index node 320 selects one or more storage nodes 330 according to the preset data sharding rule, and stores the page obtained by the encoding to the selected storage node 330. After receiving the page sent by the index node 320, the storage node 330 stores the page persistently.

[0108] Taking a hash value (hereinafter referred to as "hash value P") of a key of a piece of KV data in the target data table and a hash value range (hereinafter referred to as "hash value range Q") in the partition rule as an example, the routing node 310 can determine the matching relationship between the two in the following manner: the routing node 310 determines the start value Q1 and the end value Q2 of the hash value range Q, and the interval type of the hash value range Q. When any of the following conditions is met, it means that the hash value P belongs to the hash value range Q, that is, the hash value P matches the hash value range Q. Conversely, when all of the following conditions are not met, it means that the hash value P does not belong to the hash value range Q, that is, the hash value P does not match the hash value range Q. The above conditions include: the interval type of the hash value range Q is an open interval (that is, the hash value range Q is (Q1, Q2)), the start value Q1 < the hash value P < the end value Q2; the interval type of the hash value range Q is a closed interval (that is, the hash value range Q is [Q1, Q2]), the start value Q1 ≤ the hash value P ≤ the end value Q2; the interval type of the hash value range Q is left open right closed (that is, the hash value range Q is (Q1, Q2]), the start value Q1 < the hash value P ≤ the end value Q2; the interval type of the hash value range Q is left closed right open (that is, the hash value range Q is [Q1, Q2)), the start value Q1 ≤ the hash value P < the end value Q2.

[0109] In some embodiments, the routing node 310 can configure the partition rule in the following manner: the routing node 310 configures the partition rule according to the hash algorithm used in the above S2022 and the number of index nodes 320 available in the infrastructure 300 (that is, M). More specifically:

[0110] Since the value range of the hash value calculated using the hash algorithm is fixed, the routing node 310 can determine the value range of the hash value of the key of all KV data in the target data table according to the hash algorithm used in the above S2022. Then, the routing node 310 evenly divides the above determined value range into M small ranges according to the number of index nodes 320 available in the infrastructure 300, thereby obtaining the hash value range corresponding to each index node 320, wherein the hash value range corresponding to an index node 320 is one of the small ranges obtained by the above division.

[0111] Through the above S2021 to S2024, the KV data in the target data table can be evenly stored to the plurality of storage nodes 330, not only avoiding the problem of uneven partition hotspots in the lexicographic scenario, but also enabling the infrastructure 300 to perform multiple data operation requests on the data table in parallel, thereby improving the data processing efficiency.

[0112] S203, the cloud management platform 200 sends the hash value range of the key of the KV data in the target data table to the client 100. Correspondingly, the client 100 receives the hash value range of the key of the KV data in the target data table.

[0113] It should be understood that through the present step, the user can know the total range of the hash value of the key of the KV data in the target data table. Then, when the user wants to use multiple clients 100 to access the target data table in parallel, the user can configure the data range that each client 100 needs to scan based on the total range known above, so as to realize the parallel scanning of the target data table by multiple clients 100 without interference.

[0114] The above describes the data operation method provided by the present application in detail in combination with FIGS. 1 to 5. The structure of the cloud management platform 200 in the above method is described below in combination with FIG. 5.

[0115] Referring to FIG. 6, FIG. 6 shows a structural schematic diagram of a cloud management platform 200. It should be understood that FIG. 6 only exemplarily shows one division manner of the structure of the cloud management platform 200, and in actual application, the structure of the cloud management platform 200 can also have other division manners, which are not limited by the present application. As shown in FIG. 6, the cloud management platform 200 includes a receiving module 201, a sending module 202 and a processing module 203.

[0116] The receiving module 201 is configured to perform the receiving of the first data scanning request and the second data scanning request in S101; and the receiving of the first data operation request and the second data operation request in S103.

[0117] The sending module 202 is configured to perform the returning of the first KV data and the hash value of the key thereof to the client 100A and the returning of the second KV data and the hash value of the key thereof to the client 100B in S102; and the returning of the data operation result of the first KV data to the client 100A and the returning of the data operation result of the second KV data to the client 100B in S104.

[0118] The processing module 203 is configured to perform the notifying of the infrastructure 300 to acquire the first KV data and the hash value of the key thereof from the target data table according to the first data scanning request and to acquire the second KV data and the hash value of the key thereof from the target data table according to the second data scanning request in S102; and the notifying of the infrastructure 300 to perform the data operation on the first KV data according to the first data operation request and to perform the data operation on the second KV data according to the second data operation request in S104.

[0119] In a particular implementation, the receiving module 201, the sending module 202 and the processing module 203 can be implemented by software or by hardware. By way of example, the implementation of the receiving module 201 is described below. Similarly, the implementation of the sending module 202 and the processing module 203 can be implemented by referring to the implementation of the receiving module 201.

[0120] As an example of a software functional unit, the receiving module 201 can include code running on a computing instance. The computing instance can include at least one of a physical host (computing device), a virtual machine, a container. Further, the computing instance can be one or more. For example, the receiving module 201 can include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers for running the code can be distributed in the same region or in different regions. Further, the multiple hosts / virtual machines / containers for running the code can be distributed in the same AZ or in different AZs, each AZ including one data center or multiple data centers in close geographical proximity. Generally, one region can include multiple AZs.

[0121] Similarly, the multiple hosts / virtual machines / containers for running the code can be distributed in the same VPC or in multiple VPCs. Generally, one VPC is set in one region, and a communication gateway needs to be set in each VPC for cross-region communication between two VPCs in the same region or between VPCs in different regions, and the interconnection between VPCs is realized through the communication gateway.

[0122] As an example of a hardware functional unit, the receiving module 201 can include at least one computing device, such as a server, etc. Alternatively, the receiving module 201 can also be a device implemented by an application-specific integrated circuit (ASIC) or a programmable logic device (PLD), etc. The PLD can be implemented by a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0123] The plurality of computing devices included in the receiving module 201 can be distributed in the same region or in different regions. The plurality of computing devices included in the receiving module 201 can be distributed in the same AZ or in different AZs. Similarly, the plurality of computing devices included in the receiving module 201 can be distributed in the same VPC or in multiple VPCs. The plurality of computing devices can be any combination of servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0124] It should be noted that, in other embodiments, the receiving module 201 can be configured to perform any of the steps of the data operation method described above, the sending module 202 can be configured to perform any of the steps of the data operation method described above, and the processing module 203 can be configured to perform any of the steps of the data operation method described above. The steps implemented by the receiving module 201, the sending module 202, and the processing module 203 can be specified as needed, and the cloud management platform 200 can implement all functions by implementing different steps of the data operation method described above through the receiving module 201, the sending module 202, and the processing module 203.

[0125] The present application also provides a computing device. The computing device can be a server or a terminal device. FIG. 7 exemplarily shows a structural schematic diagram of the computing device provided by the present application. As shown in FIG. 7, the computing device 400 includes a bus 401, a processor 402, a memory 403, and a communication interface 404, wherein the processor 402, the memory 403, and the communication interface 404 communicate through the bus 401. It should be understood that the number of the processor 402 and the memory 403 in the computing device 400 is not limited by the present application.

[0126] The bus 401 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the convenience of representation, only one line is shown in FIG. 7, but this does not mean that the computing device 400 has only one bus or only one type of bus. The bus 401 can include a path for transmitting information between the components (e.g., the processor 402, the memory 403, and the communication interface 404) of the computing device 400.

[0127] The processor 402 can include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP), among other processors.

[0128] The memory 403 can include volatile memory, such as random access memory (RAM), and non-volatile memory, such as read-only memory (ROM), a floppy disk drive, a hard disk drive, or a solid state drive.

[0129] The memory 403 stores executable code. The processor 402 executes the code stored in the memory 403 to implement the functions of the receiving module 201, the sending module 202, and the processing module 203, respectively, to implement the data operation method described above. That is, the memory 403 stores instructions for executing the data operation method described above.

[0130] The communication interface 404 uses a transceiver module, such as but not limited to a network interface card or a transceiver, to enable communication between the computing device 400 and other devices or communication networks, for example, the computing device 400 communicates with the client 100 and the infrastructure 300 through the communication interface 404.

[0131] It should be understood that the computing device 400 provided according to the present application can correspond to the cloud management platform 200 in the present application, and the functions of the various components in the computing device 400 are respectively for implementing the corresponding processes of the various steps in the method shown in FIG. 2, and for the sake of brevity, will not be repeated here.

[0132] The present application also provides a computing device cluster. The computing device cluster includes at least one computing device, and the computing devices in the computing device cluster can be servers, such as central servers, edge servers, or local servers in local data centers. In some embodiments, the computing devices in the computing device cluster can also be terminal devices such as desktop computers, notebook computers, or smartphones.

[0133] FIG. 8 illustrates a structural diagram of a computing device cluster according to the present application. As shown in FIG. 8, the computing device cluster 500 includes at least one computing device 400. The memory 403 of one or more computing devices 400 in the computing device cluster 500 can store the same instructions for performing the data operation method described above.

[0134] In some possible implementations, the memory 403 of one or more computing devices 400 in the computing device cluster 500 can also respectively store partial instructions for performing the data operation method described above. In other words, the combination of one or more computing devices 400 can collectively execute the instructions of the data operation method described above.

[0135] It should be noted that the memory 403 in different computing devices 400 in the computing device cluster 500 can store different instructions for performing part of the functions of the cloud management platform 200. That is, the instructions stored in the memory 403 in different computing devices 400 can implement the functions of one or more of the receiving module 201, the sending module 202, and the processing module 203.

[0136] In some possible implementations, one or more computing devices 400 in the computing device cluster 500 can be connected through a network, which can be a wide area network or a local area network, etc. FIG. 9 illustrates one possible implementation. As shown in FIG. 9, the computing device 400A and the computing device 400B are connected through a network. Specifically, the computing device 400A and the computing device 400B are connected to the network through the communication interfaces therein. In this implementation, the memory 403 in the computing device 400A stores instructions for implementing the functions of the receiving module 201 and the sending module 202, and the memory 403 in the computing device 400B stores instructions for implementing the functions of the processing module 203.

[0137] The connection between the computing devices 400 in the computing device cluster 500 shown in FIG. 9 can be that the data operation method provided by the present application needs to interact with the client 100 and the infrastructure 300, so it is considered that the receiving module 201 and the sending module 202 for interacting with the client 100 are executed by the computing device 400A, and the processing module 203 for interacting with the infrastructure 300 is executed by the computing device 400B.

[0138] It should be understood that the functions of the computing device 400A shown in FIG. 9 can also be completed by multiple computing devices 400. Similarly, the functions of the computing device 400B can also be completed by multiple computing devices 400.

[0139] It should also be understood that the computing device cluster 500 provided in the present application can correspond to the cloud management platform 200 in the present application, and the functions of the various components in the computing device cluster 500 are respectively for implementing the corresponding processes of the various steps in the method shown in Fig. 2, and for the sake of brevity, will not be repeated here.

[0140] The present application also provides a computer program product containing instructions. The computer program product can be a software or program product containing instructions capable of running on a computing device or stored in any available medium. When the computer program product runs on the computing device, it causes the computing device to perform the data operation method described above.

[0141] The present application also provides a computer readable storage medium. The computer readable storage medium can be any available medium that the computing device can store or a data storage device such as a data center containing one or more available media. The above available media can be a magnetic medium (e.g. floppy disk, hard disk, magnetic tape), an optical medium (e.g. DVD), or a semiconductor medium (e.g. solid state disk) and the like. The computer readable storage medium includes instructions that instruct the computing device to perform the data operation method described above.

[0142] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements to some technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the protection scope of the technical solutions of the embodiments of the present application.

Claims

1. A method of data manipulation, characterized by, The application is applied to a cloud management platform for managing an infrastructure providing a key-value storage service, the infrastructure comprising computing nodes, the infrastructure comprising a plurality of data centers in different regions, each data center comprising a plurality of servers, the method comprising: receiving a data scan request sent by a first client, the data scan request comprising a target hash value range; returning one or more key-value (KV) data and a hash value of a key (key) of the one or more KV data to the first client according to the target hash value range; receiving a first data operation request sent by the first client, the first data operation request comprising a hash value of a key of at least one KV data and a first data operation, wherein the one or more KV data comprises the at least one KV data; informing the computing nodes to perform the first data operation on the at least one KV data according to the first data operation request.

2. The method of claim 1, wherein, The cloud management platform is also configured to receive a plurality of data scan requests in parallel, each data scan request carrying a hash value range, the plurality of data scan requests being used to indicate performing a data scan operation on different KV data in a same data table.

3. The method of claim 1, wherein, The target hash value range comprises a hash value, and the method further comprises: receiving a second data operation request sent by a second client, the second data operation request comprising a hash value of a key of one KV data and a second data operation; informing the computing nodes to perform the second data operation on the one KV data according to the hash value of the key of the one KV data.

4. The method according to any one of claims 1 to 3, characterized in that, The infrastructure further comprises a plurality of storage nodes, and the method further comprises: receiving a write data request, the write data request comprising a data table, the data table comprising the one or more KV data; informing the computing nodes to obtain a partition key of KV data in the data table according to the key of the KV data in the data table, the partition key of the KV data in the data table comprising the key of the KV data and a hash value of the key; informing the computing nodes to store the KV data in the data table to the plurality of storage nodes according to the partition key of the KV data in the data table.

5. The method of claim 4, wherein, In the partition key of the KV data in the data table, the hash value of the key of the KV data is a prefix of the key.

6. The method of claim 5, wherein, Before the receiving of the first data scan request sent by the first client, the method further comprises: providing the first client with a hash value range of the key of the KV data in the data table.

7. The method of claim 4, wherein, The hash value range of the key of the KV data in the data table is determined based on a partition rule; The first data operation request further comprises the key of the one or more KV data, and the method further comprises: informing the computing nodes to determine that the hash value of the key of the KV data carried in the first data operation request does not match the plurality of hash value ranges in the partition rule, and to calculate a new hash value according to the key of the KV data carried in the first data operation request.

8. A cloud management platform based key-value storage service apparatus, the apparatus is deployed in a cloud management platform, the cloud management platform is used to manage an infrastructure providing a key-value storage service, the infrastructure includes computing nodes, the infrastructure includes a plurality of data centers in different regions, each data center includes a plurality of servers, characterized in that, The device comprises: The receiving module is configured to receive a data scanning request sent by a first client, the data scanning request comprising a target hash value range; The sending module is configured to return the one or more pieces of KV data and a hash value of a key key of the one or more pieces of KV data to the first client according to the target hash value range; The receiving module is further configured to receive a first data operation request sent by the first client, the first data operation request comprising a hash value of a key key of at least one piece of KV data and a first data operation, wherein the one or more pieces of KV data comprise the at least one piece of KV data; The processing module is configured to instruct the computing node to perform the first data operation on the at least one piece of KV data according to the first data operation request.

9. The apparatus of claim 8, wherein The receiving module is further configured to receive a plurality of data scanning requests in parallel, each data scanning request carrying a hash value range, and the plurality of data scanning requests are used to instruct to perform a data scanning operation on different KV data in a same data table.

10. The apparatus of claim 8, wherein, The target hash value range comprises a hash value, The receiving module is further configured to receive a second data operation request sent by a second client, the second data operation request comprising a hash value of a key key of one piece of KV data and a second data operation; The processing module is further configured to instruct the computing node to perform the second data operation on the one piece of KV data according to the hash value of the key key of the one piece of KV data.

11. The apparatus of any one of claims 8 to 10, wherein, The infrastructure further comprises a plurality of storage nodes, The receiving module is further configured to receive a write data request, the write data request comprising a data table, and the data table comprising the one or more pieces of KV data; The processing module is further configured to instruct the computing node to obtain a partition key of KV data in the data table according to a key key of the KV data in the data table, and store the KV data in the data table to the plurality of storage nodes according to the partition key of the KV data in the data table, wherein the partition key of the KV data in the data table comprises the key key of the KV data and a hash value of the key key.

12. The apparatus of claim 11, wherein, In the partition key of the KV data in the data table, the hash value of the key key of the KV data is a prefix of the key key.

13. The apparatus of claim 12, wherein The sending module is further configured to provide the first client with a hash value range of the key key of the KV data in the data table.

14. The apparatus of claim 11, wherein, The hash value range of the hash value of the key key of the KV data in the data table is determined based on a partition rule, and the first data operation request further comprises the key key of the one or more pieces of KV data, The processing module is further configured to instruct the computing node to determine that the hash value of the key key of the KV data carried in the first data operation request does not match the plurality of hash value ranges in the partition rule, and to calculate a new hash value according to the key key of the KV data carried in the first data operation request.

15. A cluster of computing devices, characterized in that, The apparatus comprises at least one computing device, and each computing device comprises a processor and a memory; The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device to cause the cluster of computing devices to perform the method of any of claims 1 to 7.

16. A computer program product comprising instructions, characterized in that, The instructions, when executed by the cluster of computing devices, cause the cluster of computing devices to perform the method of any of claims 1 to 7.

17. A computer readable storage medium characterized by: Computer program instructions, when executed by the cluster of computing devices, cause the cluster of computing devices to perform the method of any of claims 1 to 7.

Images