Data sampling method based on cloud management platform and cloud management platform

Abstract

The invention discloses a data sampling method based on a cloud management platform and the cloud management platform. The user experience can be improved. The method comprises the steps that a user can send a data sampling request to a sampling interface provided by a cloud management platform, the cloud management platform can receive the data sampling request sent by the user through the sampling interface, the data sampling request can be used for indicating a first key range set by the user, and the cloud management platform can obtain a key value distribution table, a plurality of key ranges and the number of values corresponding to each key range are recorded in the key value distribution table, and the plurality of key ranges are obtained by dividing a plurality of keys in a plurality of key value pairs of the user by the user. And then, the cloud management platform can detect whether a first key range set by the user contains a second key range in a plurality of key ranges recorded in the key value distribution table, and if so, the cloud management platform can generate a data sampling result based on the number of values corresponding to the second key range, and return the data sampling result to the user.

Description

Technical Field

[0001] This application relates to the field of cloud technology, and in particular to a data sampling method based on a cloud management platform and a cloud management platform. Background Technology

[0002] With the development of cloud technology, more and more users are choosing to use distributed key-value storage systems provided by cloud vendors to store and process their data. In a distributed key-value storage system, user data serves as the value, and the information used to retrieve this data serves as the key. Therefore, the user's data, corresponding to the key, can be retrieved from the system to meet the user's data processing needs.

[0003] In related technologies, distributed key-value storage systems may include a cloud management platform and infrastructure, which can store multiple key-value pairs belonging to users. When a user needs to determine the distribution of data that meets certain conditions, the user can send a data scanning request to the cloud management platform. The cloud management platform then uses the keys contained in these key-value pairs to retrieve the corresponding values ​​from the infrastructure, determines the number of values ​​that meet the conditions, and returns the number of these values ​​to the user, thus satisfying the user's needs.

[0004] In the above process, since the user's multiple key-value pairs are stored in different locations in the infrastructure, a full scan of the infrastructure often requires a large time cost, resulting in excessively long waiting times for users and thus a poor user experience. Summary of the Invention

[0005] This application provides a data sampling method and a cloud management platform based on a cloud management platform. This method can save the scanning operation of the cloud management platform during the data sampling process, thereby reducing the waiting time required by the user and improving the user experience.

[0006] The first aspect of this application provides a data sampling method based on a cloud management platform. The cloud management platform implementing this method can manage infrastructure that provides cloud services to users. This infrastructure can store multiple key-value pairs for users, where multiple values ​​in these multiple key-value pairs are multiple data points for the user, and multiple keys in these multiple key-value pairs are used to retrieve these multiple values. The method includes:

[0007] For multiple keys in these key-value pairs, when a user needs to determine the distribution of one or more keys, the user can set these keys as the first key range and log in to the cloud management platform. The cloud management platform can provide a sampling interface to the user, so the user can send a data sampling request to the sampling interface provided by the cloud management platform to indicate the first key range.

[0008] Upon receiving a data sampling request, the cloud management platform can obtain a key-value distribution table based on the request. Since the key-value distribution table records multiple key ranges and the number of values ​​corresponding to each key range, these key ranges are obtained by the user dividing the multiple key-value pairs.

[0009] Then, the cloud management platform can detect whether the first key range includes one or more key ranges among the multiple key ranges recorded in the key value distribution table. If it does, the cloud management platform can generate data sampling results based on the number of values ​​corresponding to these key ranges and return the data sampling results to the user so that the user can understand the distribution of keys in the first key range using the data sampling results, thereby meeting the user's needs.

[0010] As can be seen from the above method, since the cloud management platform has a built-in key-value distribution table, once the first key range set by the user includes the second key range recorded in the key-value distribution table, the cloud management platform can use the number of values ​​corresponding to the second key range recorded in the key-value distribution table as the distribution of keys in the first key range, and notify the user of this distribution. In this way, the cloud management platform does not need to scan the infrastructure to determine the distribution of keys in the first key range, saving the cloud management platform scanning operations during the key distribution determination process, thereby reducing the waiting time for the user and improving the user experience.

[0011] In one possible implementation, the infrastructure comprises multiple storage nodes, each storing at least one key-value pair from a set of key-value pairs. A key-value distribution table records each storage node, the range of keys within each storage node, and the number of values ​​corresponding to those keys. In this implementation, for any one of the storage nodes storing multiple key-value pairs, the key-value distribution table records information such as the storage node itself, the range of keys it possesses, and the number of values ​​corresponding to those keys. For the other storage nodes besides the first one, the key-value distribution table also records relevant information about them, which will not be elaborated further here. Therefore, the distribution of keys across multiple storage nodes can be determined based on the key-value distribution table.

[0012] In one possible implementation, the data sampling request is further used to indicate a first storage node among multiple storage nodes; determining that the first key range includes a second key range among multiple key ranges recorded in the key-value distribution table, and the cloud management platform generating a data sampling result based on the number of values ​​corresponding to the second key range includes: determining that the first key range includes a second key range in the first storage node recorded in the key-value distribution table, and the cloud management platform generating a data sampling result based on the number of values ​​corresponding to the second key range. In the aforementioned implementation, when the data sampling request is also used to indicate a first storage node, it indicates that the user needs to determine the distribution of keys in the first key range within the sampling range of the first storage node. Based on this, the cloud management platform can check in the key-value distribution table whether the first key range includes the second key range possessed by the first storage node. If it does, the cloud management platform determines the number of values ​​corresponding to these second key ranges in the key-value distribution table and uses the number of values ​​corresponding to these second key ranges to generate a data sampling result, which is then returned to the user. In this way, the user can accurately understand the distribution of keys in the first key range based on the data sampling result.

[0013] In one possible implementation, the second key range has only one value. The cloud management platform generates a data sampling result based on the number of values ​​corresponding to the second key range, including using the number of values ​​corresponding to the second key range as the data sampling result. In the aforementioned implementation, when the second key range has only one value, the cloud management platform can directly use the number of values ​​corresponding to the second key range as the data sampling result. In this way, the cloud management platform can accurately obtain the data sampling result and, based on this result, accurately notify the user of the distribution of keys in the first key range.

[0014] In one possible implementation, there are multiple second key ranges. The cloud management platform generates data sampling results based on the number of values ​​corresponding to the multiple second key ranges, including summing the number of values ​​corresponding to the multiple second key ranges as the data sampling result. In the aforementioned implementation, when there are multiple second key ranges, the cloud management platform can add the number of values ​​corresponding to the multiple second key ranges to obtain the sum of the number of values ​​corresponding to these multiple second key ranges, and use this sum as the data sampling result. In this way, the cloud management platform can accurately obtain the data sampling result, and based on this result, accurately notify the user of the distribution of keys in the first key range.

[0015] In one possible implementation, the method further includes: a cloud management platform receiving a user's request to create a key-value distribution table, the request indicating multiple key ranges set by the user for the table; the cloud management platform determining the number of values ​​corresponding to each key range based on the multiple key-value pairs; and the cloud management platform generating the key-value distribution table based on the multiple key ranges and the number of values ​​corresponding to each range. In the aforementioned implementation, to determine the distribution of multiple keys across various storage nodes, the user can divide the multiple keys to obtain the multiple key ranges set by the user for the key-value distribution table. Then, the user can input a creation request for the key-value distribution table into the creation interface, the request being for these multiple key ranges. Then, for any one of these storage nodes, the cloud management platform first determines the key-value pairs stored by that node. Based on the keys in these key-value pairs, it then determines which key ranges from the multiple key ranges set by the user the storage node possesses, and records these key ranges in the key-value distribution table. Since the keys of the key-value pairs stored by the storage node fall within these key ranges, the cloud management platform can further determine the number of values ​​corresponding to each key range and record this number in the key-value distribution table. The cloud management platform can perform similar operations for the other storage nodes. In this way, the cloud management platform can successfully create a key-value distribution table, which records the distribution of keys across multiple storage nodes, allowing users to quickly understand the key distribution.

[0016] In one possible implementation, the method further includes: the cloud management platform migrating key-value pairs from the second storage node to the third storage node, wherein the third key range in the second storage node contains the keys in the key-value pairs in the key-value distribution table; the cloud management platform updating the third key range, the number of values ​​corresponding to the third key range, the fourth key range of the third storage node, and the number of values ​​corresponding to the fourth key range based on the key-value pairs, to obtain an updated key-value distribution table, wherein the updated third key range does not contain keys, and the updated fourth key range contains keys. In the aforementioned implementation, after migrating a key-value pair from the second storage node to the third storage node, the cloud management platform can update the third key range, the number of values ​​corresponding to the third key range, the fourth key range of the third storage node, and the number of values ​​corresponding to the fourth key range in the key-value distribution table based on the key-value pair, thereby obtaining an updated key-value distribution table. In the updated key-value distribution table, the updated third key range does not contain the key, while the updated fourth key range does. The number of values ​​corresponding to the updated third key range is less than the number of values ​​corresponding to the original third key range, and the number of values ​​corresponding to the updated fourth key range is greater than the number of values ​​corresponding to the original fourth key range. This demonstrates that the cloud management platform can update the key-value distribution table in real time when storage nodes send key-value pair migrations, allowing users to understand the latest key distribution.

[0017] In one possible implementation, the second and third storage nodes are two of a plurality of storage nodes, or the second storage node is one of a plurality of storage nodes and the third storage node is a newly created storage node.

[0018] A second aspect of this application provides a cloud management platform for managing infrastructure providing cloud services. The infrastructure stores multiple key-value pairs, where multiple values ​​in the multiple key-value pairs are multiple data points of a user. Multiple keys in the multiple key-value pairs are used to retrieve multiple values. The cloud management platform includes: a first receiving module for receiving a data sampling request sent by a user, the data sampling request indicating a first key range, the first key range containing at least one key among the multiple keys; an obtaining module for obtaining a key-value distribution table based on the data sampling request, the key-value distribution table recording multiple key ranges and the number of values ​​corresponding to each key range, each key range containing at least one key among the multiple keys; and a providing module for determining that the first key range contains a second key range among the multiple key ranges recorded in the key-value distribution table, generating a data sampling result based on the number of values ​​corresponding to the second key range, and sending the data sampling result to the user.

[0019] In one possible implementation, the infrastructure comprises multiple storage nodes, each storing at least one key-value pair from a plurality of key-value pairs, and a key-value distribution table is used to record each storage node, the range of keys in each storage node, and the number of values ​​corresponding to the key range.

[0020] In one possible implementation, the data sampling request is also used to indicate a first storage node among multiple storage nodes; and a providing module is used to determine a second key range in the first storage node whose first key range contains key value distribution table records, and to generate a data sampling result based on the number of values ​​corresponding to the second key range.

[0021] In one possible implementation, the number of second key ranges is one, and a module is provided to take the number of values ​​corresponding to the second key range as the data sampling result.

[0022] In one possible implementation, there are multiple second key ranges, and a module is provided to take the sum of the number of values ​​corresponding to the multiple second key ranges as the data sampling result.

[0023] In one possible implementation, the cloud management platform further includes: a second receiving module for receiving a creation request for a key-value distribution table sent by a user, the creation request indicating multiple key ranges set by the user for the key-value distribution table; a determining module for determining the number of values ​​corresponding to each key range based on multiple key-value pairs; and a generating module for generating a key-value distribution table based on the multiple key ranges and the number of values ​​corresponding to each key range.

[0024] In one possible implementation, the cloud management platform further includes: a migration module for migrating key-value pairs from a second storage node to a third storage node, wherein the third key range in the second storage node contains the keys in the key-value pairs in the key-value distribution table; and an update module for updating the third key range, the number of values ​​corresponding to the third key range, the fourth key range of the third storage node, and the number of values ​​corresponding to the fourth key range based on the key-value pairs, to obtain an updated key-value distribution table, wherein the updated third key range does not contain keys, and the updated fourth key range contains keys.

[0025] In one possible implementation, the second and third storage nodes are two of a plurality of storage nodes, or the second storage node is one of a plurality of storage nodes and the third storage node is a newly created storage node.

[0026] A third aspect of this application provides a computing device cluster, which includes at least one computing device, each computing device including a processor and a memory: the memory is used to store instructions; the processor is used to cause the computing device cluster to perform the method described in the first aspect or any possible implementation of the first aspect according to the instructions.

[0027] A fourth aspect of this application provides a computer storage medium storing one or more instructions that, when executed by one or more computers, cause the one or more computers to perform the method described in the first aspect or any possible implementation of the first aspect.

[0028] A fifth aspect of this application provides a computer program product storing instructions that, when executed by a computer, cause the computer to perform the method described in the first aspect or any possible implementation of the first aspect.

[0029] In this embodiment, the cloud management platform can treat multiple user data as multiple values ​​in multiple key-value pairs and store these key-value pairs in the infrastructure managed by the cloud management platform. For the multiple keys in these key-value pairs, when the user needs to determine the distribution of certain keys, the user can set these keys as a first key range and send a data sampling request to the sampling interface provided by the cloud management platform. Therefore, the cloud management platform can receive the data sampling request sent by the user through the sampling interface. This data sampling request can be used to indicate the first key range set by the user. Thus, the cloud management platform can obtain a key-value distribution table, which records multiple key ranges and the number of values ​​corresponding to each key range. These multiple key ranges are obtained by the user dividing the multiple keys in these key-value pairs. Then, the cloud management platform can detect whether the first key range set by the user includes a second key range among the multiple key ranges recorded in the key-value distribution table. If it does, the cloud management platform can generate a data sampling result based on the number of values ​​corresponding to the second key range and return the data sampling result to the user to meet the user's needs. In the aforementioned process, because the cloud management platform has a built-in key-value distribution table, once the first key range set by the user includes a second key range recorded in the key-value distribution table, the cloud management platform can use the number of values ​​corresponding to the second key range recorded in the key-value distribution table as the distribution of keys in the first key range, and notify the user of this distribution. In this way, the cloud management platform does not need to scan the infrastructure to determine the distribution of keys in the first key range, saving the cloud management platform scanning operations during the key distribution determination process, thereby reducing the waiting time for the user and improving the user experience. Attached Figure Description

[0030] Figure 1A schematic diagram of the structure of a distributed key-value storage system provided in an embodiment of this application;

[0031] Figure 2 A schematic diagram of the structure of a distributed key-value storage system provided in an embodiment of this application;

[0032] Figure 3 A schematic flowchart of a data sampling method based on a cloud management platform provided in this application embodiment;

[0033] Figure 4 A schematic diagram of a user interface provided for an embodiment of this application;

[0034] Figure 5a This is another schematic diagram of the distributed key-value storage system provided in the embodiments of this application;

[0035] Figure 5b This is another schematic diagram of the distributed key-value storage system provided in the embodiments of this application;

[0036] Figure 6 Another schematic diagram of the user interface provided in the embodiments of this application;

[0037] Figure 7 A schematic diagram of the structure of the cloud management platform provided in the embodiments of this application;

[0038] Figure 8 A schematic diagram of the structure of a computing device provided in an embodiment of this application;

[0039] Figure 9 A schematic diagram of the structure of a computing device cluster provided in an embodiment of this application;

[0040] Figure 10 This is a schematic diagram illustrating the network connection of computer devices in a computer cluster provided in an embodiment of this application. Detailed Implementation

[0041] This application provides a data sampling method and a cloud management platform based on a cloud management platform. This method can save the scanning operation of the cloud management platform during the data sampling process, thereby reducing the waiting time required by the user and improving the user experience.

[0042] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms are interchangeable where appropriate; this is merely a way of distinguishing objects with the same attributes in the embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of elements is not necessarily limited to those elements, but may include other elements not explicitly listed or inherent to those processes, methods, products, or apparatuses.

[0043] With the development of cloud technology, more and more users are choosing to use distributed key-value storage systems provided by cloud vendors to store and process their data. In a distributed key-value storage system, user data is stored in the form of key-value pairs. The user's data serves as the value in the key-value pair, and the information used to retrieve this data serves as the key. Therefore, the user's data can be retrieved from the system using the key, thus meeting the user's data processing needs.

[0044] In related technologies, distributed key-value storage systems may include a cloud management platform and infrastructure. This infrastructure may contain multiple physical partitions, where a user's multiple key-value pairs can be stored separately. When a user needs to determine the distribution of data that meets certain conditions across these physical partitions, the user can send a data scan request to the cloud management platform. The cloud management platform then uses the keys contained in the multiple key-value pairs to retrieve the values ​​contained in the multiple key-value pairs from each physical partition, identifies several values ​​that meet the condition, and returns the number of these values ​​to the user, thus fulfilling the user's needs.

[0045] In the above process, since the user's multiple key-value pairs are stored in different physical partitions in the infrastructure, some physical partitions may store more values ​​that meet the condition, while other physical partitions may store fewer values ​​that meet the condition. Therefore, when scanning all physical partitions, it is often necessary to wait until the last physical partition is scanned before the scanning can stop. This will consume a lot of time, resulting in excessively long waiting time for users and thus a poor user experience.

[0046] To address the aforementioned issues, this application provides a data sampling method based on a cloud management platform. This method can be implemented using a distributed key-value storage system (also known as a cloud service system). Figure 1 A schematic diagram of the structure of the distributed key-value storage system provided in the embodiments of this application is shown below. Figure 1As shown, a distributed key-value storage system includes the infrastructure that provides cloud services to users and a cloud management platform that manages this infrastructure. The cloud management platform and the infrastructure are described separately below:

[0047] A cloud management platform can centrally manage the infrastructure of the entire distributed key-value storage system. (For example, within the infrastructure, according to a user's instructions, multiple pieces of a user's data are stored as key-value pairs, where multiple values ​​in these key-value pairs represent multiple pieces of the user's data, and multiple keys in these key-value pairs are used to retrieve these values.) The cloud management platform can also be accessible to users outside the system and respond to their requests. For example, the cloud management platform can provide various interfaces such as login, creation, and sampling interfaces for user clients (e.g., the user's terminal device or the browser on that device) to access. Specifically, the cloud management platform can authenticate a user's client through the login interface, allowing the client to log in to the cloud management platform upon successful authentication. For example, the cloud management platform can also create an interface that allows a user's client to send a request to the cloud management platform to create a key-value distribution table. Since this request instructs the user to divide their multiple keys into multiple key ranges (also called key groups or key sets, etc., each key range may contain one or more keys), the cloud management platform can determine the number of values ​​corresponding to each key range based on these multiple key-value pairs. In this way, the cloud management platform can use these multiple key ranges and the number of values ​​corresponding to each key range to generate a key-value distribution table. Similarly, the cloud management platform can also use a sampling interface that allows a user's client to send a data sampling request to the cloud management platform. Since this request instructs the user to set a certain (or several) key ranges, the cloud management platform can check whether the key range contains a certain (or several) key ranges recorded in the key-value distribution table. If it does, since the key-value distribution table also records the number of values ​​corresponding to these key ranges, the cloud management platform can use the number of values ​​corresponding to these key ranges to generate data sampling results and return the data sampling results to the user, and so on.

[0048] The infrastructure comprises multiple storage nodes, which are typically represented as multiple physical servers (or other physical devices such as bare-metal servers). Since each physical server possesses certain physical storage resources, these physical servers can be considered as multiple physical partitions facing the cloud management platform, and each physical partition usually has a globally unique identifier (partition ID). Thus, once a user requests the cloud management platform to store multiple pieces of their data, the cloud management platform can store this data as multiple key-value pairs in these multiple physical partitions, i.e., in these multiple storage nodes.

[0049] It should be noted that the cloud management platform can generate multiple key-value pairs in several ways: (1) The cloud management platform can construct multiple key-value pairs using only a primary index. That is, all these key-value pairs are primary indexes. For example, suppose a user provides data 1 to data n to the cloud management platform. Data 1 contains identity document (ID) 1, name 1, age 1, and score 1. Data 2 contains ID 2, name 2, age 2, and score 2, ..., and data n contains ID n, name n, age n, and score n. Then, the cloud management platform can generate key-value pairs 1 to 2 key-value pairs n. The key of key-value pair 1 is ID 1, and the value of key-value pair 1 is data 1. The key of key-value pair 2 is ID 2, and the value of key-value pair 2 is data 2, ..., and the key of key-value pair n is ID n, and the value of key-value pair n is data n. (2) The cloud management platform can also construct multiple key-value pairs using a combination of primary and secondary indexes. That is, these key-value pairs can be divided into two parts: one part forms the primary index, and the other part forms the secondary index. Continuing with the example above, suppose the user provides data 1 to data n to the cloud management platform. The primary index generated by the cloud management platform can contain key-value pairs 1 to n, and the secondary index generated by the cloud management platform can contain key-value pairs n+1 to 2n. Specifically, key-value pair 1 has the key ID1 and the value is data 1; key-value pair 2 has the key ID2 and the value is data 2; ..., key-value pair n has the key IDn and the value is data n. Furthermore, key-value pair n+1 has the key score 1 and the value is data 1; key-value pair n+2 has the key score 2 and the value is data 2; ..., key-value pair 2n has the key score n and the value is data n.

[0050] It should also be noted that since the user's multiple key-value pairs are stored on multiple storage nodes by the cloud management platform, the user can provide the cloud management platform with multiple key ranges set by the user to determine the distribution of these keys across the storage nodes. For any one of these storage nodes, the cloud management platform can first determine the key-value pairs stored on that node, then determine which key ranges from the user-set key ranges the storage node possesses based on the keys in these key-value pairs, and record these key ranges in the key-value distribution table. Since the keys of the key-value pairs stored on the storage node fall within these key ranges, the cloud management platform can further determine the number of values ​​corresponding to each key range of the storage node and record this number in the key-value distribution table. Therefore, the key-value distribution table can describe the distribution of keys across the storage nodes.

[0051] For example, a user provided six data points to the cloud management platform: Data 1 includes Xiao Chen, age 20, and score 98; Data 2 includes Xiao Zhang, age 19, and score 92; Data 3 includes Xiao Liu, age 20, and score 93; Data 4 includes Xiao Lin, age 19, and score 91; Data 5 includes Xiao Sun, age 19, and score 98; and Data 6 includes Xiao Wu, age 19, and score 97.

[0052] Then, the cloud management platform can generate key-value pairs 1 to 12, where the key of key-value pair 1 is 20 years old and the value is data 1; the key of key-value pair 2 is 19 years old and the value is data 2; the key of key-value pair 3 is 20 years old and the value is data 3; the key of key-value pair 4 is 19 years old and the value is data 4; the key of key-value pair 5 is 19 years old and the value is data 5; and the key of key-value pair 6 is 19 years old and the value is data 12. The value of key-value pair 6 is data 6. The key of key-value pair 7 is 98 points and its value is data 1. The key of key-value pair 8 is 94 points and its value is data 2. The key of key-value pair 9 is 93 points and its value is data 3. The key of key-value pair 10 is 93 points and its value is data 4. The key of key-value pair 11 is 98 points and its value is data 5. The key of key-value pair 12 is 97 points and its value is data 6. Furthermore, the cloud management platform can store key-value pairs 1 to 3 and key-value pairs 7 to 9 in storage node 1, and key-value pairs 4 to 6 and key-value pairs 10 to 12 in storage node 2.

[0053] To determine the distribution of keys in their key-value pairs, users can set multiple key ranges to the cloud management platform: key range 1 is 19 years old, key range 2 is 20 years old, key range 3 is 90 to 95 points, and key range 4 is 96 to 100 points. Based on this, the cloud management platform can generate a key-value distribution table, as shown in Table 1.

[0054] Table 1

[0055]

[0056] Furthermore, such as Figure 2 As shown ( Figure 2(This is a schematic diagram of a distributed key-value storage system provided in an embodiment of this application). The cloud management platform may include modules such as an access module, a management module, and a sampling module. The access module provides various interfaces for users, such as the aforementioned login interface, creation interface, and sampling interface. The management module has a built-in partition routing table and a key-value distribution table. The partition routing table records the location of the storage nodes where key-value pairs reside (i.e., which key-value pairs are located in which storage nodes), and the key-value distribution table records multiple storage nodes, the key ranges of each storage node, and the number of values ​​corresponding to the key ranges of each storage node. The sampling module can be deployed remotely at the infrastructure. According to user requirements (e.g., multiple key ranges set by the user), the sampling module can periodically collect relevant information of key-value pairs in multiple storage nodes and update the contents of the partition routing table and the key-value distribution table in a timely manner. For example, during the aforementioned generation of the key-value distribution table, the sampling module can collect the distribution information such as the key ranges of each storage node and the number of values ​​corresponding to these key ranges, and write it into the key-value distribution table. For example, if key-value pairs migrate between storage nodes, the sampling module can collect the latest key range of each storage node and the latest number of values ​​corresponding to these key ranges, and update the key-value distribution table to obtain the updated key-value distribution table, etc., which will not be elaborated here.

[0057] Furthermore, for the access module, management module, and sampling module, these modules can be one or more cloud instances. These cloud instances can be presented in various forms. For example, they can be physical servers, bare metal servers, or virtual machines (VMs) created on cloud resources using virtualization technology. They can also be containers (Docker) created on cloud resources using virtualization technology, or microVMs created on cloud resources using virtualization technology, and so on.

[0058] Furthermore, for multiple storage nodes in the infrastructure, these multiple storage nodes can be set up at the same site or different sites. These sites can be presented in various forms. For example, these sites can be regions, availability zones, data centers (DCs), server rooms, physical server groups (also known as racks), and so on.

[0059] Based on the aforementioned distributed key-value storage system, the cloud management platform can treat multiple user data points as multiple values ​​in multiple key-value pairs and store these key-value pairs within the infrastructure managed by the cloud management platform. When a user needs to determine the distribution of certain keys within these key-value pairs, the user can set these keys as a target key range and send a data sampling request to the sampling interface provided by the cloud management platform. The cloud management platform can then receive this data sampling request from the user through the sampling interface. This request can be used to indicate the target key range set by the user, allowing the cloud management platform to obtain a key-value distribution table. This table records multiple key ranges and the number of values ​​corresponding to each key range. These key ranges are obtained by the user dividing the multiple keys in the key-value pairs. The cloud management platform can then detect whether the target key range set by the user includes one or more key ranges recorded in the key-value distribution table. If so, the cloud management platform can generate data sampling results based on the number of values ​​corresponding to these key ranges and return the data sampling results to the user to meet the user's needs. In the aforementioned process, because the cloud management platform has a built-in key-value distribution table, once the target key range set by the user includes one or more key ranges recorded in the key-value distribution table, the cloud management platform can use the number of values ​​corresponding to these key ranges recorded in the key-value distribution table as the distribution of keys within the target key range, and notify the user of this distribution. In this way, the cloud management platform does not need to scan the infrastructure to determine the distribution of keys within the target key range, saving the cloud management platform scanning operations during the key distribution determination process, thereby reducing the waiting time for the user and improving the user experience. To further understand the working process of the aforementioned cloud management platform, the following section combines... Figure 3 To further explain this process, Figure 3 A flowchart illustrating the data sampling method based on a cloud management platform provided in this application embodiment is shown below. Figure 3 As shown, this method can be achieved through, as Figure 1 or Figure 2 The illustrated distributed key-value store system implementation may include infrastructure providing cloud services to users and a cloud management platform managing this infrastructure. This infrastructure stores multiple key-value pairs, where multiple values ​​in these key-value pairs represent multiple pieces of user data. Multiple keys in these key-value pairs are available for the cloud management platform to retrieve these multiple values ​​from the infrastructure. The method includes:

[0060] 301. The cloud management platform receives a data sampling request sent by the user. The data sampling request is used to indicate a first key range, which includes at least one of multiple keys.

[0061] In this embodiment, according to the user's data storage needs, the cloud management platform can pre-define multiple pieces of the user's data as multiple values ​​in multiple key-value pairs and store these key-value pairs in the infrastructure managed by the cloud management platform. For the multiple keys in these key-value pairs, when the user needs to determine the distribution of one or more keys, the user can set these keys as a first key range and log in to the cloud management platform through their client. The cloud management platform can provide a sampling interface to the user's client, so the user can use their client to send a data sampling request indicating the first key range to the sampling interface provided by the cloud management platform (e.g., the data sampling bar in the user interface, etc.). Therefore, the cloud management platform can receive the data sampling request sent by the user's client through the sampling interface.

[0062] Specifically, for multiple keys contained in multiple key-value pairs, these multiple keys can be presented in a variety of ways: (1) These multiple keys can all be keys of the same type, that is, these multiple keys are multiple keys obtained by a certain type of key under different values. (2) These multiple keys can also contain keys of different types, that is, these multiple keys contain several keys obtained by a certain type of key under different values, and several keys obtained by other types of keys under different values.

[0063] Continuing with the example shown in Table 1, based on user-provided data 1 to 6, the cloud management platform generates key-value pairs 1 to 12 and stores them on storage node 1 and storage node 2 respectively. Key-value pairs 1 to 12 contain a total of 12 keys, which are of two types: age and score. For age, assigning values ​​to the "age" key six times yields the keys in key-value pairs 1 to 6: 20 years old, 19 years old, 20 years old, 19 years old, 19 years old, and 19 years old. Similarly, for score, assigning values ​​to the "score" key six times yields the keys in key-value pairs 7 to 12: 98 points, 92 points, 93 points, 91 points, 98 points, and 97 points.

[0064] More specifically, for multiple keys among these key-value pairs, when a user needs to determine the distribution of keys of a certain type (or several types) among these keys, the user can set the key of that type as the sampling key in the data sampling request, and set the number of sampling points (also known as sampling point precision, sampling point step size, etc.) for the sampling key, so that the data sampling request includes the sampling key and the number of sampling points selected by the user. Since the number of sampling points can be used to assign values ​​to the sampling key, thereby obtaining several keys with different values, these several keys constitute the target key range to be sampled, that is, the aforementioned first key range. Therefore, the data sampling request can be used to indicate the first key range.

[0065] More specifically, since a user's multiple key-value pairs are stored in multiple storage nodes, the user can use a certain storage node as the sampling range according to their own needs. For example, if the user needs to determine the distribution of keys in a certain storage node with respect to the first key range, the user can also make the data sampling request include the identifier of the storage node, that is, the identifier of the first storage node. Therefore, the data sampling request can not only be used to indicate the first key range, but also to indicate the first storage node.

[0066] As shown in Table 1, for example Figure 4 As shown ( Figure 4 (A schematic diagram of the user interface provided in this application embodiment) When a user needs to determine the distribution of "scores" in storage node 1, the user can log in to the cloud management platform. The cloud management platform can provide a user interface, which includes a data sampling bar. The user can enter the sampling key as "score", the number of sampling points as "10", and the sampling range as "storage node 1 (e.g., the identifier of storage node 1, etc.)" in the data sampling bar. Then, the data sampling bar can generate a data sampling request containing the sampling key, the number of sampling points, and the sampling range, and send the data sampling request to the cloud management platform.

[0067] Upon receiving the data sampling request, the cloud management platform can parse the request to obtain the sampling key, the number of sampling points, and the sampling range. Since the sampling key is the score and the number of sampling points is 10, the cloud management platform can determine the 10 target key ranges to be sampled (i.e., the aforementioned first key range). Target key range 1 is scores from 1 to 10, target key range 2 is scores from 11 to 20, ..., and target key range 10 is scores from 91 to 100.

[0068] 302. The cloud management platform obtains a key-value distribution table based on data sampling requests. The key-value distribution table is used to record multiple key ranges and the number of values ​​corresponding to each key range. Each key range contains at least one of the multiple keys.

[0069] 303. Determine that the first key range includes the second key range among the multiple key ranges recorded in the key value distribution table. The cloud management platform generates data sampling results based on the number of values ​​corresponding to the second key range and sends the data sampling results to the user.

[0070] Upon receiving a data sampling request, since this request indicates a first key range, the cloud management platform can determine that the user needs to know the distribution of keys within that first key range. The cloud management platform can first obtain a key-value distribution table. This table records multiple key ranges and the number of values ​​corresponding to each key range; these key ranges are obtained by the user dividing the key-value pairs into multiple key pairs. Then, the cloud management platform can detect whether the first key range includes one or more key ranges recorded in the key-value distribution table. If so, the cloud management platform can generate a data sampling result based on the number of values ​​corresponding to these key ranges and return this result to the user, allowing the user to understand the distribution of keys within the first key range and thus meeting the user's needs.

[0071] Specifically, the cloud management platform can also perform the following operations:

[0072] Since a user's multiple key-value pairs are stored across multiple storage nodes by the cloud management platform, the user can create a key-value distribution table to determine the distribution of keys across these nodes. This involves first dividing the keys into multiple key ranges for the distribution table, where each key combination can contain one or more of these keys. Next, the cloud management platform provides a creation interface to the user's client. The user can then submit a creation request for the key-value distribution table, specifying the range of keys, to this interface. The cloud management platform can then receive these creation requests from the user's client via the creation interface.

[0073] Upon receiving a creation request, the cloud management platform can determine which key-value distribution tables need to be created. For any one of the multiple storage nodes, the cloud management platform first determines the key-value pairs stored by that node, then determines which key ranges from the multiple key ranges set by the user the storage node possesses based on the keys in these key-value pairs, and records these key ranges in the key-value distribution table. Since the keys of the key-value pairs stored by the storage node fall within these key ranges, the cloud management platform can further determine the number of values ​​corresponding to each key range of the storage node and record this number in the key-value distribution table. The cloud management platform can perform similar operations for the other storage nodes. Therefore, the key-value distribution table can record multiple storage nodes, the key ranges possessed by each storage node, and the number of values ​​corresponding to each key range.

[0074] In addition, after creating the key-value distribution table, the cloud management platform can selectively return the key-value distribution table to the user for viewing and use as feedback on the creation request.

[0075] It should be noted that the creation process of the key-value distribution table is usually performed before step 301.

[0076] As in the example above, such as Figure 5a As shown ( Figure 5a (This is another schematic diagram of the distributed key-value storage system provided in this application embodiment). Based on user-provided data 1 to data 6, the cloud management platform generates key-value pairs 1 to 12, and stores key-value pairs 1 to 3 and 7 to 9 in storage node 1, and key-value pairs 4 to 6 and 10 to 12 in storage node 2. For the user, to understand the distribution of their data across the storage nodes, the user can define key ranges 1 to 4, where key range 1 is 19 years old, key range 2 is 20 years old, key range 3 is 90 to 95 points, and key range 4 is 96 to 97 points. Next, the user can log in to the cloud management platform, which provides a user interface including a key-value distribution table creation field. The user can enter key ranges 1 to 4 in this creation field. This creation field then generates a creation request for the key-value distribution table and sends it to the cloud management platform.

[0077] Upon receiving the creation request, the cloud management platform can parse it to obtain key range 1 to key range 4. Then, the cloud management platform can first obtain the partition routing table, and based on the partition routing table, retrieve key-value pairs 1 to 3 and 7 to 9 from storage node 1, and key-value pairs 4 to 6 and 10 to 12 from storage node 2.

[0078] Based on the keys contained in key-value pairs 1 to 3 and key-value pairs 7 to 9, the cloud management platform can determine that the key range of storage node 1 is key range 1 to key range 4, and the number of values ​​corresponding to these 4 key ranges (for example, if the key of key-value pair 1 and key-value pair 3 is 20 years old, then the number of values ​​corresponding to key range 2 in storage node 1 is 2; if the key of key-value pair 2 is 19 years old, then the number of values ​​corresponding to key range 1 in storage node 1 is 1, and so on).

[0079] Similarly, based on the keys contained in key-value pairs 4 to 6 and key-value pairs 10 to 12, the cloud management platform can determine that the key range of storage node 2 is key range 1, key range 3 and key range 4, and the number of values ​​corresponding to these three key ranges (for example, the keys of key-value pairs 4, 5 and 6 are all 19, so the number of values ​​corresponding to key range 1 in storage node 2 is 3, etc.).

[0080] In this way, the cloud management platform can generate a key-value distribution table for users, as shown in Table 1, which will not be described in detail here.

[0081] More specifically, the cloud management platform can also detect whether the first key range includes the second key range in several ways:

[0082] (1) When the data sampling request is also used to indicate the first storage node, it means that the user needs to determine the distribution of keys within the first key range within the sampling range of the first storage node. For ease of explanation, the key range of the first storage node recorded in the key value distribution table can be called the second key range. The cloud management platform can then check whether the first key range includes the second key range of the first storage node in the key value distribution table. If it does, the cloud management platform determines the number of values ​​corresponding to these second key ranges in the key value distribution table and uses the number of values ​​corresponding to these second key ranges to generate data sampling results, which are then returned to the user. If it does not include the second key range, the cloud management platform can scan the first storage node based on the partition routing table to obtain all the user's key-value pairs stored in the first storage node, and determine which part of the key-value pairs contains keys that fall into the first key range. The number of values ​​in this part of the key-value pairs is taken as the number of values ​​corresponding to the first key range. In this way, the cloud management platform can use the number of values ​​corresponding to the first key range to generate data sampling results, which are then returned to the user.

[0083] As in the example above, such as Figure 5b As shown ( Figure 5b (This is another schematic diagram of the distributed key-value storage system provided in this application embodiment). After receiving the data sampling request, since the data sampling request includes the sampling key as "score", the number of sampling points as "10", and the sampling range as "storage node 1", the cloud management platform can determine that it needs to determine the distribution of keys in the target key range 1 to the target key range 10 in storage node 1. Then, the cloud management platform can obtain the key-value distribution table as shown in Table 1. Since the table shows that storage node 1 has a key range of 90 to 95 points and a key range of 96 to 100 points (i.e., the second key range in the aforementioned first storage node), the cloud management platform can determine that the target key range 10 (90 to 100 points) includes these two key ranges. It can directly read the number of values ​​corresponding to these two key ranges from the key-value distribution table to generate data sampling results and return them to the user.

[0084] In another example, suppose the data sampling request contains the sampling key as "score", the number of sampling points as "25", and the sampling range as "storage node 1". In this case, there are 25 target keys to be sampled. Target key range 1 is 1 to 4 points, target key range 2 is 5 to 8 points, ..., target key range 24 is 93 to 96 points, and target key range 25 is 97 to 100 points. Based on the key value distribution table shown in Table 1, it can be seen that within these 25 target key ranges, none of the target key ranges contain any key range possessed by storage node 1. Therefore, the cloud management platform can scan the user-related key-value pairs stored in storage node 1, namely key-value pairs 1 to 3 and key-value pairs 7 to 9, based on the partition routing table. Since the key of key-value pair 7 is 98 points, the key of key-value pair 8 is 94 points, and the key of key-value pair 9 is 93 points, the cloud management platform can determine that the key of key-value pair 7 falls into target key range 25, and the keys of key-value pair 8 and key-value pair 9 fall into target key range 24. It can determine that the number of values ​​corresponding to target key range 24 is 2, and the number of values ​​corresponding to target key range 25 is 1. The platform then uses the number of values ​​corresponding to these two target key ranges to generate data sampling results and return them to the user.

[0085] (2) When the data sampling request does not specify any of the multiple storage nodes, it means that the user needs to determine the distribution of keys within the first key range across these multiple storage nodes. For ease of explanation, the key ranges of each storage node recorded in the key-value distribution table can be referred to as the second key range. The cloud management platform can then check whether the first key range includes the second key ranges of each storage node in the key-value distribution table. If it does, the cloud management platform determines the number of values ​​corresponding to these second key ranges in the key-value distribution table and uses the number of values ​​corresponding to these second key ranges to generate data sampling results, which are then returned to the user. If it does not include them, the cloud management platform can scan each storage node based on the partition routing table to obtain all key-value pairs stored by the user on each storage node, and determine which part of the key-value pairs contains keys that fall within the first key range. The number of values ​​in this part of the key-value pairs is taken as the number of values ​​corresponding to the first key range. In this way, the cloud management platform can use the number of values ​​corresponding to the first key range to generate data sampling results, which are then returned to the user.

[0086] In another example, suppose the data sampling request contains the sampling key "score", the number of sampling points is "10", and the sampling range is "storage node 1 and storage node 2". The cloud management platform can determine that it needs to determine the distribution of keys in the target key range 1 to target key range 10 in storage node 1 and storage node 2. Next, the cloud management platform can obtain the key value distribution table as shown in Table 1. Since the table shows that both storage node 1 and storage node 2 have a key range of 90 to 95 points and a key range of 96 to 100 points (i.e., the second key range in the aforementioned storage nodes), the cloud management platform can determine that the target key range 10 (90 to 100 points) includes these four key ranges. It can directly read the number of values ​​corresponding to these four key ranges from the key value distribution table to generate the data sampling results and return them to the user.

[0087] In another example, suppose the data sampling request contains the sampling key "score", the number of sampling points is "25", and the sampling range is "storage node 1 and storage node 2". In this case, there are 25 target key ranges to be sampled. Based on the key value distribution table shown in Table 1, it can be seen that in these 25 target key ranges, no target key range contains any key range possessed by storage node 1 or storage node 2. Therefore, the cloud management platform can scan the user-related key-value pairs stored in storage node 1 and storage node 2, i.e., key-value pairs 1 to 12, based on the partition routing table. Since the key of key-value pair 7 is 98 points, the key of key-value pair 8 is 94 points, the key of key-value pair 9 is 93 points, and the key of key-value pair 10 is 93 points, the key-value pair... The key for key-value pair 11 scores 98 points, and the key for key-value pair 12 scores 97 points. Therefore, the cloud management platform can determine that the key for key-value pair 7, key for key-value pair 11, and key for key-value pair 12 fall into the target key range 25, and the key for key-value pair 8, key for key-value pair 9, and key for key-value pair 10 fall into the target key range 24. It can determine that the number of values ​​corresponding to the target key range 24 is 3, and the number of values ​​corresponding to the target key range 25 is 3. The platform uses the number of values ​​corresponding to these two target key ranges to generate data sampling results and return them to the user.

[0088] More specifically, the cloud management platform can obtain data sampling results through the following methods:

[0089] (1) When there are multiple second key ranges, the cloud management platform can add up the number of values ​​corresponding to multiple second key ranges to obtain the sum of the number of values ​​corresponding to these multiple second key ranges, and use the sum as the data sampling result.

[0090] As in the example above, such as Figure 6 As shown ( Figure 6(Another schematic diagram of the user interface provided in the embodiment of this application) After determining that the target key range 10 (90 to 100 points) includes storage node 1 having a key range of 90 to 95 points and a key range of 96 to 100 points, the cloud management platform can directly read the number of values ​​corresponding to these two key ranges from the key value distribution table, and add these two numbers together. The sum can be used as the data sampling result, i.e., {90 to 100 points: 3}, and notify the user in the form of a prompt window.

[0091] (2) When the number of the second key range is one, the cloud management platform can directly use the number of values ​​corresponding to the second key range as the data sampling result.

[0092] In another example, suppose the data sampling request includes a sampling key range (equivalent to the user-defined sampling key and number of sampling points) of "age less than or equal to 19 years old" and a sampling range of "storage node 2". The cloud management platform can determine that it needs to determine the distribution of keys within the sampling key range (i.e., the target key range) in storage node 2. Next, the cloud management platform can obtain the key-value distribution table as shown in Table 1. Since this table shows that storage node 2 has the key range of 19 years old, the cloud management platform can determine that the sampling key range includes this key range. It can directly read the number of values ​​corresponding to this key range from the key-value distribution table to directly use as the data sampling result: {19 years old: 3}, and return it to the user in the form of a prompt window.

[0093] More specifically, the cloud management platform can also perform the following operations:

[0094] In a multi-node storage environment, dynamic merging or splitting of storage nodes may occur. During dynamic splitting, key-value pairs from one storage node can be migrated to a newly created storage node. Similarly, during dynamic merging, key-value pairs from the merged storage node can be migrated to the merged storage node. For clarity, the storage node from which the key-value pairs are migrated will be referred to as the second storage node, and the storage node receiving the key-value pairs will be referred to as the third storage node. When dynamic splitting or merging occurs, the cloud management platform can migrate key-value pairs from the second storage node to the third storage node. It should be noted that in the key-value distribution table, the keys of the migrated key-value pairs fall within the third key range possessed by the second storage node.

[0095] Because the key-value pair migrated from the second storage node to the third storage node, the distribution of keys in these two storage nodes changed. Therefore, the cloud management platform can update the key-value distribution table based on this key-value pair, including the third key range, the number of values ​​corresponding to the third key range, the fourth key range of the third storage node, and the number of values ​​corresponding to the fourth key range. This results in an updated key-value distribution table. In the updated key-value distribution table, the updated third key range does not contain any keys, while the updated fourth key range does. Furthermore, the number of values ​​corresponding to the updated third key range is less than the number of values ​​corresponding to the original third key range, and the number of values ​​corresponding to the updated fourth key range is greater than the number of values ​​corresponding to the original fourth key range.

[0096] In another example, suppose key-value pair 13 has a key score of 90, key-value pair 14 has a key score of 95, key-value pair 15 has a key score of 96, and key-value pair 16 has a key score of 97, and all four key-value pairs are stored in storage node 3. In this case, the key-value distribution table is shown in Table 2:

[0097] Table 2

[0098] Storage nodes: Bond distribution: Storage Node 3 Key range: 90 to 100 points, number of values ​​corresponding to the key range: 4.

[0099] Based on Table 2 above, storage node 3 (i.e. the aforementioned second storage node) has a key range of 90 to 100 points (i.e. the aforementioned third key range), and the number of its corresponding values ​​is 4.

[0100] Due to a dynamic split, the cloud management platform migrated key-value pairs 15 and 16 from storage node 3 to the newly created storage node 4 (the aforementioned third storage node). Consequently, the cloud management platform needs to update the key-value distribution table. Based on the keys of key-value pairs 15 and 16, it can determine that storage node 4 has a new key range of 96 to 97 points (the aforementioned updated fourth key range, previously empty), and the number of values ​​corresponding to this key range is 2. Furthermore, the cloud management platform can remove the keys 96 and 97 from the key range of 90 to 100 points, resulting in a new key range of 90 to 95 points (the aforementioned updated third key range), with the number of values ​​corresponding to this key range updated to 2, as shown in Table 3.

[0101] Table 3

[0102] Storage nodes: Bond distribution: Storage Node 3 Key range: 90 to 95 points, number of values ​​corresponding to the key range: 2. Storage Node 4 Key range: 96 to 97 points, number of values ​​corresponding to the key range: 2.

[0103] In this way, the cloud management platform can collaboratively update the key-value distribution table during the storage node splitting process, which is beneficial for users to continue sampling keys and understand the real-time key distribution.

[0104] It should be understood that in this embodiment, the process of creating a key-value distribution table is only illustrated by the user specifying multiple key ranges included in the key-value distribution table. In actual applications, the user may not specify these multiple key ranges, but only send a creation request for the key-value distribution table to the cloud management platform. This request only indicates a table creation operation, and the cloud management platform will automatically divide multiple key ranges based on the multiple keys in the user's multiple key-value pairs, and then generate a key-value distribution table based on these multiple key ranges. There are no restrictions here.

[0105] It should also be understood that this embodiment is only illustrated with examples of a certain primary index and a certain secondary index, and does not impose any restrictions on the types and number of primary and secondary indexes in this application. That is to say, in Table 1, the primary index is age-data and the secondary index is score-data. In actual applications, the primary index is ID-data and the secondary index is score-data and gender-data, etc., and no restrictions are imposed here.

[0106] It should also be understood that in this embodiment, when a user sends a data sampling request, the user can specify the sampling key and the number of sampling points so that the cloud management platform can determine the target key range to be sampled based on the sampling key and sampling point data, or the user can directly specify the sampling key range so that the cloud management platform can directly use the sampling key range as the target key range to be sampled, etc. There are no restrictions here.

[0107] In this embodiment, the cloud management platform can treat multiple user data as multiple values ​​in multiple key-value pairs and store these key-value pairs in the infrastructure managed by the cloud management platform. For the multiple keys in these key-value pairs, when the user needs to determine the distribution of certain keys, the user can set these keys as a first key range and send a data sampling request to the sampling interface provided by the cloud management platform. Therefore, the cloud management platform can receive the data sampling request sent by the user through the sampling interface. This data sampling request can be used to indicate the first key range set by the user. Thus, the cloud management platform can obtain a key-value distribution table, which records multiple key ranges and the number of values ​​corresponding to each key range. These multiple key ranges are obtained by the user dividing the multiple keys in these key-value pairs. Then, the cloud management platform can detect whether the first key range set by the user includes a second key range among the multiple key ranges recorded in the key-value distribution table. If it does, the cloud management platform can generate a data sampling result based on the number of values ​​corresponding to the second key range and return the data sampling result to the user to meet the user's needs. In the aforementioned process, because the cloud management platform has a built-in key-value distribution table, once the first key range set by the user includes a second key range recorded in the key-value distribution table, the cloud management platform can use the number of values ​​corresponding to the second key range recorded in the key-value distribution table as the distribution of keys in the first key range, and notify the user of this distribution. In this way, the cloud management platform does not need to scan the infrastructure to determine the distribution of keys in the first key range, saving the cloud management platform scanning operations during the key distribution determination process, thereby reducing the waiting time for the user and improving the user experience.

[0108] Furthermore, in this embodiment, users can define the sampling key and the number of sampling points according to their own business needs. This is equivalent to users being able to customize the target key range to be sampled (i.e., the aforementioned first key range) so as to understand the distribution of keys in the target key range with the help of the distributed key-value storage system. This can meet the user's personalized processing needs and business needs, and is conducive to further improving the user experience.

[0109] Furthermore, in this embodiment, the user can also specify a sampling range, such as a certain storage node among multiple storage nodes (i.e., the aforementioned first storage node, etc.). Once the target key range set by the user does not include the key range of the storage node recorded in the key value distribution table, the cloud management platform can directly scan the key-value pairs stored in the storage node. This eliminates the need for a full scan of multiple storage nodes, improves the scanning efficiency of key-value pairs, reduces the user's waiting time, and further enhances the user experience.

[0110] The above is a detailed description of the data sampling method based on a cloud management platform provided in the embodiments of this application. The cloud management platform provided in the embodiments of this application will be described below. Figure 7 A schematic diagram of the structure of the cloud management platform provided in the embodiments of this application is shown below. Figure 7 As shown, the cloud management platform is used to manage the infrastructure that provides cloud services. The infrastructure stores multiple key-value pairs, where multiple values ​​in the key-value pairs represent multiple pieces of user data. Multiple keys in the key-value pairs are used to retrieve multiple values. The cloud management platform includes:

[0111] The first receiving module 701 is used to receive a data sampling request sent by a user. The data sampling request is used to indicate a first key range, which includes at least one key among a plurality of keys.

[0112] The acquisition module 702 is used to acquire a key-value distribution table based on a data sampling request. The key-value distribution table is used to record multiple key ranges and the number of values ​​corresponding to each key range. Each key range contains at least one of multiple keys.

[0113] A module 703 is provided to determine that the first key range includes a second key range among multiple key ranges recorded in the key value distribution table, generate a data sampling result based on the number of values ​​corresponding to the second key range, and send the data sampling result to the user.

[0114] In one possible implementation, the infrastructure comprises multiple storage nodes, each storing at least one key-value pair from a plurality of key-value pairs, and a key-value distribution table is used to record each storage node, the range of keys in each storage node, and the number of values ​​corresponding to the key range.

[0115] In one possible implementation, the data sampling request is also used to indicate a first storage node among multiple storage nodes; and a providing module is used to determine a second key range in the first storage node whose first key range contains key value distribution table records, and to generate a data sampling result based on the number of values ​​corresponding to the second key range.

[0116] In one possible implementation, the number of second key ranges is one, and a module is provided to take the number of values ​​corresponding to the second key range as the data sampling result.

[0117] In one possible implementation, there are multiple second key ranges, and a module is provided to take the sum of the number of values ​​corresponding to the multiple second key ranges as the data sampling result.

[0118] In one possible implementation, the cloud management platform further includes: a second receiving module for receiving a creation request for a key-value distribution table sent by a user, the creation request indicating multiple key ranges set by the user for the key-value distribution table; a determining module for determining the number of values ​​corresponding to each key range based on multiple key-value pairs; and a generating module for generating a key-value distribution table based on the multiple key ranges and the number of values ​​corresponding to each key range.

[0119] In one possible implementation, the cloud management platform further includes: a migration module for migrating key-value pairs from a second storage node to a third storage node, wherein the third key range in the second storage node contains the keys in the key-value pairs in the key-value distribution table; and an update module for updating the third key range, the number of values ​​corresponding to the third key range, the fourth key range of the third storage node, and the number of values ​​corresponding to the fourth key range based on the key-value pairs, to obtain an updated key-value distribution table, wherein the updated third key range does not contain keys, and the updated fourth key range contains keys.

[0120] In one possible implementation, the second and third storage nodes are two of a plurality of storage nodes, or the second storage node is one of a plurality of storage nodes and the third storage node is a newly created storage node.

[0121] It should be noted that the information interaction and implementation process between the modules / units of the above-mentioned device are based on the same concept as the method embodiments of this application, and the resulting technical effects are the same as those of the method embodiments of this application. For details, please refer to the description in the method embodiments shown above in the embodiments of this application, and will not be repeated here.

[0122] Please see Figure 8 , Figure 8 This is a schematic diagram of a computing device provided in an embodiment of this application. Figure 8 As shown, the computing device 800 (which can be used to present the aforementioned cloud management platform) includes: a processor 801, a memory 802, a communication interface 803, and a bus 804. The processor 801, memory 802, and communication interface 803 are coupled via the bus (not shown in the figure). The memory 802 stores instructions. When the instructions in the memory 802 are executed, the computing device 800 executes the method performed by the cloud management platform in the above method embodiment.

[0123] The computing device 800 may be one or more integrated circuits configured to implement the methods described above, such as: one or more application-specific integrated circuits (ASICs), or one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs), or a combination of at least two of these forms of integrated circuits. Furthermore, when the units in the device can be implemented in the form of a processing element scheduler, the processing element may be a general-purpose processor, such as a central processing unit (CPU) or other processor capable of calling programs. Alternatively, these units may be integrated together to implement a system-on-a-chip (SOC).

[0124] The processor 801 can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. A general-purpose processor can be a microprocessor or any conventional processor.

[0125] The memory 802 can be volatile memory or non-volatile memory, or it can include both. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

[0126] The memory 802 stores executable program code, and the processor 801 executes this executable program code to implement the functions of the aforementioned first receiving module, acquisition module, and providing module, thereby realizing the aforementioned data sampling method based on the cloud management platform. That is, the memory 802 stores instructions for executing the aforementioned data sampling method based on the cloud management platform.

[0127] The communication interface 803 uses transceiver modules, such as, but not limited to, network interface cards and transceivers, to enable communication between the computing device 800 and other devices or communication networks.

[0128] In addition to the data bus, the 804 bus can also include a power bus, a control bus, and a status signal bus. The bus can be a Peripheral Component Interconnect Express (PCIe) bus, an Extended Industry Standard Architecture (EISA) bus, a Unified Bus (Ubus or UB), a Compute Express Link (CXL) bus, a Cache Coherent Interconnect for Accelerators (CCIX) bus, etc. The bus can be divided into address bus, data bus, and control bus.

[0129] Please see Figure 9 , Figure 9 This is a schematic diagram of a computing device cluster provided in an embodiment of this application. Figure 9 As shown, the computing device cluster 900 includes at least one computing device 800.

[0130] like Figure 9 As shown, the computing device cluster 900 includes at least one computing device 800. The memory 802 of one or more computing devices 800 in the computing device cluster 900 may store the same instructions for executing the aforementioned data sampling method based on the cloud management platform.

[0131] In some possible implementations, the memory 802 of one or more computing devices 800 in the computing device cluster 900 may also store partial instructions for executing the aforementioned data sampling method based on the cloud management platform. In other words, a combination of one or more computing devices 800 can jointly execute the aforementioned data sampling method based on the cloud management platform.

[0132] It should be noted that the memory 802 in different computing devices 800 within the computing device cluster 900 can store different instructions, each used to execute a portion of the functions of the aforementioned cloud management platform. That is, the instructions stored in the memory 802 of different computing devices 800 can implement the functions of one or more modules, such as the first receiving module, the acquisition module, and the providing module.

[0133] In some possible implementations, one or more computing devices 800 in the computing device cluster 900 can be connected via a network. This network can be a wide area network (WAN) or a local area network (LAN), etc.

[0134] Please see Figure 10 , Figure 10 This is a schematic diagram illustrating the network connection of computer devices in a computer cluster provided in an embodiment of this application. Figure 10 As shown, the two computing devices 800A and 800B are connected via a network. Specifically, they are connected to the network through the communication interfaces in each computing device.

[0135] In one possible implementation, the memory in computing device 800A stores instructions for performing the functions of modules such as the first receiving module. Meanwhile, the memory in computing device 800B stores instructions for performing the functions of modules such as the acquisition module and the provision module.

[0136] It should be understood that Figure 10 The functions of computing device 800A shown can also be performed by multiple computing devices. Similarly, the functions of computing device 800B can also be performed by multiple computing devices.

[0137] This application also relates to a computer storage medium storing a program for signal processing, which, when run on a computer, causes the computer to perform actions such as... Figure 3 The steps performed by the cloud management platform in the illustrated embodiment.

[0138] This application also relates to a computer program product that stores instructions that, when executed by a computer, cause the computer to perform actions such as... Figure 3 The steps performed by the cloud management platform in the illustrated embodiment.

[0139] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0140] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between apparatuses or units through some interfaces, and may be electrical, mechanical, or other forms.

[0141] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0142] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0143] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A data sampling method based on a cloud management platform, characterized in that, The cloud management platform is used to manage the infrastructure providing cloud services. The infrastructure stores multiple key-value pairs, where multiple values ​​in each key-value pair represent multiple pieces of user data. Multiple keys in each key-value pair are used to retrieve the multiple values. The method includes: The cloud management platform receives a data sampling request sent by the user. The data sampling request is used to indicate a first key range, which includes at least one key among the plurality of keys. The cloud management platform obtains a key-value distribution table based on the data sampling request. The key-value distribution table is used to record multiple key ranges and the number of values ​​corresponding to each key range. Each key range contains at least one of the multiple keys. The first key range is determined to include the second key range among the multiple key ranges recorded in the key value distribution table. The cloud management platform generates a data sampling result based on the number of values ​​corresponding to the second key range and sends the data sampling result to the user.

2. The method according to claim 1, characterized in that, The infrastructure includes multiple storage nodes, each storage node stores at least one key-value pair from the multiple key-value pairs, and the key-value distribution table is used to record each storage node, the key range in each storage node, and the number of values ​​corresponding to the key range.

3. The method according to claim 1 or 2, characterized in that, The data sampling request is also used to instruct the first storage node among the plurality of storage nodes; The first key range is determined to include the second key range among the multiple key ranges recorded in the key value distribution table. The cloud management platform generates data sampling results based on the number of values ​​corresponding to the second key range, including: The first key range is determined to include the second key range in the first storage node recorded in the key value distribution table, and the cloud management platform generates data sampling results based on the number of values ​​corresponding to the second key range.

4. The method according to any one of claims 1 to 3, characterized in that, The second key range has one value, and the cloud management platform generates data sampling results based on the number of values ​​corresponding to the second key range, including: The cloud management platform uses the number of values ​​corresponding to the second key range as the data sampling result.

5. The method according to any one of claims 1 to 3, characterized in that, The second key range has multiple values, and the cloud management platform generates data sampling results based on the number of values ​​corresponding to the second key range, including: The cloud management platform uses the sum of the number of values ​​corresponding to multiple second key ranges as the data sampling result.

6. The method according to any one of claims 1 to 5, characterized in that, The method further includes: The cloud management platform receives a creation request for the key-value distribution table sent by the user. The creation request is used to indicate the multiple key ranges set by the user for the key-value distribution table. The cloud management platform determines the number of values ​​corresponding to each key range based on the multiple key-value pairs; The cloud management platform generates the key-value distribution table based on the multiple key ranges and the number of values ​​corresponding to each key range.

7. The method according to claims 1 to 6, characterized in that, The method further includes: The cloud management platform migrates the key-value pairs in the second storage node to the third storage node. In the key-value distribution table, the third key range in the second storage node includes the key in the key-value pair. The cloud management platform updates the third key range, the number of values ​​corresponding to the third key range, the fourth key range of the third storage node, and the number of values ​​corresponding to the fourth key range based on the key-value pairs to obtain an updated key-value distribution table. In the updated key-value distribution table, the updated third key range does not contain the key, and the updated fourth key range contains the key.

8. The method according to claim 7, characterized in that, The second storage node and the third storage node are two of a plurality of storage nodes, or the second storage node is one of the plurality of storage nodes and the third storage node is a newly created storage node.

9. A cloud management platform, characterized in that, The cloud management platform is used to manage the infrastructure providing cloud services. The infrastructure stores multiple key-value pairs, where multiple values ​​in each key-value pair represent multiple pieces of user data. Multiple keys in each key-value pair are used to retrieve the multiple values. The cloud management platform includes: A first receiving module is configured to receive a data sampling request sent by the user, the data sampling request being used to indicate a first key range, the first key range including at least one of the plurality of keys; The acquisition module is used to acquire a key-value distribution table based on the data sampling request. The key-value distribution table is used to record multiple key ranges and the number of values ​​corresponding to each key range. Each key range contains at least one key among the multiple keys. A module is provided to determine that the first key range includes a second key range among the multiple key ranges recorded in the key value distribution table, generate a data sampling result based on the number of values ​​corresponding to the second key range, and send the data sampling result to the user.

10. The cloud management platform according to claim 9, characterized in that, The infrastructure includes multiple storage nodes, each storage node stores at least one key-value pair from the multiple key-value pairs, and the key-value distribution table is used to record each storage node, the key range in each storage node, and the number of values ​​corresponding to the key range.

11. The cloud management platform according to claim 9 or 10, characterized in that, The data sampling request is also used to instruct the first storage node among the plurality of storage nodes; The providing module is used to determine that the first key range includes the second key range in the first storage node recorded in the key value distribution table, and to generate a data sampling result based on the number of values ​​corresponding to the second key range.

12. The cloud management platform according to any one of claims 9 to 11, characterized in that, The second key range has one value, and the providing module is used to take the number of values ​​corresponding to the second key range as the data sampling result.

13. The cloud management platform according to any one of claims 9 to 11, characterized in that, The number of the second key ranges is multiple, and the providing module is used to take the sum of the number of values ​​corresponding to the multiple second key ranges as the data sampling result.

14. The cloud management platform according to any one of claims 9 to 13, characterized in that, The cloud management platform also includes: The second receiving module is used to receive a creation request for the key-value distribution table sent by the user, wherein the creation request is used to indicate the multiple key ranges set by the user for the key-value distribution table; The determining module is used to determine the number of values ​​corresponding to each key range based on the plurality of key-value pairs; The generation module is used to generate the key-value distribution table based on the plurality of key ranges and the number of values ​​corresponding to each key range.

15. The cloud management platform according to claims 9 to 14, characterized in that, The cloud management platform also includes: A migration module is used to migrate key-value pairs in a second storage node to a third storage node, wherein in the key-value distribution table, the third key range in the second storage node includes the keys in the key-value pairs. An update module is used to update the third key range, the number of values ​​corresponding to the third key range, the fourth key range of the third storage node, and the number of values ​​corresponding to the fourth key range based on the key-value pairs, to obtain an updated key-value distribution table. In the updated key-value distribution table, the updated third key range does not contain the key, and the updated fourth key range contains the key.

16. The cloud management platform according to claim 15, characterized in that, The second storage node and the third storage node are two of a plurality of storage nodes, or the second storage node is one of the plurality of storage nodes and the third storage node is a newly created storage node.

17. A computing device cluster, characterized in that, The computing device cluster includes at least one computing device, each computing device including a processor and memory: The memory is used to store instructions; The processor is configured to, according to the instructions, cause the computing device cluster to perform the method of any one of claims 1 to 8.

18. A computer storage medium, characterized in that, The computer storage medium stores one or more instructions that, when executed by one or more computers, cause the one or more computers to perform the method of any one of claims 1 to 8.

19. A computer program product, characterized in that, The computer program product stores instructions that, when executed by a computer, cause the computer to perform the method described in any one of claims 1 to 8.

Images