A data processing method based on a cloud management platform and a cloud management platform

Abstract

The application discloses a data processing method based on a cloud management platform and the cloud management platform, which can improve user experience. The method comprises the following steps: when a user has a data processing requirement, the user can send a data processing request of the user and parameters of a client of the user to a data processing interface provided by the cloud management platform through the client, the data processing request is used for indicating target data in a plurality of objects stored in a plurality of storage buckets by the user, and the parameters of the client of the user are used for indicating the performance of the client of the user. Then, the cloud management platform can select at least two storage buckets from the plurality of storage buckets based on the data processing request and the parameters of the client of the user. Subsequently, the cloud management platform can notify the client of the user to obtain the target data from the at least two storage buckets, so that the client of the user can process the target data to obtain a processing result of the target data, thereby meeting the data processing requirement of the user.

Description

Technical Field

[0001] This application relates to the field of cloud technology, and in particular to a data processing method based on a cloud management platform and a cloud management platform. Background Technology

[0002] With the rapid development of cloud technology, more and more users are choosing object storage services provided by cloud providers to store their data. In this way, users' data does not need to be stored locally, which not only reduces the user's local data storage costs, but also allows cloud providers to provide a certain level of security for user data when data is deployed in the cloud.

[0003] In related technologies, a cloud service system may include a cloud management platform and a storage bucket that provides object storage services, with user data pre-stored in the bucket. When a user needs to use their data, they can access the storage bucket through a client to retrieve the required data, process it, and obtain the processing results.

[0004] In the above process, under certain special scenarios, the user side may have a large number of accesses to a certain data. Therefore, the storage bucket will receive a large number of download requests for that data from the user side clients. Due to the limited performance of the storage bucket, it cannot provide enough download bandwidth for the clients, and thus cannot meet the data acquisition needs of these clients. Summary of the Invention

[0005] This application provides a data processing method and a cloud management platform based on a cloud management platform, which can meet the data acquisition and processing needs of users' clients, thereby improving the user experience.

[0006] A first aspect of this application provides a data processing method based on a cloud management platform. The cloud management platform implementing this method can manage infrastructure providing object storage services. This infrastructure includes multiple storage buckets located in different regions, and each of the multiple storage buckets stores multiple objects belonging to a user. The method includes:

[0007] When a user needs to process target data across multiple objects, the user can input a data processing request through their client into the data processing interface provided by the cloud management platform. In this way, the cloud management platform can receive the data processing request sent by the user's client through the data processing interface; the data processing request indicates the target data to be processed.

[0008] At the same time, users can also input their client parameters into the data processing interface through their client. Therefore, the cloud management platform can receive the client parameters sent by the user's client through the data processing interface. The client parameters can be used to indicate the client's performance.

[0009] After obtaining the data processing request and the user's client parameters, the cloud management platform can use the data processing request and the user's client parameters to select at least two storage buckets from multiple storage buckets that can provide the user with the target data.

[0010] Once these at least two storage buckets are identified, the cloud management platform can notify the user's client to retrieve the target data from these two storage buckets and process the target data to obtain the processing result, thereby meeting the user's data processing needs.

[0011] As can be seen from the above method, since each of the multiple storage buckets stores multiple user objects, when a user needs to process target data from these multiple objects, the cloud management platform can select at least two storage buckets accessible to the user's client and notify the user's client to retrieve the target data from these at least two storage buckets to complete the target data processing. In this way, these at least two storage buckets can share the pressure of providing target data to the user's client. That is to say, these at least two storage buckets as a whole can have sufficient performance. Even if there are a large number of clients on the user side that need to retrieve target data, these two storage buckets can provide sufficient download bandwidth for the target data for a large number of clients on the user side, thereby meeting the data retrieval and processing needs of these clients and improving the user experience.

[0012] In one possible implementation, the cloud management platform selects at least two storage buckets from multiple storage buckets based on data processing requests and client parameters. This includes: the cloud management platform obtaining parameters for multiple storage buckets, where these parameters indicate the performance of the multiple storage buckets; the cloud management platform obtaining evaluation values ​​for the multiple storage buckets based on the data processing request, client parameters, and the parameters of the multiple storage buckets; and the cloud management platform selecting at least two storage buckets from the multiple storage buckets whose evaluation values ​​are greater than a preset value. In the aforementioned implementation, after obtaining the data processing request and the user's client parameters, the cloud management platform can also obtain parameters describing the performance of the multiple storage buckets. Therefore, the cloud management platform can use the data processing request, the user's client parameters, and the parameters of the multiple storage buckets to obtain evaluation values ​​for the multiple storage buckets. Then, the cloud management platform can select at least two storage buckets from the multiple storage buckets whose evaluation values ​​are greater than a preset value to provide the target data to the user's client, thereby ensuring successful data acquisition.

[0013] In one possible implementation, at least two storage buckets include a first storage bucket and a second storage bucket. The method further includes: a cloud management platform determining, based on the evaluation value of the first storage bucket, a first sub-data of the target data that the first storage bucket needs to provide; the cloud management platform determining, based on the evaluation value of the second storage bucket, a second sub-data of the target data that the second storage bucket needs to provide, wherein the first sub-data and the second sub-data are different sub-data in the target data, and the first sub-data is ordered before the second sub-data in the target data; the cloud management platform notifying the client to obtain the target data from the at least two storage buckets includes: the cloud management platform notifying the client to obtain the first sub-data from the first storage bucket and then obtain the second sub-data from the second storage bucket. In the aforementioned implementation, since the at least two storage buckets include a first storage bucket and a second storage bucket, the cloud management platform can determine the first sub-data of the target data that the first storage bucket needs to provide based on the evaluation value of the first storage bucket, and determine the second sub-data of the target data that the second storage bucket needs to provide based on the evaluation value of the second storage bucket. Since the first and second sub-data are different sub-data within the target data, and the first sub-data precedes the second sub-data, the cloud management platform can instruct the client to retrieve the first sub-data from the first storage bucket and then the second sub-data from the second storage bucket to successfully obtain the target data. Therefore, under the control of the cloud management platform, the user's client retrieves different portions (the aforementioned first and second sub-data) of the target data from different storage buckets (the first and second storage buckets mentioned above), thereby fully utilizing the performance provided by each storage bucket and reducing the pressure on data delivery between storage buckets.

[0014] In one possible implementation, the method further includes: the cloud management platform instructing the client to partition the pre-configured cache to obtain a first sub-cache and a second sub-cache, wherein the first sub-cache is ordered before the second sub-cache in the cache; the cloud management platform instructs the client to store the first sub-data in the first sub-cache and the second data in the second sub-cache; the client processes the target data, and the processing result of the target data includes: the client reads the first sub-data from the first sub-cache, then reads the second sub-data from the second sub-cache to obtain the target data, and processes the target data to obtain the processing result of the target data. In the aforementioned implementation, after determining the first and second sub-data, the cloud management platform instructs the user's client to partition its own cache to obtain the first and second sub-caches. Based on this, after the user's client obtains the first sub-data from the first storage bucket, the user's client can store the first sub-data in the first sub-cache, and after the user's client obtains the second sub-data from the second storage bucket, the user's client can store the second data in the second sub-cache. Because the first sub-cache precedes the second sub-cache in this cache, the user's client can first read the first sub-data from the first sub-cache, and then read the second sub-data from the second sub-cache to successfully obtain the target data. The client can then process the target data to obtain the processing result. Thus, under the control of the cloud management platform, the user's client stores different parts of the target data obtained from different storage buckets in different parts contained in its own cache (the aforementioned first and second sub-caches). When processing is needed, the target data is directly read from these different parts of the cache to complete the processing, thereby fully utilizing its own cache to quickly achieve data processing and achieving data processing that is imperceptible to the user.

[0015] In one possible implementation, the size of the first sub-cache matches the size of the first sub-data, and the size of the first sub-data matches the estimated value of the first bucket. In the aforementioned implementation, the size of the first sub-data typically matches the estimated value of the first bucket, and the two are usually positively correlated. Similarly, the size of the second sub-data typically matches the estimated value of the second bucket, and the two are usually positively correlated. Likewise, the size of the second sub-data typically matches the size of the second sub-cache, and the two are usually the same.

[0016] In one possible implementation, the client's parameters include at least one of the following: the client's throughput, the time required for the client to retrieve data from multiple buckets, the time required for the client to process the data, and the time required for the client to wait before retrieving data from multiple buckets.

[0017] In one possible implementation, the parameters of the multiple buckets include at least one of the following: the available storage capacity of the multiple buckets and the throughput of the multiple buckets, wherein the available storage capacity of the multiple buckets is determined based on the maximum storage capacity of the multiple buckets and the used storage capacity of the multiple buckets.

[0018] A second aspect of this application provides a cloud management platform for managing infrastructure that provides object storage services. The infrastructure includes multiple storage buckets located in different regions, each of which stores multiple objects belonging to a user. The cloud management platform includes: a receiving module for receiving data processing requests sent by a user's client, wherein the data acquisition request indicates target data to be processed among the multiple objects; the receiving module is also used to receive client parameters sent by the client, wherein the client parameters indicate the client's performance; a selection module for selecting at least two storage buckets from the multiple storage buckets based on the data processing request and the client parameters; and a first notification module for notifying the client to acquire the target data from the at least two storage buckets and process the target data to obtain the processing result of the target data.

[0019] In one possible implementation, the selection module is configured to: obtain parameters of multiple storage buckets, wherein the parameters of the multiple storage buckets are used to indicate the performance of the multiple storage buckets; obtain evaluation values ​​of the multiple storage buckets based on the data processing request, the client parameters, and the parameters of the multiple storage buckets; and select at least two storage buckets from the multiple storage buckets whose evaluation values ​​are greater than a preset value.

[0020] In one possible implementation, at least two storage buckets include a first storage bucket and a second storage bucket. The cloud management platform further includes: a first determining module, used to determine a first sub-data of the target data that the first storage bucket needs to provide based on the evaluation value of the first storage bucket; a second determining module, used to determine a second sub-data of the target data that the second storage bucket needs to provide based on the evaluation value of the second storage bucket, wherein the first sub-data and the second sub-data are different sub-data in the target data, and the order of the first sub-data in the target data is before the order of the second sub-data in the target data; and a first notification module, used to notify the client to obtain the first sub-data from the first storage bucket and then obtain the second sub-data from the second storage bucket.

[0021] In one possible implementation, the cloud management platform further includes: a second notification module for notifying the client to divide the pre-set cache into a first sub-cache and a second sub-cache, wherein the first sub-cache is ordered before the second sub-cache in the cache; a third notification module for notifying the client to store the first sub-data in the first sub-cache and the second data in the second sub-cache; and a first notification module for notifying the client to read the first sub-data from the first sub-cache and then read the second sub-data from the second sub-cache to obtain the target data, process the target data, and obtain the processing result of the target data.

[0022] In one possible implementation, the size of the first sub-cache is matched with the size of the first sub-data, and the size of the first sub-data is matched with the evaluation value of the first bucket.

[0023] In one possible implementation, the client's parameters include at least one of the following: the client's throughput, the time required for the client to retrieve data from multiple buckets, the time required for the client to process the data, and the time required for the client to wait before retrieving data from multiple buckets.

[0024] In one possible implementation, the parameters of the multiple buckets include at least one of the following: the available storage capacity of the multiple buckets and the throughput of the multiple buckets, wherein the available storage capacity of the multiple buckets is determined based on the maximum storage capacity of the multiple buckets and the used storage capacity of the multiple buckets.

[0025] A third aspect of this application provides a computing device cluster, the computing device cluster including at least one computing device, each computing device including a processor and a memory: the memory is used to store instructions; the processor is used to cause the computing device cluster to perform the method described in the first aspect or any possible implementation of the first aspect according to the instructions.

[0026] A fourth aspect of this application provides a computer storage medium storing one or more instructions that, when executed by one or more computers, cause the one or more computers to perform the method described in the first aspect or any possible implementation of the first aspect.

[0027] A fifth aspect of this application provides a computer program product storing instructions that, when executed by a computer, cause the computer to perform the method described in the first aspect or any possible implementation of the first aspect.

[0028] In this embodiment, when a user has a data processing need, the user can send a data processing request and client parameters to the data processing interface provided by the cloud management platform through their client. The data processing request indicates the target data stored by the user in multiple objects across multiple storage buckets, and the client parameters indicate the performance of the user's client. Then, based on the data processing request and the user's client parameters, the cloud management platform can select at least two storage buckets from the multiple storage buckets. Subsequently, the cloud management platform can notify the user's client to retrieve the target data from these at least two storage buckets, enabling the user's client to process the target data and obtain the processing result, thus satisfying the user's data processing needs. In the aforementioned process, since each of the multiple storage buckets stores multiple objects belonging to the user, when the user needs to process target data from these multiple objects, the cloud management platform can select at least two storage buckets accessible to the user's client and notify the user's client to retrieve the target data from these at least two storage buckets to complete the target data processing. In this way, these at least two storage buckets can share the pressure of providing target data to the user's client. That is to say, these at least two storage buckets as a whole can have sufficient performance. Even if there are a large number of clients on the user side that need to retrieve target data, these two storage buckets can provide sufficient download bandwidth for the target data for a large number of clients on the user side, thereby meeting the data retrieval and processing needs of these clients and improving the user experience. Attached Figure Description

[0029] Figure 1 A schematic diagram of the structure of the cloud service system provided in the embodiments of this application;

[0030] Figure 2 This is a schematic diagram illustrating an application example of the cloud service system provided in the embodiments of this application;

[0031] Figure 3 A flowchart illustrating a data processing method based on a cloud management platform provided in an embodiment of this application;

[0032] Figure 4 A schematic diagram illustrating an application example of the data processing method provided in the embodiments of this application;

[0033] Figure 5 Another schematic diagram illustrating an application example of the data processing method provided in the embodiments of this application;

[0034] Figure 6 Another schematic diagram illustrating an application example of the data processing method provided in the embodiments of this application;

[0035] Figure 7 A schematic diagram of the structure of the cloud management platform provided in the embodiments of this application;

[0036] Figure 8 A schematic diagram of the structure of a computing device provided in an embodiment of this application;

[0037] Figure 9 A schematic diagram of the structure of a computing device cluster provided in an embodiment of this application;

[0038] Figure 10 This is a schematic diagram illustrating the network connection of computer devices in a computer cluster provided in an embodiment of this application. Detailed Implementation

[0039] This application provides a data processing method and a cloud management platform based on a cloud management platform, which can meet the data acquisition and processing needs of users' clients, thereby improving the user experience.

[0040] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms are interchangeable where appropriate; this is merely a way of distinguishing objects with the same attributes in the embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of elements is not necessarily limited to those elements, but may include other elements not explicitly listed or inherent to those processes, methods, products, or apparatuses.

[0041] With the rapid development of cloud technology, more and more users are choosing object storage services provided by cloud providers to store their data. In this way, users' data does not need to be stored locally, which not only reduces the user's local data storage costs, but also allows cloud providers to provide a certain level of security for user data when data is deployed in the cloud.

[0042] In related technologies, a cloud service system may include a cloud management platform and storage buckets that provide object storage services, with user data pre-stored in these buckets. When a user needs to access their data, they can access the storage bucket through a client to retrieve the required data, process it, and obtain the processing results. For example, a user can store training data for training a neural network model in a cloud storage bucket. When the user needs to use the training data to train the neural network model on their client, they can access the storage bucket through their client to read the training data and complete the training of the neural network model.

[0043] In the above process, under certain special scenarios (for example, the aforementioned neural network model training scenario), the user side may have a large number of accesses to a certain data. Therefore, the storage bucket will receive a large number of download requests for the data sent by the user side clients. Due to the limited performance of the storage bucket, it cannot provide enough download bandwidth for the clients, and thus cannot meet the data acquisition needs of these clients.

[0044] Furthermore, if the storage bucket fails, the user's client will be unable to access the user's data, thereby damaging the user's business and resulting in a poor user experience.

[0045] To address the aforementioned problems, this application provides a data processing method based on a cloud management platform, which can be implemented through a cloud service system. Figure 1 A schematic diagram of the structure of the cloud service system provided in the embodiments of this application is shown below. Figure 1 As shown, a cloud service system includes the infrastructure that provides object storage services and a cloud management platform that manages this infrastructure. The cloud management platform and the infrastructure are described separately below:

[0046] A cloud management platform can centrally manage the infrastructure of the entire cloud service system (for example, creating multiple buckets within the infrastructure to provide object storage services to a user, based on the user's instructions; these buckets can store multiple objects, each containing at least one piece of data, etc.). The cloud management platform can also be accessible to users outside the cloud service system and respond to their requests. For instance, the cloud management platform can provide various interfaces such as login, data storage, and data processing interfaces for user clients (e.g., the user's terminal device or browser on that device) to access. The login interface allows the cloud management platform to authenticate a user's client, granting access upon successful authentication. Similarly, the data processing interface allows user clients to send data processing requests and client parameters to the cloud management platform. The data processing request specifies the target data stored in multiple objects across multiple buckets, while the client parameters indicate the client's performance. The cloud management platform can then select at least two buckets from the multiple buckets based on the data processing request and the client's parameters. Subsequently, the cloud management platform can notify the user's client to retrieve the target data from at least two storage buckets, so that the user's client can process the target data and obtain the processing result to meet the user's data processing needs.

[0047] The infrastructure comprises multiple buckets that provide object storage services to users. Each bucket in a cloud service system (which can also be understood as multiple buckets in the cloud) can occupy one or more storage nodes. Each of these storage nodes can occupy a certain amount of computing resources (e.g., central processing unit (CPU) and graphics processing unit (GPU), etc.), a certain amount of storage resources (e.g., memory and disk), and a certain amount of network resources (e.g., network interface card, etc.). Therefore, each bucket in a cloud service system possesses a large number of storage resources, providing sufficient storage capacity to provide cloud storage services to users.

[0048] Furthermore, the storage nodes occupied by multiple storage buckets in a cloud service system can be cloud instances in the infrastructure. These cloud instances can be presented in various ways. For example, these cloud instances can be physical servers selected by the cloud management platform, bare metal servers selected by the cloud management platform, virtual machines (VMs) created by the cloud management platform on physical servers using virtualization technology, containers (Docker) created by the cloud management platform on physical servers using virtualization technology, micro virtual machines (microVMs) created by the cloud management platform on physical servers using virtualization technology, and so on.

[0049] Furthermore, regardless of how the storage nodes in the multiple buckets of the cloud service system are presented, they all possess a certain amount of physical storage resources (for example, when a storage node is a physical server, these physical storage resources can be the memory, hard disk, block device, and encapsulated persistent objects that can be read and written, etc.). Therefore, each storage node can use its storage resources to store a certain amount of data, thereby providing remote data storage services to users. It is worth noting that the storage resources of a storage node can be divided into multiple storage areas. Each storage area serves as a medium for storing data. These media have the following characteristics: data can only be appended to these media, and cannot be modified while writing to them; that is, they do not support write-in-place, only append.

[0050] Furthermore, for multiple storage buckets in a cloud service system, these multiple storage buckets can be deployed in different regions. Multiple storage nodes in the same storage bucket can be deployed in the same site or different sites. Sites can be presented in various forms, such as availability zones in the infrastructure, data centers (DCs) in the infrastructure, rooms in the infrastructure, racks in the infrastructure, and so on.

[0051] To further understand the object storage service (OBS) provided by the aforementioned cloud service system, the following section provides an illustrative example using a specific application case. Figure 2 As shown ( Figure 2 (This is a schematic diagram illustrating an application example of the cloud service system provided in this application embodiment.) In this application scenario, the cloud service system can be presented as an object storage system, and the cloud management platform can be presented as a platform containing an object storage service layer. Multiple storage buckets in the infrastructure are... Figure 2 The bucket shown can be represented by individual storage nodes, which can be seen as object storage devices (OSDs). The physical storage resources occupied by an OSD can be represented as one or more physical disks, and the storage areas within these physical storage resources can be represented as persistent storage units (PLOGs). The following is a brief introduction to the basic concepts involved:

[0052] (1) OBS: OBS is an object-based storage service with advantages such as massive capacity, security, high reliability, and low cost. OBS is an internet-accessible service. Users can establish a connection with the object storage service layer through a client, create buckets in the storage nodes managed by the object storage service layer, and then access and manage the objects in the buckets. The object storage service layer uses a sequential distribution method to store objects in buckets by default. The sequential distribution method can also be called lexicographical distribution or range distribution.

[0053] (2) Object: An object is the basic unit of data storage in an object storage system. An object is actually a collection of a file's data and its related attribute information (metadata). Data uploaded by users to OBS is stored in buckets in the form of objects. An object consists of three parts: key, metadata, and data. The key, i.e., the object name, is, for example, a character sequence with a length greater than 0 and not exceeding 1024 characters after UTF-8 encoding. Each object in a bucket has a unique object key.

[0054] (3) Bucket: A bucket is a container for storing objects in OBS. Object storage provides a flat storage method based on buckets and objects. All objects in a bucket are at the same logical level, eliminating the multi-level tree directory structure in the file system. Each bucket has its own storage category, access permissions, region, and other attributes. Users can create buckets with different storage categories and access permissions and configure more advanced attributes to meet the storage needs of different scenarios.

[0055] (4) OSD: OSD is the basic storage unit of the object storage system. It is set on the physical disk and is a fixed-size storage space of the physical disk. The object storage system manages the physical disks of multiple physical servers in the form of OSD.

[0056] exist Figure 2 In the object storage system shown, users can log in to the object storage service layer through their clients. Within the service layer, they can select and purchase object storage services, such as creating buckets and configuring bucket names. After detecting a user's action (e.g., creating a bucket), the service layer can issue a creation command to the object storage device. This command includes information such as the bucket name and domain name, and is used to notify the storage device to create the bucket and save the bucket name and domain name information. After the bucket is created, users can access the bucket domain name through their object storage client, locate the bucket on the storage device, and upload, download, and delete objects from the bucket.

[0057] The service layer includes a control unit and an operating system. The control unit runs on the operating system, which includes disk drivers and physical network card drivers. The control unit controls the disk controller through the disk drivers to set up the physical disk as multiple persistent storage units (PLOG).

[0058] The infrastructure for running object storage services can be set up across multiple data centers in different regions. For example, object storage devices are set up in multiple data centers in different regions, and each data center includes multiple object storage devices. Each object storage device includes at least one physical disk. For example, the disk controller sets physical disk 11 and physical disk 12 in Region 1 as 4 PLOGs.

[0059] After receiving the creation instruction for bucket 1, the service layer notifies the control unit to create bucket 1. The control unit creates bucket 1 on physical disk 1 and physical disk 2 in the object storage device through the operating system. The PLOG of bucket 1 is distributed on physical disk 1 and physical disk 2, and information such as bucket name and bucket domain name of bucket 1 is stored.

[0060] When a user needs to retrieve (i.e. upload) multiple objects to bucket 1 via the client, such as retrieving target data, the user can call the data storage interface provided by the object storage service layer through the client. This will cause the object storage service layer to trigger the object storage device to write the user's target data into bucket 1. For example, the target data can be written to bucket 1 by the control unit in the object storage service layer.

[0061] When a user needs to retrieve (i.e. download) target data from bucket 1 through the client, the user can call the data retrieval interface provided by the object storage service layer through the client. This will cause the object storage service layer to trigger the object storage device to read the target data from bucket 1 and send it back to the user's client for the user to use.

[0062] In addition, the object storage service layer also provides users with data processing interfaces such as data deletion, data merging, and data modification interfaces, which will not be elaborated on here.

[0063] Based on the aforementioned cloud service system, when a user has data processing needs, the user can send a data processing request and their client parameters to the data processing interface provided by the cloud management platform through their client. The data processing request indicates the target data stored in multiple objects across multiple storage buckets, and the client parameters indicate the performance of the user's client. Then, based on the data processing request and the user's client parameters, the cloud management platform can select at least two storage buckets from the multiple storage buckets. Subsequently, the cloud management platform can instruct the user's client to retrieve the target data from these at least two storage buckets, enabling the user's client to process the target data and obtain the processing result, thus satisfying the user's data processing needs. In the aforementioned process, since each of the multiple storage buckets stores multiple user objects, when a user needs to process target data from these multiple objects, the cloud management platform can select at least two accessible storage buckets for the user's client and notify the client to retrieve the target data from these two buckets to complete the processing. In this way, these at least two storage buckets can share the burden of providing target data to the user's clients. That is, these at least two storage buckets as a whole have sufficient performance; even if a large number of clients on the user side need to retrieve target data, these two storage buckets can provide sufficient download bandwidth for that target data, thereby meeting the data retrieval and processing needs of these clients and improving the user experience. To further understand the workflow of the cloud management platform described above, the following section combines... Figure 3 This workflow will be described in further detail. Figure 3 A flowchart illustrating a data processing method based on a cloud management platform provided in an embodiment of this application is shown below. Figure 3As shown, this method can be achieved through, as Figure 1 The illustrated cloud service system implementation includes infrastructure providing object storage services to users and a cloud management platform for managing this infrastructure. This infrastructure includes multiple storage buckets located in different regions, each of which stores multiple objects belonging to the user. Each of these objects may contain at least one piece of data. The method includes:

[0064] 301. The cloud management platform receives data processing requests sent by the user's client, wherein the data acquisition request is used to indicate the target data to be processed among multiple objects.

[0065] In this embodiment, when a user needs to process target data from multiple objects, the cloud management platform can provide a data processing interface to the user's client (e.g., a data processing bar in the user interface). The user can then input their configured data processing request into the data processing interface through their client. In this way, the cloud management platform can receive the data processing request sent by the user's client through the data processing interface, where the data processing request indicates the target data that the user needs to process.

[0066] For example, such as Figure 4 As shown ( Figure 4 (This is a schematic diagram illustrating an application example of the data processing method provided in this application embodiment.) Assume that storage bucket 1 in region 1, storage bucket 2 in region 2, ..., and storage bucket n in region n all store multiple objects belonging to the user. When the user needs to process target data from these multiple objects, the user can log in to the cloud management platform through their client. The cloud management platform provides a user interface for the user's client, which includes a data processing bar. Therefore, the user can input a data processing request for the target data through their client in the data processing bar, so that the cloud management platform can receive the data processing request sent by the user's client through the data processing bar.

[0067] 302. The cloud management platform receives parameters sent by the client, where the client parameters are used to indicate the client's performance.

[0068] In addition, users can input their client parameters into the data processing interface through their client. Therefore, the cloud management platform can receive these client parameters via the data processing interface. These client parameters can be used to indicate client performance. It should be noted that users can provide data processing requests and their client parameters to the cloud management platform synchronously or asynchronously via their client; there is no restriction on this.

[0069] Specifically, the parameters of the user's client may include at least one of the following: (1) the throughput of the user's client; (2) the time required for the user's client to obtain (receive) data from multiple storage buckets, which may also be referred to as the time required for the user's client to access multiple storage buckets; (3) the time required for the user's client to process the data after obtaining it from multiple storage buckets, for example, the time required for the user's client to write the data to its own memory after receiving the data, or the time required for the user's client to modify the data and perform other operations after receiving the data, etc.; (4) the waiting time required for the user's client to obtain data from multiple storage buckets, etc.

[0070] Continuing with the example above, users can also input client parameters into the data processing field through their client. This allows the cloud management platform to receive these parameters from the user's client via the data processing field. These parameters can include the client's throughput, the time required for the client to receive data from bucket 1, bucket 2, ..., bucket n respectively, the time required for the client to process the data after receiving it from bucket 1, bucket 2, ..., bucket n, and the waiting time required for the client to receive data from bucket 1, bucket 2, ..., bucket n, etc. As can be seen, client parameters can be used to describe the client's performance.

[0071] 303. The cloud management platform selects at least two storage buckets from multiple storage buckets based on data processing requests and client parameters.

[0072] After receiving the data processing request from the user and the parameters from the user's client, the cloud management platform can refer to the data processing request and the parameters from the user's client to select at least two storage buckets from multiple storage buckets to provide the target data to the tenant.

[0073] Specifically, the cloud management platform can select these at least two storage buckets in the following ways:

[0074] After receiving the data processing request from the user and the parameters from the user's client, the cloud management platform can also obtain the parameters of multiple storage buckets (the parameters of multiple storage buckets can be either stored in advance in the cloud management platform or provided to the cloud management platform in real time by multiple storage buckets, without any restrictions here). The parameters of multiple storage buckets are used to indicate the performance of multiple storage buckets.

[0075] Based on this, the cloud management platform can evaluate multiple storage buckets using a specific evaluation algorithm, taking into account data processing requests, user client parameters, and parameters from multiple storage buckets, thereby obtaining evaluation values ​​for each bucket. It should be noted that for any given bucket, a higher evaluation value indicates better performance and a greater ability to provide the target data to the user's client; conversely, a lower evaluation value indicates poorer performance and a less effective ability to provide the target data to the user's client.

[0076] After obtaining the evaluation values ​​of multiple storage buckets, the cloud management platform can select at least two storage buckets whose evaluation values ​​are greater than a preset value (the size of the preset value can be set according to actual needs and is not limited here) to provide the target data to the user.

[0077] More specifically, the parameters of multiple storage buckets include at least one of the following: (1) the available storage capacity of the multiple storage buckets, which can be calculated by the cloud management platform based on the maximum storage capacity and the used storage capacity of the multiple storage buckets. It should be noted that the available storage capacity of multiple storage buckets can also be referred to as the water level of the multiple storage buckets; (2) the throughput of the multiple storage buckets, etc.

[0078] Continuing with the example above, after receiving the data processing request and the user's client parameters, the cloud management platform can also receive parameters provided in real time by storage bucket 1, storage bucket 2, ..., and storage bucket n. Therefore, the cloud management platform can use the data processing request, the user's client parameters, and the parameters of storage bucket 1 (including throughput and water level, etc.), storage bucket 2 (including throughput and water level, etc.), ..., and storage bucket n (including throughput and water level, etc.) to evaluate storage buckets 1, 2, ..., and n, thereby obtaining the evaluation values ​​for storage bucket 1, 2, ..., and n. Since the evaluation values ​​of storage bucket 1, 2, and 3 are greater than preset values, the cloud management platform can determine that storage buckets 1, 2, and 3 are the storage buckets that provide the target data to the user's client.

[0079] 304. The cloud management platform notifies the client to retrieve the target data from at least two storage buckets, processes the target data, and obtains the processing result of the target data.

[0080] Once these at least two storage buckets are identified, the cloud management platform can notify the user's client to access these at least two storage buckets to obtain and process the target data, thereby obtaining the processing results and meeting the user's data processing needs.

[0081] Specifically, the cloud management platform can obtain target data in the following ways:

[0082] Since these at least two storage buckets can contain two storage buckets or more storage buckets (e.g., three, four, etc.), for ease of explanation, the following description will focus on two of these at least two storage buckets, and will refer to these two storage buckets as the first storage bucket and the second storage bucket, respectively.

[0083] In identifying at least two storage buckets, since these at least two storage buckets include a first storage bucket and a second storage bucket, the cloud management platform can determine the first sub-data of the target data that the first storage bucket (for the user's client) needs to provide based on the evaluation value of the first storage bucket. Similarly, the cloud management platform can also determine the second sub-data of the target data that the second storage bucket needs to provide based on the evaluation value of the second storage bucket. It should be noted that the first sub-data and the second sub-data are different sub-data in the target data (i.e., the first sub-data is one part of the target data, and the second sub-data is another part of the target data), and the order of the first sub-data in the target data is before the order of the second sub-data in the target data (the first sub-data and the second sub-data can be adjacent or not, which is not restricted here). The size of the first sub-data is usually related to the evaluation value of the first storage bucket (for example, the larger the evaluation value of the first storage bucket, the larger the first sub-data, and the smaller the evaluation value of the first storage bucket, the smaller the first sub-data). The size of the second sub-data is usually related to the evaluation value of the second storage bucket. If these at least two buckets also contain other buckets besides the first and second buckets (e.g., the third and fourth buckets, etc., which will not be elaborated here), the cloud management platform can also perform similar operations on the other buckets to determine the remaining sub-data of the target data provided by the other buckets. The second sub-data is ordered in the target data before the remaining sub-data is ordered in the target data, and the size of the remaining sub-data is usually associated with the evaluation value of the other buckets.

[0084] Then, the cloud management platform can instruct the client to retrieve the first sub-data from the first storage bucket and the second sub-data from the second storage bucket. If these at least two storage buckets contain only the first and second storage buckets, then the user's client, after obtaining the first and second sub-data, has essentially obtained the target data. If these at least two storage buckets contain not only the first and second storage buckets but also other storage buckets, the cloud management platform can also instruct the user's client to retrieve the remaining sub-data from the other storage buckets. Therefore, the user's client will ultimately obtain the first sub-data, the second sub-data, and the remaining sub-data, which is equivalent to obtaining the target data.

[0085] As in the example above, such as Figure 5 As shown ( Figure 5 (This is another schematic diagram illustrating an application example of the data processing method provided in this application embodiment). After determining storage buckets 1 to 3, since the evaluation value of storage bucket 1 (i.e., the aforementioned first storage bucket) is 20, the evaluation value of storage bucket 2 (i.e., the aforementioned second storage bucket) is 30, and the evaluation value of storage bucket 3 (i.e., the aforementioned remaining storage buckets) is 50, the cloud management platform can determine that storage bucket 1 needs to provide the first 20% of the target data, which is sub-data 1 (i.e., the aforementioned first sub-data), determine that storage bucket 2 needs to provide the middle 30% of the target data, which is sub-data 2 (i.e., the aforementioned second sub-data), and determine that storage bucket 3 needs to provide the last 50% of the target data, which is sub-data 3 (i.e., the aforementioned remaining sub-data).

[0086] Then, the cloud management platform can instruct the user's client to retrieve sub-data 1 from storage bucket 1, then sub-data 2 from storage bucket 2, and finally sub-data 3 from storage bucket 3. In this way, the user's client can successfully retrieve sub-data 1, sub-data 2, and sub-data 3, which is the target data.

[0087] More specifically, the cloud management platform can also perform the following operations:

[0088] After determining the first and second sub-data, the cloud management platform instructs the user's client to partition the pre-configured cache in the client, thereby obtaining a first sub-cache and a second sub-cache. The first sub-cache is ordered before the second sub-cache within the cache. The size of the first sub-cache typically matches the size of the first sub-data, and the size of the second sub-cache also matches the size of the second sub-data. If these at least two buckets contain not only the first and second buckets but also other buckets, the cloud management platform can determine not only the first and second sub-data but also the remaining sub-data. Therefore, after the user's client partitions the cache, it will obtain not only the first and second sub-caches but also the remaining sub-caches. The second sub-cache is ordered before the remaining sub-caches within the cache, and the size of the remaining sub-caches typically matches the size of the remaining sub-data.

[0089] After the cloud management platform notifies the user's client to retrieve the first piece of data from the first storage bucket, the user's client can store the first piece of data in the first sub-cache. Similarly, after the user's client retrieves the second piece of data from the second storage bucket, the user's client can store the second piece of data in the second sub-cache. If these at least two storage buckets contain not only the first and second storage buckets but also other storage buckets, the client can also retrieve the remaining piece of data from the other storage buckets and store the remaining piece of data in the remaining sub-caches.

[0090] Then, the user's client can read the first sub-data from the first sub-cache and the second sub-data from the second sub-cache. If these at least two buckets contain only the first and second buckets, then the user's client, after obtaining the first and second sub-data, has essentially obtained the target data and can process the target data to obtain the processing result. If these at least two buckets contain not only the first and second buckets but also other buckets, the user's client can also read the remaining sub-data from the other sub-caches, ultimately obtaining the first, second, and remaining sub-data, which is equivalent to obtaining the target data. The client can then process the target data to obtain the processing result.

[0091] As in the example above, such as Figure 6 As shown ( Figure 6 (This is another schematic diagram illustrating an application example of the data processing method provided in this application embodiment). After determining sub-data 1, sub-data 2, and sub-data 3, the cloud management platform can also notify the user's client to divide the local cache, thereby obtaining a chained sub-cache 1, sub-cache 2, and sub-cache 3 (i.e., sub-cache 1 (i.e., the aforementioned first sub-cache), sub-cache 2 (i.e., the aforementioned second sub-cache), and sub-cache 3 (i.e., the aforementioned remaining sub-caches) can be consecutive caches). The size of sub-cache 1 matches the size of sub-data 1 (i.e., sub-cache 1 can store sub-data 1), the size of sub-cache 2 matches the size of sub-data 2, and the size of sub-cache 3 matches the size of sub-data 3.

[0092] Then, the cloud management platform can notify the user's client to retrieve sub-data 1 from storage bucket 1 and store sub-data 1 in sub-cache 1, then retrieve sub-data 2 from storage bucket 2 and store sub-data 2 in sub-cache 2, and then retrieve sub-data 3 from storage bucket 3 and store sub-data 3 in sub-cache 3.

[0093] Subsequently, the user's client can continuously read sub-cache 1, sub-cache 2, and sub-cache 3 to successfully read sub-data 1, sub-data 2, and sub-data 3, which is the target data, and process the target data to obtain the processing result of the target data.

[0094] In this embodiment, when a user has a data processing need, the user can send a data processing request and client parameters to the data processing interface provided by the cloud management platform through their client. The data processing request indicates the target data stored by the user in multiple objects across multiple storage buckets, and the client parameters indicate the performance of the user's client. Then, based on the data processing request and the user's client parameters, the cloud management platform can select at least two storage buckets from the multiple storage buckets. Subsequently, the cloud management platform can notify the user's client to retrieve the target data from these at least two storage buckets, enabling the user's client to process the target data and obtain the processing result, thus satisfying the user's data processing needs. In the aforementioned process, since each of the multiple storage buckets stores multiple objects belonging to the user, when the user needs to process target data from these multiple objects, the cloud management platform can select at least two storage buckets accessible to the user's client and notify the user's client to retrieve the target data from these at least two storage buckets to complete the target data processing. In this way, these at least two storage buckets can share the pressure of providing target data to the user's client. That is to say, these at least two storage buckets as a whole can have sufficient performance. Even if there are a large number of clients on the user side that need to retrieve target data, these two storage buckets can provide sufficient download bandwidth for the target data for a large number of clients on the user side, thereby meeting the data retrieval and processing needs of these clients and improving the user experience.

[0095] Furthermore, in this embodiment, even if any one of the multiple storage buckets storing multiple objects of the user fails, the remaining storage buckets can still serve as backups to provide target data to the user's client, thereby meeting the user's data processing needs, i.e., meeting the user's business needs, and further improving the user experience.

[0096] The above is a detailed description of the data processing method based on the cloud management platform provided in the embodiments of this application. The cloud management platform provided in the embodiments of this application will be introduced below. Figure 7 A schematic diagram of the structure of the cloud management platform provided in the embodiments of this application is shown below. Figure 7 As shown, the cloud management platform is used to manage the infrastructure that provides object storage services. The infrastructure comprises multiple buckets located in different regions, each bucket storing multiple objects belonging to the user. The cloud management platform includes:

[0097] The receiving module 701 is used to receive data processing requests sent by the user's client, wherein the data acquisition request is used to indicate the target data to be processed among multiple objects;

[0098] The receiving module 701 is also used to receive parameters sent by the client, wherein the client parameters are used to indicate the performance of the client;

[0099] Selection module 702 is used to select at least two storage buckets from multiple storage buckets based on data processing requests and client parameters;

[0100] The first notification module 703 is used to notify the client to obtain target data from at least two storage buckets, process the target data, and obtain the processing result of the target data.

[0101] In one possible implementation, the selection module 702 is configured to: obtain parameters of multiple storage buckets, wherein the parameters of the multiple storage buckets are used to indicate the performance of the multiple storage buckets; obtain evaluation values ​​of the multiple storage buckets based on the data processing request, the client parameters, and the parameters of the multiple storage buckets; and select at least two storage buckets from the multiple storage buckets whose evaluation values ​​are greater than a preset value.

[0102] In one possible implementation, at least two storage buckets include a first storage bucket and a second storage bucket. The cloud management platform further includes: a first determining module, used to determine a first sub-data of the target data that the first storage bucket needs to provide based on the evaluation value of the first storage bucket; a second determining module, used to determine a second sub-data of the target data that the second storage bucket needs to provide based on the evaluation value of the second storage bucket, wherein the first sub-data and the second sub-data are different sub-data in the target data, and the order of the first sub-data in the target data is before the order of the second sub-data in the target data; and a first notification module 703, used to notify the client to obtain the first sub-data from the first storage bucket and then obtain the second sub-data from the second storage bucket.

[0103] In one possible implementation, the cloud management platform further includes: a second notification module for notifying the client to divide the pre-set cache into a first sub-cache and a second sub-cache, wherein the first sub-cache is ordered before the second sub-cache in the cache; a third notification module for notifying the client to store the first sub-data in the first sub-cache and the second data in the second sub-cache; and a first notification module 703 for notifying the client to read the first sub-data from the first sub-cache and then read the second sub-data from the second sub-cache to obtain the target data, and to process the target data to obtain the processing result of the target data.

[0104] In one possible implementation, the size of the first sub-cache is matched with the size of the first sub-data, and the size of the first sub-data is matched with the evaluation value of the first bucket.

[0105] In one possible implementation, the client's parameters include at least one of the following: the client's throughput, the time required for the client to retrieve data from multiple buckets, the time required for the client to process the data, and the time required for the client to wait before retrieving data from multiple buckets.

[0106] In one possible implementation, the parameters of the multiple buckets include at least one of the following: the available storage capacity of the multiple buckets and the throughput of the multiple buckets, wherein the available storage capacity of the multiple buckets is determined based on the maximum storage capacity of the multiple buckets and the used storage capacity of the multiple buckets.

[0107] It should be noted that the information interaction and implementation process between the modules / units of the above-mentioned device are based on the same concept as the method embodiments of this application, and the resulting technical effects are the same as those of the method embodiments of this application. For details, please refer to the description in the method embodiments shown above in the embodiments of this application, and will not be repeated here.

[0108] Please see Figure 8 , Figure 8 This is a schematic diagram of the structure of a computing device provided in an embodiment of this application. Figure 8 As shown, the computing device 800 (which can be used to present the aforementioned cloud management platform) includes: a processor 801, a memory 802, a communication interface 803, and a bus 804. The processor 801, memory 802, and communication interface 803 are coupled via the bus (not shown in the figure). The memory 802 stores instructions. When the instructions in the memory 802 are executed, the computing device 800 executes the method performed by the cloud management platform in the above method embodiment.

[0109] The computing device 800 may be one or more integrated circuits configured to implement the methods described above, such as: one or more application-specific integrated circuits (ASICs), or one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs), or a combination of at least two of these forms of integrated circuits. Furthermore, when the units in the device can be implemented in the form of a processing element scheduler, the processing element may be a general-purpose processor, such as a central processing unit (CPU) or other processor capable of calling programs. Alternatively, these units may be integrated together to implement a system-on-a-chip (SOC).

[0110] The processor 801 can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. A general-purpose processor can be a microprocessor or any conventional processor.

[0111] The memory 802 can be volatile memory or non-volatile memory, or it can include both. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

[0112] The memory 802 stores executable program code, and the processor 801 executes this executable program code to implement the functions of the aforementioned receiving module, selection module, and first notification module, thereby realizing the aforementioned data processing method based on the cloud service system. That is, the memory 802 stores instructions for executing the aforementioned data processing method based on the cloud service system.

[0113] The communication interface 803 uses transceiver modules, such as, but not limited to, network interface cards and transceivers, to enable communication between the computing device 800 and other devices or communication networks.

[0114] In addition to the data bus, the 804 bus can also include a power bus, a control bus, and a status signal bus. The bus can be a Peripheral Component Interconnect Express (PCIe) bus, an Extended Industry Standard Architecture (EISA) bus, a Unified Bus (Ubus or UB), a Compute Express Link (CXL) bus, a Cache Coherent Interconnect for Accelerators (CCIX) bus, etc. The bus can be divided into address bus, data bus, and control bus.

[0115] Please see Figure 9 , Figure 9 This is a schematic diagram of a computing device cluster provided in an embodiment of this application. Figure 9 As shown, the computing device cluster 900 includes at least one computing device 800.

[0116] like Figure 9 As shown, the computing device cluster 900 includes at least one computing device 800. The memory 802 of one or more computing devices 800 in the computing device cluster 900 may store the same instructions for executing the data processing method described above based on the cloud service system.

[0117] In some possible implementations, the memory 802 of one or more computing devices 800 in the computing device cluster 900 may also store partial instructions for executing the data processing method based on the cloud service system described above. In other words, a combination of one or more computing devices 800 can jointly execute the data processing method based on the cloud service system described above.

[0118] It should be noted that the memory 802 in different computing devices 800 within the computing device cluster 900 can store different instructions, each used to execute a portion of the functions of the aforementioned cloud management platform. That is, the instructions stored in the memory 802 of different computing devices 800 can implement the functions of one or more modules, such as the receiving module, the selection module, and the first notification module.

[0119] In some possible implementations, one or more computing devices 800 in the computing device cluster 900 can be connected via a network. This network can be a wide area network (WAN) or a local area network (LAN), etc.

[0120] Please see Figure 10 , Figure 10 This is a schematic diagram illustrating the network connection of computer devices in a computer cluster provided in an embodiment of this application. Figure 10 As shown, the two computing devices 800A and 800B are connected via a network. Specifically, they are connected to the network through the communication interfaces in each computing device.

[0121] In one possible implementation, the memory in computing device 800A stores instructions for performing the functions of modules such as the receiving module. Meanwhile, the memory in computing device 800B stores instructions for performing the functions of modules such as the selection module and the first notification module.

[0122] It should be understood that Figure 10 The functions of computing device 800A shown can also be performed by multiple computing devices. Similarly, the functions of computing device 800B can also be performed by multiple computing devices.

[0123] This application also relates to a computer storage medium storing a program for signal processing, which, when run on a computer, causes the computer to perform actions such as... Figure 3 The steps performed by the cloud management platform in the illustrated embodiment.

[0124] This application also relates to a computer program product that stores instructions that, when executed by a computer, cause the computer to perform actions such as... Figure 3 The steps performed by the cloud management platform in the illustrated embodiment.

[0125] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0126] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between apparatuses or units through some interfaces, and may be electrical, mechanical, or other forms.

[0127] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0128] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0129] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A data processing method based on a cloud management platform, characterized in that, The cloud management platform is used to manage the infrastructure that provides object storage services. The infrastructure includes multiple storage buckets located in different regions, each of which stores multiple objects belonging to a user. The method includes: The cloud management platform receives a data processing request sent by the user's client, wherein the data acquisition request is used to indicate the target data to be processed among the plurality of objects; The cloud management platform receives parameters from the client sent by the client, wherein the client's parameters are used to indicate the client's performance; The cloud management platform selects at least two storage buckets from the plurality of storage buckets based on the data processing request and the parameters of the client. The cloud management platform notifies the client to retrieve the target data from the at least two storage buckets, processes the target data, and obtains the processing result of the target data.

2. The method according to claim 1, characterized in that, Based on the data processing request and the client's parameters, the cloud management platform selects at least two storage buckets from the plurality of storage buckets, including: The cloud management platform obtains parameters of the multiple storage buckets, wherein the parameters of the multiple storage buckets are used to indicate the performance of the multiple storage buckets; The cloud management platform obtains the evaluation values ​​of the multiple storage buckets based on the data processing request, the client's parameters, and the parameters of the multiple storage buckets; The cloud management platform selects at least two storage buckets from the plurality of storage buckets whose evaluation values ​​are greater than preset values.

3. The method according to claim 2, characterized in that, The at least two storage buckets include a first storage bucket and a second storage bucket, and the method further includes: Based on the evaluation value of the first storage bucket, the cloud management platform determines the first sub-data of the target data that the first storage bucket needs to provide; Based on the evaluation value of the second storage bucket, the cloud management platform determines the second sub-data of the target data that the second storage bucket needs to provide, wherein the first sub-data and the second sub-data are different sub-data in the target data, and the first sub-data is ordered before the second sub-data in the target data. The cloud management platform notifies the client to retrieve the target data from the at least two storage buckets, including: The cloud management platform notifies the client to retrieve the first sub-data from the first storage bucket and then retrieve the second sub-data from the second storage bucket.

4. The method according to claim 3, characterized in that, The method further includes: The cloud management platform notifies the client to divide the preset cache into a first sub-cache and a second sub-cache, wherein the first sub-cache is ordered before the second sub-cache in the cache. The cloud management platform notifies the client to store the first sub-data in the first sub-cache and to store the second data in the second sub-cache; The client processes the target data to obtain the processing result of the target data, including: The client reads the first sub-data from the first sub-cache and then reads the second sub-data from the second sub-cache to obtain the target data, and processes the target data to obtain the processing result of the target data.

5. The method according to claim 4, characterized in that, The size of the first sub-cache matches the size of the first sub-data, and the size of the first sub-data matches the evaluation value of the first storage bucket.

6. The method according to any one of claims 1 to 5, characterized in that, The parameters of the client include at least one of the following: the throughput of the client, the time required for the client to obtain data from the plurality of storage buckets, the time required for the client to process data, and the time required for the client to wait before obtaining data from the plurality of storage buckets.

7. The method according to any one of claims 1 to 6, characterized in that, The parameters of the plurality of storage buckets include at least one of the following: the available storage capacity of the plurality of storage buckets and the throughput of the plurality of storage buckets, wherein the available storage capacity of the plurality of storage buckets is determined based on the maximum storage capacity of the plurality of storage buckets and the used storage capacity of the plurality of storage buckets.

8. A cloud management platform, characterized in that, The cloud management platform is used to manage the infrastructure that provides object storage services. This infrastructure includes multiple storage buckets located in different regions, and each of these buckets stores multiple objects belonging to the user. A receiving module is configured to receive a data processing request sent by the user's client, wherein the data acquisition request is used to indicate the target data to be processed among the plurality of objects; The receiving module is further configured to receive parameters sent by the client, wherein the parameters are used to indicate the performance of the client; The selection module is used to select at least two storage buckets from the plurality of storage buckets based on the data processing request and the parameters of the client; The first notification module is used to notify the client to obtain the target data from the at least two storage buckets, process the target data, and obtain the processing result of the target data.

9. The cloud management platform according to claim 8, characterized in that, The selection module is used for: Obtain the parameters of the plurality of storage buckets, wherein the parameters of the plurality of storage buckets are used to indicate the performance of the plurality of storage buckets; Based on the data processing request, the client's parameters, and the parameters of the multiple storage buckets, obtain the evaluation values ​​of the multiple storage buckets; From the plurality of storage buckets, select at least two storage buckets whose evaluation value is greater than a preset value.

10. The cloud management platform according to claim 9, characterized in that, The at least two storage buckets include a first storage bucket and a second storage bucket, and the cloud management platform further includes: The first determining module is used to determine, based on the evaluation value of the first storage bucket, the first sub-data of the target data that the first storage bucket needs to provide; The second determining module is used to determine, based on the evaluation value of the second storage bucket, the second sub-data of the target data that the second storage bucket needs to provide, wherein the first sub-data and the second sub-data are different sub-data in the target data, and the first sub-data is ordered before the second sub-data in the target data. The first notification module is used to notify the client to retrieve the first sub-data from the first storage bucket and then retrieve the second sub-data from the second storage bucket.

11. The cloud management platform according to claim 10, characterized in that, The cloud management platform also includes: The second notification module is used to notify the client to divide the preset cache into a first sub-cache and a second sub-cache, wherein the first sub-cache is ordered before the second sub-cache in the cache. The third notification module is used to notify the client to store the first sub-data in the first sub-cache and to store the second data in the second sub-cache; The first notification module is used to notify the client to read the first sub-data from the first sub-cached cache, and then read the second sub-data from the second sub-cached cache to obtain the target data, and process the target data to obtain the processing result of the target data.

12. The cloud management platform according to claim 11, characterized in that, The size of the first sub-cache matches the size of the first sub-data, and the size of the first sub-data matches the evaluation value of the first storage bucket.

13. The cloud management platform according to any one of claims 8 to 12, characterized in that, The parameters of the client include at least one of the following: the throughput of the client, the time required for the client to obtain data from the plurality of storage buckets, the time required for the client to process data, and the time required for the client to wait before obtaining data from the plurality of storage buckets.

14. The cloud management platform according to any one of claims 8 to 13, characterized in that, The parameters of the plurality of storage buckets include at least one of the following: the available storage capacity of the plurality of storage buckets and the throughput of the plurality of storage buckets, wherein the available storage capacity of the plurality of storage buckets is determined based on the maximum storage capacity of the plurality of storage buckets and the used storage capacity of the plurality of storage buckets.

15. A computing device cluster, characterized in that, The computing device cluster includes at least one computing device, each computing device including a processor and memory: The memory is used to store instructions; The processor is configured to, according to the instructions, cause the computing device cluster to perform the method of any one of claims 1 to 7.

16. A computer storage medium, characterized in that, The computer storage medium stores one or more instructions that, when executed by one or more computers, cause the one or more computers to perform the method of any one of claims 1 to 7.

17. A computer program product, characterized in that, The computer program product stores instructions that, when executed by a computer, cause the computer to perform the method described in any one of claims 1 to 7.

Images