Task processing method running on cloud computing platform, and apparatus and device

Abstract

The present application relates to the technical field of cloud computing. Disclosed are a task processing method running on a cloud computing platform, and an apparatus and a device. Data of a target object can be processed by a service pod side or an intelligent hierarchical cluster, such that the situation where only one cluster or system processes a large amount of data in a unified manner can be avoided, thereby improving the effectiveness of data processing. The method comprises: first, a cloud computing platform acquiring task information of a task to be processed and processing capability information of a service pod on the basis of a task configuration operation performed by a user on a configuration interface; then, on the basis of the task information of said task and the processing capability information of the service pod, the cloud computing platform assigning said task to a target computing device for processing, wherein the target computing device is a service pod or an intelligent hierarchical cluster; and finally, the intelligent hierarchical cluster generating an access data table of a target object on the basis of a processing result of said task, wherein the access data table is used for managing a storage area of the target object.

Description

A task processing method, device and equipment running on a cloud computing platform

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on August 15, 2023, with application number 202311027148.X and application name “A method, device and other equipment for data distribution”, and the Chinese patent application filed with the State Intellectual Property Office on January 18, 2024, with application number 202410077113.5 and application name “A task processing method, device and equipment running on a cloud computing platform”, all of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of cloud computing technology, and in particular to a task processing method, apparatus, and device running on a cloud computing platform. Background Art

[0003] In data storage systems, different storage media correspond to different performance characteristics. Data reports can be provided to users to help them choose the media in which to store their data. For example, users can choose to store hot data on high-performance media and cold data on lower-performance media. Furthermore, users can request services based on these data reports. For example, if the data volume of certain data in the report is large, additional equipment or resources can be requested to provide storage services for this data.

[0004] Currently, an agent can be deployed in each business container group (pod). The agent sends the access data corresponding to each business pod to the intelligent tiering cluster, which then centrally processes the access data reported by all business pods. After the intelligent tiering cluster completes processing of all access data, a series of data reports are generated.

[0005] However, in the above method, due to the large number of business pods, the scale of access data also increases. If only one intelligent hierarchical cluster handles a large amount of access data, the access data will not be processed in a timely manner, affecting the effectiveness of data processing.

[0006] Summary of the Invention

[0007] The present application provides a task processing method, apparatus and device running on a cloud computing platform, which solves the technical problem that a large amount of access data can not be processed in time due to only one intelligent hierarchical cluster processing the access data.

[0008] In a first aspect, a task processing method running on a cloud computing platform is provided, wherein the cloud computing platform is deployed with a container management system, wherein an intelligent hierarchical cluster and one or more business container groups are deployed in the container management system, and the cloud computing platform also provides a configuration interface. The method includes: first, the cloud computing platform obtains task information of the task to be processed and processing capacity information of the business container group according to the task configuration operation of the user in the configuration interface, wherein the task configuration operation indicates intelligent hierarchical processing of data of the target object; then, the cloud computing platform assigns the task to be processed to a target computing device for processing based on the task information of the task to be processed and the processing capacity information of the business container group, wherein the target computing device is a business container group or an intelligent hierarchical cluster; finally, the intelligent hierarchical cluster generates an access data table of the target object based on the processing result of the task to be processed, and the access data table is used to manage the storage area of ​​the target object.

[0009] It is understood that the method of the first aspect can be executed by a computing device, which can be a terminal, an apparatus including a terminal, or a chip within a terminal. Alternatively, the computing device can be a network device, an apparatus including a network device, or a chip within a network device. For ease of understanding, the following description uses execution by a computing device as an example.

[0010] In this application, after the cloud computing platform obtains the task information of the pending task and the processing capacity information of the business container group deployed in the cloud computing platform, it can allocate the pending task to the target computing device for processing based on the task information and the processing capacity information. Since the target computing device is the intelligent hierarchical cluster deployed in the business container group or the cloud computing platform, the data of the target object can be processed by the business pod side or the intelligent hierarchical cluster, which can avoid only one cluster or system processing a large amount of data uniformly, thereby improving the effectiveness of data processing. In addition, the intelligent hierarchical cluster can generate an access data table for the target object based on the processing results of the pending task. Since the access data table is used to manage the storage area of ​​the target object, the user can apply for more storage resources for the target object data based on the access data table or identify hot data based on the access data table and apply to store the hot data in a storage medium with higher performance, which can provide customized service needs for users.

[0011] Optionally, the above-mentioned cloud computing platform allocates the task to be processed to the target computing device for processing based on the task information of the task to be processed and the processing capability information of the business container group, specifically including: the cloud computing platform matches the processing capability information of the business container group with the task information of the task to be processed; the cloud computing platform determines that the processing capability information of the business container group successfully matches the task information of the task to be processed, and allocates the task to be processed to the business container group for processing; the cloud computing platform determines that the processing capability information of the business container group fails to match the task information of the task to be processed, and allocates the task to be processed to the intelligent hierarchical cluster for processing.

[0012] In this application, if the processing capability information of the business container group successfully matches the task information of the pending task, it means that the business container group has the processing capability to process the pending task. At this time, the cloud computing platform will assign the pending task to the business container group for processing, which can reduce the processing pressure of the intelligent hierarchical cluster while processing the pending task. If the processing capability information of the business container group fails to match the task information of the pending task, it means that the business container group does not have the processing capability to process the pending task. At this time, the cloud computing platform will assign the pending task to the intelligent hierarchical cluster for processing, which can ensure the effectiveness of task processing.

[0013] Optionally, the task processing method also includes: the cloud computing platform splits the task to be processed to obtain multiple subtasks to be processed; the cloud computing platform assigns at least one subtask to the business container group for processing based on the processing capability information of the business container group and the task information of the multiple subtasks to be processed, and assigns other subtasks to the intelligent hierarchical cluster for processing.

[0014] In this application, the cloud computing platform can instruct the business container group and the intelligent hierarchical cluster to collaboratively complete the processing of pending tasks. Specifically, the business container group handles at least one subtask, and the intelligent hierarchical cluster handles the remaining subtasks. This can fully utilize the processing capabilities of the intelligent hierarchical cluster and the business pod side, significantly saving the processing costs of the cloud computing platform, avoiding the need for a single cluster (or system) to process all data, and can improve data processing efficiency.

[0015] Optionally, the above-mentioned cloud computing platform is also deployed with a data acquisition system, and the task processing method also includes: the cloud computing platform obtains the acquisition performance information of the data acquisition system; the cloud computing platform instructs the data acquisition system to collect data of the target object from the business container group based on the acquisition performance information of the data acquisition system; the data acquisition system sends the data of the target object to the intelligent hierarchical cluster.

[0016] In this application, the cloud computing platform can instruct the data collection system to proactively collect data for target objects from the business container group based on the data collection system's collection performance information. This effectively controls the order in which data is collected, ensuring that the data collection system does not exceed its own collection capacity. Furthermore, the data collection system can send the target object's data to the intelligent tiering system, which then obtains the processing results of the pending tasks based on the target object's data. This ensures that the data collected by the data collection system does not exceed the maximum processing capacity of the intelligent tiering cluster, reducing the processing pressure on the intelligent tiering cluster.

[0017] Optionally, after the cloud computing platform assigns the pending task to the business container group for processing, the task processing method further includes: the business container group processes the pending task to obtain a processing result of the pending task.

[0018] In this application, the business container group processes pending tasks (or target object data) and obtains the processing results of the pending tasks. This means that the business pod can independently complete the processing of pending tasks, improving data processing efficiency. This also reduces the processing pressure on the intelligent hierarchical cluster and improves its processing capabilities.

[0019] Optionally, after the above-mentioned cloud computing platform assigns the pending tasks to the business container group for processing, the task processing method also includes: the business container group processes the pending tasks to obtain the intermediate results of the pending tasks; the data acquisition system collects the intermediate results of the pending tasks from the business container group, and sends the intermediate results of the pending tasks to the intelligent grading cluster; the intelligent grading cluster obtains the processing results of the pending tasks based on the intermediate results of the pending tasks.

[0020] In this application, the business container group and the intelligent hierarchical cluster jointly complete the processing of pending tasks, making full use of the data coupling characteristics of the business pod side to obtain the intermediate results of the pending tasks in advance. The intelligent hierarchical cluster then obtains the processing result (or final result) of the pending tasks based on the intermediate results. This can significantly reduce the processing scale and complexity of the intelligent hierarchical cluster.

[0021] Optionally, the above-mentioned target computing device includes an operator container, which is deployed in the target computing device in the form of a container image. The above-mentioned cloud computing platform assigns the task to be processed to the target computing device for processing, specifically including: the cloud computing platform assigns the task to be processed to the operator container for processing, and obtains the processing result of the task to be processed.

[0022] In this application, the computing device assigns pending tasks to an operator container in the target computing device, which then processes the pending tasks and obtains the processing results. This allows precise control of the operators in the operator container that participate in the computation, providing processing power that matches the pending tasks. This effectively completes task processing while avoiding insufficient or wasted processing power.

[0023] Optionally, the task information of the above-mentioned task to be processed includes the data volume of the task to be processed, and the task processing method also includes: the cloud computing platform performs an expansion operation or a reduction operation on the operator container included in the target computing device based on the data volume of the task to be processed.

[0024] In this application, when the data volume of the task to be processed is large, the target computing device (specifically, the operator container included in the target computing device) may not be able to meet the processing requirements of the task to be processed. The cloud computing platform can perform a capacity expansion operation on the operator container included in the target computing device to improve the processing capability of the target computing device. Correspondingly, when the data volume of the task to be processed is small, the target computing device (specifically, the operator container included in the target computing device) can easily complete the processing requirements of the task to be processed. The cloud computing platform can perform a capacity reduction operation on the operator container included in the target computing device to save the processing resources of the target computing device.

[0025] In a second aspect, a task processing device running on a cloud computing platform is provided. The cloud computing platform is deployed with a container management system, in which an intelligent hierarchical cluster and one or more business container groups are deployed. The cloud computing platform also provides a configuration interface. The task processing device includes an acquisition module and a first processing module, and the intelligent hierarchical cluster includes a second processing module. The acquisition module is used to obtain task information of the task to be processed and processing capacity information of the business container group based on the task configuration operation performed by the user in the configuration interface, wherein the task configuration operation indicates intelligent hierarchical processing of the data of the target object; the first processing module is used to allocate the task to be processed to a target computing device for processing based on the task information of the task to be processed and the processing capacity information of the business container group; the target computing device is a business container group or an intelligent hierarchical cluster; the second processing module is used to generate an access data table of the target object based on the processing result of the task to be processed, and the access data table is used to manage the storage area of ​​the target object.

[0026] In addition, the technical effects of the task processing device described in the second aspect can refer to the technical effects of the method described in the first aspect, and will not be repeated here.

[0027] In a third aspect, a computing device cluster is provided, comprising at least one computing device, each computing device comprising a processor and a memory; the processor of the at least one computing device is used to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the method described in the first aspect.

[0028] In a fourth aspect, a computer program product comprising instructions is provided. When the instructions are executed by a computing device cluster, the computing device cluster executes the method described in the first aspect.

[0029] In a fifth aspect, a computer-readable storage medium is provided, comprising computer program instructions. When the computer program instructions are executed by a computing device cluster, the computing device cluster executes the method described in the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] FIG1 is a schematic diagram showing the principle of intelligent hierarchical cluster data processing provided by an embodiment of the present application;

[0031] FIG2 shows a schematic diagram of the internal structure of a computing device provided in an embodiment of the present application;

[0032] FIG3 shows a flowchart of a task processing method provided in an embodiment of the present application;

[0033] FIG4 shows a schematic diagram of a configuration interface provided in an embodiment of the present application;

[0034] FIG5 is a flow chart showing another task processing method provided in an embodiment of the present application;

[0035] FIG6 shows a flowchart of another task processing method provided in an embodiment of the present application;

[0036] FIG7 shows a schematic diagram showing a principle of data cleaning for data in a business container group provided by an embodiment of the present application;

[0037] FIG8 shows a schematic diagram of a principle of data conversion and aggregation processing provided by an embodiment of the present application;

[0038] FIG9 is a schematic diagram showing a flow chart of another task processing method provided in an embodiment of the present application;

[0039] FIG10 is a schematic diagram showing a principle of expanding an operator container according to an embodiment of the present application;

[0040] FIG11 shows a schematic diagram of another configuration interface provided in an embodiment of the present application;

[0041] FIG12 is a schematic diagram showing the principle of a data collection system according to an embodiment of the present application collecting data in batches from a business container group;

[0042] FIG13 shows a schematic diagram illustrating a principle of allocating multiple subtasks to an intelligent hierarchical cluster and a business container group for processing according to an embodiment of the present application;

[0043] FIG14 shows a schematic structural diagram of a task processing device provided in an embodiment of the present application;

[0044] FIG15 shows a schematic structural diagram of a computing device provided in an embodiment of the present application;

[0045] FIG16 shows a schematic diagram of the structure of a computing device cluster provided in an embodiment of the present application;

[0046] FIG17 shows a schematic diagram of a network architecture of a computing device cluster provided in an embodiment of the present application. DETAILED DESCRIPTION

[0047] To facilitate understanding of the solutions provided in the embodiments of the present application, some concepts involved in the present application are explained before introducing the solutions provided in the embodiments of the present application.

[0048] Kubernetes, or K8S for short, is an open-source container management system (or container orchestration engine) used to automate the deployment, scaling, and management of containerized applications. In a K8S cluster, a pod is the foundation of all business types and the smallest deployable unit in K8S. It is a combination of one or more containers. A container can be understood as a simplified Linux environment and the applications running within it, with each container isolated from the others. A K8S cluster can include one or more business container groups, each of which contains one or more business containers used to store object data.

[0049] The intelligent hierarchical cluster (also known as the central cluster) serves as the management and control center for the Kubernetes cluster. It consists of master nodes and is responsible for cluster management and scheduling. One or more operator containers can be deployed within this intelligent hierarchical cluster. These containers process a task and generate its results. Furthermore, an operator container can contain one or more operators. An operator is a mapping from one function space to another, serving as the computational unit of an algorithm.

[0050] Object storage is a data storage method that uses objects as units, storing data and its associated metadata as a whole. A "bucket" is a container used to organize and store objects, similar to a folder that can hold multiple objects. Object storage provides a flat storage method based on buckets and objects, eliminating the multi-level tree-like directory structure of traditional file systems, making data storage and management simpler and more efficient.

[0051] In data storage systems, different storage media correspond to different performance characteristics. Intelligent tiered clusters analyze massive amounts of data, determine the hotness or coldness of the objects currently being stored, and provide users with data reports to assist them in selecting the appropriate media for storing their data. For example, users can choose to store hot data on higher-performance media and cold data on lower-performance media to improve overall cost management. Furthermore, users can request services based on these data reports. For example, if the data volume of certain data in a data report is large, additional equipment or resources can be requested to provide storage services for this data.

[0052] Due to the large scale of data access and the fragmentation of execution nodes operating on the same object bucket, related data can be dispersed across numerous nodes. Intelligent tiering clusters need to generate comprehensive data reports by reading various types of data, including metadata, access data, and configuration files. In a container-based cloud-native environment, each container generates a massive amount of data, and the scale of containers is far greater than that of physical machines. Therefore, a more efficient intelligent tiering processing method is needed.

[0053] In one possible implementation, an agent can be deployed in each business container group. This agent sends access data corresponding to each business pod to a data collection system. The data collection system then sends this access data to the intelligent tiering cluster, which then centrally processes the access data reported by all business pods. After the intelligent tiering cluster processes all access data, it generates a series of data reports.

[0054] 1 , the data processing device includes a business container group 101 , a data acquisition system 102 , an intelligent hierarchical cluster 103 , and a report display module 104 . The business container group 101 includes a business container 1011 , a file system 1012 , and an agent module 1013 .

[0055] Specifically, the business container 1011 is used to store user access data. After obtaining the user access data from the business container 1011, the file system 1012 proactively pushes this access data to the data collection system 102 via the proxy module 1013. The intelligent hierarchical cluster 103 then uniformly processes the access data sent by the data collection system 102. After processing the access data, the intelligent hierarchical cluster 103 generates relevant data reports and sends these reports to the report presentation module 104. Finally, the report presentation module 104 displays these reports.

[0056] In the above implementation, each business container group actively pushes data to the data collection system through its own internally deployed proxy module, without a unified data collection logic. In cloud-native container scenarios, there are many business container groups. This active push method can cause a large number of business container groups to push data to the data collection system simultaneously at a given moment, exceeding the data collection system's capacity and causing data push queues or even loss.

[0057] In addition, due to the large number of business container groups, the scale of access data also increases accordingly. If only one intelligent hierarchical cluster uniformly processes the access data reported by all business container groups, the processing capacity of the intelligent hierarchical cluster will become a bottleneck, and the access data cannot be processed in a timely manner, affecting the effectiveness of data processing.

[0058] At the same time, since the data is generated and reported by the business container group, the data within the business container group is highly coupled. The above implementation method does not fully utilize the processing capabilities of the business container group to process the data in advance, resulting in a considerable portion of the capabilities of the intelligent hierarchical cluster being used for events such as filtering unimportant data, converting data formats, and merging related data, further affecting the processing capabilities of the intelligent hierarchical cluster.

[0059] Based on this, an embodiment of the present application provides a task processing method running on a cloud computing platform. After the cloud computing platform obtains the task information of the task to be processed and the processing capacity information of the business container group deployed in the cloud computing platform, it can allocate the task to be processed to the target computing device for processing based on the task information and the processing capacity information. Since the target computing device is an intelligent hierarchical cluster deployed in the business container group or the cloud computing platform, the data of the target object can be processed by the business pod side or the intelligent hierarchical cluster, which can avoid only one cluster or system processing a large amount of data uniformly, thereby improving the effectiveness of data processing. And the intelligent hierarchical cluster can generate an access data table for the target object based on the processing results of the task to be processed. Since the access data table is used to manage the storage area of ​​the target object, the user can apply for more storage resources for the data of the target object based on the access data table or identify hot data based on the access data table and apply to store the hot data in a storage medium with higher performance, which can provide customized service needs for users.

[0060] The task processing method, apparatus, and device provided in the embodiments of the present application, which run on a cloud computing platform, are applicable to task processing (or data processing) scenarios (including but not limited to cloud native container scenarios, container business deployment scenarios, and distributed storage scenarios). After the cloud computing platform obtains the task information of the pending task and the processing capacity information of the business container group, it can assign the pending task to the target computing device for processing, and the intelligent hierarchical cluster can generate access data of the target object based on the processing results of the pending task.

[0061] In order to enable ordinary people in the art to better understand the technical solutions of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0062] It is understood that the terms "first," "second," and the like in the specification and claims of this application and the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate so that the embodiments of the application described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Instead, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

[0063] It will also be understood that the term “comprising” indicates the presence of described features, integers, steps, operations, elements and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements and / or components.

[0064] In one example, the cloud computing platform that executes the task processing method provided in the embodiments of the present application can be a terminal, which can also be called user equipment (UE), access terminal, subscriber unit (subscriber unit), user station, mobile station (MS), mobile station, remote station, remote terminal, mobile device, user terminal, terminal, wireless communication equipment, user agent or user device. The terminal in the embodiments of the present application can be a mobile phone, a cellular phone, a smart phone, a tablet computer, a wireless data card, a personal digital assistant (PDA), a wireless modem, a handheld device (handset), a laptop computer, a machine type communication (MTC) terminal, a computer with wireless transceiver function, a virtual reality (VR) terminal, an augmented reality (AR) terminal, a smart home device (for example, a refrigerator, a television, an air conditioner, an electric meter, etc.), an intelligent robot, a robotic arm, a workshop equipment, a wireless terminal in industrial control, a wireless terminal in self-driving, a wireless terminal in remote medical care, a wireless terminal in a smart grid, a wireless terminal in transportation safety, a wireless terminal in a smart city, a wireless terminal in a smart home, a vehicle-mounted terminal, a roadside unit with terminal function, or a wireless terminal in a smart city. The terminal of the present application may also be an onboard module, onboard module, onboard component, onboard chip or onboard unit built into a vehicle as one or more components or units. The terminal may also be other devices with terminal functions, for example, the terminal may also be a device that functions as a terminal in device-to-device (D2D) communication.

[0065] The embodiments of this application do not limit the device form factor of the terminal. The device used to implement the terminal's function can be a terminal; it can also be a device that supports the terminal in implementing the function, such as a chip system. The device can be installed in the terminal or used in conjunction with the terminal. In the embodiments of this application, the chip system can be composed of a chip or include a chip and other discrete components.

[0066] In another case, the above-mentioned cloud computing platform can also be a server, which can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, network acceleration services (content delivery network, CDN), as well as big data and artificial intelligence platforms.

[0067] As shown in Figure 2, in one implementation of the embodiment of the present application, a cloud computing platform that executes the task processing method provided in the embodiment of the present application may include a scheduler 201, a business container group 202, a data acquisition system 203, a message queue 204, an intelligent hierarchical cluster 205, and a configuration interface 206. Specifically, the business container group 202 includes an operator container 2021, a business container 2022, and a file system 2023, and the intelligent hierarchical cluster 205 includes an operator container 2051.

[0068] Specifically, the scheduler 201 can be understood as a management device (or control device) in the cloud computing platform. The scheduler 201 is used to allocate the tasks to be processed to the business container group 202 or the intelligent hierarchical cluster 205 for processing based on the task information of the tasks to be processed and the processing capacity information of the business container group 202.

[0069] The operator container 2021 includes N operators (N is greater than or equal to 1), which can process pending tasks.

[0070] The business container 2022 is used to store the data of the target object.

[0071] The file system 2023 is a data sharing channel between the business container 2022 and the operator container 2021 , which facilitates the operator container 2021 to obtain the data of the target object from the business container 2022 .

[0072] The data collection system 203 is used to collect data of the target object from the business container group 202 .

[0073] The message queue 204 is a message transmission channel between the data acquisition system 203 and the intelligent grading cluster 205 , and is used to transmit data of target objects and intermediate results of tasks to be processed.

[0074] The intelligent hierarchical cluster 205 is used to generate an access data table of a target object based on the processing results of the task to be processed.

[0075] The operator container 2051 includes M operators (M is greater than or equal to 1), which can also process pending tasks.

[0076] The configuration interface 206 is used to display the access data table of the target object.

[0077] Based on the cloud computing platform shown in FIG. 2 , the task processing method provided in the embodiment of the present application can be run on the cloud computing platform to implement the process of generating an access data table for a target object. Specifically, the cloud computing platform is deployed with a container management system that includes an intelligent hierarchical cluster and one or more business container groups. The cloud computing platform also provides a configuration interface. As shown in FIG. 3 , the task processing method can include steps S101-S103.

[0078] Step S101: The cloud computing platform obtains task information of the task to be processed and processing capacity information of the business container group according to the task configuration operation performed by the user on the configuration interface.

[0079] The task configuration operation instructs to perform intelligent hierarchical processing on the data of the target object.

[0080] It should be understood that the task configuration operation of the user in the configuration interface can specifically be that the user performs preset operations (such as input, pull-down, click and other selection operations) on the configuration items related to the task to be processed on the configuration interface, thereby determining the configuration information of the task to be processed.

[0081] Optionally, the configuration information of a task may include a combination of one or more of the following information: the identifier of the object (or bucket object), the identifier of the user (or tenant), the time interval corresponding to the object, the number of times the object is accessed, and the region (or area) to which the object belongs, etc.

[0082] For example, the configuration interface shown in Figure 4 includes four configuration items: object, tenant, region, and access attributes. The user enters the identifier of object A, the identifier of tenant B, the names of four regions (including Beijing, Shanghai, Shenzhen, and Xi'an), and the number of accesses in the corresponding input boxes for each of the four configuration items. This generates the configuration information for the pending task, specifically including the identifier of object A, the identifier of tenant B, the four region names, and the number of accesses for object A.

[0083] It should be understood that the task information of the task to be processed may include the configuration information of the task to be processed. The processing capability information of the service container group is used to represent the task processing capability of the service container group.

[0084] Step S102: The cloud computing platform allocates the task to be processed to the target computing device for processing based on the task information of the task to be processed and the processing capability information of the business container group.

[0085] The target computing device is one or more of the above-mentioned business container groups or intelligent hierarchical clusters.

[0086] As shown in FIG5 , the cloud computing platform allocates the pending tasks to the target computing device for processing based on the task information of the pending tasks and the processing capability information of the business container group, which may specifically include steps S1021 - S1023 .

[0087] Step S1021: The cloud computing platform matches the processing capability information of the business container group with the task information of the task to be processed.

[0088] In an optional implementation, the processing capability information of a business container group includes a combination of one or more of the following information: central processing unit (CPU) resources that can be allocated to the business container group, memory resources that can be allocated to the business container group, bandwidth of the business container group, disk pressure of the business container group, queue depth corresponding to the business container group, and operator types included in the business container group.

[0089] In another optional implementation, the task information of the task to be processed may include a combination of one or more of the following information: the CPU resources of the task to be processed, the memory resources occupied by the task to be processed, the bandwidth resources occupied by the task to be processed, the disk pressure occupied by the task to be processed, and the operator type corresponding to the task to be processed.

[0090] Step S1022: The cloud computing platform determines that the processing capability information of the business container group successfully matches the task information of the task to be processed, and allocates the task to be processed to the business container group for processing.

[0091] It should be understood that when the cloud computing platform determines that the processing capability information of a business container group matches the task information of a pending task, it indicates that the business container group's task processing capability matches the task information. Alternatively, it can be understood that the business container group has the processing capability to handle the pending task. At this point, the cloud computing platform assigns the pending task to the business container group for processing, specifically by having the business pod handle the pending task. This allows the pending task to be processed efficiently, allowing the business pod to process the target object's data, thereby improving data processing efficiency.

[0092] In one case, when there is only one business container group in the one or more business container groups whose processing capability information successfully matches the task information of the task to be processed, the cloud computing platform may assign the task to be processed to the business container group for processing.

[0093] In another case, when the number of business container groups whose processing capability information in the one or more business container groups successfully matches the task information of the pending task (hereinafter referred to as at least two business container groups) is greater than one, the cloud computing platform may determine the capability level (or capability evaluation index) of each business container group based on the processing capability information of each business container group in the at least two business container groups, and assign the pending task to the business container group with the highest capability level among the at least two business container groups for processing.

[0094] Step S1023: The cloud computing platform determines that the processing capability information of the business container group fails to match the task information of the task to be processed, and allocates the task to be processed to the intelligent hierarchical cluster for processing.

[0095] It should be understood that if the cloud computing platform fails to determine whether the processing capability information of a business container group matches the task information of a pending task, this indicates that the task processing capability of the business container group does not match the task information. Alternatively, it can be understood that the business container group does not have the processing capability to handle the pending task. In this case, the cloud computing platform assigns the pending task to an intelligent hierarchical cluster for processing. This improves the processing efficiency of the pending task, allowing the intelligent hierarchical cluster to process the target object's data, thereby improving data processing efficiency.

[0096] In the embodiments of the present application, if the processing capability information of a business container group successfully matches the task information of a pending task, it indicates that the business container group has the processing capability for the pending task. In this case, the cloud computing platform assigns the pending task to the business container group for processing, thereby reducing the processing pressure on the intelligent hierarchical cluster while processing the pending task. If the processing capability information of a business container group fails to match the task information of a pending task, it indicates that the business container group does not have the processing capability for the pending task. In this case, the cloud computing platform assigns the pending task to the intelligent hierarchical cluster for processing, thereby ensuring the effectiveness of task processing.

[0097] Optionally, each item of information included in the processing capability information of a business container group may have a priority, and each item of information included in the task information of a task (e.g., a pending task) may also have a priority. When multiple business container groups are deployed in the container management system, the cloud computing platform may screen the multiple business container groups according to the priority order of one or more items of information included in the processing capability information of each business container group and / or the priority order of one or more items of information included in the task information of the pending task, to determine the business container group that will process the pending task.

[0098] Specifically, for one or more items of information included in the processing capability information of each of the multiple business container groups, the cloud computing platform may filter the multiple business container groups in descending order of priority. The cloud computing platform then determines the business container group whose processing capability information successfully matches the task information of the pending task as the business container group that will process the pending task, and assigns the pending task to this business container group for processing.

[0099] In one embodiment, if the processing capability information of a service container group includes the CPU resources allocatable to the service container group, the memory resources allocatable to the service container group, the bandwidth of the service container group, the disk pressure of the service container group, the queue depth corresponding to the service container group, and the operator types included in the service container group, then the priority of the multiple items of information included in the processing capability information of the service container group is in the following order from high to low: the operator types included in the service container group, the CPU resources allocatable to the service container group, the memory resources allocatable to the service container group, the queue depth corresponding to the service container group, the bandwidth of the service container group, and the disk pressure of the service container group.

[0100] In another embodiment, if the task information of a pending task includes the CPU resources occupied by the pending task, the memory resources occupied by the pending task, the bandwidth occupied by the pending task, the disk pressure occupied by the pending task, and the operator type corresponding to the pending task, then the priority of the multiple pieces of information included in the task information are in the following order from high to low: the operator type corresponding to the pending task, the CPU resources occupied by the pending task, the memory resources occupied by the pending task, the bandwidth occupied by the pending task, and the disk pressure occupied by the pending task.

[0101] In embodiments of the present application, a computing device can filter multiple business container groups based on the priority order of one or more items of information included in the processing capability information of each of the multiple business container groups and / or the priority order of one or more items of information included in the task information of the pending task to determine the business container group that will process the pending task. In other words, the target computing device that will process the pending task can be determined based on a certain priority order or filtering order, which can improve the orderliness of determining the target computing device and thus improve the orderliness of task processing.

[0102] In an optional implementation, the processing capacity information of a business container group includes the CPU resources that can be allocated to the business container group and the operator types included in the business container group, and the task information of the pending task includes the CPU resources occupied by the pending task and the operator type corresponding to the pending task. The above-mentioned cloud computing platform allocates the pending task to the business container group for processing, which can specifically include the cloud computing platform determining at least one business container group from multiple business container groups, and allocating the pending task to the target business container group in the at least one business container group for processing. The operator types included in the at least one business container group include the operator type corresponding to the pending task, and the target business container group is any business container group in the at least one business container group whose allocateable CPU resources meet the CPU resources occupied by the pending task.

[0103] It should be understood that the cloud computing platform can first perform a preliminary screening of the multiple business container groups based on the operator types included in each of the multiple business container groups and the operator types corresponding to the pending tasks, that is, screen out at least one business container group whose included operator types successfully match the operator types corresponding to the pending tasks. Then, from the at least one business container group, a target business container group whose allocatable CPU resources meet the CPU resources occupied by the pending tasks is screened out, and the pending tasks are assigned to the target business container group for processing. Based on the specific content included in the processing capability information of the business container group and the specific content included in the task information of the pending tasks, the target computing device for processing the pending tasks can be conveniently and quickly determined, further improving the efficiency of data processing.

[0104] In one implementation of the present invention, after the cloud computing platform assigns a pending task to a business container group for processing, the business container group can independently complete the processing of the pending task. Specifically, the business container group can process the pending task and obtain a processing result for the pending task.

[0105] It should be understood that the business container group processing the pending task can also be understood as the business container group processing the data of the target object. The processing result of the pending task is the final result obtained after processing the pending task.

[0106] In this embodiment of the present application, a business container group processes pending tasks (or target object data) and obtains the processing results of the pending tasks. This allows the business pod to independently complete the processing of pending tasks, improving data processing efficiency. This also reduces the processing pressure on the intelligent hierarchical cluster and improves its processing capabilities.

[0107] In another implementation of the present embodiment, after the cloud computing platform assigns pending tasks to a business container group for processing, the business container group and the intelligent hierarchical cluster can jointly complete the processing of the pending tasks. Specifically, as shown in FIG6 , the task processing method provided in the present embodiment further includes steps S104-S106.

[0108] Step S104: The business container group processes the task to be processed and obtains an intermediate result of the task to be processed.

[0109] Step S105: The data collection system collects the intermediate results of the tasks to be processed from the business container group, and sends the intermediate results of the tasks to be processed to the intelligent grading cluster.

[0110] It should be understood that the data collection system is a data collection system deployed in the cloud computing platform. The data collection system can actively collect intermediate results of tasks to be processed from the business container group.

[0111] Step S106: The intelligent grading cluster obtains the processing result of the task to be processed based on the intermediate result of the task to be processed.

[0112] In one scenario, the cloud computing platform's processing of pending tasks specifically includes data cleansing. This data cleansing can involve deleting data that the user is not interested in, while retaining data that the user is interested in. The data of interest to the user constitutes the intermediate results of the pending task. After the data collection system collects the cleansed data from the business container group, the intelligent hierarchical cluster aggregates (or integrates) the cleansed data to obtain the processing results of the pending task.

[0113] Optionally, the data that the user is concerned about is data related to the configuration information of the task to be processed.

[0114] For example, as shown in Figure 7, assume that two business container groups are deployed in a cloud computing platform, including business container group 1 and business container group 2. Business container group 1 stores data 1, data 2, and data 3, and business container group 2 stores data 4, data 5, and data 6. The data that users are concerned about (or need to retain) includes object identifiers, user identifiers, and operations performed by users on objects.

[0115] In this way, for business container group 1, the business container group 1 can delete the task identifier (including task-1 and task-3), time (including 100, 200, 300) and result (including success and failure). The data (or intermediate results) obtained (or retained) by business container group 1 include the object identifier (i.e., 00001), user identifier (including user 1 and user 2) and the operations performed by the user on the object (including creation and deletion).

[0116] For business container group 2, the business container group 2 can also delete the task identifier (including task-2 and task-4), time (including 300, 200, 100) and result (including success and failure). The obtained data includes the object identifier (including 00001 and 00002), user identifier (including user 1 and user 2) and the operation performed by the user on the object (including creation and modification).

[0117] In addition, for each of the two business container groups, the cleaned data can be integrated. For example, object ID 00001, Create (2), Delete (1) indicates that the user has performed two create operations and one delete operation on the object with ID 00001 in business container group 1.

[0118] Furthermore, the data collection system can collect integrated data from each of the two business container groups and send the integrated data of each business container group to the intelligent hierarchical cluster. After obtaining the integrated data of each business container group, the intelligent hierarchical cluster can perform aggregation processing. For example, the user identifier: user 1, create (2), modify (1), delete (1) is used to indicate that user 1 has performed 2 create operations, 1 modify operation, and 1 delete operation on the relevant objects in the two business container groups.

[0119] In another scenario, the cloud computing platform processes pending tasks by performing data conversion on the pending tasks to generate basic indicators. These basic indicators are the intermediate results of the pending tasks. A data collection system then collects these basic indicators from the business container group and sends them to the intelligent tiering cluster. Finally, the intelligent tiering cluster aggregates these basic indicators to generate final indicators. These final indicators are the processing results (or final results) of the pending tasks.

[0120] For example, as shown in FIG8 , it is assumed that five business container groups are deployed in the cloud computing platform, and each of the five business container groups stores one million pieces of data.

[0121] In one scenario, the five business container groups each proactively push 1 million pieces of data to the data collection system. The data collection system then sends five 1 million pieces of data (i.e., 5 million pieces of data) to the intelligent tiering cluster. The intelligent tiering cluster then processes the 5 million pieces of data to generate 5,000 indicators.

[0122] In another scenario, each of the five business container groups first performs data conversion on the 1 million pieces of data it stores, generating 500 basic indicators. The data collection system then proactively collects 500 basic indicators from each of the five business container groups (i.e., 2,500 basic indicators) and sends these 2,500 basic indicators to the intelligent tiering cluster. After obtaining these 2,500 basic indicators, the intelligent tiering cluster aggregates them to produce 5,000 final indicators.

[0123] In this embodiment of the application, the business container group and the intelligent hierarchical cluster jointly complete the processing process of the pending task, which can fully utilize the data coupling characteristics of the business pod side to obtain the intermediate results of the pending task in advance, and then the intelligent hierarchical cluster can obtain the processing result (or final result) of the pending task based on the intermediate results. This can significantly reduce the processing scale and complexity of the intelligent hierarchical cluster.

[0124] In one implementation of the present embodiment, the target computing device includes an operator container, which is deployed on the target computing device as a container image. The target computing device may process the pending task using the operator container. As shown in FIG9 , the cloud computing platform assigns the pending task to the target computing device for processing, which may specifically include step S1024.

[0125] Step S1024: The cloud computing platform assigns the pending tasks to the operator container for processing, and obtains the processing results of the pending tasks.

[0126] It should be understood that the target computing device may include one or more operator containers, and an operator container may include one or more operators. After the cloud computing device assigns a task to be processed to an operator container, the operator container can perform computations on the task based on the operator corresponding to the task, thereby obtaining a processing result for the task.

[0127] It can be understood that the type of operator containers included in the above-mentioned intelligent hierarchical cluster may be the same as or different from the type of operator containers included in the business container group, and the number of operator containers included in the intelligent hierarchical cluster may be the same as or different from the number of operator containers included in the business container group. The embodiment of the present application does not specifically limit the type and number of operator containers included in the target computing device.

[0128] In the embodiments of the present application, the cloud computing device assigns pending tasks to operator containers in the target computing device, which then process the pending tasks to obtain the processing results. This allows precise control of the operators in the operator container that participate in the computation, providing processing power that matches the pending tasks. This effectively completes task processing while avoiding insufficient or wasted processing power.

[0129] To fully utilize the processing power of the cloud computing platform, the cloud computing platform can dynamically adjust the processing power of the operator containers (or operators) included in the target computing device to achieve adaptive task processing capabilities. Based on this, the task information of the pending task can also include the data volume of the pending task. The cloud computing platform can perform expansion or contraction operations on the operator containers included in the target computing device based on the data volume of the pending task.

[0130] It should be understood that the data volume of the task to be processed is the data volume required to process the task to be processed.

[0131] It is understandable that the size of the data volume of the task to be processed can be 1 kilobyte (kb), and the size of the data volume of the task to be processed can also be 1 megabyte (mb). The embodiment of the present application does not specifically limit the size of the data volume of the task to be processed.

[0132] In one case, when the data volume of the task to be processed is greater than or equal to the data volume threshold, it indicates that the data volume of the task to be processed is large, and the target computing device (specifically, the operator container included in the target computing device) may not be able to meet the processing requirements of the task to be processed. In this case, the cloud computing platform can perform capacity expansion operations on the operator container included in the target computing device to improve the processing capabilities of the target computing device.

[0133] In another case, when the data volume of the task to be processed is less than the data volume threshold, it indicates that the data volume of the task to be processed is small, and the target computing device (specifically, the operator container included in the target computing device) can easily complete the processing of the task to be processed. In this case, the cloud computing platform can perform a scaling operation on the operator container included in the target computing device to save processing resources of the target computing device.

[0134] For example, as shown in Figure 10, the cloud computing platform includes a scheduler, seven processing tasks (including Task 1, Task 2, Task 3, Task 4, Task 5, Task 6, and Task 7), and four business container groups (including Business Container Group 1, Business Container Group 2, Business Container Group 3, and Business Container Group 4). Business Container Group 1 includes four operator containers: Operator Container 1, Operator Container 2, Operator Container 3, and Operator Container 4.

[0135] Based on the task information of each of the seven processing tasks and the processing capacity information of each of the four business container groups, the scheduler assigns tasks 1, 2, and 3 to business container group 1 for processing, and assigns tasks 4 and 5 to business container group 2 for processing, and assigns tasks 6 and 7 to business container group 3 and business container group 4 for processing, respectively.

[0136] Assume that Task 1, Task 2, and Task 3 constitute the aforementioned pending tasks, and that the data volume of these pending tasks is greater than or equal to the data volume threshold. The cloud computing platform then performs a capacity expansion operation on the operator containers included in Business Container Group 1. Finally, Business Container Group 1 includes eight operator containers: Operator Container 1, Operator Container 2, Operator Container 3, Operator Container 4, Operator Container 5, Operator Container 6, Operator Container 7, and Operator Container 8.

[0137] Step S103: The intelligent hierarchical cluster generates an access data table of the target object based on the processing results of the task to be processed.

[0138] The access data table is used to manage the storage area of ​​the target object.

[0139] In an optional implementation, the intelligent hierarchical cluster may send the access data table of the target object to the configuration interface, so that the configuration interface displays the access data table.

[0140] It should be understood that after the configuration interface displays the access data table of the target object, the user can make some decisions or operations on the data of the target object based on the access data table.

[0141] In one case, the user may apply for more storage resources or storage devices for the data of the target object.

[0142] In another case, the user can identify hot data (or high-value data) and cold data (or low-value data) from the data of the target object, and determine that the hot data is stored in a storage medium with higher performance and the cold data is stored in a storage medium with lower performance.

[0143] Optionally, the access data table of the target object may include a combination of one or more of the following data: an identifier of the target object, an identifier of a user accessing the target object, an identifier of the region where the user is located, and an operation performed by the user on the target object.

[0144] For example, the configuration interface shown in FIG11 includes an access data table of the target object. Specifically, the access data table includes the identifier of the target object (i.e., 00001), the user who accessed the target object, the region where the user who accessed the target object is located, and the operations performed by the user who accessed the target object on the target object.

[0145] Specifically, the target object is accessed by three users: User 1, User 2, and User 3. These users are located in Beijing, Shanghai, and Shenzhen, respectively. User 1 performs operations on the target object, including creation and modification; User 2 performs operations on the target object, including modification and deletion; and User 3 performs operations on the target object, including creation, modification, and deletion.

[0146] In an embodiment of the present application, after the cloud computing platform obtains the task information of the task to be processed and the processing capacity information of the business container group deployed in the cloud computing platform, it can allocate the task to be processed to the target computing device for processing based on the task information and the processing capacity information. Since the target computing device is the intelligent hierarchical cluster deployed in the business container group or the cloud computing platform, the data of the target object can be processed by the business pod side or the intelligent hierarchical cluster, which can avoid only one cluster or system from uniformly processing a large amount of data, thereby improving the effectiveness of data processing. In addition, the intelligent hierarchical cluster can generate an access data table for the target object based on the processing results of the task to be processed. Since the access data table is used to manage the storage area of ​​the target object, the user can apply for more storage resources for the data of the target object based on the access data table or identify hot data based on the access data table and apply to store the hot data in a storage medium with higher performance, which can provide customized service needs for the user.

[0147] In conjunction with the description of the above embodiment, the cloud computing platform also deploys a data collection system. This data collection system can proactively collect data on target objects from the business container group, thereby enabling intelligent hierarchical processing of the target object data. Based on this, the task processing method provided in the embodiment of the present application also includes steps A through C.

[0148] Step A: The cloud computing platform obtains the collection performance information of the data collection system.

[0149] It should be understood that the acquisition performance information is used to ensure the acquisition capability of the data acquisition system.

[0150] Optionally, the acquisition performance information of the data acquisition system includes a combination of one or more of the following information: network bandwidth occupied by the data acquisition system, CPU resources occupied by the data acquisition system, and memory resources occupied by the data acquisition system.

[0151] Step B: The cloud computing platform instructs the data collection system to collect data of the target object from the business container group based on the collection performance information of the data collection system.

[0152] Optionally, when multiple business container groups are deployed in the cloud computing platform, the cloud computing platform may collect data of target objects from the multiple business container groups in batches based on collection performance information of the data collection system.

[0153] For example, as shown in FIG12 , a scheduler, a data acquisition system, and four business container groups (including business container group 1, business container group 2, business container group 3, and business container group 4) are deployed in the cloud computing platform, and the business data of the target object includes seven data (i.e., data 1, data 2, data 3, data 4, data 5, data 6, and data 7).

[0154] Based on the collection capability of the data collection system, the scheduler instructs the data collection system to collect the seven data from the four business container groups in three batches.

[0155] Specifically, the data collection system collects data 1, data 2, and data 3 from business container group 1, data 4 and data 5 from business container group 2, and data 6 and data 7 from business container group 3 and business container group 4, respectively. Data 1, data 2, and data 3 are the first batch of data collected by the data collection system, data 4, data 5, and data 6 are the second batch of data collected by the data collection system, and data 7 is the third batch of data collected by the data collection system.

[0156] Step C: The data acquisition system sends the data of the target object to the intelligent grading system.

[0157] It is understandable that after acquiring the data of the target object sent by the data acquisition system, the intelligent grading system can obtain the processing result of the task to be processed based on the data of the target object.

[0158] In an embodiment of the present application, the cloud computing platform can instruct the data collection system to proactively collect data for target objects from a business container group based on the data collection system's collection performance information. This effectively controls the order in which data is collected, ensuring that the data collection system does not exceed its own collection capacity. Furthermore, the data collection system can send the target object's data to the intelligent tiering system, which then obtains the processing results of the pending tasks based on the target object's data. This ensures that the data collected by the data collection system does not exceed the maximum processing capacity of the intelligent tiering cluster, reducing the processing pressure on the intelligent tiering cluster.

[0159] In one optional implementation, the data volume of the aforementioned pending tasks may be large, and the business container group alone may not be able to promptly and effectively process all of the pending tasks. In this case, the cloud computing platform can assign some of the pending tasks to the business container group for processing, and assign the remaining tasks to the intelligent hierarchical cluster for processing. Based on this, the task processing method provided in this embodiment of the application may also include steps D through E.

[0160] Step D: The cloud computing platform splits the task to be processed to obtain multiple subtasks to be processed.

[0161] It should be understood that the multiple subtasks to be processed are subtasks included in the task to be processed.

[0162] In an optional implementation, the task information of the pending task may further include the data volume of the pending task. The cloud computing platform may split the pending task based on the data volume of the pending task, that is, obtain the task information of each of the multiple subtasks.

[0163] In one case, the task information of the task to be processed may also include the data volume of the task to be processed. The cloud computing platform may evenly split the data volume of the task to be processed, that is, each of the multiple subtasks to be processed has the same data volume.

[0164] In another case, the cloud computing platform may also perform non-uniform splitting of the data volume of the task to be processed, that is, the data volume of each subtask to be processed is different.

[0165] Step E: The cloud computing platform allocates at least one subtask to the business container group for processing based on the processing capability information of the business container group and the task information of the multiple subtasks to be processed, and allocates the other subtasks to the intelligent hierarchical cluster for processing.

[0166] It should be understood that the other subtasks are subtasks other than the at least one subtask among the multiple subtasks to be processed.

[0167] Specifically, for any of the multiple subtasks, the cloud computing platform can determine whether the processing capability information of the business container group matches the task information of the subtask. If so, the cloud computing platform can assign the subtask to the business container group for processing. If not, the cloud computing platform can assign the subtask to an intelligent hierarchical cluster for processing.

[0168] Optionally, after splitting the data volume of the task to be processed, the cloud computing platform can obtain the data volume of the multiple subtasks to be processed. The cloud computing platform can then query the task data relationship based on the data volume of the multiple subtasks to be processed to determine the data volume of the multiple subtasks to be processed in the task data relationship. Ultimately, the cloud computing platform determines the task information corresponding to the data volume of the multiple subtasks to be processed in the task data relationship as the task information of the multiple subtasks to be processed. The task data relationship includes multiple data volumes and the task information corresponding to each of the multiple data volumes.

[0169] In this embodiment of the present application, the cloud computing platform can instruct the business container group and the intelligent hierarchical cluster to collaboratively complete the processing of pending tasks. Specifically, the business container group handles at least one subtask, while the intelligent hierarchical cluster handles the remaining subtasks. This fully utilizes the processing capabilities of the intelligent hierarchical cluster and the business pod side, significantly reducing the processing costs of the cloud computing platform, avoiding the need for a single cluster (or system) to process all data, and improving data processing efficiency.

[0170] For example, as shown in Figure 13, a scheduler, an intelligent hierarchical cluster and three business container groups (including business container group 1, business container group 2 and business container group 3) are deployed in the cloud computing platform. The tasks to be processed include seven subtasks, namely Task 1, Task 2, Task 3, Task 4, Task 5, Task 6 and Task 7.

[0171] Based on the processing capacity information of the three business container groups and the task information of the seven subtasks, the scheduler assigns Task 1, Task 2, and Task 3 to the intelligent hierarchical cluster for processing, and assigns Task 4 and Task 5 to Business Container Group 1 for processing, and assigns Task 6 and Task 7 to Business Container Group 2 and Business Container Group 3 for processing, respectively.

[0172] It is understood that in order to implement the functions in the above embodiments, the cloud computing platform includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should readily appreciate that, in combination with the units and method steps of each example described in the embodiments disclosed in this application, this application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in hardware or in a manner driven by computer software depends on the specific application scenario and design constraints of the technical solution.

[0173] The above description, in conjunction with Figures 1 to 13, details the task processing method provided in accordance with this embodiment. The following description will describe the task processing apparatus provided in accordance with this embodiment in conjunction with Figure 14. As shown in Figure 14, an embodiment of the present application provides a task processing apparatus 30 running on a cloud computing platform. The cloud computing platform is deployed with a container management system, in which an intelligent hierarchical cluster and one or more business container groups are deployed. The cloud computing platform also provides a configuration interface. The task processing apparatus includes: an acquisition module 301 and a first processing module 302. The intelligent hierarchical cluster includes a second processing module 303.

[0174] The acquisition module 301 is used to acquire task information of the task to be processed and processing capability information of the business container group according to the task configuration operation performed by the user on the configuration interface, wherein the task configuration operation indicates intelligent hierarchical processing of target object data.

[0175] A first processing module 302 is configured to allocate the task to be processed to a target computing device for processing based on the task information of the task to be processed and the processing capability information of the business container group. The target computing device is the business container group or the intelligent hierarchical cluster.

[0176] The second processing module 303 is configured to generate an access data table for the target object based on the processing result of the task to be processed, where the access data table is used to manage the storage area of ​​the target object.

[0177] Optionally, the first processing module 302 is further configured to match the processing capability information of the service container group with the task information of the task to be processed.

[0178] The first processing module 302 is further configured to determine whether the processing capability information of the business container group successfully matches the task information of the task to be processed, and to allocate the task to be processed to the business container group for processing.

[0179] The first processing module 302 is further configured to determine that the processing capability information of the business container group fails to match the task information of the task to be processed, and assign the task to be processed to the intelligent hierarchical cluster for processing.

[0180] Optionally, the first processing module 302 is further configured to split the task to be processed into a plurality of subtasks to be processed.

[0181] The first processing module 302 is further configured to allocate at least one subtask to the business container group for processing, and allocate other subtasks to the intelligent hierarchical cluster for processing based on the processing capability information of the business container group and the task information of the multiple subtasks to be processed.

[0182] Optionally, the cloud computing platform is further deployed with a data acquisition system, the task processing device 30 further includes an indication module 304 , and the data acquisition system includes a sending module 305 .

[0183] The acquisition module 301 is further used to acquire acquisition performance information of the data acquisition system.

[0184] The instructing module 304 is configured to instruct the data collection system to collect the data of the target object from the business container group based on the collection performance information of the data collection system.

[0185] The sending module 305 is configured to send the data of the target object to the intelligent grading cluster.

[0186] Optionally, the service container group includes a third processing module 306 .

[0187] The third processing module 306 is used to process the task to be processed and obtain a processing result of the task to be processed.

[0188] Optionally, the data acquisition system further includes an acquisition module 307 .

[0189] The third processing module 306 is used to process the task to be processed and obtain an intermediate result of the task to be processed.

[0190] The collection module 307 is configured to collect the intermediate results of the task to be processed from the business container group.

[0191] The sending module 305 is used to send the intermediate result of the task to be processed to the intelligent grading cluster.

[0192] The second processing module 303 is configured to obtain a processing result of the task to be processed according to the intermediate result of the task to be processed.

[0193] Optionally, the target computing device includes an operator container, which is deployed in the target computing device in the form of a container image.

[0194] The first processing module 302 is specifically configured to assign the task to be processed to the operator container for processing, and obtain a processing result of the task to be processed.

[0195] Optionally, the task information of the task to be processed includes the data volume of the task to be processed.

[0196] The first processing module 302 is further configured to perform an expansion operation or a contraction operation on the operator container included in the target computing device based on the data volume of the task to be processed.

[0197] The acquisition module 301, the processing module (including the first processing module 302, the second processing module 303, and the third processing module 306), the indication module 304, the sending module 305, and the collection module 307 can all be implemented via software or hardware. For example, the implementation of the first processing module 302 will be described below using the first processing module 302 as an example. Similarly, the implementation of the acquisition module 301, the second processing module 303, the indication module 304, the sending module 305, the third processing module 306, and the collection module 307 can refer to the implementation of the determination module.

[0198] As an example of a software functional unit, the first processing module 302 may include code running on a computing instance. The computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Furthermore, the computing instance may be one or more. For example, the first processing module 302 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code may be distributed in the same region or in different regions. Furthermore, the multiple hosts / virtual machines / containers used to run the code may be distributed in the same availability zone (AZ) or in different AZs, each AZ including one data center or multiple geographically close data centers. Typically, a region may include multiple AZs.

[0199] Similarly, multiple hosts / virtual machines / containers running the code can be distributed within the same virtual private cloud (VPC) or across multiple VPCs. Typically, a VPC is set up within a region. Cross-region communication between two VPCs within the same region, or between VPCs in different regions, requires a communication gateway within each VPC to interconnect the VPCs.

[0200] As an example of a hardware functional unit, the first processing module 302 may include at least one computing device, such as a server. Alternatively, the first processing module 302 may be implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0201] The multiple computing devices included in the first processing module 302 can be distributed in the same region or in different regions. The multiple computing devices included in the processing module 302 can be distributed in the same AZ or in different AZs. Similarly, the multiple computing devices included in the first processing module 302 can be distributed in the same VPC or in multiple VPCs. The multiple computing devices can be any combination of servers, ASICs, PLDs, CPLDs, FPGAs, GALs, and other computing devices.

[0202] It should be noted that, in other embodiments, the first processing module 302 can be used to execute any step in the task processing method, the acquisition module 301 can be used to execute any step in the task processing method, the second processing module 303 can be used to execute any step in the task processing method, the indication module 304 can be used to execute any step in the task processing method, the sending module 305 can be used to execute any step in the task processing method, the third processing module 306 can be used to execute any step in the task processing method, and the acquisition module 307 can also be used to execute any step in the task processing method. The steps that the acquisition module 301, the processing modules (including the first processing module 302, the second processing module 303, and the third processing module 306), the indication module 304, the sending module 305, and the acquisition module 307 are responsible for implementing can be specified as needed. The full functionality of the task processing device is achieved by having the acquisition module 301, the processing modules (including the first processing module 302, the second processing module 303, and the third processing module 306), the indication module 304, the sending module 305, and the acquisition module 307 respectively implement different steps in the task processing method.

[0203] This application also provides a computing device 400. As shown in Figure 15, computing device 400 includes a bus 402, a processor 404, a memory 406, and a communication interface 408. Processor 404, memory 406, and communication interface 408 communicate with each other via bus 402. Computing device 400 can be a server or a terminal device. It should be understood that this application does not limit the number of processors and memories in computing device 400.

[0204] Bus 402 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, among others. Buses may be classified as address buses, data buses, control buses, and the like. For ease of illustration, FIG15 shows only one bus line, but this does not imply a single bus or type of bus. Bus 402 may include a path for transmitting information between various components of computing device 400 (e.g., memory 406, processor 404, and communication interface 408).

[0205] The processor 404 may include any one or more processors such as a CPU, a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0206] The memory 406 may include volatile memory, such as random access memory (RAM). The processor 404 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0207] The memory 406 stores executable program code, and the processor 404 executes the executable program code to respectively implement the functions of the aforementioned acquisition module, processing module, indication module, sending module, and collection module, thereby implementing the task processing method. In other words, the memory 406 stores instructions for executing the task processing method.

[0208] The communication interface 408 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 400 and other devices or a communication network.

[0209] Embodiments of the present application also provide a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smartphone.

[0210] As shown in Figure 16, the computing device cluster includes at least one computing device 400. The memory 406 of one or more computing devices 400 in the computing device cluster may store the same instructions for executing the task processing method.

[0211] In some possible implementations, the memory 406 of one or more computing devices 400 in the computing device cluster may also store partial instructions for executing the task processing method. In other words, the combination of one or more computing devices 400 can jointly execute the instructions for executing the task processing method.

[0212] It should be noted that the memory 406 in different computing devices 400 in the computing device cluster can store different instructions, each for executing a portion of the functions of the task processing apparatus. In other words, the instructions stored in the memory 406 in different computing devices 400 can implement the functions of one or more of the acquisition module, processing module, instruction module, sending module, and collection module.

[0213] In some possible implementations, one or more computing devices in a computing device cluster may be connected via a network. The network may be a wide area network (WAN) or a local area network (LAN), among others. FIG. 17 illustrates a possible implementation. As shown in FIG. 17 , two computing devices 400A and 400B are connected via a network. Specifically, the network is connected via a communication interface in each computing device. In this type of possible implementation, the memory 406 in the computing device 400A stores instructions for executing the functions of the indication module, the sending module, and the acquisition module. Simultaneously, the memory 406 in the computing device 400B stores instructions for executing the functions of the acquisition module and the processing module.

[0214] The connection method between the computing device clusters shown in Figure 17 can be considered to be a process of acquiring a large amount of information and assigning the tasks to be processed to the target computing device for processing. Therefore, it is considered to assign the functions implemented by the acquisition module and the processing module to the computing device 400B.

[0215] It should be understood that the functionality of the computing device 400A shown in FIG17 may also be accomplished by multiple computing devices 400. Similarly, the functionality of the computing device 400B may also be accomplished by multiple computing devices 400.

[0216] The present application also provides another computing device cluster. The connection relationship between the computing devices in this computing device cluster can be similar to the connection method of the computing device cluster described in Figures 16 and 17. However, the memory 406 of one or more computing devices 400 in this computing device cluster can store the same instructions for executing the task processing method.

[0217] In some possible implementations, the memory 406 of one or more computing devices 400 in the computing device cluster may also store partial instructions for executing the task processing method. In other words, the combination of one or more computing devices 400 can jointly execute the instructions for executing the task processing method.

[0218] The present application also provides a computer program product including instructions. The computer program product may be software or a program product including instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on at least one computing device, the at least one computing device executes the task processing method.

[0219] The present application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that can be stored by a computing device or a data storage device such as a data center that contains one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive). The computer-readable storage medium includes instructions that instruct the computing device to execute the task processing method.

[0220] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the protection scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A task processing method running on a cloud computing platform, characterized in that: The cloud computing platform is deployed with a container management system, in which an intelligent hierarchical cluster and one or more business container groups are deployed, and the cloud computing platform also provides a configuration interface. The method includes: The cloud computing platform obtains task information of the task to be processed and processing capacity information of the business container group according to the task configuration operation of the user in the configuration interface, wherein the task configuration operation indicates intelligent hierarchical processing of data of the target object; The cloud computing platform allocates the task to be processed to a target computing device for processing based on the task information of the task to be processed and the processing capability information of the business container group, wherein the target computing device is the business container group or the intelligent hierarchical cluster; The intelligent hierarchical cluster generates an access data table of the target object based on the processing result of the task to be processed, and the access data table is used to manage the storage area of ​​the target object.

2. The method according to claim 1, characterized in that The cloud computing platform allocates the task to be processed to a target computing device for processing based on the task information of the task to be processed and the processing capability information of the business container group, including: The cloud computing platform matches the processing capability information of the business container group with the task information of the task to be processed; The cloud computing platform determines that the processing capability information of the business container group successfully matches the task information of the task to be processed, and allocates the task to be processed to the business container group for processing; The cloud computing platform determines that the processing capability information of the business container group fails to match the task information of the task to be processed, and allocates the task to be processed to the intelligent hierarchical cluster for processing.

3. The method according to claim 1 or 2, characterized in that: The method further comprises: The cloud computing platform splits the task to be processed to obtain a plurality of subtasks to be processed; The cloud computing platform allocates at least one subtask to the business container group for processing based on the processing capability information of the business container group and the task information of the multiple subtasks to be processed, and allocates other subtasks to the intelligent hierarchical cluster for processing.

4. The method according to any one of claims 1 to 3, characterized in that: The cloud computing platform is also deployed with a data acquisition system, and the method further includes: The cloud computing platform obtains the collection performance information of the data collection system; The cloud computing platform instructs the data acquisition system to acquire the data of the target object from the business container group based on the acquisition performance information of the data acquisition system; The data acquisition system sends the data of the target object to the intelligent grading cluster.

5. The method according to any one of claims 2 to 4, characterized in that: After the cloud computing platform assigns the to-be-processed tasks to the business container group for processing, the method includes: The business container group processes the task to be processed to obtain a processing result of the task to be processed.

6. The method according to any one of claims 2 to 4, characterized in that: After the cloud computing platform assigns the to-be-processed tasks to the business container group for processing, the method includes: The business container group processes the task to be processed to obtain an intermediate result of the task to be processed; The data collection system collects the intermediate results of the tasks to be processed from the business container group, and sends the intermediate results of the tasks to be processed to the intelligent grading cluster; The intelligent grading cluster obtains a processing result of the task to be processed according to the intermediate result of the task to be processed.

7. The method according to any one of claims 1 to 6, characterized in that: The target computing device includes an operator container, and the operator container is deployed in the target computing device in the form of a container image. The cloud computing platform allocates the task to be processed to the target computing device for processing, including: The cloud computing platform distributes the tasks to be processed to the operator container for processing, and obtains processing results of the tasks to be processed.

8. The method according to any one of claims 1 to 7, characterized in that: The task information of the task to be processed includes the data volume of the task to be processed, and the method further includes: The cloud computing platform performs expansion on the operator container included in the target computing device based on the data volume of the task to be processed. Capacity expansion or capacity reduction operation.

9. A task processing device running on a cloud computing platform, characterized in that: The cloud computing platform is deployed with a container management system, in which an intelligent hierarchical cluster and one or more business container groups are deployed, the cloud computing platform also provides a configuration interface, the device includes an acquisition module and a first processing module, and the intelligent hierarchical cluster includes a second processing module; The acquisition module is used to acquire task information of the task to be processed and processing capacity information of the business container group according to the task configuration operation of the user in the configuration interface, wherein the task configuration operation indicates intelligent hierarchical processing of the data of the target object; The first processing module is used to allocate the task to be processed to a target computing device for processing based on the task information of the task to be processed and the processing capability information of the business container group, wherein the target computing device is the business container group or the intelligent hierarchical cluster; The second processing module is used to generate an access data table of the target object based on the processing result of the task to be processed, and the access data table is used to manage the storage area of ​​the target object.

10. The device according to claim 9, characterized in that The first processing module is further used to match the processing capability information of the business container group with the task information of the task to be processed; The first processing module is further configured to determine whether the processing capability information of the business container group successfully matches the task information of the task to be processed, and to allocate the task to be processed to the business container group for processing; The first processing module is further configured to determine that the processing capability information of the business container group fails to match the task information of the task to be processed, and allocate the task to be processed to the intelligent hierarchical cluster for processing.

11. The device according to claim 9 or 10, characterized in that The first processing module is further used to split the task to be processed to obtain multiple subtasks to be processed; The first processing module is further configured to allocate at least one subtask to the business container group for processing and allocate other subtasks to the intelligent hierarchical cluster for processing based on the processing capability information of the business container group and the task information of the multiple subtasks to be processed.

12. The device according to any one of claims 9 to 11, characterized in that The cloud computing platform is also deployed with a data acquisition system, the device also includes an indication module, and the data acquisition system includes a sending module; The acquisition module is further used to acquire acquisition performance information of the data acquisition system; The instruction module is used to instruct the data acquisition system to collect the data of the target object from the business container group based on the acquisition performance information of the data acquisition system; The sending module is used to send the data of the target object to the intelligent grading cluster.

13. The device according to any one of claims 10 to 12, characterized in that The service container group includes a third processing module; The third processing module is used to process the task to be processed to obtain a processing result of the task to be processed.

14. The device according to any one of claims 10 to 12, characterized in that The data acquisition system includes an acquisition module and a sending module, and the business container group includes a third processing module; The third processing module is used to process the task to be processed to obtain an intermediate result of the task to be processed; The collection module is used to collect the intermediate results of the tasks to be processed from the business container group; The sending module is used to send the intermediate result of the task to be processed to the intelligent grading cluster; The second processing module is used to obtain the processing result of the task to be processed according to the intermediate result of the task to be processed.

15. The device according to any one of claims 9 to 14, characterized in that The target computing device includes an operator container, and the operator container is deployed in the target computing device in the form of a container image; The first processing module is specifically used to assign the task to be processed to the operator container for processing, and obtain the processing result of the task to be processed.

16. The device according to any one of claims 9 to 15, characterized in that The task information of the task to be processed includes the data volume of the task to be processed; The first processing module is further used to perform an expansion operation or a reduction operation on the operator container included in the target computing device based on the data volume of the task to be processed.

17. A computing device cluster, characterized in that: comprising at least one computing device, each computing device comprising a processor and a memory; The processor of the at least one computing device is used to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the task processing method according to any one of claims 1 to 8.

18. A computer program product comprising instructions, characterized in that When the instructions are executed by a computing device cluster, the computing device cluster executes the task processing method according to any one of claims 1 to 8.

19. A computer-readable storage medium, characterized in that: The method comprises computer program instructions. When the computer program instructions are executed by a computing device cluster, the computing device cluster executes the task processing method according to any one of claims 1 to 8.

Images