A task scheduling method, device and equipment for cloud multi-tenant isolation

Abstract

The application discloses a task scheduling method, device and equipment for multi-tenant isolation on a cloud, and is used for improving the efficiency and fairness of workflow task scheduling of multi-tenant on the cloud. The method comprises the following steps: receiving a first workflow of a first tenant on the cloud, performing load balancing on a plurality of service nodes in a service cluster through a distributed message queue, and scheduling a task of the first workflow to a target service node; storing the task of the first workflow to a dedicated message queue of the first tenant in the target service node; and controlling execution of the task of the first workflow based on a dedicated concurrency degree configured for the first tenant and the dedicated message queue of the first tenant through the target service node.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of cloud computing, and in particular to a task scheduling method, device and equipment for multi-tenant isolation on a cloud. BACKGROUND

[0002] A cloud computing vendor as a service provider uses management and control software to process user creation, deletion or resource scheduling requests. When a user as a tenant requests the service provider to create, delete or schedule resources, the service provider will automatically generate multiple tasks according to user needs through the management and control software, such as cloud computing device configuration, internal service calling, data persistence operation, etc. A workflow scheduling system responds to the calling of the management and control software, and is responsible for reasonably scheduling and executing these tasks to ensure that resources can be finally delivered to users on demand. The workflow scheduling system will arrange the tasks and parameters that need to be executed in a single user request into a workflow, and then execute the tasks in the workflow in series or in parallel according to the arranged order.

[0003] The workflow scheduling system needs to provide task scheduling for multiple users at the same time, and if some users have a request burst or some tasks cause performance bottlenecks in the backend service, the normal requests of other users will inevitably be affected. Therefore, how to achieve fair and efficient scheduling of the workflow tasks of multi-tenants on a cloud is a technical problem to be solved. SUMMARY

[0004] Therefore, the embodiments of the present application provide a task scheduling method, device and equipment for multi-tenant isolation on a cloud to improve the efficiency and fairness of the workflow task scheduling of multi-tenants on a cloud.

[0005] To solve the above problems, the technical solutions provided by the embodiments of the present application are as follows:

[0006] In a first aspect, the embodiments of the present application provide a task scheduling method for multi-tenant isolation on a cloud, which comprises:

[0007] receiving a first workflow of a first tenant on a cloud, load balancing multiple service nodes in a service cluster through a distributed message queue, and scheduling tasks of the first workflow to a target service node;

[0008] storing the tasks of the first workflow to a dedicated message queue of the first tenant in the target service node;

[0009] controlling the execution of the tasks of the first workflow based on a dedicated concurrency degree configured for the first tenant and the dedicated message queue of the first tenant through the target service node.

[0010] In a second aspect, the embodiments of the present application provide a task scheduling device for multi-tenant isolation on a cloud, which comprises:

[0011] a scheduling unit, configured to receive a first workflow of a first tenant on a cloud, load balance a plurality of service nodes in a service cluster through a distributed message queue, and schedule a task of the first workflow to a target service node;

[0012] a storage unit, configured to store the task of the first workflow to a dedicated message queue of the first tenant in the target service node;

[0013] a control unit, configured to control execution of the task of the first workflow based on a dedicated concurrency configured for the first tenant and the dedicated message queue of the first tenant by the target service node.

[0014] In a third aspect, an embodiment of the present application provides a task scheduling device for multi-tenant isolation on a cloud, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, the cloud multi-tenant isolation task scheduling method is implemented.

[0015] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, wherein instructions are stored in the computer readable storage medium, and when the instructions are executed on a terminal device, the terminal device executes the cloud multi-tenant isolation task scheduling method.

[0016] Therefore, the embodiments of the present application have the following beneficial effects:

[0017] In the embodiments of the present application, after receiving a first workflow of a first tenant on a cloud, the task of the first workflow is scheduled to a target service node through a distributed message queue, so as to load balance a plurality of service nodes in a service cluster. A dedicated message queue corresponding to a tenant is arranged in each service node. The task of the first workflow is added to the dedicated message queue of the first tenant in the target service node. The task is read from the dedicated message queue according to a dedicated concurrency configured for the first tenant, so as to control execution of the task of the first workflow. Therefore, the number of tasks of processing each tenant and the number of tasks of the same type can be limited through the dedicated message queue of the tenant and the dedicated concurrency, the multi-tenant isolation on the cloud is realized, and the fairness and efficiency of the workflow task scheduling of the multi-tenant on the cloud are ensured. BRIEF DESCRIPTION OF DRAWINGS

[0018] Figure 1 A schematic diagram of an exemplary application scenario provided by the embodiments of the present application;

[0019] Figure 2 A flowchart of a cloud multi-tenant isolation task scheduling method provided by the embodiments of the present application;

[0020] Figure 3 This is a schematic diagram illustrating the working process of the scheduler in the embodiments of this application;

[0021] Figure 4 This is a schematic diagram illustrating the process of dynamically adjusting the concurrency threshold in an embodiment of this application;

[0022] Figure 5 This is a schematic diagram illustrating the process of obtaining workflow information in an embodiment of this application;

[0023] Figure 6 This is a schematic diagram of the counter's operation process in an embodiment of this application;

[0024] Figure 7 This is a schematic diagram illustrating the implementation process of a cloud-based multi-tenant isolation task scheduling method in an embodiment of this application;

[0025] Figure 8 A schematic diagram of a cloud-based multi-tenant isolation task scheduling device provided in an embodiment of this application;

[0026] Figure 9 This is a schematic diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0027] To make the above-mentioned objectives, features and advantages of the embodiments of this application more apparent and understandable, the embodiments of this application will be further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0028] To facilitate understanding and explanation of the technical solutions provided in the embodiments of this application, the background technology of the embodiments of this application will be described first below.

[0029] For cost optimization or product design purposes, cloud computing vendors typically use a workflow scheduling system to provide scheduling services for different users (i.e., tenants) in the same region. The workflow scheduling system orchestrates the tasks and parameters to be executed in a single user request into a workflow, and then executes the tasks in this workflow sequentially or in parallel according to the orchestrated order.

[0030] If some users experience a sudden surge in requests, processing these requests will affect the normal requests of other users. Alternatively, certain tasks may cause performance bottlenecks in backend services, such as server failures or insufficient backend service processing capacity; processing these tasks will also affect the normal requests of other users. How to achieve isolation among multiple tenants in the cloud and ensure that workflow tasks of multiple tenants in the cloud are scheduled fairly and efficiently is the main technical problem that this application aims to solve.

[0031] To facilitate understanding of the cloud-based multi-tenant isolation task scheduling method provided in this application's embodiments, the following is combined with... Figure 1 The example scenario is shown below. See also...Figure 1 As shown in the figure, the figure is a schematic diagram of an exemplary application scenario provided by the embodiment of the application.

[0032] The embodiment of the application is applied to a workflow scheduling system, which can be composed of four modules, i.e., a memory, a scheduler, an executor, and a counter. Meanwhile, the workflow scheduling system is a multi-node system, which is composed of a service cluster of multiple service nodes, and the service node can be a server or other computing device. In the workflow scheduling system, the service node can include software modules such as the memory, the scheduler, the executor, and the counter. In addition, the workflow scheduling system further includes an independent scheduling node and a centralized cache node, the scheduling node includes the scheduler, and the centralized cache node includes the memory.

[0033] The scheduler is a core module for implementing the task scheduling method for multi-tenant isolation on the cloud provided by the embodiment of the application, and is used for dynamically scheduling tasks of multiple tenants according to system resource conditions. The memory uses a multi-level cache technology, stores metadata required for workflow execution, and can persist data. The executor can perform tasks according to task names and parameters. The counter is used to record the current execution state of the workflow, and uses an optimistic lock technology to ensure task concurrency safety.

[0034] Those skilled in the art can understand that Figure 1 The framework diagram shown is only one example in which the implementation of the application can be implemented. The scope of application of the implementation of the application is not limited by any aspect of the framework.

[0035] In order to facilitate understanding of the embodiment of the application, a task scheduling method for multi-tenant isolation on the cloud provided by the embodiment of the application is described below in combination with the drawings.

[0036] Referring to Figure 2 As shown in the figure, the figure is a flowchart of a task scheduling method for multi-tenant isolation on the cloud provided by the embodiment of the application, which can include S201-S204: Figure 2

[0037] S201: receiving a first workflow of a first tenant on the cloud, load balancing multiple service nodes in a service cluster through a distributed message queue, and scheduling tasks of the first workflow to a target service node.

[0038] ​When the tenant requests the service provider to create, delete or schedule resources, the service provider generates multiple tasks according to the tenant demand through the management software. The management software calls the workflow scheduling system, and the workflow scheduling system implements the scheduling of the tasks in the workflow. Usually, the workflow includes at least one task, and each task includes tenant information requesting the task, so the task has a corresponding relationship with the tenant. After the workflow scheduling system obtains a first workflow of a first tenant, the tasks in the first workflow are assigned to a target service node. The first tenant is any tenant on the cloud, and the target service node is any service node in the workflow scheduling system.

[0039] In actual application, the scheduler in the scheduling service node can add the tasks in the first workflow to the distributed message queue, and distribute each task to a service node through the distributed message queue. By distributing the tasks to the service nodes through the distributed message queue, the load balancing of multiple service nodes in the service cluster can be ensured as much as possible.

[0040] S202: Store the tasks of the first workflow to the exclusive message queue of the first tenant in the target service node.

[0041] In each service node, an exclusive message queue is established for each tenant. After the tasks of the first workflow are assigned to the target service node, the tasks are added to the exclusive message queue established for the first tenant in the target service node. For example, the target service node has established an exclusive message queue for tenant 1, an exclusive message queue for tenant 2, etc. The tasks of the first workflow correspond to tenant 1, and the tasks of the first workflow are added to the exclusive message queue of tenant 1. An exclusive message queue is established for each tenant to ensure that the tasks of different tenants are isolated during scheduling.

[0042] In actual application, the scheduler can store the tasks of the first workflow to the exclusive message queue of the first tenant in the target service node.

[0043] S203: Control the execution of the tasks of the first workflow based on the exclusive concurrency configured for the first tenant and the exclusive message queue of the first tenant through the target service node.

[0044] Each tenant has an exclusive concurrency to limit the tasks requested by each tenant. The exclusive message queue of the first tenant includes at least one task of the first workflow, and the total number of tasks read from the exclusive message queue of the first tenant according to the task addition order is less than or equal to the exclusive concurrency of the first tenant. For example, if the exclusive concurrency of the exclusive message queue of the current tenant is 5, and the number of tasks in the exclusive message queue is more than 5, 5 tasks can be read; if the number of tasks in the exclusive message queue is not more than 5, all tasks in the exclusive message queue can be read.

[0045] In actual application, the scheduler of the target service node can read the task of the first workflow from the exclusive message queue of the first tenant based on the exclusive concurrency configured for the first tenant. Then the target service node is triggered to execute the task of the first workflow.

[0046] In actual application, the scheduler transmits the task name, task parameters and other information of the task to be executed into the executor, and the executor actually executes the task of the first workflow and can return the execution result to the scheduler so that the scheduler continues to schedule the task.

[0047] In a possible implementation, the specific implementation of S203, which controls the execution of the task of the first workflow by the target service node based on the exclusive concurrency configured for the first tenant and the exclusive message queue of the first tenant, can include:

[0048] The target service node reads the tasks from the exclusive message queue of the first tenant in turn based on the exclusive concurrency configured for the first tenant and the task concurrency corresponding to each task type, so that the total number of the read tasks is less than or equal to the exclusive concurrency configured for the first tenant and meets the task concurrency corresponding to each task type.

[0049] The target service node executes the read task of the first workflow.

[0050] In the embodiments of the present application, in addition to setting the exclusive concurrency of a tenant, the task concurrency corresponding to each task type can also be set to limit the flow of the tasks of the same task type requested by a tenant at the same time. Therefore, when reading the tasks from the exclusive message queue of the tenant, in addition to meeting the exclusive concurrency set for the tenant, each read task also needs to meet the task concurrency corresponding to the task type of the task. For example, the exclusive concurrency set for the first tenant is 5, the task concurrency corresponding to the task type a is 2, the task concurrency corresponding to the task type b is 2, and the task concurrency corresponding to the task type c is 2. The tasks are read from the exclusive message queue of the first tenant in turn, the task type of task 1 is task type a, the task type of task 2 is task type a, the task type of task 3 is task type b, the task type of task 4 is task type c, and the task type of task 5 is task type a. At this time, task 5 does not meet the task concurrency corresponding to the task type a, and the task cannot be read temporarily, and task 6 of task type b is continued to be read. Then the target service node executes the read task of the first workflow.

[0051] In this way, by setting the exclusive concurrency of a tenant and the task concurrency corresponding to each task type, the concurrency of the requested tasks of each tenant is limited, and the fairness of the task scheduling between multiple tenants is ensured.

[0052] In addition, in actual application, other types of concurrency can be set, for example, the concurrency of the interface corresponding to the task request service, and the like.

[0053] In the embodiments of the present application, the exclusive concurrency of the tenant and the task concurrency corresponding to the task type can be automatically and dynamically adjusted according to the task execution, and the related description can be referred to in subsequent embodiments, which will not be described here.

[0054] Referring to Figure 3 As shown in the figure, a working process schematic diagram of the scheduler is shown. After the scheduler in the scheduling service node obtains the workflow, the tasks in the workflow are added to the task message queue, and the tasks in the task message queue are distributed to different service nodes, for example, the tasks are distributed to the service node 1, the service node 2 and the service node 3. The scheduler in each service node adds the tasks to the exclusive message queue of the tenant corresponding to each task. For example, the tasks distributed to the service node 1 all come from the tenant 1, and these tasks are added to the exclusive message queue of the tenant 1 in the service node 1; the tasks distributed to the service node 2 all come from the tenant 1, and these tasks are added to the exclusive message queue of the tenant 1 in the service node 2; the tasks distributed to the service node 3 all come from the tenant 2, and these tasks are added to the exclusive message queue of the tenant 2 in the service node 3. The scheduler of each service node reads the tasks from the exclusive message queue according to the exclusive concurrency of the tenant and the task concurrency corresponding to the task type, that is, after one or more tasks are read, the task concurrency is checked. If the read task meets the concurrency, the task can be executed by the executor in the service node, and if the read task does not meet the concurrency, the task is read from the exclusive message queue again.

[0055] In this way, in the embodiments of the present application, after the first workflow of the first tenant on the cloud is received, the tasks of the first workflow are scheduled to the target service node through the distributed message queue, so as to realize the load balancing of the multiple service nodes in the service cluster. The exclusive message queue corresponding to the tenant is set in each service node. The tasks of the first workflow are added to the exclusive message queue of the first tenant in the target service node. The tasks are read from the exclusive message queue according to the exclusive concurrency configured for the first tenant, so as to control the execution of the tasks of the first workflow. Thus, through the exclusive message queue of the tenant and the exclusive concurrency, the number of tasks processed for each tenant and the number of tasks of the same type can be limited, the isolation of the multiple tenants on the cloud is realized, and the fairness and efficiency of the workflow task scheduling of the multiple tenants on the cloud are ensured.

[0056] In the embodiments of the present application, whether the service cluster is overloaded and the abnormal source is analyzed, so as to dynamically adjust the concurrency.

[0057] In a possible implementation, the cloud multi-tenant isolation task scheduling method provided by the embodiment of the application can further include the following steps.

[0058] A1: determining whether the service cluster is overloaded according to the number of remaining tasks in the exclusive message queue of each tenant in the service cluster and the remaining exclusive concurrency degree of the tenant.

[0059] A2: if the service cluster is overloaded, determining an abnormal task type according to the execution time of multiple tasks of the same task type.

[0060] A3: adjusting the task concurrency degree corresponding to the abnormal task type.

[0061] In the embodiment of the application, it can be determined whether the service cluster is overloaded at a regular time. Specifically, the number of remaining tasks in the exclusive message queue of each tenant in the service cluster and the remaining exclusive concurrency degree of the tenant are recorded to determine whether the service cluster is overloaded. The number of remaining tasks is the number of tasks accumulated in the exclusive message queue. If the number of remaining tasks of multiple exclusive message queues is in a state of exceeding a number threshold within a time threshold, it means that the exclusive message queue is in a long-term task accumulation state, and it can be determined that the service cluster is overloaded. The remaining exclusive concurrency degree refers to the difference between the exclusive concurrency degree set for the tenant and the number of tasks read from the exclusive message queue of the tenant. For example, the exclusive concurrency degree set for the tenant is 5, and the number of tasks read from the exclusive message queue of the tenant is 3, so the remaining exclusive concurrency degree is 2. The greater the remaining exclusive concurrency degree of each exclusive message queue, the more idle the exclusive message queue is. On the contrary, if the remaining exclusive concurrency degree of multiple exclusive message queues is 0 within a time threshold, it means that the exclusive message queue is in a long-term task accumulation state, and it can be determined that the service cluster is overloaded.

[0062] If the service cluster is overloaded, the execution time of each task can be collected, and the average execution time of tasks of the same task type can be counted. If the average execution time is greater than the standard execution time of tasks of the task type, it means that the service providing the task may be abnormal, and the task type can be determined as an abnormal task type. For example, the standard execution time of tasks of task type a is 10 seconds, and the average execution time of tasks of task type a is 20 seconds at this time, which means that the service providing the execution of tasks of task type a is abnormal, and task type a is an abnormal task type.

[0063] Therefore, a log can be output, and the task concurrency degree corresponding to the abnormal task type can be adjusted downward. Since the abnormal task type is a performance bottleneck of its backend service, adjusting the task concurrency degree corresponding to the abnormal task type and reducing the execution of tasks of the abnormal task type can help the performance of the backend service of the abnormal task type to recover.

[0064] In a possible implementation, the method for task scheduling of multi-tenancy isolation on the cloud provided by the embodiments of the present application can further include:

[0065] B1: determining whether there is workflow overload in the service cluster according to the remaining task quantity in the exclusive message queue of each tenant in the service cluster and the remaining exclusive concurrency degree of the tenant.

[0066] B2: if there is workflow overload, determining an abnormal tenant according to the task quantity corresponding to the tenant.

[0067] B3: adjusting the exclusive concurrency degree of the abnormal tenant.

[0068] In the embodiments of the present application, it can be determined whether there is workflow overload in the whole service cluster at a regular time. If there is workflow overload in the service cluster, the task quantity corresponding to each tenant can be counted, and if the task quantity corresponding to the tenant exceeds a threshold value, it means that the tenant has submitted too many task requests, and the tenant can be determined as an abnormal tenant.

[0069] Thus, the log can be output, and the exclusive concurrency degree corresponding to the abnormal tenant can be adjusted downward. The abnormal tenant means that the tenant can have requested a large number of tasks to cause workflow overload, and the exclusive concurrency degree of the tenant is adjusted downward to ensure that the execution time of the workflow task of other tenants does not increase significantly, thereby significantly improving the stability of the cloud computing service.

[0070] In addition, if the abnormal tenant determined in the last time is no longer an abnormal tenant in the process of determining again whether there is workflow overload in the service cluster, the exclusive concurrency degree of the tenant can also be adjusted upward. Similarly, if the abnormal task type determined in the last time is no longer an abnormal task type, the task concurrency degree corresponding to the task type can also be adjusted upward. Thus, the dynamic adjustment of the concurrency degree is realized.

[0071] Referring to Figure 4As shown, a schematic diagram of dynamically adjusting the concurrency is shown. When the performance bottleneck occurs in the service backend or the request burst occurs in part of the tenants, the scheduler can also dynamically adjust the concurrency according to the task execution. In the process of determining whether the service cluster is overloaded, the remaining task quantity in the exclusive message queue of each tenant and the remaining exclusive concurrency of the tenant can be used to determine whether the service cluster is overloaded. In the process of analyzing the abnormal source, the execution time of multiple tasks of the same task type can be used to determine the abnormal task type, and the task quantity corresponding to the tenant can be used to determine the abnormal tenant. After determining the workflow overload, the state of the workflow overload and the abnormal task type and the abnormal tenant can be recorded in the log. In the process of self-adaptive adjustment of the concurrency, the exclusive concurrency corresponding to the abnormal tenant and the task concurrency corresponding to the abnormal task type can be adjusted. Therefore, when the request burst or the service performance bottleneck causes the workflow overload, the concurrency can be dynamically adjusted according to the task execution.

[0072] In the working process of the scheduler and the executor, the workflow information of the first workflow also needs to be obtained from the memory. In a possible implementation manner, the cloud multi-tenant isolation task scheduling method provided by the embodiment of the present application can further include the following steps.

[0073] The workflow information of the first workflow is obtained from the multi-level cache based on the priority order of the multi-level cache.

[0074] In the embodiment of the present application, the memory uses the multi-level cache technology, and therefore the workflow information of the first workflow can be obtained from the multi-level cache when the workflow information is needed. In the process of obtaining the workflow information from the multi-level cache, the workflow information is first obtained from the local cache, then obtained from the centralized cache, and finally obtained from the persistent storage. The local cache and the persistent cache are included in each service node, and the centralized cache is included in the centralized cache service node. The local cache is used to store data that will not be modified within the life cycle of a single workflow, for example, including task parameters, workflow orchestration information, header information, and static task context information; the centralized cache stores the above data, and also stores workflow execution state information, dynamic task context data, and is responsible for data synchronization between multiple service nodes; the persistent storage is responsible for storing the data in the centralized cache to the local disk to prevent the loss of cache data.

[0075] In a possible implementation manner, the specific implementation of obtaining the workflow information of the first workflow from the multi-level cache based on the priority order of the multi-level cache can include the following steps.

[0076] C1: Obtain the workflow information of the first workflow from the local cache of the target service node.

[0077] C2: If the workflow information of the first workflow does not exist in the local cache of the target service node, obtain the workflow information of the first workflow from the centralized cache, and save the workflow information of the first workflow to the local cache of the target service node.

[0078] C3: If the workflow information of the first workflow does not exist in the centralized cache, obtain the workflow information of the first workflow from the persistent storage of the target service node, and save the workflow information of the first workflow to the local cache of the target service node.

[0079] Referring to Figure 5 As shown in the figure, a schematic diagram of the process of obtaining workflow information is shown. After requesting to obtain the workflow information of the first workflow, the local cache of the target service node is queried to determine whether the workflow information exists in the local cache. If the workflow information exists in the local cache, the workflow information is obtained. If the workflow information does not exist in the local cache, the centralized cache is queried to determine whether the workflow information exists in the centralized cache. If the workflow information exists in the local cache, the workflow information is obtained and saved to the local cache. If the workflow information does not exist in the centralized cache, the persistent storage is queried to obtain the workflow information, and the workflow information is saved to the local cache.

[0080] Compared with the use of a database to save workflow information, the use of multi-level cache to store and synchronize workflow information has higher efficiency in reading workflow information.

[0081] In the embodiment of the present application, after the current task (target task) of the first workflow is successfully executed, the execution state information of the executed task of the first workflow in the centralized cache needs to be modified. The tasks of a single workflow can be concurrently executed by different service nodes, and therefore different service nodes can simultaneously modify the execution state information in the centralized cache. Therefore, the counter uses an optimistic lock to ensure concurrent safety when modifying the execution state information. In one possible implementation, the cloud multi-tenant isolated task scheduling method provided by the embodiment of the present application can further include:

[0082] D1: After the current task of the first workflow is successfully executed, read the execution state information of the executed task of the first workflow from the centralized cache.

[0083] D2: Write the current task execution success information of the first workflow to the centralized cache.

[0084] D3: Read the execution state information of the executed task of the first workflow from the centralized cache again.

[0085] D4: If the execution status information of the executed tasks of the first workflow read twice is consistent, add the current task execution success information of the first workflow to the execution status information of the executed tasks of the first workflow.

[0086] D5: If the execution status information of the executed tasks of the first workflow read twice is inconsistent, repeat D1 to read the execution status information of the executed tasks of the first workflow from the centralized cache and the subsequent steps.

[0087] After the current task of the first workflow is executed successfully, for example, the task is a target task, the state of the target task execution success needs to be written into the centralized cache by the counter. The centralized cache stores the execution status information of the executed tasks of each workflow, and the execution status information of the executed tasks records which tasks in the workflow have been executed successfully. Before writing, the execution status information of the executed tasks of the first workflow is read from the centralized cache. For example, the target task is task 2, and the execution status information of the executed tasks of the first workflow read from the centralized cache is that task 1 is executed successfully. Then the target task execution success information (i.e. task 2 is executed successfully) is written into the centralized cache, and the execution status information of the executed tasks of the first workflow is read again. If the execution status information of the executed tasks is still that task 1 is executed successfully, it means that during the process of writing the target task execution success information, no other service node modifies the execution status information of the executed tasks, and then task 2 is executed successfully is added to the execution status information of the executed tasks of the first workflow.

[0088] If the execution status information of the executed tasks of the first workflow read twice is inconsistent, for example, the execution status information of the executed tasks read again is that task 1 is executed successfully and task 3 is executed successfully, it means that during the process of writing the target task execution success information, other service nodes write other task execution success information, and this writing fails. The execution status information of the executed tasks of the first workflow needs to be read from the centralized cache again, for example, the execution status information of the executed tasks is that task 1 is executed successfully and task 3 is executed successfully, the target task execution success information (i.e. task 2 is executed successfully) is written into the centralized cache, and the execution status information of the executed tasks of the first workflow is read again. If the execution status information is still that task 1 is executed successfully and task 3 is executed successfully, task 2 is executed successfully is added to the execution status information of the executed tasks of the first workflow, and at this time the execution status information of the executed tasks is that task 1 is executed successfully, task 2 is executed successfully and task 3 is executed successfully.

[0089] Referring to Figure 6As shown in the figure, a working process diagram of the counter is shown. After the executor informs the counter that the target task execution is successful, the counter reads the execution state information of the executed task of the first workflow from the centralized cache, and writes the target task execution success information into the centralized cache. It is judged whether the execution state information of the executed task of the first workflow is changed, that is, the execution state information of the executed task of the first workflow is read again. If the execution state information of the executed task of the first workflow does not change, the execution success information is added to the execution state information of the executed task of the first workflow. If the execution state information of the executed task of the first workflow changes, the execution state information of the executed task of the first workflow is re-read, the target task execution success information is written into the centralized cache, and the target task execution success information is added to the execution state information of the executed task of the first workflow.

[0090] Compared with the pessimistic lock of locking the centralized cache to prohibit other service nodes from writing before writing the target task execution success information into the centralized cache, the counter of the embodiment of the application adopts the optimistic lock mode, and allows other service nodes to operate the centralized cache when writing the target task execution success information. Therefore, the locking time is reduced, and the writing efficiency is improved. At the same time, by comparing the execution state information before and after writing, the safety of concurrent writing is ensured.

[0091] Referring to Figure 7 As shown in the figure, a working process diagram of a task scheduling method for multi-tenant isolation in the cloud according to an embodiment of the application is shown. The embodiment of the application can be applied to a workflow scheduling system, which can be composed of four modules: a memory, a scheduler, an executor and a counter.

[0092] In actual application, when a tenant requests a service provider to create, delete or schedule resources, the service provider will automatically generate multiple tasks according to the tenant's demand through the management and control software. The management and control software submits the tasks to the workflow scheduling system, and the workflow scheduling system generates a workflow according to the tasks, triggers the memory to save the workflow information composed of task-related information by using the cache and the persistent storage, and returns a response of successful saving to the management and control software.

[0093] The counter is created by the management and control software or the workflow scheduling system itself, and the counter returns a response of successful creation. The management and control software triggers the running of the workflow, the scheduler adds the tasks to the distributed message queue, and returns a response of successful addition.

[0094] The scheduler in each service node obtains a task in a workflow from a distributed message queue, obtains workflow information from a memory, and adds the task to a dedicated message queue of a tenant corresponding to the task. The task is read from the dedicated message queue of the tenant, and is executed by an executor after a concurrency check. The executor also needs to obtain workflow information from the memory before executing the task. After the executor completes execution of the task, a counter triggers modification of execution state information of the executed task of the workflow in a centralized cache.

[0095] After receiving feedback that the counter modification is successful, the executor notifies the scheduler to schedule a next task, which is added to a dedicated message queue of a tenant corresponding to the task. The task is read from the dedicated message queue of the tenant, and is executed by an executor after a concurrency check. The executor also needs to obtain workflow information from the memory before executing the task. After the executor completes execution of the task, a counter triggers modification of execution state information of the executed task of the workflow in a centralized cache.

[0096] The executor can also check completion of the workflow through the counter. After determining that the workflow is executed completely, the scheduler sets that the workflow is executed successfully, and cleans up the workflow information in the memory.

[0097] The embodiment of the application can greatly improve efficiency and fairness of workflow task scheduling of multiple tenants on a cloud. When a request burst of some tenants causes overload of a cloud computing service, dynamic scheduling can be performed to effectively ensure that execution time of workflow tasks of the remaining tenants does not increase significantly, and stability of the cloud computing service is significantly improved.

[0098] Based on the task scheduling method for multiple tenants on a cloud provided in the above method embodiment, the embodiment of the application further provides a task scheduling device for multiple tenants on a cloud. The device will be described below with reference to the accompanying drawings.

[0099] Referring to FIG. 8, Figure 8 As shown in the figure, the figure is a structural schematic diagram of a task scheduling device for multiple tenants on a cloud provided by an embodiment of the application. As shown in the figure, Figure 8 The task scheduling device for multiple tenants on a cloud includes:

[0100] The scheduling unit 801 is configured to receive a first workflow of a first tenant on a cloud, perform load balancing on multiple service nodes in a service cluster through a distributed message queue, and schedule a task of the first workflow to a target service node.

[0101] The storage unit 802 is configured to store the task of the first workflow to a dedicated message queue of the first tenant in the target service node.

[0102] The control unit 803 is configured to control execution of tasks of the first workflow by the target service node based on the exclusive concurrency degree configured for the first tenant and the exclusive message queue of the first tenant.

[0103] In a possible implementation, the apparatus further includes:

[0104] The first determination unit is configured to determine whether there is workflow overload in the service cluster according to the number of remaining tasks in the exclusive message queue of each tenant in the service cluster and the remaining exclusive concurrency degree of the tenant.

[0105] The second determination unit is configured to determine an abnormal task type according to execution times of multiple tasks of the same task type if there is workflow overload.

[0106] The first adjustment unit is configured to adjust the task concurrency degree corresponding to the abnormal task type.

[0107] In a possible implementation, the apparatus further includes:

[0108] The first determination unit is configured to determine whether there is workflow overload in the service cluster according to the number of remaining tasks in the exclusive message queue of each tenant in the service cluster and the remaining exclusive concurrency degree of the tenant.

[0109] The third determination unit is configured to determine an abnormal tenant according to the number of tasks corresponding to the tenant if there is workflow overload.

[0110] The second adjustment unit is configured to adjust the exclusive concurrency degree of the abnormal tenant.

[0111] In a possible implementation, the control unit is specifically configured to:

[0112] read tasks from the exclusive message queue of the first tenant in turn by the target service node based on the exclusive concurrency degree configured for the first tenant and the task concurrency degree corresponding to each task type, so that the total number of read tasks is less than or equal to the exclusive concurrency degree configured for the first tenant, and the task concurrency degree corresponding to each task type is met.

[0113] execute the read tasks of the first workflow by the target service node.

[0114] In a possible implementation, the apparatus further includes:

[0115] The acquisition unit is configured to acquire workflow information of the first workflow from the multi-level cache based on a priority order of the multi-level cache; the multi-level cache includes a local cache, a centralized cache, and a persistent storage sorted in descending order of priority.

[0116] In a possible implementation, the obtaining unit is specifically configured to:

[0117] obtain the workflow information of the first workflow from a local cache of the target service node;

[0118] if the workflow information of the first workflow does not exist in the local cache of the target service node, obtain the workflow information of the first workflow from a centralized cache, and save the workflow information of the first workflow into the local cache of the target service node;

[0119] if the workflow information of the first workflow does not exist in the centralized cache, obtain the workflow information of the first workflow from a persistent storage of the target service node, and save the workflow information of the first workflow into the local cache of the target service node.

[0120] In a possible implementation, the apparatus further includes:

[0121] a reading unit configured to read, after the current task of the first workflow is executed successfully, execution state information of the executed task of the first workflow from the centralized cache;

[0122] a writing unit configured to write the current task execution success information of the first workflow into the centralized cache;

[0123] the reading unit is further configured to read again the execution state information of the executed task of the first workflow from the centralized cache;

[0124] an adding unit configured to add, if the execution state information of the executed task of the first workflow read twice is consistent, the current task execution success information of the first workflow to the execution state information of the executed task of the first workflow;

[0125] a triggering unit configured to trigger, if the execution state information of the executed task of the first workflow read twice is inconsistent, the reading unit to repeatedly execute reading the execution state information of the executed task of the first workflow from the centralized cache.

[0126] In addition, the embodiments of the present application further provide a computer program product, which includes computer program instructions, and when the computer program instructions run on a computer, the computer is caused to execute the task scheduling method for cloud multi-tenant isolation as described in any one of the above.

[0127] Based on the cloud multi-tenant isolated task scheduling method provided in the above method embodiments, the application further provides an electronic device, comprising: one or more processors; a storage device having one or more programs stored thereon, when the one or more programs are executed by the one or more processors, the one or more processors implement the cloud multi-tenant isolated task scheduling method according to any of the above embodiments.

[0128] Reference will now be made to the following description Figure 9 , which shows a structural diagram of an electronic device 1300 suitable for implementing the embodiments of the application. The terminal device in the embodiments of the application can include, but is not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (Personal Digital Assistant), PADs (portable android devices), PMPs (Portable Media Player), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), and the like, and fixed terminals such as digital TVs, desktop computers, and the like. Figure 9 The electronic device shown is only an example and should not impose any limitation on the functions and use range of the embodiments of the application.

[0129] As shown in Figure 9 , the electronic device 1300 can include a processing device (such as a central processor, a graphics processor, etc.) 1301, which can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 1302 or programs loaded from a storage device 1306 into a random access memory (RAM) 1303. In the RAM 1303, various programs and data required for the operation of the electronic device 1300 are also stored. The processing device 1301, the ROM 1302, and the RAM 1303 are connected to each other through a bus 1304. An input / output (I / O) interface 1305 is also connected to the bus 1304.

[0130] Generally, the following devices can be connected to the I / O interface 1305: input devices 1306 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, and the like; output devices 1307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, and the like; storage devices 1306 including, for example, a magnetic tape, a hard disk, and the like; and communication devices 1309. The communication devices 1309 can allow the electronic device 1300 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 9The electronic device 1300 is shown with various elements, but it will be understood that not all of these elements are required to implement or be present in every electronic device. More or fewer elements can alternatively be implemented.

[0131] In particular, the processes described above with reference to the flow charts can be implemented as a computer software program according to embodiments of the present application. For example, embodiments of the present application include a computer program product comprising a computer program carried on a non-transitory computer readable medium, the computer program comprising program code for performing the methods illustrated in the flow charts. In such embodiments, the computer program can be downloaded and installed from a network via the communication device 1309, or installed from the storage device 1306, or installed from the ROM 1302. When the computer program is executed by the processing device 1301, the above-mentioned functions defined in the methods of embodiments of the present application are performed.

[0132] The electronic device provided by the embodiments of the present application and the task scheduling method for multi-tenant isolation on cloud provided by the above-mentioned embodiments belong to the same inventive concept, and the technical details not described in detail in the present embodiment can be referred to the above-mentioned embodiments, and the present embodiment has the same beneficial effects as the above-mentioned embodiments.

[0133] Based on the task scheduling method for multi-tenant isolation on cloud provided by the above-mentioned method embodiments, the present embodiment provides a computer readable medium, which stores a computer program, wherein the program is executed by a processor to implement the task scheduling method for multi-tenant isolation on cloud as described in any of the above-mentioned embodiments.

[0134] It should be noted that the computer readable medium in the embodiments of the present application can be a computer readable signal medium or a computer readable storage medium or any combination of the two. The computer readable storage medium may, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any combination of the above. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the embodiments of the present application, the computer readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device or apparatus. In the embodiments of the present application, the computer readable signal medium can include a data signal carried in a baseband or as a part of a carrier wave, which carries computer readable program code. Such a propagated data signal can take various forms, including but not limited to an electromagnetic signal, an optical signal or any suitable combination of the above. The computer readable signal medium can also be any computer readable medium other than the computer readable storage medium, which can send, propagate or transmit a program for use by or in conjunction with an instruction execution system, device or apparatus. The program code contained in the computer readable medium can be transmitted by any suitable medium, including but not limited to a wire, a cable, a RF (radio frequency) or the like, or any suitable combination of the above.

[0135] In some embodiments, the client, server, or both can communicate using any current known or future developed network protocol, such as HTTP (HyperText Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include local area networks ("LAN"), wide area networks ("WAN"), the Internet, and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any current known or future developed networks.

[0136] The computer readable medium described above can be included in the electronic device described above; or can exist separately, without being assembled into the electronic device.

[0137] The computer readable medium described above carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the task scheduling method for multi-tenant isolation on the cloud described above.

[0138] Computer program code for carrying out operations of embodiments of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0139] The computer program instructions can also be loaded onto a computer or other programmable information processing apparatus to cause a series of operations to be performed on the computer or other programmable information processing apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable information processing apparatus implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0140] The units involved in the embodiments described in the present application can be implemented by software, or can be implemented by hardware. Among them, the name of the unit / module does not constitute a limitation to the unit itself in some cases.

[0141] The functions described in the above description herein can be performed at least in part by one or more hardware logic components. For example, and without limitation, illustrative types of hardware logic components that can be used include Field-programmable Gate Arrays (FPGAs), Program-specific Integrated Circuits (ASICs), Program-specific Standard Products (ASSPs), System-on-a-chip systems (SOCs), Complex Programmable Logic Devices (CPLDs), etc.

[0142] In the context of the embodiments of the present application, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include but not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium will include one or more of the following: an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0143] It should be noted that the various embodiments described in the specification are progressive, and each embodiment focuses on the differences from other embodiments. The same and similar parts between embodiments can be mutually referred to. For the system or device disclosed by the embodiments, since it corresponds to the method disclosed by the embodiments, the description is relatively simple, and the relevant parts can be referred to the method part.

[0144] It should be understood that in the present application, "at least one" means one or more, and "multiple" means two or more. "And / or" is used to describe the association relationship between the associated objects, which means that there can be three relationships, for example, "A and / or B" can represent three cases: only A, only B, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally represents that the associated objects before and after are in an "or" relationship. "At least one of the following" or similar expressions means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0145] It is also to be noted that, as used in the specification and the appended claims, the singular forms "a," "an" and "the" include plural referents unless otherwise indicated. Furthermore, to the extent that the terms "including," "includes," "having," "has," "with," or "contains" are used in either the detailed description and the claims, such terms are intended to be inclusive in a manner similar to the term "comprising" as an open transition term without precluding any additional or other elements.

[0146] The embodiments disclosed herein can each be implemented as a method, apparatus, or article of manufacture using programming instructions. The embodiments disclosed herein can be implemented using software, firmware, hardware, or a combination thereof. The various elements of the disclosed embodiments, as well as the procedural aspects of the disclosed embodiments, can be implemented using a variety of programming instructions, software, firmware, or the like. In addition, one or more of the disclosed embodiments can be implemented by a computer system having a processor and a memory, where the memory stores programming instructions for execution by the processor. The programming instructions can be stored in a variety of ways, including on a computer diskette, on a computer hard drive, on a computer memory, on a computer tape, or on other computer storage devices.

[0147] The above description of disclosed embodiments provides enough information to enable one of ordinary skill in the art to practice the application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein can be applied to other embodiments without departing from the spirit or scope of the application. Accordingly, the application is not to be restricted based on the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A task scheduling method for multi-tenant isolation in the cloud, characterized in that, The method includes: The system receives the first workflow from the first tenant in the cloud, performs load balancing on multiple service nodes in the service cluster through a distributed message queue, and schedules the tasks of the first workflow to the target service node. In each service node, a dedicated message queue is set up for each tenant, and the tasks of different tenants are isolated during scheduling. The tasks of the first workflow are stored in the dedicated message queue of the first tenant in the target service node; The target service node controls the task execution of the first workflow based on the dedicated concurrency level configured for the first tenant and the dedicated message queue of the first tenant; The workflow information of the first workflow is obtained from the multi-level cache based on the priority order of the multi-level cache, which includes a local cache, a centralized cache, and persistent storage sorted in descending order of priority. The step of controlling the task execution of the first workflow through the target service node based on the dedicated concurrency level configured for the first tenant and the dedicated message queue of the first tenant includes: The target service node reads tasks sequentially from the first tenant's dedicated message queue based on the dedicated concurrency level configured for the first tenant and the task concurrency level corresponding to each task type, such that the total number of tasks read is less than or equal to the dedicated concurrency level configured for the first tenant, and satisfies the task concurrency level corresponding to each task type. The task of the first workflow read is executed through the target service node.

2. The method according to claim 1, characterized in that, The method further includes: Based on the number of remaining tasks in the dedicated message queue of each tenant in the service cluster and the remaining dedicated concurrency of the tenant, determine whether the service cluster is overloaded in workflow. If workflow overload exists, determine the abnormal task type based on the execution time of multiple tasks of the same task type; Adjust the task concurrency corresponding to the abnormal task type.

3. The method according to claim 1, characterized in that, The method further includes: Based on the number of remaining tasks in the dedicated message queue of each tenant in the service cluster and the remaining dedicated concurrency of the tenant, determine whether the service cluster is overloaded in workflow. If workflow overload exists, determine the abnormal tenants based on the number of tasks corresponding to the tenants; Adjust the dedicated concurrency level for the abnormal tenant.

4. The method according to claim 1, characterized in that, The step of retrieving workflow information of the first workflow from the multi-level cache based on the priority order of the multi-level cache includes: Obtain the workflow information of the first workflow from the local cache of the target service node; If the workflow information of the first workflow does not exist in the local cache of the target service node, the workflow information of the first workflow is obtained from the centralized cache and saved to the local cache of the target service node. If the workflow information of the first workflow is not present in the centralized cache, the workflow information of the first workflow is retrieved from the persistent storage of the target service node and saved to the local cache of the target service node.

5. The method according to claim 1, characterized in that, The method further includes: After the current task of the first workflow is successfully executed, the execution status information of the executed tasks of the first workflow is read from the centralized cache; Write the current task execution success information of the first workflow into the centralized cache; The execution status information of the executed tasks of the first workflow is read again from the centralized cache; If the execution status information of the executed tasks of the first workflow read twice is consistent, the current task execution success information of the first workflow is added to the execution status information of the executed tasks of the first workflow. If the execution status information of the executed tasks of the first workflow read twice is inconsistent, repeat the process of reading the execution status information of the executed tasks of the first workflow from the centralized cache and the subsequent steps.

6. A cloud-based multi-tenant isolated task scheduling device, characterized in that, The device includes: The scheduling unit is used to receive the first workflow of the first tenant in the cloud, perform load balancing on multiple service nodes in the service cluster through a distributed message queue, and schedule the tasks of the first workflow to the target service node. In each service node, a dedicated message queue is set up for each tenant, and the tasks of different tenants are isolated during scheduling. A storage unit is used to store the tasks of the first workflow into the dedicated message queue of the first tenant in the target service node; The control unit is used to control the task execution of the first workflow through the target service node based on the dedicated concurrency level configured for the first tenant and the dedicated message queue of the first tenant; The acquisition unit is used to acquire the workflow information of the first workflow from the multi-level cache based on the priority order of the multi-level cache; the multi-level cache includes a local cache sorted in descending order of priority, a centralized cache, and persistent storage; The control unit is specifically used for: The target service node reads tasks sequentially from the first tenant's dedicated message queue based on the dedicated concurrency level configured for the first tenant and the task concurrency level corresponding to each task type, such that the total number of tasks read is less than or equal to the dedicated concurrency level configured for the first tenant, and satisfies the task concurrency level corresponding to each task type. The task of the first workflow read is executed through the target service node.

7. A cloud-based multi-tenant isolated task scheduling device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements the task scheduling method for multi-tenant isolation in the cloud as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed on a terminal device, cause the terminal device to perform the task scheduling method for multi-tenant isolation in the cloud as described in any one of claims 1-5.

Images