Resource management method and device, equipment and storage medium

Abstract

The embodiment of the invention provides a resource management method and device, equipment and a storage medium. The method includes creating a first sub-container in a host container using a portion of resources included in the host container, where a plurality of workflows are executed in the host container, and the resources allocated to the first sub-container are isolated from the resources in the host container that are not allocated to the first sub-container; distributing a first workflow in the plurality of workflows to the first sub-container; and executing the first workflow by using the first sub-container. In this way, an isolated execution environment is provided for execution tasks of different workflows, and it is ensured that each workflow can operate independently. The isolation mechanism can improve the stability and safety of the system and effectively avoid the influence of the abnormal workflow on other parts of the system.

Description

Method, device, apparatus and storage medium for resource management TECHNICAL FIELD

[0001] Example embodiments of the present disclosure generally relate to the field of computers, and in particular, to a method, device, apparatus and computer readable storage medium for resource management. BACKGROUND

[0002] With the rapid development of cloud computing and distributed technology, more and more applications and workloads are deployed in cloud platform environments. For example, in a cloud service platform, usually multiple workflows are running simultaneously. These workflows need effective resource management and scheduling mechanisms, especially in the case of resource competition and dynamic load changes, to ensure system stability and performance.

[0003] In the related art, an exception caused by a certain workflow can affect the execution of all other workflows on the same engine. At the same time, a hotspot workflow that consumes more resources can also preempt the resources of the engine, which can also affect the normal operation of other workflows. Therefore, how to solve the problem of workflow exception or resource preemption affecting other workflows in the entire system is worth attention.

[0004] SUMMARY

[0005] In a first aspect of the present disclosure, a method for resource management is provided. The method comprises: creating a first sub-container in a host container by using a part of resources included in the host container, wherein a plurality of workflows are executed in the host container, and the resources allocated to the first sub-container are isolated from the resources in the host container that are not allocated to the first sub-container; allocating a first workflow in the plurality of workflows to the first sub-container; and executing the first workflow by using the first sub-container.

[0006] In a second aspect of the present disclosure, an apparatus for resource management is provided. The apparatus comprises: a creating module configured to create a first sub-container in a host container by using a part of resources included in the host container, wherein a plurality of workflows are executed in the host container, and the resources allocated to the first sub-container are isolated from the resources in the host container that are not allocated to the first sub-container; an allocating module configured to allocate a first workflow in the plurality of workflows to the first sub-container; and an executing module configured to execute the first workflow by using the first sub-container.

[0007] In a third aspect of the disclosure, an electronic device is provided. The device includes at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit. The instructions, when executed by the at least one processing unit, cause the device to perform the method of the first aspect.

[0008] In a fourth aspect of the disclosure, a computer-readable storage medium is provided. The computer-readable storage medium has stored thereon a computer program, which is executable by a processor to implement the method of the first aspect.

[0009] It should be understood that the contents described in this section are not intended to limit the key features or important features of the embodiments of the disclosure, nor are they used to limit the scope of the disclosure. Other features of the disclosure will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0010] The above and other features, advantages, and aspects of embodiments of the present disclosure will become more apparent by describing in detail exemplary embodiments thereof with reference to the attached drawings in which:

[0011] FIG. 1 shows a schematic diagram of an example environment in which embodiments according to the present disclosure can be implemented;

[0012] FIG. 2 shows a flowchart of an example process of resource management according to some embodiments of the present disclosure;

[0013] FIG. 3A shows a schematic diagram of a deployment of resource management according to some embodiments of the present disclosure;

[0014] FIG. 3B shows a schematic diagram of an architecture of resource management according to some embodiments of the present disclosure;

[0015] FIG. 3C shows a schematic diagram of a resource limit policy according to some embodiments of the present disclosure;

[0016] FIG. 3D shows a schematic diagram of an example process of limiting resource consumption of a sub-container according to some embodiments of the present disclosure;

[0017] FIG. 4 shows a schematic structural block diagram of an example apparatus for resource management according to some embodiments of the present disclosure; and

[0018] FIG. 5 shows a block diagram of an electronic device capable of implementing a plurality of embodiments of the present disclosure. DETAILED DESCRIPTION

[0019] It can be understood that, before using the technical solutions disclosed in the embodiments of the present disclosure, the type, use range, use scenario, etc. of the personal information involved in the present disclosure should be informed to the user and the authorization of the user should be obtained through appropriate means according to relevant laws and regulations.

[0020] For example, in response to receiving an active request of a user, prompt information is sent to the user to explicitly prompt the user that the operation requested to be performed will require obtaining and using personal information of the user. Thus, the user can autonomously select whether to provide personal information to the software or hardware such as an electronic device, an application program, a server or a storage medium, etc. performing the operation of the technical solutions of the present disclosure according to the prompt information.

[0021] As an optional but non-limiting implementation manner, in response to receiving an active request of a user, the manner of sending prompt information to the user may, for example, be a pop-up window manner, in which the prompt information can be presented in a text manner. In addition, the pop-up window can also carry selection controls for the user to select "agree" or "disagree" to provide personal information to the electronic device.

[0022] It can be understood that the above notification and obtaining of user authorization process is only illustrative, and does not limit the implementation manner of the present disclosure, and other manners meeting relevant laws and regulations can also be applied to the implementation manner of the present disclosure.

[0023] It can be understood that the data (including but not limited to the data itself, the acquisition or use of the data) involved in the present technical solutions should comply with the requirements of relevant laws and regulations and relevant provisions.

[0024] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms, and should not be interpreted as being limited to the embodiments set forth herein, on the contrary, these embodiments are provided to make the present disclosure more thorough and complete. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes, and are not intended to limit the scope of protection of the present disclosure.

[0025] It should be noted that the titles of any section / subsection provided herein are not limiting. Various embodiments are described throughout this document, and any type of embodiment can be included under any section / subsection. Furthermore, embodiments described in any section / subsection can be combined with any other embodiment described in the same section / subsection and / or a different section / subsection in any manner.

[0026] In the description of embodiments of the disclosure, the term "includes" and its similar terms are understood to be open-ended, i.e., "including but not limited to". The term "based on" is understood to be "based, at least in part, on". The term "one embodiment" or "the embodiment" is understood to be "at least one embodiment". The term "some embodiments" is understood to be "at least some embodiments". Other explicit and implicit definitions can also be included below. The terms "first", "second", etc. can refer to different or the same objects. Other explicit and implicit definitions can also be included below.

[0027] As mentioned above, in cloud computing, big data processing, business logic execution, etc. scenarios, various workflows initiated by users are usually run on execution engines. The execution engine usually refers to a component responsible for executing workflow or application code, responsible for scheduling and executing specific business logic and processes. In the current complex computing environment and application scenarios, multiple workflows are usually run in parallel on the same execution engine based on the same resources. These workflows may affect each other, especially in the absence of resource isolation and security technology, which will cause the system to face greater risks in stability, capacity, security, etc. Several typical scenarios are described below.

[0028] One typical scenario involves the problem of engine crash caused by abnormal workflow. Specifically, when an abnormal workflow occurs, it may cause the execution engine to crash. In this case, all other workflows running on the engine will also be affected and exit abnormally. This overall system failure exposes the problem of lack of isolation mechanism between processes.

[0029] Another typical scenario involves the security risk of running user-defined code. Specifically, running user-defined code in the engine poses a significant security risk. For example, user code may damage the file system or running environment, causing the engine to malfunction, and even threatening the security of the entire system. This indicates that the system has vulnerabilities in file system and environment isolation.

[0030] Another typical scenario involves the problem of hot workflow preemption of system resources. Specifically, some workflows may consume excessive system resources due to high access volume or high computing demand, causing the running of other workflows to be affected. This problem is particularly prominent in scenarios where system resources are highly competitive, indicating the inadequacy of related technologies in resource management.

[0031] Traditionally, a common resource isolation solution is based on Docker containerization technology. Docker containerization technology provides resource control and isolation by using control group mechanism (Cgroup) and namespace mechanism (Namespace). However, in many business scenarios, resource isolation of a workflow is the internal logic of a process service manager (PSM). This internal logic cannot directly utilize Docker scheduling at the platform as a service (PaaS) level to implement resource isolation. In other words, resource isolation of a workflow belongs to the requirement of the internal business logic layer of an application or a service, and container scheduling and resource isolation at the PaaS level are resource isolation and allocation between different container instances of a cloud platform, which cannot meet the fine-grained isolation requirement in specific business scenarios.

[0032] In addition, to implement resource isolation between workflows based on Docker containerization technology, a container instance needs to be deployed for each Workflow. However, this will cause resource waste, especially in a high-concurrency distributed scenario. Moreover, when each container instance is started, an image needs to be pulled from an image library, and resources need to be allocated to the container. Frequent container pulling and starting operations will further aggravate performance load.

[0033] Therefore, embodiments of the present disclosure provide a solution for resource management. The solution includes: creating a first sub-container in a host container by using a part of resources included in the host container, wherein a plurality of workflows are executed in the host container, and the resources allocated to the first sub-container are isolated from the resources in the host container that are not allocated to the first sub-container; allocating a first workflow in the plurality of workflows to the first sub-container; and executing the first workflow by using the first sub-container.

[0034] In this way, embodiments of the present disclosure avoid the resource consumption problem of allocating an independent container instance for each workflow by introducing a lightweight multi-process mechanism to allocate a corresponding sub-container for a workflow. An isolated execution environment is provided for execution tasks of different workflows, ensuring that each workflow can be independently run. This isolation mechanism can improve the stability and security of the system and effectively avoid the influence of an abnormal workflow on other parts of the system.

[0035] Various example implementations of the solution are described in detail below in further conjunction with the accompanying drawings.

[0036] Example Environment

[0037] FIG. 1 shows a schematic diagram of an example environment 100 in which embodiments of the present disclosure can be implemented. As shown in FIG. 1, the example environment 100 can include an electronic device 110.

[0038] In this example environment 100, electronic devices 110 communicate with a cloud service platform 120 to collaboratively complete workflows. For example, a user 140 can initiate a workflow via an electronic device 110 and / or an attached device of the electronic device 110. The initiated workflow can be executed and managed at the cloud service platform 120. As an example, FIG. 1 illustrates workflows 130-1, 130-2, 130-N, which are collectively referred to as or individually referred to as workflows 130. The number of workflows illustrated in FIG. 1 is merely exemplary and is not intended to be limiting in any way.

[0039] The cloud service platform 120 is used for resource allocation and management of the workflows 130. The cloud service platform 120 can be a container engine environment deployed on various devices such as cloud servers, edge computing nodes, mainframes, etc., which is capable of at least providing underlying computing resources to the user 140 to schedule and execute the workflows 130 initiated by the user 140.

[0040] In some embodiments, a workflow 130 refers to a series of tasks or steps executed in sequence or in parallel, typically to accomplish a complex operation or business process. The workflow 130 can be a certain computing task or application logic that needs to be efficiently executed and managed in the cloud service platform 120.

[0041] The electronic devices 110 can be any type of mobile terminal, fixed terminal, or portable terminal including a mobile handset, a desktop computer, a laptop computer, a notebook computer, a subnotebook computer, a netbook computer, a tablet computer, a media computer, a mainframe computer, a server computer, a personal digital assistant (PDA), a television receiver, a radio broadcast receiver, an electronic book device, a game device, or any combination thereof, including accessories and peripherals of these devices, or any combination thereof. In some embodiments, the terminal device 110 can also support any type of interface to a user (such as "wearable" circuitry, etc.).

[0042] It should be understood that the structures and functions of the various elements in the environment 100 are described for illustrative purposes only and are not intended to imply any limitation on the scope of the present disclosure.

[0043] Some example embodiments of the present disclosure will be hereinafter described with continued reference to the drawings.

[0044] Example process

[0045] FIG. 2 illustrates a flowchart of an example process 200 of resource management according to some embodiments of the present disclosure. The process 200 can be implemented at the cloud service platform 120. The process 200 is described below with reference to FIG. 1.

[0046] As shown in FIG. 2, at block 210, the cloud service platform 120 creates a first child container in a host container with a portion of resources included in the host container. The host container refers to a container environment allocated by the cloud service platform 120 to support execution of a plurality of workflows 130. For example, the host container can be a physical machine, a virtual machine, or a container instance in a cloud platform responsible for hosting a plurality of workflows 130 and resources required for execution of the workflows 130. The workflow 130 refers to a series of tasks or steps executed in sequence or in parallel, for example, the workflow 130 can be a certain computing task or application logic.

[0047] In some embodiments, the host container can host execution of a plurality of workflows 130 at the same time. Each workflow requires consumption of corresponding resources in the host container when executed. In some embodiments, there are a plurality of workflows executed in the host container, and the resources allocated to the first child container are isolated from the resources in the host container that are not allocated to the first child container.

[0048] As an example, to avoid a situation where an exception of a certain workflow causes the entire system to crash or execution of other workflows in the host container to be blocked, the cloud service platform 120 can create an isolated environment, i.e., a child container, within the host container for execution of the specific workflow. Each child container can be regarded as an independent, lightweight execution environment that contains all resources and dependencies required for execution of the workflow.

[0049] In some embodiments, the cloud service platform 120 can create a plurality of corresponding child containers based on a plurality of workflows included. The cloud service platform 120 can allocate resources of the host container to the plurality of child containers on demand, and each child container can be allocated specific resources, so as to ensure that each child container is isolated from the resources in the host container that are not allocated to it.

[0050] In some embodiments, the cloud service platform 120 determines the resources of the host container allocated to the first child container according to a control group mechanism. Based on the resources of the host container allocated to the first child container, the cloud service platform 120 creates the first child container. The control group (Cgroup) mechanism is a resource management mechanism for limiting, recording, and isolating resource usage (such as processing, memory, disk, etc.) of a process group. Through the Cgroup mechanism, the cloud service platform 120 can manage and monitor allocation of resources, and ensure that resources allocated to different child containers can be called in isolation.

[0051] According to the control group mechanism, it is determined that the resources allocated to the first sub-container in the resources of the host container can provide the work flow granularity resource isolation capability, so that different work flows can be safely and stably run in the same host container without interfering with each other. In addition, by dynamically adjusting the resource quota of the sub-container, the resource utilization and system stability are improved.

[0052] With reference back to FIG. 2, at block 220, the cloud service platform 120 allocates a first work flow in the plurality of work flows to a first sub-container. For example, by allocating the first work flow to the first sub-container, the first work flow can implement a mechanism similar to sandbox running, that is, the first work flow can call the resources allocated to the first sub-container in the host container in isolation. Resource isolation between different work flows is achieved.

[0053] At block 230, the cloud service platform 120 executes the first work flow with the first sub-container. For example, when a certain work flow needs to be executed, the cloud service platform 120 will allocate the first work flow to be executed to the corresponding first sub-container. In the first sub-container, the cloud service platform 120 can start one or more sub-processes responsible for the execution of specific work flow tasks. This sub-process corresponds to the allocated first work flow and undertakes the logic of executing the work flow.

[0054] As an example, FIG. 3A shows a schematic diagram of a deployment mode 300A of resource management according to some embodiments of the present disclosure. As shown in FIG. 3A, one container instance in the cloud service platform, i.e., the host container 311, includes a plurality of work flows: a first work flow, a second work flow, a third work flow,..., an Nth work flow, etc. These work flows are executed directly in the host container 311 without independent isolation mechanism. All work flows share one large running environment, in which case if one of the work flows goes wrong (such as crashes or excessive resource occupation), it can affect the execution of other work flows, and even cause the engine service corresponding to the entire host container to be unavailable.

[0055] With reference back to FIG. 3A, the host container 312 is one container instance in the cloud service platform deployed based on the resource management mode in the embodiments of the present disclosure. In the host container 312, the cloud service platform 120 creates a plurality of sub-containers in the host container 312 using the resources included in the host container 312, and the resources allocated to each sub-container are isolated from the resources allocated to other sub-containers. Then, the cloud service platform 120 allocates each work flow to be executed to the corresponding sub-container. The work flow in each container can be executed in its isolated container environment, thereby ensuring that the abnormality of one of the work flows does not affect the normal execution of other work flows in the entire host environment 312.

[0056] Since multiple workflows are supported to be executed synchronously or asynchronously in the same host container, the cloud service platform 120 needs to manage multiple workflows and corresponding sub-containers. In some embodiments, the cloud service platform 120 can also set an agent process on top of each sub-process based on a multi-process service to execute the resource management method in the embodiments of the present disclosure to be responsible for managing and coordinating the execution of multiple workflows. Such an agent process can also be referred to as a parent process.

[0057] As an example, FIG. 3B shows a schematic diagram of an architecture 300B of resource management according to some embodiments of the present disclosure. As shown in FIG. 3B, the cloud service platform 120 can be responsible for creating new sub-containers according to the needs of the workflows through the parent process 320, and assigning the corresponding workflows to the corresponding sub-containers.

[0058] In some embodiments, the cloud service platform 120 can manage and coordinate the execution of multiple workflows through the parent process 320. The example operations of the parent process 320 are described below.

[0059] As an example, the parent process 320 can be used for process management. The parent process 320 can manage the life cycle of the sub-containers in the host container and allocate and release the resources corresponding to the sub-containers, each of which is used to execute a specific workflow and a corresponding sub-process. By dynamically managing the sub-containers, efficient use of system resources is ensured.

[0060] As another example, the parent process 320 can also be used for routing management. The parent process 320 can maintain a routing table, so that the execution tasks of each workflow are correctly assigned to the corresponding sub-containers. Through this mechanism, it is ensured that the execution data corresponding to the workflow can be correctly forwarded to the corresponding sub-containers and can be adjusted as needed.

[0061] In some embodiments, the cloud service platform 120 updates the routing table for the host container to indicate that the execution task corresponding to the first workflow is mapped to the first sub-container. As an example, after the cloud service platform 120 creates the corresponding first sub-container for the first workflow to be executed, it can update the routing table to bind the corresponding relationship between the first workflow and the first sub-container. In this way, it is ensured that the execution data related to the first workflow can be correctly forwarded to the corresponding first sub-container. At the same time, when the first workflow completes the task or the execution ends, the cloud service platform 120 can destroy the first sub-container and correspondingly update the routing table to release resources.

[0062] In the embodiments of the present disclosure, the dynamic management of the routing table ensures the correct distribution and scheduling of tasks, further optimizes the management and execution efficiency of the workflow, and improves the maintainability and scalability of the system.

[0063] With continued reference to FIG. 3B, as another example, the parent process 320 can be used to implement a base isolation capability. The cloud service platform 120 can determine resources allocated to each sub-container in the host container, thereby providing an isolation function for each sub-container, ensuring that the file system, memory, and other resources of each sub-container are independent of each other, avoiding interference with each other.

[0064] In some embodiments, the resources allocated to the first sub-container include at least one of: a memory resource, a computing resource, or a file system resource.

[0065] As an example, the memory resource can include, for example, memory, and the like, for storing data and instructions needed during execution of a workflow. The cloud service platform 120 can allocate memory resources to the corresponding sub-container through a Cgroup mechanism, and can set an upper limit on the memory resources to prevent a certain workflow from occupying too much memory, thereby affecting the execution of other workflows.

[0066] The computing resource can include, for example, a central processing unit, and the like, responsible for executing instructions and operations in a workflow. The cloud service platform 120 can set a computing resource usage limit for the corresponding sub-container, for example, a certain number of processor cores or usage time quota can be allocated to the sub-container.

[0067] As another example, the file resource can include resources responsible for managing files, data, and applications required by a workflow. In actual application scenarios, the execution of a workflow can involve read operations on the file system. The cloud service platform 120 can create a corresponding independent bundle file directory for the sub-container to ensure that file operations between workflows do not interfere with each other, thereby improving the security and stability of the system.

[0068] In the embodiments of the present disclosure, by allocating corresponding memory resources, computing resources, or file system resources to the first sub-container, an isolated execution environment is provided for the execution tasks of different workflows, ensuring that each workflow can run independently, thereby ensuring the stability of the host container under high load or abnormal conditions.

[0069] With continued reference to FIG. 3B, as another example, the parent process 320 can also be used to implement resource monitoring. The cloud service platform 120 can periodically monitor the resource consumption (such as CPU, memory, and the like) of each sub-container to ensure that system resources are reasonably utilized, and to avoid a certain container occupying too many resources affecting the overall performance.

[0070] As another example, the parent process 320 can also be used to implement resource limits based on the resource monitoring. Through static and dynamic resource allocation strategies, the resource usage of the containers is controlled. For example, including static strategies by pre-setting resource load limits (quota), and by the parent process 320 issuing a quota parameter, and dynamic strategies that can adjust resource allocation according to real-time monitoring, limit the resource usage of the child containers that occupy too much resources, and avoid overload to affect other child containers.

[0071] With continued reference to FIG. 3B, in the architecture 300B, each child container (e.g., the first child container 321 and the second child container 322) can communicate with the parent process 320 through a process communication mechanism, such as a unix socket. In this way, the child containers can receive instructions from the parent process 320 through, for example, a socket, while the parent process 320 can also monitor the running state and resource usage of the child containers through the socket.

[0072] In some application scenarios, there can be a large number of normal workflows 130 in the cloud service platform 120, and only a small number of workflows 130 are resource-intensive or abnormal workflows (such as workflows that require a large amount of computation, memory, or are abnormally interrupted due to program errors). In this case, the cloud service platform 120 can also set a corresponding threshold for the host container, and perform resource management based on the relationship between the resource load of the host container and the set threshold, so as to effectively balance the resource utilization and the stability of the system.

[0073] As an example, FIG. 3C illustrates a schematic diagram of a resource limiting strategy 300C according to some embodiments of the present disclosure. In some embodiments, the cloud service platform 120 determines whether the resource load of the host container is greater than a first threshold. In response to the resource load being greater than the first threshold, the cloud service platform 120 creates a first child container in the host container.

[0074] As shown in FIG. 3C, a plurality of normal workflows can run in the same host container and share the resources of the host container. This means that in the initial stage, the cloud service platform 120 can concentrate them in one container instance for running, thereby fully utilizing the resources of the host container. The cloud service platform 120 can determine the resource load of the host container in real time or based on a set time interval. When it is determined that the resource load of the host container reaches a set first threshold 333 (also referred to as a soft line), the cloud service platform 120 creates a first child container 331 in the host container, and the resources allocated to the first child container 331 are isolated from the resources in the host container that are not allocated to the first child container. In this way, both the resource utilization and the stability and performance of the system are improved.

[0075] In some embodiments, the cloud service platform 120 selects a first workflow from the plurality of workflows to assign to the first sub-container based on respective resource consumptions of the plurality of workflows. For example, the cloud service platform 120 can select a hot workflow or an abnormal workflow with a higher resource consumption from the plurality of workflows based on respective resource consumptions of the plurality of workflows. Then, the cloud service platform 120 creates a corresponding sub-container for the selected workflow and assigns the selected workflow to the corresponding sub-container. For another example, after creating the first sub-container 331 and assigning corresponding resources thereto, the cloud service platform 120 can select a hot workflow or an abnormal workflow with a higher resource consumption from the plurality of workflows based on respective resource consumptions of the plurality of workflows to assign to the first sub-container 331. In this way, the corresponding first workflow can be executed in isolation in the first sub-container 331, thereby avoiding affecting the normal execution of other workflows.

[0076] With continued reference to FIG. 3C, the cloud service platform 120 can also set a second threshold 334 (also referred to as a hard line) for the resource load of the host container, to ensure that when the resource load of the host container exceeds a certain limit (high load), measures can be taken to limit the resource usage of the sub-container with excessive resource consumption.

[0077] FIG. 3D illustrates a schematic diagram of an example process 300D of limiting resource consumption of a sub-container, according to some embodiments of the present disclosure. As shown in FIG. 3D, at block 341, in response to determining that the resource load of the host container is greater than the second threshold 334, the cloud service platform 120 determines respective resource consumptions of the plurality of sub-containers.

[0078] In some embodiments, the second threshold 334 is a set upper limit of resource usage, and when the cloud service platform 120 determines that the resource load of the host container exceeds this threshold, it can be considered that the existing resource allocation has reached a limit, and measures must be taken to avoid system resource exhaustion. The cloud service platform 120 determines the respective resource consumptions of the plurality of sub-containers to determine the sub-container with the most resource occupation.

[0079] At block 342, the cloud service platform 120 selects at least one second sub-container 332 from the plurality of sub-containers based on the respective resource consumptions of the plurality of sub-containers. At block 343, the cloud service platform 120 executes a resource limiting strategy for the at least one second sub-container 332.

[0080] In some embodiments, the cloud service platform 120 can determine one or more sub-containers with high resource consumption (also referred to as second sub-containers 332) by monitoring the resource usage of the plurality of sub-containers, and execute a resource limiting strategy thereon. In this way, the resource pressure of the host container can be alleviated, and the normal execution of other workflows can be avoided from being affected by the hot workflow with high load.

[0081] In the embodiments of the present disclosure, when the resource load of the host container exceeds the second threshold, the resource pressure of the system can be effectively alleviated by limiting the resource usage of the high-consumption sub-container, so as to prevent the system from collapsing or having a serious performance decline. Moreover, by actively identifying and limiting the sub-container with large resource consumption, it can be ensured that the resource allocation is more balanced, and the normal operation of other sub-containers is not affected due to the occupation of too many resources by a few sub-containers. In this way, the stability and reliability of the workflow task execution are improved.

[0082] In some embodiments, the cloud service platform 120 sorts the plurality of sub-containers based on the respective resource consumptions of the plurality of sub-containers. Based on the sorting result of the plurality of sub-containers, the cloud service platform 120 determines at least one second sub-container.

[0083] As an example, the cloud service platform 120 sorts the sub-containers based on the respective resource consumptions of the plurality of sub-containers, so as to select one or more sub-containers that occupy more resources. For example, the cloud service platform 120 can take the sub-container with the highest resource occupation in the sorting result as the second sub-container 332, and perform a resource limiting strategy for the second sub-container 332, such as limiting the processor usage time share or the calculation priority of the second sub-container 332, or reducing the upper limit of the memory usage of the second sub-container 332 to prevent it from occupying too much memory, and the like.

[0084] In the embodiments of the present disclosure, by sorting the sub-containers according to the respective resource consumptions, the sub-containers that occupy a large amount of resources can be accurately identified and preferentially limited. This not only avoids resource waste, but also releases the resources for more critical or more efficient workflows, thereby improving the utilization rate of the overall resources.

[0085] Example apparatus and device

[0086] Embodiments of the present disclosure also provide a corresponding apparatus for implementing the above method or process. FIG. 4 shows a schematic structural block diagram of an example apparatus 400 for resource management according to certain embodiments of the present disclosure. The apparatus 400 can be implemented as or included in the cloud service platform 120. Various modules / components in the apparatus 400 can be implemented by hardware, software, firmware, or any combination thereof.

[0087] As shown in FIG. 4, the apparatus 400 includes a creation module 410 configured to create a first sub-container in a host container by utilizing a part of resources included in the host container, wherein a plurality of workflows are executed in the host container, and the resources allocated to the first sub-container are isolated from the resources in the host container that are not allocated to the first sub-container; an allocation module 420 configured to allocate a first workflow in the plurality of workflows to the first sub-container; and an execution module 430 configured to execute the first workflow by utilizing the first sub-container.

[0088] In some embodiments, the execution module 430 is further configured to start a first sub-process in the first sub-container, the first sub-process being used for performing a task corresponding to the first workflow.

[0089] In some embodiments, the creation module 410 is further configured to determine, according to a control group mechanism, resources of the host container allocated to the first sub-container; and create the first sub-container based on the resources of the host container allocated to the first sub-container.

[0090] In some embodiments, the resources allocated to the first sub-container comprise at least one of: a memory resource, a computing resource, or a file system resource.

[0091] In some embodiments, the apparatus 400 further comprises a routing module 450 configured to update a routing table for the host container to indicate that the task corresponding to the first workflow is mapped to the first sub-container.

[0092] In some embodiments, the creation module 410 is further configured to determine whether a resource load of the host container is greater than a first threshold; and in response to the resource load being greater than the first threshold, create the first sub-container in the host container.

[0093] In some embodiments, the creation module 410 is further configured to select the first workflow from the plurality of workflows to be allocated to the first sub-container based on respective resource consumptions of the plurality of workflows.

[0094] In some embodiments, the apparatus 400 further comprises a resource limiting module 460 configured to, in response to determining that the resource load of the host container is greater than a second threshold, determine respective resource consumptions of the plurality of sub-containers; select at least one second sub-container from the plurality of sub-containers based on the respective resource consumptions of the plurality of sub-containers; and perform a resource limiting strategy for the at least one second sub-container.

[0095] In some embodiments, the resource limiting module 460 is further configured to sort the plurality of sub-containers based on the respective resource consumptions of the plurality of sub-containers; and determine the at least one second sub-container based on the sorting result of the plurality of sub-containers.

[0096] FIG. 5 illustrates a block diagram of a computing device 700 in which one or more embodiments of the present disclosure can be implemented. It should be understood that the computing device 500 illustrated in FIG. 5 is merely an example and should not be construed to limit the functionality and scope of the embodiments described herein. The computing device 500 illustrated in FIG. 5 can be used to implement the electronic device 110 of FIG. 1.

[0097] As shown in FIG. 5, electronic device 500 is in the form of a general-purpose electronic device. Components of electronic device 500 can include, but are not limited to, one or more processors or processing units 510, memory 520, storage 530, one or more communication units 540, one or more input devices 550, and one or more output devices 560. Processing unit(s) 510 can be actual or virtual processors and capable of executing various processing in accordance with programs stored in memory 520. In a multi-processing system, multiple processing units execute computer-executable instructions in parallel to improve the processing power of electronic device 500.

[0098] Electronic device 500 typically includes a plurality of computer storage media. Such media can be removable and / or non-removable, and can include volatile and / or nonvolatile media. Memory 520 can be volatile (such as, for example, registers, cache, random access memory (RAM)), non-volatile (such as, for example, read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. Storage 530 can be removable or non-removable and can include machine-readable media, such as, for example, flash drives, disks, or any other media capable of storing information and / or data and accessible by electronic device 500.

[0099] Electronic device 500 can further include additional removable / non-removable, volatile / non-volatile storage media. Although not shown in FIG. 5, a disk drive or other computer-readable media drive can be provided for reading from or writing to a removable, non-removable, volatile, or non-volatile media slot (such as a "floppy" disk). In such cases, each drive can be connected to the bus (not shown) by one or more data media interfaces. Memory 520 can include a computer program product 525 having one or more program modules configured to carry out the various methods or actions of the various embodiments of the present disclosure.

[0100] Communication unit(s) 540 enable communication with other electronic devices via communication media. Additionally, functionality of components of electronic device 500 can be implemented in a single computing cluster or a plurality of computer machines capable of communicating over a communication connection. As such, electronic device 500 can operate in a networked environment using logical connections to one or more other servers, network personal computers (PCs), or another network nodes.

[0101] The input device 550 can be one or more input devices, such as a mouse, a keyboard, a trackball, etc. The output device 560 can be one or more output devices, such as a display, a speaker, a printer, etc. The electronic device 500 can also communicate with one or more external devices (not shown), such as storage devices, display devices, etc., one or more devices that enable a user to interact with the electronic device 500, or any devices (e.g., a network card, a modem, etc.) that enable the electronic device 500 to communicate with one or more other electronic devices, as desired via the communication unit 540. Such communication can be carried out via an input / output (I / O) interface (not shown).

[0102] According to an example implementation of the present disclosure, a computer readable storage medium is provided having computer executable instructions stored thereon, where the computer executable instructions are executed by a processor to implement the method described above. According to an example implementation of the present disclosure, a computer program product is also provided that is tangibly stored on a non-transitory computer readable medium and includes computer executable instructions, where the computer executable instructions are executed by a processor to implement the method described above.

[0103] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0104] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0105] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0106] The computer program product of the present disclosure can be a computer program product, which is a machine-readable medium (media) having instances of the software embodied thereon, such as computer software, firmware, wireless application protocol (WAP), middleware or microcode. For example, a computer program product can be a floppy disk, a CD-ROM, an optical disc (e.g., a DVD), a flash drive, a memory stick, or a hard disk drive, having such instances of the software. These instances (or a compressed version thereof) can be presented to a processor of a user device by providing a usable device, such as a floppy disk drive, a CD-ROM drive, an optical drive, a flash drive, a memory stick, or a hard disk drive.

[0107] The implementations of the disclosure have been described above with the intent to be illustrative rather than limiting. Although the implementations of the disclosure have been described with regard to one or more implementations, it will be recognized that a variety of modifications and changes can be made to these implementations without departing from the broader spirit and scope of the implementations as set forth in the preceding disclosure. For example, certain aspects of the implementations can be performed using hardware, software, and / or firmware, or any combination thereof. The preceding description is intended to be illustrative, and not to limit the scope of the present disclosure. Other arrangements, methods, or modifications can be devised without departing from the scope of the present disclosure, the described implementations being illustrative. Numerous specific details are described to provide a thorough understanding of the implementations. However, in certain instances, well-known methods, procedures, components, and circuits have not been described in detail so as not to obscure the implementations. It will be appreciated that the features and components of the described implementations can be combined or configured in various ways. It will be appreciated that the features and components of the described implementations can be combined or configured in various ways. It will be appreciated that the features and components of the described implementations can be combined or configured in various ways.

Claims

1. A method for resource management, comprising: creating a first child container in a host container with a portion of resources included in the host container, wherein a plurality of workflows are executed in the host container and resources allocated to the first child container are isolated from resources in the host container that are not allocated to the first child container; allocating a first workflow of the plurality of workflows to the first child container; and executing the first workflow with the first child container.

2. The method of claim 1, wherein executing the first workflow with the first child container comprises: starting a first child process in the first child container, the first child process being used for an execution task corresponding to the first workflow.

3. The method of claim 1, wherein creating the first child container in the host container comprises: determining resources of the host container allocated to the first child container according to a control group mechanism; creating the first child container based on the resources of the host container allocated to the first child container.

4. The method of claim 1, wherein the resources allocated to the first child container comprise at least one of: memory resources, computing resources, or file system resources.

5. The method of claim 1, further comprising: updating a routing table for the host container to indicate mapping of an execution task corresponding to the first workflow to the first child container.

6. The method of claim 1, wherein creating the first child container in the host container comprises: determining whether a resource load of the host container is greater than a first threshold value; and in response to the resource load being greater than the first threshold value, creating the first child container in the host container.

7. The method of claim 6, further comprising: selecting the first workflow from the plurality of workflows to allocate to the first child container based on respective resource consumptions of the plurality of workflows.

8. The method of claim 1, wherein the host container has a plurality of containers created therein, the plurality of child containers including the first child container, and the method further comprises: in response to determining that a resource load of the host container is greater than a second threshold value, determining respective resource consumptions of the plurality of child containers; selecting at least one second child container from the plurality of child containers based on the respective resource consumptions of the plurality of child containers; and executing a resource limiting policy for the at least one second child container.

9. The method of claim 8, wherein selecting the at least one second child container from the plurality of child containers comprises: ranking the plurality of child containers based on the respective resource consumptions of the plurality of child containers; and determining the at least one second child container based on the ranking of the plurality of child containers.

10. An apparatus for resource management, comprising: ​ ​ ​ ​ ​ a creation module configured to create a first child container in a host container using a portion of resources included in the host container, wherein a plurality of workflows are executed in the host container, and resources allocated to the first child container are isolated from resources in the host container that are not allocated to the first child container; an allocation module configured to allocate a first workflow of the plurality of workflows to the first child container; and an execution module configured to execute the first workflow using the first child container.

11. An electronic device, comprising: at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions, when executed by the at least one processing unit, cause the electronic device to perform the method according to any one of claims 1-9.

12. A computer-readable storage medium having stored thereon a computer program, the computer program being executable by a processor to implement the method according to any one of claims 1-9. ​

Images