A method for sharing resources in a multi-tenant network

Abstract

A method in a network node configured to manage at least one processing unit resource (PU) in a network comprising a plurality of applications is described. The method includes grouping the at least one application of the plurality of applications into at least one group of applications based on hardware partitioning constraints of the plurality of PUs and an application isolation condition. The method also includes configuring, based on the grouping of the at least one application, at least one hardware partition on the at least one PU that fulfills minimization of resource wastage and meets a condition of at least one objective function. In addition, the method includes at least one of sorting, mapping and running the at least one group of applications on the at least one hardware partition using the software partitioning process and according to the objective function.

Description

[0001] A METHOD FOR SHARING RESOURCES IN A MULTI-TENANT NETWORK

[0002] TECHNICAL FIELD

[0003] The present disclosure relates to wireless communications, and in particular, to management of resources in a communication network.

[0004] BACKGROUND

[0005] The rate at which the microprocessor speeds increase has been fast outstripped by the computing needs of current workloads, many times driven by advances in artificial intelligence and machine learning (AI / ML). In order to bridge the gap between the computing capability of current microprocessors and the computing needs of workloads, datacenters have been introducing accelerators into their hardware.

[0006] One way to introduce computing acceleration to datacenters is through the use of graphics processing units (GPUs). In contrast to standard central processing unit (CPU) architectures which are generally optimized for sequential processing of data through instructions, a GPU architecture is centered around a parallel computing architecture where simple operations process data in parallel at the same time, i.e., single instruction multiple data (SIMD) operation.

[0007] As GPUs, as well as GPU-equipped servers, are generally expensive, it is quite important that they are used efficiently. For example, with respect to efficient GPU utilization, besides being costly, many applications exhibit low resource demands, leading to inefficient underutilization of the GPU in many scenarios. In cloud network setting, this means solutions that enable sharing of GPUs among several users are needed. GPU vendors have recognized this and designed mechanisms for sharing GPU resources. However, these mechanisms are yet to be incorporated into cloud management solutions to enable efficient use of GPUs under multi-tenant scenarios.

[0008] GPU sharing techniques can be divided into two techniques: hardware- implemented (e.g., Advanced Micro Devices (AMD) multiple-GPU (MxGPU)) partitioning and software implemented (e.g., NVIDIA multi-process service (MPS)) partitioning. Hardware and software implemented GPU partitioning methods allow running several processes to use the GPU hardware. Software-implemented sharing generally assumes well-behaved processes as it does not implement any strong memory isolation among the process that share the GPU. Further, software-implemented sharing allows the following: (1) GPU resource can be used more efficiently as it allows for more fine-grained control of GPU resources; and (2) for processes that benefit from sharing data, it allows a several fold performance increase by avoiding copying data from GPU memory to main memory and back. A disadvantage of software-implemented sharing is that since there is no strong isolation among the running processes, it is not suitable for multi-tenant applications. Specifically, a process may be able to access the memory of other processes (i.e., not secure) or a crash of one of the processes may affect all other processes that run on that hardware (i.e., not safe).

[0009] Hardware-implemented GPU partitioning allows partitioning the GPU into self- contained GPU units with strong isolation properties. Isolation among the different processes that run on that GPU may be provided, making it usable in multi-tenant cloud environments. A disadvantage is that hardware partitioning has limited granularity, making it difficult to maximize efficient use of the GPU.

[0010] An energy-related runtime scheduling technique for resource allocation decisions may employ GPU resources to meet the response time requirements of intermittent realtime tasks in the cloud computing platform. A problem of scheduling tasks is formulated as an optimization problem. After sorting tasks in ascending order based on their arrival time, the proposed scheduling algorithm decides on resource allocation and task placement to minimize energy consumption and maximize the success rate of all tasks.

[0011] Further, a GPU sharing approach, e.g., Transparent GPU Sharing (TGS), may be provided to share GPU resources, including memory and compute resources, between cloud workloads. Specifically, under-utilizing GPU resources may be managed when Deep Learning (DL) jobs are offloaded to the cloud GPU resources, as well as managing the throughput of the DL jobs. TGS seeks to enhance GPU utilization by allowing other jobs to opportunistically use GPU while controlling their dequeuing rate to decrease the contention impact on existing jobs. Accordingly, a monitoring component is designed to keep the throughput of DL jobs close to their throughput while running on the GPU exclusively.

[0012] A scheduling framework and resource allocation approach, i.e., Themis, may be used for GPU scheduling in machine learning training workloads. Themis may use a two- level scheduling architecture where long-running ML workloads bid on available resources in the central auction. A central arbiter determines the global winning bids based on the finish-time fairness metric to minimize the maximum finish-time fairness across all ML apps while efficiently utilizing GPUs. In addition, a dynamic GPU resource partitioning technique, i.e., multi-instance GPU solution (MISO), may be used to allocate GPU resources between co-located jobs using multi-instance GPU (MIG) capability as hardware partitioning. MISO is explicitly designed for hardware partitioning GPUs and attempts to maximize the overall system performance by reducing the job completion time.

[0013] In some cases, a GPU operation method that utilizes an MPS server process may be used in order to address the problem of GPU underutilization such as to enhance GPU utilization by leveraging the parallel processing capacity of the GPU and allow processes of different containers to co-locate and run concurrently on the shared GPU.

[0014] Further, a GPU sharing method may include receiving service creation information, creating a pod and selecting a target GPU node. By scheduling multiple pods on the same GPU, the method attempts to improve GPU utilization and facilitates parallel execution on the same GPU. In order to maximize GPU computing efficiency, the method attempts to provide a choice of parallel execution using MPS technology or GPU- exclusive services using time- sharing scheduling.

[0015] In sum, purely software-based sharing techniques provide poor isolation and security, and purely hardware -based sharing techniques suffer from inefficient resource usage.

[0016] SUMMARY

[0017] Some embodiments advantageously provide methods, systems, and apparatuses for sharing resources in a multi-tenant network.

[0018] Some embodiments provide a method for sharing resources such as GPU resources to maximize efficiency. When the method is applied in multi-tenant or multi-user scenarios, strong isolation of workloads ensures the security of applications. In some embodiments, both hardware-based sharing and software -based sharing techniques to realize an efficient use of resources while ensuring strong isolation of workloads to secure application deployment in multi-tenant environments.

[0019] Some other embodiments provide a method for sharing resources such as GPU resources in multi-tenant cloud environments, where the method uses both hardware- and software -based GPU partitioning to allocate GPU resources securely and efficiently, e.g., according to a given objective function. The method may include receiving a set of users (or information from a set of network nodes configured with user applications). For example, a system may include one or more network nodes associated with a user may include a set of applications and / or a set of network nodes (e.g., cloud servers), each with GPU resources on which the applications from the users are to be run.

[0020] The method may include one or more of the following steps:

[0021] • For each user (e.g., network node associated with a user), grouping of its applications, according to the hardware partitioning constraints of GPUs, where the grouped application can be run using software partitioning processes.

[0022] • Based on the groupings, configuring hardware partitions on the GPUs such that resource wastage is minimized while fulfilling the objective of the placement.

[0023] • Sorting, mapping and running the application groups on hardware partitions using software partitioning technology, according to the objective function.

[0024] One or more embodiments provide one or more of the following benefits:

[0025] • Sharing GPU resources in a multi-tenant environment, ensuring both efficiency and security at the same time.

[0026] • Supporting different types of software partitioning (e.g., sharing in time, sharing in space, etc.) and hardware partitioning (e.g., AMD MxGPU, NVIDIA MIG, etc.)

[0027] • Supporting a wide range of objectives for resource sharing (e.g., fair resource allocation, revenue maximization, etc.)

[0028] According to one aspect, a method in a network node configured to manage at least one processing unit resource (PU) of a plurality of PUs in a network comprising a plurality of applications is described. The at least one PU is configured to execute at least one application of the plurality of applications. The method includes grouping the at least one application of the plurality of applications into at least one group of applications based on hardware partitioning constraints of the plurality of PUs and an application isolation condition. The at least one group of applications is executable using a software partitioning process. The method also includes configuring, based on the grouping of the at least one application, at least one hardware partition on the at least one PU that fulfills minimization of resource wastage and meets a condition of at least one objective function. In addition, the method includes at least one of sorting, mapping and running the at least one group of applications on the at least one hardware partition using the software partitioning process and according to the objective function.

[0029] In some embodiments, the method further includes obtaining information associated with the plurality of applications and the plurality of PUs to group the at least one application. In some other embodiments, the at least one application is grouped for concurrent execution.

[0030] In some embodiments, the objective function includes an allocation objective associated with a minimization of a normalized completion time for a predetermined resource allocation.

[0031] In some other embodiments, the application isolation condition requires each application in the one group of applications to be isolated from the applications in other groups of applications.

[0032] In some embodiments, configuring the at least one hardware partition includes determining a combination of the at least one partition to partition the at least one PU to fulfill minimization of resource wastage and meet the condition of the at least one objective function. The condition is maximizing the at least one objective function.

[0033] In some other embodiments, at least one of sorting, mapping and running the at least one group of applications includes sorting and prioritizing the at least one group of applications based on the corresponding contribution to the objective function.

[0034] In some embodiments, at least one of sorting, mapping and running the at least one group of applications includes prioritizing at least one hardware partition combination for sequentially partitioning the at least one PU. The prioritizing includes at least one of determining and assigning a score to each hardware partition combination, arranging each hardware partition combination by the corresponding score; and determining a switch partition combination overhead and consensus characteristic values of the at least one hardware partition combination to determine the score.

[0035] In some other embodiments, the method further includes scheduling the at least one hardware partition on the at least one PU and the execution of the at least one application based on the at least one of sorting, mapping and running of the at least one group of applications.

[0036] In some embodiments, at least one of: (A) the at least one PU is comprised in the network node; (B) the at least one PU is comprised in another network node different from the network node; (C) the at least one PU is at least one graphics processing unit (GPU).

[0037] According to another aspect, a network node configured to manage at least one processing unit resource (PU) of a plurality of PUs in a network comprising a plurality of applications is described. The at least one PU is configured to execute at least one application of the plurality of applications, and the network node is configured to group the at least one application of the plurality of applications into at least one group of applications based on hardware partitioning constraints of the plurality of PUs and an application isolation condition. The at least one group of applications is executable using a software partitioning process. The network node may configure, based on the grouping of the at least one application, at least one hardware partition on the at least one PU that fulfills minimization of resource wastage and meets a condition of at least one objective function. Further, the network node is configured to at least one of sort, map and run the at least one group of applications on the at least one hardware partition using the software partitioning process and according to the objective function.

[0038] In some embodiments, the network node is further configured to obtain information associated with the plurality of applications and the plurality of PUs to group the at least one application.

[0039] In some other embodiments, the at least one application is grouped for concurrent execution.

[0040] In some embodiments, the objective function includes an allocation objective associated with a minimization of a normalized completion time for a predetermined resource allocation.

[0041] In some other embodiments, the application isolation condition requires each application in the one group of applications to be isolated from the applications in other groups of applications.

[0042] In some embodiments, configuring the at least one hardware partition includes determining a combination of the at least one partition to partition the at least one PU to fulfill minimization of resource wastage and meet the condition of the at least one objective function. The condition is maximizing the at least one objective function.

[0043] In some other embodiments, at least one of sorting, mapping and running the at least one group of applications includes sorting and prioritizing the at least one group of applications based on the corresponding contribution to the objective function.

[0044] In some embodiments, at least one of sorting, mapping and running the at least one group of applications includes prioritizing at least one hardware partition combination for sequentially partitioning the at least one PU. The prioritizing includes at least one of: (A) determining and assigning a score to each hardware partition combination; (B) arranging each hardware partition combination by the corresponding score; and (C) determining a switch partition combination overhead and consensus characteristic values of the at least one hardware partition combination to determine the score. In some other embodiments, the network node is further configured to schedule the at least one hardware partition on the at least one PU and the execution of the at least one application based on the at least one of sorting, mapping and running of the at least one group of applications.

[0045] In some embodiments, at least one of: (A) the at least one PU is comprised in the network node; (B) the at least one PU is comprised in another network node different from the network node; (C) the at least one PU is at least one graphics processing unit (GPU).

[0046] According to one aspect, a system comprising a first network node and at least one second network node is described. The first network node is configured to manage at least one processing unit resource (PU) of a plurality of PUs in a network comprising a plurality of applications. The at least one PU is configured to execute at least one application of the plurality of applications, and the first network node is configured to obtain, from the at least one second network node, information associated the plurality of applications and the plurality of PUs. The first network node is also configured to group the at least one application of the plurality of applications into at least one group of applications based on hardware partitioning constraints of the plurality of PUs, an application isolation condition, and the obtained information. The at least one group of applications is executable using a software partitioning process. The first network node may also configure, based on the grouping of the at least one application, at least one hardware partition on the at least one PU that fulfills minimization of resource wastage and meets a condition of at least one objective function. In addition, the first network node is configured to at least one of sort, map and run the at least one group of applications on the at last one hardware partition using the software partitioning process and according to the objective function.

[0047] BRIEF DESCRIPTION OF THE DRAWINGS

[0048] A more complete understanding of the present embodiments, and the attendant advantages and features thereof, will be more readily understood by reference to the following detailed description when considered in conjunction with the accompanying drawings wherein:

[0049] FIG. 1 is a schematic diagram of an example network architecture illustrating a communication system according to principles disclosed herein;

[0050] FIG. 2 is a block diagram of a network node in communication with other network nodes according to some embodiments of the present disclosure; FIG. 3 is a block diagram illustrating an example virtualization environment according to some embodiments of the present disclosure;

[0051] FIG. 4 is a flowchart of an example process in a user equipment according to some embodiments of the present disclosure;

[0052] FIG. 5 an exemplary system overview according to some embodiments of the present disclosure;

[0053] FIG. 6 is a flowchart of an example process for resource allocation for applications according to some embodiments of the present disclosure; and

[0054] FIG. 7 shows example partitioning of a processing unit following according to the hardware partitioning constraints according to some embodiments of the present disclosure.

[0055] DETAILED DESCRIPTION

[0056] Before describing in detail exemplary embodiments, it is noted that the embodiments reside primarily in combinations of apparatus components and processing steps related to sharing resources in a multi-tenant network. Accordingly, components have been represented where appropriate by conventional symbols in the drawings, showing only those specific details that are pertinent to understanding the embodiments so as not to obscure the disclosure with details that will be readily apparent to those of ordinary skill in the art having the benefit of the description herein.

[0057] As used herein, relational terms, such as “first” and “second,” “top” and “bottom,” and the like, may be used solely to distinguish one entity or element from another entity or element without necessarily requiring or implying any physical or logical relationship or order between such entities or elements. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the concepts described herein. As used herein, the singular forms “a”, “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprises,” “comprising,” “includes” and / or “including” when used herein, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0058] In embodiments described herein, the joining term, “in communication with” and the like, may be used to indicate electrical or data communication, which may be accomplished by physical contact, induction, electromagnetic radiation, radio signaling, infrared signaling or optical signaling, for example. One having ordinary skill in the art will appreciate that multiple components may interoperate and modifications and variations are possible of achieving the electrical and data communication.

[0059] In some embodiments described herein, the term “coupled,” “connected,” and the like, may be used herein to indicate a connection, although not necessarily directly, and may include wired and / or wireless connections.

[0060] In some embodiments, the term “group” or “grouping” may be used and may refer to a number of elements (e.g., software applications) that are classed together such as to correspond to one or more actions (e.g., assignment of applications to a hardware / software partition. For example, a group of applications may refer to one or more applications that are expected to perform a predetermined function or another entity or component may perform a predetermined function associated to the grouped applications. A group of applications may be assigned a predetermined hardware partition, where the application(s) in the group are to be executed using the predetermined hardware partition. The group of applications may include one application or more than one application. In some embodiments, the group is created and may be empty, e.g., where an application may be assigned to such group at a later time or not assigned at all. In some embodiments, the term software partitioning process may refer to one or more functions executed by a network node (e.g., software and / or hardware of the network node) that providing partitioning functions such as functions that allow apportionment, allocation, slicing of one or more resources associated with a computing device such as a network node.

[0061] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the concepts described herein. As used herein, the singular forms “a”, “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprises,” “comprising,” “includes” and / or “including” when used herein, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0062] The term “network node” used herein can be any kind of network node comprised in a network. For example, a network node may be any computing device such as a server or device and may be virtualized. The functions described herein as being performed by a network node may be distributed over a plurality of network nodes. In other words, it is contemplated that the functions of the network node described herein are not limited to performance by a single physical device and, in fact, can be distributed among several physical devices.

[0063] Referring to the drawing figures, in which like elements are referred to by like reference numerals, there is shown in FIG. 1 a schematic diagram of a communication system 10, according to an embodiment, which comprises a network 12, and a network 14. Further, communication system 10 comprises a plurality of network nodes 16a, 16b, 16c, 16d (referred to collectively as network nodes 16). Network nodes 16a, 16b, 16c are comprised in the network 12, and network node 16d is comprised in the second network 14. Each network node 16 is connectable to another network node 16. Further, each network node 16a, 16b, 16c is connectable to the second network 14 (and / or network node 16d comprised in the second network 14) over a wired or wireless connection 18. In some embodiments, network 12 and / or network 14 is a cloud network.

[0064] Also, it is contemplated that a network node 16 can be in simultaneous communication and / or configured to separately communicate with more than one network node 16 and more than one type of network node 16. For example, a network node 16 can have dual connectivity with another network node 16 that more than once communication protocol or communication link.

[0065] A network node 16 is configured to include a scheduler unit 20 which is configured to perform any step and / or task and / or process and / or method and / or feature described in the present disclosure, e.g., network node functions and / or prediction optimization functions.

[0066] Example implementations in accordance with an embodiment of the network node 16 and other network nodes 16 discussed in the preceding paragraphs will now be described with reference to FIG. 2.

[0067] The communication system 10 includes a network node 16 provided in a communication system 10 and includes hardware 22 enabling it to communicate with the other network nodes 16. The hardware 22 may include a communication interface 24 for setting up and maintaining at least a wireless / wired connection with the other network nodes 16, which may be located in the network 12, network 14, or any other network. The communication interface 24 may be formed as or may include transmitters and / or receivers. In the embodiment shown, the hardware 22 of the network node 16 further includes processing circuitry 26. The processing circuitry 26 may include a processor 28 and a memory 30. In particular, in addition to or instead of a processor, such as a central processing unit, and memory, the processing circuitry 26 may comprise integrated circuitry for processing and / or control, e.g., one or more processors and / or processor cores and / or FPGAs (Field Programmable Gate Array) and / or ASICs (Application Specific Integrated Circuitry) adapted to execute instructions. The processor 38 may be configured to access (e.g., write to and / or read from) the memory 40, which may comprise any kind of volatile and / or nonvolatile memory, e.g., cache and / or buffer memory and / or RAM (Random Access Memory) and / or ROM (Read-Only Memory) and / or optical memory and / or EPROM (Erasable Programmable Read-Only Memory). Hardware 22 may further include processing unit (PU) 32 configured to perform processing of one or more actions described herein such as actions associated with an application. In some embodiments, PU may be a hardware unit or a component of hardware 22 or a portion thereof. In some other embodiments, PU 32 comprises one or more GPUs. The terms PU and GPU may be used interchangeably herein.

[0068] Thus, the network node 16 further has software 34 stored internally in, for example, memory 40, or stored in external memory (e.g., database, storage array, network storage device, etc.) accessible by the network node 16 via an external connection. The software 34 may be executable by the processing circuitry 26 and may include at least an application 36 (e.g., a software application, a client application, operating system, virtualized application, cloud application, etc.). The processing circuitry 26 may be configured to control any of the methods and / or processes described herein and / or to cause such methods, and / or processes to be performed, e.g., by network node 16. Processor 28 corresponds to one or more processors 28 for performing network node 16 functions described herein. The memory 30 is configured to store data, programmatic software code and / or other information described herein. In some embodiments, the software 34 may include instructions that, when executed by the processor 28 and / or processing circuitry 26, causes the processor 28 and / or processing circuitry 26 to perform the processes described herein with respect to network node 16. For example, processing circuitry 26 of the network node 16 may include a scheduler unit 20 which is configured to perform any step and / or task and / or process and / or method and / or feature described in the present disclosure, e.g., network node functions and / or prediction optimization functions. In some embodiments, the inner workings of the network node 16 may be as shown in FIG. 2 and independently, the surrounding network topology may be that of FIG. 1.

[0069] The connection between the network node 16 and the other network nodes 16 is in accordance with the teachings of the embodiments described throughout this disclosure. More precisely, the teachings of some of these embodiments may improve the data rate, latency, and / or power consumption and thereby provide benefits such as reduced user waiting time, relaxed restriction on file size, better responsiveness, extended battery lifetime, etc.

[0070] Although FIGS. 1 and 2 show various “units” such as scheduler unit 20 as being within a respective processor, it is contemplated that these units may be implemented such that a portion of the unit is stored in a corresponding memory within the processing circuitry. In other words, the units may be implemented in hardware or in a combination of hardware and software within the processing circuitry. Further, in some embodiments, a network node 16 may include one or more of the components shown in FIG. 2. That is, in some embodiments, network node 16 is not required to include all the components shown in FIG. 2 and may include fewer components (or more components such as components described in other sections of the present disclosure).

[0071] FIG. 3 is a block diagram illustrating a virtualization environment 94 in which functions implemented by some embodiments may be virtualized. In the present context, virtualizing means creating virtual versions of apparatuses or devices which may include virtualizing hardware platforms, storage devices and networking resources. As used herein, virtualization can be applied to any device described herein, or components thereof, and relates to an implementation in which at least a portion of the functionality is implemented as one or more virtual components. Some or all of the functions described herein may be implemented as virtual components executed by one or more virtual machines (VMs) implemented in one or more virtual environments 94 hosted by one or more of network nodes, such as a hardware computing device that operates as another network node, a wireless device (e.g., user equipment), core network node, or host. Further, in embodiments in which the virtual node does not require a predetermined connectivity (e.g., a core network node or host), then the node may be entirely virtualized. In some other embodiments, the node may be entirely virtualized even when a predetermined connectivity is required. Applications 96 (which may alternatively be called software instances, virtual appliances, network functions, virtual nodes, virtual network functions, etc.) are run in the virtualization environment 94 to implement some of the features, functions, and / or benefits of some of the embodiments disclosed herein.

[0072] Hardware 98 includes processing circuitry, memory that stores software and / or instructions executable by hardware processing circuitry, and / or other hardware devices as described herein, such as a network interface, input / output interface, and so forth. Hardware 98 may be associated with and / or be and / or be comprised in and / or comprise and / or perform one or more functions associated with hardware 22 of FIG. 2. In some embodiments, VMs (e.g., VMs 102) may virtualize and / or partition PU 32 and make it available to applications. Software may be executed by the processing circuitry to instantiate one or more virtualization layers 100 (also referred to as hypervisors or virtual machine monitors (VMMs)), provide VMs 102a and 102b (one or more of which may be generally referred to as VMs 102), and / or perform any of the functions, features and / or benefits described in relation with some embodiments described herein. The virtualization layer 100 may present a virtual operating platform that appears like networking hardware to the VMs.

[0073] The VMs 102 comprise virtual processing, virtual memory, virtual networking or interface and virtual storage, and may be run by a corresponding virtualization layer 100. Different embodiments of the instance of a virtual appliance 96 may be implemented on one or more of VMs 102, and the implementations may be made in different ways. Virtualization of the hardware is in some contexts referred to as network function virtualization (NFV). NFV may be used to consolidate many network equipment types onto industry standard high volume server hardware, physical switches, and physical storage, which can be located in data centers, and customer premise equipment.

[0074] In the context of NFV, a VM 102 may be a software implementation of a physical machine that runs programs as if they were executing on a physical, non- virtualized machine. Each of the VMs 102, and that part of hardware 98 that executes that VM, be it hardware dedicated to that VM and / or hardware shared by that VM with others of the VMs, forms separate virtual network elements. Still in the context of NFV, a virtual network function is responsible for handling specific network functions that run in one or more VMs 102 on top of the hardware 98 and corresponds to the application 96.

[0075] Hardware 98 may be implemented in a standalone network node with generic or specific components. Hardware 98 may implement some functions via virtualization. Alternatively, hardware 98 may be part of a larger cluster of hardware (e.g. such as in a data center or CPE) where many hardware nodes work together and are managed via management and orchestration 104, which, among others, oversees lifecycle management of applications 96. In some embodiments, hardware 98 is coupled to one or more radio units that each include one or more transmitters and one or more receivers that may be coupled to one or more antennas. Radio units may communicate directly with other hardware nodes via one or more appropriate network interfaces and may be used in combination with the virtual components to provide a virtual node with radio capabilities, such as a radio access node or a base station. In some embodiments, some signaling can be provided with the use of a control system 106 which may alternatively be used for communication between hardware nodes and radio units.

[0076] In some embodiments, the virtualization environment 94 (which may go beyond what is illustrated in FIG. 3), may comprise systems, networks, servers, nodes, devices, etc., that are in communication with each other either through wire or wirelessly, e.g. through communication interfaces 24 (e.g., a network interface component (NIC) comprising physical network interface(s)). Some or all of the functions and steps described herein may be implemented as one or more virtual components (e.g., via one or more applications, components, functions, virtual machines, containers, etc.) executing on one or more physical apparatus in one or more networks, systems, environment, etc.

[0077] A virtualization environment 94 may provide hardware 98 comprising processing circuitry such as processing circuitry 26 and memory such as memory 30. The memory 30 can include instructions executable by the processing circuitry whereby functions and steps described herein may be executed to provide any of the relevant features and benefits disclosed herein.

[0078] The hardware 98 may also include non-transitory, persistent, machine-readable storage media having stored therein software and / or instruction executable by the processing circuitry 26 to execute functions and steps described herein.

[0079] The instructions may include a computer program for configuring the processing circuitry. The computer program may be stored in a removable memory, such as a portable compact disc, portable digital video disc, or other removable media. The computer program may be stored in a physical memory local to the hardware, which can be removable, or it could alternatively, or in part, be stored in the cloud. The computer program may also be embodied in a carrier such as an electronic signal, optical signal, radio signal, or computer readable storage medium. In some embodiments, the components of virtualization environment 94 may be that of network node 16, e.g., hardware 98 may be configured to perform the functions of hardware 22 and may be at least in part comprised in network node 16.

[0080] FIG. 4 is a flowchart of an example process in a network node 16. One or more blocks described herein may be performed by one or more elements of network node 16 such as by one or more of processing circuitry 26 (including the scheduler unit 20), processor 38, and / or communication interface 24. Network node 16 is configured to manage at least one processing unit resource (PU) 32 of a plurality of PUs 32 in a network 12, 14 comprising a plurality of applications 36. The at least one PU 32 is configured to execute at least one application 36 of the plurality of applications 36. Network node 16 such as via processing circuitry 26 and / or processor 38 and / or communication interface 24 is configured to group (Block S100) the at least one application 36 of the plurality of applications 36 into at least one group of applications 36 based on hardware partitioning constraints of the plurality of PUs 32 and an application isolation condition. The at least one group of applications 36 is executable using a software partitioning process. The network node may configure (Block S102), based on the grouping of the at least one application 36, at least one hardware partition 206 on the at least one PU 32 that fulfills minimization of resource wastage and meets a condition of at least one objective function. Further, the network node is configured to at least one of sort, map and run (Block S104) the at least one group of applications 36 on the at least one hardware partition 206 using the software partitioning process and according to the objective function.

[0081] In some embodiments, the method further includes obtaining information associated with the plurality of applications 36 and the plurality of PUs 32 to group the at least one application 36.

[0082] In some other embodiments, the at least one application 36 is grouped for concurrent execution.

[0083] In some embodiments, the objective function includes an allocation objective associated with a minimization of a normalized completion time for a predetermined resource allocation.

[0084] In some other embodiments, the application isolation condition requires each application 36 in the one group of applications 36 to be isolated from the applications 36 in other groups of applications 36.

[0085] In some embodiments, configuring the at least one hardware partition 206 includes determining a combination of the at least one partition to partition the at least one PU 32 to fulfill minimization of resource wastage and meet the condition of the at least one objective function. The condition is maximizing the at least one objective function.

[0086] In some other embodiments, at least one of sorting, mapping and running the at least one group of applications 36 includes sorting and prioritizing the at least one group of applications 36 based on the corresponding contribution to the objective function.

[0087] In some embodiments, at least one of sorting, mapping and running the at least one group of applications 36 includes prioritizing at least one hardware partition combination for sequentially partitioning the at least one PU 32. The prioritizing includes at least one of determining and assigning a score to each hardware partition combination, arranging each hardware partition combination by the corresponding score; and determining a switch partition combination overhead and consensus characteristic values of the at least one hardware partition combination to determine the score.

[0088] In some other embodiments, the method further includes scheduling the at least one hardware partition 206 on the at least one PU 32 and the execution of the at least one application 36 based on the at least one of sorting, mapping and running of the at least one group of applications 36.

[0089] In some embodiments, at least one of: (A) the at least one PU 32 is comprised in the network node 16; (B) the at least one PU 32 is comprised in another network node 16 different from the network node; (C) the at least one PU 32 is at least one graphics processing unit (GPU).

[0090] Having described the general process flow of arrangements of the disclosure and having provided examples of hardware and software arrangements for implementing the processes and functions of the disclosure, the sections below provide details and examples of arrangements for sharing resources, e.g., in a multi-tenant network. Multi-tenant network may refer to a network that comprises or is configured to serve more than one network node (e.g., more than one network node or device associated with a user).

[0091] In some embodiments, the term resource is used and may refer to any resource such as hardware and / or software resources, PU resources, GPU resources, partition, etc.

[0092] Some embodiments provide a system 10 and method for defining various objectives for resource allocation between cloud applications 36 that employ PU 32 (e.g., GPUs) to accelerate computation and, at the same time, fulfill isolation and optimal resource allocation using the hardware and software partitioning processes. One or more features of the embodiments may be applied in offline systems as an offline scheduling placement of cloud applications given the collected data regarding resource requirements and estimated completion time of the applications.

[0093] FIG. 5 shows an exemplary overview of an embodiment of system 10. Network node 16a and other network nodes 16 (e.g., network nodes 16b, 16c, 16d) comprising cloud applications 36 of N different users are shown. The users may want to accelerate their workloads using shared resources (e.g., PU resources, GPU resources, etc.) in a given data center with a network node 16 (e.g., server) having ‘g’ GPUs. Each application 36 may have specific information on resource requirements and estimated application completion time. In some embodiments, more detailed information about applications 36 may be collected and provided to network node 16, e.g., a list of affinity group applications which gives an estimated speed-up matrix capturing how much faster a group of co-scheduled applications run if co-located and sharing the same GPU partition.

[0094] Network node 16a (and / or scheduler unit 20) may be configured as an application scheduler and / or include and / or be configured to perform functions associated with application affinity manager 200, partition manager 202 (e.g., PU or GPU partition manager), and scheduler 204 (e.g., application placer). One or more embodiments provide an optimal resource allocation (e.g., PU or GPU resource allocation) and scheduling while satisfying a specific objective (e.g., minimizing the average normalized completion time in fairness allocation) for all cloud applications 36 using the components of network node 16a.

[0095] In some embodiments, the Application Affinity Manager 200 considers the resource requirements and other relevant parameters of the specific allocation objective (e.g., the estimated execution time of applications in allocation with fairness objective). Based on this information, the Application Affinity Manager 200 organizes applications from each user into groups of applications 36 according to the possible hardware partitioning sizes (e.g., PU / GPU hardware partitioning sizes). The organization provide minimization of unused resources between applications 36 with no isolation requirements to be co-located and executed in a specific hardware partition 206 (e.g., hardware partitions 206a, 206b, 206c, 206d, 206e), which may be comprised in queue of hardware partitions containing k elements. Partition 206 may comprise or be assigned one or more applications 36 (or a queue of applications 36). Considering allocation objective parameters aids in selecting applications 36. For example, considering the estimated execution time in the allocation with a fairness objective enables applications 36 with close execution times to be co-located together. Considering this aspect prevents the unnecessary allocation of idle resources before releasing the partition 206 (e.g., GPU hardware partition) upon completing all hosted applications 36. At the end of this phase, applications 36 may be organized or grouped into optimal hardware partitions 206, resulting in the creation of a k hardware partitions queue for the grouped applications 36.

[0096] The Partition Manager 202 is configured to receive grouped applications 36 with the determined required size of hardware partition as input and to determine the optimal configuration of hardware partitions 206 (e.g., for the GPUs). This step considers the isolation constraint, partition fragmentation constraints, and limitation of hardware partition configuration. The limitation of hardware partitioning may indicate the PU (e.g., GPU) policies that set up some limitations on partitioning the PU 32 (GPU) into multiple smaller PUs 32 (GPUs), e.g., the limitation on the number of smaller PUs 32 or the specific allowed combination of hardware partitions 206. Applications 36 from different users can be co-scheduled on the same PU 32 to improve PU utilization or enhance the performance of inter-dependent applications.

[0097] Although applications 36 associated with one user may run on the same PU 32 (e.g., GPU) using software partitioning techniques and spatial multitasking techniques without any issues, co-locating applications of different users may require isolation to prevent security risks. In this regard, the Application Affinity Manager 200 may be configured to execute software partitioning task to improve resource utilization between applications of the same users while supplying the isolation for applications of different users using hardware partitioning processes in Partition Manager 202. Accordingly, Partition Manager 202 determines the optimal configuration of hardware partitions 206 (e.g., GPU partitions) for isolated grouped applications 36 while minimizing the waste of PU resources (e.g., GPU resources).

[0098] A time-based allocation strategy may be used in which the scheduler 204 (application placer) is configured to find a time slot in which PUs 32 with available resources can run the grouped applications 36 at the earliest available time slot by considering the application priorities to ensure efficient execution. In the previous steps, applications 36 were grouped in the proper size of hardware partitions 206, and appropriate configurations of hardware partitions 206 are determined to allocate optimal PU resources. In this step, the scheduler 204 is configured to prioritize the grouped applications 36 to schedule applications 36 based on the objective of allocation. For example, in the allocation with fairness objective, the scheduler 204 executes applications 36 with shorter completion times earlier, while applications with longer completion times are scheduled later. That is, the embodiments provide functions that increase the efficiency of the scheduling process by potentially reducing the overall processing time of all applications 36. In some embodiments, the “fairness” objective refers to the relative completion time of all applications being the same or roughly the same (e.g., similar, not to exceed a difference threshold, etc.). In some other embodiments, running short applications first ensures fair scheduling of the execution of the applications while, at the same time, reducing the overall processing time of all applications.

[0099] FIG. 6 illustrates an example method for resource allocation for cloud applications. The method includes, at step S200, grouping applications 36 for concurrent execution based on hardware partitioning constraints. Input for step S200 may include information associated with applications 36, PU 32 (GPUs), allocation objective, etc. The method also includes, at step S202, configuring hardware partitions 206 according to application groupings, at step S204, sorting and prioritizing the grouped applications 36 based on the objective function, at step S206, sorting and prioritizing hardware partition combinations, and at step S208, scheduling partitioning of PU 32 (GPUs) and running applications 36. In some embodiments, sorting and prioritizing the grouped applications 36 is based on the contribution to the objective function (e.g., the objective function be, say a sum of ‘contributions’ from individual groupings.) However, the embodiments are not limited as such, and the sorting and / or prioritizing may be based on the objective function without requiring the sorting and / or prioritizing to be based on the contribution to the objective function.

[0100] More specifically, PU resources (e.g., GPU resources) may be allocated between diverse users with the isolation concern based on the given type of allocation objective while improving PU utilization (e.g., GPU utilization). This approach serves as a way of sharing PUs 32 (GPUs) in the cloud for a wide array of objectives. The method may be adaptable to different objective functions. In some embodiments, the method takes input from the collected information about applications, available GPUs, and the allocation objective to decide on partitioning GPU and resource allocation for the cloud applications.

[0101] The following are steps of the method which are provided as nonlimiting examples.

[0102] S200: Grouping applications for concurrent execution based on hardware partitioning constraints

[0103] The method utilizes hardware partitioning technology to partition the PU 32 (e.g., GPU) into smaller PUs 32 (e.g., smaller GPUs), for example, in order to address the isolation concerns between different users. However, different implementations of PU hardware partitioning solutions come with specific constraints on how the PU 32 can be partitioned. For example, if using the MIG technology of NVIDIA, the PU 32 may be partitioned into up to seven partitions, following the allowable combinations as shown in FIG. 7. The term “allowable combinations” may refer to predetermined combinations or any other type of combinations. In the case of AMD MxGPU technology with GIM driver, up to 16 equal-sized partitions may be created.

[0104] Applications 36 may have varying resource and execution time requirements, and considering the above constraints, using a hardware partition per application may result in underutilization of resources. Since applications from the same user may be assumed to not have isolation requirements, grouping applications (with comparable execution times) together provides flexibilities that may be exploited to minimize resource wastage for a given hardware partition size. The Application Affinity Manager 200 identifies the best grouping of a user’s applications 36 given the hardware partitioning constraints and their completion times. A group of applications 36 may be run in parallel on a given hardware partition 206, using software partitioning techniques.

[0105] Selecting a set of applications among applications 36 with various resource requirements and distinct execution times, along with deciding on the optimal hardware partition 206 to host them, may be used to reduce PU resource usage by reducing the number of partitions used. However, along with minimizing resource utilization, the requested allocation objective (e.g., minimize the normalized application completion time for the objective of fair resource allocation) may be met. Such an optimization problem can be solved either using solvers (e.g. a Linear Programming (LP) Simplex Solver) or an approximative heuristic solution.

[0106] S202: Configuring hardware partitions for isolated grouped applications

[0107] After grouping applications of each user to be hosted in the specific hardware partition 206 with a given size, in this step, the partition manager 202 determines the optimal combination for the selected hardware partitions (e.g., NVIDIA MIG unit configurations or MxGPU number of partitions) to partition available the GPUs (i.e., Pus 32), with the objective of maximizing the objective function and minimizing the wastage of resources (e.g., PU / GPU resources). Considering the constraints associated with valid combinations of hardware partitions 206 in the PU 32, optimal hardware partition configurations may be determined (e.g., to achieve one or more objectives such as reducing the waste of PU resources). In this regard, based on the number and size of selected hardware partitions 206 and the list of possible hardware partition configuration options, Partition Manager 202 may determine the optimal combination for the selected hardware partitions as the optimal solution for this linear problem, e.g., using an LP simplex solver, heuristic solutions, etc.

[0108] S204: Sorting and prioritizing the grouped applications based on their contribution to the objective function

[0109] At this step, applications 36 to be co-located in the specific hardware partition 206 are grouped, and the hardware partitions for each PU 32 are determined. In this step, the grouped applications 36 are prioritized based on the objective function. For example, if the objective is to fairly schedule the execution of all applications 36 such that their relative execution times are close to each other, then groups with shorter execution times should be scheduled for execution first. As a result, this step sorts the grouped applications 36 in each specific hardware partition size based on their maximum completion time to prioritize them for executing sequentially. The grouped applications 36 with shorter maximum completion times are scheduled earlier, while the longer ones are scheduled later. Other scheduling orders are possible.

[0110] S206: Sorting and prioritizing hardware partition combinations

[0111] After determining the priority of grouped application execution, the method prioritizes the hardware partition combinations for sequentially partitioning the available Pus 32. This partitioning may be used, e.g., when there are more selected partition combinations than available Pus 32. To prioritize the selected partition combinations, a score (e.g., priority score) may be determined and assigned to each. Partition configurations may be arranged by their score (e.g., priority score). Each selected partition combination includes a list of hardware partitions 206 with specific sizes. This step picks the grouped applications 36 for each hardware partition size in the hardware partition combination from the sorted list of grouped applications 36. The grouped applications 36 in each hardware partition size are sorted so the grouped applications 36 with higher priority (e.g., shorter execution time when the objective is fairness) are selected sooner. After selecting sufficient grouped applications 36 with specific hardware partition requirements, the score may be calculated for each hardware partition combination following this formula:

[0112] Scores = (CCV + STO)

[0113] Switch partition combination overhead (STO) enables considering the overhead of switching between partition combinations. Considering that changing the current GPU partition combination may require restarting the GPU, preventing switching the partition combination may increase the efficiency of the method.

[0114] • Consensus characteristic values of the partition combination (CCVpc) shows how the combination of grouped applications can contribute to the allocation objective. For example, in the fairness allocation, the maximum completion time of the grouped applications 36 in all the partitions of the partition combination may be calculated for each partition combination. Further, if the objective is earning more money, the sum of earnings of the grouped applications 36 in partitions of the partition combination may be calculated.

[0115] By associating the score (e.g., priority score) to each partition combination, these scores (e.g., priority scores) are used as input to the scheduler to partition available GPUs.

[0116] S208: Schedule partitioning of GPUs and running of applications

[0117] In this step, the method partitions available Pus 32 one by one by sequentially assigning prioritized partition configurations to the Pus 32. Then, the method selects and schedules the grouped applications36 in the PU partitions according to their priority. After each PU partitioning, the status of Pus 32 may be updated. In some embodiments, in the case of releasing all the resources of the PU 32, the method iterates through S206 and then chooses the next partition configuration for the PU 32 until all applications groups are executed.

[0118] Nonlimiting Example

[0119] To further illustrate the embodiment, the following example is provided. The example is described using the term GPU which may refer to PU 32 as described in the present disclosure. That is, the example is not limited to GPUs and may be used with any processing unit or hardware unit. In this example, several users want to run their applications 36 on a shared cloud infrastructure accelerated with NVIDIA GPUs. Each user has multiple cloud applications 36 (e.g., on a network node 16) requiring varying amount GPU resources to accelerate the execution of their application 36. Each application 36 also has an expected execution time, i.e., for how long it expects to run on the GPU. In this example, the objective of the cloud when allocating resources is to fairly schedule the execution of all applications such that all have minimal but comparable relative completion times. In some embodiments, if one application 36 waits for half of its expected execution time to start running, then all other applications 36 may wait for half of their expected execution times to start running. This may be achieved by defining the objective function as minimizing the average normalized completion time.

[0120] In this nonlimiting example, there are four users who want to isolate their applications from each other. Each user wants to run fifty cloud applications 36 each, resulting in a total workload of two hundred cloud applications across all the users. Additional details about the resource requirements of the applications are provided in Table 1.

[0121] Table 1: Applications resource requirements.

[0122] A cloud data center with 15 NVIDIA A100-SXM4-40GB GPUs may be used.

[0123] When using A100 GPU with MIG technology as a hardware partitioning technology, the GPU can be partitioned into up to 7 units 210, each with one unit of a specific and completely separate GPU memory slice and GPU Streaming Multiprocessor (SM) slice. In some embodiments, units 210 refer to processing units or PUs 32. In some other embodiments, a unit may represent an instance or instance of a processing unit. The A100 can be flexibly configured into one GPU (7 units 210) or multiple smaller units 210, including 1, 2, 3 or 4 units, offering adaptable computational capabilities. Along with the constraint in the size of the unit, there may be also the limitation in the MIG unit configuration and only a specific list of unit combinations is supported in A100.

[0124] FIG. 7 shows example A100 valid units 210 of a MIG Unit configuration policy. The policy of selecting the units 210 is that for a valid combination, where the first unit 210 may be selected from the left side. Once a partition is chosen, the next selections must exclude units 210 positioned below it. Consequently, the list of supported unit combinations may include 19 different combinations. Considering a unit of GPU to be the smallest possible partition in the A100 GPU, each GPU in this example has a capacity of 7 units, giving a total GPU capacity of 105 units, distributed across 15 GPUs. The 105 GPU units may be available to meet the various resource requirements of the applications, ranging between 0 to 7 units. Additionally, the execution time for each application may vary, e.g., between 5 to 20 time units, reflecting the complexities of computational demands in this user-case cloud computing scenario.

[0125] The following are nonlimiting example step-by-step results.

[0126] Results of S200: Table 2 indicates the results of selecting best-fit hardware partitions (MIG units) for applications 36 of each user using MPS as a software partitioning technique and an LP simplex solver.

[0127] Table 2: Integrating non-isolated applications results.

[0128] Results of S202: This step considers the total hardware partitions (MIG units) required for all the users and finds the optimal hardware partition combinations (MIG unit configurations) to minimize GPU utilization using the LP simplex solver. The information provided in Table 3 shows the results of this step:

[0129] Table 3: Configuring hardware partitions results.

[0130] Results of S204: In this step, the grouped applications 36 are sorted based on the maximum completion time of each group. Considering that applications 36 are executed in parallel in each group, the maximum completion time of the application within the group may be when they release the associated hardware partition. As a result, sorting them based on the maximum completion time and prioritizing grouped applications with shorter release times enables meeting the objective of allocation, which may be fairness allocation. Rather than using a list of two hundred applications, for ease of understanding, the three first and last grouped applications are used, after sorting them in this step.

[0131] Results of S206: In step S202, 112 MIG configurations in 3 distinct categories were selected: (7), (4,3), and (2,2,3) to partition available GPUs, where, for example, (2,2,3) indicates that the GPU may be partitioned into a 2-unit, 2-unit, and 3-unit instances. Since there are 15 GPUs available in this example, these configurations can be prioritized by assigning scores (e.g., priority scores). The allocation objective in this example is fairness, and the minimization of the average normalized completion time is sought. In this example, the CCV may be the maximum execution time of the MIG units in each MIG unit configuration, e.g., after scoring the method sorted MIG unit configuration ascendingly based on their score (e.g., priority score) for partitioning each GPU. The information of scoring and sorting MIG configurations for one GPU are listed in follows:

[0132] Table 5: Sorting and prioritizing hardware partition combinations results.

[0133] Results of S208: In this step, an available GPU may be partitioned by selecting the MIG unit configuration with the minimum score, and the applications 36 of each unit will be executed accordingly. In the case of having an equal score, the combination with more partitions may be selected. This allows more applications 36 to start sooner. The results of selected partitions for two GPUs are provided below:

[0134] Table 6: scheduling applications and partitioning GPUs results.

[0135] Further, Table 7 summarizes the results of the partitioning and scheduling process of 200 applications for the proposed method and the baseline method. The baseline method schedules applications by sorting them based on their execution time and then schedules the applications with shorter execution times earlier to achieve the minimum average normalized completion time for all the applications.

[0136] Table 7: Evaluation results.

[0137] The example results show how the embodiments of the present solution, by grouping applications 36, selects different unit sizes compared to the baseline schedule method. The method displays significant improvement over a baseline schedule method in terms of fair resource allocation and GPU resource utilization. Further, in some embodiments, the method achieves an average normalized completion time of 0.253, which outperforms the baseline schedule method by approximately 29.1%, with a comparatively higher average normalized completion time of 0.357. This significant reduction indicates the fair resource allocation between the cloud applications of the method. Furthermore, the waste of GPU resources of this example is 5, which, compared to the 136 waste of resources in the baseline method, shows a 96.3% reduction. This substantial decrease highlights the optimized allocation and utilization of GPU resources in the proposed method. Further, the total time required to complete all applications 36 is reduced to 96 time units, demonstrating a 15.9% improvement over the baseline method, which took 113 time units.

[0138] As will be appreciated by one of skill in the art, the concepts described herein may be embodied as a method, data processing system, computer program product and / or computer storage media storing an executable computer program. Accordingly, the concepts described herein may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects all generally referred to herein as a “circuit” or “module.” Any process, step, action and / or functionality described herein may be performed by, and / or associated to, a corresponding module, which may be implemented in software and / or firmware and / or hardware. Furthermore, the disclosure may take the form of a computer program product on a tangible computer usable storage medium having computer program code embodied in the medium that can be executed by a computer. Any suitable tangible computer readable medium may be utilized including hard disks, CD-ROMs, electronic storage devices, optical storage devices, or magnetic storage devices.

[0139] Some embodiments are described herein with reference to flowchart illustrations and / or block diagrams of methods, systems and computer program products. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer (to thereby create a special purpose computer), special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0140] These computer program instructions may also be stored in a computer readable memory or storage medium that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instruction means which implement the function / act specified in the flowchart and / or block diagram block or blocks.

[0141] The computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0142] It is to be understood that the functions / acts noted in the blocks may occur out of the order noted in the operational illustrations. For example, two blocks shown in succession may in fact be executed substantially concurrently or the blocks may sometimes be executed in the reverse order, depending upon the functionality / acts involved. Although some of the diagrams include arrows on communication paths to show a primary direction of communication, it is to be understood that communication may occur in the opposite direction to the depicted arrows.

[0143] Computer program code for carrying out operations of the concepts described herein may be written in an object oriented programming language such as Python, Java® or C++. However, the computer program code for carrying out operations of the disclosure may also be written in conventional procedural programming languages, such as the "C" programming language. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer. In the latter scenario, the remote computer may be connected to the user's computer through a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0144] Many different embodiments have been disclosed herein, in connection with the above description and the drawings. It will be understood that it would be unduly repetitious and obfuscating to literally describe and illustrate every combination and subcombination of these embodiments. Accordingly, all embodiments can be combined in any way and / or combination, and the present specification, including the drawings, shall be construed to constitute a complete written description of all combinations and subcombinations of the embodiments described herein, and of the manner and process of making and using them, and shall support claims to any such combination or subcombination.

[0145] It will be appreciated by persons skilled in the art that the embodiments described herein are not limited to what has been particularly shown and described herein above. In addition, unless mention was made above to the contrary, it should be noted that all of the accompanying drawings are not to scale. A variety of modifications and variations are possible in light of the above teachings and the following claims.

Claims

What is claimed is:

1. A method in a network node (16) configured to manage at least one processing unit resource, PU, (32) of a plurality of PUs (32) in a network (12, 14) comprising a plurality of applications (36), the at least one PU (32) being configured to execute at least one application (36) of the plurality of applications (36), the method comprising: grouping (S100) the at least one application (36) of the plurality of applications (36) into at least one group of applications (36) based on hardware partitioning constraints of the plurality of PUs (32) and an application isolation condition, the at least one group of applications (36) being executable using a software partitioning process; configuring (S102), based on the grouping of the at least one application (36), at least one hardware partition (206) on the at least one PU (32) that fulfills minimization of resource wastage and meets a condition of at least one objective function; and at least one of sorting, mapping and running (S104) the at least one group of applications (36) on the at least one hardware partition (206) using the software partitioning process and according to the objective function.

2. The method of Claim 1, wherein the method further includes: obtaining information associated with the plurality of applications (36) and the plurality of PUs (32) to group the at least one application (36).

3. The method of any one of Claims 1 and 2, wherein the at least one application (36) is grouped for concurrent execution.

4. The method of any one of Claims 1-3, wherein the objective function includes an allocation objective associated with a minimization of a normalized completion time for a predetermined resource allocation.

5. The method of any one of Claims 1-4, wherein the application isolation condition requires each application (36) in the one group of applications (36) to be isolated from the applications (36) in other groups of applications (36).

6. The method of any one of Claims 1-5, wherein configuring the at least one hardware partition (206) includes: determining a combination of the at least one partition (206) to partition the at least one PU (32) to fulfill minimization of resource wastage and meet the condition of the at least one objective function, the condition being maximizing the at least one objective function.

7. The method of any one of Claims 1-6, wherein at least one of sorting, mapping and running the at least one group of applications (36) includes: sorting and prioritizing the at least one group of applications (36) based on the corresponding contribution to the objective function.

8. The method of any one of Claims 1-7, wherein at least one of sorting, mapping and running the at least one group of applications (36) includes: prioritizing at least one hardware partition combination for sequentially partitioning the at least one PU (32), the prioritizing including at least one of: determining and assigning a score to each hardware partition combination; arranging each hardware partition combination by the corresponding score; and determining a switch partition combination overhead and consensus characteristic values of the at least one hardware partition combination to determine the score.

9. The method of any one of Claims 1-8, wherein the method further includes: scheduling the at least one hardware partition (206) on the at least one PU (32) and the execution of the at least one application (36) based on the at least one of sorting, mapping and running of the at least one group of applications (36).

10. The method of any one of Claims 1-9, wherein at least one of: the at least one PU (32) is comprised in the network node (16); the at least one PU (32) is comprised in another network node (16) different from the network node (16); and the at least one PU (32) is at least one graphics processing unit, GPU.

11. A network node (16) configured to manage at least one processing unit resource, PU, (32) of a plurality of PUs (32) in a network (12, 14) comprising a plurality of applications (36), the at least one PU (32) being configured to execute at least one application (36) of the plurality of applications (36), the network node (16) being configured to: group the at least one application (36) of the plurality of applications (36) into at least one group of applications (36) based on hardware partitioning constraints of the plurality of PUs (32) and an application isolation condition, the at least one group of applications (36) being executable using a software partitioning process; configure, based on the grouping of the at least one application (36), at least one hardware partition (206) on the at least one PU (32) that fulfills minimization of resource wastage and meets a condition of at least one objective function; and at least one of sort, map and run the at least one group of applications (36) on the at least one hardware partition (206) using the software partitioning process and according to the objective function.

12. The network node (16) of Claim 11, wherein the network node (16) is further configured to: obtain information associated with the plurality of applications (36) and the plurality of PUs (32) to group the at least one application (36).

13. The network node (16) of any one of Claims 11 and 12, wherein the at least one application (36) is grouped for concurrent execution.

14. The network node (16) of any one of Claims 11-13, wherein the objective function includes an allocation objective associated with a minimization of a normalized completion time for a predetermined resource allocation.

15. The network node (16) of any one of Claims 11-14, wherein the application isolation condition requires each application (36) in the one group of applications (36) to be isolated from the applications (36) in other groups of applications (36).

16. The network node (16) of any one of Claims 11-15, wherein configuring the at least one hardware partition (206) includes:determining a combination of the at least one partition (206) to partition the at least one PU (32) to fulfill minimization of resource wastage and meet the condition of the at least one objective function, the condition being maximizing the at least one objective function.

17. The network node (16) of any one of Claims 11-16, wherein at least one of sorting, mapping and running the at least one group of applications (36) includes: sorting and prioritizing the at least one group of applications (36) based on the corresponding contribution to the objective function.

18. The network node (16) of any one of Claims 11-17, wherein at least one of sorting, mapping and running the at least one group of applications (36) includes: prioritizing at least one hardware partition combination for sequentially partitioning the at least one PU (32), the prioritizing including at least one of: determining and assigning a score to each hardware partition combination; arranging each hardware partition combination by the corresponding score; and determining a switch partition combination overhead and consensus characteristic values of the at least one hardware partition combination to determine the score.

19. The network node (16) of any one of Claims 11-18, wherein the network node (16) is further configured to: schedule the at least one hardware partition (206) on the at least one PU (32) and the execution of the at least one application (36) based on the at least one of sorting, mapping and running of the at least one group of applications (36).

20. The network node (16) of any one of Claims 11-19, wherein at least one of: the at least one PU (32) is comprised in the network node (16); the at least one PU (32) is comprised in another network node (16) different from the network node (16); and the at least one PU (32) is at least one graphics processing unit, GPU.

21. A system (10) comprising a first network node (16) and at least one second network node (16), the first network node (16) being configured to manage at least one processing unit resource, PU, (32) of a plurality of PUs (32) in a network (12, 14) comprising a plurality of applications (36), the at least one PU (32) being configured to execute at least one application (36) of the plurality of applications (36), the first network node (16) being configured to: obtain, from the at least one second network node (16), information associated the plurality of applications (36) and the plurality of PUs (32); group the at least one application (36) of the plurality of applications (36) into at least one group of applications (36) based on hardware partitioning constraints of the plurality of PUs (32), an application isolation condition, and the obtained information, the at least one group of applications (36) being executable using a software partitioning process; configure, based on the grouping of the at least one application (36), at least one hardware partition (206) on the at least one PU (32) that fulfills minimization of resource wastage and meets a condition of at least one objective function; and at least one of sort, map and run the at least one group of applications (36) on the at least one hardware partition (206) using the software partitioning process and according to the objective function.

Images