Method and apparatus for vectorized resource scheduling in a distributed computing system using tensors

Abstract

A method and apparatus for vectorized resource scheduling using multidimensional hierarchical arrays of uniform data types are provided. According to various embodiments, in the context of scheduling, a resource hierarchy can be represented as a multidimensional array of uniform data types. This data structure is referred to as a tensor. In various embodiments, a tensor can be used to model each of resource usage, requested resources, resource capacity, tenant resource usage, and the like. Tensor scheduling operations (e.g., a tensor scheduling algorithm) are provided to utilize this resource model. Some embodiments can utilize specialized hardware such as GPUs and tensor processing units to accelerate the tensor scheduling operations.

Description

[0001] Cross-reference to related applications

[0002] This application claims priority to U.S. patent application Ser. No. 16 / 575,763, filed on Sep. 19, 2019, entitled “Method and Apparatus for Vectorized Resource Scheduling In Distributed Computing Systems Using Tensors,” the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present invention relates to resource scheduling in a distributed computing system, and in particular to a method and apparatus for resource management and scheduling in a distributed computing system using tensors. Background Art

[0004] In a distributed computing system, a resource scheduler matches work requests (i.e., requests for work to be performed) with the distributed computing system resources required to complete the work. In other words, the resource scheduler allocates resources in the distributed computing system to one or more work requests. After scheduling, the work request can be "proposed" (i.e., executed by the allocated resources) to the allocated resources.

[0005] Distributed computing systems receive a large number of work requests and manage and schedule a large number of resources. Furthermore, the number of work requests received by the distributed computing system and the number of resources in the distributed computing system are constantly increasing. Because the number of work requests and resources is large and constantly increasing, it is often necessary to manage resource management and scheduling so that the resource scheduler can meet the resource scheduling requirements. In order to manage the increasing number of work requests and resource management, effective parallelization techniques are needed because it is computationally difficult to match many resource requests with many consumable resources (e.g., CPU, memory) within many time intervals. Existing methods and systems for resource management and scheduling, even including techniques using parallelization, need to be more efficient. Therefore, a method and apparatus are needed to address one or more problems in the prior art.

[0006] This background information is provided to reveal information believed by the applicant to be potentially relevant to the present invention. It is not necessarily intended, nor should it be construed, that any of the foregoing information constitutes prior art against the present invention. Summary of the Invention

[0007] The purpose of the embodiment is to provide a method and apparatus for vectorized resource management and scheduling in a distributed computing system, which involves using tensors (multidimensional arrays with a uniform data type) to represent resource requests and representing a hierarchy of resources as tensors in a scheduling context.

[0008] According to an embodiment, a method for resource management and scheduling in a distributed computing system is provided. The method includes: using one or more resource tensors to represent the availability of resources or the amount of resource usage in the distributed computing system; receiving one or more resource requests, each resource request specifying resources required to satisfy a work request in the distributed computing system. The method also includes: representing the resource requests as one or more request tensors; and scheduling resources in the distributed computing system based on the one or more resource tensors and the one or more request tensors.

[0009] In some embodiments, the method further includes: using one or more capacity tensors to represent the total amount of the resources in the distributed computing system; and further scheduling the resources in the distributed computing system based on the one or more capacity tensors.

[0010] In some embodiments, the method further comprises: during scheduling, based on the values ​​in the one or more request tensors, updating the one or more resource tensors in response to a condition, the condition being one of: a request for work for a subset of one or more available resources; completion of the work request requiring the one or more resources; or release of the one or more resources requested by the work request.

[0011] In some embodiments, the method further comprises determining a subset of the one or more resource tensors, the subset of the one or more resource tensors having sufficient capacity for the work request. In some embodiments, the method further comprises determining a subset of the one or more resource tensors that best matches the requested resource, wherein the best match is determined based on calculating a fitness score for each subset of the one or more resource tensors when the work request is made for the requested resource. In some embodiments, the fitness score for each subset of the one or more resource tensors is determined based on a similarity (in a mathematical sense, e.g., cosine similarity) between total resource usage after the request is made for the resource and the total capacity of the resource.

[0012] According to an embodiment, a device for resource management and scheduling in a distributed computing system is provided. The device includes a processor and a memory storing machine-executable instructions. When the instructions are executed by the processor, the device is configured to: use one or more resource tensors to represent the availability of resources or the usage of resources in the distributed computing system; receive one or more resource requests, each resource request specifying the quantity of specific resources required to satisfy a work request in the distributed computing system. When the instructions are executed by the processor, the device is further configured to: represent the resource requests as one or more request tensors; and schedule resources in the distributed computing system based on the one or more resource tensors and the one or more request tensors.

[0013] According to an embodiment of the present invention, a network node for performing resource management and scheduling in a distributed computing system is provided. The network node includes: a network interface for receiving data from a component connected to a distributed computing network and transmitting data to the component; a processor; and a non-transitory memory for storing instructions. When the instructions are executed by the processor, the network node is configured to: use one or more resource tensors to represent the availability of resources or the usage of resources in the distributed computing system; receive one or more resource requests, each resource request specifying the number of specific resources required to satisfy a work request in the distributed computing system. When the instructions are executed by the processor, the network node is further configured to: represent the resource request as one or more request tensors; and schedule resources in the distributed computing system based on the one or more resource tensors and the one or more request tensors.

[0014] The above describes embodiments in conjunction with various aspects of the present invention, and these embodiments can be implemented based on these aspects. Those skilled in the art will appreciate that the embodiments can be implemented in conjunction with the aspects describing these embodiments, but can also be implemented together with other embodiments of the same aspect. When the embodiments are mutually exclusive or incompatible with each other, it will be apparent to those skilled in the art. Some embodiments may be described in conjunction with one aspect, but may also be applicable to other aspects, which will be apparent to those skilled in the art.

[0015] Some aspects and embodiments can provide numerous advantages, including increased speed due to massively parallel computing, improved predictability of execution (e.g., reduced tail latency), the ability to use appropriate levels of precision (e.g., utilizing low-precision arithmetic, improving computational efficiency), and reduced power consumption (e.g., operating at higher input-output operations per second (IOPS) per watt). Furthermore, offloading at least some scheduling to a coprocessor can free host CPU resources to complete requested work. This can result in improvements where scheduling does not interfere with completing work, nor does completing work interfere with scheduling. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Further features and advantages of the present invention will become apparent from the following detailed description taken in conjunction with the accompanying drawings, in which:

[0017] Figure 1 The invention shows a distributed computing system for vectorized resource scheduling using tensors provided by an embodiment;

[0018] Figure 2 shows the hierarchical structure of resources in a distributed computing system;

[0019] Figure 3 Shows the resource schedule for jobs planned to run in the future;

[0020] Figure 4 An example of scheduling operations in a distributed computing system is shown, specifically, checking whether a host has sufficient resources for a work request;

[0021] Figure 5A The tensor-based resource representation provided by the embodiment is shown to encode the resource hierarchy by converting the conceptual resource hierarchy into a tensor representation;

[0022] Figure 5B The concept of treating the time dimension as a consumable resource in the resource hierarchy provided by the embodiment is shown;

[0023] Figure 6A An example of resource hierarchy representation in an a priori resource model is shown;

[0024] Figure 6B shows the tensor target dimension in the resource tensor provided by the embodiment;

[0025] Figure 7 The following illustrates a logical circular buffer and circular buffer implementation method of the time dimension in a resource tensor provided by an embodiment;

[0026] Figure 8The process of determining whether a request matches a host by a resource scheduler provided by an embodiment is shown;

[0027] Figure 9 The process of using cosine similarity to determine the best host to which a request is made is shown;

[0028] Figure 10 A schematic diagram of an electronic device provided by an embodiment is shown.

[0029] It should be noted that throughout the drawings, like features are identified by like reference numerals. DETAILED DESCRIPTION

[0030] Terminology

[0031] The term "about" as used herein should be understood to include variations relative to the nominal value, for example, variations of + / - 10% relative to the nominal value. It should be understood that given values ​​provided herein always include such variations, whether specifically mentioned or not.

[0032] The term "Advanced Vector Extensions" (AVX) refers to the instruction set that supports single-instruction multiple-data (SIMD) operations. and A set of computer instructions for a microprocessor. For example, AVX can perform specific operations on multiple tasks simultaneously in the same amount of time as it would take to perform the same operation on one task (i.e., parallel processing).

[0033] The term "allocation problem" refers to one of the basic combinatorial optimization problems (i.e., a problem with a solution obtained from a combination of a set of finite elements) in which a maximum weight matching is sought in a weighted bipartite graph, i.e., identifying a maximum weight matching of nodes in one group with nodes in another group, where the nodes are in two groups (e.g., 'requests' and 'hosts'), and each request is required to go to exactly one host, and each host is connected to exactly one request. In scheduling, the weights can encode how well each request is a 'match' with each host (e.g., by using cosine similarity). If the number of elements in each group is different, then this is an 'asymmetric' allocation problem. In general, scheduling can be an asymmetric allocation problem.

[0034] The term "broadcasting" refers to the way that NumPy, PyTorch, TensorFlow, and / or other software libraries handle tensors with different shapes during vectorized arithmetic operations, essentially aligning and copying the tensor dimensions so that the operation can continue. The 'shape' of a tensor is described below, and it refers to the number of dimensions and the number of elements (i.e., component values) in each dimension. For example, when there is an operation involving two tensors with different shapes, for each dimension, the 'broadcasting' technique means that NumPy behaves as if it 'copies' the smaller array dimension (and its values) as many times as necessary to match the larger dimension so that the arrays can have compatible shapes and the arithmetic operation can continue. Using the broadcasting technique can generally ensure that the algorithm is implemented very efficiently compared to alternative approaches (e.g., explicit copying or looping in the software code).

[0035] In the context of distributed scheduling, the term "cluster" refers to a group of computers that are set up to execute distributed programs (e.g., programs that run across multiple machines). Physically, a cluster refers to multiple machines linked together. Each machine has a Central Processing Unit (CPU) core, memory, etc., and can be thought of as a 'host' or 'server' on which work should be run. From the perspective of the resource scheduler, logically, a cluster refers to a group of resources (e.g., resources can be CPU cores, memory, storage devices, networks, Graphics Processing Units (GPUs), port numbers, software licenses, etc.) that the resource scheduler must match with distributed programs (e.g., Operating System (OS) processes, serial / parallel batch jobs, Virtual Machines (VMs), containers, serverless functions, etc.) that request resources to complete their work.

[0036] The term "data type" refers to the properties of variables / data in a computer program that restrict the types of values ​​the variables can take, the operations that can be performed on them, and how they are stored and manipulated on a computer system. Common examples include Boolean data types, integer data types, and real-valued (floating-point) data types.

[0037] The term "distributed computing system" refers to a computing system whose components or resources are located on different networked computers.

[0038] The term "Graphics Processing Unit" (GPU) refers to specialized electronic circuits originally designed to quickly generate images for output to display devices. GPUs have been used to perform general-purpose parallel algorithms, particularly matrix multiplication and related calculations (e.g., General-Purpose Graphics Processing Unit (GPGPU)-based calculators). GPUs parallelize operations similarly to how AVX / SIMD instructions are processed on a CPU, but GPUs further parallelize the processing operations.

[0039] The term "host" refers to the host machine (eg, a server) on which the resource scheduler runs a job.

[0040] The term "Input-Output Operations Per Second" (IOPS) refers to a performance measurement used to benchmark storage devices.

[0041] The term "maximum weight matching problem" refers to the problem of finding a matching in a weighted graph that maximizes the sum of its weights. A special case of the maximum weight matching problem is the assignment problem (described above), where the input is restricted to a bipartite graph.

[0042] The term "NumPy" refers to a scientific (software) library for the Python programming language. NumPy adds support for large multidimensional arrays and matrices (i.e., 'tensors', but in NumPy, tensors are often called 'arrays') and a large number of operations on these data structures.

[0043] The term "Operating System" (OS) refers to the software running on a computer that schedules tasks, executes application programs, and controls the disk drive, display, and / or other peripheral devices.

[0044] The term "PyTorch" refers to a machine learning (scientific) library for the Python programming language. PyTorch provides a set of specialized operations for neural networks. PyTorch can be described as 'NumPy on the GPU.'

[0045] In the context of distributed computing systems, the term "resource management" (RM) can be understood as scheduling, where resources span multiple machines and the work to be performed consists of batch jobs (batches of individual tasks to be run in parallel). RM can also be considered a part of scheduling, including (1) resource definition and discovery, aggregation, and presentation; and (2) resource lifecycle monitoring. In this application, the resource manager tracks the resources in the distributed computing system that are available for scheduling or using resources in the distributed computing system, while the resource scheduler performs the matching part, where resource requests are matched with appropriate resources.

[0046] The term "resource model" refers to the data structure, i.e., the data organization and storage format, in which resource information, such as parameters indicating specific resources (e.g., CPU characteristics, memory characteristics, allocated resource usage, etc.), is recorded, updated, and communicated within a resource management system. A resource model is a computer-internal representation of real physical resources that can be used when requesting and scheduling computing resources.

[0047] In computing, the term "scheduling" refers to the method of allocating resources to work (e.g., 'requests') over a period of time to complete that work. Scheduling can also be thought of as 'making' requests for resources. A work request can be, for example, a request to run a process on an operating system, a user-specified batch job, or a virtual machine (VM) running on a server. Resources can include CPU cores, memory, storage devices, network ports, GPUs, and so on. Every computer, including all smartphones and laptops, has an OS that includes a resource scheduler that determines which processes should be running at any given time. Similarly, on a larger scale, such as in a distributed (multi-machine) computing system, a resource scheduler may be needed to decide which work requests should be made to which machines, and so on. It should be understood that in a distributed computing system, multiple work requests can be made to a single host resource. For example, if a host server has 10 CPU cores and the scheduler receives a request to place virtual machines (VMs) where each VM requires 1 CPU core, the scheduler can place up to 10 VMs on a host (assuming there is enough memory and other resources on the host for all 10 VMs).

[0048] The term "scheduling operation" can be understood as the matching or work placement performed in scheduling. Scheduling operations can include: (i) checking which resources have sufficient capacity to satisfy a request (a subset of resources with 'enough' capacity to satisfy a work request); (ii) determining which resources are the 'best match' for the request; and / or (iii) "making" a request for a resource (e.g., recording in its internal data structures that the resource is in use and therefore unavailable for other requests).

[0049] The term "tensor" can be considered as a multidimensional array of values ​​of uniform data type (e.g., Boolean, floating point, integer, etc.). In programming languages, the values ​​of a tensor can be accessed by index. For example, a one-dimensional array (a one-dimensional tensor) is a vector whose values ​​can be accessed by array[0], array[1], or array

[10] , etc. A two-dimensional array (a two-dimensional tensor) is a matrix whose values ​​can be accessed by two indices, such as array[0][1] or array[5][7]. The generalization of scalars, arrays, and matrices to arbitrary dimensions (e.g., indexed by an arbitrary number of indices) is collectively referred to as a tensor. In the context of this application, resources can have a hierarchical structure and can therefore be defined as tensors. For example, each data center has multiple racks, each rack has multiple hosts (i.e., machines), each host has multiple processing units, and each processing unit has multiple cores.

[0050] The term tensor "shape" (or array "shape") refers to the dimensions of the tensor (a multidimensional array), for example 40x10x3. The tensor "shape" specifies the size of the tensor in each dimension. For example, if the shape of the tensor is 40x10x3, then the size of the tensor in each dimension is 40, 10, and 3, respectively. In this case, the tensor contains a total of 40*10*3=1200 values. The size of the tensor in each dimension is equal to the length of the multidimensional array in that dimension. If zero indexing is used (i.e., array indexing starts at 0), the length of the array is equal to the maximum index value of the array plus 1. (For example, when the shape of the tensor is 40x10x3, the array sizes in each dimension are 40, 10, and 3. Using this tensor, the array

[35] [6][2] represents a valid index, but the array

[50]

[50]

[50] does not represent a valid index because each index exceeds the size of the tensor in the corresponding dimension.)

[0051] The term "Single Instruction Multiple Data" (SIMD) refers to a parallel computing operation where a single 'instruction' (eg, "increase value by one") is executed simultaneously on a 'vector' of data points. SIMD instructions require corresponding hardware to implement the SIMD operation.

[0052] The term "tail latency" refers to high percentile latency values, such as the 98th or 99th percentile latency. Each time an operation is run on a computer system, the response time for that operation will be different. When the frequency of each response time is plotted as a histogram, this distribution of response times will have a tail. The tail of the response time distribution represents values ​​in the distribution where a large percentage of all latencies are below these values, for example, 98% of latencies are below these values.

[0053] The term "tenant" refers to a user or group of users that the cluster scheduler treats consistently. Tenants typically correspond to business units within an organization (e.g., sales, data science, marketing, and management). Tenants can also be arranged in a hierarchy. The concept of tenants emerged as clusters grew in size, and enterprises sought to improve resource utilization by sharing resources across multiple user groups. Regarding shared resources, one of the simple policies enforced by the (cluster) scheduler for tenants is that each tenant is entitled or guaranteed to have a certain amount of cluster resources (e.g., a 'guaranteed share'). However, when a tenant is not using its resources, other tenants can borrow the unused resources, up to a certain amount of all cluster resources (e.g., a 'maximum share').

[0054] The term "TensorFlow" refers to a machine learning library developed by Google that is primarily accessed through Python.

[0055] The term "Tensor Processing Unit" (TPU) refers to an application-specific integrated circuit (ASIC) that Google developed specifically for artificial intelligence (AI) accelerators for neural network machine learning.

[0056] In computer science, the term "vector" refers to a group of numbers. In this application, a 'vectorized' operation refers to an instruction that operates on multiple values ​​in parallel.

[0057] The term "virtual machine" refers to a computer program that emulates a computer system (e.g., a Windows or Linux computer with an entire file system, operating system, etc.), while itself running as a process on a host computer.

[0058] Unless defined otherwise, all technical and / or scientific terms used herein have the same meanings as commonly understood by one of ordinary skill in the art.

[0059] Embodiments relate to representing resource hierarchies as multidimensional arrays of a uniform data type in a scheduling context to improve resource scheduling efficiency. The multidimensional arrays of the uniform data type are referred to as 'tensors'. In other words, embodiments relate to using a 'tensor' resource model, and related vectorized tensor operations implemented by such a model, such as broadcasts. Such a resource model can be used for each of resource usage, requested resources, resource capacity, tenant resource usage, etc. Embodiments use tensor scheduling operations (e.g., tensor scheduling algorithms) to take advantage of the new resource model and its related operations. Some embodiments may utilize specialized hardware such as GPUs and tensor processing units to accelerate the tensor scheduling operations.

[0060] As previously mentioned, according to an embodiment, scheduling problems can be handled and solved by using tensors (multi-dimensional arrays of uniform data types) to represent resources. This common general vectorization model can be used for requested resources, resource usage, resource capacity, tenant resource usage, etc. The resource model can also be used to effectively represent scheduling operations (e.g., making requests to the host). The resource representation and scheduling operation set can be referred to as "tensor scheduling". In some embodiments, a circular buffer can be used for the time dimension. Other parts of this document (e.g., in conjunction with Figure 7 ) describes the use of the circular buffer, which can improve update efficiency. In some embodiments, dedicated hardware can be used to perform tensor scheduling operations.

[0061] The basic approach to resource representation described herein can have advantages, including improving the efficiency of implementing resource scheduling and scheduling operations. According to an embodiment, the resource representation allows (automatically) applying vectorized and parallelized arithmetic operations to kernel scheduling operations. Tensor broadcast operations can be efficiently performed using scientific (software) frameworks or libraries, such as NumPy and TensorFlow. Tensor arithmetic operations can be used to more efficiently implement scheduling operations. The scheduling operations may include: (i) logically making a request for a target resource; (ii) checking whether the resource capacity or availability is sufficient to satisfy the scheduling request; (iii) calculating the 'best match' (e.g., cosine similarity) for the scheduling request; (iv) determining the number of times the request can be matched with one or more hosts; (v) resolving conflicts. It should be noted that when a request is logically made for a target resource, the operation updates the internal resource model to reflect the reduced availability of the target resource after the request is made for the target resource.

[0062] By using tensors to represent resources, scheduling operations can be performed using specialized hardware optimized for matrix operations (e.g., CPUs for SIMD instructions, or specialized "coprocessors" such as GPUs, GPU tensor cores, and TPUs). Therefore, using tensor representations improves scheduling operations in several ways. For example, some specialized hardware, such as GPUs, GPU tensor cores, and TPUs, can process scheduling operations faster than conventional processing units (e.g., conventional CPUs) because they leverage massively parallel computations and use appropriate levels of precision for scheduling arithmetic. Because extremely precise numbers (e.g., 20 decimal places) are typically not required to determine whether sufficient resources (e.g., CPUs) are available on a server, some specialized hardware can use lower levels of precision for scheduling arithmetic to improve efficiency. This specialized hardware can also reduce power consumption, for example, achieving higher IOPS per watt. Furthermore, specialized hardware, such as GPUs, GPU tensor cores, or TPUs, can improve execution predictability, thereby reducing tail latency. If scheduling operations are run on a dedicated coprocessor, execution predictability can be improved because the scheduling operations are not delayed by other processes running on the processing unit. Furthermore, when scheduling operations are offloaded by the host processing unit to specialized hardware or accelerators such as GPUs, GPU tensor cores, or TPUs, the scheduling operations do not interfere with other host processing unit operations. In other words, the host processing unit's resources are not used for scheduling operations, but for processing unit operations. This allows for constraints such that scheduling operations do not interfere with processing unit operations, and vice versa.

[0063] Figure 1 An example of a distributed computing system 100 for vectorized resource scheduling using a multi-dimensional hierarchical array with a uniform data type is shown. Figure 1 Each client device 120 can be communicatively and operatively connected to the computing device 140 and the server racks 180a and 180b via the network 130. The computing device 140 and the server racks 180a and 180b can also be communicatively and operatively connected to each other via the network 130.

[0064] According to an embodiment, the client device 120 may be any device directly used by an end user for communication, such as a mobile phone, a smartphone, a computer with network capabilities, or other devices that may be classified as user equipment (UE). The client device 120 may be a machine type communication (MTC) device (also known as a machine-to-machine (M2M) device), or other such devices that may be classified as UEs even if they do not provide direct services to users.

[0065] Depending on the embodiment, the computing device 140 may be placed in the same data center where the server racks 180a and 180b are placed, or may be placed in another data center. Figure 1 Although not shown, there may be one or more other computing devices in the distributed computing system 100. These computing devices may be located in the same data center where the computing device 140 is located, or they may be located in other data centers. The computing device 140 may include one or more resource schedulers 160. Figure 1 Although not shown, one or more other resource schedulers may be present in the computing device 140. The resource scheduler 160 may perform one or more scheduling operations as described above (e.g., making a request to a host, determining whether the request matches a host, finding the best host to make the request to, resolving conflicts, etc.).

[0066] According to an embodiment, there may be one or more data centers in the distributed computing system 100. Each of the data centers may include one or more server racks (e.g., server rack 180a or 180b). The server racks may be similar in one or more aspects (e.g., components, functions) to the server racks. Figure 1 The server racks 180a and 180b shown in FIG are similar. In some embodiments, the server racks 180a and 180b can be placed in the same data center. In some embodiments, the server racks 180a or 180b can be placed in two different data centers. Figure 1 , the server rack 180a includes multiple resource servers 110a, and the server rack 180b includes a single resource server 110b. Figure 1 It is shown that there are only two resource servers 110 a in the server rack 180 a , but it is easy to understand that more than two resource servers 110 a can be placed in the server rack 180 a .

[0067] According to an embodiment, each of the resource servers 110a and 110b may include a resource (e.g., resource 150). Each resource may in turn include one or more other resources, which in turn may include other resources until a level of consumable resources (e.g., CPU, GPU, memory, etc.) is reached. For example, referring to Figure 1 The server resources 150 may include one or more processing units (e.g., CPU 152, GPU 154, TPU, etc.), memory (e.g., memory 156), storage devices (e.g., storage device 158), and other consumable resources that may be required for the request. In some embodiments, each of these consumable resources may be a composite resource, and the composite resource may be composed of other consumable resources or composite resources.

[0068] As mentioned above, resource scheduling is the process of matching work requests with the resources needed to complete the work. Once matched, the request can be considered a 'proposal' for its assigned resources. Resource scheduling can be complex due to the hierarchical relationships between various resources. Figure 2 Shows the hierarchical structure of resources in a distributed computing system. Figure 2 , a single host resource 200 named 'myserver1' includes 'cpu' 210, 'mem' 220, 'disk_capacity' 230, and 'disk_bandwidth' 240 as its resources; therefore, the relationship between the host 200 'myserver1' and other resources (e.g., 'cpu' 210, 'mem' 220, 'disk_capacity' 230, and 'disk_bandwidth' 240) is a hierarchical relationship. The host 200 'myserver1' can be considered a 'composite' resource (i.e., composed of other resources). On the other hand, Figure 2 The "leaf-level" resources shown in , such as 'cpu' 210, 'mem' 220, 'disk_capacity' 230, and 'disk_bandwidth' 240, can be considered as 'consumable' resources. The concept of resource hierarchy can be extended. Although Figure 2 Although not shown, the host 200 named 'myserver1' may be just one host in a host rack; the host rack may be one of multiple host racks in a data center. Furthermore, the distributed computing system may have multiple data centers. Each of these higher levels can be considered a composite resource in the resource hierarchy.

[0069] For resources with a hierarchical structure, scheduling includes matching work requests with a target resource at a target level in the resource hierarchy. Specifically, at the target level, scheduling may include matching work requests (e.g., a VM) with a host (the target resource), where a single host is implicitly placed on a host rack in a data center (e.g., a work request will be matched with one host 'or' another host). At the same time, at levels below the target level, work requests typically require a specified portion of all consumable resources below the target resource (e.g., digital resources at the leaf level of the hierarchy rooted in the target resource, i.e., below the host). Therefore, at levels below the target level, scheduling may include matching consumable values ​​in a work request with all consumable resources in the resource hierarchy rooted in the target resource. For example, a VM may require the 'cpu' 210, 'mem' 220, 'disk_capacity' 230, and 'disk_bandwidth' 240 of the host 200 'myserver1'.

[0070] For example, consider a work request to place a VM, where the resource requirements of the work request are 1 CPU core and 2GB of memory. Assume that the goal is to find a host to which to make this work request. Assume that there is a data center, and the data center has a host rack, and the host rack has a host, and the host has 10 CPU cores and 35GB of available memory. Scheduling involves determining that this particular host has enough consumable resources so that the work request can be made to the host. If the resource scheduler does make the work request to the host, the resource scheduler needs to update its resource information to reflect that after the work request is made, there will now be only 9 CPU cores and 34GB of memory left on the host.

[0071] Due to recent trends in distributed computing systems, resource scheduling has become more complex and computationally intensive. Essentially, resource scheduling is being performed on a massive scale. One current trend is the large number of target resources. For example, there may be 20,000 to 100,000 hosts, potentially processing over 10,000 requests per second. Consequently, resource scheduling is now extremely complex and computationally intensive, given that each of these hosts has a large number of consumable resources, such as CPU and memory, that need to be tracked.

[0072] Another trend in computing resource scheduling is the need or expectation to manage resource scheduling over time. In the past, tasks or work requests could start immediately and be scheduled on any idle host. However, current resource scheduling often requires more resources because available resources now need to be reserved in advance, as discussed further below.

[0073] refer to Figure 3 , the "resources" shown may refer to CPU cores on a host computer, and work 301 is scheduled to execute at some time in the future (using one or more CPU cores). In this case, the resource scheduler may need to determine whether work 302, work 303, or work 304 can be placed on the currently available resources so that one or more of work 302, work 303, or work 304 can be started immediately, while still ensuring that work 301 will have sufficient resources to run at its designated time in the future. This technique is called 'backfilling'. However, if the time required to complete work 302, work 303, or work 304 is too long, backfilling cannot always be used because work 301 cannot be started as planned.

[0074] In another case, if there is planned maintenance for the host at some point in the future, and therefore all CPU cores on the host are unavailable during the planned maintenance (e.g., no work can be performed on the host during the planned maintenance), the resource scheduler may also need to determine whether there is sufficient time to start and run one or more work requests to completion on the host before the planned maintenance period begins.

[0075] In order to make more effective decisions about resource scheduling in the above situation, it is necessary to determine the time required to complete the requested work. If the time required to complete the work is unknown, the resource scheduler can predict the resource usage of the work over a period of time and the duration of the work's use of the resources.

[0076] Furthermore, since there are some work requests with fixed timeouts (e.g., a time limit or maximum duration that the work can be executed), the resource scheduler may not need to predict the exact duration of the work, but instead use the time limit as the predicted duration. For example, many serverless functions have time limits, and many cloud service providers support serverless functions (e.g., Lambda).

[0077] As shown above, in the context of resource scheduling in the distributed computing system, the time dimension can be a highly complex concept. However, in order to achieve effective resource scheduling, it may be necessary to consider the time dimension. To this end, embodiments propose using the time dimension as another resource dimension in the resource hierarchy.

[0078] Resource schedulers typically lack a common way to represent models. While some resource schedulers may include data structures in their source code, these data structures are only available for certain relevant resources (e.g., 'cpu' and 'memory'). In many prior resource representation models (e.g., YARN, OpenStack), resources are hard-coded, independent objects represented only by scalar values ​​and lack a time dimension. Furthermore, these models are not resource models that can encode user-defined resources.

[0079] There are several general resource models. One example is the flat resource model proposed by Chan and Curry in U.S. Patent No. 6,829,765. The flat resource model generally represents resources in a resource management (RM) system. However, this approach cannot be used to represent hierarchical resource structures. Another example of a general resource model is the general resource model for distributed resource management proposed by Guo, Chen, and Lam in U.S. Patent Application Publication No. 2017 / 0153918A1. This model can represent a hierarchical structure of resources. In this general resource model, the underlying resource values ​​are scalar values ​​of different data types. This resource model can have multiple data types (e.g., a combination of Boolean values, string values, and floating values), so this resource model is a flexible model. However, when the number of resources to be scheduled increases, this general resource model may become inefficient.

[0080] In the context of hierarchical resources, the primary function of a resource scheduler is to match work requests with target resources in the target dimension of the resource hierarchy. This function can encompass three subproblems: (i) determining which target resources have sufficient consumable resources to satisfy the request; (ii) determining which target resources are the 'best match' for the request based on appropriate criteria (e.g., spreading the request across hosts); and (iii) resolving conflicts when they arise, particularly when scheduling decisions are made in parallel. Solving these three subproblems can be considered a 'scheduling operation'. Figure 4 An example of scheduling operations in a distributed computing system is shown, specifically, checking whether a host has sufficient resources for a work request.

[0081] refer to Figure 4Each square in the upper grid represents the consumable resources (e.g., memory 411, CPU 412, and network 413) required by work request 410'i' over time, while each square in the lower grid represents the current usage of these consumable resources (e.g., memory 421, CPU 422, and network 423) on host 420'h' over time. Specifically, the color of each square in the upper grid represents the amount of consumable resources (e.g., memory 411, CPU 412, and network 413) required by work request 410'i' at a specific time, while the color of each square in the lower grid represents the current usage of these consumable resources (e.g., memory 421, CPU 422, and network 423) on host 420'h' at a specific time. The darker the color of the square in the upper grid, the more consumable resources the work request required at that specific time. Similarly, the darker the color of the square in the lower grid, the greater the usage of each consumable resource in the host at that specific time. On the other hand, a white square indicates that the consumable resource is not used or not needed, and a black square indicates that the consumable resource is fully used or completely needed.

[0082] refer to Figure 4 , the horizontal axis represents time or time steps. For each host / request check (e.g., work request 410 / host 420), the resource scheduler can check whether the usage of each consumable resource (e.g., memory 421, CPU 422, network 423, indexed as r=1...R) is within its capacity at each time step (e.g., t=1...T). In other words, the task of the resource scheduler is to check whether the sum of the consumable resources required by the request (i.e., each square of the upper grid) and the consumable resources used by the host (i.e., each square of the lower grid) is less than the consumable resource capacity at each time step across all time steps. When the usage reaches 100% (e.g., the square is black), the capacity of each consumable resource can be equal to the usage of the consumable resource. When there are 'I' requests and 'H' host resources (each host resource has 'R' consumable resources), the resource scheduler can determine whether each request (i.e., request i=1...I) is matched with each host resource (i.e., host resource h=1...H) so that for each time step (i.e., time step t=1...T), the usage of each consumable resource (i.e., resource r=1...R) is within its capacity after adding the requested value. For the entire checking process, the resource scheduler needs to perform a total of I*H*R*T checks.

[0083] In this regard, scheduling is computationally expensive (e.g., more checks) relative to resources and time. If there are 5,000 hosts (i.e., H=5,000) and 6 consumable resources (i.e., R=6) (e.g., CPU, memory, network, etc.) to perform scheduling checks, and the checks need to be completed for 7,200 time steps (i.e., T=7200) (note: if 1 time step equals 1 minute, then 7,200 time steps equals 5 days), then each request (i.e., I=1) will have 216 million values ​​to check. If there are 500 work requests (i.e., I=500) that need to be scheduled, then the resource scheduler needs to check 108 billion values.

[0084] To manage this computationally expensive (e.g., large-scale) resource scheduling problem, it may be necessary to parallelize resource scheduling operations. With the scalar-based resource model described above, scheduling can only be efficiently parallelized across requests (e.g., parallelized at the request level). Requests can be load balanced across multiple resource schedulers with or without multiple resource pools (e.g., islands), as shown below.

[0085] One approach to parallelizing requests is to partition resources into separate resource pools (e.g., islands) and assign a resource scheduler to each pool to match requests only with resources in that pool. This parallelization approach can lead to fragmentation. In other words, even if the total amount of available resources is sufficient to satisfy a work request, those available resources cannot be used for the work request because the resources are 'fragmented' across different resource pools.

[0086] Another approach to parallelizing requests is to have multiple resource schedulers within a common resource pool handle subsets of work requests. It should be understood that this parallel processing approach does not result in fragmentation between resource pools. However, with this approach, because each resource scheduler makes independent decisions, there is always the possibility of conflicting decisions. Therefore, when conflicts arise between different resource schedulers, the system requires a conflict resolution mechanism. This conflict resolution mechanism can add additional overhead to the system, potentially making scheduling more complex.

[0087] Because scheduling parallel processing between requests is inadequate, a better resource scheduling parallel processing method is needed. One way to overcome the above problems is to parallel process scheduling 'within' a request (i.e., perform parallel operations to schedule a single request). It should be understood that parallel processing within a request can be performed simultaneously with one or both of the above-mentioned methods for parallel processing between requests. However, parallel processing within a request may be computationally more efficient than parallel processing between requests. Parallel processing within a request may also be less complex. However, parallel processing within a request may require a new resource model. Therefore, a new resource model is provided that can be used for parallel processing within the request.

[0088] Embodiments can potentially improve several versions of scheduling. Embodiments can improve online one-to-one request scheduling, where the scheduling is performed with or without a time dimension (e.g., scheduling a request for 2 cores and 2GB of RAM for 10 minutes or other fixed time period). For example, by using the new resource model, the scheduling computation can be parallelized within a request (as described above) on the host and / or resource. Since the scheduling computation is parallelized only on the host and not on the request, this version can be considered a semi-vectorized version.

[0089] Embodiments can also improve online mini-batch scheduling, where multiple requests are batched and then all processed in parallel (within-request and across-request parallelism, as described above). Scheduling can be performed with or without a time dimension. Since the scheduling computation is parallelized on both the request and the host, the mini-batch scheduling case can be considered a fully vectorized version, in contrast to a semi-vectorized version where the scheduling computation is parallelized only on the host.

[0090] Embodiments can also improve offline lookahead scheduling (planning). In this case, a complete set of requests is known in advance. In this version, scheduling in the time dimension can be an important aspect of the scheduling problem. The offline lookahead scheduling can also be performed in the fully vectored version (or can be performed only in the semi-vectored version).

[0091] It should be noted that the line between online scheduling and offline planning may become blurred, as the time dimension may be considered in the online case (eg, a subset of work requests starts later - eg, reservations).

[0092] According to an embodiment, a new resource representation is used for scheduling, wherein resources are represented using tensors (eg, multi-dimensional arrays of uniform data types). Specifically, a resource hierarchy can be represented as a tensor with dimensions for each resource level.

[0093] Figure 5A The present invention provides a tensor-based resource model that encodes a resource hierarchy by converting a conceptual resource hierarchy into a tensor representation. According to an embodiment, the hierarchical relationship between different resource types can be encoded as different dimensions in the tensor. Each 'level' in the resource hierarchy corresponds to a 'dimension' in the resource tensor. Resources in a higher dimension can be considered to 'have' resources in a lower dimension. For example, the tensor (e.g., tensor 520) can have dimensions representing hosts, consumable resources, and time steps. Therefore, the dimensions in the one or more resource tensors sequentially represent the hierarchical relationship of the resources, so that resources in one dimension contain resources in subsequent dimensions. In this way, the resource tensor can represent the hierarchical relationship of the resources, so that resources in the second level of the hierarchical relationship contain resources in the first level of the hierarchical relationship. The size of the 'host' dimension 521 is 'H', so there are 'H' hosts (e.g., host 510) in the distributed computing system. The host dimension 521 is followed by the 'consumable resources' dimension. The consumable resources dimension 522 is of size 'R', so there are 'R' consumable resources in total, and each host 'has' all of these 'R' consumable resources. The array in the consumable resources dimension contains the values ​​of the consumable resources, for example, 'cpu' (e.g., cpu 511), 'memory' (e.g., mem 512), 'disk_capacity', 'disk_bandwidth', etc. The consumable resources dimension 522 is followed by the 'time steps' dimension 523. The 'time steps' dimension 523 is of size 'T', so there are 'T' time steps, and each consumable resource 'has' values ​​within 'T time steps. With respect to time steps, embodiments may treat the time dimension as another consumable resource in the resource hierarchy, for example, as Figure 5B It should be noted that Figure 5B The tree resource model in is used to illustrate the concept of treating time as a consumable resource or sub-resource of other consumable resources.

[0094] According to an embodiment, each array of the lowest dimension (i.e., the bottom level) contains actual values ​​encoding the amount of "consumable" resources. For example, resource_tensor[h][r][t] will contain a value representing the amount of resource 'r' in host 'h' at time 't'. Again referring to Figure 5AIn the resource structure shown in , if resource_tensor[0][1][2]=20, the resource model indicates that there are 20 'cpus' (e.g., resource[0]=cpu) on the host 510 (note: host 510=host[1]) at time step 2. The data type of the resource value in each tensor can be specified (e.g., Boolean, integer, Float16, Float32, etc.).

[0095] According to an embodiment, there may be a "resource tensor" (e.g. Figure 5A ), the resource tensor encodes the resources on all hosts, all consumable resources, and all time steps (e.g., of shape H x R x T). The resource tensor can indicate the usage of each resource ("resource usage tensor") or the remaining amount of each resource ("resource remaining tensor"). In this application, the term "resource tensor" can be used to define "resource usage tensor" and "resource remaining tensor" interchangeably, except when it is necessary to clearly distinguish whether the "resource tensor" is a resource usage tensor or a resource remaining tensor.

[0096] The resource usage tensor (a resource tensor indicating the usage of each resource) can be used in conjunction with a tensor indicating the capacity (e.g., the total amount) of each resource. The total capacity tensor can indicate the maximum amount of resource usage without exceeding the resource capacity. To ensure efficiency, the total capacity tensor can exclude any dimensions whose values ​​are immutable. Thus, for example, if the total capacity does not vary between different hosts or over time, the total capacity tensor can have a shape of 1 x R x 1. If the total capacity value varies across hosts and consumable resources, but not over time, the total capacity tensor can have a shape of H x R x 1. If the total capacity value varies across all hosts, all consumable resources, and all time steps, the total capacity tensor can have a shape of H x R x T. It should be noted that according to the broadcasting rules, whenever the size of the leading tensor dimension is 1, a dimension (level) can be excluded from the resource representation without changing the output of the scheduling operation.

[0097] According to an embodiment, the resource usage tensor may be updated each time a new work request is made (e.g., scheduled) for a resource, or each time a previously requested work is completed using its allocated resources. The resource usage tensor may also be updated over time (e.g., as time passes). In some embodiments, resource usage values ​​at past time steps may be used to predict resource usage at new future time steps.

[0098] In some embodiments, a single resource remaining tensor (ie, a 'remaining capacity' or 'available capacity' tensor) may be used instead of the resource usage tensor and the total capacity tensor. The resource remaining tensor may indicate the amount of resources remaining.

[0099] In some embodiments, one or more different types of tensors may be used for resource management. The one or more different types of tensors may be used instead of or in addition to one or more of the resource usage tensor, the total capacity tensor, and the resource remainder tensor.

[0100] According to an embodiment, there may be a request tensor for resource scheduling. The request tensor may encode the requested resources required for the work request to be scheduled. When processing a single request at a time, the request tensor may specify the consumable resources and the resources in the time dimension. In this case, the shape of the request tensor may be 1 x R x T (or equivalently, R x T), with the leading host dimension being of size 1. When processing 'I' requests in parallel at the same time, the request tensor may be a tensor of shape I x 1 x R x T because the extra dimension of 'I' requests is added. When processing the scheduling operation later, the host dimension remains equal to 1 to make efficient use of broadcasts. If the resource request is constant in the consumable resources over time, the request may be specified as an R x 1 tensor and the values ​​in the tensor may be broadcast as needed during the scheduling operation.

[0101] According to an embodiment, there may be a scheduler, which may also be referred to as a resource scheduler, for resource scheduling. The scheduler may include a set of computer-implemented instructions or methods for matching each request with a host in the resource tensor. As described above, the target dimension may be the host dimension, and the request may specify the resources required in the dimension below the target level.

[0102] Figure 6AAnother example of a resource hierarchy representation with different target dimensions is shown. In this resource model, assume that the goal is to match a request to one of the graphics processing units (e.g., GPU 610, GPU 620, etc.) in one of the hosts (e.g., Host 1, Host 2 600) in one of the racks (Rack 1, Rack 2, etc.) in one of the data centers. The target level is the processing unit (e.g., GPU 610, GPU 620). Therefore, each request can specify all the resources (or consumable resources) it requires below the processing unit level (GPU level). Here, the GPU can be equivalent to a certain amount of 'memory' and 'cores' over a specific time period (e.g., T time steps). When scheduling a single request, the shape of the request tensor can be R x T. In this case, the value of R is three (3) because there are three types of consumable resources: 'GPU memory' 611, 'GPU cores' 612, and 'GPU disk' 613.

[0103] According to an embodiment, the resource tensor may have a shape of D x K x H x G x R x T (D data centers x H hosts x G GPUs x R consumable resources x T time steps). The resources may be encoded as resources[d][k][h][g][r][t], and the requests may be encoded as requests[r][t]. In various embodiments, using the resource tensor and the request tensor, the scheduling process may be simply implemented, even using a single line of computer source code, to determine the appropriate data center, rack, host, or processing unit (e.g., GPU 610, GPU 620) for each request.

[0104] Figure 6B 6. The tensor target dimension (e.g., target dimension 650) in the resource tensor provided by an embodiment of the present invention is shown. According to an embodiment, one dimension in the resource tensor can be used as the target dimension explicitly or implicitly. Representing a dimension as the target dimension indicates how resources in all dimensions should be used. In various embodiments, a request can be considered to require 'one' resource in the resources at the target level, and implicitly require one resource in the resources at each level above the target level. For example, referring to Figure 6B , a request may require 'one' data center, 'one' rack, 'one' host, and 'one' GPU to satisfy the resource requirements for completing the request. In addition, in various embodiments, a request may be considered to require 'all' resources at all levels below the target dimension level. For example, referring to Figure 6B , a request may require "all" consumable resources (e.g., all CPU, all memory, etc.) at all time steps in the tensor.

[0105] In some embodiments, for some scheduling operations, the resource tensor can be reshaped into a J x L matrix (note: a 'matrix' is a 2-dimensional tensor). All dimensions of the resource tensor above the target level are merged into a single dimension. This single dimension is the 'J' dimension, i.e., the first dimension in the J x L matrix. Similarly, all dimensions of the resource tensor at and below the target level are merged into another single dimension. This single dimension is the 'L' dimension, i.e., the second dimension in the J x L matrix. Therefore, in Figure 6B In the case of the resource tensor model shown in , the size of the J dimension may be D*K*H*G, and the size of the L dimension may be R*T. In some embodiments, the request tensor may be reshaped into a single-dimensional array of size L.

[0106] Compared to the resource model hierarchy used in prior work, resource tensors have some constraints. Like the prior model, a single resource tensor can encode a hierarchy or "tree" of resources. However, as a data structure based on multidimensional arrays, there is less flexibility in which resources can be encoded at different levels in the hierarchy. Specifically, in a single resource tensor, all digital consumable resources must be at the leaf level (e.g., actual CPU, memory, disk values, etc.). Furthermore, as described above, all digital consumable resources must have the same data type, e.g., a uniform data type. Furthermore, all non-leaf resources are "composite" resources (e.g., resources that contain other resources). Finally, all resources at a given level in the tree (e.g., resources of a given dimension in the tensor) must have the same number of sub-resources. While more constrained than the prior model, resource tensors have the advantage of allowing faster execution of these models through parallel vectorized operations, as further described above. Furthermore, to increase flexibility when these constraints become too restrictive, those skilled in the art will understand how to adjust resource tensors to increase flexibility. Some examples of these adjustments are discussed further below.

[0107] In some embodiments, there can be multiple resource tensors (and multiple corresponding request tensors and capacity tensors) to manage several different situations. In some embodiments, there can be multiple resource tensors when consumable resource values ​​can logically exist at different levels of the resource hierarchy (for example, as described above, violating one of the constraints of a single resource tensor). For example, in addition to the resource tensor that encodes the specific resources available in each host, there can also be a separate resource tensor for resources shared between hosts. An example of a shared resource is a software license. A software license is a resource that can only be used by a certain number of requests at a time. The software license can be used by requests on all hosts. In other words, the software license is shared between requests from all hosts, and the software license can be considered a shared resource. Because this shared resource does not belong to a specific host, unlike other resources, a separate resource tensor may be required. Therefore, there can be a separate resource tensor for shared resources to manage resource scheduling. In addition, separate corresponding request tensors and capacity tensors may also be required.

[0108] In some embodiments, there may be a single tensor encoding the resource consumption of all tenants, as it may be desirable to track each tenant's resource consumption over time to implement tenant sharing policies.

[0109] In some embodiments, there may be separate resource tensors for past resource usage (e.g., observed usage) and future resource usage (e.g., predicted usage). The tensor for past resource usage can be used to predict future resource usage, and the tensor for future resource usage can be used to make resource placement decisions to ensure sufficient capacity for future requested workloads.

[0110] If some resources in the resource hierarchy logically have different numbers of sub-resources (e.g., one host has one GPU and another host has two GPUs), this would also appear to violate one of the constraints of a single resource tensor, as described above. However, in some embodiments, a single resource tensor may still be used if a 'zero (0)' capacity is encoded for the consumable resources that are missing in the capacity tensor (e.g., the GPUs that are not present). For example, when some hosts have 'two' GPUs and some hosts have 'one' GPU, a single resource tensor may be used that logically encodes each host as having two GPUs. Then, for the GPUs that are not present in the capacity tensor (e.g., the GPUs that are not present on a host that actually only has one GPU), a 'zero (0)' capacity may be encoded. However, in some embodiments, a separate resource tensor may be used for each of the hosts with one GPU and the hosts with two GPUs, rather than encoding a 'zero (0)' capacity.

[0111] The single tensor constraint may be further violated if all consumable resources in the hierarchy are not logically of the same data type (e.g., some bottom-level resource values ​​are numbers, other bottom-level resource values ​​are Boolean (True / False) values, etc.) In this case, in some embodiments, a separate tensor may be used for each resource data type.

[0112] If not all resources logically have a time dimension (e.g., if some resources are 'static' or simply 'tag-based' resources), further violations of the single tensor constraint may occur. In one embodiment, the time dimension may be used for all resources, but the static resources may be constant over time. In another embodiment, separate resource tensors may be used, e.g., one resource tensor with a time dimension and one without. In such an embodiment, the request may be checked separately for the time-based tensor and the tag-based tensor.

[0113] Figure 7 A logical circular buffer 710 and a circular buffer implementation 720 for the time dimension in a resource tensor provided by an embodiment are shown. According to an embodiment, the time dimension in the resource tensor can be encoded using the circular buffer 710. In a tensor representation, the time dimension can be uniquely viewed as a time step with a certain increment or interval (e.g., every minute). Therefore, over time, all resource values ​​may need to be updated. For example, a resource value that was previously two time steps away from a certain point in time is now only one time step away from the same point in time. Therefore, the resource values ​​in the tensor need to be updated. However, it is not necessary to update all resource values ​​in the tensor at every time step because such updates are computationally expensive.

[0114] To solve this problem, a logical circular buffer 710 can be implemented for the time dimension in the resource tensor, for example, as Figure 7 , a circular buffer implementation 720 is shown in FIG. Specifically, a start pointer (e.g., start pointer 701) and an end pointer (e.g., end pointer 702) can be implemented and kept dynamically pointing to the 'start' and 'end' of the time dimension value (or a sub-array of time dimension values), respectively. At each update, a new 'end' value for each host and resource can be inserted, extrapolated from past values, or explicitly predicted. Over time, the values ​​of the start pointer and the end pointer can be rotated around the actual end of the time dimension value as needed. For example, referring to Figure 7, the start pointer 701 may point to position 1 (e.g., array index 0) and the end pointer 702 may point to position 10 (e.g., array index 9). As time passes, the values ​​of the positions of the start pointer 701 and the end pointer 702 may increment. When the pointer is at the end of the time dimension value, the pointer will continue to shift (e.g., wrap around) to the beginning of the time dimension value over time. For example, after advancing 4 times (e.g., after 4 time steps), the start pointer 701 moves from position 1 to position 5 and the end pointer 702 moves from position 10 to position 14, which is actually position 4 of the linear buffer in the memory.

[0115] According to an embodiment, a circular buffer in the time dimension of a resource tensor can update values ​​in the tensor more quickly over time. In some embodiments, the resource tensor can be updated T times faster, where the time dimension is of size T. This speedup can be achieved by a factor of T because the update only needs to be performed at one of the T time steps, rather than at all T time steps.

[0116] According to an embodiment, tensors can be used to represent resources to efficiently perform scheduling operations. Scheduling operations may include: (i) logically making a request to a host; (ii) determining whether the request matches the host; (iii) finding the best host to which to make the request; (iv) determining the number of times the request can be matched with one or more hosts; and (iv) resolving conflicts that arise during the scheduling operation. In various embodiments, scheduling operations may be performed using scientific (software) frameworks and libraries (e.g., NumPy and PyTorch).

[0117] According to an embodiment, a tensor model can be used to more efficiently perform scheduling operations for making requests (e.g., making requests to hosts). For example, when there is a resource usage tensor of shape H x R x T and a request tensor of shape R x T, making an R x T request to the i-th 'host' can be implemented as follows:

[0118] resources[i]+=request

[0119] or

[0120] resources[i]=resources[i]+request

[0121] Because both resource[i] and request have an R x T shape, an addition operation (e.g., '+') can modify the value in the R x T subarray at position 'i' of the resource tensor. Scientific (software) frameworks or libraries (e.g., NumPy and PyTorch) can automatically compile source code to efficiently perform dispatch operations (e.g., making requests to the host). In some embodiments, source code can be compiled more efficiently on specialized hardware such as GPUs and Tensor Processing Units (TPUs). Consequently, requests to the host can be made more efficiently on specialized hardware such as GPUs and TPUs.

[0122] Once the requested work (e.g., request) is completed, the scheduling operation may proceed to remove the completed work from the 'i'th host and free up its resources for other work requests. Removing the completed request from the 'i'th host may be accomplished as follows:

[0123] resources[i]-=request

[0124] or

[0125] resources[i]=resources[i]-request

[0126] According to an embodiment, a tensor model can be used to more efficiently determine whether a request is compatible with one or more hosts during scheduling. In other words, the tensor model allows the resource scheduler to efficiently find a subset of resources (here, at the host level) that have sufficient or adequate resources to execute the request (here, to determine the hosts that can accept the request). Figure 8 An example of performing such a determination using one request and two hosts is shown. Figure 8 shows the case without the time dimension. Figure 8 Not shown, but when the tensor has a time dimension, determining whether the request matches the host can be repeated across all values ​​of the time dimension.

[0127] refer to Figure 8 , the amount of resources requested by request 810, as well as the current resource usage and total capacity, can be provided to each of hosts 821 and 822. The purpose of this operation (e.g., determining whether the request matches the host) is to find a host that can accept the request. Whether the request matches the host can be determined by finding whether the sum of the amount of resources requested by the request and the current resource usage in the host in any consumable resource dimension (e.g., CPU, memory, etc.) does not exceed the total capacity of the host. Figure 8In the illustrated case, request 810 is not matched with host 821 because the sum of the amount of resources requested by request 810 and the current resource usage of host 821 exceeds the total capacity of host 821 in the memory dimension. On the other hand, request 810 is matched with host 822 because the sum of the amount of resources requested by request 810 and the current resource usage of host 822 does not exceed the total capacity of host 821 in any resource dimension.

[0128] According to an embodiment, a vectorized operation may be used to determine whether a request matches a host. In various embodiments, a single vectorized operation may be sufficient to determine whether multiple requests match multiple hosts across all time steps. For example, when the resource tensor (e.g., resources in the code below) has a shape of H x R x T, the request tensor (e.g., requests in the code below) has a shape of I x 1 x R x T (i.e., there are 'I' requests in total), and the total capacity tensor (e.g., resource_capacity in the code below) has a shape of H x R x 1 (i.e., capacity is constant over time), determining whether each of the multiple requests matches each of the multiple hosts can be simply implemented using a single command in NumPy, as shown below:

[0129] server_fits=np.all(request+resources<=resource_capacity,axis=(2,3))

[0130] The output will be a boolean tensor of shape I x H. If the value of the output tensor at index (i,h) is 'True', then the request 'i' matches the host 'h'. If the value of the output tensor at index (i,h) is 'False', then the request 'i' does not match the host 'h'. An example of an output tensor of shape I x H can be shown below. Each Fits i,h Can hold Boolean values ​​(e.g., True or False).

[0131]

[0132] The above code illustrates the concept of 'broadcasting'. As described above, the three tensors in the arithmetic operation 'request+resources<=capacity' all have different dimensions (I x 1 x R x T, H x R x T, and H x R x 1, respectively). To perform the computation, in some embodiments, the source code implicitly broadcasts the request across all host dimensions, broadcasts the host across all request dimensions, and broadcasts the capacity across all requests and all time steps. For example, when executing a scheduling operation in which the request tensor and the resource tensor are parameters of the operation, the resource scheduler or other computing device can efficiently and automatically (i) align the dimensions between the request and resource tensors, (ii) broadcast (e.g., logically copy) the request and resource tensor values ​​so that the request tensor implicitly has the same dimensions as the resource tensor without actually copying the data in memory, and (iii) perform the required operation (e.g., adding the request value to the resource value). According to an embodiment, broadcasting can bring certain flexibility and efficiency to tensor representation. For example, for fixed requests over time (i.e., static, non-dynamic requests), a tensor of dimension R x 1 can be used (instead of a tensor of dimension R x T), and the consumable resource values ​​can be broadcast across all time steps during the scheduling operation. For another example, considering the aforementioned complex resource tensor of shape D x K x H x G x R x T, the request tensor may not need to completely match the resource tensor in terms of dimensions in order to perform the scheduling operation, where the request tensor and the resource tensor are parameters of the operation. For example, if the request tensor has a shape of R x 1, the consumable resource values ​​of the request tensor can be automatically broadcast across all complex resource dimensions (D, K, H, and G) and across all time steps during the scheduling operation. In fact, there may be no need to copy data, that is, there is no need to actually instantiate the request tensor of dimensions D x K x H x G x R x T in memory to proceed with the operation.

[0133] Referring to the source code above, the 'all' operation (e.g., np.all) can be used because the request requires all resources below the target level / dimension. The 'all' operation returns True only if 'all' elements along a given dimension or set of dimensions (one or more 'axis' in NumPy terminology) in the input tensor evaluate to True. In this case, the input tensor is the output of an addition and comparison operation (e.g., request + resources <= resource_capacity), and the axes to be used are the time dimension and the consumable resource dimension. Referring to the source code above, the dimensions to check for truth are represented as 'axis = (2, 3)'.

[0134] The above source code can be considered a 'fully vectorized' version of tensor-based scheduling, as all requests and all hosts are processed simultaneously. The same functionality can be implemented using a 'for' loop, where a single request is processed at each iteration (e.g., determining whether a request matches each host, but only for a single request at each iteration). The for loop-based approach can be considered a 'semi-vectorized' version.

[0135] According to an embodiment, a tensor model may be used when determining the best host to which to make a request. When determining the best host to which to make a request, it may be necessary to avoid a situation where the consumable resources on the host are exhausted in one dimension but not in other dimensions. For example, when most of the memory is still available, it may not be necessary to exhaust all the CPUs on the host. In this case, although there is a large amount of available memory capacity on the host, this capacity will be wasted because the CPUs on the host are fully utilized and new tasks cannot be placed on the host. To avoid such inefficient resource usage, the resource scheduler may make requests in such a way that after the request is made, the total resource usage in the host is similar to the total capacity of the host in terms of vector similarity.

[0136] According to an embodiment, by calculating 'cosine similarity', the vector similarity between the total resource usage in the host and the total capacity of the host can be found. Cosine similarity can refer to the cosine value of the angle between two vectors in a multidimensional space, where one vector represents the total resource usage in the host after the request is made and the other vector represents the total capacity of the host. If the two vectors are close to each other (i.e., the resource usage in the host and the total capacity of the host after the request is made are similar to each other), the angle between the two vectors is small and the cosine value of the angle will be close to 1. On the other hand, if the two vectors are far apart from each other (i.e., when the request is made, the resource usage of one dimension is much higher when the total resource capacity of the other dimension is considered than when the total resource capacity of the other dimension is considered), the angle between the two vectors is large and the cosine value of the angle will be close to 0.

[0137] Figure 9An example of using cosine similarity to determine the best host for which a request is made is provided in an embodiment. The purpose of this operation is to find the best host for which a request is made. According to an embodiment, the best host can be determined by calculating the cosine similarity between the total resource usage of each host (e.g., total resource usage = number of requests + current resource usage of each host) and the total resource capacity of each host. The best host may be the host whose total capacity is most similar to the sum of the requested resource amount and the resource amount used by the host.

[0138] refer to Figure 9 , can provide the amount of resources requested by request 910, and can also provide the current resource usage and total capacity for each of hosts 921 and 922. Vector 931 represents the vector of total resource usage of host 921 (e.g., the sum of the amount of resources requested by request 910 and the current resource usage of host 921), and vector 932 represents the vector of total resource capacity of host 921. Similarly, vector 941 represents the vector of total resource usage of host 922 (e.g., the sum of the amount of resources requested by request 910 and the current resource usage of host 922), and vector 942 represents the vector of total resource capacity of host 922. To find the best host for which to make a request (in this case, host 921 or host 922), the cosine similarity between the total resource usage (e.g., the sum of the number of requests and the host usage) and the total resource capacity of all hosts (e.g., hosts 921 and 922) is calculated. In this case, the cosine similarity of host 921 can be obtained by calculating the cosine of the angle between vectors 931 and 932. Similarly, the cosine similarity of host 922 can be obtained by calculating the cosine of the angle between vectors 941 and 942. Because the angle between vectors 941 and 942 is smaller than the angle between vectors 931 and 932, the cosine similarity of host 922 is greater than the cosine similarity of host 921. Therefore, making request 910 to host 921 will better balance resource usage in each dimension in proportion to its capacity.

[0139] If there are I requests and J hosts, it may be necessary to generate an I x J matrix (i.e., an I x J floating-point tensor) for cosine similarity when each request is made to each host. Cosine similarity can be calculated using the following "semi-vectorized" (i.e., one request at a time) source code in NumPy:

[0140] for each request:

[0141] proposed=request+resources

[0142] proposed_norm=torch.norm(proposed,dim=1)

[0143] cosine_dot=torch.einsum("ijk,ijm->i", proposed, resources_capac)

[0144] cosines[i]=cosine_dot / (proposed_norm*resources_capac_norm)

[0145] An example of the output tensor of the cosine similarity calculated for all request (plus host usage) vectors and all host capacity vectors may be as follows. The output tensor may have a shape of I x J. Each cosθ i,j (eg, i=1...I, j=1...J) may hold a floating point value (eg, 0.866025404) indicating the cosine of the angle between the request+usage and host capacity vectors.

[0146]

[0147] Referring to the above NumPy source code, the last line of the code (e.g., cosine[i] = cosine_dot / (proposed_norm * resources_capac_norm)) is equivalent to the following mathematical expression, which indicates that the cosine value can be obtained by calculating the quotient of the dot product value of the vector and the norm of each vector.

[0148]

[0149] Referring to the NumPy source code above, the 'Einstein summation' operation (e.g., torch.einsum) is used to calculate the dot product across all consumable resources and time steps. This dot product then becomes the numerator in the expression in the last line of code, as described above.

[0150] According to an embodiment, a "semi-vectorized" version of the source code shown above can be parallelized by dividing the request (and therefore the corresponding cosine similarity calculations performed at each iteration) among multiple CPUs and GPUs, where each CPU or GPU can complete part of the iteration work in the loop. This may increase the speed of calculation of the entire 1 x J cosine similarity output matrix.

[0151] According to an embodiment, using cosine similarity is one way to perform a 'best match' operation that can be employed in resource scheduling (e.g., determining the best host to which to make a request). Alternatively, in other embodiments, it may be desirable to 'spread out' the requests as much as possible so that resource usage remains as even as possible. Regarding the calculation of a 'best match' using a 'spread out' criterion, it can also be easily vectorized. For example, the norm of the total (i.e., aggregate) resource usage in each host can be calculated to see how much resources each host uses. There are several ways to efficiently recalculate the aggregate resource usage in each host over time, such as simply adding or subtracting values ​​that are removed or deleted over time. Once the norm of the resource usage for each host is calculated, a request can be made to the least used host (i.e., the host that uses the least amount of resources). Here, the least used host can be the host with the smallest norm.

[0152] In addition to the cosine similarity and "least used" score of hosts used to determine the "best match" score, those skilled in the art will readily appreciate that, depending on the embodiment, there are other possible methods for performing the "best match" calculation using resource and request tensors. In this application, the "score" can be viewed as a measure of the "suitability" of a given resource as a host for a given work request.

[0153] According to some embodiments, the resource scheduler may pre-compute (i.e., "predetermine") cosine similarity scores between all possible suggested placements (i.e., all possible total resource usage, which is the sum of all possible requests and host usage) and the total resource capacity on each host. Each of the pre-computed cosine similarity scores may be stored in a score function tensor. In essence, the score function tensor may provide a value for each possible request placement, including suggested placements that do not currently exist in the cluster. For example, in some embodiments, the scoring function may be pre-computed for all possible resource combinations (e.g., all possible CPU and memory combinations: <1 core, 1GB>, <1 core, 2GB>, <2 cores, 1GB>, <2 cores, 2GB>, etc.).

[0154] Once the pre-computed cosine similarity scores are stored in the score tensor, the score tensor can be indexed (in a vectorized manner) using the corresponding request placements to efficiently generate cosine similarity scores for all suggested request placements.

[0155] In some embodiments, the pre-computed scores may be accessed to find the 'best match' to a host, as shown using the following NumPy source code:

[0156] for each request:

[0157] proposed=request+resources

[0158] proposed_indices=np.split(proposed, proposed.shape[-1], axis=-1)

[0159] proposed_scores=scoring_tensor[proposed_indices]

[0160] placement_spot=proposed_scores.argmax()

[0161] Referring to the above source code, the proposed request placement (i.e., "suggestion") can be obtained in the same manner as the semi-vectorized NumPy source code that calculates the cosine similarity of each request. The obtained proposed request placement (i.e., "suggestion") will be converted into a multidimensional index. The multidimensional index (i.e., proposed_indices) can be obtained by splitting the proposed request placement along the final resource dimension. In various embodiments, as shown above, the multidimensional index can be simply obtained using the 'split' function provided by NumPy, while other alternative methods (e.g., obtaining a 'view' of resources grouped by the consumable resource dimension instead of copying them into a new array) can also be used to obtain the multidimensional index. The multidimensional index (e.g., proposed_indices) can be used to retrieve the corresponding cosine similarity score for each proposed request placement. The resource scheduler can make a request to the host with the highest proposed request placement score (e.g., if scoring_tensor encodes cosine similarity, this will be the host with the highest cosine similarity score). The above steps can be repeated for each request.

[0162] There are several reasons for using a pre-computed scoring function to determine the best host for a request. The primary reason is that it's more efficient to compute the score only once and cache it for future retrieval, rather than recalculating it multiple times during resource scheduler operation. Another reason is flexibility; for example, if you need to switch from cosine similarity to a 'least used' scoring function, you don't need to make corresponding changes to the scheduling source code; you can simply load a pre-computed score tensor using the 'least used' scoring function into the resource scheduler as a replacement for the cosine similarity function, and the scoring function can be changed.

[0163] It should be noted that all scores can be pre-computed only when there are a finite number of possible scores; and this limitation only arises when there are a finite number of possible combinations of resource values ​​in the distributed computing system. In various embodiments, the requested work can be drawn from a finite number of "categories" or discrete options for CPU and memory requirements. Therefore, a finite number of discrete requested placements can be proposed (e.g., the sum of existing resource usage and requested resource amounts). Since at most a score for each possible proposed placement is required, a finite number of possible scores need to be pre-computed before scheduling begins. On the other hand, in some embodiments, only a subset of the scores are pre-computed.

[0164] According to an embodiment, a tensor model can be used to determine the number of times a work request can be matched to one or more hosts. For example, given current resource availability, a system operator may wish to determine the number of work requests that can be successfully submitted with a particular specified resource requirement. For example, the system operator may wish to determine whether all 100 VMs can be placed into the distributed computing system, where each VM requires 1 CPU and 2GB of memory. The tensor model enables this number to be calculated in a computationally efficient manner using vectorized operations. The tensor model also enables this number to be calculated in parallel for multiple different work requests in an efficient manner using vectorized operations.

[0165] According to an embodiment, a tensor model may be used to resolve conflicts generated during scheduling operations. During scheduling, if only one request is made at a time (i.e., the semi-vectorized version described above), there may be no conflicts. However, if multiple requests are made in parallel or requests are made in batches (i.e., using the fully vectorized operation version), parallel scheduling may cause conflicts. For example, when two requests are best matched with the same host, a conflict may only arise if one of the two requests matches the host. Therefore, after finding out whether the request matches each host individually, or after determining the single best host to which each request is made, an algorithm for resolving conflicts may be required. Therefore, in various embodiments, a conflict resolution mechanism may be required.

[0166] There are several possible approaches to resolving conflicts that arise during scheduling operations. In some embodiments, when a conflict arises between one or more requests, the resource scheduler can arbitrarily select a winning request and reschedule the losing request at a later time. In some embodiments, the resource scheduler can use a special conflict resolution algorithm that can be processed in parallel. One such conflict resolution algorithm that can be processed in parallel is the parallel asymmetric auction algorithm proposed by Bertsekas and Castanon (Bertsekas, Dimitri P., and David A. Castanon, "Parallel synchronous and asynchronous implementations of the auction algorithm," Parallel Computing, 17.6-7 (1991): pp. 707-732). The proposed algorithm solves the allocation problem by finding the optimal allocation of requests to hosts. The proposed algorithm works by iteratively 'bidding' requests in parallel to determine the degree to which each host should make the request, and iteratively having each host select the request with the highest bid. These iterations are repeated until a final solution is found.

[0167] According to an embodiment, there are potential advantages in using tensors when implementing scheduling operations. One potential advantage is that scheduling operations can be implemented through very concise, easy-to-read and clear source code. Scheduling operations can usually be implemented through a single line of source code rather than complex statements in nested for loops. Another potential advantage of using tensors is that scientific (software) frameworks and libraries (e.g., NumPy and PyTorch) can have built-in functions for converting source code into vectorized instructions that can be efficiently run on a processing unit (e.g., a CPU). The converted instructions can run more efficiently on specialized hardware such as GPUs and TPUs.

[0168] According to an embodiment, scheduling using tensors can utilize dedicated hardware to perform scheduling operations. For example, modern GPUs, TPUs, and other dedicated AI chips can be optimized for tensor processing operations. By encoding scheduling operations using tensors, dedicated hardware can be (automatically) utilized to parallelize scheduling operations and run scheduling operation algorithms more efficiently. The scientific (software) frameworks and libraries can also use or allow the use of machine instructions and hardware for parallel processing. For example, if the machine supports SIMD instructions, NumPy compiles Python code into underlying C code, and further compiles the underlying C code to utilize the instructions (for example, AVX). These SIMD instructions run significantly faster than pure Python code. Compared to NumPy on Intel AVX, PyTorch can allocate and update tensors on GPU memory and (automatically) leverage GPU parallel processing capabilities for further speedups. Tensor resource models can also be deployed on dedicated AI hardware or tensor processing hardware (e.g., TPUs) or other AI accelerator chips.

[0169] According to an embodiment, a tensor (i.e., a multidimensional (hierarchical) array with a uniform data type) can be used to model resources for scheduling in a distributed computing system. The tensor can represent the hierarchical relationship between different resources. In particular, a tensor can be used when encoding the time dimension as a circular buffer. In some embodiments, one or more separate tensors for past and future times can be used (e.g., when using values ​​from past times to predict future values).

[0170] According to an embodiment, tensors can be used to encode all resource usage, resource capacity, remaining or available resources, requested resources, and resources used by each tenant in the system. When tensors are used to represent resources, standard scheduling operations can be implemented using tensor operations (e.g., tensor calculations using broadcasts). Some examples of scheduling operations that can be implemented using tensors can be: making a request to a host; calculating where the request can be matched; determining the best host to which the request is made; and resolving conflicts caused by parallel scheduling operations. These scheduling operations and corresponding tensors can be placed and executed on dedicated hardware such as dedicated vectorized CPUs, GPUs, TPUs, or other dedicated processing units. In various embodiments, the above-mentioned scheduling operations and tensor resource modeling can be performed in a distributed computing system (e.g., in which the resource scheduler can perform distributed resource management).

[0171] Figure 10 Schematic diagrams of electronic devices 1100 provided by various embodiments of the present invention are shown. The electronic device 1100 can perform any or all of the operations of the methods and features described herein, either explicitly or implicitly. For example, a UE such as a mobile phone, a smartphone, a computer with network capabilities, a network node, or a machine type communication (MTC) device can be configured as the electronic device 1100. Furthermore, a computing device including a resource scheduler and a resource server in a data center described herein can be configured as the electronic device 1100.

[0172] As shown, the device includes a processor 1110 (e.g., a central processing unit (CPU) or a dedicated processing unit (e.g., a graphics processing unit (GPU)) or other such processor unit), a memory 1120, a non-transitory mass storage device 1130, an I / O interface 1140, a network interface 1150, and a transceiver 1160, all of which are communicatively coupled via a bidirectional bus 1170. According to certain embodiments, any or all of the components described, or only a subset of the components, may be utilized. Furthermore, the device 1100 may include multiple instances of certain components, such as multiple processors (including a combination of CPUs and dedicated processors), memories, or transceivers. In addition, components of the hardware device may be directly coupled to other components without the bidirectional bus. Additionally or alternatively, in addition to the processor and memory, other electronic devices such as integrated circuits may be used to perform the required logical operations.

[0173] The memory 1120 may include any type of non-transitory memory, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), read-only memory (ROM), or any combination thereof. The mass storage element 1130 may include any type of non-transitory storage device, such as a solid-state disk, a hard disk drive, a magnetic disk drive, an optical disk drive, a USB drive, or any computer program product for storing data and machine-executable program code. According to certain embodiments, the memory 1120 or the mass storage device 1130 may record thereon statements and instructions executable by the processor 1110 for performing any of the above-described method operations.

[0174] It should be understood that although specific embodiments of the technology are described herein for illustrative purposes, various modifications may be made without departing from the scope of the technology. Accordingly, the specification and drawings are to be regarded only as illustrative of the invention as defined by the appended claims, and are intended to cover any and all modifications, variations, combinations or equivalents within the scope of the invention. Specifically, within the scope of the technology, there is provided a computer program product or program element, or a program storage device or memory device (e.g., a magnetic or optical wire, a magnetic tape or disk, etc.) for storing machine-readable signals, controlling the operation of a computer according to the method of the technology, and / or constructing some or all of its components according to the system of the technology.

[0175] The actions associated with the methods described herein may be implemented as coded instructions in a single computer program product. In other words, the computer program product is a computer-readable medium that, when loaded into a memory and executed on a microprocessor of the wireless communication device, records software code thereon to perform the methods described.

[0176] The actions associated with the methods described herein can be implemented as coded instructions in a variety of computer program products. For example, the first portion of the method can be performed by one computing device, and the second portion of the method can be performed by another computing device, a server, or the like. In this case, each computer program product is a computer-readable medium that, when loaded into a memory and executed on a microprocessor of a computing device, records software code to perform the appropriate portion of the method.

[0177] Furthermore, each operation of the method can be performed on any computing device such as a personal computer, a server, or a personal digital assistant (PDA), and according to one or more program units, modules, or objects, or a portion of one or more program units, modules, or objects, generated in any programming language such as C++ or Java. In addition, each operation, or a file or object that implements each operation, can be performed by dedicated hardware or a circuit module designed for this purpose.

[0178] Through the description of the above embodiments, the present invention can be implemented only by using hardware, or by using software and a necessary general-purpose hardware platform. Based on this understanding, the technical solution of the present invention can be embodied in the form of a software product. The software product can be stored in a non-volatile or non-transient storage medium, which can be a compact disk read-only memory (CD-ROM), a USB flash drive, or a removable hard disk. The software product includes a plurality of instructions that enable a computer device (a personal computer, a server, or a network device) to execute the method provided in the embodiment of the present invention. For example, such execution can correspond to a simulation of the logical operations described herein. The software product may additionally or optionally include a plurality of instructions that enable a computer device to execute operations for configuring or programming the digital logic device according to the embodiment of the present invention.

[0179] Although the present invention has been described with reference to specific features and embodiments thereof, it is apparent that various modifications and combinations of the present invention may be made without departing from the present invention. Therefore, the specification and drawings are to be regarded only as illustrative of the present invention as defined by the appended claims, and are intended to cover any and all modifications, variations, combinations or equivalents within the scope of the present invention.

Claims

1. A method for resource management and scheduling in a distributed computing system, characterized in that: The method comprises: Using one or more resource tensors to represent the availability of resources or the usage of resources in the distributed computing system; the dimensions of each resource tensor include a resource level dimension and a time dimension; receiving one or more resource requests, each resource request specifying resources in the distributed computing system required to satisfy a work request within a specified time; Representing the resource request as one or more request tensors; the request tensors correspond to the dimensions of the resource tensor, including dimensions corresponding to the resource level dimension and the time dimension; Resources in the distributed computing system are scheduled according to the one or more resource tensors and the one or more request tensors.

2. The method according to claim 1, characterized in that Also includes: Representing the total amount of the resources in the distributed computing system using one or more capacity tensors; Wherein, resources in the distributed computing system are further scheduled according to the one or more capacity tensors.

3. The method according to claim 2, characterized in that Scheduling also includes, at least in part, operating on the one or more resource tensors, the one or more request tensors, or the one or more capacity tensors using a vectorized tensor broadcast operation.

4. The method according to any one of claims 1 to 3, characterized in that Also includes: During scheduling, based on values ​​in the one or more request tensors, the one or more resource tensors are updated in response to a condition, wherein the condition is one of: making a request for said work on a subset of one or more available resources; completing said work request requiring said one or more resources; or The one or more resources requested by the work request are released.

5. The method according to any one of claims 4, characterized in that Also includes: A subset of the one or more resource tensors is determined, the subset of the one or more resource tensors having sufficient capacity for the work request.

6. The method according to any one of claims 5, characterized in that Also includes: Determining a subset of the one or more resource tensors that best matches the requested resource, wherein the best match is determined based at least on calculating a fitness score for each subset of the one or more resource tensors when making the work request for the requested resource.

7. The method according to claim 6, characterized in that The fitness score for each of the subsets of the one or more resource tensors is determined based on a similarity between total resource usage after the request for the resource is made and a total capacity of the resource.

8. The method according to any one of claims 7, characterized in that The fitness score is predetermined.

9. The method according to any one of claims 8, characterized in that Also includes: A total number of times work requests can be made for the requested subset of resources is determined.

10. The method according to any one of claims 9, characterized in that Scheduling also includes: Resolving conflicts arising from requests for work on a subset of the one or more resource tensors.

11. The method according to any one of claims 10, characterized in that The resources have a hierarchical relationship, wherein resources at the second level of the hierarchical relationship include resources at the first level of the hierarchical relationship, and the order of dimensions in the one or more resource tensors represents the hierarchical relationship, so that resources in one dimension include resources in subsequent dimensions.

12. The method according to any one of claims 11, characterized in that The resources in the dimensions of the one or more resource tensors represent values ​​at different time steps defining a time dimension.

13. The method according to claim 12, characterized in that The time steps of the time dimension are stored in a circular buffer.

14. The method according to any one of claims 13, characterized in that The one or more resource tensors include one or more of: one or more resource tensors for past resource usage and one or more resource tensors for future resource usage.

15. A device for resource management and scheduling in a distributed computing system, characterized in that: The device comprises: processor; a memory storing machine-executable instructions that, when executed by the processor, configure the apparatus to: Using one or more resource tensors to represent the availability of resources or the usage of resources in the distributed computing system; the dimensions of each resource tensor include a resource level dimension and a time dimension; receiving one or more resource requests, each resource request specifying resources in the distributed computing system required to satisfy a work request within a specified time; Representing the resource request as one or more request tensors; the request tensors correspond to the dimensions of the resource tensor, including dimensions corresponding to the resource level dimension and the time dimension; Resources in the distributed computing system are scheduled according to the one or more resource tensors and the one or more request tensors.

16. The device according to claim 15, characterized in that When executed by the processor, the instructions further configure the apparatus to: Representing the total amount of the resources in the distributed computing system using one or more capacity tensors; Scheduling is performed using the one or more resource tensors, the one or more request tensors, and the one or more capacity tensors.

17. The device according to any one of claims 15 to 16, characterized in that When executed by the processor, the instructions further configure the apparatus to: During scheduling, based on values ​​in the one or more request tensors, the one or more resource tensors are updated in response to a condition, wherein the condition is one of: making a request for said work on a subset of one or more available resources; completing said work request requiring said one or more resources; or The one or more resources requested by the work request are released.

18. The device according to any one of claims 17, characterized in that The instructions, when executed by the processor, further configure the apparatus to determine a subset of the one or more resource tensors that best matches the requested resource, wherein, when making the work request for the requested resource, the best match is determined based at least on calculating a fitness score for each subset of the one or more resource tensors.

19. The device according to claim 18, characterized in that The fitness score for each of the subsets of the one or more resource tensors is determined based on a similarity between total resource usage after the request for the resource is made and a total capacity of the resource.

20. A network node for resource management and scheduling in a distributed computing system, characterized in that: The network node includes: a network interface for receiving data from and transmitting data to components connected to the distributed computing network; processor; a non-transitory memory for storing instructions that, when executed by the processor, cause the network function to be configured to: Using one or more resource tensors to represent the availability of resources or the usage of resources in the distributed computing system; the dimensions of each resource tensor include a resource level dimension and a time dimension; receiving one or more resource requests, each resource request specifying resources in the distributed computing system required to satisfy a work request within a specified time; Representing the resource request as one or more request tensors; the request tensors correspond to the dimensions of the resource tensor, including dimensions corresponding to the resource level dimension and the time dimension; Resources in the distributed computing system are scheduled according to the one or more resource tensors and the one or more request tensors.

Images