Computational system and method for mixed-precision processing of computational tasks

Abstract

The present invention relates to a computing system for mixed-precision processing of computational tasks. In particular, the computing system comprises a plurality of first computing elements configured to perform arithmetic calculations with a first precision, one or more second computing elements configured to perform arithmetic calculations with a second precision, and a communication infrastructure, wherein the plurality of first computing elements and one or more second computing elements are communicatively connected to the communication infrastructure, and the second precision is different from the first precision. The system proposed herein has the advantage that corresponding parts of a computational task can be performed by computing elements suited to each, in a particularly resource-saving manner.

Description

Technical Field

[0001] The present invention relates to a computing system and a method thereof for mixed-precision processing of computing tasks.

Background Art

[0002] Supercomputers are rapidly evolving into modular systems. This was first described in Patent Document 1. This is due not only to the need to efficiently cope with the increasing complexity of applications, but also to the urgent need to reduce energy consumption. Such a system may consist of several different modules, and each module solves a specific aspect of a complex application in the sense of functional parallelism that operates optimally from the viewpoints of speed, energy consumption, or other requirements.

[0003] Double-precision floating-point arithmetic (FP64) has been the de facto standard in the fields of technical and scientific simulations for decades. However, for quite some time, computer scientists and numerical mathematicians have been researching the use of numerical formats with less than 64-bit floating-point precision. The motivations are as follows: For example, in recent years, the power requirements in high-performance computing have increased extremely with the improvement of system performance, reaching a dimension where OPEX and CAPEX become equal over the life of the system. Furthermore, the cost of data storage is increasing rapidly with the increase in the size of the system. Thirdly, the execution time of applications is increasing rapidly with the increase in the size of the problem, which is a phenomenon due to the fact that the computing power of the machine cannot catch up with the requirements of the algorithms caused by the increase in the size of the system. Excellent reviews on these developments, such as those on Deep Neural Net (DNN) and linear algebra applications, can be found in Non-Patent Documents 1 and 2.

[0004] Using 32, 16, 8, 4, or even just 2-bit numerical formats offers enormous advantages: Firstly, lower precision data requires less memory, making it possible in principle to handle large computational problems, such as training large neural networks. Secondly, these formats require less memory bandwidth, allowing for faster data transfer, which is particularly beneficial for bandwidth-constrained problems, as is the case with most large computational problems processed today. A common algorithmic challenge in this regard is the repeated multiplication of large matrices and vectors that appears in iterative solvers. Furthermore, lower precision allows mathematical operations to be computed much faster and with less power consumption.

[0005] In the field of mixed-precision computing, methods have been found to ensure that task-specific accuracy is not compromised compared to high-precision computing. This is achieved by precisely identifying the steps that require high precision, for example, using 64-bit floating-point numbers only for these steps, while 16-bit floating-point numbers may suffice for all other steps. A practical example is the matrix-vector multiplication mentioned above. [Prior art documents] [Patent Documents]

[0006] [Patent Document 1] International Publication No. 2012 / 049247 [Non-patent literature]

[0007] [Non-Patent Document 1] 'Train with Mixed Precision', User's Guide, NVIDIA Docs (DA-08617-001_v001, February 2023) [Non-Patent Document 2] Azzam Haidar et al., 'Mixed-Precision iterative refinement using tensor cores on GPUs to accelerate solution of linear systems', 25 November 2020 (doi.org / 10.1098 / rspa.2020.0110) [Overview of the project] [Means for solving the problem]

[0008] The present invention is defined by its independent claims. In particular, the present invention provides a computing system for mixed-precision processing of computational tasks. The computing system includes a plurality of first computing elements configured to perform arithmetic calculations with first precision, one or more second computing elements configured to perform arithmetic calculations with second precision, and a communication infrastructure, wherein the plurality of first computing elements and one or more second computing elements are communicably connected to the communication infrastructure, and the second precision is different from the first precision.

[0009] This invention includes aspects of both the Modular Supercomputing Architecture (MSA) and the Mixed Precision Arithmetic (MPA) concepts.

[0010] For this purpose, cluster systems equipped with computing elements (CEs) capable of representing high precision—typically 64-bit operations—are assigned to a portion of computational tasks requiring high precision. This usually represents only a small fraction of the operations performed in the MPA (Multi-Performance Automation) domain. From an MSA (Multi-Simulation Automation) perspective, most of the computation is performed on boosters, and in this case, CEs capable of performing only low precision are advantageously selected. However, these CEs perform computations much faster than high-precision CEs on a cluster, with lower memory and communication requirements, and therefore lower power requirements.

[0011] A key feature of MSA is its ability to dynamically allocate and modify resources on boosters at the request of the resource manager, triggered from code by the execution of appropriate directives. This, advantageously, extends this capability to a level of precision that should be used in the most optimal way. This can be achieved through a monitoring function (MF) within the code.

[0012] One dynamic implementation involves balancing execution between the cluster's high-precision CE and the booster's low-precision CE. The MF then determines whether higher precision is needed during actual computation and moves execution from the booster CE to the cluster CE. If lower precision is advantageous, the computation is moved accordingly from the cluster CE to the booster CE.

[0013] Another important dynamic realization is given for the case of two or more boosters, each with a CE having different precisions. For clarity, consider the following example: A cluster has a 64-bit precision CE, and five boosters are available on a single shared network, tightly connected to the cluster, one booster with a 32-bit precision CE, a second with a 16-bit precision, a third with an 8-bit precision, one with a 4-bit precision, and one with a 2-bit precision.

[0014] In this case, dynamic implementation offers the freedom to move the computational load related to the optimization criteria between any boosters and clusters. This enables hierarchical cascaded execution of MPA, a field that is currently undergoing active development. In the case of 2-bit precision, this may relate to quadratic unconstrained binary optimization problems, so-called QUBOs, which arise in the field of discrete optimization and are advantageously used in computations relating to quantum annealing and / or quantum computing (in the context of the QAOA algorithm).

[0015] Another application area of ​​MSA is in highly advanced, modern chip architectures consisting of complex cores for complex, e.g., recursive computation tasks, and cores tuned for maximum computational performance. In all modern chip architectures since 2020, all of these are dynamically allocated to each other. In practice, it is advantageous to apply the principles presented herein to a CE consisting of two types of cores: one for high-precision computation and one for low-precision computation. The on-chip controller takes over the resource manager's tasks. Furthermore, since different core sets can operate with different caching principles, the low-precision core can be operated without caching for maximum streaming I / O. The computing systems for mixed-precision processing of computational tasks described herein advantageously include a plurality of first computing elements configured to perform arithmetic calculations with first precision, one or more second computing elements configured to perform arithmetic calculations with second precision, and a communication infrastructure such as Gigabit Ethernet, InfiniBand, Omni-Path, Mellanox HDR, Slingshot-10, or any custom interconnect or proprietary network.

[0016] Multiple first computing elements and one or more second computing elements are communicated to a communication infrastructure. This provides the advantage that the first and second computing elements can be freely assigned to each other to perform multiple parts of a single computing task. The state of being communicated to a communication infrastructure also includes scenarios in which the first computing elements are connected to a separate internal communication infrastructure, which in turn is connected to the aforementioned communication infrastructure, possibly via a communication interface. This is particularly advantageous in configurations in which multiple first computing elements form a cluster or are formed by specific cores of a single computer processor, such as a system-on-a-chip. The communication interface can also adapt the protocol of the internal communication infrastructure to another protocol used by the aforementioned communication infrastructure. Similarly, with respect to second computing elements or any group of computing elements, the state of being communicated to a communication infrastructure also includes scenarios in which the second computing elements or any group of computing elements are connected by their own communication infrastructure, which in turn is connected to the aforementioned communication infrastructure, possibly via their respective communication interfaces.

[0017] A computational task can be defined by an algorithm, source code, or binary code, or by a combination of these elements. For example, a computational task may be a simulation computed by a computer cluster array, or the training of an AI (artificial intelligence) model such as a DNN or a Transformer like GPT (OpenAI). Furthermore, a computational task may consist of multiple sub-problems, which are called parts of the computational task, and together they describe the entire computational task. A computational task can be divided into multiple parts, for example, at least one first part and at least one second part. A computing system can also solve multiple parts of a computational task in parallel or sequentially. Some parts of a computational task may be suitable for parallel computing and, advantageously, can be processed by specially adapted processing elements such as GPUs. Alternatively, parts may be formed according to the precision required for use in arithmetic operations.

[0018] Preferably, multiple first computing elements are configured to perform arithmetic calculations with a first precision and can only operate precisely at this precision. This has the advantage that each piece of hardware can be optimized precisely to this precision. This can improve the energy efficiency and cost efficiency of the components. The same may apply to any computing element of the computing system proposed herein.

[0019] Currently, it has been proposed to divide a computational task into multiple parts, depending on the precision requirements of the calculations to be applied to each part. For example, there may be a part requiring double-precision floating-point arithmetic and a part where half-precision floating-point arithmetic is possible. This is advantageous when multiple different computational elements are configured to provide different precisions for arithmetic calculations, as currently proposed.

[0020] Furthermore, multiple parts can also be formed according to whether each individual part is suitable for parallelization. Such parallelizable parts can preferably be processed by computational elements optimized for parallel processing. This has proven to be particularly useful when one or more second computational elements have a different architecture from the plurality of first computational elements and are optimized for parallel processing. Preferably, the computing system includes one or more third computational elements configured to perform arithmetic calculations at a third precision, which is different from the second precision. If the second computational element has single precision (FP32) for arithmetic operations, it can be advantageous for the third computational element to have double precision (FP64). If the first computational element is also set to double precision (FP64), i.e., the same precision as the third computational element, both the first and third computational elements can be utilized for processing corresponding parts of computational tasks that require this precision. This is particularly advantageous if the third computational element has a different architecture from the first computational element, for example, if the third computational element is optimized for parallel computing. In this case, if the computational task can be parallelized, the third computational element optimized for parallel processing is particularly suitable for handling the processing of this part of the computational task.

[0021] Assume that an arithmetic task includes a first part processed at a first precision and a second part processed at a second precision. In this case, each part of the arithmetic task must be assigned to a corresponding arithmetic element configured to perform the precision required for the arithmetic operation. For this purpose, the computing system further includes a management entity. The management entity is configured to assign parts with a certain specific arithmetic precision requirement to the corresponding arithmetic elements for processing. In the above case, the management entity will assign the first part of the computational task to one or more of the plurality of first computational elements and the second part of the computational task to one of the one or more second computational elements.

[0022] Subsequently, assume that the third accuracy is equal to the first accuracy. Considering the required accuracy, this means that either the first computing element or the third computing element is suitable for processing tasks that require the first accuracy. In this case, what matters is not the required accuracy, but rather which computing element is particularly suitable for processing that part of the computing task at that time.

[0023] Here, when determining which computing element to use, other criteria can play a role. For example, if the first part of the computing task can benefit from parallel processing and the first computing element is not optimized for parallel processing like the third computing element, the management entity may allocate the first part of the computing task to one of the one or more third computing elements.

[0024] Advantageously, the management entity is configured to dynamically allocate different parts of the computing task to one of the computing elements based on their respective accuracy requirements and the performance of the tasks expected in each computing element. Further, the management entity is configured to allocate different parts of the computing task to one of the computing elements based on additional criteria such as energy consumption, priority, availability, latency, available bandwidth, and occupancy. The above criteria may result in an allocation such that a part of the computing task is computed by a computing element with a higher accuracy than required. This can occur when a computing element with the required accuracy is occupied by another process or another part of the computing task, has a long queue, is currently unavailable, or the result of that part of the computing task is required as an intermediate result for a further part of the computing task. This has the advantage of further optimizing the use of available resources.

[0025] Preferably, the management entity is also configured to dynamically change an allocation when one of the criteria changes and the expected benefit of another allocation exceeds a predetermined threshold. Advantageously, the management entity receives information about new allocations and / or changes in any of the criteria and / or updates to task requirements from one or more components within a group of components, which may include: a resource manager, an operating system, policies within program code, monitoring functions, an on-chip controller, sensors such as a temperature sensor or energy consumption sensor, or background tasks.

[0026] Advantageously, instructions are provided for each part of the computation task regarding the required precision and / or whether that part of the computation task can benefit from parallel processing, for example, whether it is proposed to process that part of the computation task using a GPU, FPU, FPGA, ASIC, neural processing unit, tensor processing unit, parallel processor, etc. These kinds of instructions are provided by directives in the source code of the computation task, which act as a kind of switch in the compiled or interbred code, indicating which hardware is preferable to use for processing that part of the computation task, for example, depending on floating-point precision or parallel computing capability.

[0027] The resource manager can also consider whether the relevant part of the computing task belongs to an application running in the foreground or background of a multitasking operating system.

[0028] With respect to managed entities, the terms assignment or assign may include allocating resources, virtually connecting two or more compute elements, creating a shared memory space accessible to two or more compute elements, reserving one or more compute elements for one or more other compute elements, and may also include instructing a compute unit which other compute units can be used to outsource a portion of a compute task.

[0029] In addition to the management entity, a resource manager is provided. The resource manager can be implemented as a combination of software and / or hardware, i.e., an application program and / or background task, or hardware and / or firmware. The resource manager can be considered a decision-making entity that determines which computing elements to use to process a particular part of a computing task, while the management entity can be considered an execution entity that carries out what the resource manager has decided. The management entity can also inform the resource manager of criteria related to the computing nodes with which the management entity communicates. It is also envisioned that the resource manager and the management entity be integrated into a single entity that performs both of these functions.

[0030] The management entity can be implemented in software or hardware. Software implementation can be advantageous in cluster-booster configurations. In such configurations, the management entity can be implemented as an application program and / or background task running on one or more selected compute elements or cluster nodes, or as an application program and / or background task running on each compute element or cluster node. Correspondingly, each function can also be provided as part of a booster. In a cluster-booster configuration, the resource manager can be implemented as part of the management entity or as a separate application program and / or background task. The resource manager can be considered a decision-making entity that determines which compute elements to use to process a particular part of a compute task, while the management entity can be considered an execution entity that carries out the decisions made by the resource manager. The management entity can also inform the resource manager of criteria related to the compute nodes with which the management entity communicates.

[0031] Implementing the management entity in hardware and / or firmware may be advantageous for scenarios in which the present invention is implemented on a computer chip or system-on-a-chip. In such scenarios, the resource manager may be implemented as part of an operating system used in a configuration that includes a computing system. In such cases, the management entity is preferably configured to transmit information to a resource manager in the operating system regarding criteria related to the computing node with which the management entity communicates, and further configured to receive instructions from the resource manager regarding the assignment of computing elements and / or the assignment of such computing elements to each other. The management entity itself may be implemented as an on-chip controller communicating with various computing elements, or as a group of on-chip controllers, each assigned to a particular computing element on a system-on-a-chip.

[0032] Advantageously, a monitoring function is provided to observe what arithmetic precision is required for at least some of the parts of a computation task, and / or whether it is beneficial to parallelize those parts of the computation task. Such a monitoring function can be implemented as part of an application program that instructs the resource manager on each requirement, such as the required floating-point precision. Additionally, or alternatively, the monitoring function can be implemented as part of the management entity and / or the resource manager. The monitoring function can observe the range of floating-point values ​​being used and / or overflow or underflow of arithmetic operations and instruct the use of different floating-point formats.

[0033] Preferably, a plurality of first computing elements include at least one group of components, the group of which includes: a processing node, a multicore processor, a general-purpose central processing unit, a scalar processor, a multiscalar processor, a processor core, a system on a chip, a computer, and a workstation.

[0034] One or more second computing elements include at least one of the following component groups: GPU (Graphics Processing Unit), FPU (Floating Point Unit), FPGA (Field Programmable Gate Array), ASIC (Application-Specific Integrated Circuit), Neural Processing Unit, and Tensor Processing Unit.

[0035] Preferably, the arithmetic operations are floating-point operations, and the first, second, and third precisions can each be one of a precision group, which includes: FP128, FP64, FP32, bfloat16, FP8, and TensorFloat forms. In computation, floating-point arithmetic (FP) is an operation that approximates a real number, scaled by an integer exponent with a fixed base, using a fixed-precision integer called the significant digits part or mantissa.

[0036] FP128 is a floating-point format according to IEEE 754, also known as a quadruple-precision binary floating-point format using 128 bits. FP64 is a 64-bit floating-point format, referring to a double-precision binary floating-point format according to IEEE 754. FP32 is a 32-bit format according to IEEE 754. FP16 is a half-precision floating-point format according to IEEE 754. bfloat16 is another 16-bit format that provides the same dynamic range as FP32. TensorFloat-32, or TF32, is a new arithmetic mode for the NVIDIA A100 GPU.

[0037] In one preferred embodiment, the computing system is a modular supercomputer, where multiple first computing elements form a computing cluster. In such a configuration, a second computing element forms a first booster, and a third computing element forms another booster, but the first and second boosters have different precisions with respect to the arithmetic operations they perform.

[0038] In another preferred embodiment, the computing system is a computer processor or a system-on-a-chip, characterized in that a plurality of first computing elements operate on a different caching principle than one or more second computing elements. The cache operates on the principle of locality of reference, which refers to the tendency of a processor to access the same memory location when an application is executed. Because these memory accesses are predictable, they can be leveraged through the cache. In this embodiment, it may be advantageous for a core operating at low precision to operate without a cache for optimal streaming I / O.

[0039] The present invention can also be implemented as a method for mixed-precision processing of computation tasks, the computation system comprising a plurality of first computation elements configured to perform arithmetic calculations with first precision, one or more second computation elements configured to perform arithmetic calculations with second precision, a communication infrastructure, and a management entity, wherein the plurality of first computation elements and one or more second computation elements are communicably connected to the communication infrastructure, the second precision differs from the first precision, and the computation task comprises a first part which is determined to be processed using the first precision and a second part which is determined to be processed using the second precision, the method comprising the following steps:

[0040] The management entity assigns the first part of the computation task to one or more of the first computation elements, and the management entity assigns the second part of the computation task to one or more of the second computation elements.

[0041] The architecture proposed here is suitable for performing various variations of large-scale computation, such as simulations, data analysis, and AI tasks:

[0042] Training Deep Neural Networks. One important application area is the training of DNNs in the field of artificial intelligence (AI), which have revolutionized cognitive tasks such as image and speech recognition. These are also called foundational networks. Experts are well aware that training large-scale DNNs is extremely computationally intensive, which has driven the search for computer architectures for this class of application. MSAs, particularly cluster booster architectures, store synaptic weights in a low-precision booster (BLP), thereby performing weighted summation very efficiently, which would otherwise be costly. High-precision weight updates are then performed in a high-precision cluster (CHP), so the training precision of this implementation is not limited. In other words, booster units that perform relatively inaccurate weighted summation with 4, 8, or 16 bits, depending on the requirements of the particular DNN, are combined with small, high-precision digital processing units of 32 or 64 bits that accumulate weight updates with high precision.

[0043] In specific embodiments of CHP and one or more BLP, the latter is implemented with one or more different accuracies and can perform a workflow that monitors the success of the training cycle and dynamically switches the booster calculation to the best partition size on the optimal booster according to selected optimization criteria, such as highest accuracy, maximum speedup, or minimum energy consumption.

[0044] In a specific implementation using a booster with 8-bit precision CE and a cluster with 64-bit precision CE, it is possible to save up to 3-4 coefficients (factors) for every power of 2, i.e., 8 × 2 3 =64 is 3 3 =27~4 3 This results in a 64x speedup. If we can reduce the implementation to a specific configuration using a booster with 4-bit precision CE and a cluster with 64-bit precision CE, it may even be possible to achieve speedup values ​​between 81 and 256.

[0045] Speeding up the solution of linear equations using mixed precision. Solving linear equations is arguably the most important application area of ​​today's supercomputers. Corresponding code can be found in virtually any field, including fluid dynamics; computational fluid dynamics (CFD); simulations of electromagnetic, thermodynamic, and biological systems; engineering or symbolic logic optimization problems; materials science; or, to a small extent, simulations in fundamental sciences such as astrophysics, particle physics, and gravitational physics.

[0046] The technique of combining different levels of floating-point precision is a relatively recent development in numerical analysis, representing a new approach in the context of the exponential increase in the complexity of computer applications and the enormous amount of data from various devices and sensors.

[0047] Non-patent document 2 applies this new approach to various numerical methods for solving a system of linear equations A·x=b (where A is a large, dense, non-singular n×n matrix) using LU decomposition or a pre-processed generalized minimal residual algorithm (GMRES).

[0048] The combination of MPA and MSA proposed herein makes it possible to directly provide the principles of mixed-precision linear algebra to hardware.

[0049] Numerous further applications of mixed-precision computation can be identified through high-performance computing, such as probabilistic algorithms in field theory or solid-state physics, density functional theory, or molecular dynamics in materials science and biology. [Brief explanation of the drawing]

[0050] [Figure 1] Figure 1 shows a calculation system 100 for mixed precision processing according to the present invention. [Figure 2]Figure 2 shows one embodiment of the present invention, namely a modular supercomputer based on the cluster booster principle. [Figure 3] Figure 3 shows a block diagram of the cluster booster system according to the present invention. [Figure 4] Figure 4 shows another embodiment of the present invention. [Modes for carrying out the invention]

[0051] Figure 1 shows a calculation system 100 for mixed-precision processing according to the present invention, which has a plurality of first calculation elements 102, a second calculation element 104, a third calculation element 105, a fourth calculation element 106, a fifth calculation element 107, and a sixth calculation element 108. The plurality of first calculation elements 102 have 64 bits of arithmetic precision. The second calculation element 104 has 32 bits of arithmetic precision, the third calculation element 105 has 64 bits of arithmetic precision, the fourth calculation element 106 has 16 bits of arithmetic precision, the fifth calculation element 107 has 8 bits of arithmetic precision, and the sixth calculation element 108 has 4 bits of arithmetic precision (arithmetic resession).

[0052] Multiple first computing elements 102 are communicatively connected to a communication infrastructure 110. Second, third, fourth, fifth, and sixth computing elements 104, 105, 106, 107, and 108 are also communicatively connected to this communication infrastructure 110. In one preferred embodiment, the multiple first computing elements 102 are formed by a CPU, while the second, third, fourth, fifth, and sixth computing elements 104, 105, 106, 107, and 108 are each formed by a GPU having the aforementioned arithmetic precision for processing arithmetic operations.

[0053] Because multiple different computing elements are interconnected with a communication infrastructure, it may be possible to offload portions of a computing task that are primarily performed on one or more of the multiple first computing elements 102 to one of the second, third, fourth, fifth, or sixth computing elements 104, 105, 106, 107, or 108. Such offloading is preferably performed such that the portion of the computing task being offloaded is offloaded to a computing element capable of performing the arithmetic precision required for that portion of the computing task.

[0054] For example, if it is determined in advance, or detected during the calculation, that a portion of the calculation task requires, for example, 32 bits of arithmetic precision, then preferably, the second calculation element 104 is used to perform this portion of the calculation task. If it is determined in advance, or detected during the calculation, that a portion of the calculation task requires even lower or higher arithmetic precision, then this portion of the calculation task is transferred accordingly to another calculation element. Thus, if only 16 bits, or even just 8 bits of precision, is required instead of the previously required 32 bits of arithmetic precision, then the portion of the calculation task is transferred to a fourth calculation element 106 operating with 16 bits of precision, or to a fifth calculation element 107 operating with 8 bits of precision. Similarly, if a portion of the calculation task is being performed with lower precision and a higher precision is required during the calculation, then it can be transferred accordingly to another calculation element, for example, from a second calculation element 104 with 32 bits of arithmetic precision to a third calculation element 105 with 64 bits of arithmetic precision.

[0055] This has the advantage that computational elements that provide the exact precision required for arithmetic operations are always used. This frees up resources that provide higher precision for other tasks. This allows all resources to be used while conserving resources, which leads to cost reduction. This is because high-cost resources can be saved by dynamically selecting the appropriate resources, and energy consumption during calculations can be reduced because computational elements that provide lower precision typically consume less energy than computational elements that provide higher arithmetic precision.

[0056] Figure 2 shows one embodiment of the present invention, namely a modular supercomputer 200 following the cluster booster principle. This system includes a cluster 202, a communication infrastructure 204, and three boosters 206, 207, and 208. The cluster comprises six computing elements 210 that are communicatively connected to an internal communication infrastructure 212. Furthermore, cluster 202 includes a communication interface 214 that connects the internal communication infrastructure 212 to the communication infrastructure 204 located between the cluster and the boosters. Each booster is configured accordingly, each comprising multiple computing elements 216, 217, and 218, each in this example comprising six computing elements. Within one booster, the six computing elements 216, 217, and 218 are communicatively connected to internal communication infrastructures 220, 221, and 222 provided within boosters 206, 207, and 208. The internal communication infrastructures 220, 221, and 222 of each booster are connected to their respective communication interfaces 224, 225, and 226, and these communication interfaces 224, 225, and 226 are linked to a communication infrastructure 204 located between the cluster and the boosters (posters).

[0057] Each computing element 210 of cluster 202 is formed by a cluster node, each cluster node comprising one or more CPUs, memory, and I / O units (not shown). Network components such as Gigabit Ethernet, InfiniBand, Omni-Pass, Melanox, and Slingshot are particularly suitable for the internal network 212 of cluster 202, the communication infrastructure 204 between cluster 202 and boosters 206, 207, and 208, and the communication infrastructures 220, 221, and 222 within the individual boosters 206, 207, and 208. The communication interface 214 of cluster 202, which connects the internal communication infrastructure 212 of cluster 202 to the communication infrastructure 204 between cluster 202 and boosters 206, 207, and 208, can be formed, for example, by a dedicated computing element of the cluster (not shown). Alternatively, one of the computing elements of the cluster can perform this task. For boosters 206, 207, and 208, it is preferable to provide communication interfaces 224, 225, and 226 that are different from the computing elements 216, 217, and 218 used in each booster 206, 207, and 208.

[0058] In one preferred embodiment, the computation elements 210 of cluster 202 have 64-bit arithmetic precision. However, computation elements 210 with different arithmetic precisions can also be provided within cluster 202. Preferably, the boosters 206, 207, and 208 used have different levels of arithmetic precision. For example, the first booster 206 may have 32-bit arithmetic precision, the second booster 207 may have 64-bit precision, and the third booster 208 may have 16-bit arithmetic precision. In order to design each booster particularly optimally with respect to the arithmetic precision they provide, it is preferable that each of the individual boosters 206, 207, and 208 has only one precision with respect to arithmetic operations. That is, in this example, the first booster 206 can perform operations only with 32-bit arithmetic precision, the second booster 207 can perform operations only with 64-bit arithmetic precision, and the third booster 208 can perform operations only with 16-bit arithmetic precision.

[0059] The functions of the modular supercomputer 200 described in Figure 2 correspond to the functions of the computing system 100 described in Figure 1. A typical use case is when a computing task processed on a cluster 202 on one or more computing elements 210 has certain parts that do not require the arithmetic precision provided by that cluster. For example, a part of the computing task may only require 16 bits of arithmetic precision, and therefore it is appropriate to execute that part on a third booster 208 that has the required 16 bits of precision. In addition to the required arithmetic precision, it is also a practical issue whether the part of the computing task to be offloaded can benefit from being executed by parallel processing. In such an example, it may be advantageous to offload a part of the computing task that requires 64 bits of arithmetic precision to a second booster that provides the same 64 bits of arithmetic precision. In this example, the processing of the computing task benefits from the fact that the outsourced part is processed at high speed in parallel by the second booster 207. Outsourcing of corresponding parts is performed automatically and dynamically during system operation based on various criteria. The criteria mentioned above include the required arithmetic precision, the ability to benefit from parallel processing, and several other criteria, to name just a few, such as expected or current energy consumption, task priority, availability of adequate booster capacity, time delays due to offloading, such as data transfer, available bandwidth for data transfer, and the current occupancy of each resource.

[0060] Figure 3 shows a block diagram of the cluster booster system 300 according to the present invention, comprising a cluster 302, a first booster 304, a second booster 305, and a communication infrastructure 307 connecting the cluster 302 to the first booster 304 and the second booster 305. Furthermore, a management entity 309 is connected to the communication infrastructure 307. The management entity 309 is then communicably connected to a resource manager 310.

[0061] Figure 4 shows another embodiment of the present invention. In this example, the computing system 400 is implemented as a processor 400, for example, as a computer chip or system-on-a-chip. In this case, the processor 400 comprises a first computing element 402, a second computing element 403, and a communication infrastructure 405 interconnecting these two computing elements 402, 403. The processor further has a management entity 407 that is communicably connected to each of the two computing elements 402, 403 shown. In this example, the first computing element 402 has 64-bit arithmetic precision, and the second computing element 403 has 32-bit arithmetic precision. The processor 400 is further configured to communicate with a resource manager 410, which may be provided while operating as part of an operating system 412, for example, Windows or Linux®. The management entity 407 can obtain information about the operating status of the individual computing elements 402, 403 and pass this information to the resource manager 410 of the operating system 412. Furthermore, the management entity 407 is configured to receive instructions from the resource manager 400 regarding which of the two computation elements 402 and 404 should be assigned to which part of the computation task. The resource manager 410 preferably considers multiple criteria when deciding which part to process with which computation element 402 and 403, including not only the required precision for the work metric calculation, but also other criteria such as power consumption and availability, as described in the embodiment of Figure 3. For simplification, Figure 4 shows only one 64-bit precision computation element, i.e., the first computation element 402, and only one 32-bit precision computation element, i.e., the second computation element 403. However, in actual implementations, there may be multiple instances of one or both of the computation elements 402 and 403. Furthermore, it is advantageous to have not only computation elements with two different arithmetic precisions, but also three or more computation elements with different arithmetic precisions. This has the advantage of allowing corresponding parts of the computation task to be executed by the appropriate computation element, in a particularly resource-efficient manner.

Claims

1. A computing system (100) for mixed-precision processing of computational tasks, wherein the computing system (100) comprises: A plurality of first computation elements (102) configured to perform arithmetic calculations with a first precision, One or more second computation elements (104) configured to perform arithmetic calculations with second precision, and Communication infrastructure (110) Equipped with, The plurality of first computing elements (102) and the one or more second computing elements (104) are communicated to the communication infrastructure (110). A calculation system (100) wherein the second precision is different from the first precision.

2. The computing system (100) according to claim 1, wherein the one or more second computing elements (104) have a different architecture from the plurality of first computing elements (102) in that the one or more second computing elements (104) are optimized for parallel computing.

3. The system further comprises one or more third computation elements (105) configured to perform arithmetic calculations with a third degree of precision. The calculation system (100) according to claim 1 or 2, wherein the third accuracy is different from the second accuracy.

4. The computing system (100) according to claim 3, wherein the one or more third computing elements (105) have a different architecture from the plurality of first computing elements (102) in that the one or more third computing elements (105) are optimized for parallel computing.

5. The calculation task includes a first part which is determined to be processed using the first precision, and a second part which is determined to be processed using the second precision. The calculation system (100) further comprises a management entity (309), The management entity (309) is configured to assign the first portion of the calculation task to one or more of the plurality of first calculation elements (102), The computing system (100) according to any one of claims 1 to 4, wherein the management entity (309) is further configured to assign the second portion of the computing task to one of the one or more second computing elements (104).

6. The third precision is the same as the first precision. The computing system (100) according to claims 3 to 5, wherein the management entity (309) is configured to assign the first part of the computing task to one of the one or more third computing elements (105) when the first part of the computing task can benefit from parallel computing.

7. The computing system (100) according to claim 5 or 6, wherein the management entity (309) is configured to dynamically assign a plurality of different parts of the computing task to any of the computing elements based on the accuracy requirements of each part and the expected task performance of each of the computing elements (102, 104, 105).

8. The computing system (100) according to claim 7, wherein the management entity (309) is further configured to assign a plurality of different parts of the computing task to any of the computing elements (102, 104, 105) based on additional criteria such as energy consumption, priority, availability, latency, available bandwidth, and occupancy.

9. The calculation system (100) according to claim 8, wherein the management entity (309) is further configured to dynamically change the allocation when any of the criteria changes and the expected benefits of different allocations exceed a predetermined threshold.

10. The management entity (309) receives information regarding new assignments or different criteria and task requirements from one or more groups of components, the computing system (100) according to any one of claims 7 to 9, the group including: a resource manager (310), an operating system, directives in program code, monitoring functions, an on-chip controller, sensors, and background tasks.

11. The computing system (100) according to any one of claims 1 to 10, wherein the computing system (100) is a modular supercomputer, and the plurality of first computing elements (102) form a computing cluster.

12. The computing system (100) is a computer processor or a system on a chip, and the plurality of first computing elements (102) operate on a different cache principle than the one or more second computing elements (104), as described in any one of claims 1 to 10.

13. The plurality of first computing elements (102) include at least one of a group of components, the computing system (100) according to any one of claims 1 to 12, wherein the group includes: a processing node, a multicore processor, a general-purpose central processing unit, a scalar processor, a multiscalar processor, a processor core, a system on a chip, a computer, and a workstation.

14. The computing system (100) according to any one of claims 1 to 13, wherein the one or more second computing elements (104) include at least one of a group of components, the group including: a GPU, an FPU, an FPGA, an ASIC, a neural processing unit, a tensor processing unit, and a parallel processor.

15. The calculation system (100) according to any one of claims 1 to 14, wherein the arithmetic calculation is a floating-point operation, and the first precision, the second precision, and the third precision can each be one of a precision group, the group including: FP128, FP64, FP32, bfloat16, FP8, and TensorFloat formats.

16. A method for mixed-precision processing of computational tasks, The computing system (100) comprises a plurality of first computing elements (102) configured to perform arithmetic calculations with a first precision, one or more second computing elements (104) configured to perform arithmetic calculations with a second precision, a communication infrastructure (110), and a management entity (309), wherein the plurality of first computing elements (102) and the one or more second computing elements (104) are communicably connected to the communication infrastructure (110), and the second precision differs from the first precision. The calculation task includes a first part which is determined to be processed using the first precision and a second part which is determined to be processed using the second precision, and the method is as follows: The management entity (309) assigns the first portion of the calculation task to one or more of the plurality of first calculation elements (102), The management entity (309) assigns the second portion of the calculation task to one of the one or more second calculation elements (104). Methods that include...

Images