Apparatus and method for efficient parallel computation

Abstract

The present invention provides a computing unit for operating in a parallel computing system, the computing unit comprising a plurality of processing elements and an interface for connecting the computing unit to other components of the computing system wherein each processing element has a nominal maximum processing rate NPR and each processing element includes a respective memory unit such thatdata can be transferred from the memory unit at a predetermined maximum data rate MBW and the interface provides a maximum data transfer rate CBW, wherein in order to provide a predetermined peak calculation performance for the computing unit PP obtainable by a number n processing elements operating at the nominal maximum processing rate such that PP = n * NPR operations per second, the computingunit includes an integer multiple f times n processing elements wherein f is greater than one and each processing element is limited to operate at a processing rate of NPR / f.

Description

technical field [0001] The present invention relates to parallel processing systems, and in particular to parallel processing systems with improved efficiency in terms of performance / power consumption. Background technique [0002] In a typical parallel processing system, multiple computing nodes, each including one or more processing elements, are connected by a high-speed network. The processing elements of the compute nodes each have internal memory. Processing elements are connected within their compute nodes. The connectivity of the nodes in the computation can be achieved using high-speed networks, high-speed networks of separate nodes in the computation, or common memory (as for example in symmetric multiprocessing SMP systems). This arrangement is shown in the figure 1 middle. [0003] figure 1 An arrangement of a plurality of computing nodes CN is shown, where each computing node comprises a plurality of processing elements PE each with a corresponding memory M...

AI Technical Summary

Problems solved by technology

The downside is that most data-intensive applications will a priori achieve η below 5% to 10% on current CPUs A and about 15% of η on current GPUs A , regardless of any further reduction due to the algorithm being executed or parallelism in terms of required processing elements

For a given R of processing elements, the more data-intensive the application, the efficiency η A become lower

Images