System and method for automatic parallelization of processing code for multiprocessor systems with optimized latency - Patents.com
The compiler system optimizes multiprocessor systems by translating code into computational block nodes and minimizing latency, addressing parallelization challenges and improving throughput across diverse hardware, particularly in computationally intensive applications.
Patent Information
- Application Number
- JP2024520843
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2021-10-27
- Publication Date
- 2024-11-13
- Estimated Expiration
- 2041-10-27
AI Technical Summary
Existing compilers struggle to efficiently manage multiprocessor systems by optimizing latency and reducing data dependencies for higher throughput, particularly in computationally rich applications, due to complex parallelization and latency issues in modern computing infrastructures.
A compiler system that translates source code into machine code for parallel processing systems, utilizing a parser module to divide code into computational block nodes, and an optimizer module to minimize latency through numerical matrix optimization techniques, ensuring efficient execution across multiple processing units.
The system achieves optimized latency and reduced data dependencies, enabling higher throughput by automatically parallelizing code and adapting it to various hardware infrastructures, including CPUs, GPUs, and clusters, thus enhancing performance in applications like machine learning, high-performance computing, and embedded systems.
Smart Images

Figure 2024541823000001_ABST
Abstract
Description
[Technical field]
[0001] The present invention generally relates to multiprocessor systems, multicore systems, and parallel computing systems that allow multiprocessing, where two or more processors work together to execute multiple program codes and / or processor instruction sets simultaneously, where the multiprocessor system or parallel computing system has several central processing units linked together to allow the multiprocessing or parallel processing to occur. In particular, the present invention relates to a compiler system that optimizes computer programs for execution by a parallel processing system having multiple processing units. More specifically, the described embodiments relate to systems and methods for automatic parallelization of code for execution in the described multiprocessor systems. In the technical field of multiprocessor systems, important characteristics and classifications arise from, among other things, how processor memory accesses are handled and whether the system processors are of a single type or of various types in the system architecture. [Background technology]
[0002] The growing need for computationally heavy applications, driven among others by the birth of Industry 4.0 technologies, the ongoing automation of traditional manufacturing with more and more industrial implementations using modern smart technologies, the large-scale machine-to-machine communication (M2M), the Internet of Things (IoT) with improved communication and monitoring means with large-scale data access and aggregation, as well as the growing importance of industrial simulation techniques, for example digital twin technologies, is causing a paradigm shift from centralized computing to parallel and distributed computing. Parallel computing involves distributing computing jobs across different computing resources. These resources generally include several central processing units (CPUs), graphics processing units (GPUs), memory, storage, and networking support. In addition to this growing demand, there has been a huge development in the performance and capabilities of computer systems, at the hardware level, during the last 50 years. This has been made possible with the help of Very Large Scale Integration (VLSI) technology. VLSI technology allows a large number of components to be accommodated on a single chip, increasing the clock speed. Thus, more operations can be performed in parallel at once. Parallel processing is also related to data locality and data communication. Thus, the field of parallel computer architecture typically refers to systems and methods that organize all the resources of a parallel computer system to maximize performance and programmability at any given time within the limitations imposed by technology and cost. Parallel computer architecture adds a dimension to the development of computer systems by using more and more processors. In principle, the performance achieved by utilizing multiple processors is higher than the performance of a single processor at a given time. However, parallelizing processor code is complex and difficult to automate for truly optimized parallelization.
[0003] While centralized computing works well for many applications, it may be inadequate for performing computationally rich applications and large amounts of data processing. Programs may be executed serially or distributed to run on multiple processors. When a program is executed serially, only one processor can be utilized, and thus throughput is limited to the speed of the processor. Such a system with one processor is sufficient for many applications, but not for computationally intensive applications such as modern computer-based simulation techniques. Processing code can be executed in parallel in a multiprocessor system, resulting in higher throughput. Multiprocessor systems require breaking down the code into smaller code blocks and efficiently managing the execution of the code. For processors to execute in parallel, the data to each processor must be independent. To improve throughput, instances of the same code block may be executed simultaneously on several processors. When a processor requires data from a previous execution or from another process that is currently performing a calculation, parallel processing efficiency may be reduced due to the latency incurred in exchanging data between and / or among processor units. In general, when a processor enters a state where the program execution is suspended or not executed for any reason and the instructions belonging to it are not fetched from memory or executed, those states cause the processor to be idle and affect parallel processing efficiency. Data dependencies need to be considered when scheduling the processor. It is difficult to efficiently manage multiple processors and data dependencies for higher throughput. It is desirable to have a method and system for efficiently managing code blocks in computationally rich applications. It is noted that the latency problem also exists in single processor systems. In that case, for example, latency-oriented processor architectures are used to minimize the problem.It is a microprocessor microarchitecture designed to provide serial computing threads with low latency. These architectures generally aim to execute as many instructions belonging to a single serial thread as possible within a given time window, and the time to fully execute a single instruction from the fetch stage to the retire stage can vary in some cases from a few cycles to hundreds of cycles. However, these techniques do not automatically apply to the latency problem of (massively) parallel computing systems.
[0004] In general, parallel computing systems require efficient parallel coding or programming, and parallel programming becomes a programming paradigm. On the one hand, it involves how to divide a computer program into individual sections that can be executed simultaneously in parallel, and on the other hand, how to synchronize the concurrent code sections. This is in contrast to classical sequential (or serial) programming and coding. Parallel execution of a program can be supported on the hardware side, in which case programming languages are typically adapted for this. For example, parallel programming can be done explicitly by having the programmer execute parts of the program in separate processes or threads, or it can be done automatically, so that causally independent (parallelizable) instruction sequences are executed side by side, i.e. in parallel. This parallelization can be done automatically by compiler systems, if a computer with a multicore processor or a parallel computer is available as a target platform. Some modern CPUs can also recognize such independence (in the machine code or microcode of the program) and distribute instructions to different parts of the processor in such a way that they are executed simultaneously (out-of-order execution). However, as soon as the individual processes or threads communicate with each other, they affect each other and are in that sense no longer concurrent as a whole. Only the individual sub-processes are still concurrent with each other. If the order of execution of the communication points of the individual processes or threads cannot be defined accordingly, conflicts may arise; in particular the so-called deadlocks, when two processes wait for each other (or block each other), or race conditions, when two processes overwrite each other's results. In the prior art, synchronization techniques are used to solve this problem, for example the mutual exclusion (mutex) technique. Such techniques are able to prevent race conditions, but do not allow an optimized parallel processing of processes or threads with minimal latency of the processor unit.
[0005] (Micro)processors are based on integrated circuits that allow to perform arithmetic and logical operations based on two binary values (simplest 1 / 0). For this, the binary values must be available to the computational units of the processor. The processor units need to get the two binary values to compute the result of the expression a=b operand c. The time it takes to get the data for these operations is known as the latency. These latency times have a wide hierarchical range, from registers, L1 cache, memory accesses, I / O operations, or network transfers, as well as the processor configuration (e.g. CPU or GPU). Since every single component has a latency time, the overall latency time for a computation is, in modern computing infrastructures, mainly the combination of hardware components needed to get the data from one location to another. In modern architectures, the different software layers (e.g. operating systems) also have a large impact. The difference between the fastest and slowest location for the CPU (or GPU) to get the data can be huge (>10 9 times range). Figure 1 shows the formation of latency times in modern computing infrastructure. As shown in Figure 1, parallel computing machines have been developed with different distinct architectures. It is important to note that parallel architecture enhances the traditional concept of computer architecture with communication architecture. Computer architecture defines the key abstractions (such as user-system boundaries and hardware-software boundaries) and organizational structures, while communication architecture defines the basic communication and synchronization behaviors. It also deals with organizational structures.
[0006] Computer applications are typically written at the top layer, i.e., in a high-level language, based on a corresponding programming model. Various parallel programming models are known, for example, as (i) shared address space, (ii) message passing, or (iii) data parallel programming, which refer to corresponding multiprocessor system architectures. Shared memory multiprocessors are one such class of parallel machines. Shared memory multiprocessor systems give better throughput for multiprogramming workloads and support parallel programs. In this case, the computer system allows a set of processors and I / O controllers to access a collection of memory modules through some hardware interconnection. Memory capacity is increased by adding memory modules, and I / O capacity is increased by adding devices to the I / O controllers or by adding additional I / O controllers. Processing power can be increased by implementing faster processors or by adding more processors. As shown in Figure 2, resources are organized around a central memory bus. Through the bus access mechanism, any processor can access any physical address in the system. Since all processors are assumed to be or are actually equidistant from all memory locations, the access time or latency for all processors is the same to any memory location. This is called a symmetric multiprocessor system.
[0007] Message-passing architectures are another class of parallel machines and programming models. They provide communication between processors as explicit I / O operations. Communication is combined at the I / O level instead of the memory system. In message-passing architectures, user communication is performed by using operating system or library calls that perform lower level actions, including the actual communication operations. As a result, there is a gap between the programming model and the communication operations at the physical hardware level. Send and receive are the most common user-level communication operations in message-passing systems. Send specifies a local data buffer (to be sent) and a receiving remote processor. Receive specifies a sending process and a local data buffer where the sent data will be placed. In a send operation, an identifier or tag is attached to the message, and a receive operation specifies a matching rule, such as a specific tag from a specific processor or any tag from any processor. The combination of a send and a matching receive completes a memory-to-memory copy. Each end specifies its local data address and a pair-wise synchronization event. Message passing and shared address spaces traditionally represented two separate programming models, each with its own paradigm for sharing, synchronization, and communication, but today the underlying machine structures are converging towards a common organization.
[0008] Finally, data parallelism, also called processor arrays, data parallel architectures, or single instruction multiple data machines, is a further class of parallel machines and programming models. The main feature of this programming model is that operations can be performed in parallel on each element of a large regular data structure (such as an array or matrix). Data parallel programming languages are usually implemented by looking at the local address spaces of a group of processes (one per processor) that form an explicit global space. Either a shared address space or message passing can be used, since all processors communicate together and there is a global view of all operations. However, the development of programming models alone cannot increase the efficiency of computers, and neither can the development of hardware alone. Furthermore, a top-level programming model necessarily introduces boundary conditions, such as model-specific architectures, that are given by the requirements of the programming model. Since a parallel program consists of one or more threads operating on data, the underlying parallel programming model defines what data the threads need, what operations can be performed on the required data, and in what order the operations follow. Thus, there are limitations to optimizing machine code for multiprocessor systems due to the boundaries of the underlying programming model: a parallel program must coordinate the activities of its threads to ensure that inter-program dependencies are enforced.
[0009] As Figure 1 shows, parallel computing machines have been developed with different distinct architectures, each of which creates different latency times in their computational infrastructure. One of the most common multiprocessor systems is the shared memory multiprocessor system. Essentially, three basic architectures are known for shared memory multiprocessor systems: (i) Uniform Memory Access (UMA), (ii) Non-uniform Memory Access (NUMA), and (iii) Cache Only Memory Architecture (COMA). In the UMA architecture (see Figure 3), all processors share the physical memory uniformly. All processors have equal access times for all memory words. Each processor may have a private cache memory. Peripheral devices follow the same rules. If all processors have equal access to all peripheral devices, the system is called a symmetric multiprocessor. If only one or a few processors can access the peripheral devices, the system is called an asymmetric multiprocessor. In a NUMA multiprocessor architecture (see Figure 4), the access time varies with the location of the memory word. The shared memory is physically distributed among all the processors, called local memory. The collection of all the local memories forms a global address space that can be accessed by all the processors. Finally, the COMA multiprocessor architecture (see Figure 5) is a special case of the NUMA multiprocessor architecture. In the COMA multiprocessor architecture, all the distributed main memory is transformed into cache memory. The COMA architecture can also be applied to distributed memory multicomputers. A distributed memory multicomputer system consists of multiple computers, typically called nodes, interconnected by a message passing network.Each node acts as an autonomous computer with a processor, local memory, and sometimes I / O devices. In this case, all local memory is private and accessible only to the local processor, hence such machines are also called no-remote-memory-access (NORMA) machines. Other known multiprocessor architectures are, for example, multivector computers, and single-instruction-multiple-data (SIMD) parallel computers, parallel random-access machines (PRAM), and parallel computers based on very large scale integration (VLSI) chips, all with different multiprocessor architectures and infrastructure characteristics. In summary, different multiprocessor architectures result in different latency time formation in their computational infrastructure, so that the development of programming models alone cannot increase the efficiency of computers, and the development of hardware alone cannot do it.
[0010] As mentioned above, code parallelization can also be performed automatically by a compiler system if a computer with multi-core processors or a parallel computer is available as a target platform. Such automatic parallelization, also called automatic parallelization, means converting sequential code into multithreaded and / or vectorized code for simultaneous use of multiple processors, for example in a shared-memory multiprocessor (SMP) machine. In prior art systems, fully automatic parallelization of sequential programs is technically difficult because it requires complex program analysis and the best approach may depend on parameter values that are not known at compile time. The programming control structure that compiler systems focus most on for automatic parallelization is loops, because typically most of the program's execution time takes place inside some form of loop. There are two main approaches to loop parallelization: pipelined multithreading and cyclic multithreading. Compiler structures for automatic parallelization usually include a parser, an analyzer, a scheduler, and a code generator. A parser in a compiler system covers the first processing stage, where for example a scanner reads the input source file and identifies all static and external uses. Each line in the file is checked against predefined patterns and separated into tokens. These tokens are stored in a file that is later used by a grammar engine. The grammar engine checks for patterns of tokens that match predefined rules to identify variables, loops, control statements, functions, etc. in the code. In the second stage, an analyzer identifies sections of code that can be executed in parallel at the same time. The analyzer uses static data information provided by the scanner-parser. The analyzer first finds all completely independent functions and marks them as individual tasks. The analyzer then finds which tasks have dependencies.In the third stage, the scheduler lists all the tasks and their dependencies on each other in terms of execution and start time. The scheduler generates an optimal schedule in terms of the number of processors used or the total execution time for the application. In the fourth and final stage, the scheduler generates a list of all the tasks and details of the cores they will run on, along with the length of time they will run. The code generator then inserts special structures into the code that are read during execution by the scheduler. These structures tell the scheduler on which cores a particular task will run, along with its start and end times.
[0011] When a circularly multithreaded parallelizing compiler is used, the compiler attempts to split each loop so that each iteration of the loop can be executed in parallel simultaneously on a separate processor. During automatic parallelization, the compiler typically performs two passes of automated evaluation before the actual parallelization to determine two fundamental prerequisites for parallelization: (i) the first pass evaluates whether it is safe to parallelize the loop based on dependency and alias analysis, and (ii) the second pass evaluates whether it is worthwhile to parallelize it based on an estimation (modeling) of the program workload and the capacity of the parallel system. The first pass of the compiler performs a data dependency analysis of the loop to determine whether each iteration of the loop can be executed independently of other iterations. Data dependencies can sometimes be addressed, but may incur additional overhead in the form of message passing, shared memory synchronization, or some other method of processor communication. The second pass attempts to justify the parallelization effort by comparing the theoretical execution time of the parallelized code with the sequential execution time of the code. It is important to understand that code does not necessarily benefit from parallel execution. The extra overhead that may be associated with using multiple processors can eat into the potential speedup of parallelized code.
[0012] When a pipelined multithreaded parallelizing compiler is used for automatic parallelization, the compiler attempts to divide a sequence of operations within a loop into a series of code blocks such that each code block can be executed simultaneously in parallel on a separate processor.
[0013] In a concrete system that uses pipes and filters, there are many parallel problems that have relatively independent blocks of code like that: when producing a live broadcast, for example, many different tasks must be executed many times per second.
[0014] A pipelined, multithreaded parallelizing compiler tries to assign each of these operations to a different processor, usually arranged in a systolic array, and inserts the appropriate code to forward the output of one processor to the next. For example, in modern computer systems, one of the focuses is to use the power of GPUs and multicore systems to compute such independent code blocks (or independent iterations of a loop) at run-time. The memory accessed (directly or indirectly) can then be marked for different iterations of the loop and compared for dependency detection. With this information, iterations are grouped into levels, so that iterations belonging to the same level can be executed independently and in parallel.
[0015] In the prior art, there are many compilers for automatic parallelization. However, most recent prior art compilers for automatic parallelization rely on the use of Fortran as a high-level language, i.e., they are only applicable to Fortran programs. This is because Fortran makes stronger guarantees about aliasing than languages such as C. Typical examples of such prior art compilers are (i) the Paradigm compiler, (ii) the Polaris compiler, (iii) the Rice Fortran D compiler, (iv) the SUIF compiler, and (v) the Vienna Fortran compiler. A further drawback of automatic parallelization with prior art compilers is that it is often difficult to achieve a high degree of optimization of the code. This is due to the fact that (a) for code that uses indirect addressing, pointers, recursion, or indirect function calls, dependency analysis is difficult because such dependencies are difficult to detect at compile time, (b) loops often have unknown numbers of iterations, (c) it is difficult to coordinate access to global resources in terms of memory allocation, I / O, and shared variables, and (d) irregular algorithms that use input-dependent indirection interfere with compile-time analysis and optimization.
[0016] One important task of a compiler is to try to deal with latency times efficiently. Compilation is the transcription from so-called high-level languages that are human readable (C, python, java, etc.) to assembler / processor code, which consists only of instructions available on a given processor. As already mentioned, modern applications with large demands on data or computation must target an appropriate infrastructure, introducing many different latency times, only some of which can currently be solved by prior art compiler optimization techniques.
[0017] For every level of complexity (hardware components), solutions have been developed and evolved historically, from compiler optimization techniques, to multi-threaded libraries for concurrent data structures to prevent race conditions, vectorization of code, GPU systems with corresponding programming languages (e.g., OpenCL (Open Computing Language)), frameworks such as "TensorFlow" to distribute computations by programmers, and big data algorithms such as "MapReduce". Here, MapReduce is a programming technique and related implementation for processing and generating big data sets using parallel distributed algorithms on clusters. In the field of high performance computing, theoretical mathematics-based techniques have been developed and defined for partitioning large matrices into special gridding techniques for finite difference or element methods. This includes, for example, protocols in cluster infrastructures. The Message Passing Interface (MPI) supports the transfer of data to different processes through the infrastructure.
[0018] As mentioned above, the list of prior art optimization techniques is long. However, from a systems theory point of view, the problem is more or less always the same: how can code (any code) interact most efficiently with latency times in a complex hardware infrastructure? Compilers work well when using a single CPU. As soon as the complexity of the hardware increases, compilers are no longer able to really make the code parallel. Parallelization becomes merely CPU-approximated, for example by introducing microprocesses. The hardware industry for CPUs, GPUs and their clusters is mainly focused on their specific domains, and developers and research are focused on implementation techniques and framework developments and have not moved so far into the field of more general (cross-industry) approaches. Furthermore, Non-Patent Document 1 revealed the partitioning of algorithms that process data streams onto different processor cores. The author's model is a simple communication network with a simple additive communication model between the processor cores. This does not allow realistic conclusions about the real communication load caused by partitioning on multiple cores.
[0019] Generally, known processor manufacturers focus on their processors and related hardware components, while other developers, e.g., High Performance Computing (HPC) research groups, focus on the use of numerical methods and libraries. Currently, there is no attempt to solve the problem of optimizing compiler systems from a system theoretical perspective by accessing the latency dynamics resulting from a given source code. Prior art source code simply consists of a sequence of statements that result in direct reads and writes for a given target infrastructure. [Prior art documents] [Non-patent literature]
[0020] [Non-Patent Document 1] Ruggiero, Martino; Guerri, Alessio; Bertozzi, Davide; Milano, Michaela; Benini, Luca, Brochure A Fast and Accurate Technique for Mapping Parallel Applications on Stream-Oriented MPSoC Platforms with Communication Awareness, International Journal of Parallel Programming, Vol. 36, No. 1, Feb 08 Summary of the Invention [Problem to be solved by the invention]
[0021] One object of the present invention is to provide a compiler system for multiprocessor and multicomputer systems that compiles program code into machine code with optimized latency of the processing units of the multiprocessor system, thereby efficiently managing multiprocessor and data dependencies for higher throughput, without the drawbacks of the prior art systems as described above. In particular, it is an object of the present invention to provide a system and techniques that can be used to achieve the best performance in multiprocessor machines through automatic parallelization, thereby optimizing the utilization of low levels of parallelism (time and space) at the level of processing machine instructions. A further object of the present invention is to overcome the drawbacks of the prior art and the limitations of dealing with parallelized sections, e.g. loops or specific sections of code, that are typically constrained to the specific system envisaged.
[0022] An automatic parallelization system should be able to optimize identifying parallelization opportunities as a pivotal step while generating a multi-threaded application. [Means for solving the problem]
[0023] According to the invention, these objects are achieved in particular by means of the features of the independent claims. Furthermore, further advantageous embodiments can be derived from the dependent claims and the associated description.
[0024] According to the present invention, a compiler system and a corresponding method for optimizing compiling a program code for execution by a parallel processing system having a plurality of processing units which simultaneously process data in the parallel processing system by executing the program code are achieved, in particular, the compiler system has means for translating a source programming language of a source machine code of the computer program into a machine code as a target programming language and generating the machine code as a parallel processing code comprising several instructions executable by the plurality of processing units of the parallel processing system or controlling the operation of the plurality of processing units; the parallel processing system has memory units, comprising at least a main execution memory unit comprising a plurality of memory banks for holding at least a part of data of the processing code, and a transition buffer unit storing starting positions and data segments of the processing code and comprising at least a high-speed memory containing branch or jump instructions and / or memory references and data values used, the main execution memory unit providing a slower access time than the transition buffer unit; the execution of the processing code by the parallel processing system includes the occurrence of a latency time, the latency time being given by an idle time of the processing units between sending data back to the parallel processing system after processing a particular block of instructions of the processing code on the data by one processing unit and receiving by the processing unit data necessary for execution of a successive block of instructions of the processing code; the compiler system has a parser module for translating the source programming language into code having a flow of basic instructions executable by the processing units, the basic instructions being selectable from a processing unit specific set of basic instructions, the basic instructions including basic arithmetic and / or logical operations and / or control operations and / or memory operations for the several processing units;the parser module has means for dividing a code of basic instructions into computation block nodes, each computation block node consisting of a smallest possible segmentation of a further indecomposable sequence of basic instructions of the code processable by a single processing unit, the smallest possible segmentation of the basic instructions being characterized by a sequence of basic instructions consisting of consecutive read and write instructions, the sequence not being further decomposable by smaller sequences of basic instructions between consecutive read and write instructions, the read and write instructions being required to receive data required to process the sequence of basic instructions by the processing unit and to send back data after processing by the sequence; the compiler system has a matrix builder for generating a numerical matrix from computation chains divided from the code in dependence on the latency time, the numerical matrix including a computation matrix and a transfer matrix, the computation chains being formed by one or more computation block nodes creating an ordered flow of computation block nodes, each computation chain being executed by one processing unit; the computation matrix includes the computation chain of processing units in each row, and each column includes the sequence of primitives of computation block nodes in the computation chain of the row, and the transfer matrix includes transfer characteristics associated with data transfer from one computation block node to a successive computation block node; the compiler system includes an optimizer module that uses a numerical matrix optimization technique to minimize an aggregated incurred latency time that aggregates all incurred latency times by providing an optimized structure of computation chains processed by the multiple processing units, and a code generator generates optimized machine code for the multiple processing units, the optimized aggregated latency time being. If matrices or tensors are used, the optimization by the optimization module can be based, for example, on numerical matrix optimization techniques (or more general numerical tensor optimization techniques). Technically, the optimization problem is formulated by using tensors and / or matrices, thus obtaining a matrix / tensor field optimization problem. For linear optimization, for example, matrices and linear programming can be used by the optimizer module. In certain applications of the invention, the concept of tensors can be technically useful, for example. In optimization, tensor techniques can solve systems of nonlinear relations and equations, for unconstrained optimization using second derivatives. Tensor methods can be used as general-purpose methods, especially for problems where the Jacobian matrix in the solution is singular or ill-conditioned. Tensor methods can also be used for linear optimization problems. An important feature of tensors is that their values do not change when they cause regular nonlinear coordinate transformations, and thus this concept can be technically useful for the characterization of structural properties that do not depend on regular nonlinear coordinate transformations. Thus, tensor optimization can also be applied within the framework of nonlinear optimization. However, it should be noted that one of the technical advantages of the present invention is that all matrices known to date are linear for optimization, in contrast to prior art optimization techniques in the field of automatic parallelization of source code, while prior art systems have to rely mainly on non-linear optimization. Here, optimization means the problem of finding a set of inputs to an objective function that results in a maximum or minimum function evaluation. For this technically challenging problem, various machine learning algorithms can also be used together with the optimizer module, for example, from fitting logistic regression models to training artificial neural networks. If the optimizer module is realized by a realized machine learning structure, it can be formulated in such a way that it can be provided by using a continuous function optimization, where the input arguments to the function are real-valued numerical values, for example floating point values. The output from this function is also a real-valued evaluation of the input values.However, as a variant of the embodiment, optimization functions that take discrete variables, i.e. provide a combinatorial optimization problem, can also be used. To technically select the best optimization structure, for example, one approach can be to group selectable optimization structures based on the amount of information available about the target function being optimized, which can be used and utilized by the optimization algorithm. It is clear that the more information is available about the target function, the easier it is to optimize the function by machine learning. Of course, it depends on the fact that the available information can be effectively used in the optimization. Thus, one selection criterion can be associated with a differentiable target function, for example, by the question of whether the first derivative (gradient or slope) of the function can be calculated for a given candidate solution. This criterion divides the available machine learning structures into those that can utilize the calculated gradient information and those that do not, i.e., machine learning structures that use differential information and machine learning structures that do not use differential information. For applications where a differential objective function can be used, it must be noted that in this specification, a differentiable function denotes a function whose derivative can be generated for any given point in the input space. The derivative of a function with respect to a value is the rate or amount of change of the function at that point, which is also called the slope. The first derivative is defined as the slope or rate of change of the objective function at a given point, and the derivative of a function with more than one input variable (e.g., multivariate inputs) is called the gradient. Thus, the gradient can be defined as the derivative of a multivariate continuous objective function. The derivative of a multivariate objective function is a vector, and each element of the vector can be called a partial derivative, or the rate of change with respect to a given variable at that point, assuming all other variables are held constant. Furthermore, a partial derivative can be defined as an element of the derivative of a multivariate objective function. The derivative of the derivative of the objective function can then be generated, i.e., the rate of change of the rate of change of the objective function. This is called the second derivative. Thus, the second derivative can be defined as the rate at which the derivative of the objective function changes.In this case of a function that takes multiple input variables, this is a matrix, called the Hessian matrix, and the Hessian matrix is defined as the second derivative of a function with two or more input variables. Simple differentiable functions can be analytically optimized using known analytics. However, the objective function may not be analytically solvable. If the gradient of the objective function can be generated, the optimization used becomes significantly easier. Some machine learning structures that can use the gradient information and may be used for this application include bracketing algorithms, local descent algorithms, first order algorithms, and second order algorithms.
[0025] The present invention has, among other things, the advantage of providing and achieving extensive optimizations based on the lowest possible code structure, reducing high programming language code to a small number of basic instructions that cannot be further reduced in terms of data input and data output points due to the limited set of machine instructions executed on the CPU / microprocessor. The basic instructions include, for example, (i) arithmetic operations: +, -, *, / in applied numerical applications and ->, i.e., mathematical operations such as integral or differential analysis are reduced to these basic instructions, (ii) logical operations: AND, OR, etc., (iii) variable and array declarations, (iv) comparison operations: same, greater than, less than, etc., (v) code flow: jump, call, etc., (vi) if(condition){codeA}else{codeB}, (vii) loop(condition). The interaction between today's modern high-level languages (e.g., python, C, java, etc.) and the limited resources of processor instructions can be analyzed and made accessible by creating a "mapping" of reads and writes of "data points" by their operations. In other words, by mapping the read and write interactions of single instructions with an appropriate representation (which can also be represented graphically), they can be made available for numerical optimization techniques, which can automatically parallelize the source code and always result in executable concurrent parallel code. There are several techniques to access these interactions, but none that ultimately map the read and write patterns of the source code to the data introduced by the programmer's choice of variable definitions, and then proceed to extract the required sequential chains and introduce potential communication patterns, thereby "mapping" the code to a wide range of hardware infrastructures. This method discloses a new technique for "fitting" source code to a given hardware infrastructure across all levels (CPU, GPU, cluster).
[0026] The invention further has the advantage that the disclosed method and system can solve known technical problems such as solving nested loops with arrays solving PDEs (partial differential equations) in a new perspective, or well-known problems occurring in the optimization step of SOTA compilers. The method gives a new perspective to the code, this new scope is based on physical effects occurring in all classical computing infrastructures, resulting in a generic method to map the computation to a given hardware structure and derive a concurrent representation of the code on a given hardware or an ideal hardware for a given code. This is based on the result of preserving all the "read" and "write" dependencies of the introduced data nodes and building a chain of instructions depending on these dependencies. The resulting computation block nodes and their flow graphs return a well-formed base of matrices, resulting in a generic method to return code applicable to different computing units (e.g. CPU, GPU, FPGA, microcontroller, etc.). The method has a new perspective on the interaction of ICT software and hardware according to the principles of system theory. This results in a method that can bring novel solutions in a wide range of fields, as follows:
[0027] (a) Adaptive Hardware - FPGA (Field Programmable Gate Array) / ACID (Atomicity, Consistency, Isolation, Durability): The invention decomposes code into chains with instructions that clearly represent logical elements in an integrated circuit. The method gives a general form for optimizing a combination of computation and communication, and can be used to optimize groups of instructions based on the same "bit pattern" / "signal", respectively, to automatically optimize the transfer of software to FPGAs for example, or to bring new methods to bridge the gap from code to chip floor planning.
[0028] (b) ML / AI: Machine learning and artificial intelligence codes are resource intensive, especially in the training phase. The method can be used, for example, (i) to optimize known codes, (ii) to support code development that adapts complexity at run-time and is therefore difficult to parallelize up-front (as the method always results in optimized code), and (iii) to support upcoming methods (e.g. genetic algorithms, see, for example, Inside HPC Special Report by R. Farber, AI-HPC is Happening Now) rather than neural network based methods.
[0029] (c) HPC (High Performance Computing) Applications: The method can bridge gaps that exist in different research fields (e.g., from HPC to AI development) since the code can be converted from, e.g., python to C code with the MPI support library. Another application can be an adaptive mesh refinement implementation used in numerical model software packages for engineering applications, weather forecasting models, etc. Alternatively, it can be used to combine models with different spatial and temporal resolutions (e.g., computational fluid dynamics models and agent-based models, etc.) and improve existing software packages in different fields, such as modeling and analysis software packages.
[0030] (d) Automated Business Processes: The invention can also be used in process management, where the decision of whether a unit should work on a task or send the task to another unit is a well-known problem, for which the present method represents one approach.
[0031] (e) Cloud, Desktop Operating Systems, Virtual Machines, Deployment in General: Having available a generic approach to "shrink" code with respect to basic required operations and possible concurrency options supports a wide range of solutions at the interface between software and hardware. This interface obviously arises in particular for any form of operating system, software virtualization and / or deployment, or more specifically for example for virtualization solutions for cloud infrastructures, operating systems (with multi-core systems), virtual machines supporting a mix of different operating systems, or similar examples.
[0032] (f) Heterogeneous Platforms, (i) IoT and Edge Computing: Heterogeneous platforms govern a given situation in different domains, such as IoT projects, autonomous driving, combined mobile and cloud applications, and other forms of applications running with and / or on a mixed hardware infrastructure. The method can adapt the code to determine how to best distribute data, computation, and / or data communication on the platform. Furthermore, it can be incorporated into the process of deploying / developing software for different characteristics of the hardware components of a given network of computing units, optimizing the code to meet the target characteristics, for example reducing latency for some parts of the software system.
[0033] (g) Embedded Systems: Embedded systems have high requirements for a given code, e.g., on power consumption or other specific adaptations, e.g., only reduced instruction sets on some microprocessors, or similar challenges. The method can directly support this mapping, since it can optimize for the given physical characteristics, thus resulting in the most efficient code representation for any given code.
[0034] (h) Self-optimizing algorithms: The present invention allows for a completely autonomous circle. This means that the algorithm can optimize itself on a given platform without any manual interaction. This enables new applications and fields that were not known before. [Brief description of the drawings]
[0035] The invention will now be explained in more detail, by way of example, with reference to the drawings, in which:
[0036] [Figure 1] A diagram is shown that shows a schematic representation of the formation of latency times under different modern computing infrastructures. (Micro)processors are based on integrated circuits that allow to perform arithmetic and logical operations based on two binary values. For this, the binary values must be available to the processor's computational units. The processor units need to get the two binary values to compute the result of the expression a=b and operand c. The time it takes to get the data for these operations is known as the latency time. There is a wide hierarchical range for these latencies, from registers, L1 cache, memory accesses, I / O operations, or network transfers, as well as the processor configuration (e.g. CPU or GPU). Since every single component has a latency time, the overall latency time for a computation is mainly the combination of hardware components required to get the data from one location to another in modern computing infrastructures.
[0037] [Diagram 2]A block diagram showing a schematic of a shared memory multiprocessor as a class of parallel machines that provides the basis for the shared address programming model as the top-level parallel programming. In a shared memory multiprocessor architecture, the assumption is made that in this computer system, the processors and a set of I / O controllers can access a collection of memory modules through some hardware interconnect. Memory capacity is increased by adding memory modules, and I / O capacity is increased by adding devices to the I / O controllers or by adding additional I / O controllers. Processing power can be increased by waiting for faster processors to become available or by adding more processors. All resources are organized around a central memory bus. Through the bus access mechanism, any processor can access any physical address in the system. Since all processors are assumed to be, or are actually, equidistant from all memory locations, the access time or latency of all processors is the same to the memory locations. This is called a symmetric multiprocessor.
[0038] [Diagram 3] A block diagram is shown which shows a schematic of UMA architecture. In UMA architecture, all processors share the physical memory equally. All processors have equal access time for all memory words. Each processor may have a private cache memory. They follow the same rules for peripheral devices. If all processors have equal access to all peripheral devices, the system is called a symmetric multiprocessor. If only one or few processors have access to peripheral devices, the system is called an asymmetric multiprocessor.
[0039] [Figure 4]A block diagram showing a schematic of a NUMA multiprocessor architecture is shown. In a NUMA multiprocessor architecture, the access time varies with the location of the memory word. The shared memory is physically distributed among all the processors, called local memory. The collection of all the local memories forms a global address space that can be accessed by all the processors.
[0040] [Diagram 5] A block diagram showing the schematic of the COMA multiprocessor architecture is shown. The COMA multiprocessor architecture is a special case of the NUMA multiprocessor architecture, in which all the distributed main memory is transformed into cache memory.
[0041] [Figure 6] As a schematic example, we present a block diagram that illustrates the different scopes of the units "basic block" (or "block unit") and "computation block node". The "computation block node" used in this application is essentially different from the block unit used in the SOTA (State-Of-The-Art) compiler.
[0042] [Figure 7] A block diagram is shown that illustrates a simplified but more realistic example, where the method of the invention exploits the fact that logarithmic instructions (as an example in modern CPUs: FYL2X) require much longer than floating-point addition and / or multiplication instructions (e.g. FADD, FMUL). The method of the invention adds the statements "x:=a+b" and "y:=a*b" to two different computation block nodes (cbn), since both are based on the same information "a" and "b". The "log2(x)" statement is appended to "x=a+b" since it uses the information "x". The information "y" is forwarded, so that it is available to both independent computation chains.
[0043] [Figure 8] Here is an example of how the code from Figure 7 is divided into machine code that can be executed on two computation units synchronized by any form of IPC (it should be noted that IPC means communication between two computation units rather than "InterProcessCommunication"). This matrix shows the required clock cycles required to compute the different machine instructions. As an example, how to get the value of the time to compute the instruction. In this form, in Figure 8 we can see that unit 2 can compute the long execution instruction of the statement "log2(x)" and unit 1 processes the loop by incrementing a ("a=a+1") (see Figure 7).
[0044] [Figure 9]A block diagram is shown which shows in a schematic way the method of the invention forming two matrices, referred to herein as "numerical matrices", from the flow graph of computation block nodes (it should be noted that "numerical" may also refer in certain cases to text data, since all data on the IT infrastructure can be understood at a low level as numerical, e.g. binary, as a variant). One contains the instruction chain, the other contains the possible transfer characteristics (from and to other computation block nodes). Thus, the code extracted from these matrices always forms the pattern "compute->communicate", as can be seen in detail in the following description. Obviously, if the code is mapped to one unit, the communication part disappears (computation (= chain of instructions) is summarized) and the method of the invention is reduced to a technique using "basic blocks", which can be processed by the SOTA compiler, respectively. Moreover, the matrices, which represent access information in a much more scalable form than graphs, respectively, show the well-formed nature of the control flow graph. The definition of computation block node (cbn) also shows the general nature of the matrix of the invention. In one matrix, each row has an independent flow of instructions (=computation), in the other matrix, the necessary transfers (=communications) to other CBNs. As a consequence, this guarantees that a) no further information is needed to compute all instructions in a computation block node, b) the information used is not changed anywhere else during the same computation step in the whole code, and c) only information can be transferred during this time step without being affected by the computation.
[0045] [Figure 10]Figure 10 shows a block diagram that shows,schematically how each row of the compute and forward matrices together,represents a combination of a chain of instructions and,a communication entry for one unit, depending on the level of,implementation (e.g. bare assembly, thread, process, compute,node, etc.).,Clearly, empty compute cells or unused communication entries (empty,cells in the compute and / or forward matrices) will disappear,as well as the origin and destination communication links,as seen in Figure 10.
[0046] [Figure 11] Fig. 11 shows a block diagram that shows one possible optimization by the system of the present invention, which provides an ideal hardware optimized for technical computing and processing problems. The most obvious way for such optimization is by constructing different combinations of rows from the computation and transfer matrix, each combination being one possible new parallel / concurrent code (since transfers on the same unit disappear and computations are bundled), and then evaluating its properties on the target infrastructure, as shown by Fig. 11 for a simple example.
[0047] [Figure 12] A block diagram is shown that illustrates a schematic of an exemplary multi-processor system 2 having two or more processing units 21 (multiple processors) each sharing a main memory 22 / 221 / 222 / 223 and peripheral devices 25 for simultaneously processing program code 3.
[0048] [Figure 13]1 shows a block diagram illustrating generally an exemplary embodiment variant according to the present invention, where source code 31 is input code to an auto-parallelizing compiler system 1 comprising a parser 11, a computational block chain module 14, a matrix builder 15, an optimizer 16 and a code generator 17, and where the auto-parallelizing compiler system 1 generates parallelized and optimized target code 32 as machine or assembler code for execution by a parallel processing system 2 having multiple processing units 21.
[0049] [Figure 14] 1 shows a block diagram that illustrates, in schematic form, an exemplary compiler system as known from prior art systems, in comparison with the inventive auto-parallelizing compiler system 1.
[0050] [Figure 15]1 shows block diagrams each showing schematic representations of exemplary basic elements for a tree structure and / or computation graph. A computation is reduced to one type of statement: result = param1 operation param2. Numeric operands are +, -, *, / -> and only these in numerical terms. In the source code 31, which is a high-level language, variables are assigned values in the form variablename = value. This is a link to a "virtual" location (in the variable name) to save the value. The source code 31 is usually sequential - that is the programmer's mindset. According to the invention, the code is a different approach to looking at the code as savelocation1 = savelocation2 operation savelocation3. Based on this, basic graph or tree structure elements are now introduced. For example, two input data nodes 33 / 331 / 3311 (datanodein1 and datanodein2), an operation node 332, and an output data node 33 / 332 / 3312 (datanodeout). They are connected by directed edges as seen in Fig. 15. A node 33, as used herein, is the basic unit of a data structure, such as a sequence of linked processor instructions (computation block node 333 / operation node 332) or a tree data structure as a data entry or data output structure (datanodein / datanodeout). A node 33 contains data and can also be linked to other nodes 33. Links between nodes 33 are often given by pointers (arrows) in the graph of Fig. 15.
[0051] [Figure 16]A block diagram is shown that shows a schematic representation of exemplary basic elements of source code. In the lexer and parser 11, operations from the high-level language are reduced to basic graph or tree structure elements. It is thus possible to use (i) arithmetic operations: +, -, *, / , (ii) logical expressions such as conditions: a==b or a>=b or similar expressions resulting in true or false output, (iii) variable assignments name=value, (iv) flow control operations as branching expressions such as (1) if(condition){block 1}, and (2) loop(condition: increment variable: max value of increment variable){block 1}. Here, there is an analogy with basic operations in assembler code, where the instruction set in the processor 21 usually has three parameters. The statement and the basic graph elements describe an operation (operation node 332), followed by two reads (datanodein1 3311 and datanodein2 3312) and one write access (datanodeout 3312). Multiple assignments to a variable name create versions of this variable, creating different data nodes 331.
[0052] [Figure 17] As FIG. 17 shows, each version builds a new data node 331.
[0053] [Figure 18]A block diagram is shown which shows a schematic representation of an exemplary simple sequence of operations 332 in a tree structure or graph. A sequence of program codes 31 can be added to the tree structure or graph, statement by statement. Complex expressions are reduced (by the need to reduce calculations on binary computing systems to such a form) to a series of expressions of the type result=param1 operand param2. If an operation node 332 accesses a data node 331 to which another operation node 332 is writing, the construction of the sequence begins by connecting these operation nodes 332 by a directed edge (next edge). Writes and reads are mapped in this way in the tree structure or graph for all data nodes 331 (see FIG. 19). As a rule to simplify the following steps, only one write is allowed for a data node 331 (this helps to simplify later operations on the graph). If this is not the case, a new operation 332 is introduced to model several writes to a data node 331. [Figure 19] [See explanation of Figure 18]
[0054] [Figure 20]A block diagram is shown that shows a schematic example of how Δtreadwrite 35 / 351 describes the time between a write access and a read access to a data node 331. With this interpretation, the latency time Δtreadwrite 351 can be considered as a number that represents the time that the data node 331 must be transferred to another process in the multiprocessing system 2. The interpretation of Δtreadwrite 351 depends on the level of the hardware setup. It is the technical and physical time between the write access 3311 (I / O process, network card, or similar) and the read access. Δtreadwrite 351 is a unit given by the hardware infrastructure. Δtreadwrite 351 can be composed of different Δt: Δttotal=Δt1+Δt2+Δt3+Δtn (for example, L1-cache access-memory access). For one dependency (operation node read from one dependent datanodein), Δtreadwrite 351 is given and must be very short respectively. Note that other problems arise when there are two dependencies (see below).
[0055] [Figure 21]3 shows block diagrams that show schematic example dependencies for basic tree structure elements and graph elements, respectively. As a variant, it becomes simple if a rule is set that only one computation node 332 can write to one data node 331. Then, there are different but finite situations for adding a new computation node 332 in the sequence: (i) 0 dependency: placement is independent, (ii) 1 dependency: placement is after dependent computation node 332, (iii) 2 dependency: -unclear->Δtreadwrite 351 because both data nodes 331 (in1 3311 and in2 3312) depend on the history of their previous computation nodes 332; - at this point, in order to find Δtreadwrite 351, the history of all previous Δtreadwrite 351 of datanodein1 3311 and datanodein2 3311 must be known. This history of Δtreadwrite 351 must be placed in contrast to Δtlatency 352, which is the time required to distribute information or data points to different locations throughout the system 2.
[0056] [Figure 22]23 shows a block diagram that shows an example of how data / information can be considered. For example, it is possible to transfer data from one process to another, or in case of threading, for example, to prevent simultaneous writes. The movement of data can be seen as, for example, an elementary operation (operation K), and the transfer is of length Δtlatency 35. It is important to note that the interpretation of Δtlatency 35 strongly depends on the hardware system and can be seen as the time to transfer data / information through the system. Furthermore, during this time, computation nodes that are not affected by the transfer can be computed if the system allows it, for example in the case of Direct Memory Access (DMA) network transfers. Alternatively, in case of a multi-threaded system 2 (two or more processes have access to the same data, but race conditions are critical), this Δtlatency 35 can be seen as a blocking time during which one process reads / writes the corresponding data and other processes are not allowed. It can be seen that a send (see FIG. 23) transfers information or data through the system 2 from a source to a target location. Transmission is not yet a clearly defined pattern (e.g. send and receive) and it is not given whether it will be required or not in the final code - this can only be done after optimization to the corresponding platform / hardware infrastructure.
[0057] [Diagram 23]1 shows a block diagram that illustrates a schematic of exemplary computation block nodes 332 / CB1, CB2, ..., CBx. A computation block node 333 can be introduced as follows: (i) a computation block node 333 consists of connected computation nodes 332, (ii) computation nodes 332 are arranged in one chain 34, (iii) a computation block node 333 can be connected to a next computation block node 333 or a control flow node 334, e.g., a branch node 3341, (iv) Δtcomputation 352 is the time to compute all computation nodes 332 (representing instructions) in a computation block node 333, and (v) it has a minimum length Δtlatency 35. During this time, data can be transferred within the computational infrastructure and (vii) to describe the communication, the following transmissions are introduced: (a) source: position within computational block node 333 (start, end), computational block node 333 id, computation node id, datanodein1 / datanodein2 3311, or output data node 3312; (b) target: position within computational block node 333 (start, end), computation block node 333 id, computation node 332 id, datanodein1 / datanodein2 3311, or output data node 3312. Sending must not explicitly send and receive information -> a computation block node 333 can start sending and / or receiving data information at the beginning and end of the block 333 (before and after all computation nodes 332 have been computed) -> this model is typically applied in explicit non-blocking send and receive communication methods (e.g., Message Passing Interface (MPI)) that may be able to send and receive independently of the computation (e.g., by a Direct Memory Access (DMA) network card). However, explicitly introducing a computation node with send and receive operations blocks send and receive operations in the same way (and therefore has its own Δtreadwrite 351).For example, in a multithreaded approach, if communication is not allowed (threads share the same data, but time-dependence can only be guaranteed by locking mechanisms), sends and writes can be interpreted as lock sets and releases, and Δtlatency 351 must be interpreted accordingly.
[0058] [Figure 24] 1 shows a block diagram that illustrates a schematic of an exemplary processing of program flow control elements in a tree structure or graph. The condition node 335 is realized on the basis that (i) each condition in an if condition or loop condition generates a new "level", e.g., cond:0->cond:1, (ii) each condition knows how many levels are given between its own level and the base level of the program (condition 0) (e.g., by cond:0->cond:1->cond:2; this means that two conditions have been computed to get to this position in the code), and (iii) the condition node 335 is connected by a directed edge to branch nodes 334 / 3341 with the same condition. Furthermore, branch nodes 3341 are introduced: (i) each condition has at least one branch node 3341, (ii) each branch node 3341 is connected to the computation block node 333 that belongs to this branch (the code in the section of the if clause). A condition arises from an operation node 332 that has a (logical) expression with a "conditional" result, such as true or false. The introduction of a condition is one way of making it possible to determine to which computation block node a new operation can be assigned in the "fastest" way (the computation block node must be in the same condition level group, but can be in another branch node). Note that here a branch node does not correspond to a block in a branch in the definition of "basic block".
[0059] [Diagram 25]A block diagram is shown that shows a schematic of an exemplary processing of an "if statement". The if statement results in a branching of the code under one condition (if(condition){code A}). As a result of the logical condition expression that needs to be forwarded in the system -> a subsequent empty calculation block node 333 is added after the calculation block node 333 of the operation node 332 with the logical command -> this is the time needed to forward the result of the comparison (meaning the condition) to other processes -> it can be used to calculate other things - this calculation block node 333 can be marked as branch-send type (CB2). After this, two directed edges of CB2 333 are added to a new branch node with a new calculation block node for the code in branch 1 and to a so-called hidden branch node (see below) that is used if the condition is not met (however, it is not an else clause, but it could be an else clause). The result of the condition expression must be forwarded to the subsequent branches -> a send package is added to the end of CB1. Edges from the branch nodes and hidden branch nodes 3342 can be set only after parsing all statements in the branch code section, and thus only after the closing statement for the if condition (see below).
[0060] [Figure 26] A block diagram is shown that shows a schematic of an exemplary processing of nested "if statements". Since each statement in the code is added to the graph in the sequence in which it is read, the control edge connection between a branch node 3341 and subsequent branch nodes 3341 can be made only after the closing mark of the if statement -> nested if statement is reached. The correct condition node 335 with the correct branch node 3342 can be found by calculating the last condition level (which is reduced by closing the if statement) and traversing the graph to the correct node 3342.
[0061] [Figure 27]A block diagram is shown that shows a schematic of an exemplary processing of a "loop statement". It should be noted that according to the branching shown above for the if statement, the loop statement can be handled with some adaptation. The loop statement results in a branch of the code, as long as a certain condition is met, in this example by incrementing a variable until a certain value is reached (loop(condition, i, 2) {code A}). This is the same as in the if statement -> add CB4 333 for a transmission with type loopbranch-transmission. The increment operation node 332 is added to the variable defined in the expression. The result of the condition expression must be transferred to the subsequent branches -> a transmission package is added to the end of CB3 333. The branch node 3341 (branch 2) after the loop branch node 3343 (branch 1) and the edge from the computation block node 333 with type transmission can only be set after the closing statement for the loop statement (see below) is reached.
[0062] [Figure 28]1 shows a block diagram that shows a schematic of an exemplary processing of nested loop statements. Since each statement in the code is added to the graph in the sequence in which it is read, the control edge connection between the computation block node 333 with type loopbranch-transmission and the subsequent branch node 3341 can be made only after the closing mark for the loop statement -> nested loop statement is reached. This can be done by finding the lower condition level based on the last added statement and finding the correct loopbranch-transmission node 333. The computation block node 333 with the compare and increment operation node 332 as well as the computation block node 333 with type loopbranch-transmission must be reconnected to the branch node 3341 of the previous loop. The branch node connections after the loop must be updated as well. branch0 | cond0 for (i=0; cond1; i++) {branch1 | cond1 for (j=0; cond2; j++) {branch2 | cond1 -> cond2 for (k=0; cond3; k++) {branch3 | cond1 -> cond2 -> cond3} branch4 | cond1 -> cond2} branch5 | cond1}branch6 | cond0 Regarding variable assignments, for a=0 and a=b, where assignments are also operation nodes but have only one data node (in1) instead of two (in1 and in2), (i) a=0(-->variablename=number): -0 dependency and 1 dependency are possible, (ii) a=b: -1 dependency must be present since a copy operation node must be added since it explicitly copies data from a data node to another data node with the correct version number, (iii) redefinition of a variable name results in a new version of the variable and a new data node 331. Additionally, array variables and pointer variables can be handled similarly.
[0063] [Figure 29] A block diagram is shown that shows an example process of branching and variable versions in a schematic manner. Since the compiler looks at the code with an explicit data-centric view, non-explicit changes to data under conditions must also be incorporated. In Fig. 30, c is computed in branch 0 under condition cond0. c is redefined in branch 1 under condition cond0->cond1. If the compare function in CB1 is false -> the program flow proceeds to hidden branch node 3342 hbranch1 -> copy the value from the data node at position mmapid:12 (the data node for c=1+5) to the new location as if CB3 were true. This is important and is the reason why there are hidden branch nodes that are not (only) else statements. The reason we have to do this is because the programmer programs the data flow for true conditions but not for false conditions (where nothing is expected to happen), but this is not the case in the data dependency graph. That is because while parsing the code, this c=8 in branch 1 does a new version for the variable. Also, this needs to be done for all variable assignments under conditions.
[0064] [Diagram 30] A block diagram is shown which shows a schematic example of a branching and transmission process. In this situation, for example, if a 1-dependency or a 2-dependency occurs under different conditions, -> a different branch node means -> a branch node indicating that the information transmission must have occurred before this. The same is true for the case of a 2-dependency with the same conditions but different branch nodes. However, there are other possibilities to solve this problem. It means that a data transfer to this new computation block node in the corresponding branch node is necessary. Thus, the branch nodes can also be processed in other ways.
[0065] [Diagram 31]1 shows a block diagram that shows a schematic of an exemplary handling of arrays and dynamic arrays. Arrays can be viewed as a basevariable name and offset (e.g., a pointer variable and offset in C) in a sequence of data with the same length. Thus, they can be handled by a two-step operation: find the address / location of the base variable, get the offset, then go to the offset found by the variable definition (type / length, e.g., u_int8<->u_int32). Note that in this special case, the operation is not in fact a direct CPU-based primitive, since it is performed by the processor itself, such as by first looking in the L1-Ln caches.
[0066] [Diagram 32] A block diagram is shown that shows a schematic of an exemplary split and merge situation for a computation block node 333. In case of 2-dependency, a merge situation of two computation blocks 333 must exist. If two computation nodes read from the same data node -> this creates a split situation of two computation block nodes 333. For both cases, splitting or (re)merging two computation chains 34 (chains of computation block nodes) is allowed by introducing a new computation block node 333 (e.g. merge-send or split-send) to send and forward data / information through the system. Send information is added to the newly created send type computation block node.
[0067] [Diagram 33]A block diagram is shown which shows a schematic example of how to close a transmission gap in a graph flow. This step can only occur after all statements from the source code have been added to the graph. Then, additional computation block nodes 333 of type b2b-transmission are added to branch nodes that are not connected by computation block nodes 333 of type-transmission. This is necessary to propagate the branch change through the system. In a later step, these transmission blocks have to be set to the latest block number (computation step) so that the information of the branch change can be propagated through the system 2.
[0068] [Diagram 34]A block diagram is shown that shows a schematic example of the introduction of a time perspective into a tree structure or graph. From the connected computation nodes 332 and the flow-controlled computation block nodes 333, the tree structure or graph can be seen as a chain 34 of computation blocks 333 with Δtcomputation 352. In every computation block node, the computation nodes are grouped together, so Δtreadwrite 351 is "optimal" (in the sense of locality; therefore, copying data is slower than computing data). These sequential chains in the computation block nodes should not be disturbed, since they represent sequential parts of the code. To make the code parallel, the computation block nodes need to be balanced by time, so that both input chains (and therefore chains of operations) of the computation block node are balanced against each other -> so in a fusion situation, both input chains must have the same runtime. Otherwise, the overall computation will stall when the other chain has to be computed. The most concurrent / optimized parallel code is achieved if in all fusion situations of the computation block node 333, the input chains (CB1 and CB2) have the same runtime and starting point. In the split situation, it is possible to start different chains at the same time and thus create parallel computations. It must be noted that a series of "+" and under some conditions "*" may occur in the computation block node, which may again be split into parallel computations. However, this is easy to detect during the optimization phase. Each computation block node requires Δtcompute to compute all the computation nodes 332 within it. Now, since each computation block node 333 can be seen as being of minimum Δtlatency 35 length (because it was introduced as such in the previous steps), it is possible to try to balance this Δtcompute with the available Δtlatency 35 of the system. By assuming that all computation block nodes 333 must be at least Δtlatency 35 long, it is possible to transfer data in the system 2.Finite block numbers can be introduced for each computation block node. These represent the required sequence of computation blocks 333 in the sequential chain 34 of computation nodes 332. This produces a distinct and well-defined grouped graph, where every group of operations has a unique number and can be transformed into matrices according to these numbers. These matrices can then be used to optimize / map to a given or ideal infrastructure, for example with real-world times.
[0069] Regarding assigning a block graph or tree structure to runtime numbers, the numbering can be implemented as a recursive function that for example parses the graph of computation block nodes and their edges, branch node by branch node. At each branch node 3341, the computation block node 333 is started to be numbered. Then, step recursively through the computation block node chain 341 of the branch node 3341. After a branch node, the maximum block number is known, given the computation block node in the branch node. This is used for the next branch node. The rule is that a computation block node 333 is stepped to a computation block node 333. If there is only one previous computation block or no previous computation block -> set its block number to the actual block number and increase the block number (for the next computation block node). If there are two preceding computation block nodes -> this is a fusion situation -> this is the first visit to this node -> append to the local list of block numbers. Else, if this is the second visit -> use the highest block number (actual from the function call or saved in the list of nodes). If there is one subsequent computation block node -> call the function for the computation block node (recursive approach). If there are two subsequent computation block nodes -> split situation -> call both computation block node numbering functions in a recursive manner. If not (so there is no next computation block node) -> give the next branch node, if there is, end the actual recursive call. By appropriately adjusting the block numbers in the branch node transmissions, the call graph is numbered in the form of a discrete-time graph based on the computation block node connections, and each computation block node has a finite number of computations that must be computed during the same time period.
[0070] [Diagram 35]FIG. 1 is a block diagram showing a schematic example of splitting the graph into a single / "transmission coupled" call chain. At this point, it is possible to split the call graph or tree structure into a single chain 34 of computation block nodes 333, which leads to a single series of operation nodes with corresponding transmission information for each computation block node 333. Each computation block node has a unique number. The required transmissions / communications are stored in the computation block nodes.
[0071] [Diagram 36] Figures 36 and 37 show block diagrams that show an example possible process from sending to directed communication. The direction of communication can depend on the number of the computation block nodes 333, as the examples of Figures 36 and 37 show. It must be ensured that all sending computation block nodes are at the highest +1 block number in one branch node (to ensure that the program flow information has enough time to propagate to all computation units / processes). Send the command in the correct computation block node by explicitly forwarding the sending package with source and target information. The receiving part can be done only after an optimization step. Signal sending, like branch or loop sending, must be implemented for all connected next computation block nodes 333 in the corresponding branch. [Figure 37] (See the description of Figure 36.)
[0072] [Figure 38]A block diagram is shown that shows a schematic of an exemplary process from a graph to a matrix. At this point, the graph consists of a branch node with a single chain 34 of computation block nodes 333. Each computation block node 333 knows what information (data and / or signals) must be transferred to which other computation block nodes 333. Each computation block node 333 has a discrete number that represents the order in which each block must be calculated over the program's time. Thus, the graph can be interpreted as a matrix where each cell is a computation block node, each row is a single unit, each column is a discrete block number (sequence number), and each cell (computation block node) knows what information it must send to other computation block nodes 333 (cell entries). Each cell knows which operations (chain of computation nodes) must be calculated in which order and between their block numbers, i.e. they must be independent and in the correct order -> it has a clear nearest write and read connection. Each computation node knows the correct global data needed for reading (in1 and in2) and writing (out). Here, the matrices can be optimized, e.g. numerically, to the target hardware infrastructure (CPU, e.g. CPU with explicit network transfers via MPI, GPU (memory transfers then vectorized operations), etc.).
[0073] [Figure 39]FIG. 1 shows a block diagram that illustrates example program paths and matrices. The z-dimension is the code perspective for each path through the code (each condition change introduces a different path through the code). The third dimension (or z-dimension) is the condition switch. That is, each set of transfer and computation matrices is one way through the code under one condition. In this perspective, each path through the code can be represented by two matrices with rows for independent chains and columns (x- and y-axes) for block numbers. Branches evolve by switching to a corresponding set of z-dimensional matrices (paths through the code), and each process node needs to know all possible chains 34 for itself. In this description, each condition change is captured by one set of computation and transfer matrices, but the matrices can also be considered as tensors, where, for example, the computation and transfer tensors can contain in their rows, respectively independent operations and corresponding transfers, in each column, discrete computation steps, and in their third dimension, different conditions. Alternatively, everything can be packed into one tensor that combines the calculations and transfers for each block number, unit, and condition, etc.
[0074] Regarding optimization, there are many automatic optimization techniques that can be applied today: combining rows to reduce parallelism, moving a single operation chain to the earliest point (the send command in the cell is like a barrier), reducing communication by best combination of rows, etc. A compiler system1 can be used to get an estimate about the runtime of the cell entry, or a method like that of Agne Fog at Technical University in Denmark can be used to extract the CPU and cache interaction, or a table from the CPU manufacturer can be used, or it can be compiled with openCL, etc. For example, in a view having one tensor for computation and one tensor for transfer, different parallel code versions can be obtained by combining the same rows in each tensor, thus generating a new combination of "computation and transfer" for each block number. This leads to a reduction in the number of parallel / concurrent units. By combining rows, it is possible to reduce or group transfers (in the final code communication) and computations can be grouped together. If there is no send() or read() at the beginning of the block, the operation nodes in the block can be moved to the previous block. Each cell knows how much data (memory, etc.) it needs. Different given Δt on the target platform latency35 can be used to make decisions about which sequential parts (cell entries in the computation matrix) have to be computed on which hardware units. The communication type in the infrastructure can be implemented as needed, from asynchronous or non-blocking, to explicit blocking of send and receive commands in MPI frameworks, preventing race conditions by ensuring that the correct barriers are set and released, bulk copy transfers in GPU infrastructures, etc. Thus, the choice of optimization technique can be easily made by choosing the appropriate prior art optimization technique, for example by using the SOTA compiler for each code per unit (computation and transfer). The optimization technique used introduces a much broader perspective to obtain a more "perfectly parallel" code than the others (perfect in the sense of linear dependence of speedup on the number of processes with a slope of 1). Thus, it is possible to numerically optimize the matrix to a new parallelized optimized / concurrent code for the target hardware. This can be done automatically and is therefore a big step compared to other methods. Since this is done by software, the software can now parallelize its own code, which is novel and brings new possibilities. For example, adaptive models in Machine Learning (ML) or Artificial Intelligence (AI) applications, meshes in Computational Fluid Dynamics (CFD) calculations, or particle sources in vortex methods, or the combination of different modeling methods with different spatial and temporal resolutions (Finite Volume Method (FVM) with Agent-Based Models and Statistical Models).
[0075] [Diagram 40]A block diagram is shown that shows a schematic of an exemplary code extraction. Code can be extracted from the matrix by a wide range of implementations: extracting assembler code directly, writing code files and compiling them with state-of-the-art compilers, explicitly implementing send and receive methods for example for a message passing interface (MPI). Computation matrix cells have code=statements in the form of operations, each column has to be computed during the "same" time interval and each row is a separate process. Transfer matrix cells know what information has to be shared with which other cells.
[0076] [Diagram 41] A block diagram is shown that illustrates a schematic of a computation block node (CB) with operations and corresponding data nodes (similar to tokens in Table 1) for a doubly recursive call of a function.
[0077] [Diagram 42] 1 shows a block diagram that illustrates a schematic of a recursive call of a function resulting in additional forwarding. A forwarding to a location within a computation block node (either the start or end of a computation block node) is shown.
[0078] [Diagram 43] Figures 43, 44 and 45 show block diagrams that show the steps of numbering the computation block nodes depending on the call location in the code. The numbering results in a pseudo graph as shown in Figure 43, giving computation and transfer matrices for each path number as shown in Figures 44 and 45. In terms of considering the matrices as m x n objects, a set of computation and transfer matrices is created for each path number, giving two computation matrices and two transfer matrices. The switching of paths (or conditions) can be seen in Figure 46, where the "true" / "false" signaling is shown. [Diagram 44] (See the description of Figure 43.) [Diagram 45] (See the description of Figure 43.)
[0079] [Figure 46] A block diagram is shown that shows the schematic result of combining the start and end communication cells in the transfer matrix, removing the empty cells in the computation matrix (for each path) and transferring them back to different code segments. Based on this code segment, the code can be generated directly (as a compiler) or transferred back to the code and then machine code can be generated using the SOTA compiler (as a transpiler), for example using unit-specific optimization techniques implemented in the SOTA compiler that mostly target single unit compilation.
[0080] [Figure 47] A block diagram showing,schematically how function calls in parsed code can be handled in the presented,way.,After parsing a function definition, the resulting computational,block nodes (or matrix entries) can be "copied" / used at the,location where the function is called.
[0081] [Figure 48] FIG. 4 shows a block diagram that shows in schematic form how the method of the present invention maps and / or optimizes code in a more parallel solution than the input code. As shown to optimize the code, it is a combination between rows and the graph is shown with the combination of CBNs in the branch node marked as branch2b (see FIG. 43) and combine them (see FIG. 48). Invoking a function, in the present method, is a matter of placing the computation block nodes in the correct position in the matrix to apply the corresponding transfers, respectively, as shown in FIG. 47. With this in mind, the recursive invocation of a function can be seen in the present method as a transfer of the result variables and function parameters in the return statement as well as a "read" and a "write" (see FIG. 48).
[0082] [Figure 49]Following the example of a function call to calculate the Fibonacci number for an input of 4 (fib(4)), we show a block diagram that shows step by step how the reduction in additional computational block nodes and transfers (since everything is on one computational chain) results in a more optimized source code.
[0083] [Figure 50] A block diagram showing step 4 is shown showing cbn for the call with "n=4". Note that since recursive calls can be detected very easily in the code, it is easy to not implement them in their full dimensions in the final application, as shown by some simplifications in Figures 49 to 53. Here, the depth of this chain depends directly on the number n in fib(n).
[0084] [Figure 51] A block diagram is shown that outlines step 5 showing the transfer that occurs (transporting the information of the last ``write'' to the data node to where the ``read'' of the data node occurs and storing this information in the corresponding computation block node).
[0085] [Figure 52] As can be seen in Figure 52, a block diagram is shown that illustrates in a simplified manner that transfers disappear since all computations are on one chain.
[0086] [Figure 53] A block diagram is shown which illustrates diagrammatically how the individual steps, when resolved, result in a program of the form shown in FIG.
[0087] [Figure 54] A block diagram is shown which illustrates, in a simplified manner, that a function call, and therefore also a recursive call, can be interpreted like an array operation applying the method of the present invention, since it transfers information (function parameters param[i]) to the corresponding cbn in the branch node of the function declaration and then transfers the return value(s) back to a[i].
[0088] [Figure 55] Figures 55 and 56 show a block diagram that shows a schematic of an implementation in python for solving the 2D heat equation. Each entry in the array is a data node. In the method of the present invention, a read from an array index is an operation node with a data node for the index and the base address of the array. An array operation can be viewed as an operation node with a corresponding data node (see Figure 56). [Figure 56] [See explanation of Figure 55]
[0089] [Figure 57] For an example loop array with array operation a[i+Δiw]=a[i], a block diagram is shown that shows a schematic of the corresponding computational block nodes.
[0090] [Figure 58] FIG. 57 shows a block diagram illustrating in schematic form the initial block (FIG. 55) which is detailed in the graph as shown in FIG. 58.
[0091] [Figure 59] A block diagram is presented that illustrates, in schematic form, how the model approach is derived by using 1D array notation and applying the basic rules of the method.
[0092] [Figure 60] FIG. 60 shows a block diagram that illustrates, in a simplified manner, how a statement such as an array operation in the j-loop (FIG. 55) results in five computation block nodes that represent a "read" operation of the array, then a computation block node that computes the arithmetic answer, then a computation block node that has a write to the array at position [k+1][i][j]. This representation is a schematic perspective to more clearly show how the method takes such array operations into account, resulting in the situation shown in FIG.
[0093] [Figure 61] A block diagram is shown that shows a schematic of a scheme that can be derived from the fact that each loop thus creates a new computation block node with a "read" or "write" operation node and a corresponding forwarding. Thus, each loop iteration introduces computation block nodes, forming a chain of read, compute, and write steps for one unit (the latter unit that computes the operation). If a "write" and "read" dependency occurs between these computation block nodes, a forwarding is introduced (see Fig. 62).
[0094] [Figure 62] FIG. 1 shows a block diagram that illustrates, in a simplified manner, how offsets in the indexes of respective “read” and “write” operations can result in transfers between computational block nodes that contain the respective “read” and “write” operation nodes.
[0095] [Figure 63] 1 shows a block diagram that illustrates, in a simplified manner, the conclusions that can be derived from the dependencies in the "read" and "write" of data nodes of an array indicated by an index in a loop.
[0096] [Figure 64] A block diagram is shown that illustrates the schematic dependency between the “read” and “write” operations in the “read” and “write” computational block nodes and the size of the loop (in this case, the mesh size) resulting from the 2D heat equation using a central difference scheme with three nested loop implementations as shown in Figure 55.
[0097] [Figure 65] We show a block diagram that shows, for one nested loop, how for each loop nest, consequences for "forwarding" in the proposed model and rules for dealing with gaps can be derived. Gaps arise from looping not over the complete array / mesh / dimension, denoted by j0 and Bi in the loop definition.
[0098] [Figure 66] A block diagram is shown which shows, in schematic form, how a transfer pattern of the discretized equations can be derived incorporating the gaps caused by looping over a subset of the arrays, and how a model for transfer can be constructed depending on the gap size, the size of the loop (=mesh in this case) represented by the computational block node of the method of the invention. The method can also be applied for each successive iteration for each loop and adding the corresponding elements to the graph / tree structure, although this may introduce some performance issues for applying the method to large problems (e.g. large meshes).
[0099] [Figure 67] Using a very small example of nX=5 and nY=4, we show a block diagram that shows roughly how this results in a set of computation and transfer matrices as in Figure 67. In the transfer matrices, the arrows "->" indicate getting information and "<-" indicate sending data to another computation block node.
[0100] [Figure 68] A block diagram is shown that shows in schematic form the transfers that occur between units. Each unit is represented by a row in the transfer matrix, and the computations are represented in the computation matrix. In this illustration, the entries of the computation and transfer matrices are shown together. This is a combination of the computation and transfer matrices.
[0101] [Figure 69]At this point it is important to note that this does not mean that they must be sent and received, but can also be shared, for example, by a shared memory segment and guarded by a barrier or lock or disappearance (as they are shared by a cache and therefore handled by the CPU). It depends on the transfer / communication mechanism used. This can be transferred to the model and result in code of the form shown in Figure 69 respectively. The method is based on keeping all the dependencies of "write" -> "read", and a calculation of the form a[i1] = a[i2] first "reads" the information in a[i2] and then "writes" the information in a[i1], so the operation of the first "read" (p0-p1) is introduced.
[0102] [Figure 70] Figure 1 shows a block diagram that shows, in schematic form, how transfers can be used to create a time model. In grey are meta-values (e.g. known from variable type definitions or similar) that can also be used to map / optimize matrices to a given hardware infrastructure. These values are not used in this example.
[0103] [Figure 71] FIG. 1 shows a block diagram that illustrates in schematic form a simple model that is fully decomposed to show the possible optimization steps.
[0104] [Figure 72] A block diagram is shown that illustrates in schematic form the behavior that can be derived for a grid of nX=2048 and nY=1024 for two different sets of cpuPower and networkLatency, with Δt=combined cbn*cpuPower+number of transfers*networkLatency.
[0105] [Figure 73]A block diagram is shown which shows, in a simplified manner, how a Fibonacci source can be implemented using loops. For the example using the 2D heat equation, this leads to the result in Figure 73.
[0106] [Figure 74] 7 shows a block diagram that shows in schematic form the application of the method of the present invention to the technical problem of pointer disambiguation, which obtains pointers as information and therefore resolves the disambiguation caused by passing function parameters as pointers, which can be shown to be resolved in the step of forward elimination as shown by Fig. 74. It should be noted that the labels "Intel P5 Infiniband" and "Intel Haswell L3-cache" can generally represent configurations 1 and 2. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0107] definition (i) "Computation block node" The definition of the term "computation block node" as a grouping of instructions and related communication / transmission data to other computation block nodes is important to this application. The term "computation block node" as used herein does not have a commonly recognized meaning, but is distinct from similar terms used in the state of the art.
[0108] The well-known basic blocks (see for example Proceedings of a symposium on Compiler optimization; July 1970, pp. 1-19, https: / / doi.org / 10.1145 / 800028.808479)) are the central definition in classical control flow graphs (CFGs). Simplified, they group statements that have no jumps or jump targets inside them. Thus, for a given input, they can execute their operation without interruption until the end, respectively the output. This is a fundamental concept in compilers today. The definition of a basic block historically targeted a single computational unit and is very well established. Optimization methods exist for a wide range of problems, showing how they solve different technical problems. However, when looking at code with the aim of splitting statements into different subordinate units (connected, for example, by a shared cache, through a bus or network, etc.), this definition lacks granularity and the classical scope prevents a broader perspective. By scoping blocks based on any unique information given in the code (looking at information as bit patterns), and combining this scope with the associated time to compute and transfer information in the system, we create a different, yet well-physically based, view of a given code. This alternative scoping allows new options and solves several well-known technical problems in today's SOTA compilers (see examples below for PDE, Fibonacci, or pointer disambiguation).
[0109] The term "computation block node" as used in this application is based on the correlation of transfer and computation time for a given set of different and thus non-applicable units of importance, i.e. statements, in the SOTA compiler. These newly defined "computation block nodes" group together instructions that use the same information that is not modified by any other instruction (statement) in any other computation block node during a specific time in the complete code. In this way, they group instructions that can be processed or computed independently of any other statement that has a scope on information (information as a distinct bit pattern), which is referred to in this application as "indivisible instruction chains". These instruction chains in each computation block node each have a physically based "time" that is related to how long it takes a unit to process or compute them on a given hardware. Since hardware characteristics have a fundamental impact on the time required to process or compute an instruction (as well as software components such as OS, drivers, etc.), the "computation block nodes" also correlate them to the time required for the possible transfer of any other information during a specific time to another "computation block node" in the complete code, if it is required during a specific program step. Each "computation block node" knows what information for its instructions must be exchanged (communicated / transferred, "received" or "sent", respectively) to other "computation block nodes" and when. Thus, "computation block node" as used in this application brings a new scope and decision criterion, which in the context implies one of the central aspects of code parallelism: compute information (a bit pattern) on a unit, or transfer this information to another unit to compute in parallel. This decision can only be made if it is guaranteed that the information used will not change during a particular program step (or time) in any other part of the program.Moreover, building block nodes with this scope not only provide the benefit of parallelizing a given code, but this scope also shows some benefits for problems that are not well handled by SOTA compiler optimization techniques. Such problems and various solutions are well documented, for example, in the publication "Modern compiler design" by D. Grune. To illustrate the technical benefits, some benefits resulting from using the new scope for some of these known technical problems are shown below, e.g., different performance for pointer disambiguation and Fibonacci sequence codes, and furthermore, how the method solves previously unsolvable problems such as PDE parallelization is shown below.
[0110] First, the different scopes are illustrated in FIG. 6 using a schematic example. This figure has been slightly adapted to more easily show what the inventive system and method does. For example, the statement "y>a" does not occur twice in the same computation chain, and the resulting "compute" -> "communicate" model leads to a corresponding placement of loops, jumps, or flow control instructions. Nevertheless, this example shows what the different scopes of computation block nodes give compared to basic blocks: "a" and "b" do not change in block 1, the information of the condition of "y>a" is already known after the evaluation of the statement "y:=a*b", and the alternative scope of computation block nodes takes these properties into account. The proposed perspective of computation block nodes groups statements together in a new way. This change of perspective results from the approach of grouping together operations based on information that does not change at the same time step in the code. In this example, it is shown that two independent computation chains are developed, both of which are independent with respect to the information of "y".
[0111] In the simplified but more realistic example of Figure 7, the method of the invention exploits the fact that logarithmic instructions (as an example for modern CPUs: FYL2X) take much longer than floating-point addition and / or multiplication instructions (e.g. FADD, FMUL). The method of the invention appends the statements "x:=a+b" and "y:=a*b" to two different computation block nodes (cbn), since both are based on the same information "a" and "b". The "log2(x)" statement is appended to "x=a+b" since it uses the information "x". The information "y" is forwarded, and that information is available to both independent computation chains.
[0112] (ii) "Calculation matrix" and "Transfer matrix" From the flow graph of computation block nodes as described above, the method of the present invention forms two technically defined matrices, called in this application the "computation matrix" and the "transfer matrix". Both are numerical matrices. The "computation matrix" contains the instruction chains, while the "transfer matrix" contains the possible transfer properties (from and to other computation block nodes). Thus, the code extracted from these matrices always forms the pattern "compute->communicate", as explained in more detail in the following sections. If the code is mapped to one unit, the communication part disappears and the method of the present invention is reduced to an approach with basic blocks, each of which can be processed by the SOTA compiler. Referring to the example in Figure 7 of this form, we can see that unit 2 can calculate the long-running execution instructions of the statement "log2(y)" and unit 1 processes the loop with the increment of a ("a=a+1") (see Figure 8). To see how this works, in the next section, we will show the matrix builder in detail.
[0113] The name "matrix" is used to name a structure of the form (m×n×p), where m,n,p∈N0. Since m,n,p depend on the code, this can include different forms of mathematical objects, especially in terms of dimensions such as points, vectors, matrices, tensors, etc. m is the number of the maximum number of calculation blocks, which are the block number and segment number as in Figure 43, respectively, and n is the number that indicates the calculation block nodes that are independent but have the same segment number as in Figure 43 as the chain number. Depending on the maximum level of conditions in the code (or a series of branch nodes as in Figure 26), p is defined, which is shown as the path number in Figure 43. Therefore, whenever the term "matrix" is used, it can be said that it can be a vector, a matrix, or a tensor, or any other object that has a representation of the form (m×n×p), or in the form of a graph or tree. Thus, the optimization can be done using one tensor, for example by combining the computation and transfer matrices into one structure, or in a code with only one block / segment number, the transfer and computation matrices can each be a tensor, or both can be vectors. The dimensions of these "matrices" depend on the format of the code and the way it handles / represents information in the way the method is applied.
[0114] So-called "numeric matrices" can also contain textual forms such as the transfer "1->2". Depending on the optimization / mapping techniques used, the text in the matrix can be numeric or reduced to numeric values (depending on the character encoding used) so that they can be searched or compared, for example. Alternatively, the text transfer "1->2" can be represented / encoded by numeric values from the start and compared directly with other transfers, thus omitting the character encoding.
[0115] It is also possible to work without the "numerical matrix" and do all optimizations / mappings in the form of graphs / trees, since the "numerical matrix" can be used as another properly formatted form of representing a graph / tree-like structure. Whatever the mathematical or computational form of representing a group of instructions (here named computation block nodes) and their transfer dynamics, represented here by a "transmission package" as mentioned for example in FIG. 23, is based on the physically based dependency of transfer and computation latency that occurs in all binary-based electronic computing systems. It is based on grouping instructions (representing any form of instruction / operation, e.g., electronic circuit form) in a code (any form of code (high level, assembly, etc.)) based on the rule of placing the instruction that "reads" information A in the same group (here named computation block nodes) where the instruction that "writes" information A is located. This is significantly different from the grouping of instructions with the well-known basic blocks. If the dependency is not clear (as shown in Fig. 21 for the case of a graph element with two "read" nodes and one "write" node per instruction), a "forward" is introduced between two computation block nodes (from the one holding the "write" instruction to the one holding the "read" instruction). In this form, the code can be represented in the form of computation block nodes connected by the required forwardings. This information can be included in a graph / tree or an array or any suitable structure. To execute these instructions on a given hardware, a well-known method in the SOTA compiler can be used to optimize the chain of computation block nodes that group the instructions and then for each unit to execute on the involved computing unit (in this case called transpiling), but it must not be used.
[0116] (iii) "Matrix Builder" and how to get back into the code After parsing the code and adding all instructions to computation block nodes (CBNs), each computation block node is enumerated depending on its position in the flow graph. This leads to a similar form of the control flow graph, given by the edges between CBNs and the defined branch node connections. Using these positioning numbers, the unique position in the flow of the code can be used to place the information of what to compute and what to transfer in two matrices, i.e., the "computation matrix" for computation and the "transfer matrix" for transfer. It is clear that metadata such as the size of the data required for each CBN, the size of the transfer between CBNs, etc. can be easily derived.
[0117] Matrices represent a much more scalable form of information access than graphs, respectively, showing the well-formed nature of control flow graphs. They are not absolutely necessary for the method, and this step can also be performed directly on a graph / tree structure. However, the definition of the block nodes also shows the general nature of the matrix of the invention. Each row has an independent flow of instructions (=computation) (dependence by forwarding) and the required forwarding (=communication) to other cbns. It is guaranteed that a) no further information is required to compute all instructions in the computation block node (which is similar to a basic block, but the scope of independence is quite different), b) the information used is not changed anywhere else during the same computation step in the whole code, and c) only information that is not affected by the computation during this time step can be forwarded. Due to this fact, each cell in the computation matrix has all the instructions that can be calculated independently in parallel with all the instructions in the other cells in the same column. In the transfer matrix in each cell, the required forwarding at the beginning and end of each computation step (corresponding cell of the computation matrix) is now known. Returning code to run on different units results in a representation of the form "communication -> computation -> communication -> computation" etc. Each row represents a chain of computation and communication properties forming a set of computations joined by communication with other rows = chains of computations. The required information to communicate with other rows = chains is in the transfer matrix. Each row in the computation matrix (and the same combination in the transfer matrix) can also be combined with any other row in the matrix (computing all instructions of both combined cells and making the necessary transfers of both cells based on the transfer matrix) to create a new combination of computation <-> transfer behavior of the given code. In this step, computations are combined and transfers on the same unit (by combination) disappear. This later leads to simple forms of optimization, as well as the fact that the optimization / mapping step reliably results in executable code since no iterative or similar solution is required.This is also a well-known problem in the art of code parallelism, since debugging parallel code is very complex (see, for example, "ParaVis: A Library for Visualizing and Debugging Parallel Applications", A. Danner et al.).
[0118] Each row of the computation matrix defined here represents a chain of instructions for one unit. The unit depends on the level of implementation (e.g. bare assembly, thread, process, computation node, etc.). Obviously, empty blocks (empty cells in the computation matrix) or unused communication entries (empty cells in the transfer matrix or transfers on the same unit) disappear. As seen in Figure 10, so do start and end communications that link together. This occurs when there are no transfers when the code runs on a single unit and all transfers disappear due to variable reassignment / renaming (respectively applying well-known optimization methods in the SOTA compiler to obtain optimized code for a given unit).
[0119] Depending on the way of implementing the communication, non-blocking or blocking mechanisms can be used, since it is guaranteed that between computation block nodes, information used at the same time in the instruction or in another cbn with the same number is not transferred. Depending on the level, the computation and communication parts can be implemented, so that the method can be used as a compiler or transpiler. The transfer back to the code in the form of a "computation->communication" approach also makes it easy to use the most suitable language for the application (e.g., C, C++, python), depending on the target infrastructure such as IPC (InterProcess Communication) methods (e.g., queues / pipes, shared memory, MPI (Message Passing Interface), etc.), libraries (e.g., eventlibs, multiprocessinglibs, etc.), and the use of available SOTA compilers.
[0120] (iv) "Optimization" The general well-defined nature of the matrix is a unique basis for a wide range of possibilities to map / optimize code to a given hardware or to evaluate the best hardware configuration for a given code. This structure guarantees, as a result, executable code. Each hardware infrastructure has its own performance characteristics and, in combination with the available software layers, modern ICT (Information and Communication Technology) infrastructure is very complex. The method of the invention allows optimizing the code for the hardware or providing the ideal hardware. Most obviously, by constructing different row combinations from the computation and transfer matrices, it is important that the same combinations are constructed in both matrices. For each combination (e.g. computation and The resulting code (combining rows 1 and 2 in the transfer matrix) is a new version of the parallel / concurrent code for the given input code, whose properties on the target infrastructure (see Figure 11 for a simple example) can be evaluated.
[0121] Different combinations of compute<->communication ratios are obtained by different combinations of rows in the matrix (e.g., combining rows 1 and 2, or rows 2 and 5 in the compute and transfer matrix). Each combination can then be examined, including other known metadata for the given hardware / software infrastructure. For example, the data type of the data node() can be used to evaluate hardware characteristics such as cache length, available memory, or other characteristics of the target platform. This form of combining and searching for optimal compute / communication ratios for a given code and given hardware always results in runnable concurrent code, since no iterative solving or similar techniques are required to find a solution in the optimization step, and no solutions with, for example, race conditions, deadlocks, etc., occur, or deadlocks can be detected.
[0122] The grouping of instructions is based on the inherent physical constraint that transfers are usually larger than computing at the same "location", and the method yields an optimal solution space shape for the most divisible form of the code, resulting in a well-defined way and search space for a unique way to find the optimal map for a given hardware, or ideal hardware for a given code. This makes the method very generic, solving the technical problem of automatically adapting a given code to a target platform. When the method is used as a transpiler, the SOTA compiler can be used to optimize the code to the target hardware / unit.
[0123] No other method takes advantage of the inherent time characteristics and dependencies given in the code and generates this form of unique solution space to optimize in the form of the method's definition of computational blocks / nodes (by grouping by scope of unchanging information), which in many ways directly solves the technical problem. The examples in the detailed description show this in more detail.
[0124] (v) "Computation block node" according to the present invention and "basic block or slice" of the prior art Computation block nodes are not the same as basic blocks or slices. They have a different scope. They do not follow the definition of a basic block by, for example, connecting independent program parts based on jumps / branches. Instruction chains in computation block nodes have fundamental data dependencies, meaning that information used in these chains is neither changed anywhere else in a given code nor transported at the same time / program step.
[0125] Thus, the time or location of an instruction in a program is made a dependency in the computation block nodes. A computation block node consists of a chain of instructions based on the same information, where information is defined as any form of bit pattern (e.g. data variables, pointer addresses, etc.). By introducing transfers / communications that are combined with instructions, as well as information used by time steps in the code, a correlation of both these physically based times is achieved in the computation block nodes. The method arranges the computation blocks in such a way that for every time point it is clear which information can be transferred to where and which information can be computed in parallel. This gives a different perspective, especially for optimization techniques. There is a wide range of technical problems that can be solved by this change of perspective. The computation block node connects the location of information (bit patterns) in the computation framework with the time this information is used in the program. This is supported by the basic physical principles of computing in classical infrastructures. The following example with nested loops and arrays shows that this effect is good. Branches in a loop definition can be transformed into reading and writing of data points according to the array when distributed across parallel computation block nodes.
[0126] The method of the invention divides the code into segments of computation, and the resulting information must be transported. Thus, by the definition of computation block nodes, the method of the invention generates a matrix system, where instructions based on the same information are grouped, given the constraint that during any given time, the same information is not needed anywhere else in the infrastructure. The grouped instructions are not further divisible, since it is not possible to reach a faster computation for a particular group of instructions, since any form of transport is longer than computing this given chain in the computation block nodes. As shown, the general nature of the computation and transport matrices allows optimizing the partitioned code to the most concurrent solution possible for a given hardware. The ratio of computation to transport depends on the target hardware, and the method provides different solutions for partitioning into different ratios for a given code. By converting the graph of dependencies into matrices, they can be used in a more effective way to map / optimize the partitioned code to a target platform that includes the specific characteristics of this infrastructure (e.g., a GPU requires a different way of dealing with transport / computation distribution than a CPU). However, it is not mandatory to use matrices, and the optimization / mapping can be done directly on the graph / tree structure.
[0127] (vi) "Idle Time" - "Latency Time" The latency time as defined herein is given by the idle time of the processing unit 21 between sending back data to the parallel processing system 2 after processing a particular block of instructions of the processing code 32 on the data by the processing unit 21 and receiving the data necessary for the execution of a successive block of instructions of the processing code 32 by the same processing unit 21 (i.e. after retrieving and / or fetching said data). In contrast, the idle time of a processing unit may be defined herein as the amount of time that the processing unit is not busy between two computation block nodes or otherwise executing an idle process of the system. Thus, the idle time allows to measure the unused capacity of the processing unit of a parallel processing system. Although maximum speedup, efficiency and throughput are ideal cases of parallel processing, these are not achieved in the real case, since the speedup is limited due to various factors that contribute to the idle time of the processing unit. The idle time of a processing unit as used herein may find its origin in various causes, including, among others: (A) Dependencies between successive computational blocks / nodes (B) Dependencies may exist between instructions in two computational blocks / nodes. For example, an instruction cannot start until the previous instruction has returned its result, since they are independent. Another example of a data dependency is when both instructions try to modify the same data object, also known as a data hazard; (C) Resource constraints : Delays occur in pipelining if resources are not available at run time. For example, if one common memory is used for both data and instructions and instructions need to be read / written and fetched at the same time, only one can execute and the other must wait. Another example is that of limited resources such as execution units that may be busy at the required time; (C) Branching instructions and interrupts in programs: A program is not a linear flow of sequential instructions. There may be branch instructions that change the normal flow of the program, which may delay execution and affect performance. Similarly, interrupts may occur, postponing the execution of the next instruction until the interrupt is serviced. Branches and interrupts may have a detrimental effect on minimizing idle time.
[0128] Note that the task of minimizing idle time is sometimes called "load balancing", which means the goal of distributing work among processing units such that in the ideal case all processing units are kept busy at all times.
[0129] (vii) "Elementary Commands" The term "elementary operation" as used herein refers to a machine operation that does not include simpler operations. Execution of an instruction typically consists of the successive execution of several operations, including resetting registers, resetting memory stores, shifting characters in a register one place left or right, transferring data between registers, and operations such as comparing data items and ORing and multiplying. A set of elementary operations may provide a structure for executing a particular instruction. Elementary operations include the basic logic functions of logic gates, including AND, OR, XOR, NOT, NAND, NOR, and XNOR. Such elementary operations may be assumed to take a fixed amount of time on a given processing unit and may vary only by a fixed factor when executed on different processing units 21 or parallel processing systems 2.
[0130] In a first step, which will be described in more detail below, the inventive system transforms source code into a sequence or code 32 of elementary operations 321, ..., 325 structured in loops, branches, and sequences. It is independent of platform and compiler optimization level, so the same transformation can be used to optimize execution time on any platform.
[0131] The approach is based on decomposing a source code 31 written in a programming language into basic operations 32 / 321, ..., 325, i.e. separate transformed parts of the source code 31. The set of basic operations is finite for each processing unit, with several subsets: integer, floating point, logical and memory operations. These sets correlate to parts of the processor's architecture and to memory data paths. The basic operations used here can be classified into different levels, for example: The top level includes four operation classes: integer, floating point, logical and memory. The second level classification can be based on the origin of the operands (i.e. their location in the memory space), i.e. local, global or procedure parameters. Each group can exhibit different timing behavior: local variables are used frequently and are almost always in the cache, while global and parameter operands must be loaded from any address and can cause cache misses. The third level classification is by operand type, i.e. (1) scalar variables and (2) arrays of one or more dimensions. A pointer is treated as a scalar variable when the pointer's value is given using a single variable, or as an array when the pointer's value is given using multiple variables. The operations belonging to the INTEGER and FLOATING POINT classes are ADD, MUL, and DIV. The LOGIC class includes the logical operations LOG: (i.e., AND, OR, XOR, and NOT) and SHIFT: operations that perform bitwise movements (e.g., rotate, shift, etc.). The operations of the MEMORY class are: single memory assign, BLOCK, and PROC. A MEMORY BLOCK represents a transaction of a block of size 1000 and can only have array operands.A MEMORY PROC represents a function call with one argument and one return value. The arguments can be variables and arrays declared locally or provided as parameters to the calling function rather than globally.
[0132] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 12 and 13 show schematic architectures for possible implementations of an embodiment of a compiler system 1 for optimized compilation of program code 3 for execution by a parallel processing system 2 having multiple processing units 21. It should be noted that, as used herein, when an element is referred to as being "connected" or "coupled" to another element, the element may be directly connected or coupled to the other element, or there may be intervening elements. In contrast, when an element is referred to as being "directly connected" or "directly coupled" to another element, there are no intervening elements present. Furthermore, some portions of the detailed descriptions which follow are presented in terms of procedures, logic blocks, processes, and other symbolic representations of operations on data bits within a computer memory. These descriptions and representations are the means used by those skilled in the data processing arts to most effectively convey the substance of their work to others skilled in the art. In this application, a procedure, method, logic block, process, etc., is conceived to be a self-consistent sequence of steps or instructions leading to a desired result. Variations of the embodiments described herein may be discussed in the general context of processor-executable instructions residing on some form of non-transitory processor-readable medium, such as program code or code blocks, executed by one or more processors or other devices. Generally, program code includes routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The functionality of the program code may be combined or distributed as desired in various embodiments. The techniques described herein may be implemented in hardware, software, firmware, or any combination thereof, unless specifically described as implemented in a particular way. Any features described as modules or components may also be implemented together in an integrated logic device or separately as discrete but interoperable logic devices.When implemented in software, the technique may be realized at least in part by a non-transitory processor-readable storage medium having instructions that, when executed, perform one or more of the methods described above. The non-transitory processor-readable data storage medium may form part of a computer program product. For a firmware or software implementation, the method may be implemented with modules (e.g., procedures, functions, etc.) having instructions that perform functions described herein. Any machine-readable medium tangibly embodying instructions may be used in implementing the methodologies described herein. For example, the software code may be stored in a memory and executed by one or more processors. The memory may be implemented within the processor, for example as registers, or external to the processor.
[0133] The various illustrative logic blocks, modules, circuits, and instructions described in connection with the embodiments disclosed herein may be executed by one or more processors or processor units 21. The processor 21 may be, for example, a control unit 2101, a processor 2102 having registers 21021 and combinatorial logic 21022, and / or a graphics processing unit (GPU) 211, and / or a sound chip 212 and / or a vision processing unit (VPU) 213, and / or a tensor processing unit (TPU) 214, and / or a neural processing unit (NPU) 215, and / or a physics processing unit (PPU) 216, and / or a digital processing unit (DSP) 217, and / or a synergistic processing unit (SPU) 218, and / or a field programmable gate array (FPGA) 219, or any other processor unit 21 known in the art as, for example, a motion processing unit (MPU) and / or a general purpose microprocessor and / or an application specific integrated circuit (ASIC) and / or an application specific instruction set processor (ASIP), or other equivalent integrated or discrete logic circuitry. The term "processor" or "processing unit" 21 as used herein may refer to any of the above structures or any other structure suitable for implementing the techniques described herein. Furthermore, in some aspects, the functionality described herein may be provided in dedicated software or hardware modules configured as described herein. The techniques may also be fully implemented in one or more circuits or logic elements. The general-purpose processor 21 may be a microprocessor, but alternatively, the processor may be any conventional processor, controller, microcontroller, or state machine. In the described embodiment, the processing element refers to multiple processors 21 and associated resources such as memory or memory unit 22.Some exemplary methods and apparatus disclosed herein may be implemented in whole or in part to facilitate or support one or more operations or techniques for processing code in multiple processors. The multiprocessing system 2 may also include a processor array including multiple processors 21. Each processor 21 of the processor array may be implemented in hardware or a combination of hardware and software. The processor array may represent one or more circuits capable of performing at least a portion of an information computing technique or process. By way of example and not limitation, each processor of the processing array may include one or more processors, controllers, microprocessors, microcontrollers, application specific integrated circuits, digital signal processors, programmable logic devices, field programmable gate arrays, etc., or any combination thereof. As discussed above, the processor 21 may be any of a general purpose central processing unit (CPU) or a special purpose processor such as a graphics processing unit (GPU), digital signal processor (DSP), video processor, or any other special purpose processor.
[0134] The present invention comprises a compiler system 1 having subsystems 11, ..., 16. In a non-limiting embodiment, the subsystems include at least a lexer / parser 11, and / or an analyzer 12, and / or a scheduler 13, and / or a matrix module 14, and / or an optimizer module, and / or a code generator 16. In addition, they may comprise a processor array and / or a memory. The compiler 1 segments the code into code blocks. For the described embodiment, a block or code block refers to a section or portion of code that is grouped together. The grouping allows a group of statements / instructions to be treated as if they were one statement, as well as constraining the scope of variables, procedures, and functions declared within a block so that they do not collide with variables with the same name used elsewhere in the program for a different purpose.
[0135] The aforementioned memory or memory unit 22 of the parallel processing system 2 may comprise any memory for storing code blocks and data. The memory 22 may represent any suitable or desired information storage medium. The memory may be coupled to the processor unit 21 and / or the processing array. As used herein, the term "memory" 2 refers to any type of long-term, short-term, volatile, non-volatile, or other memory and should not be limited to any particular type or number of memories or to the particular type of medium on which the memory is stored. The memory 2 may include, for example, a primary storage unit 211 as processor registers 2111, and / or a processor cache 2112 including multi-level caches as L1 cache 21221, L2 cache 21222, etc., and / or a random access memory (RAM) unit 2113. It should be noted that with respect to this application, the problem with multi-level caches is the trade-off between cache latency and hit rate. The larger the cache, the better the hit rate but the longer the latency. To address this trade-off, multiple levels of caches can be used, with a small, fast cache backed up by a larger, slower cache. Multi-level caches generally work by checking the fastest level 1 (L1) cache first. If there is a hit, the processor can proceed faster. If that smaller cache misses, the next fastest cache (level 2, L2) is checked, and so on until external memory is accessed. As the latency difference between main memory and the fastest cache (see Figure 1) grows, some processors are beginning to utilize as many as three levels of on-chip cache.The memory 2 may further include, for example, a secondary storage unit 212 and / or a third storage unit 213 (e.g., tape backup, etc.), where the secondary storage unit 212 may include, for example, a hard disk drive (HDD) 2121 and / or a solid state drive (SSD) 2122 and / or a universal serial bus (USB) memory 2123 and / or a flash drive 2124 and / or an optical storage device (CD or DVD drive) 2125 and / or a floppy disk drive (FDD) 2126 and / or a RAM disk 2127 and / or a magnetic tape 2128, etc. In at least some implementations, one or more portions of the storage medium described herein may store signals representing information that is represented by a particular state of the storage medium. For example, an electronic signal representing information may be "stored" in a portion of the storage medium (e.g., memory) by affecting or changing the state of the portion of the storage medium to represent the information. Thus, in certain implementations, such a change in the state of the portion of the storage medium to store the signal representing the information constitutes a transformation of the storage medium to a different state or thing. As mentioned above, memory 2 may include, for example, random access memory (RAM), such as synchronous dynamic random access memory (SDRAM), first-in-first-out (FIFO) memory, or other known storage media.
[0136] (Micro)processors are based on integrated circuits that allow to perform arithmetic and logical operations based on (two) binary values (1 / 0 in the simplest case). For this, the binary values must be available for the computational units of the processor. The processor units need to get two binary values to compute the result of the expression a=b operand c. The time it takes to get the data for these operations is known as the latency time. There is a wide hierarchical range for these latency times from registers, L1 cache, memory accesses, I / O operations, or network transfers, as well as the processor configuration (e.g. CPU or GPU). Since every single component has a latency time, the overall latency for a computation is mainly the combination of hardware components required to get the data from one location to another in modern computing infrastructures. The difference between the fastest and slowest location for a CPU (or GPU) to get the data can be large (in the range of more than 10^9 times).
[0137] Latency in general terms is the time delay between the cause and the result of some physical change in the system that is observed or measured. Latency as used herein is directly related to the physical structure of the multiprocessing system 2. The multiprocessing system 2 comprises a processor unit 21 based on an integrated circuit, which allows to perform arithmetic and logical operations based on (two) binary values (1 / 0 in the simplest case). These binary values must be available for the computational units of the processor. The processor unit needs to obtain two binary values to compute the result of the equation a=b operand c. The time it takes to obtain data for these operations is known as the latency time. There is a wide hierarchical range for these latency times from registers, L1 cache, memory accesses, I / O operations, or network transfers, as well as the processor configuration (e.g., CPU or GPU). Since every single component has a latency time, the overall latency for a computation is mainly the combination of the hardware components required to get the data from one location to another in the infrastructure of the multiprocessing system. It is worth noting that while the speed of microprocessors has increased by more than tenfold per decade, the speed of general purpose memory (DRAM) has only doubled, i.e. the access time has halved. Thus, the latency of memory access in terms of processor clock cycles has increased by six times per decade. Multiprocessor systems2 exacerbate this problem even further. In bus-based systems, establishing high bandwidth buses between processors and memory tends to increase the latency of retrieving data from memory. When memory is physically distributed, the latency of the network and network interfaces is added to the latency of accessing local memory on a node. Latency typically increases with the size of the multiprocessing machine1, since more nodes mean more communication for computation, more jumps in the network for general communication, and more contention.A primary goal of parallel computing hardware design is to reduce the overall usage latency of data access by maintaining a high scalable bandwidth, while a primary goal of parallel processing coding design is to reduce the overall idle time of the processor units 21. In general, idle time of the processor units 21 can have several causes, such as memory access, deadlock or race conditions (e.g., when the sequence or timing of code blocks or threads processed by the processor units 21 are interdependent, i.e., dependent on the relative timing between the interfering threads). As used herein, a deadlock is a condition in which a member of a processor unit 21 is waiting for the output of another member, e.g., the output of an instruction block processed by another processor unit 21. Deadlock is a common problem in multiprocessing systems 1, parallel computing, and distributed systems, where software and hardware locks are used to arbitrate shared resources and enforce process synchronization. Thus, a deadlock as used herein occurs when a process or thread enters a waiting state because a requested system or data resource is held by another waiting process or has not yet been achieved by the process (the process may be waiting for another resource or data held by another waiting process). If a processor unit 21 is unable to proceed further because a resource requested by it is being used by another waiting process (data access or output of a not yet finished process of another processor unit 21), this is denoted herein as a deadlock leading to idle time of the respective processor unit 21.
[0138] The compiler system 1 includes means for translating a source programming language 31 of a computer program 3 into a machine code 32 as a target programming language, generating processing codes 3.1, ..., 3.n including several instructions executable by or controlling the operation of the multiple processing units 21 of the parallel processing system 2. The source programming language may be, for example, a high-level programming language 31. The high-level programming language 31 may include, for example, C and / or C++ 311 and / or python 312 and / or Java 313, Fortran 314, OpenCL (Open Computing Language) 315 or any other high-level programming language 31. It is important to note that the automatic parallelizing compiler system 1 may also be applied to a machine code 31 or an assembler code 31 as a source code in order to achieve parallelization of the code. In this case, the translation of the high-level language into machine code instructions does not have to be performed by the compiler system 10.
[0139] The parallel processing system 2 comprises a memory unit 22, which includes at least a main execution memory unit 221 / 2212 including a plurality of memory banks for holding at least a portion of data of the processing code 32, and a transition buffer unit 221 / 2211 including a high-speed memory for storing the start location of the processing code 32 and a data segment including at least a branch or jump instruction and / or used memory references and data values, where the main execution memory unit 2212 provides a slower access time than the transition buffer unit 2211. The transition buffer unit 2211 may include, for example, a cache memory module 2211 and / or an L1 cache 22121.
[0140] Execution of the processing code 32 by the parallel processing system 2 involves the occurrence of latency time 26, the latency time being given by idle time of the processing unit 21 for retrieving and / or storing data required for execution of a particular block of instructions of the processing code 32 by the processing unit 21. The latency time may include, for example, access time of the register 2211, and / or access time of the L1 cache 22121, and / or access time of the memory 2213, and / or I / O operation time, and / or data network transfer time, and / or processor configuration time.
[0141] The compiler system 1 comprises a parser module 11 for translating a source programming language 31 into a code 32 of basic instructions executable by said several processing units 21 and directly executable by said processing units, the basic instructions being selectable from a processing unit specific set of basic instructions including arithmetic operations 321 and / or logical operations 322 and / or control operations and / or I / O operations for said several processing units 21, in particular variable and array declaration instructions 323, comparison operation instructions 324 and code flow instructions 325. The arithmetic operations 321 may include, for example, addition, subtraction, multiplication and division operations. The logical operations 322 may include, for example, several logical expressions such as equal, not equal, greater than, less than, greater than or equal to, less than or equal to. The control operations may include, for example, a "branch expression" and / or a "loop expression".
[0142] In a variation of the embodiment, at least two of the processing units may have, for example, different sets of primitives. The different processing units having different sets of primitives may include, for example, a central processing unit (CPU) 210, a graphics processing unit (GPU) 211, a sound chip 212, a vision processing unit (VPU) 213, a tensor processing unit (TPU) 214, a neural processing unit (NPU) 215, a physics processing unit (PPU) 216, a digital signal processor (DSP) 217, a synergistic processing unit (SPU) 218, a field programmable gate array (FPGA) 219, etc.
[0143] The parser module 11 comprises means for splitting the code of primitives into computation block nodes 333, each of which consists of the smallest possible segmentation of non-further decomposable units, each of which contains a sequence of primitives that require the same input data. Two or more computation block nodes 333, each of which has a chain of primitives, form a computation chain 34 that creates an ordered flow of operations / instructions on the input data. The chains (sequences of primitives) in the computation block nodes 333 are constructed by fixed rules. Instructions are placed in positions in the chain 34 where a primitive that writes a data point is followed by a new primitive that reads. This automatically forms a chain 34 that is data-centric and maps the necessary physically constrained READ and WRITE operations to the computation registers 2211 of the CPU 21, the L1 cache 2212, network I / O, etc.
[0144] The compiler system 1 comprises a matrix builder 15 for generating several numerical matrices 151, ..., 15i from the computation chains 34 depending on the latency time 26. In the graph from the computation chains 34, system dependencies can be evaluated, but they cannot be simply decomposed into individual independent chains. Within the chains 34, there are merging and splitting situations of computation block 333 chains, resulting from data dependencies and / or code branching. The latency for distributing information in the hardware system is introduced as physical time. By assigning at least this time interval length to each computation block node 333, each computation block node 333 in the graph or tree structure is numbered according to its position in the graph and given a block number, and thus computation block nodes 333 having the same "time position" in the graph can be given the same number. If a computation block node 333 must be at least as long as it takes to distribute information in the system, and as many instructions as possible must be computed during this time, and each computation block node 333 has a number based on its position in the graph that depends on the program flow, then a set of matrices 151, ..., 15i can be constructed.
[0145] Matrices 151,...,15i also show that they can be captured mathematically (graph models based on CB, CC, etc. are not simply captured as tables / matrices). Thus, data manipulation and communication are given to build numerical matrices, which can be optimized (modified) for example by using ML or AI depending on the target platform and / or hardware setup.
[0146] The compiler system comprises a numerical matrix optimization module 16 that uses a numerical matrix optimization technique to minimize the overall incurred latency time as a total latency time 26 by providing an optimized structure of the computation chain 34 processed by the multiple processing units 21. Now, by the code generator 17, an optimized machine code for the multiple processing units of the parallel processing system with the optimized overall latency time 26 is generated. The optimization can now be applied to the hardware infrastructure. Quantities important for the optimization are known in numerical form in matrices 151, ..., 15i for each time unit and for each independent chain and branch. For example, from the contents of the matrices, it is known (i) the number of basic instructions that must be sequential, (ii) the size of the data transfer from computation block x in computation chain u and when this transfer is needed again in computation block y (where y>x) in computation chain v (for example, whether it is possible over a network or to combine computation blocks to have the data on the same cache line). As an example, for a GPU, data should be copied from memory to GPU memory in one process step, then all primitives with the same characteristics should be executed at once, whereas on a CPU, operations with the same data should be in the same cache line (CPU dependent), or operations on a particular data type can be computed on the corresponding CPU with a better instruction set as needed. In our system, since primitives are grouped into sequential groups in the CB, this always results in parallel code, even without optimizations.
[0147] In summary, since the (micro)processor only understands basic instructions, the source code 31 is divided into these basic instructions to achieve the most basic level of parallelization. (It should be noted that the present invention is also applicable to the technical problem of optimizing (micro)processors based on the principle of integrated circuits (ICs), which are a set of electronic circuits. The above-mentioned instructions are linked with the configuration of electronic circuits on the (micro)processor, and therefore the following topics are also applicable to any form of integrated circuit, or conversely can be used to derive an optimized integrated circuit (or electronic circuit configuration or direct electronic circuit) for a given code, since the instructions can be considered as a form of electronic circuit configuration that represents a calculation operation (e.g. +, -, data manipulation, etc.).) The following core points are key to the system of the present invention: (1) The basic instructions of the processor system are combined according to their inherent "READ" and "WRITE" behavior, i.e. in the chain of nodes and links to the calculation block 333, they form a chain of instructions according to the rule "an instruction writes to X1, and a new instruction that reads to X1 is appended after the last instruction that writes to X1". (2) When information / data needs to be propagated in the multiprocessor system 2, a new computation block 333 is started. (3) Each computation block 333 has a minimum length of time, which is proportional to the amount of time (latency) required to propagate information (data or signals) to / from the block 333 in the hardware system. (4) For a graph model with two read data nodes, an instruction node, and one write data node connected by a link, a chain from a computation block 333 has places where the two chains (a) merge (e.g., because an instruction reads from two data points written in two different computation blocks 333, or because a branch exists) and (b) arise (e.g., when two computation blocks 333 can start by reading at the same time).If necessary, the graph model can be based, for example, on more than two read nodes and several write nodes, or combine several instructions in one operation / instruction node. (5) These chains 34 can be decomposed, and the instructions and necessary information transfers for each discrete time interval can be captured in a matrix. For example, each row is divided into columns (one time interval) and contains an independent instruction chain, with the necessary information transferred to the other instruction chain. They are thus tangible for automatic optimization processes, especially numerical optimization processes. This provides the basis for a fully automatic parallelization of source code. It should be noted that in the present automatic parallelization system 1, the graph model is not simply based on a matrix or table representation, but provides, among other things, a multidimensional nested tree structure of the computation block graph as a task graph associated with the computation parallel chains 34, allowing the system 1 to evaluate properties that can be exploited for automatic parallelization, code optimization, computation block 333 scheduling, and even automatic cost estimation or automatic mapping to different architectures of the multiprocessing system 2.
[0148] As mentioned above, in the computation matrix, a cell contains a chain 34 of instructions given by a sequence of computation block nodes 333 forming a chain 34, while in the transfer matrix, a cell contains a transfer characteristic, i.e. a required transfer to and from another computation block node 333. Note that the transfer characteristic includes what information is needed in another computation block node 333. Depending on the level of the target infrastructure, this can be solved by classical compilation / controlled by the processor = transferred in cache (classical compilers distribute data to registers and caches), or explicitly shared by shared memory, guarded by locks, or explicitly sent and received between different nodes in a cluster using, for example, a Message Passing Interface (MPI) protocol, or sent and received by sockets in multicore inter-process communication (IPC), etc. The transfer characteristic can have any form of communication, from handled by the processor (cache) to explicit by a communication pattern (e.g. IPC via queues, MPI, etc.). This makes the method of the present invention scalable to a wide range of platforms and / or infrastructures. Thus, the transfer characteristic may include information such as, for example, send data %1 (integer) to cell (1,7). The transfer and / or communication requires a certain time, which typically depends directly on the (transfer / communication) latency time in a particular system. Finally, as mentioned above, it is noted that in a particular row of a computation matrix, the column cells contain a flow or sequence of instructions, and each column cell of a row contains one computation block node 333 of the sequence of computation block nodes 333, forming a chain 34 of a particular row. However, in a variant of a particular embodiment, each cell of a row does not necessarily contain a sequence or instruction of instructions. One or more of the cells of one or more particular rows of the computation matrix may be empty. This is also true for the transfer matrix. The computation matrix and the transfer matrix are usually equal in size, i.e. in the number of rows and columns.The computation matrix, the matrix, and the transfer matrix provide possible technical structures for automatic parallelization.
[0149] Technically, the matrix builder connects computation block nodes 333 according to the program flow (e.g., FIG. 35) and / or the code 32 or from the parser module 11. The program flow is represented by connecting computation block nodes 333 (similar to basic blocks), and a new computation block node 333 can be generated, for example, when adding a new instruction (=operation node), where, in general, there is no clear dependency (see FIG. 21), and the new computation block node 333 is placed after another computation block node 333 or under a branch node. After adding all the instructions, it is possible to number each computation block node 333 along the program flow forming a sequence of the chain 34 thus generated. These numbers (block numbers) are equal to the columns of the matrix. Computation block nodes 333 with the same block number are distributed in different rows in the same column of the computation matrix.
[0150] Application of the present invention to known technical problems In the following, we will show how the system and method of the present invention derive a technical solution to a well-known technical problem in the field of parallel processing.
[0151] (i) Calculating the Fibonacci sequence Different parallel processing implementations are known for generating and computing Fibonacci series numbers, with different performance characteristics. In the following, we show how the present invention is applied to an implementation using a) recursive function calls and b) loops. Both implementations show different performance. To understand the processing issues, for example, https: / / www.geeksforgeeks.org / program-for-nth-fibonacci-number / may be referenced.
[0152] Below is an example of processing code that uses recursion to generate the Fibonacci sequence. / / Fibonacci sequence using recursion #include<studio.h> int fib(int n) { if (n<=1) return n; return fib(n-1)+fib(n-2); } int main() { int n = 9; printf("%d", fib(n)); getchar(); return 0; }
[0153] Furthermore, below we show how parsing the function code of the declaration of fib(n) gives us a pseudo-token language. The function code of fib(n) is given by: int fib(n) { if (n<=1) return n; return fib(n-1) + fib(n-2); } The pseudo-token language is given by Table 1 below. [Table 1]
[0154] Figures 41 and 42 show a schematic of a computation block node (CB) with operations and corresponding data nodes (similar to the tokens in Table 1 above). (outside 1) JPEG2024541823000003.jpg5115 shows a forwarding to a position within a computation block node (either the start or end of a computation block node). As can be seen in Figure 42, recursive calls to the function result in additional forwarding.
[0155] The next step is to number the computation block nodes depending on their call location in the code. This results in a pseudograph as shown diagrammatically in Figure 43, and consequently in computation and transfer matrices as shown in Figures 44 and 45.
[0156] By linking the source and destination communication cells in the transfer matrix, removing the empty cells in the computation matrix, and transferring them back to different code segments, we obtain the result in Figure 46. Based on this code segment, the code can be generated directly (as a compiler) or transferred back to code, which can then be used to generate machine code using the SOTA compiler (as a transpiler).
[0157] To show how the system and method of the present invention provides benefits to the compilation of recursive implementations according to the Fibonacci series, the following sections describe how the method of the present invention maps and / or optimizes the code with a more concurrent solution than the input code. As shown to optimize the code, it is a combination between the rows, and the graph is shown with the combination of cbn in the branch node marked as branch 2b (see FIG. 43) and combine them (see FIG. 48). Invoking a function, in the present method, is a matter of placing the computation block nodes in the correct positions in the matrix to apply the corresponding transfers, as shown in FIG. 47. With this in mind, the recursive invocation of a function, in the present method, can be seen as a transfer of the result variable and the function parameters in the return statement as well as a "read" and a "write" (see FIG. 48).
[0158] Using the fib(4) example, Figure 49 shows step by step how additional computation block nodes and reduced transfers (since everything is on one computation chain) result in a more optimized source code.
[0159] The depth of this chain depends directly on the number n in fib(n). Since recursive calls are very easy to detect in code, it is easy to not implement recursive calls in their full dimension in the final application. For ease of understanding, Figure 50 shows this somewhat simplified. Step 4 shows the cbn for the call with "n=4".
[0160] The next step in Figure 51 shows the transfer that is made (transporting the information of the last "write" to the data node to where the "read" of the data node is made, and remembering this information in the corresponding computation block node). Since all computations are done on one chain, the transfer disappears, as can be seen in Figure 52. After solving all the steps, we get a program of the form in Figure 53.
[0161] Converting this back into code using the matrix representation, we can see that we get the following code as a result, which results in a more efficient code than the original fib(n=4) implemented by Table 1 above. Compiling this code with the SOTA compiler respectively gives a better optimized code than without applying our method. f(n) { if (n <= 1): return n; t1 = n-1 t2 = f(t1) t3 = n-2 t4 = f(t3) t5 = t2+t4 return t5 } main() { r1w = f(1) r0w = f(0) r2w = r1w+r0w r3w = r2w+r1w r4w = r3w+r2w }
[0162] A recursive call can also be interpreted like an array operation applying the method of the invention, since it transfers information (function parameters param[i]) to the corresponding cbn in the branch node of the function declaration, and then transfers the return value(s) back to a[i] (see Figure 54). This leads to an aspect that is also found in partial differential equations. This form of implementation of the Fibonacci series is referred to further below. However, as a next step, the following paragraphs show how to deal with partial differential equations, most of which result in heavy use of nested loops and array operations.
[0163] (ii)Partial Differential Equation (PDE) By arranging the operation nodes in a "read" and "write" pattern of dependencies and resolving non-obvious dependencies by transfer, it is possible, according to the present application, to derive a computation and communication model for a freely given discretized implementation of the PDE. The PDE for the 2D heat equation is used here, which is
number
[0164] Using finite difference methods,
number
[0165] Figure 55 shows part of the implementation in python. Each entry in the array is a data node. A read from an array index is in this method an operation node with a data node of the index and the base address of the array. Array operations can be viewed as operation nodes with corresponding data nodes (see Figure 56).
[0166] For an example of a loop array with an array operation a[i+Δiw]=a[i], Figure 57 shows the corresponding computation block nodes. With this in mind, the initial block (Figure 55) can be detailed in a graph such as Figure 58. All computations in the computation block (see Figure 55) are performed in j loops. By using 1D array notation and applying the basic rules of the method, a schematic form for the computation block nodes in a loop is derived as shown in Figure 59.
[0167] An array read (e.g., u[k][i+1][j]) creates a computation block node and metadata for transferring the value to this index is added to both cbns. One is the "read" node, and this data node was last written to (e.g., at a[k+1][i][j]). This leads to the fact that a statement such as the array operation in the j-loop (Figure 55) results in five computation block nodes representing the "read" operation of the array, then a computation block node that computes the arithmetic solution, then a computation block node with a write to the array at position [k+1][i][j]. This representation is a schematic perspective to show more clearly how the method takes such array operations into account. This leads to the situation shown in Figure 60.
[0168] One of the most basic principles of the proposed method is to find the last operation node A that "writes" to data node B, then place a new operation node C that "reads" from B after operation node A. If there is no clear 0 or 1 dependency, add a transfer to the computation block node that contains operation nodes A, C, respectively. Thus, a[i1]=a[i2] is a "read" of the data node with base address "a" and index "i2" and a "write" to the data node with base address "a" and index "i1" (see Figure 56). Thus, each loop creates a new computation block node with a "read" or "write" operation node and the corresponding transfer. It can be derived from the following scheme (Figure 61):
[0169] The loop can be seen in Figure 61, showing that every loop pass is a new computation block node on a new computation chain (row in the matrix). As long as there is no transfer, the empty computation block nodes and entries in the transfer matrix disappear, so these computation block nodes are concurrent (resulting in the same block or segment number in the numbering process step). If a transfer occurs, the CBN numbers will be different and they will no longer be computable in the same step. An offset in the index of each of the "read" and "write" operations can result in transfers between computation block nodes that contain each of the "read" and "write" operation nodes, as seen in Figure 62.
[0170] The dependency on the "read" and "write" of the index in the loop leads to the conclusion in Figure 63. If ΔI is smaller than 0, it indicates that the loop can be registered in as many computation block nodes as the length of the loop. The length means the number of successive iterations defined by the start value "i0" and the maximum value "Ai" and the increment value "Bi" in the loop definition, for example "for i=i; i<>Ai; Bi". If ΔI is greater than 0, only ΔI computation block nodes can be executed in parallel. If less than this number is used, forwarding occurs. If the loop is solved on more than ΔI units, the computation block nodes are not parallel (in the sense of having the same number in the flow graph). ΔI is therefore a unique number to determine how the "read", which means using the information in the array at the index but not changing it, can be distributed on different computation units.
[0171] This has some important consequences. Extracting one single dimension from the nested loops, it is possible to determine for each "read" to the array which of these reads lead to a "write" in the loop and which value is a "read" from the data node before the loop. Thus, for each "read", it can be derived whether there is a transfer to the computation block node (= computation chain) of the "write" part. This can be implemented as a model and leads to a new, rather general perspective on arrays in nested loops. Furthermore, according to the concept of Figure 63, since there is no transfer between the "read" data node and the "write" data node, it can be derived which of the nested loops can be solved, i.e., calculated in parallel in the method. For a 2D heat equation with a central difference scheme, this leads to the dependency shown in Figure 64. This means that the i and j loops can be solved. Since ΔI is always less than or equal to the number of successive iterations in the i and j loops, but not in the k loop. It is the i and j loops that can be solved in the method. This means that the "reads" and "writes" in the array are distributed to the computation block nodes, which can run in parallel, but the k loops must be iterative. Then, after each k loop, a transfer between units is performed. Note that at this point, the method eliminates the transfers in the mapping / optimization steps, if they are on the same unit.
[0172] In most implementations for solving PDEs, the calculations are done only for a subset of the arrays (e.g. by treating the boundary conditions differentially), which makes this step a bit tedious. In the example of two nested loops, rules can be derived for each loop nest to handle the results for gaps and "transfers" in the proposed model, as shown in Figure 65. The constant values are values from the appropriate or calculation of the previous operation of the loop in the "calculation block" (see Figure 55).
[0173] By incorporating the gaps that arise by looping over a subset of the array, a transfer pattern for the discretized equations can be derived, and depending on the gap size, the size of the loop (=mesh in this case), a model for the transfer is obtained, as shown in Figure 66, which represents the computational block node of the method.
[0174] Using a very small example of nX=5 and nY=4, this results in the computation and transfer matrices shown in Figure 67. In the transfer matrix, the arrows "->" indicate getting information and "<-" indicate sending data to another computation block node.
[0175] The method of the invention derives all the necessary transfers / communications between grid elements in the grid required for the discretized PDE. Figure 68 shows the transfers that occur between the units. Each unit is represented by one row in the matrix. In Figure 68, an extraction of the transfer and computation matrix is seen, with the dark grey being the "receiving" side of the transfer and the light grey being the "sending" part of the transfer. At this point, it is important to note that this does not mean that it must be sent and received. It can be protected by a barrier or lock or disappearance, for example, shared by a shared memory segment, shared by a cache and therefore processed by the CPU. It depends on the transfer / communication mechanism used. This can be transferred to the model and result in code of the form shown in Figure 69, respectively. It must be noted that the computation between p0 and p1 evolves. This situation arises because the method is defined to add a transfer from the "written" data node to the "read" data node, and this situation arises because the array operations in the nested loop start from the "read" operation node.
[0176] Figure 70 shows how the transfers can be used to create a time model. In grey are meta values (e.g. known from variable type definitions or similar) that can also be used to map / optimize the matrices to a given hardware infrastructure. These values are not used in this example. To illustrate the possible optimization steps, a very simple model is shown fully decomposed in Figure 71 below, where the combination of rows (1,2), (3,4) and (5,6) results in three units that compute:
number
number
number
[0177] These steps exemplarily show how the method generates different ratios between computation and communication, and how depending on the combination the transfer is reduced and the computation per unit is higher. To better illustrate this, a very simple model of computation and communication can be used, and some "real values" for time can be assumed. According to the performance values for floating-point capability (FP) per cycle and its frequency, two generations of Intel CPUs yield power values for these to compute floating-point arithmetic (like T in the model). We performed addition and multiplication of floating values in 4 cycles using P5 FP32:0.5 and Haswell FP32:32, with frequencies for cycles of 66MHz and 3.5GHz. To obtain a value for Δtransfer, which represents the communication latency in the network (the author is aware that many assumptions are made and that latency is not the only key value for the network), two types of latency were used: 1ns for InfiniBand type and 50ns for cache latency. The behavior in Figure 72 can be derived by Δt = combined cbn * cpu + number of transfers * network latency for a grid of nX = 2048 and nY = 1024.
[0178] It is always easy to get the code returned from this method or the two matrices adapted to the available IPC options. This generic form of code optimization brings innovations in the field of automatic code parallelization. For example, the code can be optimally mapped to, for example, an MPI cluster infrastructure, or with some finer grain tuning that applies a hybrid approach by combining MPI (shared memory) and local threading on the node. By different combinations (for example combining the first x cbns of the i direction with the y cbns of the j direction), it is possible to obtain an optimal ratio between "computation" and "communication" for any given discretized PDE, but depending on the available hardware infrastructure. This can be done without any manual interaction, since the required properties can also be tested or calculated for the available hardware infrastructure, solving a real technical problem of the prior art.
[0179] (iii) Fibonacci using loops The Fibonacci source can also be implemented using a loop. / / Fibonacci sequence using dynamic programming #include<studio.h> int fib(n) { / * Declare an array to store the Fibonacci numbers * / int f[n+2]; / / one extra to handle the case n=0 int 1; / * The 0th and 1st numbers in the sequence are 0 and 1 * / f[0] = 0; f[1] = 1; for (i=2; i<=n; i++) { / * Add the two leading numbers in the sequence and store them * / f[i] = f[i-1] + f[i-2]; } return f[n]; } int main() { int n=9; printf("%d", fib(n)); getchar(); return 0; }
[0180] For our example using the 2D heat equation, this leads to the result in Figure 73, which in turn leads to source code of the following form: main() { a1 = f(1); b1 = f(0) for (i=0; i<=(n-1) / 2; i++) { c1 = a1+b1 a2 = c1 b2 = b1 c2 = a2+b2 a1 = c2 b1 = c1 } } This has similar performance characteristics as the results of the code in Figure 53 shown above.
[0181] (iv) Pointer Disambiguation Pointer disambiguation is also a technical problem that has not been completely solved (see, for example, Runtime pointer disambiguation by P. Alves). By applying the method of the present invention, it can be shown that the method of the present invention solves the disambiguation caused by passing function parameters as pointers, since it takes the pointer as information and resolves this disambiguation in the step of forward destruction as shown by Figure 74.
[0182] Differences Between the System of the Present Invention and Exemplary Prior Art Systems The various approaches known from the prior art are essentially different in their technical approaches. In the following, the essential differences from three of the prior art systems will be described in detail to further illustrate the system and method of the present invention.
[0183] Slicing based code parallelization for minimizing inter-process communication by M. Kandemir et al. discloses a method for scalable parallelization that minimizes inter-processor communication in distributed memory multicore architectures. Using the concept of iterative spatial slicing, Kandemir discloses a code parallelization method for data-intensive applications. The method targets distributed memory multicore architectures and uses slicing to formulate the problem of data computation distribution (partitioning) across parallel processors, starting with the partitioning of the output array and iteratively determining the partitions of other arrays as well as the iterative space of loop nests in the application code. The goal is to minimize inter-processor data communication based on this iterative spatial slicing based problem formulation. However, Kandemir does this using the partitioning of the output array by iteratively determining the partitions of other arrays in the application code (see page 87). The method disclosed here is different because it does not involve iterative determination of the sequence parts to be combined. It obtains this information directly from the code. Program slicing was originally introduced by Weiser in his seminal paper, as disclosed in Kandemir, p. 88. Slicing means extracting statements from a program that potentially affect a particular statement of interest that serves as a slicing criterion (see Advanced slicing of sequential and concurrent programs by J. Krinke). Slicing techniques show similar effects to the well-known point data / control dependences (see Computer architecture by D. Patterson, p. 150ff.) and data flow analysis (e.g., [8]). Our method is similar to these methods at first glance.This stems from the fact that statements, blocks in a program, data dependencies are one of the most central points in programming and compilation, but the inventive method disclosed herein has a different perspective. The novelty, as shown by the report, is that this new approach offers a new perspective, namely the definition of computational block nodes. The inventive method explicitly takes each variable as a unique piece of information (represented by one specific bit pattern), and since computational instructions (= statements) can only be executed with the available bit patterns (in accessible storage, e.g. CPU registers), the inventive method extracts the time between the modification and the use of a variable in a certain "location" (e.g. a register). Known SOTA compilers do not extract this time from the (source) code, nor do they focus on single data entities (bit patterns) in a statement, all known prior art systems always use the result of a complete statement as the focus. As Kandemir discloses (p. 89), iterative space slicing can be used to answer questions such as "Which iterations of which statements can affect the value of a given set of elements from array A?" This shows a different perspective. The method of the present invention derives exactly this effect in the general form of finding the "reads" and "writes" for each element of the array, and correlates this delay of the "reads" and "writes" to the time required to transfer the elements. The method of the present invention discovers this "effect" and does not attempt to discover it by "simulating" the interaction with linear algebra methods or by iterative methods. On page 91, Kandemir summarizes a function that returns the set of loop iterations assigned to processor p from loop nest s, where Zp,r is the set of data elements accessed by processor p from array Ar. In contrast, the system and method of the present invention are not based on this approach.The method and system of the present invention first derives all dependencies of the code (even within nested loops), then creates matrices for all the code in the program, and finally maps / optimizes all the behavior of the program by combining the matrices. Kandemir's method derives by creating matrices for array loop index dependencies and assigning them to processors, then takes the Presbuerger set, then generates code as output "(a set of potentially nested loops)" (see Kandemir p. 92), and then iterates over the unknowns. Kandemir's method does not consider hardware specifications either.
[0184] (b) Automatic Parallelization: Executing Sequential Programs on a Task-Based Parallel Runtime by A. Fonseca (hereafter Fonseca) discloses another prior art system for automatically parallelizing sequential code in modern multicore architectures. Fonseca discloses a parallelizing compiler that analyzes read and write instructions in a program, as well as control flow modifications, to identify a set of dependencies between instructions in the program. The compiler then rewrites the program and organizes it into a task-oriented structure based on the generated dependency graph. Parallel tasks consist of instructions that cannot be executed in parallel. A work-stealing based parallel runtime is responsible for scheduling and managing the granularity of the generated tasks. In addition, a compile-time granularity control mechanism also avoids the creation of unnecessary data structures. Although Fonseca focuses on the Java language, the technique may be applicable to other programming languages as well. However, in contrast to the inventive method disclosed in this application, Fonseca's approach requires analyzing the memory accessed to understand the dependencies between parts of the program in order to automatically parallelize the program (see Fonseca, p. 6). The inventive method disclosed in this application is different. Fonseca uses data groups and memory layouts and then checks the dependencies. This is not the same scope as the inventive method, since Fonseca explicitly holds that it is task parallelism. For example, in the Fibonacci example, as mentioned above, the cost of creating a new task is higher than the cost of running the method for the case of low input numbers (see Fonseca, p. 7). This shows that this is not the same approach. The inventive method deals with this example in a completely different way. On the other hand, it also directly proves the technical problem that can be solved by the inventive method. Moreover, Fonseca (see p. 9) must define the main requirements for the location of future creations.In contrast, the method of the present invention relies on the data dependency of a single piece of information / variable to know where to place each instruction or statement. Fonseca (see page 9) also discloses that algorithm 18 must be used to find the best position to generate the future. In contrast, the method of the present invention places the instruction exactly where it is based on the new scope. In Fonseca (see page 10), operations that are both commutative and associative are used. The method of the present invention excludes, for example, division (used in many mathematical models), so the approach is not based on this form. This limitation is also typically due to the map-reduce approach (described herein).
[0185] (c) US Patent Application Publication No. 2008 / 0263530 discloses a system for converting application code into optimized application code or into executable code suitable for execution on an architecture with at least first and second level data memory units. The method includes obtaining application code, the application code including data transfer operations between levels of memory units. The method further includes converting at least a portion of the application code. The conversion of the application code includes scheduling data transfer operations from the first level memory unit to the second level memory unit such that accesses of data accessed multiple times are closer in time than in the original code. The conversion of the application code further includes determining a layout of data in the second level memory unit to improve data layout locality such that after scheduling the data transfer operations, data accessed closer in time is also closer in layout than in the original code. US2008 / 0263530A1 allows to improve layout locality (see US2008 / 0263530A1, p. 4, para. 0078). In contrast, the inventive method disclosed herein has a different scope. The inventive method finds a form of this "locality" and orders the instructions in such a way that the instructions that must be "local" are grouped together, so that this intrinsic form of the code is then arranged in a matrix, where there can be a respectively general non-iterative one by combining the elements, where an optimal mapping to the given hardware can be set, which always results in a concurrent form of the code. Iterative solutions may miss the solution and end up with an unclear result. Furthermore, US2008 / 0263530A1 (p. 6, para. 0092) discloses that it can be considered as a complex non-linear problem, for which a reasonable, near-optimal and scalable solution is provided.In contrast, the method of the present invention avoids obtaining a nonlinear problem that is not, but rather, a complex nonlinear problem that requires an iterative numerical solution / optimization approach / algorithm. US2008 / 0263530A1 (page 6, paragraph 0093) considers that the access locality is improved by computing reuse vectors and applying them to find a suitable transformation matrix T. In contrast, the method of the present invention disclosed herein does not require a transformation matrix or the computation of such a transformation matrix. The method of the present invention reads the intrinsic logical connection between array operations based on a given code (in the form of a loop definition or loop block in a compiler IR language, or a jump definition in assembler code). Finally, US2008 / 0263530A1 (page 6, paragraph 0093) claims that after fixing T, the alignment M for arrays accessed within the loop nest is fixed, whose alignment is not yet fixed. This discloses an iterative numerical solver-based approach. The inventive method disclosed herein reads this dependency in the code without iterative solving techniques that use linear algebraic methods to find the solution to the simultaneous equations. [Explanation of symbols]
[0186] List of References 1 Compiler System 11 Lexer / Parser 12. Analyzer 13 Scheduler 14 Calculation Blockchain Module 15. Matrix Builder 151,…,15i matrix 16 Optimization Module 17. Chord Generator 2 Parallel Processing Systems / Multiprocessor Systems 21 processor units 210 Central Processing Unit (CPU) 2101 Control Unit 2102 processor 21021 Register 21022 Combinational Logic 211 Graphics Processing Unit (GPU) 212 Sound Chip 213 Vision Processing Unit (VPU) 214 Tensor Processing Units (TPU) 215 Neural Processing Unit (NPU) 216 Physical Processing Unit (PPU) 217 Digital Signal Processor (DSP) 218 Synergistic Processing Unit (SPU) 219 Field Programmable Gate Array (FPGA) 22 memory units 221 Primary Storage Unit 2211 Processor Registers 2212 processor cache 22121 L1 Cache 22122 L2 Cache 2213 Random Access Memory (RAM) Unit 222 Secondary Memory Unit 2221 Hard Disk Drive (HDD) 2222 Solid State Drive (SSD) 2223 Universal Serial Bus (USB) memory 2224 Flash Drive 2225 Optical storage device (CD or DVD drive) 2226 Floppy Disk Drive (FDD) 2227 RAM disk 2228 Magnetic Tape 223 Tertiary storage units (tape backup, etc.) 23 Memory Bus 231 Address Bus 232 Data Bus 24 Memory Management Unit (MMU) 25 Input / Output (I / O) Interfaces 251 Memory Mapped I / O (MMIO) or Port Mapped I / O (PMIO) Interface 252 Input / Output (I / O) Channels (Processor) 3 Program Code 31 High-Level Languages 311 C / C++ 312 Phython 313 Java 314 Fortran 315 OpenCL (Open Computing Language) 32 Low-level languages (machine code / assembly language) 321 Arithmetic operations instructions 322 Logical Operation Instructions 323 Variable and array declaration behavior 324 Comparison operation command 325 Code Flow Instructions / Memory Operations / I / O Operations 33 Nodes 331 Data nodes (holding some kind of data value) 3311 Input data node for reading in (Read access) 3312 Output Datanode for writing out , write access) 332 Operation node (operation node, operation node) (performing an operation) 333 Calculation block nodes (CB1, CB2, …, CBx) 334 Control Flow Nodes 3341 Branch Node 3342 Hidden Branch Nodes 3343 Loop Branch Node 335 Condition Node 34 Chain 341 Calculation chain (chain of calculation blocks / nodes) 342 Action Chain (Chain of Action Nodes) 35 Latency time Δt (time to transfer data from one process to another) 351 Δt readwrite = Time between write and read access to a data node 352 Δt computation = Time to compute all the operation nodes in a computation block node 353 Total (overall) latency time Δt total 4. Network 41 Network Controller instruction latency Processor Core Cache non-blocking unit Compute Computing 2 units computechain computation calculation communication minimal runtime resultat result operation datanode Data Node Operation node source target variable direction direction start start end End forward backward returnvalue value base index read write compare comparison loopstate Loop state computations target unit True True False Datenknoten Data Node Operationsknoten Operation node type: branch transmission type: fusion transmission type: split transmission type: b2b transmission
Claims
1. A compiler system (1) for optimizing compiling and automatically parallelizing program code (3) for execution by a parallel processing system (2) having a plurality of processing units (21) that simultaneously process instructions on data in the parallel processing system (2) by executing the program code (3), comprising: the compiler system (1) has means for translating a source programming language of a source machine code of the computer program (3 / 31) into a machine code (3 / 32) as a target programming language by generating a machine code (32) as a parallel processing code which includes several instructions executable by the processing units (21) of the parallel processing system (2) or which control the operation of the processing units (21); The parallel processing system (2) has a memory unit, the memory unit including at least a main execution memory unit (221 / 2212) including a plurality of memory banks for holding at least a part of the data of the processing code (32), and a transition buffer unit (221 / 2211) storing the start position and data segments of the processing code (32) and including a high-speed memory containing at least branch or jump instructions and / or memory references and data values used, the main execution memory unit (2212) providing a slower access time than the transition buffer unit (2211), the execution of the processing code (32) by the parallel processing system (2) involves the occurrence of a latency time (26), the latency time being given by an idle time of a processing unit (21) between sending data back to the parallel processing system (2) after processing a particular block of instructions of the processing code (32) on the data, and receiving by the processing unit (21) the data necessary for the execution of a successive block of instructions of the processing code (32); said compiler system (1) having a parser module (11) for translating said source programming language (31) into a code (32) having a flow of basic instructions executable by said processing units (21), said basic instructions being selectable from a processing unit specific set of basic instructions, said basic instructions comprising basic arithmetic operations (321) and / or logical operations (322) and / or control operations (325) and / or memory operations (325) for said several processing units (21); said parser module (11) having means for dividing said code (32) of basic instructions into computation block nodes (333), each computation block node (333) consisting of the smallest possible segmentation of a non-decomposable sequence of basic instructions of said code (32) processable by a single processing unit (21), said smallest possible segmentation of basic instructions being characterized by a sequence of basic instructions constituted by successive read and write instructions, said sequence not being further decomposable by smaller sequences of basic instructions between successive read and write instructions, said read and write instructions being required for receiving data necessary for processing said sequence of basic instructions by said processing unit (21) and for sending back data after processing by said sequence, the compiler system (1) comprises a matrix builder (15) for generating numerical matrices (151, ..., 15i) from computation chains (34) divided from the code (32) depending on the latency times (26), the numerical matrices (151, ..., 15i) including computation matrices and transfer matrices, the computation chains (34) being formed by one or more computation block nodes (333) creating an ordered flow of computation block nodes (333), each computation chain (34) being executed by one processing unit (21); the computation matrix includes in each row the computation chain (34) of processing units (21), and each column includes the sequence of primitives of computation block nodes (333) in the computation chain (34) of the row, and the transfer matrix includes transfer characteristics associated with data transfer from one computation block node to a successive computation block node (333); the compiler system (1) has an optimizer module (16) that uses a numerical matrix optimization technique to minimize an aggregated incurred latency time (26) that aggregates all incurred latency times (261) by providing an optimized structure of a calculation chain (34) processed by the multiple processing units (21); and a code generator (17) generates an optimized machine code having an optimized aggregated latency time (26) for the multiple processing units (21); Compiler system.
2. 2. The compiler system of claim 1, wherein the source programming language is a high-level programming language.
3. 3. The compiler system of claim 2, wherein the high level programming languages include C and / or C++ and / or python and / or java.
4. 4. The compiler system of claim 1, wherein the transition buffer unit comprises a cache memory module and / or an L1 cache.
5. 5. The compiler system according to claim 1, wherein the latency time includes register access time, and / or L1 cache access time, and / or memory access time, and / or I / O operation time, and / or data network transfer time, and / or processor configuration time.
6. 6. A compiler system according to claim 1, wherein at least two of the processing units have different sets of basic instructions.
7. 7. The compiler system of claim 1, wherein the arithmetic operations include addition, subtraction, multiplication, and division operations.
8. 8. The compiler system according to claim 1, wherein the logical operations include several logical expressions such as equal, not equal, greater than, less than, greater than or equal to, less than or equal to.
9. 9. The compiler system according to claim 1, wherein the control operations include a "branch expression" and / or a "loop expression."
10. 10. The compiler system of claim 1, wherein the computation block nodes (333) are generated by creating a mapping of reads and writes of data nodes (331) by their operation nodes (332) by mapping the interactions of the reads and writes of the single instruction.
11. 11. A compiler system according to claim 1, wherein the computation block nodes (333) are made up of connected operation nodes (332), the operation nodes (332) being arranged in a chain (34) having one or more input data nodes (331 / 3311) and one or more output data nodes (331 / 3312).
12. 12. The compiler system of claim 11, wherein the computation block node (333) is connected to a next computation block node (333), or to a control flow node (334), or to one or more input data nodes (331 / 3311) and one or more output data nodes (331 / 3312) that build a chain (34).
13. 13. A compiler system according to claim 1, wherein each pair of computation and transfer matrices is implemented to form a single numerical matrix and / or a multidimensional numerical tensor, respectively.
14. 14. The compiler system of claim 13, wherein the numerical matrix optimization is implemented as a tensor optimization.
15. 15. A compiler system according to claim 13 or 14, characterized in that the numerical matrix or tensor optimization by the optimizer module is based on a dedicated applied machine learning structure.
16. 16. The compiler system of claim 1, wherein each computation block node (333) of a computation chain (34) is numbered along the ordered flow of computation block nodes (333) in the computation chain (34) by assigning a block number to the computation block node (333), and wherein in the computation matrix and the transfer matrix, the block numbers of the computation block nodes (333) are equal to the columns of the matrices, and computation block nodes (333) having the same block number are distributed in different rows of the same column.
17. 17. A compiler system according to claim 1, wherein in the computation matrix, each row contains a computation chain (34) of a processing unit (21), the cells of the computation matrix contain the computation block nodes (333) of the computation chain (34), and in the transfer matrix, the cells contain required transfer characteristics to and / or from other computation block nodes (210).
18. 20. The compiler system of claim 17, wherein the transfer matrix having cells with the transfer characteristics includes at least what information is needed at another computation block node (333).
Citation Information
Patent Citations
Scheduler, processor system, program generating method, and program generating program
JP2010108153A
Program, code generation method and information processing apparatus
JP2013206289A
System and method for job scheduling optimization
US20130191843A1
