Computer-implemented method, computer program, and system (integrating loop unrolling and loop splitting to reduce control overhead)

Abstract

To describe techniques for reducing overhead controls.SOLUTION: A loop tree is constructed from a program, such as a structured control flow program. Structured control flow refers to a programming concept where the flow of control to a block or region is based on single entry and single-exist methodology (SESE). A loop tree refers to a tree-like data structure that graphically represents loop(s) and / or an if-condition(s) in a program, such as a structured control flow program. A loop splitting operation or a loop unrolling operation may then be performed in connection with the node of the loop tree that is identified as having the highest benefit (ratio of execution cycles gained to the increase in code size) representing an if-condition or a loop, respectively, provided that the resultant code fits in the instruction buffer.SELECTED DRAWING: Figure 14

Description

[Technical Field]

[0001] The present disclosure relates generally to control overhead. [Background technology]

[0002] Overhead is any combination of extra or indirect computation time, memory, bandwidth, or other resources required to perform a particular task. Control overhead in program execution refers to the additional computational cost incurred by a program due to the need to perform various control operations such as branches and loops.

[0003] Branching is the process of making decisions in a program by choosing one of two or more paths based on a condition. Overhead is the additional processing time required to evaluate the condition and determine the correct path to follow.

[0004] A loop involves repeating a set of instructions multiple times, and overhead is the additional processing time required to execute each iteration of the loop.

[0005] Control overhead is necessary for program execution, but excessive control overhead can result in slower program execution, increased memory usage, and other performance problems. Summary of the Invention [Problem to be solved by the invention]

[0006] One classic transformation for reducing control overhead is loop unrolling. Loop unrolling involves reducing the number of iterations a loop executes by executing multiple iterations (an unrolling factor) of the loop in a single pass. In loop unrolling, the loop is completely unrolled, so that each iteration of the loop is replaced by a single instance of the loop body. Unfortunately, loop unrolling can result in an increase in the size of the result code, which can have a negative impact on cache efficiency and instruction fetching. For example, in the case of a programmable accelerator, instructions related to a loop unrolling transformation may be stored in an instruction buffer (a storage for holding instructions), which may have a limited size. As a result, the instruction buffer may not have enough space to store such instructions.

[0007] Another classic transformation for reducing control overhead is loop splitting. Loop splitting involves splitting a loop into multiple subloops, each with a different loop index set, so that each subloop executes a different subset of the original loop iterations. In particular, loop splitting reduces the number of branches and conditional statements required to manage loop control. By splitting a loop into smaller subloops, each subloop can be executed without the need for complex control statements. Unfortunately, loop splitting increases the size of the result code. In the case of programmable accelerators, similar to loop unrolling transformations, instructions associated with loop splitting transformations may be stored in an instruction buffer, which may be of limited size. As a result, the instruction buffer may not have enough space to store such instructions.

[0008] Therefore, loop unrolling and loop splitting transformations may compete for space in the instruction buffer, and it is not always optimal to perform one before the other. For example, performing a loop unrolling transformation followed by a loop splitting transformation may not leave enough room in the instruction buffer after unrolling the loop to perform the loop splitting transformation. Similarly, in another example, performing a loop splitting transformation followed by a loop unrolling transformation may not leave enough room in the instruction buffer after performing the loop splitting transformation.

[0009] Therefore, there is currently no means to effectively reduce program control overhead, such as in the case of a programmable accelerator, using an instruction buffer of limited size. [Means for solving the problem]

[0010] In one embodiment of the present disclosure, a computer-implemented method for reducing control overhead includes programmatically constructing a loop tree, the method further including performing an operation selected from the group consisting of: a loop splitting operation in response to a node of the loop tree representing an if condition, and a loop unrolling operation in response to a node of the loop tree representing a loop.

[0011] Furthermore, in one embodiment of the present disclosure, the method additionally comprises calculating a metric for each node in the loop tree, and further comprising identifying the node in the loop tree with the metric associated with the ratio of beneficial execution cycles to code size increase having the highest value.

[0012] Additionally, in one embodiment of the present disclosure, in response to the node being associated with the ratio of execution cycles that are beneficial to code size increase having the highest value and in response to the node representing an if condition, a loop splitting operation is performed on the loop associated with the if condition.

[0013] Furthermore, in one embodiment of the present disclosure, a loop unrolling operation is performed on a loop in response to the node being associated with the ratio of execution cycles that is beneficial to code size increase having the highest value and in response to the node representing a loop.

[0014] Additionally, in one embodiment of the present disclosure, the method further comprises dividing the loop by considering multiple if conditions in response to the program including an equivalent loop condition with multiple conditional branches.

[0015] Furthermore, in one embodiment of the present disclosure, the method additionally comprises determining a number of partitions to be used to divide the loop iteration space based on an estimated cycle reduction and an estimated code size increase, and further comprising dividing the loop iteration space into the determined number of partitions in response to the program including different loop conditions with multiple conditional branches.

[0016] Additionally, in one embodiment of the present disclosure, the program is a structured control flow program.

[0017] Other aspects of the embodiments of the computer-implemented methods described above are systems and computer program products.

[0018] Thus, embodiments of the present disclosure use a size-limited instruction buffer to reduce control overhead for programs, such as in the case of a programmable accelerator.

[0019] The foregoing has outlined rather generally the features and technical advantages of one or more embodiments of the present disclosure in order that the detailed description of the disclosure that follows may be better understood. Additional features and advantages of the present disclosure will be described hereinafter and may form the subject of the claims of the present disclosure. [Brief explanation of the drawings]

[0020] The present disclosure may be better understood from the following detailed description when considered in conjunction with the following drawings.

[0021] [Figure 1] 1 illustrates a communication system embodying the principles of the present disclosure, according to an embodiment of the present disclosure.

[0022] [Figure 2] 1 illustrates an edge computing device reducing the overall time to complete a task, according to an embodiment of the present disclosure.

[0023] [Figure 3] FIG. 10 is a diagram of software components used by a control overhead reducer to reduce the control overhead of a program using a size-limited instruction buffer, according to an embodiment of the present disclosure.

[0024] [Figure 4] 1 illustrates an example of a loop tree constructed from a program according to an embodiment of the present disclosure.

[0025] [Figure 5] 10 illustrates computing metrics for nodes that represent loops in a program, according to an embodiment of the present disclosure.

[0026] [Figure 6] 10 illustrates computing metrics for nodes that represent if conditions in a program, according to an embodiment of the present disclosure.

[0027] [Figure 7] 10 illustrates inputting metrics in a constructed loop tree according to an embodiment of the present disclosure.

[0028] [Figure 8] 1 illustrates splitting a loop considering multiple if conditions when a program has equivalent loop conditions with multiple conditional branches, according to an embodiment of the present disclosure.

[0029] [Figure 9] 1 illustrates dividing a loop iteration space into partitions when a program has different loop conditions with multiple conditional branches, according to an embodiment of the present disclosure.

[0030] [Figure 10] 10 illustrates a partitioning table for determining the number of partitions to use in dividing the loop iteration space in the program of FIG. 9 according to an embodiment of the present disclosure.

[0031] [Figure 11] 1 illustrates identifying partitions by grouping consecutive repeating variable rows in a partitioned table that have the same set of conditions on columns of the partitioned table, according to an embodiment of the present disclosure.

[0032] [Figure 12] 1 illustrates selecting a partition based on estimated beneficial execution cycles and estimated code size increments according to an embodiment of the present disclosure.

[0033] [Figure 13] 1 illustrates an embodiment of the present disclosure of a hardware configuration of a control overhead reducer, representing a hardware environment for practicing the present disclosure.

[0034] [Figure 14] 1 is a flowchart of a method for reducing program control overhead using an instruction buffer of limited size, according to an embodiment of the present disclosure.

[0035] [Figure 15] 1 is a flowchart of a method for enhancing loop partitioning according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0036] In one embodiment of the present disclosure, a computer-implemented method for reducing control overhead includes programmatically constructing a loop tree, the method further including performing an operation selected from the group consisting of: a loop splitting operation in response to a node of the loop tree representing an if condition, and a loop unrolling operation in response to a node of the loop tree representing a loop.

[0037] This scheme uses an instruction buffer of limited size to reduce program control overhead, such as in the case of a programmable accelerator.

[0038] Furthermore, in one embodiment of the present disclosure, the method additionally comprises calculating a metric for each node in the loop tree, and further comprising identifying the node in the loop tree with the metric associated with the ratio of beneficial execution cycles to code size increase having the highest value.

[0039] In this manner, the node in the loop associated with the highest gain, corresponding to the ratio of beneficial execution cycles to increased code size, can be determined. Upon identifying the node with the highest gain, a loop splitting or loop unrolling operation is performed based on whether the node represents an if command or a loop, respectively.

[0040] Additionally, in one embodiment of the present disclosure, in response to the node being associated with the ratio of execution cycles that are beneficial to code size increase having the highest value and in response to the node representing an if condition, a loop splitting operation is performed on the loop associated with the if condition.

[0041] In this scheme, a loop splitting operation is performed for the loop associated with the if condition represented by the node if the resulting code can fit within the instruction buffer.

[0042] Furthermore, in one embodiment of the present disclosure, a loop unrolling operation is performed on a loop in response to the node being associated with the ratio of execution cycles that is beneficial to code size increase having the highest value and in response to the node representing a loop.

[0043] In this scheme, a loop unrolling operation is performed for the loop represented by the node if the resulting code can fit within the instruction buffer.

[0044] Additionally, in one embodiment of the present disclosure, the method further comprises dividing the loop by considering multiple if conditions in response to the program including an equivalent loop condition with multiple conditional branches.

[0045] In this manner, loop decomposition can be enhanced for programs that contain equivalent loop conditions with multiple conditional branches.

[0046] Furthermore, in one embodiment of the present disclosure, the method additionally comprises determining a number of partitions to be used to divide the loop iteration space based on an estimated cycle reduction and an estimated code size increase, and further comprising dividing the loop iteration space into the determined number of partitions in response to the program including different loop conditions with multiple conditional branches.

[0047] In this manner, loop decomposition can be enhanced for programs that contain different loop conditions with multiple conditional branches.

[0048] Additionally, in one embodiment of the present disclosure, the program is a structured control flow program.

[0049] In this manner, the control overhead of programs such as structured control flow programs may be reduced.

[0050] Other aspects of the embodiments of the computer-implemented methods described above are systems and computer program products.

[0051] As mentioned above, branching is the process of making decisions in a program by choosing one of two or more paths based on a condition. Overhead is the additional processing time required to evaluate the condition and determine the correct path to follow.

[0052] A loop involves repeating a set of instructions multiple times, and overhead is the additional processing time required to execute each iteration of the loop.

[0053] Control overhead is necessary for program execution, but excessive control overhead can result in slower program execution, increased memory usage, and other performance problems.

[0054] One classic transformation for reducing control overhead is loop unrolling. Loop unrolling involves reducing the number of iterations a loop executes by executing multiple iterations (an unrolling factor) of the loop in a single pass. In loop unrolling, the loop is completely unrolled, so that each iteration of the loop is replaced by a single instance of the loop body. Unfortunately, loop unrolling can result in an increase in the size of the result code, which can have a negative impact on cache efficiency and instruction fetching. For example, in the case of a programmable accelerator, instructions related to a loop unrolling transformation may be stored in an instruction buffer (a storage for holding instructions), which may have a limited size. As a result, the instruction buffer may not have enough space to store such instructions.

[0055] Another classic transformation for reducing control overhead is loop splitting. Loop splitting involves splitting a loop into multiple subloops, each with a different loop index set, so that each subloop executes a different subset of the original loop iterations. In particular, loop splitting reduces the number of branches and conditional statements required to manage loop control. By splitting a loop into smaller subloops, each subloop can be executed without the need for complex control statements. Unfortunately, loop splitting increases the size of the result code. In the case of programmable accelerators, similar to loop unrolling transformations, instructions associated with loop splitting transformations may be stored in an instruction buffer, which may be of limited size. As a result, the instruction buffer may not have enough space to store such instructions.

[0056] Therefore, loop unrolling and loop splitting transformations may compete for space in the instruction buffer, and it is not always optimal to perform one before the other. For example, performing a loop unrolling transformation followed by a loop splitting transformation may not leave enough room in the instruction buffer after unrolling the loop to perform the loop splitting transformation. Similarly, in another example, performing a loop splitting transformation followed by a loop unrolling transformation may not leave enough room in the instruction buffer after performing the loop splitting transformation.

[0057] Therefore, there is currently no means to effectively reduce program control overhead, such as in the case of a programmable accelerator, using an instruction buffer of limited size.

[0058] Embodiments of the present disclosure provide a means for reducing program control overhead, such as in cases of programmable accelerators, using instruction buffers with limited size by constructing a loop tree from a program, such as a structured control flow program. Structured control flow, as used herein, refers to a programming concept in which the flow of control for a block or region is based on a single-entry and single-exist methodology (SESE). In such a programming concept, during execution, execution of a (structured) statement starts from a defined point, and execution ends at a defined point. Metrics such as estimated cycle reduction and estimated code size increase are calculated for each node in the loop tree, where each node in the tree represents a loop (a loop is a sequence of instructions that is continuously repeated until a specific condition is reached) or an if condition (a conditional statement that executes a block of code if a specified condition is true). As used herein, estimated cycle reduction refers to the number of cycles saved by performing transformations such as loop splitting for an if condition or loop unrolling for a loop. Furthermore, as used herein, an estimated code size increase refers to an increase in code size as a result of performing a transformation such as loop splitting for an if condition or loop unrolling for a loop. Upon identifying a node with a metric associated with the highest gain, such as the largest increase in beneficial execution cycles with the smallest increase in the number of instructions, loop splitting is performed on the loop associated with the condition for the identified node representing the if condition, or loop unrolling is performed on the loop for the identified node representing such a loop, if the resulting code fits into the instruction buffer. As used herein, "highest gain" refers to the ratio of beneficial execution cycles to an increase in code size (number of instructions) that has the highest value.After performing loop splitting or loop unrolling, optimizations such as removing dead code (code that does not affect the outcome of the program) and constant propagation (substituting the value of a known constant in an expression at compile time) may then be performed. The loop tree may then be reconstructed using the parents of the identified nodes. In this manner, a limited-size instruction buffer may be used to reduce program control overhead, such as in the case of a programmable accelerator. Further discussion of these and other features is provided below.

[0059] In some embodiments of the present disclosure, the present disclosure comprises a computer-implemented method, system, and computer program product for reducing overhead control. In one embodiment of the present disclosure, a loop tree is constructed from a program, such as a structured control flow program. As used herein, structured control flow refers to a programming concept in which the flow of control to a block or region is based on a single entry and single-exist methodology (SESE). In such a programming concept, during execution, execution of (structured) statements begins at a single defined point, and execution ends at a single defined point. As used herein, a loop tree refers to a tree-like data structure that graphically represents loops and / or if-conditions in a program, such as a structured control flow program. In one embodiment, a loop tree is constructed from a program by creating a set of classes or structures to represent the nodes and relationships in the tree. Such nodes can then be inserted (nodes added) or deleted (nodes removed) via the AddChild or DeleteChild methods, respectively. If the resulting code fits into the instruction buffer, a loop splitting operation or a loop unrolling operation may then be performed in association with the node of the loop tree identified as having the highest gain (the ratio of execution cycles that benefit per increase in code size (number of instructions) that has the highest value). In one embodiment, if the resulting code fits into the instruction buffer, a loop splitting operation is performed on the loop associated with the condition for the node representing an if condition (the node with the highest gain), or a loop unrolling operation is performed on the loop for the node representing such a loop (the node with the highest gain), if the resulting code fits into the instruction buffer. In this manner, an instruction buffer of limited size may be used to reduce program control overhead, such as in the case of a programmable accelerator.

[0060] In the following description, numerous specific details are set forth to provide a thorough understanding of the present disclosure. However, it will be apparent to those skilled in the art that the present disclosure may be practiced without such specific details. In other instances, well-known circuits are shown in block diagram form so as not to obscure the present disclosure with unnecessary detail. For the most part, details regarding timing considerations and the like have been omitted unless such details are necessary to obtain a complete understanding of the present disclosure and are within the skill of those skilled in the relevant art.

[0061] Referring now in detail to the drawings, Figure 1 illustrates a disclosed embodiment of a communication system 100 for implementing the principles of the present disclosure. Communication system 100 includes computing devices 101A-101C (identified as "computing device A," "computing device B," and "computing device C," respectively, in Figure 1) connected to a software development system 102 via a network 103. Computing devices 101A-101C may be referred to collectively or individually as multiple computing devices 101 or computing device 101, respectively.

[0062] Computing device 101 may be any type of computing device (e.g., a portable computing unit, a personal digital assistant (PDA), a laptop computer, a mobile device, a tablet personal computer, a smartphone, a mobile phone, a navigation device, a gaming unit, a desktop computer system, a workstation, an Internet appliance, and the like) configured with the ability to connect to network 103 and thus communicate with other computing devices 101 and software development system 102. Note that both computing device 101 and the user of computing device 101 may be identified by element number 101.

[0063] Software development system 102 is a system utilized by software programmers, developers (e.g., users of computing device 101), and the like, in the process of creating, designing, deploying, and supporting software, including programs such as structured control flow programs. Examples of such software development systems may include, but are not limited to, RAD Studio®, embold®, Collaborator®, Studio 3T®, NetBeans®, Zend Studio®, Microsoft® Expression Studio, and the like.

[0064] Network 103 may be, for example, a local area network, a wide area network, a wireless wide area network, a circuit-switched telephone network, a Global System for Mobile Communications (GSM) network, a Wireless Application Protocol (WAP) network, a WiFi network, an IEEE 902.11 standard network, various combinations thereof, etc. Other networks (descriptions of which are omitted herein for brevity) may also be used in conjunction with system 100 of FIG. 1 without departing from the scope of this disclosure.

[0065] System 100 further includes a control overhead reducer 104 connected to network 103. Control overhead reducer 104 is configured to reduce the control overhead of a program, such as in the case of a programmable accelerator, using an instruction buffer of limited size. As previously explained, control overhead in program execution refers to the additional computational cost incurred by a program due to the need to perform various control operations, such as branches and loops.

[0066] In one embodiment, control overhead reducer 104 reduces the control overhead of a program by constructing a loop tree from the program, such as a structured control flow program. As used herein, structured control flow refers to a programming concept in which the flow of control to a block or region is based on a single entry and single-exist methodology (SESE). In such a programming concept, during execution, execution of a (structured) statement starts from one defined point, and execution ends at one defined point.

[0067] Additionally, in one embodiment, the control overhead reducer 104 calculates metrics such as an estimated cycle reduction and an estimated code size increase for each node in the loop tree, where each node in the tree represents a loop (a loop is a sequence of instructions that is continuously repeated until a specific condition is reached) or an if condition (a conditional statement that executes a block of code if a specified condition is true). As used herein, an estimated cycle reduction refers to the number of cycles that are reduced by performing a transformation such as loop splitting for an if condition or loop unrolling for a loop. Additionally, as used herein, an estimated code size increase refers to the increase in code size as a result of performing a transformation such as loop splitting for an if condition or loop unrolling for a loop.

[0068] Upon identifying a node with a metric associated with the highest gain, such as the largest increase in beneficial execution cycles with the smallest increase in code size (number of instructions), loop splitting is performed on the loop associated with the condition for the identified node representing an if condition if the resulting code fits into the instruction buffer, or loop unrolling is performed on the loop for the identified node representing such a loop if the resulting code fits into the instruction buffer. As used herein, "highest gain" refers to the ratio of beneficial execution cycles to an increase in code size (number of instructions) that has the highest value. As used herein, "execution cycles" refers to the processor performing actual calculations or manipulations on data. As used herein, "beneficial execution cycles" refers to a reduction in the number of execution cycles that need to be performed.

[0069] When loop splitting or loop unrolling is performed, the control overhead reducer 104 performs optimizations such as removing dead code (code that does not affect the program result) and constant propagation (substituting the value of known constants in expressions at compile time) during each loop splitting or unrolling.

[0070] In one embodiment, the control overhead reducer 104 enhances loop partitioning by considering all conditions of a loop iterator together and partitioning the loop, as opposed to the traditional approach of considering conditions one by one. In one embodiment, a loop is partitioned by considering multiple if conditions when the program has equivalent loop conditions with multiple conditional branches, including after optimization. In one embodiment, when the program has different loop conditions with multiple conditional branches, the number of partitions used to partition the loop iteration space (the iterations executed by the program, such as a structured control flow program) is determined based on the estimated cycle gain and the estimated code size increase. The loop iteration space is then partitioned based on the determined number of partitions.

[0071] Additionally, the control overhead reducer 104 performs speculative analysis to estimate beneficial execution cycles and program size increases. In one embodiment, such analysis involves determining whether a naive approach to estimating beneficial execution cycles and program size increases or using a cost model to estimate beneficial execution cycles and program size increases is a better approach. In one embodiment, when the program size increase is less than or equal to the program size increase using the cost model, the naive approach is selected to be used to estimate beneficial execution cycles and program size increases. Otherwise, the cost model is used to estimate beneficial execution cycles and program size increases. As used herein, a naive approach refers to a simple approach, such as assuming an overall loop execution overhead of one cycle with sufficient pipelining. As used herein, a cost model refers to a model used in the analysis of an algorithm to define what constitutes a single step in the execution of the algorithm. For example, a cost model may assume that if the loop execution overhead is one cycle, but the parent loop has M iterations, then the execution overhead of the child loop is M cycles.

[0072] In one embodiment, the control overhead reducer 104 reconstructs the loop tree using the parent of the node identified as associated with the metric with the highest gain. After reconstructing the loop tree, a metric is calculated for each node in the reconstructed loop tree, as described above.

[0073] A description of the software components of control overhead reducer 104, which is used to reduce control overhead for programs, such as in the case of a programmable accelerator that uses an instruction buffer of limited size, is provided below in conjunction with Figure 3. A description of the hardware configuration of control overhead reducer 104 is further provided below in conjunction with Figure 13.

[0074] System 100 is not limited in scope to any one particular network architecture and may include any number of computing devices 101, software development systems 102, networks 103, and control overhead reducers 104.

[0075] In one embodiment, the control overhead reducer 104 may be utilized in edge computing to reduce the overall time it takes a processor of an edge computing device to complete the tasks described below in connection with FIG. 2.

[0076] Referring to FIG. 2, FIG. 2 illustrates a reduction in the overall time for an edge computing device to complete a task, according to an embodiment of the present disclosure.

[0077] Edge computing, as used herein, is a distributed computing paradigm that brings computation and data storage closer to the source of the data. In particular, in edge computing, data processing, analysis, and storage are performed where rapid, near-real-time analysis and response can be performed. Such data processing, analysis, and storage are performed by an edge computing device 201. Due to constraints on the computing resources of the edge computing device 201, the edge computing device 201 does not perform computationally intensive tasks.

[0078] As a result, computationally intensive tasks may be performed by an external computing device, such as a trainer. In one embodiment, the control overhead reducer 104 corresponds to such a trainer. In one embodiment, the control overhead reducer 104 serves the purpose of generating an inference model 202 from the dataset, which is subsequently used by the edge computing device 201.

[0079] Additionally, in one embodiment, the control overhead reducer 104, functioning as a trainer, processes a description program 203 derived from the neural network. In one embodiment, the description program 203 is used to encapsulate a high-level abstraction of the computations to be performed, such as computations performed on a training dataset.

[0080] In one embodiment, the control overhead reducer 104 includes a compiler 204 configured to convert the description program 203 into an executable program 205 to be executed by the edge computing device 201.

[0081] In one embodiment, while converting the description program 203 into the executable program 205, the control overhead reducer 104 reduces the control overhead in the description program 203, thereby reducing the overall time it takes the processor of the edge computing device 201 to complete its tasks, such as processing the executable program 205. Further discussion regarding such control overhead reduction is provided below.

[0082] Referring now to FIG. 3, FIG. 3 is a diagram of software components used by control overhead reducer 104 to reduce program control overhead, such as in the case of a programmable accelerator, using a size-limited instruction buffer, in accordance with an embodiment of the present disclosure.

[0083] 3, the control overhead reducer 104 includes a construction engine 301 configured to construct a loop tree from a program, such as a structured control flow program. As used herein, structured control flow refers to a programming concept in which the flow of control to a block or region is based on a single entry and single-exist methodology (SESE). In such a programming concept, during execution, execution of a (structured) statement starts from a single defined point, and execution ends at a single defined point.

[0084] As used herein, a loop tree refers to a tree-like data structure that graphically represents loops and / or if-conditions in a program, such as a structured control flow program. In one embodiment, construction engine 301 constructs a loop tree from a program by creating a set of classes or structures to represent the nodes and relationships in the tree. Such nodes can then be inserted (adding nodes) or deleted (removing nodes) via the AddChild or DeleteChild methods, respectively.

[0085] In one embodiment, the build engine 301 builds a loop tree from the program by adding the root node to the queue using the put method and iterating while the queue is not empty. The build engine 301 then gets the first node in the queue and prints its value. Later, the build engine 301 adds both the left and right children to the queue (if the current node has children).

[0086] In one embodiment, the construction engine 301 utilizes various software tools for programmatically constructing loop trees, which may include, but are not limited to, Grpahviz®, Gource, CodeSee®, and the like.

[0087] An example of constructing a loop tree from a program such as a structured control flow program is provided in FIG. 4.

[0088] FIG. 4 shows an example of a loop tree constructed from a program such as a structured control program according to an embodiment of the present disclosure.

[0089] As shown in FIG. 4, the code 401 includes various loops, such as a for loop. In one embodiment, the construction engine 301 constructs a loop tree 402 as a tree-like data structure that graphically represents loops (a loop is a sequence of instructions that is continuously repeated until a specific condition is reached) and / or if conditions (a conditional statement that executes a block of code when a specified condition is true) in a program such as a structured control flow program.

[0090] In one embodiment, each node 403 in the loop tree 402 represents either a loop or an if condition. For example, as shown in FIG. 4, the construction engine 301 graphically represents loops L0, L1, L2, and L3 from the code 401 as nodes 403 in the tree structure 402. Further, the construction engine 301 graphically represents an if condition (If L1C0) as a node 403 in the loop tree 402, where a comparison is made to determine whether L1 < C0 in the code 401. Further, in one embodiment, the loop tree 402 graphically shows the relationships between such nodes 403.

[0091] In one embodiment, each node 403 is associated with content 404 that can correspond to an increase in execution cycles and code size (number of instructions) that is beneficial by performing loop unrolling or loop splitting as further described below. [[ID=1&]]

[0092] In one embodiment, the edges of the loop tree 402 represent control flow.

[0093] Referring again to FIG. 3 in conjunction with FIGS. 1 and 4, the control overhead reducer 104 further includes a computing engine 302 configured to calculate metrics for each node 403 in the loop tree 402. For example, the computing engine 302 may calculate metrics corresponding to an estimated cycle reduction and an estimated code size increase for each node 403 in the loop tree 402. As used herein, an estimated cycle reduction refers to the number of cycles reduced by performing a transformation, such as loop splitting for an if condition or loop unrolling for a loop. Furthermore, as used herein, an estimated code size increase refers to the increase in code size as a result of performing a transformation, such as loop splitting for an if condition or loop unrolling for a loop. While such transformations are performed to determine metrics associated with each node 403 in the loop tree 402, it should be noted that such transformations may not actually be implemented in the final code version. As further described below, transformations are implemented in the final code version for those nodes 403 that rank in the metrics with the highest gains.

[0094] As used herein, loop unrolling (also known as loop unrolling) is a loop transformation technique that attempts to optimize a program's execution speed at the expense of its binary size, a technique known as a space-time tradeoff. In one embodiment, the transformation is performed by the computing engine 302 via an optimizing compiler. As used herein, an optimizing compiler is a compiler that attempts to minimize or maximize some attribute of an executable computer program. For example, an optimizing compiler may be configured to perform loop unrolling, which aims to increase program speed by reducing or eliminating loop-control instructions, such as pointer arithmetic and "end-of-loop" tests for each iteration, reducing branch penalties, and hiding latency, including delays when reading data from memory. To remove this computation / control overhead, loops may be rewritten as a repeated sequence of similar independent statements.

[0095] Loop partitioning, as used herein, is a computer optimization technique that attempts to simplify a loop or remove dependencies by decomposing it into multiple loops that have the same body but iterate over different consecutive portions of an index range.

[0096] In one embodiment, the computing engine 302 utilizes a performance analysis tool, such as LoopTool or LoopReport by Oracle®, to read files created by these compilers that perform loop unrolling and loop splitting. Such analysis tools can be used to evaluate the cycle reduction (or beneficial execution cycles) and program size increase by performing such transformations. As used herein, "beneficial execution cycles" refers to a reduction in the number of execution cycles that need to be executed, such as as a result of a transformation (e.g., loop splitting, loop unrolling). As used herein, "program size" refers to the number of lines of code that increase as a result of a transformation (e.g., loop splitting, loop unrolling).

[0097] In one embodiment, the computing engine 302 performs speculative analysis to calculate such metrics, such as estimating beneficial execution cycles and program size increase. In one embodiment, such analysis involves determining whether a naive approach to estimating beneficial execution cycles and program size increase or using a cost model to estimate beneficial execution cycles and program size increase is a better approach. In one embodiment, when the code size increase is less than or equal to the code size increase using a cost model, the naive approach is selected to be used to estimate beneficial execution cycles and program size increase. Otherwise, the cost model is used to estimate beneficial execution cycles and program size increase. As used herein, a naive approach refers to a simple approach, such as assuming an overall loop execution overhead of one cycle with sufficient pipelining. As used herein, a cost model refers to a model used in the analysis of an algorithm to define what constitutes a single step in the execution of the algorithm. For example, a cost model may assume that if the loop execution overhead is one cycle, but the parent loop has M iterations, then the execution overhead of the child loop is M cycles.

[0098] An example of computing engine 302 that calculates metrics for nodes that represent loops in a program (eg, node 403) is provided in FIG.

[0099] FIG. 5 illustrates calculating metrics for a node (eg, 403) that represents a loop in a program, according to an embodiment of the present disclosure.

[0100] 5 in conjunction with FIGS. 1-4, node 403 in tree structure 402 may be associated with a loop, such as represented by code 501. Because such a node is associated with a loop, computing engine 302 performs a loop unrolling transformation on code 501.

[0101] In one embodiment, the loop unrolling transformation for code 501 involves rewriting the loop in code 501 as a repeated sequence of similar independent statements shown in result code 502.

[0102] As described above, such loop unrolling transformations may be performed by an optimizing compiler, where files created for such transformations are analyzed using a performance analysis tool, such as LoopTool or LoopReport by Oracle®. Such analysis tools may be utilized to evaluate beneficial execution cycle and program size increases. For example, in the example of FIG. 5, it is assumed that code block A() contains N instructions and requires K cycles to execute those N instructions. In one embodiment, such analysis to evaluate beneficial execution cycle and program size increases may utilize a naive approach where, with sufficient pipelining, the overall loop execution overhead is one cycle. In one embodiment, such analysis may utilize a cost model that assumes loop execution overhead is one cycle, but if a parent loop (e.g., an m-loop) has M iterations, then the execution overhead of a child loop (e.g., an i-loop) is M cycles.

[0103] Based on the analysis performed by the analysis tool, unrolling the loop surrounding code block A() results in a code size increase from N+2 to 3N+1, but the execution cycles change from 1+M(KN+1) to 1+M*KN, thereby saving M cycles at the cost of a 2N-1 increase in code size.

[0104] An example of computing engine 302 calculating metrics for a node (eg, node 403) that represents an if condition in a program is provided in FIG.

[0105] FIG. 6 illustrates computing metrics for a node (eg, node 403) that represents an if condition in a program, according to an embodiment of the present disclosure.

[0106] 6 in conjunction with FIGS. 1-4, node 403 in tree structure 402 may be associated with an if condition, such as represented by code 601. Because such a node is associated with the if condition, computing engine 302 performs a loop partitioning transformation on code 601.

[0107] In one embodiment, the loop splitting transformation for code 601 involves decomposing the loop into multiple loops that have the same body but iterate over different consecutive portions of the index range, as shown in result code 602.

[0108] As explained above, such loop splitting transformations may be performed by an optimizing compiler, where the files created for such transformations are analyzed using a performance analysis tool, such as LoopTool or LoopReport by Oracle®. Such analysis tools may be utilized to evaluate the beneficial execution cycle and program size increase. For example, in the example of FIG. 6, the code size remains unchanged (a+b+2), but the number of executions changes from 1+M(1+2(a+2)+(b+1)) to 1+M(2(a+1)+(b+1)), saving 2M execution cycles.

[0109] In one embodiment, such metrics (profitable execution cycles, program size increase) calculated by the computing engine 302 may be reflected in the loop tree 402 shown in FIG. 7, for example.

[0110] FIG. 7 illustrates inputting metrics in a constructed loop tree according to an embodiment of the present disclosure.

[0111] Referring to FIG. 7, metrics 701 are entered into the content 404 in the loop tree 402 by the computing engine 302 after the analysis of the transformation is performed as described above.

[0112] Referring again to FIG. 3 in conjunction with FIGS. 1 and 4-7, the control overhead reducer 104 further includes an analysis engine 303 configured to identify a node (e.g., node 403) in the loop tree 402 with a metric associated with the highest gain. As described above, metrics such as estimated beneficial execution cycles and estimated code size increase are calculated for each node (e.g., node 403) in the loop tree (e.g., loop tree 402), where each node in the loop tree represents a loop (a loop is a sequence of instructions that is continuously repeated until a particular condition is reached) or an if condition (a conditional statement that executes a block of code if a specified condition is true). As used herein, "beneficial execution cycles" refers to a reduction in the number of execution cycles that need to be executed, such as as a result of a transformation (e.g., loop splitting, loop unrolling). Furthermore, as used herein, estimated code size increase refers to an increase in code size as a result of performing a transformation, such as loop splitting for an if condition or loop unrolling for a loop. As used herein, "highest gain" refers to the ratio of beneficial execution cycles to an increase in code size (number of instructions) that has the highest value.

[0113] In one embodiment, the analysis engine 303 identifies the node 403 in the loop tree 402 with the metric associated with the highest gain by analyzing the metrics 701 in the loop tree 402 and identifying the metric with the ratio of execution cycles that benefit to an increase in program size that has the highest value.

[0114] Upon identifying a node (e.g., node 403) in a loop tree (e.g., loop tree 402) with a metric associated with the highest value, analysis engine 303 performs loop splitting on the loop associated with the condition if node 403 represents an if condition. In one embodiment, analysis engine 303 performs loop splitting on the loop associated with the condition if node 403 is associated with an if condition and, particularly for programs of a programmable accelerator, if the resulting code fits within an instruction buffer of limited size. As explained above, loop splitting, as used herein, is a computer optimization technique that attempts to simplify a loop or remove dependencies by decomposing it into multiple loops that have the same body but iterate over different consecutive portions of an index range. As explained above, such transformations are performed on the final version of the code.

[0115] In one embodiment, upon identifying a node (e.g., node 403) in a loop tree (e.g., loop tree 402) with a metric associated with the highest value, analysis engine 303 performs loop splitting on the loop represented by node 403 if node 403 represents a loop rather than an if condition. In one embodiment, analysis engine 303 performs loop unrolling on the loop represented by node 403 if node 403 represents a loop rather than an if condition, and if the resulting code fits into an instruction buffer of limited size, particularly for programs on a programmable accelerator. As described above, loop unrolling, as used herein, is a loop transformation technique that attempts to optimize the execution speed of a program at the expense of its binary size, a technique known as a space-time tradeoff. In one embodiment, the transformation is performed by analysis engine 303 via an optimizing compiler. As described above, such transformations are performed on the final version of the code.

[0116] In one embodiment, analysis engine 303 is configured to enhance loop partitioning by partitioning a loop by considering all conditions of a loop iterator together, as opposed to the traditional approach of considering conditions one at a time.

[0117] In one embodiment, the analysis engine 303 enhances loop partitioning by splitting the loop considering multiple if conditions when the program (e.g., a structured control flow program) has equivalent loop conditions with multiple conditional branches as shown in FIG. 8, including after optimization (described further below).

[0118] FIG. 8 illustrates splitting a loop considering multiple if conditions when a structured control flow program has an equivalent loop condition with multiple conditional branches, according to an embodiment of the present disclosure.

[0119] 8, code 801 corresponds to the original code of a program, such as a structured control flow program, that includes equivalent loop conditions with multiple conditional branches (e.g., i-loop<10 and i-loop≦9). In such a scenario, analysis engine 303 considers multiple if conditions and splits the loop (e.g., i-loop=0 to 9 and i-loop=10 to M) as shown in result code 802. Considering multiple conditions together can result in better decision results, including cost models that help recognize potential benefits from dead code elimination (removing code that does not affect program outcome).

[0120] In one embodiment, when a program, such as a structured control flow program, has different loop conditions with multiple conditional branches as shown in FIG. 9, analysis engine 303 enhances loop partitioning by dividing the loop iteration space (the iterations executed by the structured control flow program) into multiple partitions.

[0121] FIG. 9 illustrates dividing a loop iteration space into partitions when a program, such as a structured control flow program, has different loop conditions with multiple conditional branches, according to an embodiment of the present disclosure.

[0122] 9, code 901 corresponds to the original code of a program, such as a structured control flow program, that includes different loop conditions with multiple conditional branches (e.g., i-loop=0 to 10 and j-loop=0 to M). In such a scenario, analysis engine 303 divides the loop iteration space (e.g., i-loop iteration space) into partitions (e.g., i-loop=0 to 1, i-loop=2 to 5, i-loop=6 to 8, and i-loop=9 to 10), as shown in result code 902.

[0123] However, dividing the loop iteration space into partitions may not fit into the instruction buffer. As a result, a smaller number of partitions may need to be utilized.

[0124] In one embodiment, the analysis engine 303 builds a table (referred to herein as a "partition table") for loops based on conditions with reference to the loop iterator shown in FIG.

[0125] FIG. 10 illustrates a partitioning table for determining the number of partitions to use in dividing the loop iteration space in the program of FIG. 9 according to an embodiment of the present disclosure.

[0126] Referring to FIG. 10 in conjunction with FIG. 9, FIG. 10 illustrates a partitioning table 1000 for a loop based on all conditions (eg, <2) referencing the loop iterator (eg, i-loop) of code 901.

[0127] As shown in FIG. 10 , column 1001 contains the iteration variable (iv) values, column 1002 indicates which iterations (e.g., iteration 0, iteration 1) have iteration values less than 2, column 1003 indicates which iterations (e.g., iteration 2, iteration 3, iteration 4, iteration 5, iteration 6, iteration 7, iteration 8, iteration 9) have iteration values greater than or equal to 2, column 1004 indicates which iterations (e.g., iteration 0, iteration 1, iteration 2, iteration 3, iteration 4) have iteration values less than 5, column 1005 indicates which iterations (iteration 9) have iteration values greater than 8, and column 1006 indicates which iterations (iteration 0, iteration 1, iteration 2, iteration 3, iteration 4, iteration 5, iteration 6, iteration 7, iteration 8) have iteration values less than or equal to 8. In one embodiment, such columns 1002-1006 indicate which iterations meet a criterion (e.g., an iteration value less than 2) based on having a logical value of one in the column identifying the iteration in the corresponding row of partition table 1000.

[0128] In one embodiment, the analysis engine 303 identifies partitions by grouping consecutive repeating variable rows using a set of conditions on the columns of a partitioning table 1000 identical to that shown in FIG.

[0129] FIG. 11 illustrates identifying partitions by grouping consecutive repeating variable rows in partitioned table 1000 that have the same set of conditions on columns (e.g., columns 1002-1006) of partitioned table 1000, according to an embodiment of the present disclosure.

[0130] As shown in FIG. 11, grouped rows 1101, 1102, 1103, and 1104 of partitioned table 1000 correspond to those repeat variable rows with the same set of conditions in columns 1002-1006. For example, grouped row 1101 corresponds to the grouping of repeat variable rows 0 and 1. Grouped row 1102 corresponds to the grouping of repeat variable rows 2, 3, and 4. Grouped row 1103 corresponds to the grouping of repeat variable rows 5, 6, 7, and 8. Grouped row 1104 corresponds to a single repeat variable row, row 9.

[0131] Because not all of the partitions computed from the grouping may fit into the instruction buffer, a metric such as a cost metric may be utilized to determine an optimal set of partitions that fit into the instruction buffer and maximize cycle reduction. In one embodiment, such a metric corresponds to estimated beneficial execution cycles and estimated code size increment. In one embodiment, the estimated beneficial execution cycles correspond to (number of partitioned IfOps) * 2 * outer bound. As used herein, IfOps refer to if operations in the code. As used herein, outer bound refers to a loop bound, which is an upper limit on the number of loop iterations for a given program. In one embodiment, the estimated code size increment corresponds to (code size not within the partition) * 2. In one embodiment, the cost metric corresponds to the ratio of beneficial execution cycles to code size increment. An illustration of selecting a partition (e.g., partitions 1101, 1102) based on estimated beneficial execution cycles and estimated code size increment is provided in FIG. 12 .

[0132] FIG. 12 illustrates selecting a partition (eg, partitions 1101, 1102) based on estimated beneficial execution cycles and estimated code size increments, according to an embodiment of the present disclosure.

[0133] 12 in conjunction with FIGS. 10-11, partition table 1000 includes a column 1201 for a cost metric (e.g., A, B, C, D) associated with each of the partitions (e.g., partitions 1101, 1102, 1103, 1104). In one embodiment, analysis engine 303 selects an optimal set of partitions (e.g., partitions 1101, 1102) that fit into a limited-size instruction buffer based on those partitions that maximize profitable execution cycles while limiting code size growth.

[0134] After analysis engine 303 determines the optimal set or number of partitions that fit into the size-limited instruction buffer, analysis engine 303 divides the loop iteration space for programs having different loop conditions with multiple conditional branches (e.g., structured control flow programs) based on the determined number of partitions, as described above in connection with FIG. 9.

[0135] The control overhead reducer 104 further includes an optimization engine 304 that is configured to perform optimizations on the result code (the code after the transformations have been performed), such as removing dead code (code that does not affect the program outcome), constant propagation (substituting values for known constants in expressions at compile time), etc.

[0136] In one embodiment, optimization engine 304 utilizes various software tools to perform such optimizations, which may include, but are not limited to, LinearB®, Aivosto's Project Analyzer, SCARF, etc. Additionally, in one embodiment, optimization engine 304 utilizes an optimizing compiler to perform such optimizations.

[0137] In one embodiment, upon optimizing the resulting code, the build engine 301 rebuilds the loop tree 402 starting at the parent of the node 403 identified using the metric associated with the highest gain described above. That is, the build engine 301 rebuilds the loop tree 402 using the root node of the tree that corresponds to the parent of the node 403 identified using the metric associated with the highest gain.

[0138] In this manner, an instruction buffer of limited size may be used to reduce program control overhead, such as in the case of a programmable accelerator.

[0139] Further description of these and other features is provided below in connection with a description of a method for using a size-limited instruction buffer to reduce control overhead for programs, such as in the case of a programmable accelerator.

[0140] Before describing a method for reducing control overhead for a program using a size-limited instruction buffer, such as in the case of a programmable accelerator, a description of the hardware configuration of control overhead reducer 104 (FIG. 1) is provided below in connection with FIG. 13.

[0141] Referring now to FIG. 13 in conjunction with FIG. 1, FIG. 13 illustrates an embodiment of the present disclosure of a hardware configuration for control overhead reducer 104, which represents a hardware environment for implementing the present disclosure.

[0142] Various aspects of the present disclosure are described through narrative text, flowcharts, block diagrams of computer systems, and / or block diagrams of machine logic included in computer program product (CPP) embodiments. For any flowchart, depending on the technology involved, operations may be performed in an order different from that shown in a given flowchart. For example, again depending on the technology involved, two operations shown in successive flowchart blocks may be performed in reverse order, as a single integrated step, simultaneously, or in an at least partially overlapping manner.

[0143] A computer program product embodiment ("CPP embodiment" or "CPP") is a term used in this disclosure to describe any set of one or more storage media (also referred to as "media") collectively included in a set of one or more storage devices that collectively contain machine-readable code corresponding to instructions and / or data for performing computer operations specified in a given CPP claim. A "storage device" is any tangible device that can hold and store instructions for use by a computer processor. The computer-readable storage medium may be, but is not limited to, an electronic storage medium, a magnetic storage medium, an optical storage medium, an electromagnetic storage medium, a semiconductor storage medium, a mechanical storage medium, or any suitable combination of the foregoing. Some known types of storage devices that include these media include diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), compact disc read-only memory (CD-ROM), digital versatile disk (DVD), memory stick, floppy disk, mechanically encoded device (such as pits / lands formed on a major surface of a punch card or disk), or any suitable combination of the foregoing. Computer-readable storage media, as the term is used in this disclosure, is not to be construed as storage in the form of a transitory signal per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide, light pulses passing through fiber optic cables, electrical signals communicated over wires, and / or other transmission media. As those skilled in the art will appreciate, data is typically moved at some infrequent time during the normal operation of a storage device, such as during access, defragmentation, or garbage collection, but the above does not make a storage device transient because the data is not transient while it is stored.

[0144] Computing environment 1300 includes an example environment for executing at least some of the computer code (stored in block 1301) associated with performing the disclosed methods, such as using instruction buffers of limited size to reduce control overhead for programs, such as in the case of a programmable accelerator. In addition to block 1301, computing environment 1300 includes, for example, a control overhead reducer 104, a network 103, such as a wide area network (WAN), an end user device (EUD) 1302, a remote server 1303, a public cloud 1304, and a private cloud 1305. In this embodiment, the control overhead reducer 104 includes a set of processors 1306 (including processing circuitry 1307 and cache 1308), a communication fabric 1309, volatile memory 1310, persistent storage 1311 (including an operating system 1312 and the above-identified blocks 1301), a set of peripheral devices 1313 (including a set of user interface (UI) devices 1314, storage 1315, and an Internet of Things (IoT) sensor set 1316), and a network module 1317. The remote server 1303 includes a remote database 1318. The public cloud 1304 includes a gateway 1319, a cloud orchestration module 1320, a set of host physical machines 1321, a set of virtual machines 1322, and a set of containers 1323.

[0145] The control overhead reducer 104 may take the form of a desktop computer, a laptop computer, a tablet computer, a smartphone, a smartwatch or other wearable computer, a mainframe computer, a quantum computer, or any other form of computer or mobile device now known or later developed that is capable of executing programs, accessing a network, or querying a database, such as the remote database 1318. As is well understood in the field of computer technology, and depending on the technology, execution of a computer-implemented method may be distributed among multiple computers and / or among multiple locations. However, in this presentation of the computing environment 1300, the detailed description focuses on a single computer, specifically the control overhead reducer 104, to keep the presentation as simple as possible. Although the control overhead reducer 104 is not shown in the cloud in FIG. 13, it may be located in the cloud. However, the control overhead reducer 104 is not required to be in the cloud unless expressly indicated.

[0146] Processor set 1306 includes one or more computer processors of any type now known or later developed. Processing circuitry 1307 may be distributed across multiple packages, e.g., multiple tailored integrated circuit chips. Processing circuitry 1307 may implement multiple processor threads and / or multiple processor cores. Cache 1308 is memory located within the processor chip package and is typically used for data or code that should be available for fast access by threads or cores executing on processor set 1306. Cache memory is typically organized into multiple levels depending on relative proximity to the processing circuitry. Alternatively, some or all of the cache for a processor set may be located “off-chip.” In some computing environments, processor set 1306 may be designed to operate with qubits and perform quantum computing.

[0147] Computer-readable program instructions are typically loaded into the control overhead reducer 104 and cause a series of operational steps to be performed by the processor set 1306 of the control overhead reducer 104, thereby affecting a computer-implemented method; the instructions so executed instantiate the method specified in the flowcharts and / or narrative descriptions of the computer-implemented method contained in this document (collectively referred to as the "disclosed method"). These computer-readable program instructions are stored in various types of computer-readable storage media, such as cache 1308 and other storage media discussed below. The program instructions and associated data are accessed by the processor set 1306 to control and direct the execution of the disclosed method. In the computing environment 1300, at least some of the instructions for performing the disclosed method may be stored in block 1301 in persistent storage 1311.

[0148] The communications fabric 1309 is a signaling path that allows the various components of the control overhead reducer 104 to communicate with each other. Typically, this fabric is made of switches and conductive paths, such as switches and conductive paths that make up buses, bridges, physical input / output ports, and the like. Other types of signaling paths may be used, such as fiber optic and / or wireless communication paths.

[0149] Volatile memory 1310 may be any type of volatile memory, now known or later developed. Examples include dynamic random access memory (RAM) or static RAM. Typically, volatile memory is characterized by random access, although this is not required unless expressly indicated. In control overhead reducer 104, volatile memory 1310 is located in a single package and is internal to control overhead reducer 104; however, alternatively or additionally, volatile memory may be distributed across multiple packages and / or located external to control overhead reducer 104.

[0150] Persistent storage 1311 is any form of non-volatile storage for a computer, now known or later developed. The non-volatility of this storage means that stored data is maintained regardless of whether power is supplied to control overhead reducer 104 and / or to persistent storage 1311 directly. While persistent storage 1311 can be read-only memory (ROM), typically at least a portion of persistent storage allows data to be written, data to be deleted, and data to be rewritten. Some well-known forms of persistent storage include magnetic disks and solid-state storage devices. Operating system 1312 may take several forms, such as various known proprietary operating systems or open-source Portable Operating System Interface-type operating systems that utilize a kernel. The code contained in block 1301 typically includes at least some of the computer code associated with performing the disclosed methods.

[0151] The peripheral device set 1313 includes a set of peripheral devices of the control overhead reducer 104. Data communication connections between the peripheral devices and other components of the control overhead reducer 104 can be implemented by various means, such as Bluetooth® connections, near field communication (NFC) connections, cabled connections (such as universal serial bus (USB) type cables), insertion-type connections (e.g., Secure Digital (SD) cards), connections over local area communication networks, and even connections over wide area networks such as the Internet. In various embodiments, the UI device set 1314 can include components such as display screens, speakers, microphones, wearable devices (such as goggles and smartwatches), keyboards, mice, printers, touchpads, game controllers, and haptic devices. The storage 1315 can be external storage, such as an external hard drive, or insertable storage, such as an SD card. The storage 1315 can be persistent and / or volatile. In some embodiments, storage 1315 may take the form of a quantum computing storage device for storing data in the form of qubits. In embodiments where control overhead reducer 104 is required to have a large amount of storage (e.g., control overhead reducer 104 stores and manages a large database locally), this storage may be provided by a peripheral storage device designed to store very large amounts of data, such as a storage area network (SAN) shared by multiple geographically distributed computers. IoT sensor set 1316 consists of sensors that can be used in Internet of Things applications. For example, one sensor may be a thermometer and another sensor may be a motion detector.

[0152] The network module 1317 is a collection of computer software, hardware, and firmware that enables the control overhead reducer 104 to communicate with other computers over the WAN 103. The network module 1317 may include hardware such as a modem or Wi-Fi® signal transceiver, software for packetizing and / or depacketizing data for communication network transmission, and / or web browser software for communicating data over the Internet. In some embodiments, the network control and network forwarding functions of the network module 1317 are performed on the same physical hardware device. In other embodiments (e.g., embodiments utilizing Software-Defined Networking (SDN)), the control and forwarding functions of the network module 1317 are performed on physically separate devices, such that the control function manages several different network hardware devices. Computer-readable program instructions for performing the disclosed methods may be downloaded to the control overhead reducer 104 from an external computer or external storage device, typically through a network adapter card or network interface included in the network module 1317.

[0153] WAN 103 is any wide area network (e.g., the Internet) capable of communicating computer data over non-local distances by any technology for communicating computer data now known or later developed. In some embodiments, a WAN may be replaced and / or supplemented by a local area network (LAN) designed to communicate data between devices located in a local area, such as a Wi-Fi network. WANs and / or LANs typically include copper transmission cables, optical fiber transmissions, wireless transmissions, and computer hardware such as routers, firewalls, switches, gateway computers, and edge servers.

[0154] The end-user device (EUD) 1302 is any computer system used and controlled by an end user (e.g., a customer of the enterprise operating the control overhead reducer 104) and may take any of the forms described above in connection with the control overhead reducer 104. The EUD 1302 typically receives useful and actionable data from the operation of the control overhead reducer 104. For example, in the hypothetical case where the control overhead reducer 104 is designed to provide recommendations to the end user, the recommendations would typically be communicated to the EUD 1302 from the network module 1317 of the control overhead reducer 104 over the WAN 103. In this manner, the EUD 1302 can display or otherwise present the recommendations to the end user. In some embodiments, the EUD 1302 may be a client device such as a thin client, a heavy client, a mainframe computer, a desktop computer, and the like.

[0155] The remote server 1303 is any computer system that services at least some data and / or functionality to the control overhead reducer 104. The remote server 1303 may be controlled and used by the same entity that operates the control overhead reducer 104. The remote server 1303 represents a machine that collects and stores useful and useful data for use by other computers, such as the control overhead reducer 104. For example, in the hypothetical case where the control overhead reducer 104 is designed and programmed to provide recommendations based on historical data, this historical data may be provided to the control overhead reducer 104 from a remote database 1318 of the remote server 1303.

[0156] Public cloud 1304 is any computer system available for use by multiple entities that provides on-demand availability of computer system resources and / or other computer functionality, particularly data storage (cloud storage) and computing power, without direct, active management by users. Cloud computing typically leverages resource sharing to achieve coherence and economies of scale. Direct, active management of public cloud 1304's computing resources is performed by computer hardware and / or software in cloud orchestration module 1320. The computing resources provided by public cloud 1304 are typically implemented by virtual computing environments running on various computers comprising host physical machine set 1321, which is the universe of physical computers within and / or available in public cloud 1304. Virtual computing environments (VCEs) typically take the form of virtual machines from virtual machine set 1322 and / or containers from container set 1323. It is understood that these VCEs may be stored as images and transferred among and between various hosts of physical machines either as images or after instantiation of the VCE. Cloud orchestration module 1320 manages the transfer and storage of images, deploys new instantiations of VCEs, and manages active instantiations of VCE deployments. Gateway 1319 is a collection of computer software, hardware, and firmware that enables public cloud 1304 to communicate over WAN 103.

[0157] Some further description of virtualized computing environments (VCEs) is now provided. A VCE can be stored as an "image." A new, active instance of a VCE can be instantiated from the image. Two well-known types of VCEs are virtual machines and containers. A container is a VCE that uses operating system-level virtualization. This refers to a feature of an operating system in which the kernel allows the existence of multiple isolated user space instances, called containers. These isolated user space instances typically behave as actual computers from the perspective of programs running within them. A computer program running on a typical operating system can utilize all of the computer's resources, such as connected devices, files and folders, network shares, CPU power, and quantifiable hardware capabilities. However, a program running inside a container can only use the contents of the container and of the devices assigned to the container; this feature is known as containerization.

[0158] Private cloud 1305 is similar to public cloud 1304, except that its computing resources are available only for use by a single enterprise. While private cloud 1305 is depicted as communicating with WAN 103 in other embodiments, a private cloud may be completely disconnected from the Internet and accessible only through a local / private network. A hybrid cloud is a composite of multiple clouds of different types (e.g., private, community, or public cloud types), often implemented by different vendors. While each of the multiple clouds remains a separate, discrete entity, the larger hybrid cloud architecture is bound together by standardized or proprietary technologies that enable orchestration, management, and / or data / application portability between the constituent clouds. In this embodiment, public cloud 1304 and private cloud 1305 are both part of a larger hybrid cloud.

[0159] Block 1301 further includes the software components described above in connection with FIGS. 3-12 for reducing program control overhead, such as in the case of a programmable accelerator, using a size-limited instruction buffer. In one embodiment, such components may be implemented in hardware. The functions discussed above performed by such components are not general-purpose computer functions. As a result, control overhead reducer 104 is machine specific, resulting in the implementation of specific, non-general-purpose computer functions.

[0160] In one embodiment, the functionality of such software components of the control overhead reducer 104, including functionality for reducing program control overhead, such as in the case of a programmable accelerator, using instruction buffers of limited size, may be integrated into an application specific integrated circuit.

[0161] As mentioned above, branching is the process of making decisions in a program by selecting one of two or more paths based on a condition. Overhead is the additional processing time required to evaluate the condition and determine the correct path to follow. A loop is associated with repeating a set of instructions multiple times. Overhead is the additional processing time required to execute each iteration of the loop. Although control overhead is necessary for program execution, excessive control overhead can result in slower program execution, increased memory usage, and other performance issues. One classic transformation to reduce control overhead is loop unrolling. Loop unrolling involves reducing the number of iterations a loop executes by executing multiple iterations (the unrolling factor) of the loop in a single pass. In loop unrolling, the loop is completely unrolled, so that each iteration of the loop is replaced by a single instance of the loop body. Unfortunately, loop unrolling results in an increase in the size of the result code, which can negatively impact cache efficiency and instruction fetching. For example, in the case of a programmable accelerator, instructions related to a loop unrolling transformation may be stored in an instruction buffer (a storage for holding instructions), which may be limited in size. As a result, the instruction buffer may not have enough space to store such instructions. Another classic transformation for reducing control overhead is loop splitting. Loop splitting involves splitting a loop into multiple subloops, each of which has a different loop index set, so that each subloop executes a different subset of the original loop iterations. In particular, loop splitting reduces the number of branches and conditional statements required to manage loop control. By splitting a loop into smaller subloops, each subloop can be executed without the need for complex control statements. Unfortunately, loop splitting increases the size of the result code. In the case of a programmable accelerator, instructions related to a loop splitting transformation, similar to a loop unrolling transformation, may be stored in an instruction buffer, which may be limited in size.As a result, the instruction buffer may not have enough space to store such instructions. Thus, loop unrolling and loop splitting transformations may compete for space in the instruction buffer, and it is not always optimal to perform one before the other. For example, performing a loop unrolling transformation followed by a loop splitting transformation may not leave enough room in the instruction buffer after unrolling the loop to perform the loop splitting transformation. Similarly, in another example, performing a loop splitting transformation followed by a loop unrolling transformation may not leave enough room in the instruction buffer after performing the loop unrolling transformation. Thus, there is currently no means for effectively reducing program control overhead, such as in the case of a programmable accelerator, using an instruction buffer of limited size.

[0162]

[0004] Embodiments of the present disclosure provide means for reducing program control overhead, such as in the case of a programmable accelerator, using a size-limited instruction buffer, as described below in connection with Figures 14-15. Figure 14 is a flowchart of a method for reducing program control overhead, such as in the case of a programmable accelerator, using a size-limited instruction buffer. Figure 15 is a flowchart of a method for enhancing loop partitioning.

[0163] As noted above, FIG. 14 is a flowchart of a method 1400 for using a size-limited instruction buffer to reduce program control overhead, such as in the case of a programmable accelerator, according to an embodiment of the present disclosure.

[0164] Referring to FIG. 14 in conjunction with FIGS. 1-13, during operation 1401, construction engine 301 of control overhead reducer 104 constructs a loop tree (eg, loop tree 402) from a program, such as a structured control flow program.

[0165] As mentioned above, structured control flow as used herein refers to a programming concept in which the flow of control for a block or region is based on a single entry and single-exist methodology (SESE), in which, during execution, the execution of (structured) statements starts from one defined point and the execution ends at one defined point.

[0166] As used herein, a loop tree (e.g., loop tree 402) refers to a tree-like data structure that graphically represents loops and / or if-conditions in a program, such as a structured control flow program. In one embodiment, construction engine 301 constructs a loop tree (e.g., loop tree 402) from a program by creating a set of classes or structures to represent the nodes (e.g., node 403) and relationships in the tree. Such nodes (e.g., node 403) can then be inserted (add a node) or deleted (remove a node) via the AddChild or DeleteChild methods, respectively.

[0167] In one embodiment, the build engine 301 builds a loop tree (loop tree 402) from the program by adding the root node to a queue using the put method and iterating while the queue is not empty. The build engine 301 then gets the first node in the queue and prints its value. Later, the build engine 301 adds both the left and right children to the queue (if the current node has children).

[0168] In one embodiment, the construction engine 301 utilizes various software tools to programmatically construct loop trees (e.g., loop tree 402), which may include, but are not limited to, Grpahviz®, Gource, CodeSee®, etc.

[0169] An example of constructing a loop tree (e.g., loop tree 402) from a program such as a structured control flow program is provided in FIG. 4.

[0170] As shown in FIG. 4, code 401 includes various loops such as for loops. In one embodiment, construction engine 301 constructs loop tree 402 as a tree-like data structure that graphically represents loops (loops are sequences of instructions that are continuously repeated until a specific condition is reached) and / or if conditions (conditional statements that execute a block of code when a specified condition is true) in a program such as a structured control flow program.

[0171] In one embodiment, each node 403 in loop tree 402 represents either a loop or an if condition. For example, as shown in FIG. 4, construction engine 301 graphically represents loops L0, L1, L2, and L3 from code 401 as nodes (e.g., node 403) in tree structure 402. Further, construction engine 301 graphically represents an if condition (If L1C0) as a node (e.g., node 403) in loop tree 402, where a comparison is made to determine if L1 < C0 in code 401. Further, in one embodiment, loop tree 402 graphically shows the relationships between such nodes (e.g., node 403).

[0172] In one embodiment, each node 403 is associated with content 404 that can correspond to an increase in execution cycles and code size (number of instructions) that is beneficial by performing loop unrolling or loop splitting as further described below.

[0173] In one embodiment, the edges of loop tree 402 represent control flow.

[0174] During operation 1402 , the computing engine 302 of the control overhead reducer 104 calculates a metric for each node 403 in the loop tree 402 .

[0175] As described above, for example, the computing engine 302 may calculate metrics corresponding to an estimated cycle reduction and an estimated code size increase for each node 403 in the loop tree 402. As used herein, an estimated cycle reduction refers to the number of cycles reduced by performing a transformation, such as loop splitting for an if condition or loop unrolling for a loop. Furthermore, as used herein, an estimated code size increase refers to the code size increase as a result of performing a transformation, such as loop splitting for an if condition (a conditional statement that executes a block of code if a specified condition is true) or loop unrolling for a loop (a loop is a sequence of instructions that is repeated continuously until a particular condition is reached). While such transformations are performed to determine a metric associated with each node 403 in the loop tree 402, it should be noted that such transformations may not actually be implemented in the final code version. As described herein, transformations are implemented in the final code version only for those nodes 403 associated with the metric with the highest gain.

[0176] As used herein, loop unrolling (also known as loop unrolling) is a loop transformation technique that attempts to optimize a program's execution speed at the expense of its binary size, a technique known as a space-time tradeoff. In one embodiment, the transformation is performed by the computing engine 302 via an optimizing compiler. As used herein, an optimizing compiler is a compiler that attempts to minimize or maximize some attribute of an executable computer program. For example, an optimizing compiler may be configured to perform loop unrolling, which aims to increase program speed by reducing or eliminating loop-control instructions, such as pointer arithmetic and "end-of-loop" tests for each iteration, reducing branch penalties, and hiding latency, including delays when reading data from memory. To remove this computation / control overhead, loops may be rewritten as a repeated sequence of similar independent statements.

[0177] Loop partitioning, as used herein, is a computer optimization technique that attempts to simplify a loop or remove dependencies by decomposing it into multiple loops that have the same body but iterate over different consecutive portions of an index range.

[0178] In one embodiment, the computing engine 302 utilizes a performance analysis tool, such as LoopTool or LoopReport by Oracle®, to read files created by these compilers that perform loop unrolling and loop splitting. Such analysis tools can be used to evaluate the cycle reduction (or beneficial execution cycles) and program size increase by performing such transformations. As used herein, "beneficial execution cycles" refers to a reduction in the number of execution cycles that need to be executed, such as as a result of a transformation (e.g., loop splitting, loop unrolling). As used herein, "program size" refers to the number of lines of code that increase as a result of a transformation (e.g., loop splitting, loop unrolling).

[0179] In one embodiment, the computing engine 302 performs speculative analysis to calculate such metrics, such as estimating beneficial execution cycles and program size increase. In one embodiment, such analysis involves determining whether a naive approach to estimating beneficial execution cycles and program size increase or using a cost model to estimate beneficial execution cycles and program size increase is a better approach. In one embodiment, when the code size increase is less than or equal to the code size increase using a cost model, the naive approach is selected to be used to estimate beneficial execution cycles and program size increase. Otherwise, the cost model is used to estimate beneficial execution cycles and program size increase. As used herein, a naive approach refers to a simple approach, such as assuming an overall loop execution overhead of one cycle with sufficient pipelining. As used herein, a cost model refers to a model used in the analysis of an algorithm to define what constitutes a single step in the execution of the algorithm. For example, a cost model may assume that if the loop execution overhead is one cycle, but the parent loop has M iterations, then the execution overhead of the child loop is M cycles.

[0180] An example of computing engine 302 that calculates metrics for nodes that represent loops in a program (eg, node 403) is provided in FIG.

[0181] 5, node 403 in tree structure 402 may be associated with a loop, such as that represented by code 501. Because such a node is associated with a loop, computing engine 302 performs a loop unrolling transformation on code 501.

[0182] In one embodiment, the loop unrolling transformation for code 501 involves rewriting the loop in code 501 as a repeated sequence of similar independent statements shown in result code 502.

[0183] As described above, such loop unrolling transformations may be performed by an optimizing compiler, where files created for such transformations are analyzed using a performance analysis tool, such as LoopTool or LoopReport by Oracle®. Such analysis tools may be utilized to evaluate beneficial execution cycle and program size increases. For example, in the example of FIG. 5, it is assumed that code block A() contains N instructions and requires K cycles to execute those N instructions. In one embodiment, such analysis to evaluate beneficial execution cycle and program size increases may utilize a naive approach where, with sufficient pipelining, the overall loop execution overhead is one cycle. In one embodiment, such analysis may utilize a cost model that assumes loop execution overhead is one cycle, but if a parent loop (e.g., an m-loop) has M iterations, then the execution overhead of a child loop (e.g., an i-loop) is M cycles.

[0184] Based on the analysis performed by the analysis tool, unrolling the loop surrounding code block A() results in a code size increase from N+2 to 3N+1, but the execution cycles change from 1+M(KN+1) to 1+M*KN, thereby saving M cycles at the cost of a 2N-1 increase in code size.

[0185] An example of computing engine 302 calculating metrics for a node (eg, node 403) that represents an if condition in a program is provided in FIG.

[0186] 6, node 403 in tree structure 402 may be associated with an if condition, such as that represented by code 601. Because such a node is associated with an if condition, computing engine 302 performs a loop splitting transformation on code 601.

[0187] In one embodiment, the loop splitting transformation for code 601 involves decomposing the loop into multiple loops that have the same body but iterate over different consecutive portions of the index range, as shown in result code 602.

[0188] As explained above, such loop splitting transformations may be performed by an optimizing compiler, where the files created for such transformations are analyzed using a performance analysis tool, such as LoopTool or LoopReport by Oracle®. Such analysis tools may be utilized to evaluate the beneficial execution cycle and program size increase. For example, in the example of FIG. 6, the code size remains unchanged (a+b+2), but the number of executions changes from 1+M(1+2(a+2)+(b+1)) to 1+M(2(a+1)+(b+1)), saving 2M execution cycles.

[0189] In one embodiment, such metrics (profitable execution cycles, program size increase) calculated by the computing engine 302 may be reflected in the loop tree 402 shown in FIG. 7, for example.

[0190] Referring to FIG. 7, metrics 701 are entered into the content 404 in the loop tree 402 by the computing engine 302 after the analysis of the transformation is performed as described above.

[0191] Referring again to FIG. 14 in conjunction with FIGS. 1-13, during operation 1403, the analysis engine 303 of the control overhead reducer 104 identifies the node in the loop tree 402 (eg, node 403) with the metric associated with the highest gain.

[0192] As described above, metrics such as estimated beneficial execution cycles and estimated code size increase are calculated for each node (e.g., node 403) in a loop tree (e.g., loop tree 402), where each node in the loop tree represents a loop (a loop is a sequence of instructions that is continuously repeated until a specific condition is reached) or an if condition (a conditional statement that executes a block of code if a specified condition is true). As used herein, "beneficial execution cycles" refers to a reduction in the number of execution cycles that need to be executed, such as as a result of a transformation (e.g., loop splitting, loop unrolling). Furthermore, as used herein, estimated code size increase refers to an increase in code size as a result of performing a transformation, such as loop splitting for an if condition or loop unrolling for a loop. As used herein, "highest gain" refers to the ratio of beneficial execution cycles to code size (number of instructions) increase that has the highest value.

[0193] In one embodiment, the analysis engine 303 identifies the node 403 in the loop tree 402 with the metric associated with the highest gain by analyzing the metrics 701 in the loop tree 402 and identifying the metric with the ratio of execution cycles that benefit to an increase in program size that has the highest value.

[0194] During operation 1404, the analysis engine 303 of the control overhead reducer 104 determines whether the identified node 403 during operation 1403 is a node that represents an if condition.

[0195] If the identified node 403 during operation 1403 is a node representing an if condition, then during operation 1405 the analysis engine 303 of the control overhead reducer 104 determines whether the resulting code (after performing loop splitting on the loop associated with the condition of the if condition) fits into the instruction buffer.

[0196] In one embodiment, analysis engine 303 obtains the available space in the instruction buffer using the remaining() method in Java for class Buffer. In one embodiment, analysis engine 303 performs loop partitioning on the loop associated with the condition of the if condition to determine the code size. In one embodiment, analysis engine 303 determines whether such code size can fit within the available space in the instruction buffer.

[0197] If the resulting code does not fit into the instruction buffer, then during operation 1406, the analysis engine 303 of the control overhead reducer 104 does not perform the loop splitting operation. However, if the resulting code does fit into the instruction buffer, then during operation 1407, the analysis engine 303 of the control overhead reducer 104 proceeds to perform the loop splitting operation on the loop associated with the condition of the if condition in the final version of the code.

[0198] As explained above, in one embodiment, analysis engine 303 performs loop splitting on the loop associated with the condition when a node is associated with an if condition and, particularly for programs of a programmable accelerator, when the resulting code fits within an instruction buffer of limited size. Loop splitting, as used herein, is a computer optimization technique that attempts to simplify a loop or remove dependencies by decomposing it into multiple loops that have the same body but iterate over different consecutive portions of an index range.

[0199] A description of enhancing loop partitioning operations by partitioning a loop by considering all conditions of a loop iterator together, as opposed to the traditional approach of considering conditions one at a time (such enhancement is referred to herein as "inclusive loop partitioning") is provided below in connection with FIG. 15.

[0200] FIG. 15 is a flowchart of a method 1500 for enforcing loop partitioning, according to an embodiment of the present disclosure.

[0201] Referring to FIG. 15 in conjunction with FIGS. 1-14, during operation 1501, the analysis engine 303 of the control overhead reducer 104 determines whether there are equivalent loop conditions in a program, such as a structured control flow program, with multiple conditional branches, including after optimization.

[0202] If a program, such as a structured control flow program, with multiple conditional branches has equivalent loop conditions, including after optimization, during operation 1502 the analysis engine 303 of the control overhead reducer 104 splits the loop taking into account the multiple if conditions, as shown in FIG. 8.

[0203] 8, code 801 corresponds to the original code of a program, such as a structured control flow program, that includes equivalent loop conditions with multiple conditional branches (e.g., i-loop<10 and i-loop≦9). In such a scenario, analysis engine 303 considers multiple if conditions and splits the loop (e.g., i-loop=0 to 9 and i-loop=10 to M) as shown in result code 802. Considering multiple conditions together can result in better decision results, including cost models that help recognize potential benefits from dead code elimination (removing code that does not affect program outcome).

[0204] However, if there is no equivalent loop condition in a program with multiple conditional branches, then during operation 1503, the analysis engine 303 of the control overhead reducer 104 determines whether there are different loop conditions in a program, such as a structured control flow program, with multiple conditional branches, including after optimization.

[0205] If a program with multiple conditional branches does not have different loop conditions, then during operation 1504, the analysis engine 303 of the control overhead reducer 104 does not perform comprehensive loop division.

[0206] However, if a program with multiple conditional branches has different loop conditions, during operation 1505, the analysis engine 303 of the control overhead reducer 104 determines the number of partitions to use to divide the loop iteration space (the iterations executed by the structured control flow program) based on the estimated profitable cycles and estimated code size increase, as described below in connection with FIG. 9.

[0207] 9, code 901 corresponds to the original code of a program, such as a structured control flow program, that includes different loop conditions with multiple conditional branches (e.g., i-loop=0 to 10 and j-loop=0 to M). In such a scenario, analysis engine 303 divides the loop iteration space (e.g., i-loop iteration space) into partitions (e.g., i-loop=0 to 1, i-loop=2 to 5, i-loop=6 to 8, and i-loop=9 to 10), as shown in result code 902.

[0208] However, dividing the loop iteration space into partitions may not fit into the instruction buffer. As a result, a smaller number of partitions may need to be utilized.

[0209] In one embodiment, the analysis engine 303 builds a table (referred to herein as a "partition table") for loops based on conditions with reference to the loop iterator shown in FIG.

[0210] Referring to FIG. 10 in conjunction with FIG. 9, FIG. 10 illustrates a partitioning table 1000 for a loop based on all conditions (eg, <2) referencing the loop iterator (eg, i-loop) of code 901.

[0211] As shown in FIG. 10 , column 1001 contains the iteration variable (iv) values, column 1002 indicates which iterations (e.g., iteration 0, iteration 1) have iteration values less than 2, column 1003 indicates which iterations (e.g., iteration 2, iteration 3, iteration 4, iteration 5, iteration 6, iteration 7, iteration 8, iteration 9) have iteration values greater than or equal to 2, column 1004 indicates which iterations (e.g., iteration 0, iteration 1, iteration 2, iteration 3, iteration 4) have iteration values less than 5, column 1005 indicates which iterations (iteration 9) have iteration values greater than 8, and column 1006 indicates which iterations (iteration 0, iteration 1, iteration 2, iteration 3, iteration 4, iteration 5, iteration 6, iteration 7, iteration 8) have iteration values less than or equal to 8. In one embodiment, such columns 1002-1006 indicate which iterations meet a criterion (e.g., an iteration value less than 2) based on having a logical value of one in the column identifying the iteration in the corresponding row of partition table 1000.

[0212] In one embodiment, the analysis engine 303 identifies partitions by grouping consecutive repeating variable rows using a set of conditions on the columns of a partitioning table 1000 identical to that shown in FIG.

[0213] As shown in FIG. 11, grouped rows 1101, 1102, 1103, and 1104 of partitioned table 1000 correspond to those repeat variable rows with the same set of conditions in columns 1002-1006. For example, grouped row 1101 corresponds to the grouping of repeat variable rows 0 and 1. Grouped row 1102 corresponds to the grouping of repeat variable rows 2, 3, and 4. Grouped row 1103 corresponds to the grouping of repeat variable rows 5, 6, 7, and 8. Grouped row 1104 corresponds to a single repeat variable row, row 9.

[0214] Because not all of the partitions computed from the grouping may fit into the instruction buffer, a metric such as a cost metric may be utilized to determine an optimal set of partitions that fit into the instruction buffer and maximize cycle reduction. In one embodiment, such a metric corresponds to estimated beneficial execution cycles and estimated code size increment. In one embodiment, the estimated beneficial execution cycles correspond to (number of partitioned IfOps) * 2 * outer bound. As used herein, IfOps refer to if operations in the code. As used herein, outer bound refers to a loop bound, which is an upper limit on the number of loop iterations for a given program. In one embodiment, the estimated code size increment corresponds to (code size not within the partition) * 2. In one embodiment, the cost metric corresponds to the ratio of beneficial execution cycles to code size increment. An illustration of selecting a partition (e.g., partitions 1101, 1102) based on estimated beneficial execution cycles and estimated code size increment is provided in FIG. 12 .

[0215] 12 in conjunction with FIGS. 10-11, partition table 1000 includes a column 1201 for a cost metric (e.g., A, B, C, D) associated with each of the partitions (e.g., partitions 1101, 1102, 1103, 1104). In one embodiment, analysis engine 303 selects an optimal set of partitions (e.g., partitions 1101, 1102) that fit into a limited-size instruction buffer based on those partitions that maximize profitable execution cycles while limiting code size growth.

[0216] As described above in connection with FIG. 9, after the analysis engine 303 determines the optimal set or number of partitions that fit into the instruction buffer of limited size, during operation 1506, the analysis engine 303 of the control overhead reducer 104 divides the loop iteration space for a program having different loop conditions with multiple conditional branches (e.g., a structured control flow program) into the determined number of partitions.

[0217] 14, if a node (e.g., node 403) of loop tree 402 does not represent an if condition, then such node (e.g., node 403) represents a loop. As previously explained, each node 403 of loop tree 402 represents either a loop or an if condition.

[0218] If a node (e.g., node 403) of the loop tree 402 represents a loop, during operation 1408, the analysis engine 303 of the control overhead reducer 104 determines whether the resulting code (after performing loop unrolling on the loop represented by the node) fits into the instruction buffer.

[0219] In one embodiment, analysis engine 303 obtains the available space in the instruction buffer using the remaining() method in Java for class Buffer. In one embodiment, analysis engine 303 performs loop unrolling on the loop represented by the node. In one embodiment, analysis engine 303 determines whether such code size can fit within the available space of the instruction buffer.

[0220] If the resulting code does not fit into the instruction buffer, then during operation 1406, the analysis engine 303 of the control overhead reducer 104 does not perform the loop unrolling operation.

[0221] However, if the resulting code fits into the instruction buffer, then during operation 1409, the analysis engine 303 of the control overhead reducer 104 proceeds to perform a loop unrolling operation on the loop represented by the identified node (e.g., node 403) of operation 1403 in the final version of the code.

[0222] As explained above, loop unrolling, as used herein, is a loop transformation technique that attempts to optimize the execution speed of a program at the expense of its binary size, a technique known as a space-time tradeoff. In one embodiment, the transformation is performed by the analysis engine 303 via an optimizing compiler. As explained above, such transformation is performed on the final version of the code.

[0223] Upon performing the loop splitting or loop unrolling transformations of operations 1407 and 1409, respectively, the optimization engine 304 of the control overhead reducer 104 performs optimizations on the resulting code (the code after the transformations have been performed) during operation 1410, such as removing dead code (code that does not affect the program outcome), constant propagation (substituting values of known constants in expressions at compile time), etc.

[0224] As noted above, in one embodiment, optimization engine 304 utilizes various software tools to perform such optimizations, which may include, but are not limited to, LinearB®, Aivosto's Project Analyzer, SCARF, etc. Additionally, in one embodiment, optimization engine 304 utilizes an optimizing compiler to perform such optimizations.

[0225] During operation 1411, the build engine 301 of the control overhead reducer 104 reconstructs the loop tree 402 starting with the parent of the node 403 identified using the metric associated with the highest gain. That is, the build engine 301 reconstructs the loop tree 402 using the root node of the tree that corresponds to the parent of the node 403 identified using the metric associated with the highest gain.

[0226] The metrics associated with each node 403 in the reconstructed loop tree 402 are then calculated during operation 1402, as described above.

[0227] In this manner, an instruction buffer of limited size may be used to reduce program control overhead, such as in the case of a programmable accelerator.

[0228] Furthermore, the principles of the present disclosure improve upon techniques or fields related to control overhead.

[0229] As explained above, branching is the process of making decisions in a program by selecting one of two or more paths based on a condition. Overhead is the additional processing time required to evaluate the condition and determine the correct path to follow. A loop is associated with repeating a set of instructions multiple times. Overhead is the additional processing time required to execute each iteration of the loop. Although control overhead is necessary for program execution, excessive control overhead can result in slower program execution, increased memory usage, and other performance issues. One classic transformation to reduce control overhead is loop unrolling. Loop unrolling involves reducing the number of iterations a loop executes by executing multiple iterations (the unrolling factor) of the loop in a single pass. In loop unrolling, the loop is completely unrolled, so that each iteration of the loop is replaced by a single instance of the loop body. Unfortunately, loop unrolling results in an increase in the size of the result code, which can negatively impact cache efficiency and instruction fetching. For example, in the case of a programmable accelerator, instructions related to a loop unrolling transformation may be stored in an instruction buffer (a storage for holding instructions), which may be limited in size. As a result, the instruction buffer may not have enough space to store such instructions. Another classic transformation for reducing control overhead is loop splitting. Loop splitting involves splitting a loop into multiple subloops, each of which has a different loop index set, so that each subloop executes a different subset of the original loop iterations. In particular, loop splitting reduces the number of branches and conditional statements required to manage loop control. By splitting a loop into smaller subloops, each subloop can be executed without the need for complex control statements. Unfortunately, loop splitting increases the size of the result code. In the case of a programmable accelerator, instructions related to a loop splitting transformation, similar to a loop unrolling transformation, may be stored in an instruction buffer, which may be limited in size.As a result, the instruction buffer may not have enough space to store such instructions. Thus, loop unrolling and loop splitting transformations may compete for space in the instruction buffer, and it is not always optimal to perform one before the other. For example, performing a loop unrolling transformation followed by a loop splitting transformation may not leave enough room in the instruction buffer after unrolling the loop to perform the loop splitting transformation. Similarly, in another example, performing a loop splitting transformation followed by a loop unrolling transformation may not leave enough room in the instruction buffer after performing the loop unrolling transformation. Thus, there is currently no means for effectively reducing program control overhead, such as in the case of a programmable accelerator, using an instruction buffer of limited size.

[0230] Embodiments of the present disclosure improve upon such techniques by constructing a loop tree from a program, such as a structured control flow program. As used herein, structured control flow refers to a programming concept in which the flow of control to a block or region is based on a single-entry and single-exist methodology (SESE). In such a programming concept, during execution, execution of (structured) statements begins at a single defined point, and execution ends at a single defined point. As used herein, a loop tree refers to a tree-like data structure that graphically represents loops and / or if-conditions in a program, such as a structured control flow program. In one embodiment, a loop tree is constructed from a program by creating a set of classes or structures to represent the nodes and relationships in the tree. Such nodes can then be inserted (adding nodes) or deleted (removing nodes) via the AddChild or DeleteChild methods, respectively. If the resulting code fits into the instruction buffer, a loop splitting operation or a loop unrolling operation may then be performed in association with the node of the loop tree identified as having the highest gain (the ratio of execution cycles that benefit per increase in code size (number of instructions) that has the highest value). In one embodiment, if the resulting code fits into the instruction buffer, a loop splitting operation is performed on the loop associated with the condition for the node representing an if condition (the node with the highest gain), or if the resulting code fits into the instruction buffer, a loop unrolling operation is performed on the loop for the node representing such a loop (the node with the highest gain). In this manner, an instruction buffer of limited size may be used to reduce program control overhead, such as in the case of a programmable accelerator. Furthermore, this manner provides an improvement in technical fields related to control overhead.

[0231] The technical solutions provided by the present disclosure cannot be implemented in the human mind or by a human using pen and paper, i.e., the technical solutions provided by the present disclosure cannot be realized in the human mind or by a human using pen and paper in any reasonable amount of time and with any reasonable expectation of accuracy without the use of a computer. The descriptions of various embodiments of the present disclosure have been presented for illustrative purposes, but are not intended to be exhaustive or limiting of the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein has been selected to best explain the principles of the embodiments, practical applications, or technical improvements over commercially available technologies, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. 1. A computer-implemented method for reducing control overhead, comprising: constructing a loop tree from the program; and performing an action selected from the group consisting of: a loop splitting action in response to a node of the loop tree representing an if condition; and a loop unrolling action in response to a node of the loop tree representing a loop. A method for providing the above.

2. calculating a metric for each node in the loop tree; and identifying the node in the loop tree with a metric associated with the ratio of beneficial execution cycles to code size increase having the highest value; The method of claim 1 further comprising:

3. 3. The method of claim 2, wherein the loop splitting operation is performed on a loop associated with the if-condition in response to the node being associated with the ratio of beneficial execution cycles to the code size increase having the highest value and in response to the node representing the if-condition.

4. 3. The method of claim 2, wherein the loop unrolling operation is performed on the loop in response to the node being associated with the ratio of beneficial execution cycles to the code size increase having the highest value and in response to the node representing the loop.

5. In response to the program containing an equivalent loop condition with multiple conditional branches, dividing the loop into multiple if conditions. The method of claim 1 further comprising:

6. determining the number of partitions to be used to divide the loop iteration space based on the estimated cycle reduction and the estimated code size increase; and Dividing the loop iteration space into the determined number of partitions in response to the program including different loop conditions with multiple conditional branches. The method of claim 1 , further comprising:

7. The method of claim 1 , wherein the program is a structured control flow program.

8. 1. A computer program for reducing control overhead, the computer program comprising: A procedure for programmatically constructing a loop tree; and performing an operation selected from the group consisting of: a loop splitting operation in response to a node of the loop tree representing an if condition; and a loop unrolling operation in response to a node of the loop tree representing a loop. A computer program comprising programming instructions for causing a processor to execute the program.

9. The program code further comprises: calculating a metric for each node in the loop tree; and identifying the node in the loop tree with the metric associated with the ratio of beneficial execution cycles to code size increase having the highest value; 9. The computer program product of claim 8, comprising programming instructions for causing the processor to execute:

10. 10. The computer program product of claim 9, wherein the loop splitting operation is performed on a loop associated with the if-condition in response to the node being associated with the ratio of beneficial execution cycles to the code size increase having the highest value and in response to the node representing the if-condition.

11. 10. The computer program product of claim 9, wherein the loop unrolling operation is performed on the loop in response to the node being associated with the ratio of beneficial execution cycles to the code size increase having the highest value and in response to the node representing the loop.

12. The program code further comprises: A procedure for dividing a loop by considering multiple if conditions in response to the program containing an equivalent loop condition with multiple conditional branches.

9. The computer program product of claim 8, comprising programming instructions for causing the processor to execute:

13. The program code further comprises: determining the number of partitions to be used to divide the loop iteration space based on the estimated cycle reduction and the estimated code size increase; and Dividing the loop iteration space into the determined number of partitions according to whether the program contains different loop conditions with multiple conditional branches.

13. A computer program product according to any one of claims 8 to 12, comprising programming instructions for causing the processor to execute the

14. 13. A computer program according to any one of claims 8 to 12, wherein the program is a structured control flow program.

15. a memory for storing a computer program for reducing control overhead; and a processor coupled to said memory, said processor comprising: A procedure for programmatically constructing a loop tree; and performing an operation selected from the group consisting of: a loop splitting operation in response to a node of the loop tree representing an if condition; and a loop unrolling operation in response to a node of the loop tree representing a loop. configured to execute program instructions of the computer program, A system comprising:

16. The program instructions of the computer program further comprise: calculating a metric for each node in the loop tree; and identifying the node in the loop tree with the metric associated with the ratio of beneficial execution cycles to code size increase having the highest value; The system of claim 15, comprising:

17. 17. The system of claim 16, wherein the loop splitting operation is performed on a loop associated with the if-condition in response to the node being associated with the ratio of beneficial execution cycles to the code size increase having the highest value and in response to the node representing the if-condition.

18. 17. The system of claim 16, wherein the loop unrolling operation is performed on the loop in response to the node being associated with the ratio of beneficial execution cycles to the code size increase having the highest value and in response to the node representing the loop.

19. The program instructions of the computer program further comprise: A procedure for dividing a loop by considering multiple if conditions in response to the program containing an equivalent loop condition with multiple conditional branches. The system of claim 15, comprising:

20. The program instructions of the computer program further comprise: determining the number of partitions to be used to divide the loop iteration space based on the estimated cycle reduction and the estimated code size increase; and Dividing the loop iteration space into the determined number of partitions according to whether the program contains different loop conditions with multiple conditional branches.

20. The system of any one of claims 15 to 19, comprising:

Images