System and method for balancing computing resources

Abstract

Methods and systems for balancing computing resources are described. According to one aspect, a method for balancing computing resources may be provided. The method includes generating a plurality of vectoring loops of scalar code. The method may also include interleaving each vectorization loop of the plurality of vectorization loops to generate a plurality of interleaved vectorization loops. The interleaving may be based on available vector resources. The method may also include interpolating one or more scalar iterations of a scalar loop into each of the plurality of interleaving vectoring loops to generate a plurality of interleaving and scalar interpolated vectoring loops. The method may also include selecting one of the plurality of interleaving and scalar interpolation vectoring loops according to a cost model.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This is the first application filed for this invention. Technical Field

[0003] The present invention relates to the field of computer performance engineering, and in particular to a method and system for balancing computing resources such as processor resources. Background Art

[0004] The utilization of computing resources, including processor resources, plays a crucial role in modern computing systems. However, it also faces a series of challenges. One challenge is inefficient resource utilization, where not all available resources are optimally utilized during program execution. This can lead to reduced performance and efficiency. Another challenge is overuse of resources, which can cause processor stalls or contention, resulting in reduced latency and throughput. The limited number of registers available in the processor leads to high register pressure, which can further complicate resource optimization because it can cause frequent register spills and reloads, resulting in reduced performance.

[0005] Furthermore, dynamically scheduled out-of-order processors can create challenges in managing instruction dependencies and scheduling instruction execution. Processors with different memory configurations (e.g., separate memory locations for scalar and vector values) can also increase the complexity of optimizing data movement between different memory types.

[0006] Therefore, there is a need for systems and methods for balancing computing resources that obviate or mitigate one or more limitations of the prior art.

[0007] The purpose of the background art is to disclose information that the applicant believes may be relevant to the present invention. It is not necessary to admit, nor should it be construed, that any of the above information constitutes prior art against the present invention. Summary of the Invention

[0008] Methods and systems for balancing computing resources may be provided. In some aspects, the methods and systems relate to scalar interpolation to balance transmit slot utilization of a dynamically scheduled processor having a vector unit. According to one aspect, a method may be provided for balancing computing resources, such as processor resources, during a static compilation process. The method includes generating a plurality of vectorized loops of scalar code. The method may also include interpolating one or more scalar iterations of the scalar loop into each of the plurality of vectorized loops to generate a plurality of scalar interpolation vectorized loops. The method may also include selecting one of the plurality of scalar interpolation vectorized loops based on a cost model.

[0009] The method may further include determining that vectorization of the scalar loop is legal. The method may further include determining that scalar interpolation is legal for each of the plurality of vectorized loops. The method may further include determining, for each of the plurality of vectorized loops, a number of scalar iterations to interpolate based on available scalar resources. Interpolating the one or more scalar iterations into each of the plurality of vectorized loops may be based on the determined number of scalar iterations.

[0010] The cost model may be based on a heuristic method calculated based on the number of instructions in each of the plurality of interleaved and scalar interpolation vectorization loops. The cost model may also be based on the latency of the instructions in each of the plurality of interleaved and scalar interpolation vectorization loops. The cost model may also be based on the instruction-level parallelism (ILP) present in each of the plurality of interleaved and scalar interpolation vectorization loops.

[0011] The method may further include interleaving one or more vectorized loops of the plurality of vectorized loops to generate one or more interleaved vectorized loops, the interleaving being based on available vector resources.

[0012] Interpolating one or more scalar iterations of a scalar loop into each vectorized loop in a plurality of vectorized loops to generate a plurality of scalar interpolation vectorized loops may also include interpolating one or more scalar iterations of a scalar loop into each interleaved vectorized loop in one or more interleaved vectorized loops to generate one or more interleaved and scalar interpolation vectorized loops.

[0013] Selecting a scalar interpolation vectorization loop from the plurality of scalar interpolation vectorization loops according to the cost model may include selecting one of the following according to the cost model: a plurality of scalar interpolation vectorization loops and one or more interleaved and scalar interpolation vectorization loops.

[0014] According to another aspect, a method may be provided for balancing computing resources, such as processor resources, based on feedback or profiling information obtained during program execution. The method includes obtaining a vectorized loop from a program. The method may also include interpolating one or more scalar iterations of the vectorized loop into the vectorized loop to generate a scalar interpolated vectorized loop. Obtaining the vectorized loop from the program includes vectorizing the program. The program may be a precompiled program.

[0015] The method may further include determining that scalar interpolation is legal for the vectorized loop. The method may further include obtaining runtime data for the vectorized loop. The obtained runtime data may be performance data indicating resource utilization. The method may further include determining, based on the obtained runtime data, that scalar interpolation is beneficial for the vectorized loop in terms of resource utilization.

[0016] Interpolating one or more scalar iterations of the vectorized loop into the vectorized loop may include interpolating based on the acquired runtime data. Interpolating one or more scalar iterations of the vectorized loop into the vectorized loop may include generating one or more equivalent scalar iterations of the vectorized loop. Interpolating one or more scalar iterations of the vectorized loop into the vectorized loop may also include interpolating the generated one or more equivalent scalar iterations into the vectorized loop.

[0017] The method may further include determining available scalar resources for scalar interpolation. The method may further include determining a number of scalar iterations to interpolate into the vectorized loop based on one or more of: the retrieved runtime data and the available scalar resources. Interpolating the one or more scalar iterations into the vectorized loop may be based on the determined number of scalar iterations.

[0018] The method may further include scheduling vector instructions and scalar instructions in the scalar interpolation vectorization loop in order of performance in terms of execution time to generate a scheduled scalar interpolation vectorization loop.

[0019] The method may further include: unrolling one or more iterations of the vectorized loop to generate an unrolled and scalar interpolation vectorized loop. The method may further include: determining a number of iterations of the vectorized loop to unroll based on one or more of: retrieved runtime data, available scalar resources, and available vector resources. The unrolling of the one or more iterations of the vectorized loop may be based on the determined number of iterations of the vectorized loop to unroll.

[0020] The method may further include scheduling vector instructions and scalar instructions in the unrolled and scalar interpolation vectorized loop in a performance order based on execution time to generate a scheduled unrolled and scalar interpolation vectorized loop.

[0021] Determining, based on the acquired runtime data, that scalar interpolation benefits the vectorized loop in terms of resource utilization may include determining, using a first machine learning model, that scalar interpolation benefits the vectorized loop in terms of resource utilization. Determining a number of scalar iterations to be interpolated into the vectorized loop may include determining the number of scalar iterations using a second machine learning model. Using the machine learning model to determine the number of scalar iterations may achieve both or one of improved performance and improved resource utilization. Determining a number of iterations of the vectorized loop to be unrolled may include determining the number of iterations of the vectorized loop to be unrolled using a third machine learning model. Using the machine learning model to determine the number of iterations of the vectorized loop to be unrolled may achieve both or one of improved performance and improved resource utilization. The third machine learning model may be different from or the same as the second machine learning model. Scheduling vector instructions and scalar instructions in sequence may include scheduling the vector instructions and scalar instructions using a fourth machine learning model. Using the machine learning model to schedule the vector instructions and scalar instructions may achieve both or one of improved performance and improved resource utilization. The fourth machine learning model may be different from or the same as the third machine learning model.

[0022] According to another aspect, a method for balancing computing resources may be provided. The method includes generating multiple versions of a vectorized loop. The method may also include interpolating one or more scalar iterations of the vectorized loop into each of the multiple versions of the vectorized loop to generate multiple scalar interpolation vectorized loops. The method may also include selecting a version of the vectorized loop from the vectorized loop and the multiple scalar interpolation vectorized loops.

[0023] The method may further include executing the vectorized loop and each of the plurality of scalar interpolation vectorized loops for k iterations of the vectorized loop. The vectorized loop may terminate at an nth iteration, where n may be greater than k. The method may further include measuring an execution time for each of the vectorized loop and the plurality of scalar interpolation vectorized loops based on the k iterations. Selecting a version of the vectorized loop may be based on the measured execution time.

[0024] The method may further include: obtaining runtime data of the vectorized loop. The method may further include: determining whether scalar interpolation is legal for the vectorized loop based on the obtained runtime data.

[0025] According to another aspect, an apparatus is provided, comprising modules for performing one or more of the methods and systems described herein.

[0026] According to one aspect, a device is provided, wherein the device includes: a memory for storing a program; a processor for executing the program stored in the memory, wherein when the program stored in the memory is executed, the processor is used to perform one or more of the methods and systems described herein.

[0027] According to another aspect, a computer-readable medium is provided, wherein the computer-readable medium stores program code executed by a device, the program code being configured to perform one or more of the methods and systems described herein.

[0028] According to one aspect, a chip is provided, wherein the chip includes a processor and a data interface, and the processor reads instructions stored in a memory through the data interface to execute one or more of the methods and systems described herein.

[0029] Other aspects of the present invention provide apparatus and systems for implementing the methods according to the first aspect disclosed herein. For example, wireless stations and access points may be configured with machine-readable memory comprising instructions that, when executed by a processor of such devices, cause the devices to perform one or more of the methods and systems disclosed herein.

[0030] The above describes embodiments in conjunction with various aspects of the present invention, and these embodiments can be implemented based on these aspects. It will be understood by those skilled in the art that the embodiments can be implemented in conjunction with the aspects describing these embodiments, but can also be implemented together with other embodiments of the aspects. When the embodiments are mutually exclusive or inconsistent with each other, this will be apparent to those skilled in the art. Some embodiments may be described in conjunction with one aspect, but can also be applied to other aspects, which will be apparent to those skilled in the art. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Other features and advantages of the present invention will become apparent from the following detailed description taken in conjunction with the accompanying drawings, in which:

[0032] Figure 1 An example of code transformation performed by a compiler in a program loop in pseudo code similar to a Reduced Instruction Set Computer (RISC) is shown.

[0033] Figure 2 An example of a processor architecture is shown in which scalar instructions and SIMD instructions have different processing lanes.

[0034] Figure 3 Shown are examples of compiler-generated conventional vectorization 'vloop' and selective vectorization 'vloop_sel' derived from a scalar loop 'loop'.

[0035] Figure 4 Exemplary code is shown for a conventional vectorized loop 'vloop' and a compiler transformation based on scalar interpolation 'vloop_si' for a scalar loop 'loop' according to one aspect.

[0036] Figure 5 A scalar interpolation method during static compilation according to one aspect is shown.

[0037] Figure 6 A scalar interpolation method in profiling-driven optimization or feedback-driven optimization according to one aspect is shown.

[0038] Figure 7 A scalar interpolation method using a machine learning model according to one aspect is shown.

[0039] Figure 8 A scalar interpolation method using auto-tuning optimization according to one aspect is shown.

[0040] Figure 9 An example of applying Superword-Level Parallelism (SLP) vectorization to code suitable for vectorization is shown.

[0041] Figure 10 Exemplary code according to one aspect is shown, showing conventional SLP vectorization 'after-slp' applied to scalar code 'before-slp' and SLP vectorization 'after-slp-si' using scalar interpolation.

[0042] Figure 11 A method for balancing computing resources according to one aspect is shown.

[0043] Figure 12 Another method for balancing computing resources according to one aspect is shown.

[0044] Figure 13 Another method for balancing computing resources according to one aspect is shown.

[0045] Figure 14 Apparatus according to various aspects of the present invention are shown, wherein the apparatus can perform any or all of the operations of the above methods and features described herein, either explicitly or implicitly.

[0046] It should be noted that similar features are identified by similar reference numerals throughout the drawings. DETAILED DESCRIPTION

[0047] Methods and systems for balancing computational resources are described. In some aspects, the methods and systems relate to scalar interpolation to balance transmit slot utilization of a dynamically scheduled processor with a vector unit. According to one aspect, method 1100 can balance computational resources, such as processor resources, during a static compilation process. Method 1100 can include generating 1101 a plurality of vectorized loops of scalar code. The method can also include interleaving 1102 each of the plurality of vectorized loops to generate a plurality of interleaved vectorized loops. The interleaving can be based on available vector resources. The method can also include interpolating 1103 one or more scalar iterations of the scalar loop into each of the plurality of interleaved vectorized loops to generate a plurality of interleaved and scalar interpolation vectorized loops. The method can also include selecting 1104 one of the plurality of interleaved and scalar interpolation vectorized loops based on a cost model.

[0048] According to another aspect, method 1200 can balance computing resources, such as processor resources, based on feedback or profiling information obtained during program execution. Method 1200 can include obtaining 1201 runtime data of a vectorized loop. The method can also include interpolating 1202 one or more scalar iterations of the vectorized loop into the vectorized loop based on the obtained runtime data to generate a scalar interpolation vectorized loop.

[0049] According to another aspect, method 1300 may include generating 1301 multiple versions of a vectorized loop. The method may also include interpolating 1302 one or more scalar iterations of the vectorized loop into each of the multiple versions of the vectorized loop to generate multiple scalar interpolation vectorized loops. The method may also include selecting 1303 one version of the vectorized loop from the vectorized loop and the multiple scalar interpolation vectorized loops.

[0050] Many programs exhibit both data-level parallelism (DLP) and instruction-level parallelism (ILP). Code exhibits DLP when a single operation can be applied to multiple contiguous data simultaneously. Code exhibits ILP when multiple different independent instructions can be scheduled simultaneously. Modern compilers and processors exploit these program behaviors to improve performance. Modern processors include dedicated resources to process single-instruction multiple-data (SIMD) instructions, which are used to compute code that exhibits DLP. Modern high-performance processors are also capable of dynamically scheduling and issuing multiple instructions simultaneously to exploit ILP. On the software side, compilers generate code that executes on these processors. Compilers include multiple code conversion optimization passes, some of which generate SIMD instructions. Compilers can also optimize code by generating longer instruction windows, enabling the processor to perform dynamic scheduling, thereby enabling the processor to maximize ILP. The loop vectorization optimization pass is an example of an optimization pass that generates SIMD instructions. The optimization pass that performs loop unrolling and loop iteration interleaving is an example of an optimization pass that generates longer instruction windows.

[0051] Figure 1This example shows a compiler transforming code in a program loop in pseudocode similar to a Reduced Instruction Set Computer (RISC). Referring to code 100, the statements appearing below the 'loop' label are scalar computations. In a scalar computation, a single instruction operates on a single data value. This loop code 100 multiplies the value stored at A[i] by the value stored at B[i] and stores the result at C[i]. Referring to code 102, the code below the 'vloop' label is a vectorized version of the 'loop' in code 100. This vectorization is created by replacing scalar instructions with SIMD instructions. 'vloop' code 102 performs the same computation as loop code 100. However, vloop processes four data elements of a vector in each loop iteration to take advantage of the Directed Programming Interface (DLP). This example assumes a processor architecture with a vector width of four elements. Increasing data processing throughput compared to the original scalar loop can improve performance. The compiler can also optimize the code to take advantage of ILP by unrolling the loop and interleaving the loop iterations, as shown in the 'vloop2' code 104. This transformation unrolls the 'vloop' by a factor of 2, and the instructions of the unrolled iterations are interleaved to place independent instructions adjacent to each other. However, the processor is able to schedule and execute the independent instructions in the same clock cycle. This transformation 104 takes advantage of ILP to further improve performance compared to both the scalar 'loop' 100 and the only vectorized 'vloop' 102.

[0052] It will be appreciated that in modern processor architectures, scalar and vector processing operations may be performed in separate units, which are treated as separate resources.

[0053] Figure 2 An example of a processor architecture is shown in which scalar instructions and SIMD instructions have different processing channels. Instructions ready for execution by the processor instruction scheduler 202 can be cached in an issue queue before execution. SIMD instructions can be cached in the SIMD issue queue 204, and scalar instructions can be cached in the scalar issue queue 206. In one embodiment, compiler vectorization can convert most (if not all) scalar instructions into SIMD instructions. The number of generated SIMD instructions can be greater than the original scalar code, especially when both or one of the unrolling and interleaving transformations is used to expose ILP.

[0054] Applying vectorization at compile time can improve performance by leveraging DLP and ILP. However, the large number of SIMD instructions to be processed at runtime can cause the SIMD issue queue 204 to become full, causing the processor to stall until some SIMD instructions can be issued. This occurs when the rate at which instructions are cached in the issue queue is higher than the rate at which instructions are released from the issue queue to be processed by the functional units. Processor stalls caused by a full issue queue can limit the performance of the code. In addition, when most of the computations in the loop code are converted to SIMD instructions, the SIMD issue queue 204 and subsequent vector resources may be over-utilized to execute the vectorized code. At the same time, since there are few or no scalar instructions to be executed in the code after compiler vectorization, the scalar resources are almost idle.

[0055] Therefore, traditional compiler vectorization of processor architectures may suffer from inefficiencies, including: over-utilization of SIMD resources in vectorized code, causing processor stalls and thus limiting performance; and under-utilization of the processor's scalar resources in vectorized code.

[0056] Some existing compiler code transformation techniques are based on selectively applying vectorization to certain instructions in a loop and then applying software pipelining (modulo scheduling) to the loop to generate a static schedule of instructions.

[0057] Figure 3 Shown are examples of compiler-generated conventional vectorization 'vloop' and selective vectorization 'vloop_sel' derived from a scalar loop 'loop'. Figure 3 RISC-like code is shown with traditional vectorization 302 and selective vectorization 304 applied to a scalar loop 'loop' 300. Traditional vectorization 302 applies vectorization to all modifiable scalar instructions in the code region 'vloop'. In contrast, selective vectorization 304 only vectorizes a subset of applicable scalar instructions. The code in 'vloop_sel' 304 illustrates selective vectorization applied to scalar code in 'loop'. The original load from A[i] is selectively vectorized, while the load from B[i] remains scalar. In this example, the dependent multiplication and store instructions are selectively vectorized. The loop is vectorized by a factor of 2, where SIMD instructions process two consecutive data elements simultaneously. Therefore, two consecutive scalar loads, B[i] and B[i+1], are required to match the dependent SIMD instructions.

[0058] Existing approaches are subject to numerous limitations. For dynamically scheduled out-of-order processors, existing approaches may not be feasible. For example, software pipelining of loops (module scheduling) may require a detailed model of the processor to generate an optimal static schedule of the loop's instructions. While this approach may be feasible for simple statically scheduled in-order processors, it is difficult for complex out-of-order processors, where instructions are dynamically scheduled by the processor itself.

[0059] Existing schemes may assume that instructions can have both vector and scalar operands, for example, the instruction vx = vmul va, [b0, b1] in the 'vloop_sel' code 304, where the vector operand 'va' is element-wise multiplied with the scalar operands b0 and b1. However, most processors have separate intra-processor data memories (or separate memory locations) to store scalar and vector values, and therefore may not support instructions that operate on mixed types. Therefore, selective vectorization may require additional register moves between scalar and vector registers to make it feasible. Register moves can incur significant overhead during loop execution.

[0060] Existing solutions may be limited to architectures with short vectors (small number of elements in the vectors). For example, selective vectorization may require duplication of scalar code to match the vectorized width of the loop. Architectures that support long vectors may cause selective vectorization to generate too many scalar instructions, leading to high register pressure (register overutilization) on the scalar registers. High register pressure may lead to performance degradation.

[0061] According to one aspect, the utilization of processor resources (vector or SIMD resources and scalar resources) can be improved. For example, during the execution of compiler-vectorized code, overutilization of the processor's vector or SIMD resources can be improved, thereby reducing the possibility of processor stalls. In addition, during the execution of compiler-vectorized code, underutilized or idle scalar resources of the processor can be improved.

[0062] According to one aspect, a compiler transformation can be provided that interpolates scalar iterations of a loop into a vector loop. Interpolating scalar code into a vector loop can offload efficient work to processor scalar resources while reducing overutilization of vector resources, thereby reducing wasted processor cycles due to stalls.

[0063] According to one aspect, scalar interpolation can be performed, wherein scalar iterations of a loop are interpolated into a vectorized version of the loop. Figure 4Example code for a conventional vectorized loop 'vloop' and compiler transformation based on scalar interpolation 'vloop_si' for a scalar loop 'loop' according to one aspect is shown. Code 400 is the untransformed and unoptimized scalar loop 'loop'. Code 400 can be transformed into code 402, 'vloop' code, based on conventional compiler loop vectorization applied to scalar loop 'loop' 400.

[0064] Traditional loop vectorization (reference code 402) determines that it is legal to vectorize the loop and converts the scalar instructions into SIMD instructions to take advantage of DLP. In the 'vloop' example 402, the compiler has vectorized 'loop' by a factor of 4—each SIMD instruction processes four consecutive elements. Therefore, each loop iteration of 'vloop' is equivalent to four iterations of the original scalar loop 'loop'. The code 'vloop_si' shows scalar interpolation applied on top of the traditional vectorized loop 'vloop'.

[0065] According to one aspect, the scalar iterations of 'loop' can be interpolated into the vectorized loop 'vloop' to generate the code shown in 'vloop_si' 404. Thus, a single iteration of 'vloop_si' 404 is equivalent to five iterations of the original scalar loop 'loop'—the first four iterations being completed via a sequence of SIMD instructions, and the fifth iteration being completed via an interpolated scalar iteration. It should be understood that for simplicity, the 'vloop_si' 404 example shows only one scalar iteration interpolated into either or both a non-unrolled and a non-interleaved vector loop. In some embodiments, scalar interpolation can interpolate any number of scalar iterations into either or both an unrolled and an interleaved vector loop.

[0066] By integrating scalar iterative interpolation into a vectorized loop, vector resources can be fully utilized while using otherwise idle scalar resources of the processor, thereby improving DLP and ILP. According to one or more aspects, scalar iterative interpolation can also improve performance compared to conventional techniques and existing techniques.

[0067] According to one or more aspects, scalar interpolation can be applied to Figure 2 A processor architecture is shown. Processors with different and disjoint resource subsets may experience resource utilization imbalances, which can be addressed by using scalar interpolation according to one or more aspects to improve processor performance. Such a processor may have one subset dedicated exclusively to processing scalar instructions, while another subset exclusively processes SIMD or vector instructions.

[0068] One or more aspects can be implemented in any tool that generates or optimizes executable code for a processor (e.g., compilers and binary optimizers). According to one aspect, a scalar loop can be converted to a vector loop body that includes a functionally equivalent scalar loop body interleaved or interpolated within the vector loop body.

[0069] Static compilation is a technique used to compile a program into executable code that does not require information about the program's runtime behavior. Figure 5 A scalar interpolation method during a static compilation process according to one aspect is illustrated. Method 500 describes using scalar interpolation during a static compilation process of a program. Method 500 is based on applying loop vectorization and scalar interpolation to a scalar loop. The method begins 501 with a scalar loop that is a candidate for loop vectorization and scalar interpolation. The method may also include performing 502 a validity check on the scalar loop to determine whether vectorization of the loop is legal, i.e., whether the vectorization will produce a functionally correct equivalent of the original scalar loop. The validity portion may include, but is not limited to, checking memory aliasing and cross-iteration loop dependencies. If the loop passes the validity check, the method further includes vectorizing the loop. There may be more than one way to vectorize the loop to produce functionally correct code. In some embodiments, the method may include generating 503 multiple legal versions through loop vectorization. If the validity check fails, neither vectorization nor scalar interpolation is applied to the original loop.

[0070] In some embodiments, the method may further include: interleaving 504 zero, one, or more vectorized loop versions of the plurality of vectorized loop versions to maximize ILP by fully utilizing vector resources in the processing unit. In some embodiments, interleaving 504 may be optional.

[0071] The method may further include performing 505 another validity check on the one or more vectorized loops and the one or more interleaved vectorized loops (where interleaving 504 is applied) to determine whether the scalar interpolation is legal. If the validity check determines that the scalar interpolation is not legal, the scalar interpolation is not applied to the vectorized loop version. The method may then further include selecting 506 a loop from the one or more vectorized loops and the interleaved vectorized loops to replace the original scalar loop based on a cost model. The cost model may be based on one or more of: a heuristic method for calculating the number of instructions in the loop version, the latency of the instructions in the loop version, and the ILP present in the loop version. In some embodiments, the cost model may be based on other factors that will be understood by those skilled in the art.

[0072] For each loop in the one or more loops (the vectorized loops and the interleaved vectorized loops), if scalar interpolation is legal, the method may further include determining 507 a number of scalar iterations to interpolate in each of the one or more loops. The optimal number of scalar iterations for each loop in the one or more loops may be a function of one or more of (but not limited to) the number of scalar resources in the processing unit, the number of scalar instructions that can be issued simultaneously, and memory alignment of instructions in the loop.

[0073] The method may further include, for each of the one or more loops (vectorized loops and interleaved vectorized loops) in which scalar interpolation is legal, interpolating 508 the determined (as determined in 507) number of scalar iterations into each of the one or more loops. Thus, interpolation may add the number of scalar iterations determined in the previous step across multiple vectorized versions. The method may further include interleaving 509 zero, one, or more scalar interpolation loops to further maximize ILP by fully utilizing vector resources in the processing unit. In some embodiments, interleaving 509 may be optional.

[0074] The method may also include selecting 510, based on the cost model, a version of the vectorized loop that applies scalar interpolation based on the cost model. The selection may be made from one or more loops (vectorized loops and interleaved vectorized loops) that use scalar interpolation. The cost model may be based on one or more of: a heuristic for calculating the number of instructions in the loop version, the latency of the instructions in the loop version, and the ILP present in the loop version. In some embodiments, the cost model may be based on other factors that will be understood by those skilled in the art.

[0075] Method 500 can provide simplicity and reduce code generation time. In some embodiments, parameters used for the conversion, such as vectorization factors, expansion and interleaving factors, and the number of scalar iterations to be interpolated, can be determined by a static cost model. Therefore, the quality of the conversion may depend on the cost model. The quality of the conversion may also depend on the runtime behavior of the code, which in some cases may be difficult or impossible to accurately capture with a static model.

[0076] According to one aspect, a program can be optimized by either profiling-driven optimization or feedback-driven optimization. The compiler can use the runtime data collected when the program is executed to decide whether a given code conversion should be applied to the code. The compiler can also use runtime data to determine the parameters of the code conversion that should be applied to the code. In certain embodiments, compared with static compilation, profiling-driven optimization can produce executable code with better performance.

[0077] Figure 6A method for scalar interpolation in profiling-driven optimization or feedback-driven optimization according to one aspect is shown. Method 600 can be based on applying scalar interpolation to vectorized loops given runtime data. The method begins 601 with a vectorized loop in a precompiled program. In some embodiments, the vectorized loop is generated from an original program that was vectorized during compilation. In some embodiments, the original program can have scalar loops that a static compiler can vectorize to generate a "precompiled" program. Method 600 can operate on the "precompiled" program to add scalars that are interpolated based on either or both feedback and profiling information obtained when running the program.

[0078] In some embodiments, method 600 can be applied to any program that includes vectorized loops. For example, if the program has no vectorizable code at all, method 600 will not begin because there are no vectorized loops to start with. If method 800 finds that the program has vectorized loops, method 600 can be applied.

[0079] In some embodiments, additional code transformations may be required before the method is applied. For example, one or more loop transformation optimization passes may be performed to simplify or optimize the code in the loop to make it more suitable for vectorization and scalar interpolation.

[0080] In some embodiments, method 600 may be applied in addition to or after method 500. For example, the vectorized loop at 601 may refer to the selected 510 loop of method 500.

[0081] The method may also include performing a legality phase 602 to determine whether scalar interpolation is legal for use in the vectorized loop. If not, the method terminates without applying scalar interpolation. If scalar interpolation is legal, the method may also include obtaining and analyzing 603 runtime data or information about the vectorized loop to determine whether applying scalar interpolation offers a performance advantage. For example, runtime data collected during program execution may be input into a compiler. The method may also include determining 604 whether scalar interpolation is beneficial for the vectorized loop based on the data obtained and analyzed 603. In some embodiments, determining whether scalar interpolation is beneficial for the vectorized loop may include determining 605 available scalar resources for scalar interpolation. The data obtained 602 may include, but is not limited to, data from hardware performance counters of a processor, which measure resource utilization within the processor. If the cost analysis (see determination 604) indicates that scalar interpolation is not beneficial, the method does not apply the transformation and terminates. If scalar interpolation is deemed beneficial, the method may also include determining, based on the input data obtained 202, a number of scalar iterations of the vector loop to be interpolated into the loop. The number of interpolation iterations may depend on the utilization balance between scalar and vector resources of the processor. Based on determining 605 the number of scalar iterations, the method may further include applying 606 a transformation, wherein equivalent scalar iterations of the vector loop are generated and interpolated into the vector loop.

[0082] In method 600 , runtime behavior can be captured before applying the scalar interpolation transformation. Thus, in some embodiments, method 600 can produce better performing code than method 500 which relies on a static cost model that does not understand the runtime behavior of the program.

[0083] It will be appreciated that the program first needs to be compiled and executed to generate the runtime information or behavior of the loop 602. Method 600 may be sensitive to program input, and the same program may have different runtime behaviors depending on the program input. For example, one input set for the program may result in different runtime behaviors compared to different input sets. Therefore, in some embodiments, the method may further include: generating multiple versions of the loop code and selecting the best performing version for each input set. The best performing version may be selected by, but is not limited to, measuring the processor's resource utilization, measuring the latency of the loop, and measuring the instruction throughput of the loop using hardware performance counters.

[0084] Figure 7 A scalar interpolation method using a machine learning model is shown according to one aspect. Method 700 can use a machine learning module to determine the profitability of scalar interpolation and make decisions about factors used for code generation. According to one aspect, a machine model 720 can be used to determine whether a vectorized loop will benefit from scalar interpolation.

[0085] Method 700 begins 701 with a vectorized loop in a precompiled program. In some embodiments, the vectorized loop is generated from an original program that was vectorized during compilation. In some embodiments, the original program may have scalar loops that a static compiler may vectorize to generate a "precompiled" program. Method 700 may operate on the "precompiled" program to add scalars that are interpolated based on either or both feedback and profiling information obtained when running the program.

[0086] In some embodiments, method 700 can be applied to any program that includes vectorized loops. For example, if the program has no vectorizable code at all, method 700 will not begin because there are no vectorized loops to start with. If method 700 finds a vectorized loop in the program, then method 700 can be applied.

[0087] In some embodiments, additional code transformations may be required before the method is applied. For example, one or more loop transformation optimization passes may be performed to simplify or optimize the code in the loop to make it more suitable for vectorization and scalar interpolation.

[0088] In some embodiments, method 700 may be applied in addition to or after method 500. For example, the vectorized loop at 701 may refer to the selected 510 loop of method 500.

[0089] The method may also include: performing a 702 legality check to determine whether it is legal for scalar interpolation to be applied to the vectorized loop. If it is not legal, the method ends and the scalar interpolation is not applied. If the scalar interpolation is legal, the method may also include: obtaining and analyzing 703 runtime data or information of the vectorized loop to determine whether applying the scalar interpolation has a performance advantage as determined in 704. For example, the runtime data collected when the program is executed can be input into the compiler. The method may also include: determining 704 whether scalar interpolation is beneficial to the vectorized loop based on the data obtained and analyzed 702. In some embodiments, whether scalar interpolation is beneficial to the vectorized loop 704 can be determined by a machine learning model 720. The data obtained 702 can be passed to the machine learning module 720 to determine the profitability of the scalar interpolation 704.

[0090] Thus, once the machine learning model 720 is constructed, at compile time, the method can include consulting the profitability machine learning model 720 to make a decision 704 regarding applying scalar interpolation. The method can also include consulting one or more code generation machine learning models 722 to determine one or more parameters for code generation. The data obtained 702 can also be passed to the one or more machine learning models 722 to determine one or more parameters for code generation. For example, a compiler code generator can consult the code generation machine learning model 722 to determine factors (one or more parameters) for code generation.

[0091] In some embodiments, target processor characteristics 724 may be used as input to each of machine learning models 720 and 722 used to make decisions according to method 700. Target processor characteristics may include, but are not limited to, the number of scalar processing resources, the number of vector processing resources, latency and throughput of each scalar and vector instruction.

[0092] According to one aspect, one or more machine learning models 722 may be used to determine one or more parameters for generating code for a vectorized loop that uses scalar interpolation. The one or more parameters may include a loop expansion factor. The loop expansion factor may determine how many loop iterations should be expanded to create an appropriate amount of computation to maintain efficient utilization of vector and scalar units without creating unfavorable register pressure. The one or more parameters may also include a scalar interpolation factor that determines how many scalar computations should be interpolated in the loop to utilize the scalar units in the processor. The one or more parameters may also include an instruction schedule that determines the order in which vector and scalar instructions appear in the generated code to improve performance (e.g., reduce execution time).

[0093] Machine learning models 720 and 722 can be constructed by specifying a set of features extracted from loop nests and a set of features describing relevant parts of the processor architecture. Examples used to train machine learning models may come from a large number of related programs or from synthetically constructed program examples to cover a wider range of possible programs for which code can be generated. Processor features used to train machine learning models (e.g., target processor features 724) may come from various possible designs of the architecture of vector and scalar units and may not be limited to existing architectures.

[0094] In some embodiments, a single machine learning model 722 may be used to make a decision (e.g., to determine one or more parameters for generating code for a vectorized loop using scalar interpolation). In some embodiments, more than one machine learning model 722 may be used to determine one or more parameters for generating code for a vectorized loop using scalar interpolation. In some embodiments, a separate machine learning model 722 may be used to determine each of the one or more parameters.

[0095] The method may also include interpolating 705 one or more scalar iterations into a vectorized loop based on the results of the one or more machine learning models 722 .

[0096] According to one aspect, method 700 may not require designing and adjusting equations for decision 704 regarding the profitability of applying scalar interpolation or scalar interpolation factors 722 for code generation because these decisions are made by the machine learning model.

[0097] It will be appreciated that method 700 may require building machine learning models 720 and 722, including creating a sufficient training set using a sample of both possible computer programs and related processor architecture designs. The cost of creating machine learning models in terms of design time, processing time, and energy consumption may be significant. Method 700 may also require running inference on the machine learning models during code generation.

[0098] Figure 8 A scalar interpolation method according to one aspect is shown. Method 800 is an automatic tuning method in which a compiler generates multiple versions of a vectorized loop, and each of the multiple versions runs k iterations of the vectorized loop at runtime. After k iterations, the best performing version is selected to execute the remainder of the loop.

[0099] Method 800 begins 801 with a vectorized loop in a precompiled program. In some embodiments, the vectorized loop is generated from an original program that was vectorized during compilation. In some embodiments, the original program may have scalar loops that a static compiler may vectorize to generate a "precompiled" program. Method 800 may operate on the "precompiled" program to add scalars that are interpolated based on either or both feedback and profiling information obtained when running the program.

[0100] In some embodiments, method 800 can be applied to any program that includes vectorized loops. For example, if the program has no vectorizable code at all, method 800 will not begin because there are no vectorized loops to start with. If method 800 finds a vectorized loop in the program, then method 800 can be applied.

[0101] In some embodiments, additional code transformations may be required before the method is applied. For example, one or more loop transformation optimization passes may be performed to simplify or optimize the code in the loop to make it more suitable for vectorization and scalar interpolation.

[0102] In some embodiments, method 800 may be applied after method 500. For example, the vectorized loop at 801 may refer to the selected 510 loop of method 500.

[0103] The method may also include: performing a legality check 802 to determine whether it is legal to apply scalar interpolation to the vectorized loop. If it is not legal, the method ends and the scalar interpolation is not applied. If the scalar interpolation is legal, the method may also include: obtaining and analyzing 803 runtime data or information of the vectorized loop to determine whether applying scalar interpolation has a performance advantage. For example, the runtime data collected when executing the program can be input into the compiler. The method may also include: generating 804 the original vector loop in the precompiled program that does not use scalar interpolation and multiple versions of the loop with various scalar interpolation factors. Thus, one version does not use scalar interpolation, while other versions use different amounts of scalar interpolation.

[0104] It will be appreciated that in method 800, a generate 804 operation is performed instead of the following operations: determining whether the loop will benefit from scalar interpolation (e.g., 604 with respect to method 600 and 704 with respect to method 700) and determining the amount of scalar resources and scalar interpolation (e.g., 605 with respect to method 600 and 705 with respect to method 700).

[0105] The method may further include, at execution time, executing 805 k iterations of the loop for each version of the generated loop. The execution time of executing the k iterations may be measured. The value of k may be an implementation parameter that can be determined experimentally.

[0106] The method may also include, at runtime, selecting 806 the best performing version of the loop for the processor executing the code (e.g., the loop with the shortest execution time, or other performance criteria as would be appreciated by one skilled in the art). The method may also include executing 807 the remainder of the loop using the selected best performing version of the loop. It will be appreciated that in method 800, the performance advantage may be measured or determined by running the loop for "k" iterations of each version of the loop generated 804. The best performing version may then be selected for execution.

[0107] Thus, the best loop is selected by executing "k" iterations of different versions of the loop. If the loop does not benefit from scalar interpolation, running "k" iterations may indicate that the loop version without scalar interpolation performs best, wherein the method may continue to execute the selected 806 loop without scalar interpolation.

[0108] Method 800 may require an effective and efficient way to accurately measure the execution time of k iterations of a loop. Method 800 may be independent of processor architecture design. Since the decision regarding the scalar interpolation factor is not specific to a given processor architecture design, method 800 may be applicable to a variety of processor designs.

[0109] It will be appreciated that method 800 may incur some additional overhead due to code generation 804, executing 805 multiple versions of the loop, and selecting 806 the best performing version. This overhead can be reduced by performing an estimate at compile time to reduce the number of loop versions to be generated 804.

[0110] According to one aspect, scalar computations can be interpolated within vectorized loops. Since scalar computation resources are not idle while executing a vectorized loop, scalar interpolation can improve processor resource utilization. Consequently, the overall execution time of the loop can be faster.

[0111] According to one aspect, one or more methods herein are described for determining when scalar interpolation may be legal. Determining when scalar interpolation may be legal can prevent the generation of incorrect code. According to one aspect, one or more methods herein are described for determining when scalar interpolation is advantageous or beneficial. Determining when scalar interpolation is advantageous can prevent the generation of code that performs worse than traditional vectorized code for a loop.

[0112] According to one aspect, decisions about whether scalar interpolation is legal and whether to apply the scalar interpolation can be made at runtime. Making these decisions at runtime can provide a more general and easily applicable code transformation approach for newer processor architectures, as the decision-making process used in the compiler may not need to be changed.

[0113] According to one aspect, multiple versions of a vectorized loop can be generated, and the best performing version of the multiple versions can be selected to execute the code. The selection can be made statically at compile time or at runtime. Since the selection can be based on actual values, such as the actual trip count of the loop and parameters of the processor architecture, it may be advantageous to select the best performing loop version at runtime. In some embodiments, the runtime decision (e.g., the selection at runtime) can use a just-in-time compilation approach, wherein an analytical model is used to select the best loop version based on the values ​​found at runtime. In some embodiments, the runtime decision can use an automatic tuning approach, wherein various alternatives are run, each for a relatively small number of loop iterations, the execution time of the loop iterations is measured, and then the best option is selected.

[0114] Although one or more aspects are described in the context of compiler loop vectorization (wherein the compiler applies vectorization to loops identified in a program), one or more aspects may also be applied to straight-line code (code that does not necessarily reside in loops). According to one aspect, a compiler transformation optimization pass may apply vectorization to straight-line code. Applying vectorization to straight-line code may be referred to as superword-level parallelism (SLP) vectorization. SLP vectorization may identify independent, isomorphic instructions in straight-line code and replace them with equivalent SIMD instructions.

[0115] Figure 9 An example of applying Superword-Level Parallelism (SLP) vectorization to code suitable for vectorization is shown. Code 900 is an independent isomorphic instruction suitable for vectorization 'before-slp'. Code 902 is based on applying SLP vectorization 'after-slp' to code 900. Scalar loads from A[i] and A[i+1] access continuous data and can be expressed as a single SIMD load accessing two elements. The same transformation can be applied to scalar loads from B[i] and B[i+1]. Subsequent multiplication and storage instructions can also be expressed as SIMD instructions. The code obtained after SLP vectorization is shown in 'after-slp' code 902.

[0116] Figure 10 Exemplary code according to one aspect is shown, showing conventional SLP vectorization 'after-slp' applied to scalar code 'before-slp' and SLP vectorization 'after-slp-si' using scalar interpolation.

[0117] Code 1000 is a straight-line code that can be converted using traditional SLP vectorization. Traditional SLP vectorization 'after-slp' 1002 applies vectorization to all applicable instructions in the scalar code 'before-slp'. The large number of SIMD instructions generated by SLP vectorization may lead to over-utilization of vector resources and under-utilization of scalar resources. Scalar interleaving can be applied to straight-line code vectorization, such as SLP vectorization. According to one aspect, scalar interpolation in the context of SLP vectorization can generate code shown as 'after-slp-si' 1004, in which some scalar instructions are not vectorized (i.e., loads and related add instructions from C[i], C[i+1], D[i], and D[i+1]), in contrast to traditional SLP vectorization, in which all applicable instructions are vectorized. It will be appreciated that scalar interpolation applied in SLP vectorization can also achieve the same benefit of balancing processor resource utilization.

[0118] According to one aspect, the scalar interpolation transformation can be applied by a compiler.In some aspects, the application of scalar interpolation (whether static or feedback driven) can require employing some cost model to determine the number of scalar iterations to interpolate into a vector loop.

[0119] In some aspects, the number of scalar iterations to be applied can be dynamically determined by the processor at runtime. A compiler can generate information marking vector code as legal for scalar interpolation, which is then sent to the processor during program execution. When the processor executes vector code marked as legal for scalar interpolation, the processor can dynamically determine the number of scalar iterations to be interpolated into the vectorized code. It will be appreciated that dynamically determining the one or more scalar iterations to be interpolated eliminates the need for cost models used in static compilation and the generation of complex code, such as is used in feedback-driven, machine learning, and automated tuning approaches.

[0120] According to one or more aspects, other computational domains may also benefit from using scalar operations instead of vector operations, or mixed with vector operations. For example, when a scalar unit is present in a graphics processing unit (GPU), a scalar unit may be used instead of a vector unit. In graphics processing, sometimes the input parameters for a given operation may all be the same. In such cases, performing scalar operations rather than performing SIMD calculations may be advantageous in terms of resource utilization and energy consumption. According to one aspect, scalar interpolation may be suitable for application in the context of a GPU. The one or more methods described herein for determining when scalar interpolation is legal and when it is beneficial may be applicable to the GPU case.

[0121] According to one or more aspects, scalar interpolation can be applied in the context of accelerators. Similar to the case of GPUs, in some types of accelerators (e.g., accelerators designed for ray tracing), performing scalar operations rather than SIMD calculations can be advantageous in terms of resource utilization and energy consumption.

[0122] Figure 11 A method for balancing computing resources according to one aspect is shown. Method 1100 can balance computing resources, such as processor resources, during a static compilation process. Method 1100 can include generating 1101 a plurality of vectorized loops of scalar code. The method can also include interpolating 1102 one or more scalar iterations of the scalar loop into each of the plurality of vectorized loops to generate a plurality of scalar interpolation vectorized loops. The method can also include selecting 1103 a scalar interpolation vectorized loop from the plurality of scalar interpolation vectorized loops based on a cost model.

[0123] The method may further include determining that vectorization of the scalar loop is legal. The method may further include determining that scalar interpolation is legal for each of the plurality of vectorized loops. The method may further include determining, for each of the plurality of vectorized loops, a number of scalar iterations to interpolate based on available scalar resources. Interpolating the one or more scalar iterations into each of the plurality of vectorized loops may be based on the determined number of scalar iterations.

[0124] The cost model may be based on a heuristic method calculated based on the number of instructions in each of the plurality of interleaved and scalar interpolation vectorization loops. The cost model may also be based on the latency of the instructions in each of the plurality of interleaved and scalar interpolation vectorization loops. The cost model may also be based on the instruction-level parallelism (ILP) present in each of the plurality of interleaved and scalar interpolation vectorization loops.

[0125] The method may further include interleaving one or more vectorized loops of the plurality of vectorized loops to generate one or more interleaved vectorized loops, the interleaving being based on available vector resources.

[0126] Interpolating one or more scalar iterations of a scalar loop into each vectorized loop in a plurality of vectorized loops to generate a plurality of scalar interpolation vectorized loops may also include interpolating one or more scalar iterations of a scalar loop into each interleaved vectorized loop in one or more interleaved vectorized loops to generate one or more interleaved and scalar interpolation vectorized loops.

[0127] Selecting a scalar interpolation vectorization loop from the plurality of scalar interpolation vectorization loops according to the cost model may include selecting one of the following according to the cost model: a plurality of scalar interpolation vectorization loops and one or more interleaved and scalar interpolation vectorization loops.

[0128] Figure 12 Another method for balancing computing resources according to one aspect is shown. Method 1200 can balance computing resources, such as processor resources, based on feedback or profiling information obtained during program execution. Method 1200 can include obtaining 1201 a vectorized loop from a program. The method can also include interpolating 1202 one or more scalar iterations of the vectorized loop into the vectorized loop to generate a scalar interpolation vectorized loop.

[0129] Obtaining a vectorized loop from a program includes vectorizing the program. The program may be a precompiled program. The method may further include determining whether scalar interpolation is legal for the vectorized loop. The method may further include obtaining runtime data for the vectorized loop.

[0130] The obtained runtime data may be performance data indicating resource utilization. The method may further include: determining, based on the obtained runtime data, that scalar interpolation is beneficial to the vectorized loop in terms of resource utilization.

[0131] Interpolating one or more scalar iterations of the vectorized loop into the vectorized loop may include interpolating based on the acquired runtime data. Interpolating one or more scalar iterations of the vectorized loop into the vectorized loop may include generating one or more equivalent scalar iterations of the vectorized loop. Interpolating one or more scalar iterations of the vectorized loop into the vectorized loop may also include interpolating the generated one or more equivalent scalar iterations into the vectorized loop.

[0132] The method may further include determining available scalar resources for scalar interpolation. The method may further include determining a number of scalar iterations to interpolate into the vectorized loop based on one or more of: the retrieved runtime data and the available scalar resources. Interpolating the one or more scalar iterations into the vectorized loop may be based on the determined number of scalar iterations.

[0133] The method may further include scheduling vector instructions and scalar instructions in the scalar interpolation vectorization loop in order of performance in terms of execution time to generate a scheduled scalar interpolation vectorization loop.

[0134] The method may further include: unrolling one or more iterations of the vectorized loop to generate an unrolled and scalar interpolation vectorized loop. The method may further include: determining a number of iterations of the vectorized loop to unroll based on one or more of: retrieved runtime data, available scalar resources, and available vector resources. The unrolling of the one or more iterations of the vectorized loop may be based on the determined number of iterations of the vectorized loop to unroll.

[0135] The method may further include scheduling vector instructions and scalar instructions in the unrolled and scalar interpolation vectorized loop in a performance order based on execution time to generate a scheduled unrolled and scalar interpolation vectorized loop.

[0136] Determining, based on the acquired runtime data, that scalar interpolation is beneficial to the vectorized loop in terms of resource utilization may include: determining, through a first machine learning model, that scalar interpolation is beneficial to the vectorized loop in terms of resource utilization. Determining a number of scalar iterations to be interpolated into the vectorized loop may include: determining the number of scalar iterations through a second machine learning model. Determining a number of iterations of the vectorized loop to be unrolled may include: determining the number of iterations of the vectorized loop to be unrolled through a third machine learning model. The third machine learning model may be different from or the same as the second machine learning model. Scheduling vector instructions and scalar instructions in sequence may include: scheduling vector instructions and scalar instructions through a fourth machine learning model. The fourth machine learning model may be different from or the same as the third machine learning model.

[0137] Figure 13 Another method for balancing computing resources according to one aspect is shown. Method 1300 may include generating 1301 multiple versions of a vectorized loop. The method may also include interpolating 1302 one or more scalar iterations of the vectorized loop into each of the multiple versions of the vectorized loop to generate multiple scalar interpolation vectorized loops. The method may also include selecting 1303 a version of the vectorized loop from the vectorized loop and the multiple scalar interpolation vectorized loops.

[0138] The method may further include executing the vectorized loop and each of the plurality of scalar interpolation vectorized loops for k iterations of the vectorized loop. The vectorized loop may terminate at an nth iteration, where n may be greater than k. The method may further include measuring an execution time for each of the vectorized loop and the plurality of scalar interpolation vectorized loops based on the k iterations. Selecting a version of the vectorized loop may be based on the measured execution time.

[0139] The method may further include: obtaining runtime data of the vectorized loop. The method may further include: determining whether scalar interpolation is legal for the vectorized loop based on the obtained runtime data.

[0140] Figure 14 An apparatus 1400 according to different aspects of the present invention is shown, wherein the apparatus 1400 can perform any or all of the operations of the above-described methods and features explicitly or implicitly described herein. For example, a computer equipped with network capabilities can be configured as the apparatus 1400. In some aspects, the apparatus 1400 can be a device connected to a network infrastructure via a wireless interface, such as a mobile phone, a smartphone, or other device that can be classified as a user equipment (UE). In some aspects, the apparatus 1400 can be a machine type communication (MTC) device (also known as a machine-to-machine (m2m) device) or another such device that can be classified as a UE although it does not provide direct services to a user. In some aspects, the apparatus 1400 can be used to implement one or more aspects described herein. For example, the apparatus 1400 can be a network device (a router, an edge router, or a switch) used to implement or perform one or more operations described herein.

[0141] As shown, the device 1400 may include a processor 1410, such as a central processing unit (CPU) or a special-purpose processor (e.g., a graphics processing unit (GPU) or other such processor unit), a memory 1420, a non-transitory mass storage 1430, an input / output interface 1440, a network interface 1450, and a transceiver 1460, all of which are communicatively coupled via a bidirectional bus 1470. According to certain aspects, any or all of the elements described may be utilized, or only a subset of the elements may be utilized. In addition, the device 1400 may include multiple instances of certain elements, such as multiple processors, multiple memories, or multiple transceivers. Furthermore, elements of the hardware device may be directly coupled to other elements without the need for a bidirectional bus. Additionally or alternatively, in addition to processors and memories, other electronic components such as integrated circuits may be used to perform the required logical operations.

[0142] The memory 1420 may include any type of non-transitory memory, such as static random-access memory (SRAM), dynamic random-access memory (DRAM), synchronous DRAM (SDRAM), read-only memory (ROM), or any combination thereof. The mass storage element 1430 may include any type of non-transitory storage device, such as a solid-state drive, a hard disk drive, a magnetic disk drive, an optical disk drive, a USB drive, or any computer program product for storing data and machine-executable program code. According to certain aspects, the memory 1420 or the mass storage 1430 may record thereon statements and instructions executed by the processor 1410 for performing any of the above-described method operations.

[0143] Various aspects of the present invention may be implemented using electronic hardware, software, or a combination thereof. In some aspects, this may be implemented by one or more computer processors executing program instructions stored in a memory. In some aspects, the present invention may be implemented in part or in whole in hardware, for example, by rapidly executing processing operations using one or more field programmable gate arrays (FPGAs) or application specific integrated circuits (ASICs).

[0144] It should be understood that although specific aspects of the technology have been described herein for illustrative purposes, various modifications may be made without departing from the scope of the technology. Accordingly, the specification and drawings are to be regarded only as illustrative of the invention as defined by the appended claims, and are intended to cover any and all modifications, variations, combinations or equivalents within the scope of the invention. Specifically, a computer program product or program element for storing machine-readable signals, or a program storage or memory device such as a magnetic wire, tape, disk or light, optical tape or optical disc is provided, within the scope of the technology, for controlling the operation of a computer according to the method of the technology and / or constructing some or all of its components according to the system of the technology.

[0145] The actions associated with the methods described herein may be implemented as coded instructions in a computer program product. In other words, a computer program product is a computer-readable medium on which software codes are recorded to perform the methods when the computer program product is loaded into a memory and executed on a microprocessor of a wireless communication device.

[0146] Furthermore, each operation of the method may be executed on any computing device (e.g., a personal computer, server, PDA, etc.) according to one or more of one or more program elements, modules, or objects generated from any programming language (e.g., C++, Java, etc.), or a portion of one or more program elements, modules, or objects. Furthermore, each operation or a file or object implementing each operation may be executed by dedicated hardware or circuit modules designed for this purpose.

[0147] Through the description of the above aspects, the present invention can be implemented only by hardware, or by software and a necessary general hardware platform. Based on this understanding, the technical solution of the present invention can be embodied in the form of a software product. The software product can be stored in a non-volatile or non-transient storage medium, which can be a compact disc read-only memory (CD-ROM), a USB flash drive, or a removable hard disk. The software product includes a plurality of instructions that enable a computer device (a personal computer, a server, or a network device) to execute the methods provided in various aspects of the present invention. For example, such execution may correspond to a simulation of the logical operations described herein. The software product may additionally or alternatively include a plurality of instructions that enable a computer device to execute operations for configuring or programming a digital logic device according to various aspects of the present invention.

[0148] Although the present invention has been described with reference to certain features and aspects thereof, it is apparent that various modifications and combinations of the present invention may be made without departing from the present invention. Therefore, the specification and drawings are to be regarded only as illustrative of the present invention as defined by the appended claims, and are intended to cover any and all modifications, variations, combinations or equivalents within the scope of the present invention.

Claims

1. A method, characterized in that include: Generate multiple vectorized loops of scalar code; interpolating one or more scalar iterations of the scalar loop into each vectorized loop of the plurality of vectorized loops to generate a plurality of scalar interpolation vectorized loops; A scalar interpolation vectorization loop is selected from the plurality of scalar interpolation vectorization loops according to a cost model.

2. The method according to claim 1, characterized in that Also includes: It is determined that it is legal to vectorize the scalar loop.

3. The method according to claim 1 or 2, characterized in that Also includes: A scalar interpolation is determined to be legal for each of the plurality of vectorized loops.

4. The method according to any one of claims 1 to 3, characterized in that Also includes: For each vectorized loop in the plurality of vectorized loops, a number of scalar iterations to be interpolated is determined based on available scalar resources, wherein the one or more scalar iterations interpolated into each vectorized loop in the plurality of vectorized loops are based on the determined number of scalar iterations.

5. The method according to any one of claims 1 to 4, characterized in that The cost model is based on one or more of the following: a heuristic method for calculating based on the number of instructions in each of the plurality of interleaved and scalar interpolation vectorized loops; a latency of instructions in each of the plurality of interleaved and scalar interpolation vectorized loops; Instruction-level parallelism (ILP) exists in each of the plurality of interleaved and scalar interpolation vectorized loops.

6. The method according to any one of claims 1 to 5, characterized in that Also includes: interleaving one or more vectorized loops of the plurality of vectorized loops to generate one or more interleaved vectorized loops, the interleaving being based on available vector resources, wherein: The interpolating one or more scalar iterations of the scalar loop into each of the plurality of vectorized loops to generate a plurality of scalar interpolation vectorized loops further comprises: interpolating one or more scalar iterations of the scalar loop into each of the one or more interleaved vectorized loops to generate one or more interleaved and scalar interpolation vectorized loops; The selecting one of the multiple scalar interpolation vectorization loops according to the cost model includes: selecting one of the following according to the cost model: the multiple scalar interpolation vectorization loops and the one or more interleaved and scalar interpolation vectorization loops.

7. A method, characterized in that include: Get vectorized loops from your program; One or more scalar iterations of the vectorized loop are interpolated into the vectorized loop to generate a scalar interpolation vectorized loop.

8. The method according to claim 7, characterized in that The obtaining of a vectorized loop from a program includes: vectorizing the program.

9. The method according to claim 7, characterized in that The program is a precompiled program.

10. The method according to any one of claims 7 to 9, characterized in that Also includes: Determine whether scalar interpolation is legal for the vectorized loop.

11. The method according to any one of claims 7 to 10, characterized in that Also includes: Get runtime data for the vectorized loop.

12. The method according to claim 11, characterized in that Also includes: Determining that scalar interpolation benefits the vectorized loop in terms of resource utilization is based on the acquired runtime data, wherein the acquired runtime data is performance data indicative of resource utilization.

13. The method according to claim 11 or 12, characterized in that Interpolating the one or more scalar iterations of the vectorized loop into the vectorized loop includes interpolating according to the acquired runtime data.

14. The method according to any one of claims 7 to 13, characterized in that Interpolating one or more scalar iterations of the vectorized loop into the vectorized loop comprises: generating one or more equivalent scalar iterations of the vectorized loop; The generated one or more equivalent scalar iterations are interpolated into the vectorized loop.

15. The method according to claim 13, characterized in that Also includes: Determines the available scalar resources for scalar interpolation.

16. The method according to claim 15, characterized in that Also includes: A number of scalar iterations to be interpolated into the vectorized loop is determined based on one or more of: the acquired runtime data and the available scalar resources, wherein the one or more scalar iterations interpolated into the vectorized loop are based on the determined number of scalar iterations.

17. The method according to any one of claims 7 to 16, characterized in that Also includes: The vector instructions and the scalar instructions in the scalar interpolation vectorization loop are scheduled in a performance order based on execution time to generate a scheduled scalar interpolation vectorization loop.

18. The method according to any one of claims 7 to 16, characterized in that Also includes: One or more iterations of the vectorized loop are unrolled to generate an unrolled and scalar interpolation vectorized loop.

19. The method according to claim 18, characterized in that Also includes: A number of iterations of the vectorized loop to be unrolled is determined based on one or more of: the acquired runtime data, the available scalar resources, and the available vector resources, wherein the unrolling of the one or more iterations of the vectorized loop is based on the determined number of iterations of the vectorized loop to be unrolled.

20. The method according to claim 18 or 19, characterized in that Also includes: The vector instructions and the scalar instructions in the unrolled and scalar interpolation vectorized loop are scheduled in a performance order based on execution time to generate a scheduled unrolled and scalar interpolation vectorized loop.

21. The method according to any one of claims 12 to 20, characterized in that Determining, based on the acquired runtime data, that scalar interpolation is beneficial to the vectorized loop in terms of resource utilization includes: determining, through a first machine learning model, that scalar interpolation is beneficial to the vectorized loop in terms of resource utilization.

22. The method according to claim 21, characterized in that Determining the number of scalar iterations to be interpolated into the vectorized loop includes determining the number of scalar iterations through a second machine learning model.

23. The method according to claim 22, characterized in that Determining the number of iterations of the vectorized loop to be expanded includes: determining the number of iterations of the vectorized loop to be expanded through a third machine learning model, wherein the third machine learning model is different from or the same as the second machine learning model.

24. The method according to claim 23, wherein The sequential scheduling of vector instructions and scalar instructions includes: scheduling the vector instructions and the scalar instructions through a fourth machine learning model, wherein the fourth machine learning model is different from or the same as the third machine learning model.

25. A method, characterized in that include: Generate multiple versions of vectorized loops; interpolating one or more scalar iterations of the vectorized loop into each of the plurality of versions of the vectorized loop to generate a plurality of scalar interpolation vectorized loops; A version of the vectorized loop is selected from the vectorized loop and the plurality of scalar interpolation vectorized loops.

26. The method according to claim 25, characterized in that Also includes: For k iterations of the vectorized loop, execute: the vectorized loop; Each scalar interpolation vectorization loop in the plurality of scalar interpolation vectorization loops, wherein the vectorization loop terminates at an nth iteration, n>k.

27. The method according to claim 25 or 26, characterized in that Also includes: Based on the k iterations, execution time is measured for each of: the vectorization loop and the plurality of scalar interpolation vectorization loops.

28. The method according to claim 27, characterized in that The selecting a version of the vectorized loop is based on the measured execution time.

29. The method according to any one of claims 25 to 28, characterized in that Also includes: Obtaining runtime data of the vectorized loop; A scalar interpolation is determined to be legal for the vectorized loop based on the obtained runtime data.

30. A device, characterized in that include: at least one processor; At least one machine-readable medium storing executable instructions, which, when executed by the at least one processor, cause the apparatus to perform the method according to any one of claims 1 to 29.

31. A computer device, characterized in that: A non-transitory computer-readable medium having stored thereon instructions which, when executed by a computer processor, cause the computer to perform the method according to any one of claims 1 to 29.

Images