Computer-implemented method for estimating software execution parameters

Abstract

The present invention relates to a computer-implemented method for estimating a parameter for executing software on a destination platform, so as to estimate the parameter, which may be time or energy, for example, in a more optimal manner, by means of dynamic simulation on a host platform which has a main processor; comprising a phase that is the portion of application software code whose parameters for execution on the destination platform are to be estimated, the code being a set of instructions, comprising the following sequence of steps: - starting the phase with an initialisation function of a hardware performance counter, resetting the counter to zero; - executing the code; - executing a read function of the performance counter, providing the value of same; - ending the phase by estimating the parameter by means of analysis with an estimation function based on the value of the counter; such that the parameter is estimated during the simulation.

Description

[0001] COMPUTER-IMPLEMENTED METHOD FOR ESTIMATING SOFTWARE PERFORMANCE PARAMETERS

[0002] TECHNICAL FIELD OF THE INVENTION

[0003] The present invention falls within the field of simulation and analysis of performance, such as time invested or energy consumed, of a computer program or software on a platform other than where it is intended to be executed.

[0004] BACKGROUND OF THE INVENTION

[0005] Software simulation is the execution of the code of a software application on a platform or equipment capable of modeling the parameters derived from its execution, usually the execution time and potentially others such as energy consumption, delays, performance ("throughput"), etc.

[0006] In most cases, simulation is necessary in order to know the execution parameters of the software on a platform, hardware, target, other than the host platform on which the simulation takes place. In many cases, the target platform does not even exist since it is being designed at the same time. In other cases, the target platform or platforms are available but their deployment is complex enough, fault detection difficult enough or the cost associated with their detection high enough to justify the development of virtual models. This is why the simulation is performed on a host platform (or platforms) other than the target platform.

[0007] This section includes bibliographic references with numbers in brackets. These references are listed at the end of the section.

[0008] Software simulation has long been an active field of study in which different solutions have been proposed depending on the degree of compromise required between precision and simulation speed [1-3]. We understand precision as the degree of similarity between the estimates obtained from the software model and the real ones, provided by the execution of said software on the target platform, that is, the real platform whose parameters are to be analyzed. We understand the simulation speed "vs" as the quotient between the simulated time and the simulation time: vs = tso / tsn where "tso" is the simulated time, that is, the execution time of the software to be executed on the target platform; "tsn" is the simulation time, that is, the time required for the simulator to simulate the execution of the software on the host platform on the target platform. Depending on the objective of the simulation, several alternatives appear.Software simulation technologies can be analyzed by the level of detail they provide.

[0009] Simulation at the register transfer level (RTL) using hardware description languages ​​(HDL) (such as VHDL) is very accurate, but it is very time consuming to simulate since the simulation speed is very low [1] , and is only useful in specific situations such as detailed verification of hardware before implementation.

[0010] With high accuracy, instruction set simulators (ISS) can speed up simulation, but it is still too slow for simulating complex applications, and is only useful in specific situations such as detailed verification of a processor architecture before its design. A well-known open source ISS is Gem5 [2].

[0011] Virtualization [3] achieves high simulation speeds. Just-In-Time (e.g., OVP [3] ) based virtualization is the main technique used today for software simulation. However, the accuracy is low since it is limited to knowing the number of instructions executed, and one cycle per instruction is usually assumed.

[0012] Host-compiled simulation has been a field of intense research in recent years, allowing it to reach a pre-commercial level of development scalable to islanded cyber-systems. This technology offers accuracy equivalent to commercial virtualization technologies at a significantly faster simulation speed. The primary method of cross-compiled simulation is to instrument the application code with the parameters (time, power, cache misses, etc.) of the target binary (or ASM) executed by a given processor (CPU), and to compile and execute the code on the host platform. Through this instrumentation, code execution on the host platform provides the execution parameters on the target platform, while maintaining functionality.With more than ten years of study and development, cross-compiled simulation is currently a mature technology for simulation and performance analysis [4]. The simulation speed of the technique is limited by the speed with which the annotation code executes. The size of the instrumentation code can vary depending on the desired accuracy level. Although cross-compiled simulation [4] improves simulation speed while maintaining accuracy, the continuous growth in complexity of electronic systems and, fundamentally, of the software applications they run, demand improvements in SW simulation speed on heterogeneous platforms while maintaining sufficient accuracy in execution times and energy and minimizing the effort of creating the model.

[0013] Typically, analysis is performed at the basic block (BB) level, i.e., code blocks whose execution has a single entry point, corresponding to the first instruction, and lacks internal conditional branches, ensuring the execution of all instructions from entry to exit. Simply noting the number of cycles involved in the execution of a code block has little impact on simulation speed, with the execution speed ("tsn") close to that of the application code on the host platform (which is the asymptotic minimum). The annotation code can be as simple as:

[0014] C+=C(BB)E+=E(BB) at the end of each BB basic block in the application code, where "C" is the number of cycles and "E" the energy required by the code execution, and "C(BB)" and "E(BB)" the contribution of the BB block to the total number of execution cycles and energy, respectively. Multiplying the number of cycles by the clock period of the target CPU at the time the code is executed gives the code execution time assuming an infinite cache. Here too, modeling the memory hierarchy decreases the simulation speed by increasing the complexity of the instrumentation code. However, the impact is smaller than in the virtualization case because it does not require a model of the "bus" and additional memory [5].

[0015] Machine learning has proven useful in many areas of electronic design automation. However, its application to system simulation and performance analysis is limited [6] . In [7] , C++ source code is analyzed for its functional complexity characteristics, conditional branches, memory accesses, etc. This data is fed into the training of a neural network for a set of performance benchmarks with corresponding execution times and energy on the target platform.

[0016] Performance counters have been used in several design and analysis tasks. Most works derive characteristics of the hardware or software on the platform from the performance counters on the same platform. This possibility can be applied to a graphics processor -GPU-. Based on the values ​​of the performance counters, the performance and power consumption of a particular GPU configuration is estimated [8]. The technique can be applied to a CPU. Its performance counters are used in proactive power and performance management of time-varying applications, deciding the next optimal configuration based on the performance values ​​of the previous one [9]. In

[0010] , the authors proposed a STT-RAM Cache Retention Time (SCART) model that uses a KNN classifier to predict the best retention time for an STT-RAM L1 cache.In

[0011] , the analysis is performed using information provided by the host CPU's performance counters. In this case, the neural network is trained with the counter data as input and the target CPU's execution time as the target. The problem with these techniques is that they are static. The code is executed and the neural network provides its speed and energy characteristics. This information can be fed back to the model to perform time and energy simulations in a new execution, but since this information is static, independent of the data and, consequently, of the specific execution flow of the program, the accuracy decreases significantly. To avoid instrumenting the code and to allow the technique to be applied to legacy or third-party binary code, the performance counter values ​​are read in consecutive phases of the application code execution on the host CPU.From these, the execution time of the code on the target CPU is estimated for each phase. From the correlation of execution on both CPUs, the execution time of the target CPU is deduced

[0012] .

[0017] The effects of memory architecture, and specifically cache misses, are not known, although this is the main effect on execution time and consumption. Simulating the memory hierarchy requires a model of the bus and the memory, which slows down the simulation

[0013] . The language mostly used to model the entire platform is SystemC

[0014] . The effort required to create the model on multi-processor platforms is high

[0015] .

[0018] None of these technologies is fast enough to allow the simulation and performance analysis of complex systems. In many cases, software applications simply run, without adding time to the model, i.e., assuming a zero target execution time. It is external events that increase the simulation time [16-17]. However, there are many situations related to the optimization or design of complex systems in which this simple software model is not sufficient, such as the design of system networks

[0018] or hardware / software co-design.

[0019] .

[0019] References cited:

[0020] [1] E. Villar: "VHDL: Standard Electronic Design Language", McGraw-Hill, 1997.

[0021] [2] www.gem5.org.

[0022]

[0003] https: / / www. ovpworld . or rg / technology .

[0023] [4] O. Bringmann, W. Ecker, A. Gerstlauer, et al., "The Next Generation of Virtual Prototyping: Ultra-fast Yet Accurate Simulation of HW / SW Systems", Proc, of DATE 2015.

[0024] [5] K. Lu, D. Muller-Gritschneder & U. Schlichtmann : "Fast Cache Simulation for Host-Compiled Simulation of Embedded Software", In Proceedings of the 2013 Design, Automation & Test in Europe (DATE 2013) , IEEE, 2013.

[0025] [6] G. Huang, J. Hu, Y. He, J. Liu, M. Ma, Z. Shen, J. Wu, Y. Xu, H.i Zhang, K. Zhong, X. Ning, Y. Ma, H. Yang, B. Yu, H. Yang and Yu Wang: "Machine Learning for Electronic Design Automation: A Survey", ACM Trans. Des. Autom. Electron. Systems, V.26, 1.5, 2021.

[0026] [7] V. Muttillo, P. Giammatteo and V. Stoico: "Statement- Level Timing Estimation for Embedded System Design Using Machine Learning Techniques", Proceedings of the ACM / SPEC International Conference on Performance Engineering, ICPE '21, ACM, 2021.

[0027] [8] G. Wu, J. L. Greathouse, A. Lyashevsky, N. Jayasena and D. Chiou: "GPGPU performance and power estimation using machine learning", 21st International Symposium on High Performance Computer Architecture (HPCA) , IEEE, 2015. [9] A. Iranfar, W. S. D. Souza, M. Zapater, K. Olcoz, S. X. d. Souza and D. Atienza, "A Machine Learning-Based Framework for Throughput Estimation of Time-Varying Applications in Multi-Core Servers, " 2019 IFIP / IEEE 27th International Conference on Very Large Scale Integration (VLSI-SoC) , 2019.

[0028]

[0010] D. Gajaria, K. Kuan and T. Adegbija, "SCART: Predicting STT-RAM Cache Retention Times Using Machine Learning, " 2019 Tenth International Green and Sustainable Computing Conference (IGSC) , 2019.

[0029]

[0011] X. Zheng, L.K., John & A. Gerstlauer: "LACross: Learning-Based Analytical Cross-Platform Performance and Power Prediction", Int. Journal of Parallel Programming, V.45, Springer, 2017.

[0030]

[0012] X. Zheng, H. Vikalo, S. Song, L. K. John and A. Gerstlauer: "Sampling-based binary-level cross-platform performance estimation", proc, of the Design, Automation & Test in Europe Conference & Exhibition (DATE) , IEEE, 2017.

[0031]

[0013] L. Zaourar et al. : "Multilevel simulation-based codesign of next generation HPC microprocessors, " 2021 International Workshop on Performance Modeling, Benchmarking and Simulation of High Performance Computer Systems (PMBS) , 2021.

[0032]

[0014] C. Silvano, W. Fornaciari and E. Villar, E. : "Multiobjective Design Space Exploration of Multiprocessor SoC Architectures: The MULTICUBE Approach", Springer, 2011.

[0033]

[0015] B. Bailey: "Design And Verification Methodologies Breaking Down", Semiconductor Engineering, January 12, 2023. https : / / semiengineering. com / design-and- veri float ion-methodologies -brea king- down / .

[0034]

[0016] N. Li, D. W. Oyler, M. Zhang, Y. Yildiz, I. Kolmanovsky & A. R. Girard: "Game Theoretic Modeling of Driver and Vehicle Interactions for Verification and Validation of Autonomous Vehicle Control Systems", in IEEE Transactions on Control Systems Technology, V.26, N.5, 2018.

[0017] N. Grigoropoulos and S. Lalis: "Simulation and Digital Twin Support for Managed Drone Applications", in proc, of the 2020 IEEE / ACM 24th International Symposium on Distributed Simulation and Real Time Applications (DS- RT) , Prague, Czech Republic, 2020.

[0035]

[0018] Z. Zhao, K.M. Barijough and A. Gerstlauer, Andreas: "Network-Level Design Space Exploration of Resource- Constrained Networks-of-Systems", ACM Transactions on Embedded Computing Systems, V.19, 1.4, No.22, ACM, 2020.

[0036]

[0019] S. Ha and J. Teich: "Handbook of Hardware / Sof tware Codesign", Springer, 2017.

[0037] DESCRIPCIÓN DE LA INVENCIÓN

[0038] The present invention is set forth and characterized in the independent claim, while the dependent claims describe other features thereof.

[0039] The object of the invention is a computer-implemented method for estimating at least one execution parameter of a software program on a target platform by means of simulation on a host platform. The technical problem to be solved is to configure the method to estimate the parameter in a more optimal manner than the prior art, for example, by reducing the time required for the estimation or improving its accuracy.

[0040] In view of the foregoing, the present invention relates to a computer-implemented method for estimating software execution parameters, such as the time invested or the energy consumed, on a target platform, by means of dynamic simulation on a host platform, which - the host platform - includes a main processor; phase is the portion of application software code whose execution parameters on the target platform are going to be estimated, the code being a set of instructions, regardless of the volume thereof, from one line of code to thousands of them; which is characterized by comprising the following stages in sequence:

[0041] -start of the instrumented phase with an initialization function of a hardware performance counter, said initialization function including a clearing or resetting of the performance counter;

[0042] -execution of the software code of the phase , ;

[0043] -execution of a performance counter reading function providing the performance counter value;

[0044] - end of the phase by parameter estimation through analysis with a parameter estimation function based on the performance counter value; such that parameter estimation takes place during the simulation.

[0045] Here we refer to a performance counter as a minimum, although there may be as many as the host platform has available.

[0046] The use of host platform counters dynamically, at simulation time, to estimate execution time and energy consumed on the target platform is a technological solution not known to the applicant. Its impact on software simulation is significant due to the advantages it provides in accuracy, speed and flexibility. For example, dynamic estimation allows the estimated execution time to be used directly in the simulation, thus making direct time simulation of the software code possible on any target platform. By "parameter" we mean any execution property of the software on the target platform that is desired to be estimated from the host platform counter(s). The most important parameters may be execution time and energy consumed, but the technique is applicable to any other property derivable from said counters.

[0047] Other advantages related to features of the dependent claims are cited here below in the detailed description.

[0048] BRIEF DESCRIPTION OF THE FIGURES

[0049] This descriptive report is complemented with a set of figures, illustrative of the preferred example, and never limiting the invention.

[0050] Figure 1 shows a diagram of the method of the invention, by means of a sequential representation of a phase, which advances in time from the left of the figure to the right, the host platform includes a main processor.

[0051] Figure 2 shows a schematic of the method when consecutive phases are chained together. Unlike Figure 1, the time progress is represented from top to bottom.

[0052] Figure 3 shows a schematic of the method of the invention, similar to the schematic representation of Figure 1, with the option that the host platform includes a main processor and a graphics processor.

[0053] DETAILED EXPLANATION OF THE INVENTION

[0054] The present invention is a computer-implemented method for estimating software execution parameters, such as time spent or energy consumed, on a target platform, by means of dynamic simulation on a host platform, which includes a main processor (CPU); phase (Fi) is the portion of application software code whose execution parameters on the target platform are to be estimated, the code being a set of instructions, regardless of their volume, from one line of code to thousands of them; comprising the following steps in sequence: -start of phase (Fi) instrumented with an initialization function of a hardware performance counter, said initialization function including clearing or resetting the performance counter; -execution of the software code of phase (Fi);-execution of a performance counter reading function providing the performance counter value;

[0055] -end of phase (Fi) by estimating the parameter using an analysis with a parameter estimation function based on the performance counter value; such that the parameter estimation takes place during the simulation.

[0056] Figure 1 shows a representation of the method, the performance counter(s) are initialized before the execution of the software code and are read immediately after, thus capturing the code execution events such as executed instructions, memory accesses, failures in accessing nearby memory, etc., on the host platform to subsequently determine the parameters on the target platform using code inserted after the application code. Figure 1 includes the different execution times of the Fi phase: anti c=counter start time, adt±=phase Fi software execution time, anti r =counter reading time, anti e =parameter estimation time .

[0057] One option to take into account the software execution time is that it is simulated as a delay during the simulation execution.

[0058] Another option, Figure 2, is that the simulation is performed by dividing the software into consecutive chained phases (Fi, Fi+1, Fi+2), being a concatenation of phases for temporal simulation of the complete software code, in such a way that the analysis affects the complete program and not isolated phases; the diamond in Figure 2 implies a condition of the software code.

[0059] Another option is that a concurrent software application can be simulated with several threads, which includes a simulation core that synchronizes the temporal execution of the different concurrent simulations and manages the progress of the global time, for which different performance counters are associated with the different threads on the host platform.

[0060] Another option is that the simulation speed is increased by using a neural network for estimation of the simulated parameter.

[0061] Another option, figure 3, is that the host platform includes, in addition to the main processor (CPU), a graphics processor (GPU) connected to it.

[0062] Another option is to run the simulation in a discrete event-driven manner, with the software running in cycles or with time-advanced synchronization points. Specifically, the time between synchronization points for parameter estimation is used by replacing the delay with the performance counter reading, then the parameter estimation is performed, then the performance counter is cleared, and finally the delay is included in the simulation execution.

[0063] Example 1

[0064] This example illustrates the GPU-based configuration as a very efficient one, although it could also be implemented with just a CPU. The configuration has been tested on a set of benchmarks. Functions from the MiBench [http: / / cucis.ece.northwestern.edu / projects / DMS / MineBenchDownload.html] and NU-MineBench-3.0.1 [M.R. Guthaus, J.S. Ringenberg, D. Ernst, T.M. Austin, T. Mudge and R.B. Brown: "MiBench: A free, commercially representative embedded benchmark suite", Proceedings of the Fourth Annual IEEE International Workshop on Workload Characterization (WWC-4), IEEE, 2001] benchmark suites have been used for training and performance evaluation. MiBench is a free and commercially representative embedded benchmark suite covering six categories: automotive, consumer, networking, office, security and telecommunications. NU-MineBench-3.0.1 is a data mining benchmark suite containing a combination of several representative data mining applications from different domains, such as bioinformatics, network intrusion, customer relationship management, and marketing. These two benchmark suites were chosen because they are widely used in computer architecture research, systems research, and high-performance computing, and provide a diverse set of workloads that can generate a rich and varied dataset for model training.

[0065] An Intel Xeon Gold was used as the host platform.

[0066] 6138 at 3.7 GHz. To access the CPU performance counters, the PAPI 6.0.0.1 library (Jan 29, 2020) was used with the following characteristics:

[0067] Table I: Number of execution cycles of the PAPI functions used.

[0068] From this data: average anti c = t_cycle * 13,433 = 3,630 ns ( PAPI_reset ( ) + PAPI_start ( ) ) average anti r = t_cycle * 10,097 = 2,750 ns ( PAPI_stop ( ) )

[0069] The functions of the above benchmarks are implemented with the code for cleaning (zeroing) and reading performance counters proposed in the invention using the PAPI library. This code is compiled and executed on the host platform. The execution provides the values ​​of the performance counters for each phase (Fi) executed.

[0070] An Arm_v8 Nvidia Jetson AGX has been chosen as the target CPU. The same set of benchmarks used to obtain the performance counter values ​​is implemented at exactly the same points using the "std: :dock()" function from the "chrono" library. In this way, the execution times of the phase (Fi) on the target platform that the network must estimate for each set of counter values ​​are obtained. All these values ​​​​constitute the dataset. This set is divided into two groups. The first group includes 90% of the functions and is used for network training. The second group, with 10% of functions, is dedicated to validation. The mean absolute error is used as an accuracy metric. The definition and training of the neural network are done with Python / Keras and exported in ONNX format to be used from C++ with CppFlow.

[0071] The trained neural network is used to make inferences on the evaluation part of the dataset.

[0072] The mean absolute error is (MAE): with a very narrow distribution of values: 26.5% at the 99th percentile and 5.1% at the 95th percentile, giving an average absolute error of 3.97%.

[0073] A Quadro RTX4000 was used as the GPU. The average estimation time on this machine is: anti e = 1.095 ps . Summing the average times we obtain the average time due to the modeling technology as: antm = anti c + anti r + anti e

[0074] From this data it is possible to calculate the simulation time added by the technology as: tsim = adt + Np*antm with "adt" the execution time of the application software on the host CPU, Np the number of phases into which the complete simulation has been divided and "antm" the average time added by the proposed technology.

[0075] Defining the modeling efficiency as: eff = adt / tsim = 1 / (1 + Np*antm / adt) = 1 / (1 + antm / (adt / Np) ) = 1 / (1 + antm / adtm) where "adtm" is the average execution time of the phases. In this case: adtm = 5,771 ps so the efficiency is: eff = 1 / (1 + 1,101.38 / 5,771) = 84 %.

Claims

CLAIMS 1. - A computer-implemented method for estimating software execution parameters, such as time spent or energy consumed, on a target platform, by means of dynamic simulation on a host platform, which includes a main processor (CPU); phase (Fi) is the portion of application software code whose execution parameters on the target platform are to be estimated, the code being a set of instructions; characterized in that it comprises the following stages in sequence: -start of phase (Fi) instrumented with an initialization function of a hardware performance counter, said initialization function including clearing or resetting the performance counter; -execution of the software code of phase (Fi); -execution of a performance counter reading function providing the performance counter value; -end of phase (Fi) by estimating the parameter using an analysis with a parameter estimation function based on the performance counter value; such that the parameter estimation takes place during the simulation.

2. -Method according to claim 1 wherein the execution time of the software is simulated as a delay during the execution of the simulation.

3. -Method according to claim 1 in which the simulation is performed by dividing the software into consecutive chained phases ( Fi , Fi+1 , Fi+2 ).

4. -Method according to claim 1 in which the simulation is carried out a concurrent software application with several threads, which includes a simulation core that synchronizes the temporal execution of the different concurrent simulations and manages the progress of the global time, for which different performance counters are associated with the different threads on the host platform.

5. -Method according to claim 1 wherein the simulation speed is increased by using a neural network for estimating the simulated parameter.

6. -Method according to claim 1, wherein the host platform, in addition to the main processor (CPU), includes a graphics processor (GPU) connected thereto.

7. -Method according to claim 1 wherein the simulation is executed in a discrete event-driven manner, executing the software in cycles or with time advance between synchronization points.

8. -Method according to claim 7, wherein the time between synchronization points for estimating the parameter is used by replacing the delay with the reading of the performance counter, then the parameter is estimated, then the performance counter is cleaned, and finally, the delay is included in the execution of the simulation.

Images