Hardware counter multiplexing estimation implementation method

Abstract

The application discloses a hardware counter reuse estimation implementation method, which constructs a training set and trains a Transform model in an offline stage, and performs fitting on estimation data collected by a hardware counter of performance monitoring software in an online stage, so as to improve the precision of hardware counter reuse. The training set is obtained by running selected applications, simultaneously monitoring events in a selected hardware event set by using a hardware counter reuse mode and a hardware event mode of performance monitoring software, recording values of obtained hardware counters, and then performing abnormal data screening, first-order difference processing, logarithmic processing and disorder arrangement processing in sequence. The application utilizes the time series analysis capability of the Transform model to perform data fitting on estimation data collected by the hardware counter, so as to significantly improve the precision of hardware counter reuse and improve the accuracy of performance monitoring (such as VTune and PAPI).

Description

Technical Field

[0001] This invention relates to a technology in the field of neural network applications, specifically a hardware counter multiplexing estimation implementation method. Background Technology

[0002] To increase the number of hardware events captured in a single data acquisition, commonly used performance analysis tools such as PAPI and IntelVTune offer hardware counter multiplexing functionality. Hardware counter multiplexing involves two main steps: time-division multiplexing data acquisition and accuracy restoration through estimation algorithms. First, hardware counter multiplexing reads the hardware event data acquired by the hardware counter at fixed time intervals. Second, it uses linear interpolation to supplement any hardware events not acquired within a time slice. Through multiplexing, performance monitoring software can cover a larger number of hardware events with a smaller hardware counter, enabling researchers to perform data modeling of processor performance and quantitative analysis. Therefore, the hardware counter multiplexing function in processor performance monitoring software is of significant value for the quantitative study of processor performance.

[0003] However, current methods for hardware counter reuse, such as numerical fitting methods like fixed interpolation, linear interpolation, and nonlinear interpolation, suffer from poor versatility and low accuracy. These methods fail to consider the temporal nature of hardware event counting; they merely guess the interpolation method based on probability distributions, while hardware events do not exhibit specific stochastic processes. Therefore, probability-based interpolation methods yield poor results in certain applications and with specific hardware events, failing to meet the requirements of performance modeling. Summary of the Invention

[0004] To address the aforementioned shortcomings of existing technologies, this invention proposes a hardware counter reuse estimation method. By utilizing the timing analysis capabilities of the Transformer model, data fitting is performed on the estimated data collected by the hardware counter, thereby significantly improving the accuracy of hardware counter reuse and enhancing the accuracy of performance monitoring (such as VTune, PAPI).

[0005] This invention is achieved through the following technical solution:

[0006] This invention relates to a method for hardware counter reuse estimation, which constructs a training set and trains a Transformer model in the offline stage, and fits the estimated data collected by the hardware counters of performance monitoring software in the online stage, thereby improving the accuracy of hardware counter reuse.

[0007] The training set is constructed in the following way:

[0008] Step 1: Select the computing mode from Roidinia as a common application in high-performance computing and the set of hardware events to be monitored;

[0009] The applications mentioned include, but are not limited to:

[0010] Step 2: Run the application selected in Step 1, and simultaneously use the hardware counter reuse mode and hardware event mode of the performance monitoring software to monitor the events in the hardware event set selected in Step 1, and record the obtained hardware counter values. Specifically: ① In hardware counter reuse mode, organize the set of hardware events to be recorded using a string array format. The main thread runs the application selected in Step 1 and the performance monitoring software. The slave thread parses the string array at fixed time intervals, calls the interface of the performance monitoring software to obtain the count value of the corresponding hardware event, and writes it sequentially into the .txt file of the recording results; ② In the mode where one hardware counter corresponds to one hardware event, the main thread also runs the application selected in Step 1 and the performance monitoring software. The slave thread reads the hardware count of the monitored hardware events at fixed time intervals and writes it into the .txt file of the recording results.

[0011] Step 3: After obtaining the values ​​of multiple hardware counters by repeatedly executing step 2, the training set is obtained after preprocessing.

[0012] The preprocessing includes sequentially performing abnormal data filtering, first-order difference processing, logarithmic processing, and disordered arrangement processing.

[0013] The aforementioned abnormal data filtering refers to: using the maximum sum of hardware counters as Max, discarding 15% of all data collected where the sum of hardware counters is less than Max, in order to remove erroneous data collected due to abnormal overflow of hardware counters, and then discarding the last 4% of the data collection step. Considering that some hardware events have a small number of time slices collected, the data from the last 4 time steps are further discarded in addition to discarding the last 4%.

[0014] The aforementioned random arrangement refers to randomly shuffling the obtained data in a ratio of 7:1:2, and using it as the training set, test set, and validation set in sequence, with no duplicate elements between any two sets.

[0015] The training process involves using data collected in the hardware counter reuse mode as estimated values ​​and data collected in the hardware event mode (with one hardware counter recording one hardware event) as true values, with the loss set to mean squared error. The hardware counter reuse data and the hardware event data (with one hardware counter recording one hardware event) are input into the Transformer model in batches of size 1. Through backpropagation of error, the new weight values ​​in each neuron are calculated, and the loss value after each update is calculated until the loss stabilizes.

[0016] The fitting process involves recording the sum of the hardware counters collected during the online phase and comparing it with the sum of historical hardware counters. If the sum of the collected hardware counters is greater than the sum of historical hardware counters, the historical hardware counters are updated; otherwise, the collected data is deemed to have a problem and the system waits for the next input. The collected hardware counters are then subjected to first-order differencing and input into the trained Transformer model to obtain more accurate hardware counter estimates, thereby achieving more accurate performance monitoring results.

[0017] This invention relates to a system for implementing the above method, comprising: a data collection unit, a data preprocessing unit, a training unit, and a prediction unit, wherein: the data collection unit uses performance monitoring software to collect hardware event counts in a hardware counter reuse mode or a one-hardware-counter-one-hardware-event mode, obtaining the hardware event count value for a single application run; the data preprocessing unit, based on the information from the data collection unit, performs abnormal data removal, tail data cleaning, first-order differencing, and logarithmic transformation to obtain training samples; the training unit uses the training samples to train a model, obtaining different trained models; the prediction unit uses the trained model to fit the data in the data preprocessing unit and recovers the logarithmically transformed data to obtain a more accurate hardware count estimate.

[0018] Technical effect

[0019] The data cleaning method employed in this invention eliminates data with significant biases and removes tail-end data, effectively reducing the inclusion of invalid data in the training set and improving model accuracy. Matching the model to the computational pattern refers to training a model specifically for applications that conform to the computational pattern of high-performance computing applications, and then using this model for inference within those applications. Compared to training with all collected data, using the data cleaned according to this invention results in more stable model convergence, achieving an average improvement in model accuracy of over 10%. Furthermore, compared to existing performance monitoring software that outputs data without any discrimination, this method can detect errors in the performance monitoring software—specifically, hardware counter overflows leading to incorrect performance monitoring results. Training the model according to the computational pattern of this invention achieves higher accuracy than training a general-purpose model and is more efficient than training a model for each application. Attached Figure Description

[0020] Figure 1 This is a flowchart of the present invention;

[0021] Figure 2 This is a schematic diagram of the Transformer model structure for an example.

[0022] Figure 3 A diagram illustrating data cleaning strategies;

[0023] Figure 4 This is a schematic diagram of a hardware counter multiplexing estimation strategy based on nonlinear interpolation.

[0024] Figure 5 This is a schematic diagram illustrating the effect of an example. Detailed Implementation

[0025] like Figure 1 As shown, this embodiment relates to a hardware counter reuse estimation method, including: offline model training, running and collecting data from the application to be analyzed, and online data processing. Specifically, it includes: running typical high-performance computing applications in Rodinia in the offline stage and collecting hardware event counts using a hardware counter reuse mode and a hardware counter recording a hardware event mode; then using the preprocessed data to train the model as follows: Figure 2 The Transformer model shown ( GitHub mlpotter / Transformer Time Series : Enhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series Forecasting (NeurIPS) 2019))。

[0026] like Figure 3 As shown, the preprocessing method is as follows: Using the maximum sum of hardware counters as Max, 15% of all data collected with sums less than Max are discarded to remove erroneous data collected due to abnormal overflow of the hardware counters. The last 4% of the data collection step size is then discarded. Considering that some hardware events have a small number of time slices collected, the data from the last four time steps are further discarded in addition to discarding the last 4%. After obtaining several Transformer models covering typical high-performance computing modes, they are stored in a fixed order under a specified path, and each computing mode is assigned a Transformer model number. During the online phase, the hardware counter reuse mode is used to collect data from the application whose performance is to be monitored. Then, the computing mode of the application is input, and the system parses the number corresponding to the input computing mode, selects the specified Transformer model under the stored model path, and performs post-processing on the data to obtain more accurate hardware counter values.

[0027] and Figure 4Compared to the nonlinear interpolation hardware counter reuse post-processing strategy shown, this embodiment uses the Transformer model with stronger fitting ability. This model has the best timing analysis capability, can consider the context of program execution, and matches the actual growth law of hardware counter values, thereby improving the accuracy of the hardware counter reuse estimation algorithm.

[0028] Based on this embodiment, by running common high-performance computing applications in Rodinia and comparing the hardware event count values ​​obtained from the "hardware counter multiplexing acquisition + post-processing" mode and the hardware counter recording one hardware event mode, the correctness of the invention was verified. Furthermore, in terms of accuracy, this embodiment is superior to... Figure 4 The nonlinear interpolation method shown has improved performance to varying degrees in different high-performance computing applications.

[0029] Based on practical experience, using PAPI as the performance monitoring software and a server equipped with two Intel Xeon Gold 6248 processors, 192GB of main memory, and CentOS 7.6 operating system, and after pre-collecting hardware counts of KMeans and KNN and training a model with a dense linear algebra computation mode, and then fitting this model to an LU decomposition application with the same dense linear algebra computation mode, the experimental data obtained by comparing the average relative accuracy obtained from collecting six hardware events in MPX mode are as follows: The average relative accuracy of the model trained directly without data cleaning is 73.45%, while the average relative accuracy of the model trained after cleaning the data is 86.73%. However, the average relative accuracy of the model trained using data collected from SRAD (an application with a structured grid computation mode), which is not dense linear algebra, is only 58.39% for LU decomposition applications.

[0030] Compared to existing technologies, this invention uses data cleaning techniques to remove abnormal data, resulting in a relative improvement in model prediction accuracy of over 10%. This technique improves model accuracy for two reasons: First, the execution time of different program runs varies, and removing tail data avoids mismatches in the number of data acquisition time slices caused by different program execution completion times. Second, low hardware counter values ​​and acquisitions indicate hardware counter overflow, manifested as a decrease rather than monotonically increasing hardware counter values. This can lead to negative values ​​after first-order differencing, which is clearly unreasonable for hardware counter values. Removing this abnormal data ensures the use of appropriate data to train the model. Furthermore, training a model using data with the same computational pattern achieves nearly 30% accuracy improvement compared to training a model using a generalized computational pattern or data with mismatched computational patterns. This is because hardware counter performance varies significantly across different computational patterns, but hardware counter performance within the same computational pattern is highly correlated. Targeted selection of models with the same computational pattern balances efficiency and model accuracy. In summary, this method improves accuracy by 31.27% compared to the hardware counter reuse estimation algorithm based on linear interpolation. This is because the reuse estimation method based on linear interpolation makes relatively simple assumptions and does not take into account the timing relationships between hardware counter values.

[0031] The above-described specific implementations can be partially adjusted by those skilled in the art in different ways without departing from the principles and purpose of the present invention. The scope of protection of the present invention is defined by the claims and is not limited to the above-described specific implementations. All implementation schemes within the scope of the claims are bound by the present invention.

Claims

1. A hardware counter multiplexing estimation implementation method, characterized in that, In the offline phase, a training set is built and the Transformer model is trained. In the online phase, the estimated data collected by the hardware counters of the performance monitoring software is fitted to improve the accuracy of hardware counter reuse. The training set is constructed in the following way: Step 1: Select the computing mode from Roidinia as a common application in high-performance computing and the set of hardware events to be monitored; Step 2: Run the application selected in Step 1, and simultaneously use the hardware counter reuse mode and hardware event mode of the performance monitoring software to monitor the events in the hardware event set selected in Step 1, and record the obtained hardware counter values. Specifically, this includes: ① In hardware counter reuse mode, organizing the hardware event set to be recorded using a string array format, with the main thread running the application selected in Step 1 and the performance monitoring software, and the slave thread parsing the string array at fixed time intervals, calling the interface of the performance monitoring software to obtain the count value of the corresponding hardware event, and writing it sequentially into the .txt file of the recording results; ② In the mode where one hardware counter corresponds to one hardware event, the main thread also runs the application selected in Step 1 and the performance monitoring software, with the slave thread reading the hardware count of the monitored hardware events at fixed time intervals and writing it into the .txt file of the recording results. Step 3: After obtaining the values ​​of multiple hardware counters by repeatedly executing Step 2, the training set is obtained after preprocessing; The training process involves using data collected in the hardware counter reuse mode as estimated values ​​and data collected in the hardware event mode (with one hardware counter recording one hardware event) as true values, with the loss set to mean squared error. The hardware counter reuse data and the hardware event data (with one hardware counter recording one hardware event) are input into the Transformer model in batches of 1. Through backpropagation of error, the new weight values ​​in each neuron are calculated, and the loss value is calculated after each update until the loss stabilizes. The fitting process involves recording the sum of the hardware counters collected during the online phase and comparing it with the sum of historical hardware counters. If the sum of the collected hardware counters is greater than the sum of historical hardware counters, the historical hardware counters are updated; otherwise, the collected data is deemed to have a problem and the system waits for the next input. The collected hardware counters are then subjected to first-order differencing and input into the trained Transformer model to obtain more accurate hardware counter estimates, thereby achieving more accurate performance monitoring results.

2. The hardware counter multiplexing estimation implementation method according to claim 1, characterized in that, The preprocessing includes sequentially performing abnormal data filtering, first-order difference processing, logarithmic processing, and disordered arrangement processing.

3. The hardware counter multiplexing estimation implementation method according to claim 2, characterized in that, The aforementioned abnormal data filtering refers to: using the maximum sum of hardware counters as Max, discarding 15% of all data collected with sums less than Max to clear erroneous data collected due to abnormal overflow of hardware counters, then discarding the last 4% of the data collection step size, and further discarding the data from the last 4 time steps.

4. The hardware counter multiplexing estimation implementation method according to claim 2, characterized in that, The aforementioned random arrangement refers to randomly shuffling the obtained data in a ratio of 7:1:2, and using it as the training set, test set, and validation set in sequence, with no duplicate elements between any two sets.

5. A system for implementing the hardware counter multiplexing estimation method according to any one of claims 1-4, characterized in that, include: The system comprises a data collection unit, a data preprocessing unit, a training unit, and a prediction unit. Specifically: the data collection unit uses performance monitoring software to collect hardware event counts in either a hardware counter reuse mode or a one-hardware-count-one-hardware-event mode, obtaining the hardware event count value for a single application run; the data preprocessing unit, based on the information from the data collection unit, performs outlier removal, tail data cleaning, first-order differencing, and logarithmic transformation to obtain training samples; the training unit uses the training samples to train the model, obtaining different trained models; and the prediction unit uses the trained model to fit the data from the data preprocessing unit and recovers the logarithmically transformed data to obtain a more accurate hardware count estimate.

Images