Synthetic Workload Generation for ML Training Accuracy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current machine learning models for computer system design exploration face limited prediction accuracy due to insufficient coverage of program state-space by traditional training sets, which are often based on standard benchmarks or limited micro-benchmarks, failing to represent the diverse range of modern applications.
Innovation Solution
A workload generation framework that systematically creates synthetic applications with controlled core metrics, allowing users to input fixed or random values to generate training programs covering a broader range of program behavior, thereby improving the representativeness and accuracy of machine learning models.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If standard benchmarking suites are used for training machine learning models, then the training process is simplified and execution time is reduced, but the coverage of program state-space is limited and prediction accuracy deteriorates
Solution Approach 1:
The patent creates synthetic copies of real-world application behaviors through programmatically generated workloads that replicate the essential characteristics of diverse application classes. Instead of using actual benchmark applications, the system generates synthetic traces that copy the behavioral patterns, memory access patterns, and execution characteristics of real applications, thereby expanding training data coverage without the time cost of executing numerous real benchmarks.
Solution Approach 2:
The system systematically varies workload parameters such as instruction mix, loop characteristics, memory access patterns, and control flow patterns to generate diverse synthetic applications. By changing these parameters across multiple dimensions, the training set achieves broader program state-space coverage, improving prediction accuracy while maintaining efficient synthetic generation rather than expensive real application execution.
2Measurement precision
If the number of training applications is increased to improve coverage, then prediction accuracy improves, but the computational cost and training time increase significantly
Solution Approach 1:
The patent generates large numbers of synthetic training applications by copying and varying fundamental workload patterns rather than executing numerous real applications. This synthetic copying approach enables creation of thousands of diverse training samples with minimal computational overhead, achieving both broad coverage and high prediction accuracy without the linear time cost of processing equivalent numbers of real benchmarks.
Solution Approach 2:
The system performs preliminary analysis of real application characteristics to extract key behavioral patterns, then uses these insights to pre-configure synthetic workload generators. This preliminary characterization allows the subsequent generation of large-scale diverse training data to proceed efficiently without repeating the expensive analysis phase, thereby achieving both comprehensive coverage and training efficiency.
3Measurement precision
If detailed simulation-based techniques are used for accurate performance analysis, then measurement precision improves, but the computational cost and execution time increase
Solution Approach 1:
The patent creates simplified synthetic copies of detailed application behaviors that capture essential performance characteristics without requiring full detailed simulation execution. The synthetic workloads replicate key performance-relevant patterns such as memory access sequences, instruction mix, and control flow, enabling accurate performance model training at a fraction of the computational cost of running detailed simulators on real applications.
Data Source
AI summary
A method, system and computer program product for generating sets of training programs for machine learning models. Fixed values of one or more workload metrics are received from a user, where the workload metrics correspond to low-level program features which define particular low-level application behavior. A profile using the fixed values of the workload metrics is then created. A suite of synthetic applications is generated using the created profile to form a set of training programs which target particular aspects of program behavior. A machine learning model is then trained using the set of training programs. Since the generated synthetic applications provide a broader coverage of the program state-space, the formed set of training programs more accurately targets performance behavior thereby improving the prediction accuracy of the machine learning based predictive models.