Multi-Core Simulator Threading for Communication-Heavy Functional Units
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The execution speed of software simulators for multi-core processors, particularly deep learning accelerators, is a critical bottleneck that hinders software development productivity.
Innovation Solution
Implementing a host multi-core simulator using multiple operating system (OS) threads to simulate layers of a target multi-core processor with a hierarchical architecture, assigning user-level threads to functional units that frequently communicate, and skipping synchronization for communication conflicts.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If multiple OS threads are used to simulate multiple cores, then simulation speed is improved, but thread management complexity and synchronization overhead increase
Solution Approach 1:
The simulation is segmented into multiple independent layers (functional units, corelets, cores) that can be simulated on separate OS threads. Each layer is assigned to a specific thread based on the formula (1.0+t/c*s)/min(c, t), where c is the number of cores and t is the number of threads. This segmentation allows parallel execution while maintaining manageable thread complexity through structured layer assignment.
Solution Approach 2:
The patent introduces a hierarchical dimension to thread management by organizing simulations across multiple layers (functional units → corelets → cores). Instead of flat thread assignment, the hierarchical structure allows threads to operate at different levels of the hierarchy simultaneously, reducing synchronization overhead by confining communication patterns within specific hierarchical boundaries.
2Productivity
If user-level threads are assigned to functional units that frequently communicate, then communication efficiency is improved, but synchronization requirements increase
Solution Approach 1:
Multiple user-level threads that simulate functional units with frequent communication patterns are merged onto the same OS thread. This merging reduces the number of inter-thread synchronization operations required, as communications within the same OS thread can proceed without complex synchronization protocols. The patent identifies communication frequency thresholds to determine when merging is beneficial.
Solution Approach 2:
The patent converts the potential harm of frequent communication (which would require extensive synchronization) into a benefit by using communication frequency as a criterion for thread merging. Functional units that communicate frequently are deliberately assigned to the same user-level thread, transforming the synchronization overhead problem into an opportunity for optimized thread consolidation and reduced synchronization barriers.
3Productivity
If synchronization is skipped for communication conflicts on the same OS thread, then execution speed is improved, but simulation accuracy may be compromised
Solution Approach 1:
The patent applies different synchronization qualities to different communication scenarios. Within the same OS thread, synchronization is relaxed (skipped for communication conflicts) to improve execution speed. Between different OS threads, full synchronization protocols are maintained to ensure simulation accuracy. This local differentiation of synchronization quality allows the system to optimize performance where safe while maintaining reliability where necessary.
Data Source
AI summary
A method improves an execution speed of a host multi-core simulator simulating a target multi-core processor that has a hierarchical architecture including multiple corelets per core that, in turn include multiple functional units. The host multi-core simulator is implemented using multiple OS threads. The method selects layers in the hierarchical architecture to simulate on one of the OS threads, based on a shortest estimated layer execution time determined by (1.0+t/c*s)/min(c, t), wherein c is a number of cores in the simulator, t is a number of OS threads, and s is a threading overhead coefficient. The method respectively executes, from among the selected layers, a parallel simulation of the units therein that frequently communicate with each other on one of the multiple OS threads based on a communication frequency threshold, by assigning and using a respective user-level thread for each of the units from among a plurality of user-level threads.