Microprocessor performance prediction based on sampling and caching

Abstract

A method comprising executing a plurality of threads using a plurality of processor cores of a processor, wherein each thread in the plurality of threads comprises a plurality of instructions; sampling, at every N cycle, to obtain a program counter (PC) corresponding to one instruction in the plurality of instructions; calculating an instructions per cycle (IPC) corresponding to the PC; caching the PC and the IPC as a cache entry in a cache; searching, at every J new instructions, the cache for a matching PC cache entry corresponding to a new instruction to be executed by the plurality of processor cores; updating an IPC value in a register with a corresponding IPC in the matching PC cache entry when the matching PC cache entry found in the cache; and switching a thread in the plurality of threads to a new thread based on the IPC value in the register.

Description

Microprocessor Performance Prediction Based on Sampling and CachingTECHNICAL FIELD

[0001] This disclosure is related to microprocessor performance prediction based on sampling and caching.BACKGROUND

[0002] Program, process, or thread level switching is a crucial aspect of multitasking and resource management of system by ensuring efficient utilization of a processor (e g., a central processing unit (CPU)). A software program is a collection of instructions. When a program is executed, the operating system creates a process, which has its own memory space, system resources (like file handles), and at least one thread of execution. A thread is the smallest unit of instructions within a process that can be executed by a processing unit. Multiple threads within the same process can run independently and concurrently, making tasks more efficient. Thread switching in a CPU refers to the process of shifting execution from one thread to another within a processor or a core of a multi-core processor Modern CPUs often support multithreading, allowing them to manage multiple threads of execution either simultaneously or in rapid succession. When a thread switch occurs, the CPU saves the current thread’s state — such as register values and program counter — into memory and loads the state of the next thread to be executed. This enables the system to maintain the illusion of parallelism, even on single-core processors, by quickly alternating between threads.SUMMARY

[0003] A first aspect relates to a microprocessor performance prediction process method implemented by a processor. The method includes executing a plurality of threads using a plurality of processor cores of a processor, wherein each thread in the plurality of threads comprises a plurality of instructions; sampling, at every N cycle, to obtain a program counter (PC)corresponding to one instruction in the plurality of instructions, wherein N is a first positive integer; calculating an instructions per cycle (IPC) corresponding to the PC; caching the PC and the IPC as a cache entry in a cache; searching, at every J new instructions, the cache for a matching PC cache entry corresponding to a new instruction to be executed by the plurality of processor cores, wherein J is a second positive integer; updating an IPC value in a register with a corresponding IPC in the matching PC cache entry when the matching PC cache entry found in the cache; and switching a thread in the plurality of threads to a new thread based on the IPC value in the register.

[0004] Optionally, in a first implementation according to the first aspect, the sampling comprises: removing a last (PC, Instruction Count (IC)) pair from a second end of a sampling queue, wherein the last (PC, IC) pair comprises a first PC (PCM) and a first IC (ICM), and wherein the sampling queue comprises M entries, and wherein M is a third positive integer; shifting all remaining (PC, IC) pairs in the sampling queue by one entry space towards the second end of the sampling queue to create an empty entry space at a first end of the sample queue; inserting a new (PC, IC) pair into the sampling queue in the empty entry space at the first end of the sampling queue, wherein the new (PC, IC) pair comprises a second PC (PCi) and a second IC (ICi); and setting the first PC as the PC.

[0005] Optionally, in a second implementation according to the first aspect or any implementation thereof, determining the IPC corresponding to the PC comprises: calculating a number of instructions that has been processed in between the last (PC, IC) pair and the new (PC, IC) pair by subtracting the first IC (ICM) from the second IC (ICi); and dividing the number of instructions by (N*M) to obtain the IPC corresponding to the PC.

[0006] Optionally, in a third implementation according to the first aspect or any implementation thereof, caching the IP and the IPC in the cache comprises: determining whether the cache comprises an existing cache entry for the PC, wherein the cache stores K cache entries, wherein each cache entry in the K cache entries comprises a unique PC and a corresponding IPC value, and wherein K is a fourth positive integer; and updating the existing cache entry with the IPC when the cache comprises the existing cache entry for the PC.

[0007] Optionally, in a fourth implementation according to the first aspect or any implementation thereof, K is less than a number of unique instructions in the plurality of instructions.

[0008] Optionally, in a fifth implementation according to the first aspect or any implementation thereof, caching the IP and the IPC in the cache further comprises: determining, when the cache does not comprise the existing cache entry for the PC, whether the cache is foil; inserting, when the cache is not foil, a new cache entry comprising the PC and the IPC into the cache; and selecting, when the cache is foil, a replacement cache entry in the cache and replacing the replacement cache entry with the cache entry for the PC and the IPC.

[0009] Optionally, in a sixth implementation according to the first aspect or any implementation thereof, the replacement cache entry has a lowest least recently used (LRU) count in the cache.

[0010] A second aspect relates to a processor comprising a plurality of processor cores, one or more memory or storage means configured to store instructions and a cache; and one or more circuits or processing means coupled to the memory or the storage means and configured to execute the instructions to cause the processor to: execute a plurality of threads using a plurality of processor cores of a processor, wherein each thread in the plurality of threads comprises a plurality of instructions; sample, at every N cycle, to obtain a program counter (PC) corresponding to one instruction in the plurality of instructions, wherein N is a first positive integer; calculate an instructions per cycle (IPC) corresponding to the PC; cache the PC and the IPC as a cache entry in the cache; search, at every J new instructions, the cache for a matching PC cache entry corresponding to a new instruction at every J new instructions to be executed by the plurality of processor cores, wherein J is a second positive integer; update an IPC value in a register with a corresponding IPC in the matching PC cache entry when the matching PC cache entry found in the cache; determine, at a thread switching time, a thread in the plurality of threads to switch with a new thread based on the IPC in the IPC register; and switch a thread in the plurality of threads to a new thread based on the IPC value in the register.

[0011] Optionally, in a first implementation according to the second aspect, the plurality of processor cores comprises heterogenous cores comprising different processing capability cores, and wherein a higher processing capability core is less energy-efficient than a lower processing capability core.

[0012] Optionally, in a second implementation according to the second aspect or any implementation thereof, the instructions, when executed by one or more circuits, processors, processor cores, or processing means, further cause the processor to: remove a last (PC, Instruction Count (IC)) pair from a second end of a sampling queue, wherein the last (PC, IC) pair comprises a first PC (PCM) and a first IC (ICM), and wherein the sampling queue comprises M entries, and wherein M is a third positive integer; shift all remaining (PC, IC) pairs in the sampling queue by one entry space towards the second end of the sampling queue to create an empty entry space at a first end of the sample queue; insert a new (PC, IC) pair into the sampling queue in the empty entry space at the first end of the sampling queue, wherein the new (PC, IC) pair comprises a second PC (PCi) and a second IC (ICi); and set the first PC as the PC.

[0013] Optionally, in a third implementation according to the second aspect or any implementation thereof, the instructions, when executed by one or more circuits, processors, processor cores, or processing means, further cause the processor to: calculate a number of instructions that has been processed in between the last (PC, IC) pair and the new (PC, IC) pair by subtracting the first IC (ICM) from the second IC (ICi); and divide the number of instructions by (N*M) to obtain the IPC corresponding to the PC.

[0014] Optionally, in a fourth implementation according to the second aspect or any implementation thereof, the instructions, when executed by one or more circuits, processors, processor cores, or processing means, further cause the processor to: determine whether the cache comprises an existing cache entry for the PC, wherein the cache stores K cache entries, wherein each cache entry in the K cache entries comprises a unique PC and a corresponding IPC value, and wherein K is a fourth positive integer; and update the existing cache entry with the IPC when the cache comprises the existing cache entry for the PC.

[0015] Optionally, in a fifth implementation according to the second aspect or any implementation thereof, K is less than a number of unique instructions in the plurality of instructions.

[0016] Optionally, in a sixth implementation according to the second aspect or any implementation thereof, the instructions, when executed by one or more circuits, processors, processor cores, or processing means, further cause the processor to: determine, when the cache does not comprise the existing cache entry for the PC, whether the cache is full; insert, when the cache is not full, a new cache entry comprising the PC and the IPC into the cache; and select, when the cache is full, a replacement cache entry in the cache and replacing the replacement cache entry with the cache entry for the PC and the IPC.

[0017] Optionally, in a seventh implementation according to the second aspect or any implementation thereof, the replacement cache entry has a lowest least recently used (LRU) count in the cache.

[0018] A third aspect relates to an apparatus comprising: one or more memory or storage means configured to store instructions and a cache; and one or more processors or processing means coupled to the memory or the storage means and configured to execute the instructions to cause the apparatus to: execute a plurality of threads using a plurality of processor cores of a processor, wherein each thread in the plurality of threads comprises a plurality of instructions; sample, at every N cycle, to obtain a program counter (PC) corresponding to one instruction in the plurality of instructions, wherein N is a first positive integer; calculate an instructions per cycle (IPC) corresponding to the PC; cache the PC and the IPC as a cache entry in the cache; search, at every J new instructions, the cache for a matching PC cache entry corresponding to a new instruction at every J new instructions to be executed by the plurality of processor cores, wherein J is a second positive integer; update an IPC value in a register with a corresponding IPC in the matching PC cache entry when the matching PC cache entry found in the cache; determine, at a thread switching time, a thread in the plurality of threads to switch with a new thread based on theIPC in the IPC register; and switch a thread in the plurality of threads to a new thread based on the IPC value in the register.

[0019] A fourth aspect relates to a computer program product comprising computer-executable instructions stored on a non-transitory computer-readable storage medium, the computerexecutable instructions when executed by one or more processors of an apparatus, cause the apparatus to perform a method according to any of the preceding aspects or any implementation thereof.

[0020] For clarity, any one of the foregoing embodiments may be combined with any one or more of the other foregoing embodiments to create a new embodiment within the scope of the present disclosure.

[0021] These and other features, and the advantages thereof, will be more clearly understood from the following detailed description taken in conjunction with the accompanying drawings and claims.BRIEF DESCRIPTION OF DRAWINGS

[0022] For a more complete understanding of this disclosure, reference is now made to the following brief description, taken in connection with the accompanying drawings and detailed description, wherein like reference numerals represent like parts.

[0023] FIG. 1 is a diagram illustrating a multi-core processor comprising an IPC predictor according to an embodiment of the present disclosure.

[0024] FIG. 2 is a diagram illustrating a sampling architecture according to an embodiment of the present disclosure.

[0025] FIG. 3 is a diagram illustrating an IPC cache according to an embodiment of the present disclosure.

[0026] FIG. 4 is a flowchart illustrating a sampling and caching process according to an embodiment of the present disclosure.

[0027] FIG. 5 is a flowchart illustrating a cache lookup process according to an embodiment of the present disclosure.

[0028] FIG. 6 is a flowchart illustrating a thread switching process according to an embodiment of the present disclosure.

[0029] FIG. 7 is a flowchart illustrating a microprocessor performance prediction process according to an embodiment of the present disclosure.

[0030] FIG. 8 is a diagram illustrating an apparatus according to an embodiment of the present disclosure.DESCRIPTION OF EMBODIMENTS

[0031] It should be understood at the outset that although illustrative implementations of one or more embodiments are provided below, the disclosed systems and / or methods may be implemented using any number of techniques, whether currently known or in existence. The disclosure should in no way be limited to the illustrative implementations, drawings, and techniques illustrated below, including the exemplary designs and implementations illustrated and described herein, but may be modified within the scope of the appended claims along with their full scope of equivalents.

[0032] The present disclosure describes various embodiments of a microprocessor performance prediction method and apparatus that are based on continuous sampling and caching of program counters (PCs) corresponding to specific instructions and their IPC values during program execution, which are then used, at a thread switching time, to determine which processing thread to replace / switch. Embodiments of the present disclosure improve upon existing thread switching methods by providing direct IPC. Direct IPC indicates the number of instructions processed per processor cycle and is used as a standard performance index instead of relative performance prediction. Some or all embodiments of the present disclosure can be implemented using existing hardware counters through a performance monitoring unit (PMU) in most, if not all, commercial processors. In contrast to machine learning based mechanisms that rely on off-linetraining to correlate past behavioral patterns to predict relative performances of a thread on heterogeneous cores, embodiments of the present disclosure capture the performance profile of presently running threads obviating pre-trainings and pre-profiling. Further, one or more designs of the various embodiments of the present disclosure may lead to less hardware complexity and less overhead in hardware delay and hardware area.

[0033] FIG. 1 is a diagram illustrating a multi-core processor 100 comprising an IPC predictor according to an embodiment of the present disclosure. In the depicted example, the multi-core processor 100 comprises ten processing cores or execution units (labeled CPU core 0 - CPU core 9). It should also be noted that the present disclosure is applicable to a multi-core processor with any number of cores (e.g., 2-16, 2-32, etc.) or a system with multiple single core processors. Each of the processing cores is configured to execute one or more threads in parallel with the other processing cores of the multi-core processor 100. In some embodiments, CPU core 0 - CPU core 9 are heterogenous cores. For example, some of the cores may be high performance cores that are designed for demanding tasks and that consume more power, while other cores may be lower performance cores that are optimized for less demanding tasks and that are more energy-efficient.

[0034] The performance of a processor or processing core is often measured by how many instructions the processor or processing core can execute in a single clock cycle (i.e., IPC). Instructions may perform various tasks including performing arithmetic operations or data reads / writes from / to memory. For instance, a non-limiting example of an arithmetic instruction is ADD Rl, R2, R3, which adds the values in registers R1 and R2 and stores the result in register R3. Whereas, LOAD / STORE instructions may read / write a value from / to a specified memory address. A clock cycle is a single tick of the processor’s clock, which controls how fast the processor executes instructions since each cycle represents one opportunity for the processor or processing core to do work (e.g., moving data, performing calculations, or accessing memory). Clock speed is usually measured in gigahertz (GHz) (e.g., 1 GHz = 1 billion cycles per second). Modern processors include a hardware component known as a PMU that tracks the number of instructions executed by the processor or by each processing core. For instance, assume it takes1 billion clock cycles to execute 3 billion instructions, then IPC = 3 billion instructions / 1 billion clock cycles, which equals 3.0 instructions per cycle.

[0035] As described above, to make multiple processes of various programs appear to run simultaneously when there are more processes running than there are processing units, the threads of the various processes are constantly switched out in the multi-core processor 100. In an embodiment, threads to be executed by the multi-core processor 100 are first placed in a thread queue 110. For instance, thread queue 110 currently holds thread A (THA) through thread Y (THY), with THA being the next thread to be executed by the multi-core processor 100. One common method for thread replacement is round-robin switching. In round-robin switching, each process is assigned a fixed time slice to execute. Once a process uses up its time slice, the processor switches to the next thread in the thread queue 110, even if the current process is not finished. The process that is switched out is placed in the back of the thread queue 110.

[0036] Another approach to thread switching is dynamically switching threads based on their execution characteristics in a CPU system composed of multiple heterogeneous cores. In such an approach, a thread in a compute-intensive phase is switched to a high-performance core, while another thread in a memory-intensive phase is switched to a low-performance, energy-efficient core. Still, another approach is to replace, at predetermined intervals, the thread that is performing the worse according to an IPC corresponding to the thread, with the next thread to be executed THA. For example, if IPC of the thread in CPU core 0 is less than that of CPU core 1, then replace the thread in CPU core 0 with THA.

[0037] However, current approaches that rely on IPC may not be ideal because the IPC is based on past performance (i.e., IPC is measured up to the switch time based on previously executed instructions by the processor / processor core). For instance, when instructions in the new thread differ from the instructions in which the IPC was determined, switching based on a previously determined IPC may not provide optimal performance.

[0038] Accordingly, the present disclosure describes various embodiments of a microprocessor performance prediction method and apparatus that are based on continuoussampling and caching of program counters (PCs) and corresponding IPC values during program execution to determine a predicted future IPC that more likely correlates to the processing performance associated with instructions of to be executed threads in the thread queue 110. A PC is an identifier (e.g., a memory address or a unique line number in a program) that identifies one instruction in the program. Any instruction in the program can be located using the PC of the instruction. Instructions in program are not always executed sequentially. For instance, a branch instruction may alter the normal sequential flow of execution by making the program jump to a different instruction address (e.g., if a certain condition is met). Additionally, some instructions may be repeated numerous times, such as during a loop function.

[0039] In an embodiment, as shown in FIG. 1, each of the processing cores (CPU core 0 - CPU core 9) in the multi-core processor 100 includes an IPC predictor 120 or IPC predictor module / means configured to predict a future IPC as described in the present disclosure. Alternatively, in some embodiments, the multi-core processor 100 may include just a single IPC predictor or IPC predictor module / means that is able to sample and predict a future IPC for each of the processing cores of the multi-core processor 100.

[0040] FIG. 2 is a diagram illustrating a sampling architecture 200 according to an embodiment of the present disclosure. The sampling architecture 200 may be implemented as part of the IPC predictor 120 in FIG. 1. In the depicted embodiment, the sampling architecture 200 includes a sampling queue 202, an enqueue unit 204, a dequeue unit 206, a PMU 208, a retire / commit unit 210, an IPC calculator unit 212, an IPC cache 214, and an IPC register 216. The sampling queue 202 is a data structure such as an array that is used to store a plurality of (PC, IC) pairs. In an embodiment, the sampling queue 202 is configured to store up to M entries, where M is a positive integer. For example, in an embodiment, M may be 16. The size of M is not limited in the present disclosure.

[0041] As stated above, the PMU 208 is a special hardware module / component integrated in most modern microprocessors. The PMU 208 is configured to perform performance monitoring including maintaining an IC indicating a number of instructions executed by the microprocessoror a microprocessor core. In some embodiments, the PMU 208 may be a basic instruction counter unit configured to maintain an IC. In an embodiment, the PMU 208 is configured to send in an N-cycle signal every N cycles to the enqueue unit 204 and the dequeue unit 206 to indicate that bicycles have occurred, where N can be any positive integer (e.g., 5, 10, 12, etc.). The value of N is not limited in the present disclosure. As stated above, a cycle or clock cycle is a single tick of the processor’s clock, and a processor performance can be based on the number of instructions that are executed per cycle (i.e., IPC). In addition, in an embodiment, the PMU 208 is configured to send an IC to the enqueue unit 204 and the dequeue unit 206 every N cycles. The IC indicates the number of instructions that had been executed by the microprocessor or a microprocessor core.

[0042] In an embodiment, in response to receiving the N-cycle signal and IC from the PMU 208, the enqueue unit 204 configured to obtain a PC of last instruction from the retire / commit unit 210. The retire / commit unit is configured to commit / write the results of the last executed instructions in the correct order (as instructions may be executed out of order) in memory. As stated above, the PC is an identifier that uniquely identifies the particular instruction in the program. In an embodiment, the enqueue unit 204 is configured to create a (PC, IC)i pair based on the received PC and IC. The enqueue unit 204 inserts (i.e., enqueue) the (PC, IC)i pair into the sampling queue 202 at a first / insert end of the sampling queue 202.

[0043] At the same time, or prior to enqueuing, assuming the sampling queue 202 is full, the dequeue unit 206, in response to receiving the N-cycle signal from the PMU 208, removes (i.e., dequeue) the last / M (PC, IC)M pair from a second / remove end of the sampling queue 202. The dequeue (PC, IC)M pair is passed to the IPC calculator unit 212. In an embodiment, the dequeue unit 206 is configured to forward the ICPMU received from the PMU 208 to the IPC calculator unit 212.

[0044] In an embodiment, the IPC calculator unit 212 is configured to calculate an IPC (a.k.a., an IPC value) for the PC in the dequeued (PC, IC)M pair based on the size of the sampling queue 202 (referred to as IPCSAMPLE). IPCSAMPLE is calculated to determine an IPC value between the time (PC, IC)M was first inserted / enqueued into the sampling queue 202 to the time (PC, IC)M isremoved / dequeue from the sampling queue 202. In an embodiment, IPCSAMPLE is calculated by first subtracting the ICM in the dequeue (PC, IC)M pair from the ICPMU received from the PMU 208 (ICPMU- ICM) to determine the number of instructions that were executed during the time the (PC, IC)M was in the sampling queue 202. For example, IC in the dequeue (PC, IC) pair may be 10,000 to indicate that 10,000 instructions had been executed when (PC, IC)M was first enqueued into the sampling queue 202, and ICPMU may be 10,245 to indicate that 10,245 instructions had been executed when (PC, IC)M is dequeued. Thus, the number of instructions that were processed between the time (PC, IC) was first inserted / enqueued into the sampling queue 202 to the time (PC, IC)M was removed / dequeued from the sampling queue 202 is 10,245-10,000, which is 245 instructions. Then the determined number of instructions is divided by the number of cycles which is (M*N) because one entry is inserted every N-cycles and there are M entries in the sampling queue 202. Therefore, IPCSAMPLE=(ICPMU- ICM) / (M*N). For instance, in the given example, if N is 5 (i.e., every 5 cycles an entry is enqueued / dequeued) and M is 16 (i.e., the sampling queue 202 can store up to 16 (PC, IC) pairs, then IPCSAMPLE for the given example is 245 / (16*5)= 3.0625.

[0045] In the an embodiment, the PCM from the dequeue (PC, IC)M pair and the IPCSAMPLE are written to or used to update an entry in the IPC cache 214. As will be further described, in some embodiments, the IPC cache 214 can be used to periodically during a lookup process to update an IPC value contained in the IPC register 216, which is then used at a thread switching time by a switching unit 220 to obtain a predicted future IPC for determining which threads switch. The switching unit 220 is shown in dashed lines to indicate that the switching unit 220 may not be part of the sampling architecture 200 (i.e., may be external or existing switching unit). For instance, the IPC value determined and set in the IPC register 216 in accordance with the disclosed embodiments may be used by an existing switching unit 220 configured to switch threads based on the IPC value in the IPC register 216. However, as described in the present disclosure, the IPC value determined and set in the IPC register 216 is a predicted future IPC value. The predicted future IP value corresponds to the instructions or PC recently encountered during thedisclosed continuous sampling process, which more accurately reflects an IPC for future instructions because future instructions tend to produce similar patterns or trends as the recently executed instructions.

[0046] FIG. 3 is a diagram illustrating an IPC cache 300 according to an embodiment of the present disclosure. In an embodiment, the IPC cache 300 is an example of the IPC cache 214 in FIG. 2. In an embodiment, the IPC cache 300 is stored in a memory unit of the multi-core processor 100 or a processing core of the multi-core processor 100 (e.g., CPU core 0 - CPU core 9) in FIG. 1. In the depicted embodiment, the IPC cache 300 is of a fixed size (e.g., stores K entries), where K where is a positive integer. For example, in an embodiment, K may be 30. The size of K is not limited in the present disclosure. In an embodiment, each entry the IPC cache 300 comprises a PC, an IPCSAMPLE, and a least recently used (LRU) count (LRUCOUNT). In an embodiment, the PC is used as a tag / label for searching / querying for matching entries in the IPC cache 300.

[0047] In an embodiment, after IPCSAMPLE is calculated as described in FIG. 2, the IPC cache 300 is queried / search for a cache entry having a PC matching PC from the dequeue (PC, IC) pair. In an embodiment, if a match is found in the IPC cache 300, the IPCSAMPLE corresponding to the matched entry is updated with the latest calculated IPCSAMPLE, and the LRUCOUNT corresponding to the matched entry is increased. The LRUCOUNT is used to indicate how frequently the instruction corresponding to the PC is encounter during the sampling the process. The higher the LRUCOUNT indicates a higher frequency. Alternatively, the LRUCOUNT may contain some timestamp value indicating when the instructions were last used / encountered during the sampling the process. Because the IPC cache 300 is limited in size due to the cache memory of a processor being limited, the LRUCOUNT can be used to determine which PC / instruction and corresponding IPCSAMPLE to keep in the IPC cache 300 when the IPC cache 300 is full and a new entry is to be written to the IPC cache 300. For example, in an embodiment, when the IPC cache 300 is queried / searched for a cache entry having a PC matching PCM from the dequeue (PC, IC) pair, no match is found, and the IPC cache 300 is full (i.e., all K entries are filled), then the entrywith the LRUCOUNT that indicates that the instruction corresponding to the PC in the entry is least frequently used / encountered among all entries in the IPC cache 300 is replaced with a new entry for the PCM and corresponding IPCSAMPLE.

[0048] FIG. 4 is a flowchart illustrating a sampling and caching process 400 according to an embodiment of the present disclosure. In an embodiment, the sampling and caching process 400 correlates to the sampling architecture 200 in FIG. 2 and IPC cache 300 in FIG. 3. The sampling and caching process 400 begins at step 402 by determining whether N cycles of the processor or microprocessor have occurred. For example, as described in FIG. 2, a PMU such as the PMU 208 in FIG. 2 may be configured to send in an N-cycle signal every N cycles to indicate that bicycles have occurred. In an embodiment, for every N cycles, the sampling and caching process 400, at step 404, begins by dequeuing / removing a last (PC, IC) pair from a second end of a sampling queue. The last (PC, IC) pair comprises a first PC (PCM) and a first IC (IC ). AS stated above, M is the size or number of entries that can be stored in the sampling queue. In an embodiment, all remaining (PC, IC) pairs in the sampling queue are shifted by one entry space towards the second end of the sampling queue to create an empty entry space at a first end of the sample queue. The sampling and caching process 400, at step 406, then enqueues / inserts a new (PC, IC) pair into the sampling queue in the empty entry space at the first end of the sampling queue. The new (PC, IC) pair comprises a second PC (PCi) and a second IC (ICi). As described above, in an embodiment, PCi is a PC corresponding to a last completed executed instruction and the ICi is a current instruction count provided by the PMU 208 in FIG. 2. At step 408, the sampling and caching process 400 is configured to calculate IPCSAMPLE as described above in FIG. 2 (e.g., (IPCSAMPLE=(ICI-ICM) / (M*N))). The sampling and caching process 400, at step 410, then searches / queries an IPC cache (e g., IPC cache 300 in FIG. 3) for an entry matching PCDEQUEUED (i.e., PCM). When, at step 412, there is a matching entry / hit, the sampling and caching process 400, at step 414, updates the IPCSAMPLE in the matching entry in the IPC cache with the IPCSAMPLE that was determined at step 408. The sampling and caching process 400 then returns to step 402 and waits for the next N cycle.

[0049] However, when, at step 412, there is no matching entry / hit, the sampling and caching process 400, at step 416, determines an entry in the IPC cache to remove / replace assuming that the IPC cache is full. As described above, in some embodiments, the entry in the IPC cache corresponding to a PC / instruction that has not been used for the longest time is removed / replaced to make space for a new entry corresponding to PCDEQUEUED. At step 418, the sampling and caching process 400 inserts the new entry corresponding to PCDEQUEUED and IPCSAMPLE into the IPC cache. The sampling and caching process 400 returns to step 402 and waits for the next N cycle.

[0050] FIG. 5 is a flowchart illustrating a cache lookup process 500 according to an embodiment of the present disclosure. The cache lookup process 500 may be performed by a lookup unit or an instruction fetch unit (not shown) that is part of the multi-core processor 100 or part of a processing core (e.g., CPU core 0 - CPU core 9) of the multi-core processor 100 in FIG. 1. In an embodiment, the cache lookup process 500 is performed / initiated, at step 502, every J new instructions that is fetched by the instruction fetch unit for execution. J may be any positive integer (e.g., 4, 10, 16, etc ). At step 504, the cache lookup process 500 searches an IPC cache (e.g., IPC cache 300 in FIG. 3) for a matching entry corresponding to the PC of the new instruction. As stated above, a PC is an identifier that uniquely identifies / corresponds to one instruction in a program. However, that one instruction may be executed multiple times. Thus, it is possible that the PC corresponding to the new instructions has previously been cached in the IPC cache if the instruction was encountered during the continuous sampling process described above. In an embodiment, when there is no matching entry / hit corresponding to the PC of the new instruction, the cache lookup process 500 terminates and returns to step 502 for the next J new instructions. When, at step 506, there is a matching entry / hit corresponding to the PC of the new instruction, the cache lookup process 500 reads the IPCSAMPLE of the matching entry and updates a register (e.g., IPC register 216 in FIG. 1) that stores an IPC value with the IPCSAMPLE value at step 508. The IPCSAMPLE value provides a better predicted IPC value for future instructions than prior methodology by basing the IPC value on recently encountered instructions, which more likelyreflects an IPC value for future instructions because the future instructions may include the same or similar instructions as the recent instructions and therefore, are likely to have similar processing performance statistics. Thus, by updating the IPC in the register with IPCSAMPLE value, which is then used to make switching decisions as described in FIG. 6, a processor or processor core can operate more efficiently by making better thread switching decisions.

[0051] FIG. 6 is a flowchart illustrating a thread switching process 600 according to an embodiment of the present disclosure. The thread switching process 600 may be implemented by an existing switching unit of a processor or multi-processor core, (e.g., switching unit 220 in FIG. 2). The thread switching process 600 determines, at step 602, to switch threads (i.e., swap out a currently running thread with a new thread waiting in the thread queue 110 in FIG. 1). As stated above, the thread switching time may be based on various approaches such as round robin. The disclosed embodiments are not limited to any particular switching time approach. At step 604, the thread switching process 600 reads the IPC value from an IPC register that stores an IPC used in switching threads / processes. Again, as described above, the disclosed embodiments use a continuous sampling / caching process and a lookup process to continuously update the IPC value with an IPC (i.e., a predicted IPC value) that more likely correlates to the processing performance corresponding to future instructions to provide improve thread switching decisions and thus, improve overall processor efficiency. At step 606, the thread switching process 600 switches one or more threads based on the IPC read from the from an IPC register. As described above, the disclosed embodiments may be implemented in conjunction with existing hardware (e.g., existing PMUs and switching units) and thus provide a straightforward approach to improving processor efficiency without significant hardware / chip modifications. Additionally, because the disclosed embodiments is implemented in real-time time (i.e., predicts future IPC based on currently executing instructions), there is no need for offline training as used in machine learning method / models that are used to predict future IPC.

[0052] FIG. 7 is a flowchart illustrating a microprocessor performance prediction process 700 according to an embodiment of the present disclosure. The microprocessor performanceprediction process 700 may be implemented by any microprocessor architecture that supports multi-programming, multi-processing or multi-threading, which is typical in modern CPU designs. The microprocessor performance prediction process 700 begins, at step 702, by executing a plurality of threads using a plurality of processor cores of a processor, wherein each thread in the plurality of threads comprises a plurality of instructions. In an embodiment, the plurality of threads corresponds to one or more programs. In an embodiment, each thread in the plurality of threads is executed in one processor core of the plurality of processor cores. At every N cycle , the microprocessor performance prediction process 700, at step 704, samples to obtain a PC, where N is a first positive integer. The microprocessor performance prediction process 700, at step 706, calculates an IPC corresponding to the PC (i.e., the sampled PC). In an embodiment, the sampled PC corresponds to one PC in a plurality of PCs, where each PC in the plurality of PCs uniquely corresponds to one instruction in the plurality of instructions. The microprocessor performance prediction process 700 begins, at step 708, caches the PC and the IPC as a cache entry in a cache. At every I new instructions to be executed by the plurality of processor cores, the microprocessor performance prediction process 700, at step 710, searches the cache for a matching PC cache entry corresponding to a new instruction to be executed by the plurality of processor cores, wherein J is a second positive integer. When the matching PC cache entry found in the cache, the microprocessor performance prediction process 700, at step 710, updates an IPC value in a register with a corresponding IPC in the matching PC cache entry. At a thread switching time, the microprocessor performance prediction process 700, at step 712, switches a thread in the plurality of threads to a new thread based on the IPC value in the register.

[0053] FIG. 8 is a diagram illustrating an apparatus 800 according to an embodiment of the present disclosure. The apparatus 800 can be used to implement embodiments of the present disclosure. For example, the apparatus 800 may be configured to perform the functions of a sending device or a receiving device according to any of the embodiments of the present disclosure. The apparatus 800 includes receiver units (RX) 820 or receiving means for receiving data viaingress ports 810. The apparatus 800 also includes transmitter units (TX) 840 or transmitting means for transmitting via data egress ports 850.

[0054] The apparatus 800 includes a memory 860 or data storing means for storing the instructions and various data. The memory 860 can be any type of, or combination of, memory components capable of storing data and / or instructions. For example, the memory 860 can include volatile and / or non-volatile memory such as read-only memory (ROM), random access memory (RAM), ternary content-addressable memory (TCAM), and / or static random-access memory (SRAM). The memory 860 can also include one or more disks, tape drives, and solid- state drives. In some embodiments, the memory 860 can be used as an over-flow data storage device to store programs when such programs are selected for execution, and to store instructions and data that are read during program execution. In some embodiments, the memory 860 can be memory that is integrated with the processor 830.

[0055] The apparatus 800 includes one or more processors 830 or processing means (e.g., central processing unit (CPU) or specially configured circuitry) to process instructions. The one or more processors 830 may be implemented as one or more microprocessors chips or processor cores (e.g., as a multi-core processor such as the multi-core processor 100 in FIG. 1), field- programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), and digital signal processors (DSPs). The one or more processors 830 are communicatively coupled via a system bus with the ingress ports 810, RX 820, TX 840, egress ports 850, and memory 860. The one or more processors 830 can be configured to execute instructions stored in the memory 860. In an embodiment, the one or more processors 830 may include an IPC predictor module 870 as shown in FIG. 1 and described in FIG. 2. In an embodiment, the IPC predictor module 870 may include memory for storing data such as the IPC cache 300 in FIG. 3. Additionally, in some embodiments, the IPC predictor module 870 is configured to perform one or more processes described in FIG. 4-FIG. 7. Thus, the one or more processors 830 and the IPC predictor module 870 provide a means for performing any computational, comparison, determination, initiation, configuration, or any other action corresponding to the claims when the appropriate instruction isexecuted by the processor 830 or the IPC predictor module 870. Thus, the inclusion of the IPC predictor module 870 substantially improves the functionality of the one or more processors 830 and the apparatus 800.

[0056] While several embodiments have been provided in the present disclosure, a person of ordinary skill in the art would understood that the disclosed systems and methods might be embodied in many other specific forms without departing from the spirit or scope of the present disclosure. The present examples are to be considered as illustrative and not restrictive, and the disclosure is not to be limited to the details given herein. For example, the various elements or components may be combined or integrated in another system or certain features may be omitted, or not implemented.

[0057] In addition, techniques, systems, subsystems, and methods described and illustrated in the various embodiments as discrete or separate may be combined or integrated with other systems, modules, techniques, or methods without departing from the scope of the present disclosure. Other items shown or discussed as coupled or directly coupled or communicating with each other may be indirectly coupled or communicating through some interface, device, or intermediate component whether electrically, mechanically, or otherwise. Other examples of changes, substitutions, and alterations are ascertainable by one skilled in the art and may be made without departing from the spirit and scope disclosed herein.

Claims

CLAIMSWhat is claimed is:

1. A method comprising: executing a plurality of threads using a plurality of processor cores of a processor, wherein each thread in the plurality of threads comprises a plurality of instructions; sampling, at every N cycle, to obtain a program counter (PC) corresponding to one instruction in the plurality of instructions, wherein N is a first positive integer; calculating an instructions per cycle (IPC) corresponding to the PC; caching the PC and the IPC as a cache entry in a cache; searching, at every J new instructions, the cache for a matching PC cache entry corresponding to a new instruction to be executed by the plurality of processor cores, wherein J is a second positive integer; updating an IPC value in a register with a corresponding IPC in the matching PC cache entry when the matching PC cache entry found in the cache; and switching a thread in the plurality of threads to a new thread based on the IPC value in the register.

2. The method of claim 1, wherein sampling comprises: removing a last (PC, Instruction Count (IC)) pair from a second end of a sampling queue, wherein the last (PC, IC) pair comprises a first PC (PCM) and a first IC (ICM), and wherein the sampling queue comprises M entries, and wherein M is a third positive integer; shifting all remaining (PC, IC) pairs in the sampling queue by one entry space towards the second end of the sampling queue to create an empty entry space at a first end of the sample queue; inserting a new (PC, IC) pair into the sampling queue in the empty entry space at the first end of the sampling queue, wherein the new (PC, IC) pair comprises a second PC (PCi) and a second IC (ICi); and setting the first PC as the PC.

3. The method according to any of claims 1-2, wherein determining the IPC corresponding to the PC comprises: calculating a number of instructions that has been processed in between the last (PC, IC) pair and the new (PC, IC) pair by subtracting the first IC (ICM) from the second IC (ICi); and dividing the number of instructions by (N*M) to obtain the IPC corresponding to the PC.

4. The method according to any of claims 1-3, wherein caching the PC and the IPC in the cache comprises: determining whether the cache comprises an existing cache entry for the PC, wherein the cache stores K cache entries, wherein each cache entry in the K cache entries comprises a unique PC and a corresponding IPC value, and wherein K is a fourth positive integer; and updating the existing cache entry with the IPC when the cache comprises the existing cache entry for the PC.

5. The method according to claim 4, wherein K is less than a number of unique instructions in the plurality of instructions.

6. The method according to any of claims 4-5, wherein caching the PC and the IPC in the cache further comprises: determining, when the cache does not comprise the existing cache entry for the PC, whether the cache is full; inserting, when the cache is not full, a new cache entry comprising the PC and the IPC into the cache; and selecting, when the cache is full, a replacement cache entry in the cache and replacing the replacement cache entry with the cache entry for the PC and the IPC.

7. The method according to claim 6, wherein the replacement cache entry has a lowest least recently used (LRU) count in the cache.

8. A processor comprising: plurality of processor cores; one or more memory or storage means configured to store instructions and a cache; and one or more circuits or processing means coupled to the memory or the storage means and configured to execute the instructions to cause the processor to: execute a plurality of threads using a plurality of processor cores of a processor, wherein each thread in the plurality of threads comprises a plurality of instructions; sample, at every N cycle, to obtain a program counter (PC) corresponding to one instruction in the plurality of instructions, wherein N is a first positive integer; calculate an instructions per cycle (IPC) corresponding to the PC; cache the PC and the IPC as a cache entry in the cache; search, at every J new instructions, the cache for a matching PC cache entry corresponding to a new instruction at every J new instructions to be executed by the plurality of processor cores, wherein J is a second positive integer; update an IPC value in a register with a corresponding IPC in the matching PC cache entry when the matching PC cache entry found in the cache; determine, at a thread switching time, a thread in the plurality of threads to switch with a new thread based on the IPC in the IPC register; and switch a thread in the plurality of threads to a new thread based on the IPC value in the register.

9. The processor of claim 8, wherein the plurality of processor cores comprises heterogenous cores comprising different processing capability cores, and wherein a higher processing capability core is less energy-efficient than a lower processing capability core.

10. The processor according to any of claims 8-9, wherein the instructions, when executed by the one or more circuits or processing means, further cause the processor to:remove a last (PC, Instruction Count (IC)) pair from a second end of a sampling queue, wherein the last (PC, IC) pair comprises a first PC (PCM) and a first IC (ICM), and wherein the sampling queue comprises M entries, and wherein M is a third positive integer; shift all remaining (PC, IC) pairs in the sampling queue by one entry space towards the second end of the sampling queue to create an empty entry space at a first end of the sample queue; insert a new (PC, IC) pair into the sampling queue in the empty entry space at the first end of the sampling queue, wherein the new (PC, IC) pair comprises a second PC (PCi) and a second IC (ICi); and set the first PC as the PC.

11. The processor according to any of claims 8-10, wherein the instructions, when executed by the one or more circuits or processing means, further cause the processor to: calculate a number of instructions that has been processed in between the last (PC, IC) pair and the new (PC, IC) pair by subtracting the first IC (ICM) from the second IC (ICi); and divide the number of instructions by (N*M) to obtain the IPC corresponding to the PC.

12. The processor according to any of claims 8-11, wherein the instructions, when executed by the one or more circuits or processing means, further cause the processor to: determine whether the cache comprises an existing cache entry for the PC, wherein the cache stores K cache entries, wherein each cache entry in the K cache entries comprises a unique PC and a corresponding IPC value, and wherein K is a fourth positive integer; and update the existing cache entry with the IPC when the cache comprises the existing cache entry for the PC.

13. The processor according to claim 12, wherein K is less than a number of unique instructions in the plurality of instructions.

14. The processor according to any of claims 12-13, wherein the instructions, when executed by the one or more circuits or processing means, further cause the processor to: determine, when the cache does not comprise the existing cache entry for the PC, whether the cache is full; insert, when the cache is not full, a new cache entry comprising the PC and the IPC into the cache; and select, when the cache is full, a replacement cache entry in the cache and replacing the replacement cache entry with the cache entry for the PC and the IPC.

15. The processor according to claim 14, wherein the replacement cache entry has a lowest least recently used (LRU) count in the cache.

16. An apparatus comprising: one or more memory or storage means configured to store instructions and a cache; and one or more processors or processing means coupled to the memory or the storage means and configured to execute the instructions to cause the apparatus to: execute a plurality of threads using a plurality of processor cores of a processor, wherein each thread in the plurality of threads comprises a plurality of instructions; sample, at every N cycle, to obtain a program counter (PC) corresponding to one instruction in the plurality of instructions, wherein N is a first positive integer; calculate an instructions per cycle (IPC) corresponding to the PC; cache the PC and the IPC as a cache entry in the cache; search, at every J new instructions, the cache for a matching PC cache entry corresponding to a new instruction at every J new instructions to be executed by the plurality of processor cores, wherein J is a second positive integer; update an IPC value in a register with a corresponding IPC in the matching PC cache entry when the matching PC cache entry found in the cache;determine, at a thread switching time, a thread in the plurality of threads to switch with a new thread based on the IPC in the IPC register; and switch a thread in the plurality of threads to a new thread based on the IPC value in the register.

17. A computer program product comprising computer-executable instructions stored on a non-transitory computer-readable storage medium, the computer-executable instructions when executed by one or more processors of an apparatus, cause the apparatus to perform a method according to any of claims 1-7.

Images