Adaptive thread management for heterogeneous computing architecture

Abstract

An apparatus and method for efficiently scheduling tasks in a dynamic manner to a plurality of cores supporting a heterogeneous computing architecture. A computing system includes a plurality of cores, where at least two cores are capable of executing instructions of the same Instruction Set Architecture (ISA), and are thus architecturally compatible. In one implementation, each of the at least two cores is a universal central processing unit (CPU) core capable of executing instructions of the same ISA. However, based on its hardware design, throughput and power consumption differ greatly between the at least two cores. An operating system scheduler allocates a thread to a first core, and the first core measures a thread dynamic behavior of the thread over a time interval. Based on the thread dynamic behavior, the scheduler reallocates the thread to a second core different from the first core.

Description

Background Art

[0001] Description of Related Technology

[0002] In some designs, the microarchitecture of each of the first core and the second core can perform a specific type of task. For example, each of the first core and the second core can execute instructions of the same instruction set architecture (ISA) and is therefore architecturally compatible. The first core and the second core are allowed to access the same area of ​​the memory, so it is possible to exchange workloads between them. However, the hardware design of the first core emphasizes high performance (high throughput), which also increases power consumption. On the contrary, the hardware design of the second core emphasizes low power consumption.

[0003] The first core may use a general purpose microarchitecture and include hardware that emphasizes high performance rather than low power consumption. In contrast, the second core includes hardware that emphasizes low power consumption rather than high performance. For example, the first core utilizes hardware of standard cells from a cell library focused on high performance (high throughput), such as using transistors with shorter channel lengths, higher doping concentrations of source and drain regions, and thinner gate oxide thicknesses. In contrast, the second core utilizes hardware of standard cells from a cell library focused on low power consumption, such as using transistors with longer channel lengths, lower doping concentrations of source and drain regions, and larger gate oxide thicknesses. In some designs, it may also be that the first core uses more hardware than the second core to support multiple instruction issuance, scheduling, execution, and retirement; additional routing and logic to determine data forwarding for multiple instructions simultaneously per clock cycle; complex branch prediction schemes, support for deep pipelines, support for simultaneous multithreading; access to relatively large caches, and other design features. In contrast, the second core uses the same general purpose microarchitecture, but includes significantly less hardware than the listed hardware of the first core. When an architecture includes at least two cores that are architecturally compatible but have significant differences in throughput and power consumption between them, the architecture is referred to as a "heterogeneous computing architecture" or a "heterogeneous processing architecture."

[0004] Do not confuse the above architecture labels with "heterogeneous architectures," which include at least two architecturally incompatible cores such that the at least two cores use different microarchitectures, such as a general-purpose microarchitecture and a relatively wide single instruction multiple data (SIMD) microarchitecture. To schedule workloads running on a computer system with a heterogeneous computing architecture, an operating system (OS) scheduler uses a round-robin scheme or a scheme based on the availability of cores. However, these scheduling schemes can limit performance or consume considerable power when the scheduling schemes do not match the system resources.

[0005] In view of the above, there is a need for effective methods and mechanisms to efficiently and dynamically schedule tasks to multiple cores. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Figure 1 A block diagram of a computing system that efficiently schedules threads on cores supporting a heterogeneous computing architecture.

[0007] Figure 2 is another schematic block diagram of a computing system for efficiently scheduling threads on cores supporting a heterogeneous computing architecture.

[0008] Figure 3 is a high-level block diagram of thread allocation for a computing system using cores supporting a heterogeneous computing architecture.

[0009] Figure 4 is a general block diagram of a method for allocating threads in a computing system using cores supporting a heterogeneous computing architecture.

[0010] Figure 5 is a general block diagram of a method for updating information used in allocating threads in a computing system using cores supporting a heterogeneous computing architecture.

[0011] Figure 6 is a general block diagram of a method for allocating threads in a computing system using cores supporting a heterogeneous computing architecture.

[0012] Although the present invention is susceptible to various modifications and alternative forms, specific implementations are shown by way of example in the drawings and described in detail herein. However, it should be understood that the drawings and detailed description thereof are not intended to limit the invention to the specific forms disclosed, but on the contrary, the invention is to cover all modifications, equivalents and alternatives within the scope of the invention defined by the appended claims. DETAILED DESCRIPTION

[0013] In the following description, many specific details are set forth to provide a thorough understanding of the present invention. However, it will be appreciated by those of ordinary skill in the art that the present invention may be practiced without these specific details. In some cases, well-known circuits, structures, and techniques are not shown in detail to avoid obscuring the present invention. In addition, it should be understood that for simplicity and clarity of illustration, the elements shown in the figures are not necessarily drawn to scale. For example, the size of some elements is enlarged relative to other elements.

[0014] It is conceivable to efficiently schedule tasks to multiple cores supporting heterogeneous computing architectures in a dynamic manner and to use a device and method. In various specific implementations, a computing system includes multiple processor cores (or multiple cores) supporting heterogeneous computing architectures. At least two of the multiple cores are capable of executing instructions of the same instruction set architecture (ISA) and are therefore architecturally compatible. The at least two cores are allowed to access the same area of ​​memory, so it is possible to exchange workloads between them. However, the hardware design of one of the at least two cores emphasizes high performance (high throughput), which also increases power consumption. On the contrary, the hardware design of another of the at least two cores emphasizes low power consumption. In a specific implementation, each of the at least two cores is a general-purpose central processing unit (CPU) core capable of executing instructions of the same instruction set architecture (ISA).

[0015] When an architecture includes at least two cores that are architecturally compatible but have significant differences in throughput and power consumption between them, the architecture is referred to as a "heterogeneous computing architecture" or "heterogeneous processing architecture". This type of architecture should not be confused with a "heterogeneous architecture", which includes at least two architecturally incompatible cores such that the at least two cores use different microarchitectures. An example of a heterogeneous architecture is a computing system having a first core (such as a CPU core) and a second core (such as a graphics processing unit (GPU) core), the first core including a general-purpose microarchitecture and the second core including a relatively wide single instruction multiple data (SIMD) microarchitecture. It should be noted that a computing system can include both a heterogeneous computing architecture (or heterogeneous processing architecture) and a heterogeneous architecture. For example, in one specific implementation, a computing system includes at least two architecturally compatible CPU cores, but have significant differences in throughput and power consumption between them. The computing system also includes at least a GPU core or other core that uses a wide SIMD microarchitecture.

[0016] The computing system uses at least a multi-core heterogeneous computing architecture. The hardware (such as a circuit) of a specific core of the multiple cores of the computing system executes the instructions of the operating system scheduler. The specific core may be a general-purpose CPU core. It should be noted that over time, one or more cores execute the instructions of the operating system scheduler. For example, the hardware (such as a CPU core) of one or more cores respectively executes the instructions of the kernel at different time points and accesses one or more areas of the shared memory. One of the shared data structures maintained by the one or more cores executing the operating system kernel is a list of threads ready to run. Therefore, at a later time point, it is possible and conceivable that the hardware of a core different from a specific core executes the instructions of the operating system scheduler. As used herein, the terms "scheduler" and "scheduling core" refer to one or both of the software and hardware that executes and schedules threads for execution. In some specific implementations, a core that acts as a scheduler or otherwise executes scheduling software is called a scheduling core. For example, a core in a plurality of cores may be an instruction that executes the operating system scheduler. The scheduler assigns a thread to the first core in the plurality of cores of the computing system. The first core may be any one of the plurality of cores of the computing system. The scheduler assigns the thread to the first core based on the availability of the plurality of cores, a round-robin scheduling scheme, or another initial scheduling scheme. The first core determines the classification of the thread based on a hardware performance metric measured over a time interval. In various implementations, the first core includes a hardware performance counter that monitors hardware events that occur in the first core when the first core executes the assigned thread. In some implementations, the first core includes one or more multi-bit registers that function as hardware performance counters capable of counting a plurality of hardware-related activities.

[0017] The first core compares the values ​​stored in one or more hardware performance counters in the hardware performance counters with a threshold value to determine the dynamic behavior of the assigned thread. The number of events that at least reach a predetermined threshold is used to determine the classification of the assigned thread. Examples of classifications are non-scalable threads, input / output (I / O) or memory bottleneck threads, vector threads, or other threads. The first core transmits the classification to the scheduler, which is an indication of the thread dynamic behavior of the thread. As previously described, the scheduler is a core among the multiple cores that executes instructions of the operating system scheduler at the current point in time. In order to transmit, in some specific implementations, the first core writes a specific value (or multiple specific values) in a predetermined configuration register such as a machine-specific register (MSR) assigned to the first core. The scheduler confirms, checks, or polls the predetermined configuration register to determine the value stored in the predetermined configuration register. Based on the storage value previously written by the first core, the scheduler determines a classification, which is an indication of the dynamic behavior of the thread determined by the first core. Based at least in part on the indication of the dynamic behavior of the thread, the scheduler reallocates the thread to a second core different from the first core.

[0018] In one specific implementation, the classification of the above-mentioned thread indicates that the thread is part of a process running a background task. However, the scheduler initially assigns the thread to the first core, which is a high-performance CPU core, based on availability, polling scheme, or other. The first core uses hardware performance counters to classify the thread as a non-scalable thread or other types of threads that are preferably assigned to low-power cores. The first core transmits the classification to the scheduling core (the core running the OS scheduler), which removes the thread from the high-performance core and reallocates the thread to the low-power core. It should be noted that, as used herein, a "high-performance" core refers to a core designed to operate at a higher performance and / or power level than a "low-power" core. Further details of efficiently scheduling tasks to multiple cores supporting heterogeneous computing architectures in a dynamic manner are provided in the following discussion.

[0019] See also Figure 1 , shows a schematic block diagram of a computing system 100 that efficiently schedules threads on cores supporting a heterogeneous computing architecture. Computing system 100 includes a semiconductor chip 110 and a memory 130. Semiconductor chip 110 (or chip 110) includes multiple types of integrated circuits. For example, chip 110 includes at least multiple processor cores, such as the core within processing unit 150 and cores 112 and 116 of processing units 115 and 119. There are multiple options for placing the circuits of chip 110 in a system package to integrate multiple types of integrated circuits. Some examples are system on chip (SOC), multi-chip module (MCM), and system-in-package (SiP). For ease of explanation, Figure 1 Not shown are clock sources, such as a phase-locked loop (PLL), an interrupt controller, a power controller, an interface for input / output (I / O) devices, etc.

[0020] Chip 110 includes a memory controller 120 to communicate with memory 130, interface logic 140, processing units 115 and 119, communication fabric 160, shared cache memory subsystem 170, and shared processing unit 150. Processing unit 115 includes core 112 and corresponding cache memory subsystem 114. Processing unit 119 includes core 116 and corresponding cache memory subsystem 118. Memory 130 is shown to include operating system code 132. It should be noted that various portions of operating system code 132 may reside in memory 130, in one or more caches (114, 118), stored on a non-volatile storage device such as a hard disk (not shown), etc. In one implementation, the illustrated functionality of chip 110 is combined on a single integrated circuit.

[0021] Although two units 115 and 119 are shown each having a single core (core 112 and core 116, respectively), it is possible and conceivable that chip 110 includes another number of units and each unit has another number of cores based on design requirements. Similarly, although a single processing unit 150 is shown, in other implementations, the chip includes another number of other processing units that can communicate with other components of chip 110 via communication fabric 160. In various implementations, cores 112 and 116 can execute one or more threads and share at least a shared cache memory subsystem 170, processing unit 150, and coupled input / output (I / O) devices connected to interface (IF) 140.

[0022] In various specific implementations, the two cores 112 and 116 are capable of executing instructions of the same instruction set architecture (ISA) and are therefore architecturally compatible. The two cores 112 and 116 are allowed to access the same area of ​​​​the memory, so it is possible to exchange workloads between them. In a specific implementation, each of the two cores 112 and 116 is a general-purpose central processing unit (CPU) core that can execute instructions of the same ISA. However, in a specific implementation, the hardware design of the core 112 emphasizes high performance (high throughput), which also increases power consumption. On the contrary, the hardware design of the core 116 emphasizes low power consumption. In some specific implementations, the high-performance core 112 utilizes hardware of standard cells from a cell library focused on high performance (high throughput), such as using transistors with shorter channel lengths, higher doping concentrations of source and drain regions, and thinner gate oxide thicknesses. In contrast, the low-power core 116 utilizes hardware of standard cells from a cell library focused on low power consumption, such as using transistors with longer channel lengths, lower doping concentrations of source and drain regions, and larger gate oxide thicknesses, etc. The high-performance core 112 includes hardware (such as circuits) for multiple instruction issue, scheduling, execution, and retirement; additional routing and logic to determine data forwarding for multiple instructions per clock cycle simultaneously; complex branch prediction schemes, support for deep pipelining, support for simultaneous multithreading; access to relatively large caches, and other design features are also possible and contemplated.

[0023] It is also possible and conceivable that the low-power core 116 includes significantly less hardware than the above-listed hardware of the high-performance core 112 compared to the high-performance core 112. Therefore, the low-power core 116 consumes significantly less power, but also has a throughput that is much lower than the high-performance core 112. When an architecture includes at least two cores that are architecturally compatible but have a significant difference in throughput and power consumption between them, the architecture is referred to as a "heterogeneous computing architecture" or "heterogeneous processing architecture." This type of architecture should not be confused with a "heterogeneous architecture," which includes at least two architecturally incompatible cores such that the at least two cores use different microarchitectures. An example of a heterogeneous architecture is a computing system having a first core (such as a high-performance CPU core 112) and a second core (such as a core of a processing unit 150). In one specific implementation, one or more cores of the processing unit 150 use a relatively wide single instruction multiple data (SIMD) microarchitecture. Other types of data processing and corresponding microarchitectures are also possible and conceivable.

[0024] It should be noted that computing system 100 may include both heterogeneous computing architectures (or heterogeneous processing architectures) and heterogeneous architectures. For example, in various implementations, computing system 100 includes high-performance CPU cores 112 and low-power CPU cores 116 that are architecturally compatible, but throughput and power consumption vary greatly between them. Computing system 100 also includes one or more cores of processing unit 150 that use a microarchitecture that is different from a general-purpose CPU microarchitecture. An example of a different microarchitecture is a wide SIMD microarchitecture.

[0025] As previously described, the hardware (such as circuitry) of a particular core among the multiple cores executes instructions of the operating system scheduler at a current point in time, and the particular scheduling core assigns the thread to the high-performance core 112. The scheduling core assigns the thread to the high-performance core 112 based on the availability of the multiple cores in the computing system 100, a round-robin scheduling scheme, or another initial scheduling scheme. It is possible and conceivable that other cores among the multiple cores include hardware that executes instructions of the operating system scheduler at other points in time. The high-performance core 112 determines the classification of the thread based on hardware performance metrics measured over time intervals. In various specific implementations, the high-performance core 112 includes a hardware performance counter that monitors hardware events that occur in the high-performance core 112 when the first core executes the assigned thread. In some specific implementations, the high-performance core 112 includes one or more multi-bit registers that serve as hardware performance counters capable of counting multiple hardware-related activities.

[0026] Alternatively, the counter counts the number of clock cycles that the high-performance core 112 takes to perform a predetermined event. Examples of events include pipeline flushes, data cache snoops and snoop hits, cache and translation lookaside buffer (TLB) misses, read and write operations, data cache line writebacks, branch operations, accepted branch operations, the number of instructions in the integer or floating point pipeline, and bus utilization. Several other events are also possible and conceivable. In addition to storing absolute numbers corresponding to hardware-related activities, the counters of the high-performance core 112 determine and store relative numbers, such as the percentage of cache read operations that hit the cache. In addition to the hardware performance counters, in one specific implementation, the high-performance core 112 includes a timestamp counter that is used to determine the time rate or frequency of hardware-related activities. For example, the first core uses a timestamp counter to store and update the number of cache read operations per second, the number of pipeline refreshes per second, the number of floating point operations per second, or other. In various specific implementations, the low-power core 116 also includes one or more registers of these types of registers for implementing hardware performance counters.

[0027] The high performance core 112 compares the values ​​stored in one or more hardware performance counters in the hardware performance counters with a threshold value to determine the dynamic behavior of the assigned thread. The number of events that at least reach a predetermined threshold is used to determine the classification of the assigned thread. Examples of classifications are non-scalable threads, input / output (I / O) or memory bottleneck threads, vector threads, or other threads. The high performance core 112 transmits the classification to the scheduling core, which is an indication of the thread dynamic behavior of the thread. In one specific implementation, the high performance core 112 writes the indication in a predetermined configuration register that is later read by the scheduling core.

[0028] The scheduling core is capable of removing threads from the high-performance core 112 and reallocating the threads to the low-power core 116 based at least in part on the indication of the dynamic behavior of the threads. It should also be noted that in various specific implementations, one or more of the high-performance core 112 and the low-power core 116 include hardware capable of executing multiple threads at a particular point in time due to simultaneous multithreading (SMT). If the high-performance core 112 supports SMT for four threads, the scheduling core uses four separate logical processor identifiers (IDs) to allocate and reallocate threads. However, the scheduling core does not reallocate threads from a first logical processor ID of the high-performance core 112 to a second logical processor ID of the high-performance core 112 based on the indication of the dynamic behavior of the threads.

[0029] In one specific implementation, the classification of the above thread indicates that the thread is part of a process running video playback, a process running an input / output (I / O) bottleneck or memory bottleneck task, or a process running a pause or spin loop (busy spin or wait spin). However, the scheduling core initially assigns threads to the high-performance core 112 based on availability, polling scheme, or other. These types of threads are preferably assigned to the low-power core 116. The high-performance core 112 communicates the classification (indication of dynamic thread behavior) to the scheduling core, which removes the thread from the high-performance core 112 and reallocates the thread to the low-power core 116.

[0030] In other specific implementations, the reallocation is from the low-power core 116 to the high-performance core 112. For example, when the scheduling core initially allocates a thread of a real-time process (such as a real-time audio / visual (A / V) process of a video game) to the low-power core 116 based on availability, a polling scheme, or otherwise, after a time interval, the low-power core 116 sends an indication of the dynamic thread behavior to the scheduling core. Based on the indication, the scheduling core removes the real-time thread from the low-power core 116 and reallocates the real-time thread to the high-performance core 112. Before continuing with further details of efficiently scheduling tasks to multiple cores in support of a heterogeneous computing architecture in a dynamic manner, a further description of the components of the computing system 100 is provided.

[0031] The interface 140 is generally provided for various types of input / output (I / O) devices from the chip 110 to the shared cache memory subsystem 170 and the processing unit 115. Generally, the interface logic 140 includes a buffer for receiving packets from the corresponding link and for buffering packets to be sent on the corresponding link. Any suitable flow control mechanism can be used to send packets to and from the chip 110. The memory 130 can be used as the system memory of the chip 110 and includes any suitable memory device, such as one or more RAMBUS dynamic random access memory (DRAM), synchronous DRAM (SDRAM), static RAM, etc.

[0032] The address space of the chip 110 is divided between multiple memories corresponding to the multiple cores. In one specific implementation, the consistency point of the address is a memory controller 120 that communicates with the memory storing the bytes corresponding to the address. The memory controller 120 includes a circuit for interfacing to the memory and a request queue for queuing memory requests. In general, the communication structure 160 generates control packets in response to control packets received on the link of the IF 140, in response to the cores 112 and 116 and / or the cache memory subsystems 114 and 118, in response to transactions selected for service by the memory controller 120, generates probe commands and response packets, and routes the packets to other nodes through the interface logic 140. The communication structure supports various packet transmission protocols and includes one or more of a system bus, a packet processing circuit and a packet selection arbitration logic, and a queue for storing requests, responses, and messages.

[0033] The cache memory subsystems 114 and 118 include relatively high-speed cache memories to store data blocks. The cache subsystem 114 can be integrated into the corresponding high-performance core 112. Alternatively, the cache memory subsystem 114 is connected to the high-performance core 112 in a backside cache configuration or an inline configuration as needed. The cache memory subsystem 114 can be implemented as a hierarchical structure of caches. In a specific implementation, the cache memory subsystem 114 represents an L2 cache structure, and the shared cache subsystem 170 represents an L3 cache structure. The cache memory subsystem 118 can be implemented as any of the above-described methods described for the cache memory subsystem 114.

[0034] Now turn to Figure 2 , shows another schematic block diagram of a computing system 200 that efficiently schedules threads on cores that support heterogeneous computing architectures. The computing system 200 includes a microcontroller 210, an assigned processor core 220 (or cores 220), a classification table 230, and a scheduler core 240. It should be noted that although various references are made to "tables" herein, in various specific implementations, the data / content identified as being stored by these tables can be stored in any of a variety of forms other than the tables themselves. For example, various data structures (trees, indexes, associative searches, etc.) allocated in memory can be used to store data. All such implementations are possible and conceivable. For ease of illustration, Figure 2Clock sources, such as phase-locked loops (PLLs), interrupt controllers, power controllers, interfaces for input / output (I / O) devices, etc. are not shown. In addition, in various implementations, in addition to the assigned core 220 and the scheduler core 240, the computing system 200 also includes other cores, and the computing system supports a multi-core heterogeneous computing architecture. In some implementations, the assigned core 220, the scheduler core 240, and the other cores are general-purpose CPU cores. In another implementation, the functions of the microcontroller 210 are implemented by a core or another integrated circuit.

[0035] The allocated core 220 includes a thread class table 222 (or table 222) that maps a class identifier (ID) or an enumeration value (Enum) to a specific thread class type. As shown, examples of thread class types or classifications are non-scalable threads, input / output (I / O) or memory bottleneck threads, vector threads, or other threads. Non-scalable threads come from processes that run pauses or spin loops (busy spins or wait spins), and in a specific implementation, non-scalable threads are preferably assigned to low-power cores. In some specific implementations, threads from processes that run input / output (I / O) bottleneck or memory bottleneck tasks are also preferably assigned to low-power cores. Threads from processes that run vector and / or floating-point operations are preferably assigned to high-performance cores. Threads that are not identified as being in any other class are identified as default thread class types (or default classifications). In a specific implementation, default threads are preferably assigned to high-performance cores. The "HP" indicator specifies that thread class types 0-1 are preferably assigned to high-performance cores, and the "LP" indicator specifies that thread class types 2-3 are preferably assigned to low-power cores. Other allocation and thread class types are possible and contemplated.

[0036] In one specific implementation, the microcontroller 210 executes instructions of firmware that implements an algorithm for generating and updating values ​​stored in a classification table 230 (or table 230). Table 230 stores information that maps cores of the computing system 200 participating in a heterogeneous computing architecture to identified thread class types. The mapping includes rankings that are weight values ​​and determine how strong the preference is for a particular thread class type to be assigned to a high-performance core or a low-power core. The cores are identified by core identifiers (IDs), which are shown as "c0", "c1", and so on. The thread class types use the same identifiers as those used in table 222.

[0037] The ranking or weight of the mapping of allocations between core "c0" and thread class type 0 is shown as "eff_0". The ranking "eff_0" indicates that thread class type 0 is not preferably allocated to low power cores, but instead, is allocated to high performance cores. Here, low power cores are indicated as "e" and "eff" for effective cores, and high performance cores are indicated as "p" and "perf" for performance cores. The ranking or weight of the mapping of allocations between core "c0" and thread class type 0 is also shown as "perf_3". Here, the higher the numerical value, the stronger the preference for allocation. Therefore, the ranking "perf_3" indicates that thread class type 0 is strongly preferably allocated to high performance cores, such as core "c0". Here, the ranking or weight "0" indicates a strong avoidance of making corresponding allocations between cores and thread class types. Other value types, ranges, and indications for the ranking in Table 230 are possible and conceivable.

[0038] In some implementations, hardware (such as a circuit) of the microcontroller 210 sends table updates 212 to a specific area of ​​memory storing the table 230. Copies of the table 222 and the table 230 are stored in various types of data storage areas. For example, a copy of the data stored in the table 222 and the table 230 is stored in one or more of the following: a register, a flip-flop circuit, one of various random access memories, a content addressable memory (CAM), etc. Copies of the contents of the table 222 and the table 230 are also stored in a non-volatile memory such as a hard disk, a solid-state drive, a read-only memory (ROM), or others. When the microcontroller 210 generates or later updates the content stored in the table 230, the microcontroller 210 also sends a notification 214 to the scheduler core 240. In some implementations, the notification 214 is an interrupt sent from the microcontroller 210 to the scheduler core 240. When the scheduler core 240 receives the notification 214, the scheduler core 240 retrieves the most recent copy of the updated content and updates its own local copy of the table 230.

[0039] Microcontroller 210 determines whether a system event has occurred in computing system 200. Examples of system events are notification of a detected throttling condition from a power manager (due to a high thermal measurement above a threshold), notification of a particular type of application being run (such as a real-time audio / visual application), or others. If microcontroller 210 determines that a system event has occurred, microcontroller 210 updates the contents of table 230 and sends notification 214 to scheduler core 240.

[0040] The scheduler core 240 executes the instructions of the operating system scheduler and assigns threads to the assigned cores 220 based on availability, polling scheme or other. At this time, the scheduler core 240 is unaware of the dynamic thread behavior of the assigned threads. In various specific implementations, the assigned core 220 includes a hardware performance counter that monitors hardware events that occur in the assigned core 220 when the assigned core 220 executes the assigned thread. After the elapsed time interval, the assigned core 220 compares the values ​​in one or more hardware performance counters stored in the hardware performance counter with a threshold value to determine the dynamic behavior of the assigned thread. The assigned core 220 accesses the table 222 to perform classification of the dynamic behavior of the assigned thread. The assigned core 220 reports the dynamic thread behavior to the scheduler core 240. In some specific implementations, the assigned core 220 writes a specific value (or multiple specific values) indicating the thread classification 224 in a predetermined configuration register such as a machine-specific register (MSR) assigned to the assigned core 220. The scheduler core 240 periodically polls the configuration register and reads the stored value. The thread classification 224 provides scheduling feedback to the scheduler core 240. The scheduling feedback relies on the measured dynamic behavior of the threads running on the assigned cores 220.

[0041] The scheduler core 240 uses the identifier of the assigned core 220 and the information of the received thread classification 224 to access the copy of the table 230. In some specific implementations, the scheduler 240 first verifies whether enough time has elapsed since the thread was assigned to the assigned core 220 for accurate thread classification to occur. If yes, the scheduler core 240 accesses the copy of the table 230. If not, the scheduler core 240 sends an instruction to the assigned core 220 to perform the classification again. Using the ranking (weight) in the table 230, the scheduler core 240 determines whether the thread should remain running on the assigned core 220. If not, the scheduler core 240 identifies another core different from the assigned core 220 to run the thread. The scheduler core 240 sends the thread allocation 242 to the core, which removes the thread from the assigned core 220 and allocates the thread to run on the identified different core.

[0042] See also Figure 3, shows a high-level block diagram of thread allocation 300 for a computing system using a core supporting a heterogeneous computing architecture. Here, the partitioning of hardware and software resources and their interrelationships and allocations during the execution of one or more software applications 320 is shown. In one specific implementation, the circuitry of the scheduling core of the execution operating system 318 allocates memory areas for processes 308a-308q. When the application 320 or computer program is executed, each application includes multiple processes, such as processes 308a-308j and 308k-308q. In such specific implementations, each of the multiple processes 308a-308q includes its own resources, such as an image of the memory, or an instance of instructions and data before the application is executed. In addition, each of the multiple processes 308a-308q includes process-specific information, such as address spaces for addressing code, data, and possibly heaps and stacks; variables in data and control registers (such as stack pointers, general and floating point registers), program counter values, etc.; and operating system descriptors (such as stdin, stdout, etc.) and security attributes (such as the processor owner and the permission set of the process).

[0043] In each of the multiple processes 308a-308q, there are one or more software threads. For example, process 308a includes software (SW) threads 310a-310d. Threads execute independently of other threads in their corresponding processes, and threads can execute concurrently with other threads in their corresponding processes. In general, each process in the multiple threads 310a-310q belongs only to one of the processes 308a-308q. Therefore, for multiple threads of the same process (such as SW threads 310a-310d of process 308a), the same data content of the memory row (e.g., the row of address 0xff38) can be the same for all threads. This assumes that inter-thread communication is safe and handles conflicts in which the first thread (e.g., SW thread 310a) writes to the memory row, which is read by the second thread (e.g., SW thread 310d).

[0044] However, for multiple threads of different processes (such as SW thread 310a in process 308a and SW thread 310e in process 308j), the data content of the memory row with address 0xff38 can be different for the threads. However, if multiple threads of different processes share the same part of the address space, then multiple threads of the different processes can access the same data content at a specific address. Generally, for a given application, the scheduling core of the executing operating system 318 establishes an address space for the application, loads the code of the application into memory, establishes a stack for the program, branches to a given location inside the application, and starts execution of the application. Typically, the part of the operating system 318 that manages these activities is the operating system kernel 312.

[0045] As described above, an application can be divided into multiple processes, and the hardware computing system 302 can run multiple applications. Therefore, there can be several processes running in parallel. The scheduling core of the execution kernel 312 decides which of the processes that are executed simultaneously should be assigned to the processor core at any time. The scheduler 316 in the operating system 318 (which can be within the kernel 312) includes decision logic for assigning threads of the process to the core. In addition, as further described below, the scheduling core of the execution scheduler 316 decides to assign a specific one of the software threads 310a-310q to a specific hardware processor core of the hardware processor cores 314a-314g and 314h-314r within the hardware computing system 302.

[0046] In various implementations, cores 314a-314g and 314h-314r are general purpose CPU cores and include hardware capable of handling the execution of the one or more threads 310a-310q within one of processes 308a-308q. Hardware computing system 302 supports a heterogeneous computing architecture. Dashed lines are used to separate high performance cores 314a-314g from low power cores 314h-314r. Figure 3 The dashed lines in also represent assignments but do not necessarily represent direct physical connections. Thus, for example, the SW thread 310d may be assigned to the high-performance core 314a. However, later (e.g., after thread reallocation), the scheduling core removes the SW thread 310d from the high-performance core 314a and reallocates the SW thread 310d to the low-power core 314h based on an indication of the dynamic behavior of the SW thread 310d as reported by the high-performance core 314a.

[0047] In one specific implementation, an identifier (ID) is assigned to each of the cores 314a-314g and 314h-314r. The hardware thread ID (not shown) is used to assign one of the SW threads 310a-310q to one of the cores 314a-314g for execution. The scheduling core of the scheduler 316 within the execution kernel 312 handles the assignment. For example, the scheduling core uses the hardware thread ID to assign the SW thread 310m to the low power core 314r based on the availability of the high performance cores 314a-314g and the low power cores 314h-314r, a polling scheduling scheme, or another initial scheduling scheme. The scheduling core of the execution scheduler 316 performs thread reallocation based on the dynamic thread behavior reported by the high performance cores 314a-314g and the low power cores 314h-314r. For example, after a time interval of executing the SW thread 310m, the low power core 314r determines the dynamic behavior of the SW thread 310m. In one implementation, the low performance core 314r compares the values ​​stored in one or more hardware performance counters in the hardware performance counters with a threshold value to determine the dynamic behavior of the SW thread 310m. The low power core 314r writes a specific value (or multiple specific values) in a predetermined configuration register such as a machine specific register (MSR) assigned to the low power core 314r. In one implementation, the predetermined configuration register is a specific area of ​​memory.

[0048] The scheduler confirms, checks or polls the predetermined configuration register to determine the value stored in the predetermined configuration register. Based on the storage value previously written by the low-power core 314r, the scheduler determines a classification, which is an indication of the dynamic behavior of the thread determined by the low-power core 314r. Based on the classification indicating the dynamic behavior of the SW thread 310m reported by the low-power core 314r, the scheduling core removes the SW thread 310m from the low-power core 314r and reallocates the SW thread 310m to the high-performance core 314g. In order to assist thread migration, user data allocated by one thread is used only by the thread, and inter-thread data sharing is implemented via read-only global variables and fast local message passing is performed via the scheduling core of the execution thread scheduler 316. If any, the scheduling core also handles any changes to the stack pointer.

[0049] In various implementations, the high performance cores 314a-314g and the low power cores 314h-314r include hardware performance counters that monitor hardware events that occur during a predetermined time interval while the high performance cores 314a-314g and the low power cores 314h-314r execute the assigned threads. After the time interval elapses, the high performance cores 314a-314g and the low power cores 314h-314r compare the values ​​stored in one or more hardware performance counters in the hardware performance counters with a threshold value to determine the dynamic behavior of the assigned threads. The high performance cores 314a-314g and the low power cores 314h-314r report the dynamic thread behavior to the scheduling core, which determines whether to reallocate the threads.

[0050] In one implementation, the scheduling core preferentially assigns threads of processes running video playback, processes running input / output (I / O) bottleneck or memory bottleneck tasks, or processes running pauses or spin loops (busy spin or wait spin) to low power cores 314h-314r. Conversely, the scheduling core preferentially assigns threads of real-time processes (such as real-time A / V video games) to high performance cores 314a-314g. However, in some implementations, the preferences used by the scheduling core are programmable, and updated values ​​of these preferences are stored in a table or other data storage area accessible by the scheduling core.

[0051] Now turn to Figure 4 , shows a schematic block diagram of a method 400 for allocating threads in a computing system using cores supporting a heterogeneous computing architecture. For discussion purposes, this specific implementation (and Figures 5 and 6 However, in other implementations, some steps occur in a different order than shown, some steps are performed simultaneously, some steps are combined with other steps, and some steps are not present.

[0052] The computing system includes multiple cores that support a heterogeneous computing architecture. The hardware of a general-purpose CPU processor core in the multiple cores executes instructions of an operating system scheduler, and the CPU processor core (or scheduler) assigns threads to a specific processor core in the multiple processor cores of the computing system (box 402). The scheduler assigns threads to the specific processor core (or specific core) based on the availability of the multiple cores, a polling scheduling scheme, or another initial scheduling scheme. The specific core determines the classification of the thread assigned to it based on hardware performance metrics measured over time intervals (box 404). In various specific implementations, the specific core includes a hardware performance counter that monitors hardware events that occur in the specific core when the specific core executes the assigned thread. In some specific implementations, the specific core includes one or more multi-bit registers that are used as hardware performance counters that can count multiple hardware-related activities.

[0053] Alternatively, the counter counts the number of clock cycles taken by the particular core to perform a predetermined event. Examples of events include pipeline refreshes, data cache snoops and snoop hits, cache and TLB misses, read and write operations, data cache line writebacks, branch operations, accepted branch operations, the number of instructions in integer or floating point pipelines, and bus utilization. Several other events are also possible and conceivable. In addition to storing absolute numbers corresponding to hardware-related activities, the counter of a particular core determines and stores relative numbers, such as the percentage of cache read operations that hit the cache. In addition to the hardware performance counters, in a specific implementation, a particular core includes a timestamp counter that is used to determine the time rate or frequency of hardware-related activities. For example, a particular core uses a timestamp counter to store and update the number of cache read operations per second, the number of pipeline refreshes per second, the number of floating point operations per second, or other.

[0054] The particular core compares the values ​​stored in one or more hardware performance counters in the hardware performance counters with a threshold value to determine the dynamic behavior of the assigned thread. The number of events that at least reach the predetermined threshold is used to determine the classification of the assigned thread. Examples of classifications are non-scalable threads, input / output (I / O) or memory bottleneck threads, vector threads, or other threads. The particular core transmits the classification to the scheduler (block 406). In some specific implementations, the particular core writes a specific value (or multiple specific values) in a predetermined configuration register such as a machine-specific register (MSR) assigned to the particular core. The scheduler verifies whether the particular core has hardware that matches the dynamic behavior of the thread by checking the classification table (block 408).

[0055] The classification table stores an ordered mapping of defined matches between the plurality of cores and a plurality of predefined classifications of thread dynamic behavior. In some implementations, the classification table is generated and updated by another core of the plurality of cores, a microcontroller, or other processing unit assigned with circuitry to execute specific firmware using instructions of an algorithm that determines how and when to update the contents of the classification table. The scheduler loads the contents of the classification table into a local cache and accesses the classification table when evaluating initial thread assignments.

[0056] If the scheduler determines based on the classification table that there is a match between the assigned thread and the specific core (the "yes" branch of conditional box 410), the scheduler maintains the thread as being assigned to the specific core (box 412). The match indicates that the initial thread assignment is found to be a valid assignment in the classification table. If the scheduler determines based on the classification table that there is no match (the "no" branch of conditional box 410), the scheduler assigns the thread to a different core that matches the dynamic behavior of the thread based on the classification table (box 414). The different core is not a specific core, but another core of a computing system using a multi-core heterogeneous computing architecture. The scheduler reallocates the thread from a specific core to a different core. It should be noted that in addition to the steps performed in boxes 408 to 414 of method 400, it is possible and conceivable that the scheduler also includes other decision logic for a specific scheduling strategy.

[0057] See also Figure 5 , a schematic block diagram of a method 500 for updating information used in allocating threads in a computing system using cores supporting a heterogeneous computing architecture is shown. The computing system includes multiple cores supporting a heterogeneous computing architecture. Hardware (such as circuitry) of a particular processor core of a plurality of cores or microcontrollers used in the computing system generates an ordered mapping between the plurality of cores and types of thread dynamic behaviors (block 502). These ordered mappings are stored in a classification table. The circuitry notifies an operating system scheduler of the mappings (block 504). In some implementations, a particular core sends an interrupt to another core executing an operating system scheduler.

[0058] The circuit determines whether a system event has occurred. Examples of system events are notification of a throttling condition detected from a power manager (due to a high thermal measurement above a threshold), notification of a particular type of application being run, or other. If the circuit determines that a system event has occurred (the "yes" branch of conditional block 506), the circuit updates the sorting map in the classification table based on the system event (block 508). For example, the circuit updates the sorting map to indicate that more threads and possibly all threads are allocated to low-power cores rather than high-performance cores of a computing system using a heterogeneous computing architecture. The circuit notifies the scheduler of the updated map (block 510). If the circuit determines that a system event has not occurred (the "no" branch of conditional block 506), the circuit maintains the sorting map in the classification table (block 512).

[0059] Now turn to Figure 6 , a schematic block diagram of a method 600 for allocating threads in a computing system using cores supporting a heterogeneous computing architecture is shown. The computing system includes multiple cores supporting a heterogeneous computing architecture. Hardware (such as circuitry) of a particular processor core of the multiple cores determines the quality of service (QoS) of a thread (block 602). In some implementations, the particular core that determines the QoS value of the thread is a core that executes instructions of an operating system kernel. In other implementations, the thread (or process) includes an indication of a QoS value. If the particular core determines that there is no indication of a QoS value (the "yes" branch of conditional block 604), the particular core allocates the thread to a high-performance core (block 606). In some implementations, each thread has a QoS value, and conditional block 604 is unnecessary. In other implementations, as performed in conditional block 604, a confirmation is performed to determine whether the thread has an assigned QoS value.

[0060] If the particular core determines that there is an indication of a QoS value (the "No" branch of conditional block 604) and the indicated QoS value is greater than the threshold (the "Yes" branch of conditional block 608), the particular core assigns the thread to the high-performance core (block 606). However, if the particular core determines that the indicated QoS value is less than or equal to the threshold (the "No" branch of conditional block 608), the particular core assigns the thread to the low-power core (block 610). It should be noted that in another specific implementation, these assignments can be switched based on the results of the comparison with the threshold. It should also be noted that the particular core can compare the assignment steps of method 600 with methods 400 to 500 ( Figures 4 to 5 In some other specific implementations, a specific core is based on the allocation steps of method 600 and methods 400 to 500 ( Figures 4 to 5 ) to calculate the weighted score. For example, when reallocating a thread from an initially assigned core to another core, the scheduler uses the QoS value in addition to the weight and the classification indicating the dynamic behavior of the thread. Various combinations and methods are possible and conceivable for determining the final allocation of threads based on dynamic thread behavior in a heterogeneous computing architecture.

[0061] It should be noted that one or more of the above-mentioned specific implementations include software. In such specific implementations, the program instructions of the implementation method and / or mechanism are transmitted or stored on a computer-readable medium. Many types of media configured to store program instructions are available and include hard disks, floppy disks, CD-ROMs, DVDs, flash memory, programmable ROMs (PROMs), random access memories (RAMs) and various other forms of volatile or non-volatile storage devices. Generally speaking, a computer-accessible storage medium includes any storage medium that can be accessed by a computer during use to provide instructions and / or data to a computer. For example, a computer-accessible storage medium includes such as magnetic or optical media, such as storage media such as disks (fixed or removable), tapes, CD-ROMs or DVD-ROMs, CD-Rs, CD-RWs, DVD-Rs, DVD-RWs or blue-rays. The storage medium also includes volatile or non-volatile storage media, such as RAM (e.g., synchronous dynamic RAM (SDRAM), double data rate (DDR, DDR2, DDR3, etc.) SDRAM, low power DDR (LPDDR2, etc.) SDRAM, Rambus DRAM (RDRAM), static RAM (SRAM), etc.), ROM accessible via a peripheral device interface (such as a universal serial bus (USB) interface, etc.), flash memory, non-volatile memory (e.g., flash memory). The storage medium includes micro-electromechanical systems (MEMS), and storage media accessible via communication media such as a network and / or a wireless link.

[0062] In addition, in various specific implementations, the program instructions include a behavioral level description or a register transfer level (RTL) description of the hardware functionality in a high-level programming language such as C or a design language (HDL) such as Verilog, VHDL, or a database format such as GDS II stream format (GDSII). In some cases, the description is read by a synthesis tool, which synthesizes the description to produce a netlist including a list of gates from a synthesis library. The netlist includes a set of gates, which also represents the functionality of the hardware comprising the system. The netlist is then placed and threaded to produce a data set describing the geometry to be applied to the mask. The mask is then used in various semiconductor manufacturing steps to produce a semiconductor circuit or circuit corresponding to the system. Alternatively, the instructions on the computer-accessible storage medium are a netlist (with or without a synthesis library) or a data set as desired. In addition, the instructions are used by the computer-accessible storage medium such as and Mentor Such vendors' hardware-based types of emulators are used for emulation purposes.

[0063] Although the above embodiments have been described in considerable detail, numerous variations and modifications will become apparent to those skilled in the art once the above disclosure is fully appreciated. It is intended that the following claims be interpreted to encompass all such variations and modifications.

Claims

1. A device, comprising: A scheduler, the scheduler being configured to: receiving an indication of thread dynamic behavior for a given thread assigned to a first core of the plurality of cores; as well as Based at least in part on the indication of the thread dynamic behavior of the given thread, the given thread is reassigned to a second core different from the first core of the plurality of cores. 2 . The apparatus of claim 1 , wherein the scheduler is further configured to check an ordering map indicating a defined match between the plurality of cores and a plurality of classifications of thread dynamic behavior.

3. The apparatus of claim 2, wherein the scheduler is further configured to reallocate the given thread from the first core to the second core in response to determining, using the indication of the thread dynamic behavior, that the ordering map indicates that the second core has a defined match. The apparatus of claim 2 , wherein the scheduler is further configured to receive a notification indicating that the ordering map has been updated. 5 . The apparatus of claim 1 , wherein the scheduler is further configured to execute an operating system scheduler of a computing system using a multi-core heterogeneous computing architecture.

6. The device according to claim 5, wherein: The first core executes threads using a first microarchitecture; and The second core executes threads using a second microarchitecture that is different from the first microarchitecture.

7. The apparatus of claim 1, wherein the indication of the thread dynamic behavior is based on a hardware performance counter in the first core.

8. A method comprising: One or more applications are executed by multiple cores; receiving, by a given core, an indication of thread dynamic behavior for a given thread of the one or more applications assigned to a first core of the plurality of cores; as well as Based at least in part on the indication of the thread dynamic behavior of the given thread, the given thread is reallocated, by the given core, to a second core different from the first core of the plurality of cores.

9. The method of claim 8, further comprising examining, by the given core, an ordered map indicating defined matches between the plurality of cores and the plurality of classifications of thread dynamic behavior.

10. The method of claim 9, further comprising reallocating, by the given core, the given thread from the first core to the second core in response to determining, using the indication of the thread dynamic behavior, that an ordering map indicates that the second core has a defined match.

11. The method of claim 9, further comprising receiving, by the given core, a notification indicating that a sorting map has been updated.

12. The method of claim 8, further comprising executing, by the given core, an operating system scheduler of a computing system using a multi-core heterogeneous computing architecture.

13. The method according to claim 12, further comprising: Executing a thread by the first core using a first micro-architecture; as well as Threads are executed by the second core using a second microarchitecture different from the first microarchitecture.

14. The method of claim 8, wherein the indication of thread dynamic behavior is based on hardware performance counters in the first core.

15. A computing system, comprising: a memory configured to store one or more applications of the workload; and a plurality of cores configured to execute the one or more applications; and wherein a given core of the plurality of cores is configured as: receiving an indication of thread dynamic behavior for a given thread assigned to a first core of the plurality of cores; as well as Based at least in part on the indication of thread dynamic behavior of the given thread, the given thread is reassigned to a second core different from the first core of the plurality of cores.

16. The computing system of claim 15, wherein the given core is further configured to check an ordered map indicating a defined match between the plurality of cores and a plurality of classifications of thread dynamic behavior.

17. The computing system of claim 16, wherein the given core is further configured to reallocate the given thread from the first core to the second core in response to determining, using the indication of the thread dynamic behavior, that the ordering map indicates that the second core has a defined match.

18. The computing system of claim 16, wherein the given core is further configured to receive a notification indicating that the ordering map has been updated.

19. The computing system of claim 15, wherein the given core is further configured to execute an operating system scheduler of the computing system using a multi-core heterogeneous computing architecture.

20. The computing system of claim 19, wherein: The first core executes threads using a first microarchitecture; and The second core executes threads using a second microarchitecture that is different from the first microarchitecture.

Images