Thread replay to reserved state in barrel processor

Abstract

Described herein are apparatuses and techniques for thread replay to a reserved state in a barrel processor. One apparatus includes a barrel processor including a temporary storage; and a thread scheduling circuit; wherein the barrel processor is configured to perform operations using the thread scheduling circuit, the operations including scheduling a current thread on a clock cycle to be placed in a pipeline for the barrel processor, the barrel processor to schedule a thread on each clock cycle; storing the current thread in the temporary storage; detecting that no thread is available on a clock cycle following the cycle in which the current thread is scheduled; and in response to detecting that no thread is available on the subsequent clock cycle, rescheduling the current thread based on contents of the temporary storage.

Description

Technical Field

[0001] This disclosure relates to chiplets, and more specifically, to thread reloading in a barrel processor to a reserved state. Background Technology

[0002] Chiplets are an emerging technology for integrating various processing functions. Typically, a chiplet system consists of discrete modules (each referred to as a "chiplet") integrated on an interposer layer and, in many instances, interconnected via one or more established networks as needed to provide the required functionality to the system. The interposer layer and the contained chiplets may be packaged together for easy interconnection with other components of a larger system. Each chiplet may contain one or more individual integrated circuits (ICs) or "chips," which may be combined with discrete circuit components and are typically coupled to a corresponding substrate for attachment to the interposer layer. Most or all of the chiplets in a system can be individually configured for communication via one or more established networks.

[0003] Chipsets, configured as individual modules within a system, differ from such systems implemented on a single chip containing different device blocks (e.g., intellectual property (IP) blocks) on a substrate (e.g., a single die), such as a system-on-a-chip (SoC) or multiple discrete packaged devices integrated on a printed circuit board (PCB). Generally, chiplets offer better performance (e.g., lower power consumption, less latency, etc.) than discrete packaged devices, and offer greater manufacturing efficiency than a single die chip. These manufacturing efficiency benefits may include improved yield or reduced development costs and time.

[0004] A chiplet system may comprise, for example, one or more application (or processor) chiplets and one or more support chiplets. Here, the distinction between application chiplets and support chiplets is merely a reference to possible design scenarios for chiplet systems. Thus, for example, a synthetic vision chiplet system may comprise (by example only) application chiplets for generating synthetic vision output, and support chiplets, such as memory controller chiplets, sensor interface chiplets, or communication chiplets. In typical use cases, synthetic vision designers may design the application chiplets and obtain support chiplets from other sources. Therefore, design expenditures (e.g., in terms of time or complexity) are reduced by avoiding the design and manufacture of the functionality contained in the support chiplets. Chiplets also support the tight integration of IP blocks that might otherwise be difficult, such as IP blocks manufactured using different processing technologies or with different feature sizes (or utilizing different contact technologies or pitches). Therefore, multiple ICs or IC components with different physical, electrical, or communication characteristics can be assembled in a modular manner to provide components that implement the required functionality. Chiplet systems also facilitate adaptation to the needs of different larger systems that will be incorporated into the chiplet system. In examples, ICs or other components can be optimized for power, speed, or heat generation for specific functions, as may be the case with sensors. Compared to attempting integration with other devices on a single die, these ICs or other components can be integrated more easily. Furthermore, by reducing the overall size of the die, the yield of small chips is often higher than that of more complex single-die devices. Summary of the Invention

[0005] One aspect of this disclosure provides an apparatus comprising: a bucket processor including: temporary memory; and thread scheduling circuitry; wherein the bucket processor is configured to perform operations using the thread scheduling circuitry, the operations comprising: scheduling a current thread for placement in a pipeline for the bucket processor on a clock cycle, the bucket processor scheduling threads on each clock cycle; storing the current thread in the temporary memory; detecting that no thread is available on a clock cycle following the cycle in which the current thread was scheduled; and, in response to detecting that no thread is available on the subsequent clock cycle, repeatedly scheduling the current thread based on the contents of the temporary memory.

[0006] Another aspect of this disclosure provides a method comprising: scheduling a current thread for placement in a pipeline for the bucket processor on a clock cycle at a bucket processor including thread scheduling circuitry, the bucket processor scheduling threads on each clock cycle; storing the current thread in a temporary memory; detecting that no thread is available on a clock cycle following the cycle in which the current thread was scheduled; and, in response to detecting that no thread is available on the subsequent clock cycle, repeatedly scheduling the current thread based on the contents of the temporary memory.

[0007] Another aspect of this disclosure provides an apparatus comprising: a memory controller chiplet in a chiplet system, the memory controller chiplet including: a programmable atom unit, the programmable atom unit including: a barrel processor, the barrel processor including: temporary memory; and thread scheduling circuitry; wherein the barrel processor is configured to perform operations using the thread scheduling circuitry, the operations including: scheduling a current thread for placement in a pipeline for the barrel processor on a clock cycle, the barrel processor scheduling threads on each clock cycle; storing the current thread in the temporary memory; detecting that no thread is available on a clock cycle following the cycle in which the current thread was scheduled; and, in response to detecting that no thread is available on the subsequent clock cycle, repeatedly scheduling the current thread based on the contents of the temporary memory. Attached Figure Description

[0008] This disclosure will be more fully understood from the detailed description given below and the accompanying drawings of various embodiments thereof. However, the drawings should not be construed as limiting this disclosure to the specific embodiments, but are for explanation and understanding only.

[0009] Figure 1A and 1B An example of a chiplet system according to an embodiment is described.

[0010] Figure 2 This describes the components of an example of a memory controller chiplet according to an embodiment.

[0011] Figure 3 This describes the components in an example of a programmable atomic unit (PAU) according to an embodiment.

[0012] Figure 4 This is a block diagram illustrating the control and data flow in the pipeline according to an embodiment.

[0013] Figure 5 This is a flowchart of an example of a method for thread execution control in a bucket processor according to an embodiment.

[0014] Figure 6 This is a block diagram of an instance of a machine, which embodiments of this disclosure may utilize, operate in, or be performed via. Detailed Implementation

[0015] The following description Figure 1A-1BExamples of chiplet systems and components operating therein are provided. The illustrated chiplet system includes a memory controller. This memory controller includes programmable atomic units (PAUs) for executing custom programs, programmable atomic operations, in response to memory requests for programmable atomic operations. The following will refer to... Figure 2 and 3 Provide additional details about the PAU. The PAU's processor can be either bucket-based multithreaded or pipelined.

[0016] Power management within a barrel-style multithreaded processor presents an increasingly challenging task: balancing performance and throughput. In a multithreaded processing system, the processor may advance to a low-power state when no thread is ready to execute. The cycle between low-power and ready states can cause latency, as threads wait for the processor to reach the ready state. Alternative techniques increase circuit complexity, such as gating the clock of certain logic blocks or other power-intensive aspects of the computation pipeline that are unnecessary for a given thread. Furthermore, these techniques incur their own power costs as the processor logic switches between idle and active states.

[0017] This disclosure describes a control mechanism for replaying or repeating the scheduling of the current thread when no thread is available for pipeline processing. By maintaining the scheduler output, the logic in the computation pipeline remains in a steady state. The power consumption associated with switching logic states from idle to active is eliminated. Therefore, the improved control mechanism reduces overall power consumption.

[0018] The significant advantages offered by this control mechanism include reduced circuit complexity, overall power consumption reduction, and minimal additional circuitry required compared to other proposed systems for solving this problem.

[0019] Figure 1A and 1B An example of a chiplet system 110 according to an embodiment is described. Figure 1A This is an illustration of a chiplet system 110 mounted on a peripheral board 105, which can be connected to a wide range of computer systems via, for example, a peripheral component interconnect high-speed (PCIe) interface. The chiplet system 110 includes a package substrate 115, an interposer 120, and four chips: an application chiplet 125, a host interface chiplet 135, a memory controller chiplet 140, and a memory device chiplet 150. Other systems may include numerous additional chipsets to provide additional functionality, as will be apparent from the following discussion. The package of the chiplet system 110 is illustrated with a cap or cover plate 165, but other packaging techniques and structures used for chiplet systems may be used. Figure 1B This is a block diagram for the purpose of clearly labeling the components in a chiplet system.

[0020] Application chip 125 is described as including an on-chip network (NOC) 130 to support a chiplet network 155 for inter-chiplet communication. In an example embodiment, NOC 130 may be included on application chip 125. In an example, NOC 130 may be defined in response to the selected supporting chips (e.g., chips 135, 140, and 150), allowing the designer to select an appropriate number of chiplet network connections or switches for NOC 130. In an example, NOC 130 may reside on a single chiplet or even within intermediary layer 120. In the example discussed herein, NOC 130 implements a chiplet protocol interface (CPI) network.

[0021] CPI is a packet-based network that supports virtual channels to enable flexible, high-speed interaction between chiplets. CPI bridges the chiplet-internal network to chiplet network 155. For example, the Advanced Extensible Interface (AXI) is a widely used specification for designing intra-chip communication. However, the AXI specification covers a large number of physical design options, such as the number of physical channels, signal timing, and power. Within a single chip, these options are generally selected to meet design goals, such as power consumption and speed. However, to achieve flexibility in chiplet systems, adapters such as CPI are used to intersect between various AXI design options that can be implemented in various chiplets. By implementing a mapping from physical channels to virtual channels and encapsulating time-based signaling using packetization protocols, CPI bridges the chiplet-internal network across chiplet network 155.

[0022] CPI can utilize various physical layers to transmit packets. A physical layer may contain simple conductive connections, or it may contain drivers to increase voltage, or otherwise facilitate signal transmission over longer distances. Examples of such physical layers may include an Advanced Interface Bus (AIB), which in various instances may be implemented in intermediate layer 120. The AIB uses source-synchronous data transfer with a forwarding clock to transmit and receive data. Packets are transmitted across the AIB at Single Data Rate (SDR) or Double Data Rate (DDR) relative to the transmitted clock. The AIB supports various channel widths. When operating in SDR mode, the AIB channel width is a multiple of 20 bits (20, 40, 60…), and for DDR mode, the AIB channel width is a multiple of 40 bits (40, 80, 120…). The AIB channel width includes both transmitted and received signals. Channels can be configured with a symmetrical number of transmit (TX) and receive (RX) inputs / outputs (I / O), or with an asymmetrical number of transmitters and receivers (e.g., all transmitters or all receivers). Depending on which chiplet provides the master clock, the channel can act as either AIB control or slave (i.e., "master" or "slave"). The AIB I / O unit supports three timing modes: asynchronous (i.e., non-timing), SDR, and DDR. In various instances, the non-timing mode is used for the clock and some control signals. SDR mode can use a dedicated SDR-only I / O unit or a dual SDR / DDR I / O unit.

[0023] In this example, the CPI packet protocol (e.g., point-to-point or routable) can use symmetrical receive and transmit I / O units within an AIB channel. The CPI streaming protocol allows for more flexible use of AIB I / O units. In this example, a streaming AIB channel can configure I / O units as all TX, all RX, or half TX and half RX. The CPI packet protocol can use AIB channels in SDR or DDR operating modes. In this example, AIB channels are configured in increments of 80 I / O units (i.e., 40 TX and 40 RX) for SDR mode and in increments of 40 I / O units for DDR mode. The CPI streaming protocol can use AIB channels in SDR or DDR operating modes. Here, in this example, AIB channels are configured in increments of 40 I / O units for both SDR and DDR modes. In this example, a unique interface identifier is assigned to each AIB channel. This identifier is used during CPI reset and initialization to determine the paired AIB channels across the interconnect chiplets. In this example, the interface identifier is a 20-bit value comprising a seven-bit chiplet identifier, a seven-bit column identifier, and a six-bit link identifier. The AIB physical layer uses an AIB out-of-band shift register to transmit the interface identifier. Bits 32 to 51 of the shift register are used to transmit the 20-bit interface identifier in both directions across the AIB interface.

[0024] AIB defines a stacked group of AIB channels as an AIB channel column. An AIB channel column has a certain number of AIB channels, plus auxiliary channels. The auxiliary channels contain signals used for AIB initialization. All AIB channels within a column (except for the auxiliary channels) have the same configuration (e.g., all TX, all RX, or half TX and half RX, and have the same number of data I / O signals). In this example, AIB channels are numbered sequentially, starting with the AIB channel adjacent to the AUX channel. The AIB channel adjacent to the AUX channel is defined as AIB channel zero.

[0025] Typically, the CPI interface of an individual chiplet may include serialization-deserialization (SERDES) hardware. SERDES interconnects are well-suited for applications requiring high-speed signaling and low signal counts. However, SERDES can introduce additional power consumption and longer latency for multiplexing and demultiplexing, error detection or correction (e.g., using block-level cyclic redundancy check (CRC)), link-level retries, or forward error correction. However, when low latency or power consumption is a primary concern for ultra-short-distance chiplet-to-chiplet interconnects, parallel interfaces that allow data transfer with minimal latency can be utilized. CPIs contain elements designed to minimize both latency and power consumption in these ultra-short-distance chiplet interconnects.

[0026] For flow control, CPI employs a credit-based technique. For example, the receiving side of application chip 125, or the sender side of memory controller chip 140, provides credits indicating available buffers. In this example, the CPI receiver contains buffers for each virtual channel for a given transmission time unit. Therefore, if the CPI receiver supports five messages and a single virtual channel in time, the receiver has five buffers arranged in five columns (e.g., one row per unit time). If four virtual channels are supported, the receiver has twenty buffers arranged in five columns. Each buffer holds the payload of one CPI packet.

[0027] When a sender transmits to a receiver, the sender decrements its available credits based on the transmission. Once the receiver has used up all its credits, the sender stops sending packets to the receiver. This ensures that the receiver always has an available buffer to store transmissions.

[0028] When the receiver processes the received packet and releases the buffer, it relays the available buffer space back to the sender. The sender can then use this credit to allow the transmission of additional information.

[0029] The document also describes a chiplet mesh network 160 that uses direct chiplet-to-chiplet technology without requiring a NOC130. The chiplet mesh network 160 can be implemented in a CPI or another chiplet-to-chiplet protocol. The chiplet mesh network 160 typically implements a chiplet pipeline, where one chiplet acts as an interface to the pipeline, while other chipslets in the pipeline interface only interface with themselves.

[0030] In addition, dedicated device interfaces can be used to interconnect chiplets, such as one or more industry-standard memory interfaces 145 (e.g., synchronous memory interfaces, such as DDR5, DDR6). Connections from chiplet systems or individual chiplets to external devices (e.g., larger systems) can be made through a desired interface (e.g., a PCIe interface). In an example, for instance, an external interface can be implemented via a host interface chiplet 135, which, in the depicted example, provides a PCIe interface external to the chiplet system 110. Such interfaces are typically used when industry practices or standards have converged on them. The illustrated example of connecting a memory controller chiplet 140 to a Double Data Rate (DDR) interface 145 of a dynamic random access memory (DRAM) memory device 150 is such an industry practice.

[0031] Among the various possible supporting chiplets, the memory controller chiplet 140 is likely to be present in the chiplet system 110 due to the ubiquitous use of storage for computer processing and the application of advanced technologies to memory devices. Therefore, using the memory device chiplet 150 and the memory controller chiplet 140, both produced using other technologies, allows chiplet system designers to obtain robust products manufactured by established companies. Typically, the memory controller chiplet 140 provides a memory device-specific interface for reading, writing, or erasing data. The memory controller chiplet 140 often provides additional features such as error detection, error correction, maintenance operations, or atomic operation execution. For some types of memory, maintenance operations are often specific to the memory device 150, such as garbage collection in NAND flash or storage-class memory, or temperature regulation (e.g., cross-temperature management) in NAND flash memory. In instances, maintenance operations may involve logic-to-physical (L2P) mapping or management to provide an indirection hierarchy between the physical and logical representations of data. In other types of memory, such as DRAM, some memory operations, such as refresh, may be controlled at certain times by the host processor or memory controller, and at other times by the DRAM memory device or logic associated with one or more DRAM devices, such as interface chips (in this example, buffers).

[0032] Atomic operations are data manipulations, for example, that can be performed by the memory controller chiplet 140. In a chiplet system, atomic operations can be performed by other chipsets. For example, an atomic operation can be specified as an "increment" in a command by the application chiplet 125, the command containing a memory address and possibly an increment value. Upon receiving the command, the memory controller chiplet 140 retrieves a number from the specified memory address, increments the number by the amount specified in the command, and stores the result. Upon successful completion, the memory controller chiplet 140 provides the application chiplet 125 with an indication that the command was successful. Atomic operations avoid transmitting data across the chiplet network 160, thereby reducing the waiting time for executing such commands.

[0033] Atomic operations can be categorized into built-in atoms or programmable (e.g., custom) atoms. Built-in atoms are a finite set of operations that are implemented immutably in the hardware. Programmable atoms are small programs that can be executed on programmable atom units (PAUs) (e.g., custom atom units (CAUs)) of the memory controller chiplet 140. Figure 2 Explain and discuss examples of PAU memory controller chiplets.

[0034] The memory device chiplet 150 may be or contain any combination of volatile memory devices or non-volatile memory. Examples of volatile memory devices include, but are not limited to, random access memory (RAM), such as DRAM, synchronous DRAM (SDRAM), graphics dual data rate type 6 SDRAM (GDDR6 SDRAM), and so on. Examples of non-volatile memory devices include, but are not limited to, NAND flash memory, memory-class memory (e.g., phase-change memory or memristor-based technology), ferroelectric RAM (FeRAM), and so on. The illustrated example includes the memory device 150 as a chiplet; however, the memory device 150 may reside elsewhere, such as in different packages on peripheral board 105. For many applications, multiple memory device chipsets may be provided. In the examples, these memory device chipsets may each implement one or more memory technologies. In one example, the memory chiplet may contain multiple stacked memory dies of different technologies, such as stacked or additionally communicating with one or more DRAM devices or SRAM devices. The memory controller 140 can also be used to coordinate the operation between multiple memory chips in the chiplet system 110; for example, utilizing one or more memory chips in one or more tiers of cache storage and using one or more additional memory chips as main memory. The chiplet system 110 may also include multiple memory controllers 140, which can be used to provide memory control functionality for individual processors, sensors, networks, etc. For example, the chiplet architecture of the chiplet system 110 offers the advantage of allowing adaptation to different memory storage technologies and different memory interfaces through updated chiplet configurations without requiring redesign of the rest of the system architecture.

[0035] Figure 2The components of an example of a memory controller chiplet 205 according to an embodiment are described below. The memory controller chiplet 205 includes a cache 210, a cache controller 215, an off-die memory controller 220 (e.g., for communicating with off-die memory 275), a network communication interface 225 (e.g., for interfacing with a chiplet network 285 and communicating with other chiplets), and a set of atom and merging units 250. This set of components may include, for example, a write merging unit 255, a memory danger clearance unit 260, a built-in atom operation unit 265, or a PAU 270. The various components are described logically and they may not necessarily be implemented. For example, the built-in atom operation unit 265 may include different means along a path to off-die memory. For example, the built-in atom operation unit 265 may be located in an interface means / buffer on the memory chiplet, as discussed above. In contrast, the programmable atom unit 270 may be implemented in a separate processor on the memory controller chiplet 205 (but in various instances, it may be implemented in other locations, such as on the memory chiplet itself).

[0036] The off-die memory controller 220 is directly coupled to off-die memory 275 (e.g., via a bus or other communication connection) to provide write and read operations to and from one or more off-die memory devices, such as off-die memory 275 and off-die memory 280. In the depicted example, the off-die memory controller 220 is also coupled to atom and merge cells 250 for outputs and to cache controller 215 (e.g., a memory-side cache controller) for inputs.

[0037] In the instance configuration, the cache controller 215 is directly coupled to the cache 210 and can be coupled to the network communication interface 225 for input (e.g., incoming read or write requests) and coupled to the off-die memory controller 220.

[0038] Network communication interface 225 includes packet decoder 230, network input queue 235, packet encoder 240, and network output queue 245 to support packet-based chiplet network 285, such as CPI. Chiplet network 285 can provide packet routing between and within processors, memory controllers, mixed-thread processors, configurable processing circuitry, or communication interfaces. In such packet-based communication systems, each packet typically contains destination and source addressing, as well as any data payload or instructions. In some instances, depending on the configuration, chiplet network 285 may be implemented as a collection of crossbar switches with a folded Clos configuration, or as a mesh network providing additional connectivity.

[0039] In various instances, the chiplet network 285 can be part of an asynchronous switching architecture. Here, data packets can be routed along any of various paths, such that any selected data packet can arrive at its addressed destination at any time among multiple different times, depending on the routing. Alternatively, the chiplet network 285 can be implemented at least partially as a synchronous communication network, such as a synchronous mesh communication network. Both configurations of the communication network are intended for use in embodiments according to this disclosure.

[0040] The memory controller chip 205 can receive packets having, for example, a source address, a read request, and a physical address. In response, the off-die memory controller 220 or the cache controller 215 reads data from the specified physical address (which may be in off-die memory 275 or cache 210) and assembles a response packet into a source address containing the requested data. Similarly, the memory controller chip 205 can receive packets having a source address, a write request, and a physical address. In response, the memory controller chip 205 writes data to the specified physical address (which may be in cache 210 or off-die memory 275 or 280) and assembles a response packet into a source address containing confirmation that data has been stored in memory.

[0041] Therefore, where possible, the memory controller chiplet 205 may receive read and write requests via chiplet network 285 and process the requests using cache controller 215, which interfaces with cache 210. If cache controller 215 is unable to handle the request, then off-die memory controller 220 handles the request by communicating with off-die memory 275 or 280, atom and merge unit 250, or both. As described above, one or more levels of cache may also be implemented in off-die memory 275 or 280; and in some such instances, they may be directly accessed by cache controller 215. Data read by off-die memory controller 220 may be cached in cache 210 by cache controller 215 for later use.

[0042] Atom and merge unit 250 is coupled to receive (as input) the output of off-die memory controller 220 and provides the output to cache 210, network communication interface 225, or directly to chiplet network 285. Memory hazard clearing unit 260, write merge unit 255, and built-in (e.g., predetermined) atom operation unit 265 may each be implemented as a state machine with other combinational logic circuitry (e.g., adders, shifters, comparators, AND gates, OR gates, XOR gates, or any suitable combination thereof) or other logic circuitry. These components may also include one or more registers or buffers to store operands or other data. PAU 270 may be implemented as one or more processor cores or control circuitry, and various state machines with other combinational logic circuitry or other logic circuitry, and may also include one or more registers, buffers, or memories to store addresses, executable instructions, operands, and other data, or may be implemented as a processor.

[0043] Write merging unit 255 receives read data and request data, and merges the request data and read data to create a single unit having the read data and the source address to be used in the response or return data packet. Write merging unit 255 provides the merged data to the write port of cache 210 (or equivalently, to cache controller 215 for writing to cache 210). Optionally, write merging unit 255 provides the merged data to network communication interface 225 to encode and prepare response or return data packets for transmission on chiplet network 285.

[0044] When the requested data is for a built-in atomic operation, the built-in atomic operation unit 265 receives the request and reads the data from the write merging unit 255 or directly from the off-chip memory controller 220. The atomic operation is performed, and using the write merging unit 255, the resulting data is written to the cache 210 or provided to the network communication interface 225 to encode and prepare response or return packets for transmission on the chiplet network 285.

[0045] Built-in atomic operation unit 265 handles predefined atomic operations, such as fetch and increment or compare and swap. In examples, these operations perform simple read-modify-write operations on a single memory location of 32 bytes or less. An atomic memory operation is initiated from a request packet transmitted via chiplet network 285. The request packet has a physical address, atomic operator type, operand size, and optionally up to 32 bytes of data. The atomic operation performs a read-modify-write operation on a cache line of cache 210, thereby filling the cache memory as necessary. The atomic operator response can be a simple complete response or a response with up to 32 bytes of data. Example atomic memory operators include fetch and AND, fetch and OR, fetch and XOR, fetch and add, fetch and subtract, fetch and increment, fetch and decrement, fetch and minimum, fetch and maximum, fetch and swap, and compare and swap. In various example embodiments, 32-bit and 64-bit operations and operations on 16 or 32 bytes of data are supported. The methods disclosed herein are also compatible with hardware that supports larger or smaller operations and more or less data.

[0046] Built-in atomic operations may also involve requests for "standard" atomic operations on the requested data, such as relatively simple single-cycle integer atoms, such as fetch and increment or compare and swap, whose throughput will be the same as regular memory read or write operations that do not involve atomic operations. For these operations, cache controller 215 can typically preserve cache lines in cache 210 by setting a danger bit (in hardware) so that the cache line cannot be read by another process during translation. Data is obtained from off-chip memory 275 or cache 210 and provided to built-in atomic operation unit 265 to perform the requested atomic operation. After the atomic operation, in addition to providing the obtained data to packet encoder 240 to encode outgoing data packets for transmission on chiplet network 285, built-in atomic operation unit 265 also provides the obtained data to write merging unit 255, which writes the obtained data back to cache 210. After the obtained data is written to cache 210, memory danger clearing unit 260 clears any corresponding danger bits that were set.

[0047] The PAU 270 implements high performance (high throughput and low latency) for programmable atomic operations (also known as "custom atomic transactions" or "custom atomic operations"), comparable to the performance of built-in atomic operations. Instead of performing multiple memory accesses, in response to an atomic operation request specifying a programmable atomic operation and a memory address, circuitry in the memory controller chiplet 205 transmits the atomic operation request to the PAU 270 and sets a danger bit in a memory danger register corresponding to the memory address used in the atomic operation to ensure that no other operation (read, write, or atomic) is performed on that memory line. The danger bit is then cleared after the atomic operation is completed. The additional, direct data path provided to the PAU 270 for performing programmable atomic operations allows for additional write operations without being limited by the bandwidth of the communication network and without increasing any congestion on the communication network.

[0048] The PAU 270 includes a multi-threaded processor, such as a RISC-VIS-based multi-threaded processor, with one or more processor cores and further featuring an extended instruction set for performing programmable atomic operations. When equipped with the extended instruction set for performing programmable atomic operations, the PAU 270 can be embodied as one or more hybrid-threaded processors. In some example implementations, the PAU 270 provides bucket-style polling instantaneous thread switching to maintain a high instruction-per-clock rate.

[0049] Programmable atomic operations can be executed by PAU 270, which relate to requests for programmable atomic operations on requested data. Users can prepare programming code to provide such programmable atomic operations. For example, a programmable atomic operation can be a relatively simple multi-loop operation, such as floating-point addition, or it can be a relatively complex multi-instruction operation, such as a Bloom filter insert. Programmable atomic operations can be the same as or different from built-in atomic operations, as long as they are defined by the user and not the system vendor. For these operations, cache controller 215 can preserve cache lines in cache 210 by setting a danger bit (in hardware), preventing the cache lines from being read by another process during translation. Data is obtained from cache 210 or off-chip memory 275 or 280 and provided to PAU 270 to execute the requested programmable atomic operation. After the atomic operation, PAU 270 provides the obtained data to network communication interface 225 to directly encode outgoing data packets containing the obtained data for transmission on chiplet network 285. Furthermore, PAU 270 provides the obtained data to cache controller 215, which in turn writes the obtained data to cache 210. After writing the obtained data to cache 210, cache controller 215 clears any corresponding dangerous bits that were set.

[0050] In the selected example, the approach taken for programmable atomic operations is to provide multiple generic custom atomic request types, which can be sent from a source of origin, such as a processor or other system component, to the memory controller chiplet 205 via chiplet network 285. Cache controller 215 or off-die memory controller 220 recognizes the request as a custom atom and forwards the request to PAU 270. In a representative embodiment, PAU 270: (1) is a programmable processing element capable of efficiently performing user-defined atomic operations; (2) can perform load and store operations on memory, arithmetic and logic operations, and control flow decisions; and (3) utilizes a RISC-V ISA with a new set of dedicated instructions to facilitate interaction with such controllers 215, 220, thereby performing user-defined operations atomically. In desirable instances, the RISC-V ISA contains a complete instruction set supporting high-level language operators and data types. PAU 270 may utilize the RISC-V ISA, but typically supports a more limited instruction set and a limited register file size to reduce the die size of the unit when contained within the memory controller chiplet 205.

[0051] As mentioned above, before writing read data to cache 210, the memory danger clearing unit 260 clears the set danger bits of the reserved cache lines. Therefore, when the write merging unit 255 receives a request and read data, the memory danger clearing unit 260 can send a reset or clear signal to cache 210 to reset the set memory danger bits of the reserved cache lines. Furthermore, resetting this danger bit will also release pending read or write requests involving the specified (or reserved) cache line, thereby providing the pending read or write requests to the inbound request multiplexer for selection and processing.

[0052] Figure 3 This describes components in an example of the programmable atom unit (PAU) 300 according to an embodiment, such as those described above. Figure 1B (For example, in memory controller 140) and Figure 2 (For example, the components mentioned in PAU 270). As illustrated, PAU 300 includes a processor 305, local memory 310 (e.g., SRAM), and a controller 315 for the local memory 310.

[0053] In one example, processor 305 is a barrel processor with circuitry for switching between different register files (e.g., a set of registers containing the current processing state) immediately after each clock cycle of processor 305. While processor 305 is discussed herein in an example implementation of the PAU 300 of memory controller chiplet 205, the configuration and operation of processor 305 can also be used for other functions of memory controller chiplet 205 (e.g., performing built-in atomic operations), or in a variety of other scenarios, including but not limited to other chiplets in chiplet system 110. This configuration of the barrel processor enables efficient context switching between currently executing threads. In this example, processor 305 supports eight threads, resulting in eight register files. In this example, some or all of the register files 330 are not integrated into processor 305 but reside in local memory 310. This reduces circuitry complexity in processor 305 by eliminating conventional flip-flops for these registers.

[0054] Local memory 310 may also house caches for atomic operations and atomic instructions 320. Atomic instructions 320 include instruction sets supporting atomic operations loaded by various applications. When an atomic operation is requested, for example, by application chip 125, the instruction set corresponding to the atomic operation is executed by processor 305. In one example, atomic instructions 320 reside in a partition of local memory 310. In this example, a specific programmable atomic operation requested by a requesting process can be identified by a partition number. Partition numbers can be established when a programmable atomic operation is registered with (e.g., loaded onto) PAU 300. Additional metadata for programmable atomic instructions, such as partition tables, may also be stored in local memory 310.

[0055] Atomic operations manipulate cache 325, which is substantially synchronized (e.g., flushed) when the thread used for the atomic operation completes. Therefore, latency for most memory operations is reduced during the execution of the programmable atomic operation thread, except for initial loads from external memory such as off-die memory 275 or 280.

[0056] For example, a barrel processor such as processor 305 can be configured to serve several threads in a spinning manner. Processor 305 is configured with thread scheduling circuitry 335 to iterate through a scheduling queue and identify threads ready for execution. The thread scheduling queue may contain instructions, operations, operands, thread identifiers, or other metadata about the threads.

[0057] The thread scheduling circuit 335 identifies the first thread ready to execute and injects it into the computation pipeline. Thread information, such as thread metadata, is stored in a backup storage device for later reference. If no other thread is ready to execute during subsequent loops, the thread scheduling circuit 335 enters an idle state. Specifically, the thread scheduling circuit 335 propagates a signal along the computation pipeline that the thread is invalid and then re-deploys or replays the previous thread using the thread information from the backup storage device. By reusing the previous thread, the logic in the computation pipeline remains in a stable state. Therefore, when the processor 305 is idle, the components in the computation pipeline do not need to be powered down, store and load states, or perform other complex operations. Although the computation pipeline does use power to replay the same thread instructions, this power is less than the power used to change the gate state.

[0058] Thread scheduling circuitry 335 provides replay functionality for threads in the bucket processor to a reserved state. Thread scheduling circuitry 335 is configured to schedule the current thread for placement into the pipeline used by bucket processor 305 during a clock cycle. Bucket processor 305 is used to attempt to schedule threads on each clock cycle.

[0059] The thread scheduling circuit 335 updates or causes an update of the current thread in temporary memory. In one example, the temporary memory includes registers. Registers may be dedicated registers for the thread scheduling circuit 335 or the processor 305, with a direct data path from the processor 305 or the thread scheduling circuit 335 to the registers. Registers may be of a limited size for storing specific information, such as four bits, six bits, eight bits, etc. In one example, registers are used to store thread identifiers and thread metadata.

[0060] In another embodiment, the registers may be stored in the local random access memory of the bucket processor 305, such as local memory 310. In this embodiment, the registers may have reserved address space in local memory 310 for use by the thread scheduling circuitry 335.

[0061] The thread scheduling circuit 335 detects that no thread is available for scheduling in the clock cycle following the cycle in which the current thread is scheduled. For example, in one embodiment, in a processor handling eight concurrent threads, each allocated a clock cycle, the thread scheduling circuit 335 may detect that no thread is available for service during the next seven clock cycles (seven other threads). In another example embodiment, only active threads are considered for scheduling. Therefore, when all other threads are busy, for example, waiting for memory responses, these threads may not be ready for rescheduling. In this case, the scheduler detects that no thread is available for service and reschedules the current thread to the next available cycle. The detection that all other threads are busy takes one cycle, so the current thread can be rescheduled for two cycles after its first attempt to execute.

[0062] In response to detecting that no thread is available in subsequent clock cycles, the thread scheduling circuit 335 reschedules, replays, or reschedules the current thread based on the contents of temporary memory. In one instance, the thread scheduling circuit rewrites or causes a thread to be rewritten to temporary memory in response to detecting that no thread is available in subsequent clock cycles.

[0063] The signal can be asserted to indicate whether a loop is valid or invalid. Therefore, a valid loop signal can be asserted or propagated through the compute pipeline to indicate that the current thread or its instruction is a valid thread or instruction. This can be executed the moment the thread or instruction passes through the pipeline. When the thread or instruction is replayed, effectively idling the processor 305, the signal can be changed to an invalid loop signal to indicate to the compute pipeline that the thread was the replayed thread. In this case, the compute pipeline can remain in a stable state based on the asserted invalid loop signal.

[0064] Therefore, in one embodiment, in conjunction with scheduling the current thread to be placed in the pipeline, the thread scheduling circuit 335 propagates or causes a valid loop signal to be propagated to the pipeline based on the instructions of the current thread, thereby causing the pipeline to change state.

[0065] When different threads become available for scheduling, the current thread can then be released from its stagnation in the pipeline. Therefore, when the thread scheduling circuit 335 detects a thread in the thread queue with valid instructions for the pipeline, it schedules different threads. The thread scheduling circuit 335 can store different thread identifiers in a temporary memory, which in turn causes the pipeline to fetch instructions for the newly identified thread and pass them to the computation pipeline. Additionally, the thread scheduling circuit 335 can propagate or cause a valid loop signal to be propagated to the pipeline based on the instructions of different threads, thereby causing the pipeline to change its state.

[0066] like Figure 3 As shown, the thread scheduling circuitry 335 is integrated into the barrel processor 305, and the barrel processor 305 is contained within the programmable atom unit 300. The programmable atom unit 300 is contained within the memory controller. In one example, the memory controller is a chiplet in the chiplet system 110 (e.g., memory controller chiplet 220 or memory controller chiplet 140).

[0067] Figure 4 This is a block diagram illustrating the control and data flow in the pipeline according to an embodiment. Thread scheduling circuitry 335 is configured to serve a plurality of thread scheduling queues 405A-N (collectively referred to herein as 405). Thread scheduling queues 405 may contain information for each thread and instructions for queuing the corresponding thread. Queue contents may include thread identifiers, instructions, priority information, status or status flags, or the like.

[0068] When the thread scheduling circuit 335 schedules a thread, it obtains information from a selected queue. Thread information, such as a thread identifier, is sent to the computation pipeline 410. The computation pipeline 410 contains various components that use the thread identifier to retrieve instructions to be executed, perform memory operations to obtain operand data from one or more source addresses, use the instructions to obtain the results, and store the results at the destination address. Actions performed in the computation pipeline 410 are performed as atomic operations.

[0069] exist Figure 4 In the example shown, thread information is fed into a multiplexer (MUX) 415 and stored in temporary storage 420. Temporary storage 420 may be a register dedicated to the thread scheduling circuit 335 or other backup memory. Thread information is then propagated to the computation pipeline 410.

[0070] The valid signal 425 serves as a selection signal to the MUX 415, controlling whether the compute pipeline 410 uses a value from temporary storage 420 or data from the thread scheduling circuit 335. When a new thread is scheduled, the valid signal 425 is set to indicate that the thread is valid, and the thread information for the new thread is stored in temporary storage 420 and passed to the compute pipeline 410. When no other thread is ready to be scheduled, the valid signal 425 is set to indicate that the thread is invalid (e.g., in a loop). In this case, the same thread information is retrieved from temporary storage 420 and propagated to the compute pipeline 410. The compute pipeline 410 also receives the valid signal 425, and when the valid signal 425 is set to indicate that the thread is invalid, the compute pipeline 410 will maintain a steady state and not change its state. This allows the compute pipeline 410 to remain idle or standstill without changing its power state. In other words, no components in the compute pipeline 410 are powered down, and no state is stored for later recovery.

[0071] When a different thread is ready to be scheduled, the valid signal 425 changes to indicate that the different thread is valid (e.g., logic one). A different thread identifier or other information is sent to the MUX 415. This thread information is stored in temporary memory 420, replacing the previous thread identifier. The thread information is also passed to the compute pipeline 410 for execution.

[0072] Figure 5 This is a flowchart illustrating an example of a method for replaying threads in a bucket processor to a reserved state, according to an embodiment. The operation of method 500 is performed by computer hardware, for example, relative to… Figure 1A-1B (For example, memory controller chiplets) Figure 2 , Figure 3 or Figure 6(For example, processing circuitry) as described. In 505, the current thread is scheduled to be placed in a pipeline for a bucket processor (e.g., processor 305). Thread scheduling circuitry (e.g., thread scheduling circuitry 335) schedules threads on a clock cycle. The bucket processor is configured to attempt to schedule threads on each clock cycle.

[0073] In one embodiment, in conjunction with scheduling the current thread to be placed into the pipeline, a valid loop signal is propagated to the pipeline based on the current thread's instruction to cause the pipeline to change its state. In another embodiment, an invalid loop signal is propagated to the pipeline to cause the pipeline to maintain its state.

[0074] At 510, the current thread is stored in temporary memory, such as temporary storage device 420. In one embodiment, the temporary memory includes registers. In yet another embodiment, the registers are stored in the local random access memory of the bucket processor.

[0075] At 515, the thread scheduling circuitry detects that no threads are available in subsequent clock cycles.

[0076] In 520, in response to detecting that no thread is available on a clock cycle following the current thread's scheduling cycle, the thread scheduling circuitry reschedules, replays, or repeats the scheduling of the current thread based on the contents of temporary memory. When no other thread is ready to be scheduled, the thread scheduling circuitry may rewrite the thread to temporary memory in response to detecting that no thread is available on a subsequent clock cycle.

[0077] In one embodiment, repeatedly scheduling the current thread includes instructions for repeatedly scheduling the current thread. The instructions associated with the thread may be atomic operations. Atomic operations may be built-in atoms or programmable atomic operations. The atomic operation may be stored as atomic instructions 320 in local memory 310.

[0078] Method 500 further includes determining which threads are available for scheduling, and then scheduling the instructions of the different threads as operations to be placed in the pipeline. These different threads are stored in temporary memory, and a valid loop signal is propagated to the pipeline based on the instructions of the different threads, causing the pipeline to change state.

[0079] In one embodiment, the thread scheduling circuitry is integrated into a barrel processor, which is contained within a programmable atomic unit, and the programmable atomic unit is contained within a memory controller. In another embodiment, the memory controller is a chiplet in a chiplet system, such as System-on-Chips 110.

[0080] Figure 6The block diagram illustrating example machine 600 illustrates that any or more of the techniques (e.g., methods) discussed herein can be implemented using, in, or through said machine. As described herein, an example may include, or be operated by, logic or multiple components or mechanisms in machine 600. A circuit (e.g., processing circuitry) is a collection of circuits implemented in a tangible entity of machine 600, which includes hardware (e.g., simple circuits, gates, logic, etc.). Circuit membership can become flexible over time. A circuit includes components that, when operated, can perform a specified operation individually or in combination; thus, in various cases, such as in the various controllers described herein, processing circuitry may include one or more processors or processor cores or other instruction execution means. In the example, the hardware of the circuit can be designed in an immutable manner to perform a specific operation (e.g., hardwired). In this example, the hardware of the circuit may include physically connected components (e.g., execution units, transistors, simple circuits, etc.) and machine-readable media that are physically modified (e.g., the magnetic, electrical, movable placement, etc., of an unchanging number of particles) to encode instructions for specific operations. When connecting the physical components, the fundamental electrical properties of the hardware components are altered, for example, from an insulator to a conductor, or vice versa. The instructions enable the embedded hardware (e.g., execution unit or loading mechanism) to generate circuit components in the hardware via the variable connections to perform specific operations when in operation. Thus, in this example, the machine-readable media element is part of the circuit or communicatively coupled to other components of the circuit during device operation. In this example, any one of the physical components can be used in more than one component in more than one circuit. For example, under operation, an execution unit may be used at one point in time in a first circuit and reused by a second circuit in the first circuit, or reused at a different time by a third circuit in the second circuit. Further examples of these components of machine 600 are given below.

[0081] In alternative embodiments, machine 600 may operate as a standalone device or may be connected (e.g., network-connected) to other machines. In a networked deployment, machine 600 may operate as a server machine, a client machine, or both in a server-client network environment. In an example, machine 600 may act as a peer-to-peer (P2P) (or other distributed) network environment. Machine 600 may be a personal computer (PC), tablet PC, set-top box (STB), personal digital assistant (PDA), mobile phone, network device, network router, switch, or bridge, or any machine capable of (sequentially or otherwise) executing instructions specifying actions to be taken by said machine. Furthermore, although only a single machine is shown, the term "machine" should also be considered to include any collection of machines, such as cloud computing, software as a service (SaaS), and other computer cluster configurations, that individually or jointly execute one or more sets of instructions to perform any one or more of the methodologies discussed herein.

[0082] Machine (e.g., computer system) 600 may include a hardware processor 602 (e.g., a central processing unit (CPU), graphics processing unit (GPU), hardware processor core, or any combination thereof), main memory 604, static memory (e.g., firmware, microcode memory or storage device, basic input / output (BIOS), unified extensible firmware interface (UEFI), etc.) 606, and mass storage device 608 (e.g., hard disk drive, tape drive, flash memory device, or other block device), some or all of which may communicate with each other via interconnect (e.g., bus) 630. Machine 600 may further include a display unit 610, an alphanumeric input device 612 (e.g., keyboard), and a user interface (UI) navigation device 614 (e.g., mouse). In an example, the display unit 610, input device 612, and UI navigation device 614 may be a touchscreen display. Machine 600 may additionally include a storage device (e.g., a drive unit) 608, a signal generating device 618 (e.g., a speaker), a network interface device 620, and one or more sensors 616, such as a Global Positioning System (GPS) sensor, a compass, an accelerometer, or other sensors. Machine 600 may include an output controller 628, for example, serial (e.g., Universal Serial Bus (USB), parallel, or other wired or wireless (e.g., infrared (IR), near field communication (NFC), etc.) connection to communicate with or control one or more peripheral devices (e.g., a printer, a card reader, etc.).

[0083] The registers of processor 602, main memory 604, static memory 606, or mass storage device 608 may be or contain machine-readable media 622 on which one or more sets of data structures or instructions 624 (e.g., software) are stored, embodying or being utilized by any one or more of the techniques or functions described herein. Instructions 624 may also reside wholly or at least partially within any of the registers of processor 602, main memory 604, static memory 606, or mass storage device 608 during execution by machine 600. In an example, one or any combination of hardware processor 602, main memory 604, static memory 606, or mass storage device 608 may constitute machine-readable media 622. While machine-readable media 622 is described as a single medium, the term "machine-readable media" may include a single medium or multiple media (e.g., a centralized or distributed database, or associated caches and servers) configured to store one or more instructions 624.

[0084] The term "machine-readable media" can include any medium capable of storing, encoding, or carrying instructions executable by machine 600 and causing machine 600 to perform any or more of the technologies disclosed herein, or any medium capable of storing, encoding, or carrying data structures used by or associated with such instructions. Examples of non-limiting machine-readable media can include solid-state memory, optical media, magnetic media, and signals (e.g., radio frequency signals, other photon-based signals, sound signals, etc.). In examples, non-transitory machine-readable media includes machine-readable media having a plurality of particles having invariant (e.g., rest) mass and thus being a component of matter. Therefore, non-transitory machine-readable media is machine-readable media that does not contain transiently propagating signals. Specific examples of non-transitory machine-readable media can include: non-volatile memory, such as semiconductor memory devices (e.g., electrically programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM)) and flash memory devices; magnetic disks, such as internal hard disks and removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks.

[0085] In an example, information stored or otherwise provided on machine-readable medium 622 may represent instructions 624, such as instructions 624 themselves or a format from which instructions 624 can be derived. This format from which instructions 624 can be derived may include source code, encoded instructions (e.g., in compressed or encrypted form), encapsulated instructions (e.g., split into multiple encapsulations), etc. Information representing instructions 624 on machine-readable medium 622 may be processed by processing circuitry into instructions to perform any of the operations discussed herein. For example, deriving instructions 624 from information (e.g., processed by processing circuitry) may include: compiling (e.g., from source code, object code, etc.), interpreting, loading, organizing (e.g., dynamically or statically linking), encoding, decoding, encrypting, decrypting, encapsulating, decapsulating, or otherwise manipulating information into instructions 624.

[0086] In an example, the derivation of instruction 624 may involve (e.g., via processing circuitry) assembling, compiling, or decompiling information to create instruction 624 from some intermediate or preprocessed format provided by machine-readable media 622. When information is provided in multiple parts, the information may be combined, decapsulated, and modified to create instruction 624. For example, information may be contained in multiple compressed source code packages (or object code, or binary executable code, etc.) on one or more remote servers. The source code packages may be encrypted when transmitted over a network and, if necessary, decrypted, decompressed, assembled (e.g., linked), and compiled or decompiled at the local machine (e.g., into a stand-alone executable library, etc.) and executed by the local machine.

[0087] Instruction 624 may further utilize any of a number of transport protocols (e.g., Frame Relay, Internet Protocol (IP), Transmission Control Protocol (TCP), User Datagram Protocol (UDP), Hypertext Transfer Protocol (HTTP), etc.) to transmit or receive on the communication network 626 via the network interface device 620 using a transmission medium. Example communication networks may include local area networks (LANs), wide area networks (WANs), packet data networks (e.g., the Internet), mobile phone networks (e.g., cellular networks), conventional telephone (POTS) networks, and wireless data networks (e.g., the IEEE 802.11 standard series known as Wi-Fi®, the IEEE 802.16 standard series known as WiMax®), the IEEE 802.15.4 standard series, peer-to-peer (P2P) networks, etc. In examples, network interface device 620 may include one or more physical jacks (e.g., Ethernet, coaxial, or telephone jacks) or one or more antennas for connection to the communication network 626. In an example, network interface device 620 may include multiple antennas to wirelessly communicate using at least one of single-input multiple-output (SIMO), multiple-input multiple-output (MIMO), or multiple-input single-output (MISO) technologies. The term "transmitting medium" should be considered as encompassing any intangible medium capable of storing, encoding, or carrying instructions for execution by machine 600, and containing digital or analog communication signals or other intangible media used to facilitate communication of this software. The transmitting medium is a machine-readable medium. For better illustration of the methods and apparatus described herein, a set of non-limiting examples is set forth below as numbered examples.

[0088] Example 1 is a device comprising: a bucket processor including: temporary memory; and thread scheduling circuitry; wherein the bucket processor is configured to perform operations using the thread scheduling circuitry, the operations comprising: scheduling a current thread for placement in a pipeline for the bucket processor on a clock cycle, the bucket processor scheduling threads on each clock cycle; storing the current thread in the temporary memory; detecting that no thread is available on a clock cycle following the cycle in which the current thread was scheduled; and, in response to detecting that no thread is available on a subsequent clock cycle, repeatedly scheduling the current thread based on the contents of the temporary memory.

[0089] In Example 2, the subject of Example 1 includes a bucket processor that combines scheduling the current thread into the pipeline with instructions from the current thread to propagate a valid loop signal to the pipeline so that the pipeline changes state.

[0090] In Example 3, the subject matter of Examples 1 and 2 includes temporary memory including registers.

[0091] In Example 4, the subject of Example 3 includes the fact that registers are stored in the local random access memory of the bucket processor.

[0092] In Example 5, the topics of Examples 1 through 4 include instructions for repeatedly scheduling the current thread.

[0093] In Example 6, the subject of Examples 1 through 5 includes a barrel processor that propagates invalid loop signals to the pipeline so that the pipeline remains in a state.

[0094] In Example 7, the subject of Examples 1 through 6 includes a bucket processor that will: determine which threads are available for scheduling; schedule instructions for which threads are available for scheduling into the pipeline; store which threads are available for scheduling into temporary memory; and propagate valid loop signals to the pipeline based on the instructions of which threads are available for scheduling, so that the pipeline changes state.

[0095] In Example 8, the subject of Examples 1 through 7 includes a thread scheduling circuit that rewrites a thread to temporary memory in response to detecting that no thread is available on a subsequent clock cycle.

[0096] In Example 9, the subject matter of Examples 1 through 8 includes thread scheduling circuitry integrated into a bucket processor, which is contained in a programmable atomic unit, and the programmable atomic unit is contained in a memory controller.

[0097] In Example 10, the subject of Example 9 includes the memory controller being a chiplet in a chiplet system.

[0098] Example 11 is a method comprising: scheduling a current thread for placement in a pipeline for the bucket processor on a clock cycle at a bucket processor containing thread scheduling circuitry, the bucket processor scheduling threads on each clock cycle; storing the current thread in temporary memory; detecting that no thread is available on a clock cycle following the cycle in which the current thread was scheduled; and, in response to detecting that no thread is available on a subsequent clock cycle, repeatedly scheduling the current thread based on the contents of the temporary memory.

[0099] In Example 12, the subject of Example 11 includes combining scheduling the current thread into the pipeline with propagating a valid loop signal to the pipeline based on the current thread's instructions, causing the pipeline to change state.

[0100] In Example 13, the subject of Examples 11 to 12 includes temporary memory including registers.

[0101] In Example 14, the subject of Example 13 includes a register stored in the local random access memory of the bucket processor.

[0102] In Example 15, the topics of Examples 11 to 14 include instructions for repeatedly scheduling the current thread.

[0103] In Example 16, the subject of Examples 11 through 15 includes propagating invalid loop signals to a pipeline so that the pipeline remains in a state.

[0104] In Example 17, the topics of Examples 11 through 16 include: determining which threads are available for scheduling; scheduling instructions from which threads are scheduled to be placed in the pipeline; storing which threads are stored in temporary memory; and propagating valid loop signals to the pipeline based on the instructions from which threads are scheduled to cause the pipeline to change state.

[0105] In Example 18, the topics of Examples 11 through 17 include rewriting threads to temporary memory in response to detecting that no threads are available on subsequent clock cycles.

[0106] In Example 19, the subject matter of Examples 11 to 18 includes thread scheduling circuitry integrated in a bucket processor, which is contained in a programmable atom unit, and the programmable atom unit is contained in a memory controller.

[0107] In Example 20, the subject of Example 19 includes a memory controller that is a chiplet in a chiplet system.

[0108] Example 21 is a machine-readable medium containing instructions that, when executed by a bucket processor having thread scheduling circuitry, cause the bucket processor to: schedule the current thread for placement in a pipeline for the bucket processor on a clock cycle, the bucket processor scheduling threads on each clock cycle; store the current thread in temporary memory; detect that no thread is available on a clock cycle following the cycle in which the current thread was scheduled; and, in response to detecting that no thread is available on a subsequent clock cycle, repeat the scheduling of the current thread based on the contents of the temporary memory.

[0109] In Example 22, the subject of Example 21 includes instructions for combining with scheduling the current thread into the pipeline, propagating a valid loop signal to the pipeline based on the instructions of the current thread so that the pipeline changes state.

[0110] In Example 23, the subjects of Examples 21 and 22 include temporary memory including registers.

[0111] In Example 24, the subject of Example 23 includes the fact that registers are stored in the local random access memory of the bucket processor.

[0112] In Example 25, the topics of Examples 21 to 24 contain instructions that repeatedly schedule the current thread.

[0113] In Example 26, the subject of Examples 21 to 25 contains instructions for propagating invalid loop signals to the pipeline so that the pipeline remains in a state.

[0114] In Example 27, the subject matter of Examples 21 to 26 contains instructions for: determining which threads are available for scheduling; scheduling instructions for which threads are available for scheduling into the pipeline; storing which threads are available for scheduling into the pipeline; and propagating valid loop signals to the pipeline based on the instructions for which threads are available for scheduling, so that the pipeline changes state.

[0115] In Example 28, the topics of Examples 21 through 27 contain instructions for rewriting threads to temporary memory in response to detecting that no threads are available on subsequent clock cycles.

[0116] In Example 29, the subject of Examples 21 to 28 includes a thread scheduling circuit integrated in a bucket processor, which is contained in a programmable atom unit, and the programmable atom unit is contained in a memory controller.

[0117] In Example 30, the subject of Example 29 includes a memory controller that is a chiplet in a chiplet system.

[0118] Example 31 is an apparatus for resetting a thread to a reserved state in a bucket processor, the apparatus comprising: means for scheduling a current thread to be placed into a pipeline for the bucket processor on a clock cycle at a bucket processor including thread scheduling circuitry, the bucket processor scheduling threads on each clock cycle; means for storing the current thread in temporary memory; means for detecting that no thread is available on a clock cycle following the cycle in which the current thread was scheduled; and means for repeatedly scheduling the current thread based on the contents of the temporary memory in response to detecting that no thread is available on a subsequent clock cycle.

[0119] In Example 32, the subject of Example 31 includes components for combining with scheduling the current thread to be placed in the pipeline, and propagating a valid loop signal to the pipeline based on the instructions of the current thread so as to cause the pipeline to change state.

[0120] In Example 33, the subjects of Examples 31 to 32 include temporary memory including registers.

[0121] In Example 34, the subject of Example 33 includes the fact that registers are stored in the local random access memory of the bucket processor.

[0122] In Example 35, the topics of Examples 31 to 34 contain instructions for repeatedly scheduling the current thread.

[0123] In Example 36, the subject matter of Examples 31 to 35 includes components for propagating invalid loop signals to the pipeline so that the pipeline maintains its state.

[0124] In Example 37, the subject matter of Examples 31 to 36 includes: a component for determining which different threads are available for scheduling; a component for scheduling instructions of different threads to be placed in the pipeline; a component for storing different threads in temporary memory; and a component for propagating a valid loop signal to the pipeline based on the instructions of different threads so that the pipeline changes state.

[0125] In Example 38, the subjects of Examples 31 to 37 contain components for rewriting threads to temporary memory in response to detecting that no threads are available on subsequent clock cycles.

[0126] In Example 39, the subject matter of Examples 31 to 38 includes thread scheduling circuitry integrated in a bucket processor, the bucket processor being contained in a programmable atom unit, and the programmable atom unit being contained in a memory controller.

[0127] In Example 40, the subject of Example 39 includes a memory controller that is a chiplet in a chiplet system.

[0128] Example 41 is a device comprising: a memory controller chiplet in a chiplet system, the memory controller chiplet including: programmable atomic units, the programmable atomic units including: a barrel processor, the barrel processor including: temporary memory; and thread scheduling circuitry; wherein the barrel processor is configured to perform operations using the thread scheduling circuitry, the operations comprising: scheduling a current thread for placement in a pipeline for the barrel processor on a clock cycle, the barrel processor scheduling threads on each clock cycle; storing the current thread in temporary memory; detecting that no thread is available on a clock cycle following the cycle in which the current thread was scheduled; and, in response to detecting that no thread is available on a subsequent clock cycle, repeatedly scheduling the current thread based on the contents of the temporary memory.

[0129] In Example 42, the subject of Example 41 includes a barrel processor configured to perform operations using a thread scheduling circuit, including scheduling the current thread to be placed in a pipeline, and propagating a valid loop signal to the pipeline based on the instructions of the current thread to cause the pipeline to change state.

[0130] In Example 43, the subject of Examples 41 to 42 includes temporary memory including registers.

[0131] In Example 44, the subject of Example 43 includes the fact that registers are stored in the local random access memory of the bucket processor.

[0132] In Example 45, the topics of Examples 41 to 44 contain instructions for repeatedly scheduling the current thread.

[0133] In Example 46, the subject of Examples 41 to 45 includes a barrel processor configured to perform operations using a thread scheduling circuit, including propagating an invalid loop signal to the pipeline so that the pipeline remains in a state.

[0134] In Example 47, the subject of Examples 41 to 46 includes a barrel processor configured to perform operations using a thread scheduling circuit, including: determining which threads are available for scheduling; scheduling instructions for which threads are scheduled to be placed in a pipeline; storing which threads are stored in temporary memory; and propagating a valid loop signal to the pipeline based on the instructions of which threads, causing the pipeline to change state.

[0135] In Example 48, the subject of Examples 41 through 47 includes a barrel processor configured to perform operations using a thread scheduling circuit, including rewriting threads to temporary memory in response to detecting that no threads are available on subsequent clock cycles.

[0136] Example 49 is at least one machine-readable medium containing instructions that, when executed by processing circuitry, cause the processing circuitry to perform an operation to implement any one of Examples 1 to 48.

[0137] Example 50 is a device that includes components for implementing any one of Examples 1 to 48.

[0138] Example 51 is a system for implementing any of Examples 1 through 48.

[0139] Example 52 is a method used to implement any one of Examples 1 through 48.

[0140] The above detailed description includes reference to the accompanying drawings, which form a part of the detailed description. The drawings illustrate, by means of illustration, specific embodiments in which the invention may be practiced. These embodiments are also referred to herein as “examples.” Such examples may include elements other than those shown or described. However, the inventors also contemplate that only examples of those elements shown or described are provided herein. Furthermore, the inventors also contemplate examples (or one or more aspects thereof) of any combination or arrangement of those elements shown or described with respect to a particular example (or one or more aspects thereof) or with respect to other examples (or one or more aspects thereof) shown or described herein.

[0141] In this document, as is common in patent documents, the term “a” is used to include one or more, regardless of any other examples or uses of “at least one” or “one or more.” In this document, unless otherwise indicated, the term “or” is used to refer to a non-exclusive “or,” such that “A or B” can include “A but not B,” “B but not A,” and “A and B.” In the appended claims, the terms “comprising” and “in which” are used as common equivalents to the corresponding terms “including” and “wherein.” Furthermore, in the appended claims, the terms “comprising” and “including” are open-ended, meaning that a system, apparatus, article, or process that includes elements other than those listed after this term in the claims is still considered to be within the scope of the claims. Additionally, in the appended claims, the terms “first,” “second,” and “third,” etc., are used merely as labels and are not intended to impose numerical requirements on their objects.

[0142] The above description is illustrative and not restrictive. For example, the examples described above (or one or more aspects thereof) can be used in combination with each other. Other embodiments may be used by those skilled in the art upon review of the above description. This abstract is provided under the understanding that it will not be used to interpret or limit the scope or meaning of the claims. Similarly, in the above detailed description, various features may be grouped together to simplify this disclosure. This should not be construed as expecting that any unclaimed disclosed feature is necessary for any claim. In fact, the subject matter of the invention may contain fewer features than all of the particular disclosed embodiments. Therefore, the appended claims are hereby incorporated into the detailed description, wherein each claim exists independently as a separate embodiment, and these embodiments are contemplated to be combined or arranged in various ways. The scope of the invention should be determined by reference to the appended claims and the full scope of the equivalents granted by those claims.

Claims

1. An apparatus comprising: A barrel-type processor, comprising: Temporary storage; and Thread scheduling circuit; The bucket processor is configured to perform operations using the thread scheduling circuit, the operations including the following: The current thread is scheduled to be placed into the pipeline for the bucket processor on each clock cycle; The current thread is stored in the temporary storage. The system detects that there are no threads available for scheduling in the thread scheduling queue on a clock cycle following the clock cycle that scheduled the current thread; and In response to the detection that no thread is available in subsequent clock cycles, the current thread is repeatedly scheduled based on the current thread stored in the temporary memory.

2. The device of claim 1, wherein the bucket processor is combined with scheduling the current thread to be placed in the pipeline, and propagating a valid loop signal to the pipeline based on the instructions of the current thread to cause the pipeline to change state.

3. The device of claim 1, wherein the temporary memory includes a register.

4. The device of claim 3, wherein the register is stored in the local random access memory of the barrel processor.

5. The device of claim 1, wherein repeatedly scheduling the current thread includes an instruction to repeatedly schedule the current thread.

6. The device of claim 1, wherein the barrel processor propagates an invalid loop signal to the pipeline so that the pipeline remains in a state.

7. The device according to claim 1, wherein the barrel processor will: Determine which threads are available for scheduling; The instructions from the different threads are scheduled to be placed into the pipeline; The different threads are stored in the temporary memory; and The instructions based on the different threads propagate a valid loop signal to the pipeline, causing the pipeline to change its state.

8. The device of claim 1, wherein the thread scheduling circuit rewrites the thread to the temporary memory in response to detecting that no thread is available on the subsequent clock cycle.

9. The device of claim 1, wherein the thread scheduling circuit is integrated in the bucket processor, the bucket processor is contained in a programmable atomic unit, and the programmable atomic unit is contained in a memory controller.

10. The device of claim 9, wherein the memory controller is a chiplet in a chiplet system.

11. A method comprising: At the bucket processor, which includes thread scheduling circuitry, the current thread is scheduled to be placed into the pipeline for the bucket processor on a clock cycle, and the bucket processor schedules the thread on each clock cycle. Store the current thread in a temporary memory; If, in the clock cycle following the clock cycle that schedules the current thread, there are no threads available for scheduling in the thread scheduling queue; as well as In response to the detection that no thread is available in subsequent clock cycles, the current thread is repeatedly scheduled based on the current thread stored in the temporary memory.

12. The method of claim 11, further comprising, in conjunction with scheduling the current thread to be placed in the pipeline, propagating a valid loop signal to the pipeline based on an instruction from the current thread to cause the pipeline to change state.

13. The method of claim 11, wherein the temporary memory includes a register.

14. The method of claim 13, wherein the register is stored in the local random access memory of the barrel processor.

15. The method of claim 11, wherein repeatedly scheduling the current thread includes an instruction to repeatedly schedule the current thread.

16. The method of claim 11, further comprising propagating an invalid loop signal to the pipeline to maintain the pipeline in a state.

17. The method of claim 11, further comprising: Determine which threads are available for scheduling; The instructions from the different threads are scheduled to be placed into the pipeline; The different threads are stored in the temporary memory; as well as The instructions based on the different threads propagate a valid loop signal to the pipeline, causing the pipeline to change its state.

18. The method of claim 11, further comprising rewriting the thread to the temporary memory in response to detecting that no thread is available on the subsequent clock cycle.

19. The method of claim 11, wherein the thread scheduling circuit is integrated in the bucket processor, the bucket processor is contained in a programmable atomic unit, and the programmable atomic unit is contained in a memory controller.

20. The method of claim 19, wherein the memory controller is a chiplet in a chiplet system.

21. An apparatus comprising: A memory controller chip in a chiplet system, the memory controller chiplet comprising: Programmable atomic units, the programmable atomic units comprising: A barrel-type processor, the barrel-type processor comprising: Temporary storage; and Thread scheduling circuit; The bucket processor is configured to perform operations using the thread scheduling circuit, the operations including: The current thread is scheduled to be placed into a pipeline for a bucket processor on each clock cycle. The current thread is stored in the temporary storage. The system detects that there are no threads available for scheduling in the thread scheduling queue on a clock cycle following the clock cycle that scheduled the current thread; and In response to the detection that no thread is available in subsequent clock cycles, the current thread is repeatedly scheduled based on the current thread stored in the temporary memory.

22. The device of claim 21, wherein the barrel processor is configured to perform operations using the thread scheduling circuitry, comprising coordinating the scheduling of the current thread to be placed in the pipeline, and propagating a valid loop signal to the pipeline based on instructions from the current thread to cause the pipeline to change state.

23. The device of claim 21, wherein the temporary memory includes a register.

24. The device of claim 23, wherein the register is stored in the local random access memory of the barrel processor.

25. The device of claim 21, wherein repeatedly scheduling the current thread includes an instruction to repeatedly schedule the current thread.

Images