Thread state monitoring in systems with multithreaded self-scheduling processors

Abstract

The present invention discloses representative apparatus, method, and system embodiments for a self-scheduling processor that also provides additional functionality. Representative embodiments include a self-scheduling processor that includes: a processor core that is used to execute received instructions; and a core control circuit that is used to automatically schedule instructions for execution by the processor core in response to a received work descriptor data packet. In another embodiment, the core control circuit is also used to schedule shred creation instructions for execution by the processor core, reserve a predetermined amount of memory space in a thread control memory to store return variable parameters, and generate one or more work descriptor data packets to another processor or hybrid thread organization circuit for executing corresponding multiple execution threads. The present invention also discloses event handling, data path management, system calls, memory requests, and other new instructions.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of and priority to U.S. Patent Application No. 16 / 399,769, filed on April 30, 2019, entitled “Thread State Monitoring in a System Having a Multi-Threaded, Self-Scheduling Processor,” by inventor Tony M. Brewer, which is a non-provisional application and claims the benefit of and priority to U.S. Provisional Patent Application No. 62 / 667,780, filed on May 7, 2018, entitled “Thread State Monitoring in a System Having a Multi-Threaded, Self-Scheduling Processor,” by inventor Tony M. Brewer, which are commonly assigned hereto and are all hereby incorporated herein by reference in their entirety as if their entirety were set forth herein with the same full force and effect. Technical Field

[0003] The present invention relates generally to configurable computing circuitry, and more particularly to a heterogeneous computing system including a self-scheduling processor, a configurable computing circuitry having an embedded interconnect network, dynamic reconfiguration, and dynamic control of energy or power consumption. Background Art

[0004] Most existing computing systems are significantly limited in computing processing capabilities in terms of computing speed, energy (or power) consumption, and associated heat dissipation. For example, due to the growing demand for advanced computing technologies, such as to accommodate artificial intelligence and other important computing applications, existing computing solutions have become increasingly inadequate.

[0005] Therefore, there is a need for a computing architecture that can provide high performance using sparse data sets, which involves limited or no data reuse, which typically results in poor cache hit rates. Such a computing architecture should be tolerant of latency to memory and allow the number of instructions executed per clock to remain high.

[0006] There is also a need for compute architectures that can provide high-performance, energy-efficient solutions for compute-intensive cores, for example, to compute Fast Fourier Transforms (FFTs) and Finite Impulse Response (FIR) filters for sensing, communication, and analytics applications such as, but not limited to, synthetic aperture radar, 5G base stations, and graph analytics applications such as graph clustering using spectroscopy techniques, machine learning, 5G networking algorithms, and large die codes.

[0007] There is also a current need for a processor architecture that is capable of massively parallel processing and further interacting with and controlling a configurable computing architecture to perform any of these various applications. Summary of the invention

[0008] As discussed in more detail below, representative devices, systems, and methods provide a computing architecture capable of providing high-performance and energy-efficient solutions for compute-intensive cores, for example, to compute fast Fourier transforms (FFTs) and finite impulse response (FIR) filters for sensing, communication, and analytical applications, such as synthetic aperture radar, 5G base stations, and graph analytics applications, such as graph clustering using spectral techniques, machine learning, 5G networking algorithms, and large mold codes, but not limited to these.

[0009] As mentioned above, sparse data sets typically result in poor cache hit rates. Representative devices, systems, and methods provide a computing architecture that allows some threads to wait for responses from memory while other threads continue to execute instructions. This style of computing tolerates latency to memory and allows instructions executed per clock to continue to be high.

[0010] As also discussed in more detail below, representative devices, systems, and methods provide a processor architecture that is capable of self-scheduling, massively parallel processing, and further interacts with and controls a configurable computing architecture to perform any of these various applications.

[0011] A self-scheduling processor is disclosed. In a representative embodiment, the processor includes: a processor core for executing received instructions; and a core control circuit coupled to the processor core, the core control circuit for automatically scheduling instructions for execution by the processor core in response to a received work descriptor data packet. In another representative embodiment, the processor includes: a processor core for executing received instructions; and a core control circuit coupled to the processor core, the core control circuit for automatically scheduling instructions for execution by the processor core in response to a received event data packet.

[0012] Also disclosed is a multi-threaded self-scheduling processor that can create threads on local or remote computing elements. In a representative embodiment, the processor includes: a processor core that is used to execute a shred creation instruction; and a core control circuit that is coupled to the processor core, the core control circuit being used to automatically schedule the shred creation instruction for execution by the processor core and generate one or more work descriptor packets to another processor or a hybrid thread organization circuit for execution by the corresponding multiple execution threads. In another representative embodiment, the processor includes: a processor core that is used to execute a shred creation instruction; and a core control circuit that is coupled to the processor core, the core control circuit being used to schedule the shred creation instruction for execution by the processor core to reserve a predetermined amount of memory space in a thread control memory to store return convenience parameters, and generate one or more work descriptor packets to another processor or a hybrid thread organization circuit for execution by the corresponding multiple execution threads.

[0013] In another representative embodiment, a processor includes: a core control circuit, which includes: an interconnect network interface; a thread control memory, which is coupled to the interconnect network interface; an execution queue, which is coupled to the thread control memory; a control logic and thread selection circuit, which is coupled to the execution queue and the thread control memory; and an instruction cache, which is coupled to the control logic and the thread selection circuit; and the processor further includes a processor core, which is coupled to the instruction cache of the core control circuit.

[0014] In another representative embodiment, a processor includes: a core control circuit including: an interconnect network interface; a thread control memory coupled to the interconnect network interface; a network response memory; an execution queue coupled to the thread control memory; a control logic and thread selection circuit coupled to the execution queue and the thread control memory; an instruction cache coupled to the control logic and thread selection circuit; and a command queue; the processor further includes a processor core coupled to the instruction cache and the command queue of the core control circuit.

[0015] In another representative embodiment, a processor includes: a processor core and a core control circuit coupled to the processor core, wherein the core control circuit includes: an interconnect network interface, which can be coupled to the interconnect network to receive a work descriptor data packet, decode the received work descriptor data packet into an execution thread having an initial program counter and any received variable parameters; an execution queue, which is coupled to a thread control memory; and control logic and thread selection circuitry, which is coupled to the execution queue, the control logic and thread selection circuitry being used to assign available thread identifiers to execution threads, automatically place thread identifiers in the execution queue, and periodically select thread identifiers to execute execution threads.

[0016] In another representative embodiment, a processor includes: a processor core and a core control circuit coupled to the processor core, wherein the core control circuit includes: an interconnect network interface, which can be coupled to the interconnect network to receive a work descriptor data packet, decode the received work descriptor data packet into an execution thread having an initial program counter and any received variable parameters; an execution queue, which is coupled to a thread control memory; and control logic and thread selection circuitry, which is coupled to the execution queue, the control logic and thread selection circuitry being used to assign available thread identifiers to execution threads, automatically place thread identifiers in the execution queue, and periodically select thread identifiers for the processor core to execute instructions of the execution thread.

[0017] In another representative embodiment, a processor includes: a processor core and a core control circuit coupled to the processor core, wherein the core control circuit includes: an execution queue, which is coupled to a thread control memory; and control logic and thread selection circuitry, which are coupled to the execution queue, the control logic and thread selection circuitry are used to assign available thread identifiers to execution threads, automatically place thread identifiers in the execution queue, and periodically select thread identifiers for the processor core to execute instructions of the execution thread.

[0018] In another representative embodiment, a processor includes: a processor core and a core control circuit coupled to the processor core, wherein the core control circuit includes: a thread control memory, which includes a plurality of registers, the plurality of registers including a thread identifier pool register storing a plurality of thread identifiers, a program counter register storing a received program counter, a data cache, and a general register storing a received variable parameter; an execution queue, which is coupled to the thread control memory; and a control logic and a thread selection circuit, which are coupled to the execution queue, the control logic and the thread selection circuit being used to assign available thread identifiers to execution threads, automatically place thread identifiers in the execution queue, and periodically select thread identifiers for the processor core to execute instructions of the execution thread, the processor core using data stored in the data cache or the general registers.

[0019] In another representative embodiment, a processor includes: a processor core and a core control circuit coupled to the processor core, wherein the core control circuit includes: a thread control memory, which includes a plurality of registers, the plurality of registers including a thread identifier pool register storing a plurality of thread identifiers, a program counter register storing a received program counter, and a thread state register storing an effective state or a suspended state of each of the plurality of thread identifiers; an execution queue, which is coupled to the thread control memory; and a control logic and a thread selection circuit, which are coupled to the execution queue, the control logic and the thread selection circuit being used to assign an available thread identifier to an execution thread, automatically placing the thread identifier in the execution queue when the thread identifier has an effective state, and periodically selecting the thread identifier for the processor core to execute instructions of the execution thread for a duration during which the effective state remains unchanged until the execution thread is completed.

[0020] In another representative embodiment, a processor includes: a processor core and a core control circuit coupled to the processor core, wherein the core control circuit includes: a thread control memory, which includes a plurality of registers, the plurality of registers including a thread identifier pool register storing a plurality of thread identifiers, a program counter register storing a received program counter, and a thread state register storing an effective state or a pause state of each of the plurality of thread identifiers; an execution queue, which is coupled to the thread control memory; and a control logic and a thread selection circuit, which are coupled to the execution queue, the control logic and the thread selection circuit being used to assign an available thread identifier to an execution thread, automatically place the thread identifier in the execution queue when the thread identifier has an effective state, periodically select the thread identifier for the processor core to execute instructions of the execution thread during a duration during which the effective state remains unchanged, and suspend thread execution by not returning the thread identifier to the execution queue when the thread identifier has a pause state.

[0021] In another representative embodiment, a processor includes: a processor core and a core control circuit coupled to the processor core, wherein the core control circuit includes: a thread control memory, which includes a plurality of registers, the plurality of registers including a thread identifier pool register storing a plurality of thread identifiers, a thread status register, a program counter register storing a received program counter, a data cache, and a general register storing received variable parameters; an execution queue, which is coupled to the thread control memory; and a control logic and a thread selection circuit, which are coupled to the execution queue, the control logic and the thread selection circuit being used to assign an available thread identifier to an execution thread, automatically place a thread identifier in the execution queue, and periodically select a thread identifier for the processor core to execute instructions of the execution thread.

[0022] In another representative embodiment, a processor includes: a processor core for executing a plurality of instructions; and a core control circuit coupled to the processor core, wherein the core control circuit includes: an interconnect network interface that can be coupled to the interconnect network to receive a work descriptor packet, decode the received work descriptor packet into an execution thread having an initial program count and any received variable parameters; a thread control memory that is coupled to the interconnect network interface and includes a plurality of registers, the plurality of registers including a thread identifier pool register storing a plurality of thread identifiers, a thread state register, a program count register storing a received program count, a data cache, and a general register storing received variable parameters; an execution queue that is coupled to the thread control memory; a control logic and thread selection circuit that is coupled to the execution queue and the thread control memory, the control logic and thread selection circuit being used to assign an available thread identifier to an execution thread, place a thread identifier in the execution queue, select a thread identifier for execution, use the thread identifier as an index to access the thread control memory to select the initial program count of the execution thread; and an instruction cache that is coupled to the processor core and the control logic and thread selection circuit to receive the initial program count and provide the processor core with a corresponding instruction from a plurality of instructions for execution.

[0023] In another representative embodiment, a processor includes: a core control circuit, which includes: an interconnect network interface, which can be coupled to the interconnect network to receive a work descriptor data packet, decode the received work descriptor data packet into an execution thread having an initial program count and any received variable parameters; a thread control memory, which is coupled to the interconnect network interface and includes a plurality of registers, the plurality of registers including a thread identifier pool register storing a plurality of thread identifiers, a thread state register, a program count register storing a received program count, a data cache, and a general register storing received variable parameters; an execution queue, which is coupled to the thread control memory; a control logic and a thread selection circuit, which are coupled to the execution queue and the thread control memory, the control logic and the thread selection circuit being used to assign available thread identifiers to execution threads, automatically place thread identifiers in the execution queue, periodically select thread identifiers for execution, use the thread identifier as an index to access the thread control memory to select the initial program count of the execution thread; and an instruction cache, which is coupled to the control logic and the thread selection circuit to receive the initial program count and provide corresponding instructions for execution; the processor also includes a processor core, which is coupled to the instruction cache of the core control circuit, the processor core being used to execute the corresponding instructions.

[0024] In another representative embodiment, a processor includes: a core control circuit, which includes: an interconnect network interface, which can be coupled to the interconnect network to receive a work descriptor data packet, decode the received work descriptor data packet into an execution thread having an initial program count and any received variable parameters; a thread control memory, which is coupled to the interconnect network interface and includes a plurality of registers, the plurality of registers including a thread identifier pool register storing a plurality of thread identifiers, a thread state register, a program counter register storing a received program count, and a general register storing received variable parameters; an execution queue, which is coupled to the thread control memory; a control logic and a thread selection circuit, which are coupled to the execution queue and the thread control memory, the control logic and the thread selection circuit being used to assign available thread identifiers to execution threads, place thread identifiers in the execution queue, select thread identifiers for execution, use the thread identifier as an index to access the thread control memory to select the initial program count of the execution thread; an instruction cache, which is coupled to the control logic and the thread selection circuit to receive the initial program count and provide corresponding instructions for execution; and a command queue; the processor also includes a processor core, which is coupled to the instruction cache and the command queue of the core control circuit, the processor core being used to execute the corresponding instructions.

[0025] In another representative embodiment, a processor includes: a core control circuit coupled to an interconnect network interface and including: an interconnect network interface that can be coupled to the interconnect network to receive a work descriptor packet, decode the received work descriptor packet into an execution thread having an initial program count and any received variable parameters; a thread control memory coupled to the interconnect network interface and including a plurality of registers, the plurality of registers including a thread identifier pool register storing a plurality of thread identifiers, a thread state register, a program count register storing a received program count, and a general register storing received variable parameters; an execution queue coupled to the thread control memory; a control logic and a thread selection circuit coupled to the execution queue and the thread control memory, the control logic and the thread selection circuit being used to assign an available thread identifier to an execution thread, place a thread identifier in the execution queue, select a thread identifier for execution, use the thread identifier as an index to access the thread control memory to select the initial program count of the execution thread; and an instruction cache coupled to the control logic and the thread selection circuit to receive the initial program count and provide corresponding instructions for execution; the processor additionally includes a processor core coupled to the instruction cache of the core control circuit, the processor core being used to execute the corresponding instructions.

[0026] In another representative embodiment, a processor includes: a core control circuit including: an interconnect network interface that can be coupled to an interconnect network to receive a call work descriptor data packet, decode the received work descriptor data packet into an execution thread with an initial program counter and any received variable parameters, and encode the work descriptor packet for transmission to other processing elements; a thread control memory that is coupled to the interconnect network interface and includes a plurality of registers, the plurality of registers including a thread identifier pool register storing a plurality of thread identifiers, a thread status register, a program counter register storing a received program counter, and a general register storing received variable parameters; an execution queue that is coupled to the thread control memory; a network response memory that is coupled to the interconnect network interface; A network interface; a control logic and thread selection circuit coupled to an execution queue, a thread control memory, and an instruction cache, the control logic and thread selection circuit being used to assign an available thread identifier to an execution thread, place the thread identifier in the execution queue, select the thread identifier for execution, and use the thread identifier as an index to access the thread control memory to select an initial program count of the execution thread; an instruction cache coupled to the control logic and thread selection circuit to receive the initial program count and provide corresponding instructions for execution; and a command queue storing one or more commands to generate one or more work descriptor packets; the processor further includes a processor core coupled to the instruction cache and command queue of the core control circuit, the processor core being used to execute the corresponding instructions.

[0027] For any of the various representative embodiments, the core control circuitry may further include: an interconnect network interface that may be coupled to the interconnect network, the interconnect network interface for receiving a work descriptor data packet to decode the received work descriptor data packet into an execution thread having an initial program counter and any received variable parameters. For any of the various representative embodiments, the interconnect network interface may further be used to receive an event data packet to decode the received event data packet into an event identifier and any received variable parameters.

[0028] For any of the various representative embodiments, the core control circuitry may further include: control logic and thread selection circuitry coupled to the interconnect network interface, the control logic and thread selection circuitry for assigning available thread identifiers to execution threads.

[0029] For any of the various representative embodiments, the core control circuitry may further include: a thread control memory having a plurality of registers, wherein the plurality of registers include one or more of the following in any selected combination: a thread identifier pool register storing a plurality of thread identifiers; a thread status register; a program counter register storing a received initial program count; a general register storing received variable parameters; a pending fiber return count register; a return variable parameter buffer or register; a return variable parameter linked list register; a custom atomic transaction identifier register; an event status register; an event mask register; and a data cache.

[0030] For any of the various representative embodiments, the interconnect network interface may be further operable to store the execution thread with the initial program count and any received variable parameters in the thread control memory using the thread identifier as an index into the thread control memory.

[0031] For any of the various representative embodiments, the core control circuitry may further include control logic and thread selection circuitry coupled to the thread control memory and the interconnect network interface, the control logic and thread selection circuitry for assigning available thread identifiers to execution threads.

[0032] For any of the various representative embodiments, the core control circuitry may further include: an execution queue coupled to the thread control memory, the execution queue storing one or more thread identifiers.

[0033] For any of the various representative embodiments, the core control circuitry may further include: control logic and thread selection circuitry coupled to the execution queue, the interconnect network interface, and the thread control memory, the control logic and thread selection circuitry for assigning available thread identifiers to execution threads, placing thread identifiers in the execution queue, selecting thread identifiers for execution, and accessing the thread control memory using the thread identifier as an index to select an initial program count for the execution thread.

[0034] For any of the various representative embodiments, the core control circuitry may further include an instruction cache coupled to the control logic and the thread selection circuitry to receive the initial program count and provide corresponding instructions for execution.

[0035] In another representative embodiment, the processor may further include: a processor core coupled to an instruction cache of the core control circuit, the processor core being configured to execute corresponding instructions.

[0036] For any of the various representative embodiments, the core control circuitry may be further configured to assign an initial active state to an execution thread. For any of the various representative embodiments, the core control circuitry may be further configured to assign a pause state to an execution thread in response to the processor core executing a memory load instruction. For any of the various representative embodiments, the core control circuitry may be further configured to assign a pause state to an execution thread in response to the processor core executing a memory store instruction.

[0037] For any of the various representative embodiments, the core control circuitry may be further configured to terminate execution of the selected thread in response to execution of a return instruction by the processor core. For any of the various representative embodiments, the core control circuitry may be further configured to return a corresponding thread identifier of the selected thread to a thread identifier pool register in response to execution of the return instruction by the processor core. For any of the various representative embodiments, the core control circuitry may be further configured to clear a register of a thread control memory indexed by a corresponding thread identifier of the selected thread in response to execution of the return instruction by the processor core.

[0038] For any of the various representative embodiments, the interconnect network interface may be further operable to generate a return work descriptor packet in response to the processor core executing the return instruction.

[0039] For any of the various representative embodiments, the core control circuitry may further include: a network response memory. For any of the various representative embodiments, the network response memory may include one or more of the following, in any selected combination: a memory request register; a thread identifier and transaction identifier register; a request cache line index register; a byte register; and a general register index and type register.

[0040] For any of the various representative embodiments, the interconnect network interface may be used to generate a point-to-point event data message.For any of the various representative embodiments, the interconnect network interface may be used to generate a broadcast event data message.

[0041] For any of the various representative embodiments, the core control circuitry may be further used to respond to a received event data packet using an event mask stored in an event mask register. For any of the various representative embodiments, the core control circuitry may be further used to determine an event number corresponding to the received event data packet. For any of the various representative embodiments, the core control circuitry may be further used to change the state of a thread identifier from suspended to valid in response to the received event data packet to resume execution of the corresponding execution thread. For any of the various representative embodiments, the core control circuitry may be further used to change the state of a thread identifier from suspended to valid in response to the event number of the received event data packet to resume execution of the corresponding execution thread.

[0042] For any of the various representative embodiments, the control logic and thread selection circuitry may be further used to continuously select the next thread identifier from the execution queue for executing a single instruction of the corresponding execution thread. For any of the various representative embodiments, the control logic and thread selection circuitry may be further used to perform a cyclic selection of the next thread identifier from the execution queue among the plurality of thread identifiers, respectively for executing a single instruction of the corresponding execution thread. For any of the various representative embodiments, the control logic and thread selection circuitry may be further used to perform a cyclic selection of the next thread identifier from the execution queue among the plurality of thread identifiers, respectively for executing a single instruction of the corresponding execution thread, until the execution thread is completed. For any of the various representative embodiments, the control logic and thread selection circuitry may be further used to perform a bucket selection of the next thread identifier from the execution queue among the plurality of thread identifiers, respectively for executing a single instruction of the corresponding execution thread.

[0043] For any of the various representative embodiments, the control logic and thread selection circuitry may further be used to assign an active state or a pause state to a thread identifier. For any of the various representative embodiments, the control logic and thread selection circuitry may further be used to assign a priority state to a thread identifier. For any of the various representative embodiments, the control logic and thread selection circuitry may further be used to return the corresponding thread identifier to the execution queue with the assigned active state and the assigned priority after executing the corresponding instruction.

[0044] For any of the various representative embodiments, the core control circuitry may further include: a network command queue coupled to the processor core.

[0045] For any of the various representative embodiments, the interconnect network interface may include: an input queue; a packet decoder circuit coupled to the input queue, control logic and thread selection circuitry, and a thread control memory; an output queue; and a packet encoder circuit coupled to the output queue, a network response memory, and a network command queue.

[0046] For any of the various representative embodiments, the execution queue may further include: a first priority queue; and a second priority queue. For any of the various representative embodiments, the control logic and thread selection circuitry may further include: a thread selection control circuitry coupled to the execution queue, the thread selection control circuitry configured to select thread identifiers from the first priority queue at a first frequency and to select thread identifiers from the second priority queue at a second frequency, the second frequency being lower than the first frequency. For any of the various representative embodiments, the thread selection control circuitry may be configured to determine the second frequency as a skip count from the selection of the thread identifiers from the first priority queue.

[0047] For any of the various representative embodiments, the core control circuitry may further include: data path control circuitry for controlling access sizes on the first interconnect network. For any of the various representative embodiments, the core control circuitry may further include: data path control circuitry for increasing or decreasing memory load access sizes in response to a time average usage level. For any of the various representative embodiments, the core control circuitry may further include: data path control circuitry for increasing or decreasing memory storage access sizes in response to a time average usage level. For any of the various representative embodiments, the control logic and thread selection circuitry may further be used to increase the size of the memory load access request to a cache line boundary corresponding to the data cache.

[0048] For any of the various representative embodiments, the core control circuitry may further include: a system call circuitry for generating one or more system calls to the host processor. For any of the various representative embodiments, the system call circuitry may further include: a plurality of system call credit registers storing a predetermined credit count to accommodate a plurality of system calls in any predetermined time period.

[0049] For any of the various representative embodiments, the core control circuitry may be further operative to generate, in response to a request from a host processor, a command to cause a command queue of an interconnect network interface to copy and transfer all data corresponding to a selected thread identifier from a thread control memory to monitor a thread state.

[0050] For any of the various representative embodiments, the processor core can be used to execute the shred creation instruction to generate one or more commands to a command queue for an interconnect network interface to generate one or more call work descriptor packets to another processor core or to a hybrid thread fabric circuit. For any of the various representative embodiments, the core control circuit can be further used to reserve a predetermined amount of memory space in a general register or a return variable parameter register in response to the processor core executing the shred creation instruction.

[0051] For any of the various representative embodiments, in response to generating one or more call work descriptor packets to another processor core or to a hybrid thread fabric, the core control circuitry may be used to store a thread return count in a thread return register. For any of the various representative embodiments, in response to receiving a return data packet, the core control circuitry may be used to decrement the thread return count stored in the thread return register. For any of the various representative embodiments, in response to the thread return count in the thread return register being decremented to zero, the core control circuitry may be used to change the pause state of the corresponding thread identifier to an active state for subsequent execution of a thread return instruction to complete the created shred or thread.

[0052] For any of the various representative embodiments, the processor core can be used to execute a wait or non-wait shred join instruction. For any of the various representative embodiments, the processor core can be used to execute a shred join all instruction.

[0053] For any of the various representative embodiments, the processor core can be used to execute a non-cached read or load instruction to specify a general purpose register for storing data received from the memory. For any of the various representative embodiments, the processor core can be used to execute a non-cached write or store instruction to specify data in a general purpose register for storage in the memory.

[0054] For any of the various representative embodiments, the core control circuitry may be used to assign a transaction identifier to any load or store request to memory and to correlate the transaction identifier with a thread identifier.

[0055] For any of the various representative embodiments, the processor core can be used to execute the first thread priority instruction to assign the first priority to the execution thread with the corresponding thread identifier. For any of the various representative embodiments, the processor core can be used to execute the second thread priority instruction to assign the second priority to the execution thread with the corresponding thread identifier.

[0056] For any of the various representative embodiments, the processor core can be used to execute a custom atomic return instruction to complete the execution thread of the custom atomic operation. For any of the various representative embodiments, in conjunction with a memory controller, the processor core can be used to perform floating point atomic memory operations. For any of the various representative embodiments, in conjunction with a memory controller, the processor core can be used to perform custom atomic memory operations.

[0057] A method for self-scheduling execution of instructions is also disclosed, wherein a representative method embodiment includes: receiving a work descriptor data packet; and automatically scheduling instructions for execution in response to the received work descriptor data packet.

[0058] Another method of self-scheduled execution of instructions is also disclosed, wherein a representative method embodiment includes: receiving an event data packet; and automatically scheduling instructions for execution in response to the received event data packet.

[0059] A method for a first processing element to generate multiple execution threads for a second processing element to execute is also disclosed, wherein a representative method embodiment includes: executing a shred creation instruction; and in response to the execution of the shred creation instruction, generating one or more work descriptor packets to the second processing element to execute the multiple execution threads.

[0060] Also disclosed is a method for a first processing element to generate multiple execution threads for execution by a second processing element, wherein a representative method embodiment includes: executing a fiber creation instruction; and in response to executing the fiber creation instruction, reserving a predetermined amount of memory space in a thread control memory to store return variable parameters, and generating one or more work descriptor data packets to the second processing element to execute the multiple execution threads.

[0061] A method for self-scheduling execution of instructions is also disclosed, wherein a representative method embodiment includes: receiving a work descriptor data packet; decoding the received work descriptor data packet into an execution thread having an initial program counter and any received variable parameters; assigning an available thread identifier to the execution thread; automatically queuing the thread identifier to execute the execution thread; and periodically selecting the thread identifier to execute the execution thread.

[0062] Another method for self-scheduled execution of instructions is also disclosed, in which a representative method embodiment includes: receiving a work descriptor data packet; decoding the received work descriptor data packet into an execution thread having an initial program counter and any received variable parameters; assigning an available thread identifier to the execution thread; when the thread identifier has a valid state, automatically queuing the thread identifier to execute the execution thread; and periodically selecting the thread identifier to execute instructions of the execution thread for a duration that the valid state remains unchanged until the execution thread is completed.

[0063] Another method for self-scheduled execution of instructions is also disclosed, in which a representative method embodiment includes: receiving a work descriptor data packet; decoding the received work descriptor data packet into an execution thread having an initial program counter and any received variable parameters; assigning an available thread identifier to the execution thread; when the thread identifier has a valid state, automatically queuing the thread identifier in an execution queue to execute the execution thread; and periodically selecting the thread identifier to execute instructions of the execution thread for a duration that the valid state remains unchanged; and when the thread identifier has a paused state, pausing the thread execution by not returning the thread identifier to the execution queue.

[0064] Another method of self-scheduled execution of instructions is also disclosed, wherein a representative method embodiment includes: receiving a work descriptor packet; decoding the received work descriptor packet into an execution thread having an initial program count and any received variable parameters; storing the initial program count and any received variable parameters in a thread control memory; assigning an available thread identifier to the execution thread; automatically queuing the thread identifier for execution of the execution thread when the execution thread has a valid state;

[0065] The thread control memory is accessed using the thread identifier as an index to select an initial program count for an execution thread; and the thread identifier is periodically selected for executing instructions of the execution thread until the execution thread is completed as long as the valid state remains unchanged.

[0066] For any of the various representative embodiments, the method may further include: receiving an event data packet; and decoding the received event data packet into an event identifier and any received variable parameters.

[0067] For any of the various representative embodiments, the method may further include assigning an initial valid state to the execution thread.

[0068] For any of the various representative embodiments, the method may further include: assigning a pause state to the execution thread in response to execution of a memory load instruction. For any of the various representative embodiments, the method may further include: assigning a pause state to the execution thread in response to execution of a memory store instruction.

[0069] For any of the various representative embodiments, the method may further include: terminating execution of the selected thread in response to execution of the return instruction. For any of the various representative embodiments, the method may further include: returning the corresponding thread identifier of the selected thread to the thread identifier pool in response to execution of the return instruction. For any of the various representative embodiments, the method may further include: clearing the register of the thread control memory indexed by the corresponding thread identifier of the selected thread in response to execution of the return instruction. For any of the various representative embodiments, the method may further include: generating a return work descriptor packet in response to execution of the return instruction.

[0070] For any of the various representative embodiments, the method may further include: generating a point-to-point event data message. For any of the various representative embodiments, the method may further include: generating a broadcast event data message.

[0071] For any of the various representative embodiments, the method may further include: using the event mask to respond to the received event data packet. For any of the various representative embodiments, the method may further include: determining an event number corresponding to the received event data packet. For any of the various representative embodiments, the method may further include: changing the state of the thread identifier from suspended to valid to resume execution of the corresponding execution thread in response to the received event data packet. For any of the various representative embodiments, the method may further include: changing the state of the thread identifier from suspended to valid to resume execution of the corresponding execution thread in response to the event number of the received event data packet.

[0072] For any of the various representative embodiments, the method may further include: continuously selecting the next thread identifier from the execution queue for executing a single instruction of the corresponding execution thread. For any of the various representative embodiments, the method may further include: performing a circular selection of the next thread identifier from the execution queue among the plurality of thread identifiers, respectively for executing a single instruction of the corresponding execution thread. For any of the various representative embodiments, the control logic and the thread selection circuit may further include: performing a circular selection of the next thread identifier from the execution queue among the plurality of thread identifiers, respectively for executing a single instruction of the corresponding execution thread, until the execution thread is completed. For any of the various representative embodiments, the method may further include: performing a bucket selection of the next thread identifier from the execution queue among the plurality of thread identifiers, respectively for executing a single instruction of the corresponding execution thread.

[0073] For any of the various representative embodiments, the method may further include: assigning an active state or a suspended state to the thread identifier. For any of the various representative embodiments, the method may further include: assigning a priority state to the thread identifier.

[0074] For any of the various representative embodiments, the method may further include, after executing the corresponding instruction, returning the corresponding thread identifier with the assigned valid state and the assigned priority to the execution queue.

[0075] For any of the various representative embodiments, the method may further include: selecting thread identifiers from the first priority queue at a first frequency, and selecting thread identifiers from the second priority queue at a second frequency, the second frequency being lower than the first frequency. For any of the various representative embodiments, the method may further include: determining the second frequency as a skip count from the selection of the thread identifiers from the first priority queue.

[0076] For any of the various representative embodiments, the method may further include: controlling the data path access size. For any of the various representative embodiments, the method may further include: increasing or decreasing the memory load access size in response to the time average usage level. For any of the various representative embodiments, the method may further include: increasing or decreasing the memory storage access size in response to the time average usage level. For any of the various representative embodiments, the method may further include: increasing the size of the memory load access request to correspond to a cache line boundary of the data cache.

[0077] For any of the various representative embodiments, the method may further include: generating one or more system calls to the host processor. For any of the various representative embodiments, the method may further include: using a predetermined credit count, throttling a plurality of system calls in any predetermined time period.

[0078] For any of the various representative embodiments, the method may further include, in response to a request from a host processor, copying and transmitting all data corresponding to the selected thread identifier from the thread control memory to monitor the thread state.

[0079] For any of the various representative embodiments, the method may further include: executing a shred creation instruction to generate one or more commands to generate one or more call work descriptor packets to another processor core or to a hybrid thread fabric circuit. For any of the various representative embodiments, the method may further include: in response to the execution of the shred creation instruction, reserving a predetermined amount of memory space for storing any return variable parameters. For any of the various representative embodiments, the method may further include: in response to the generation of one or more call work descriptor packets, storing a thread return count in a thread return register. For any of the various representative embodiments, the method may further include: in response to receiving a return data packet, decrementing the thread return count stored in the thread return register. For any of the various representative embodiments, the method may further include: in response to the thread return count in the thread return register being decremented to zero, changing the pause state of the corresponding thread identifier to a valid state for subsequent execution of the thread return instruction to complete the created shred or thread.

[0080] For any of the various representative embodiments, the method may further include: executing a wait or non-wait shred join instruction. For any of the various representative embodiments, the method may further include: executing a shred join all instruction.

[0081] For any of the various representative embodiments, the method may further include executing a non-cached read or load instruction to specify a general purpose register for storing data received from the memory.

[0082] For any of the various representative embodiments, the method may further include executing a non-cached write or store instruction to designate data in the general register for storage in the memory.

[0083] For any of the various representative embodiments, the method may further include assigning a transaction identifier to any load or store request to memory and correlating the transaction identifier with the thread identifier.

[0084] For any of the various representative embodiments, the method may further include: executing a first thread priority instruction to assign a first priority to an execution thread having a corresponding thread identifier. For any of the various representative embodiments, the method may further include: executing a second thread priority instruction to assign a second priority to an execution thread having a corresponding thread identifier.

[0085] For any of the various representative embodiments, the method may further include executing the custom atomic return instruction to complete the thread of execution of the custom atomic operation.

[0086] For any of the various representative embodiments, the method may further include performing a floating point atomic memory operation.

[0087] For any of the various representative embodiments, the method may further include performing a custom atomic memory operation.

[0088] Numerous other advantages and features of the invention will become apparent from the following detailed description of the invention and embodiments thereof, from the claims and from the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0089] The objects, features and advantages of the present invention will be more readily understood when the following disclosure is considered in conjunction with the accompanying drawings in which like reference numerals are used to indicate the same components in the various figures and reference numerals with letter designations are used to indicate additional types, examples or variations of selected component embodiments in the various figures, wherein:

[0090] Figure 1 is a block diagram of a representative first embodiment of a hybrid computing system.

[0091] Figure 2 is a block diagram of a representative second embodiment of a hybrid computing system.

[0092] Figure 3 is a block diagram of a representative third embodiment of a hybrid computing system.

[0093] Figure 4 A high-level block diagram of a portion of a representative embodiment of a hybrid thread fabric circuit group.

[0094] Figure 5 is a high-level block diagram of a representative embodiment of a hybrid threaded processor 300 .

[0095] Figure 6 is a detailed block diagram of a representative embodiment of a thread memory for a hybrid threaded processor.

[0096] Figure 7 is a detailed block diagram of a representative embodiment of a network responsive memory for a hybrid threaded processor.

[0097] Figure 8 is a detailed block diagram of a representative embodiment of a hybrid threaded processor.

[0098] Fig.9A and 9B (collectively referred to as FIG. 9 ) is a flow chart of a representative embodiment of a method for self-scheduling and thread control of a hybrid threaded processor.

[0099] Fig.10 is a detailed block diagram of a representative embodiment of thread selection control circuitry for a hybrid threaded processor and control logic and thread selection circuitry.

[0100] Fig.11 A block diagram of a representative embodiment and representative data packets that are part of a first interconnect network.

[0101] Fig.12 is a detailed block diagram of a representative embodiment of data path control circuitry for a mixed thread processor.

[0102] Fig.13 is a detailed block diagram of a representative embodiment of system call circuitry and host interface circuitry for a hybrid threaded processor. DETAILED DESCRIPTION

[0103] Although the present invention allows for many different forms of embodiments, it is understood that the present disclosure will be regarded as an example of the principles of the present invention and it is not desired to limit the present invention to the specific embodiments shown, and specific exemplary embodiments of the present invention are shown in the drawings and will be described in detail herein. In this regard, before explaining in detail at least one embodiment according to the present invention, it should be understood that the present invention is not limited to the construction details and component arrangements described above and below, shown in the drawings, or described in the examples. The method and apparatus according to the present invention can have other embodiments and can be practiced and executed in various ways. And, it should be understood that the wording and terminology used herein and the summary contained below are for the purpose of description and should not be regarded as limiting.

[0104] Figure 1 , 2 3 are block diagrams of representative first, second, and third embodiments of hybrid computing systems 100A, 100B, 100C (collectively referred to as system 100). Figure 4 is a high-level block diagram of a portion of a representative embodiment of a hybrid thread fabric circuit group 205 having a second interconnect network 250. Figure 5 is a high-level block diagram of a representative embodiment of a hybrid thread processor (“HTP”) 300 . Figure 6is a detailed block diagram of a representative embodiment of a thread memory 320 (also equivalently referred to as thread control memory 320 or thread context memory 320) of HTP 300. Figure 7 is a detailed block diagram of a representative embodiment of the network response memory 325 of the HTP 300. Figure 8 FIG. 9 is a flow chart of a representative embodiment of a method for self-scheduling and thread control of HTP 300.

[0105] refer to Figure 1 -9, the hybrid computing system 100 includes a hybrid thread processor ("HTP") 300, which is coupled to one or more hybrid thread fabric ("HTF") circuits 200 via a first interconnect network 150. It should be understood that as used herein, the term "fabric" refers to and includes an array of computing circuits, which in this case are reconfigurable computing circuits. Figure 1 , 2 3 show different system 100A, 100B, and 100C arrangements that include additional components that form relatively larger and smaller systems 100, any and all of which are within the scope of the present disclosure. Figure 1 and 2 As shown in , each of which may be an arrangement suitable for, for example but not limited to, a system on a chip (“SOC”), in various combinations as described, the hybrid computing systems 100A, 100B may optionally also include a memory controller 120 (which may also be a separate integrated circuit) that may be coupled to a memory 125, any of various communication interfaces 130 (such as a PCIe communication interface), one or more host processors 110, and a host interface (“HIF”) 115. Figure 3 As shown, each of which may be an arrangement suitable for, for example, but not limited to, a "chiplet" configuration on a common substrate 101, the hybrid computing system 100C may also optionally include a communication interface 130 with or without these other components. Any and all of these arrangements are within the scope of the present disclosure and are collectively referred to herein as the system 100. Any of these hybrid computing systems 100 may also be considered a "node" that operates under a single operating system ("OS") and may also be coupled to other such local and remote nodes.

[0106] As used herein, hybrid threading refers to the ability to spawn multiple computational filaments and threads across different heterogeneous types of processing circuits (hardware), such as across HTF circuits 200 (e.g., a reconfigurable computing fabric) and across processors, such as HTP 300 or another type of RISC-V processor. Hybrid threading also refers to a programming language / style in which worker threads transition from one computing element to the next to move computations to where the data is, which is also implemented in a representative embodiment. The host processor 110 is typically a multi-core processor that may be embedded within the hybrid computing system 100, or may be an external host processor coupled to the hybrid computing system 100 via a communication interface 130 (e.g., a PCIe-based interface). These processors, such as the HTP 300 and one or more host processors 110, are described in more detail below.

[0107] The memory controller 120 may be implemented in a manner that is or becomes known in the electronic arts. Alternatively, in a representative embodiment, the memory controller 120 may be implemented as described in the related applications. The first memory 125 may also be implemented in a manner that is or becomes known in the electronic arts and as described in more detail below.

[0108] Also in a representative embodiment, the HTP 300 is a multithreaded processor based on the RISC-V ISA having a processor core 305 including an extended instruction set, and having one or more core control circuits 310 and one or more second memories 315 (referred to as core control memories 315), as discussed in more detail below. In general, the HTP 300 provides barrel-cycle instantaneous thread swapping to maintain a high instruction rate per clock.

[0109] For purposes herein, the HIF 115 enables the host processor 110 to send work to the HTP 300 and the HTF circuit 200, and the HTP 300 to send work to the HTF circuit 200, both as "work descriptor packets" transmitted on the first interconnect network 150. A unified mechanism is provided to start and end work on the HTP 300 and the HTF circuit 200: work on the HTP 300 and the HTF circuit 200 is started with a call work descriptor packet, and work on the HTP 300 and the HTF circuit 200 is ended with a return work descriptor packet. The HIF 115 includes scheduling circuitry and queues (referred to as "scheduling queues" 105 for short), which also provide management functionality for monitoring the loads provided to the HTF circuit 200 and / or the HTP 300 and the availability of resources of the HTF circuit 200 and / or the HTP 300. When resources on the HTF circuit 200 and / or HTP 300 are available, the scheduling queue 105 determines the least loaded HTF circuit 200 and / or HTP 300 resource. In the case of multiple HTF circuit groups 205 with the same or similar work load, if possible, the HTF circuit group 205 that is currently executing the same kernel is selected (to avoid having to load or reload the kernel configuration). Similar functionality of the HIF 115 may also be included in the HTP 300, for example, especially for system 100 arrangements that may not include a separate HIF 115. Other HIF 115 functions are described in more detail below. The HIF 115 can be implemented in a manner that is or will become known in the electronic art, such as being implemented as one or more state machines with registers (forming FIFOs, queues, etc.).

[0110] The first interconnect network 150 is a packet-based communication network that provides routing of data packets between and among the HTF circuit 200, the hybrid thread processor 300, and other optional components (e.g., the memory controller 120, the communication interface 130, and the host processor 110). The first interconnect network 150 is typically implemented as a plurality of crossbar switches having a folded Claus configuration, and is typically a mesh network for additional connections, depending on the system 100 embodiment. For purposes of the present disclosure, the first interconnect network 150 forms part of an asynchronous switch fabric ("AF"), meaning that data packets can be routed along any of a variety of paths so that any selected data packet can arrive at an addressed destination at any of a number of different times, depending on the routing. This is in contrast to the synchronous mesh communication network 275 of the second interconnect network 250, discussed in more detail below. Referring below to the first interconnect network 150, the first interconnect network 150 is typically implemented as a plurality of crossbar switches having a folded Claus configuration, and is typically a mesh network for additional connections. Fig.10 and 11 Aspects of the first interconnection network 150 are discussed in greater detail.

[0111] The HTF circuit 200, in turn, generally includes a plurality of HTF circuit groups 205, wherein each HTF circuit group 205 is coupled to a first interconnect network 150 for data packet communication. Each HTF circuit group 205 can operate independently of each of the other HTF circuit groups 205. Each HTF circuit group 205, in turn, includes an array of a plurality of HTF reconfigurable computing circuits 210 (which are equivalently referred to herein as "tiles" 210), and a second interconnect network 250. The tiles 210 are embedded in or otherwise coupled to the second interconnect network 250, which includes two different types of networks, as discussed in the related applications. As an overview, the HTF circuit 200 is a coarse-grained reconfigurable computing fabric that includes interconnected computing tiles 210 for performing a plurality of different computing operations.

[0112] The HTP 300 is a barrel multithreaded processor designed to perform well in applications with a high degree of parallelism acting on sparse data sets (i.e., applications with a minimal degree of data reuse). The HTP 300 is based on an open source RISC-V processor and executes in user mode. The HTP 300 includes more RISC-V user mode instructions, plus a set of custom instructions that allow thread management, sending and receiving events to / from other HTPs 300, HTF circuits 200, and one or more host processors 110, as well as instructions for efficient access to memory 125.

[0113] The processor core 305 has an associated cache memory, such as data cache registers 346, illustrated as part of the core control memory 315 or which may be arranged internally and / or contained within the processor core 305 (not separately illustrated). Such data cache registers 346 are typically used to store data that will be reused so that the data can be fetched from the cache and a memory load operation to the memory 125 is not required. However, many applications reuse very little data, and an application typically requires one or more memory load operations to the memory 125. Since the data is not reused in the future, in a representative embodiment, the data held in such data cache registers 346 associated with the processor core 305 will not be evicted or overwritten by data fetched from the memory 125 during a memory load operation, as discussed in more detail below, but will remain available in the cache for potential reuse purposes.

[0114] The HTP 300 of multiple threads per HTP processor core 305 allows some threads to wait for the response from the memory 125 while other threads continue to execute instructions. This computing style allows the delay to the memory 125 and allows the number of instructions to be executed per clock to continue to be high. The event mechanism allows the threads from multiple HTP cores 305 to be transmitted in an efficient manner. The thread suspends the execution of instructions when waiting for the memory 125 response or event message, thereby allowing other threads to use the instruction execution resource. The HTP 300 is self-scheduling and event-driven, thereby allowing threads to efficiently create, destroy and communicate with other threads.

[0115] The work descriptor packet is used to start work on the HTP 300 and HTF circuit 200. Receiving the work descriptor packet by the HTP 300 constitutes an "event" that triggers hardware-based self-scheduling and subsequent execution of associated functions or work (called thread execution) in the HTP 300 without further access to the main memory 125. Once the thread is started, instructions are executed until a thread return instruction is executed. The thread return instruction sends the return work descriptor packet to the original caller.

[0116] For purposes of this disclosure, at a high level or general level, a work descriptor packet includes, for example but not limited to, information required to initialize a thread context for the HTP 300, such as a program counter (e.g., a 64-bit address) to start thread execution in a stored instruction (stored in the instruction cache 340), and any variable parameters or addresses in the first memory 125 to obtain variable parameters or other information to be used in thread execution, and a return address for transmitting computational results. There may be a variety of different categories of work descriptor packets, depending on the operation or instruction to be performed, and a number of examples are illustrated and discussed below. The instruction cache 340 is populated prior to any execution, such as in an initial system 100 configuration.

[0117] Therefore, in many cases, HTP 300 allows to create threads and start execution without single access main memory 125. This "lightweight" thread creation allows to create multiple threads when entering the parallel region of application program with minimum delay and extremely low latency, which is in sharp contrast with prior art computing architecture, in which thread creation is completed by the following operations: a small (64B) work descriptor packet in the initialization hardware, then the packet is sent to the destination HTP 300 where the thread is started. Receiving HTP 300 can obtain the packet and initialize the hardware context of the thread from the work descriptor packet. The thread is an execution instruction that starts immediately. As mentioned above, the work descriptor packet only contains the instruction PC that is about to start executing and a certain number of call variable parameters (e.g., four at most). Receiving HTP 300 can autonomously initialize the remainder of the thread context state to prepare to start the thread execution instruction.

[0118] The execution thread has a memory stack space and a main memory 125 context space. The context space is used only when the state of the thread needs to be written to the memory 125 to be accessed by the host processor 110. Each HTP 300 is initialized by a core stack base address and a core context base address, where the base address points to the stack block and the context space block. The thread stack base address is obtained by taking the core stack base address and adding the thread ID multiplied by the thread stack size. The thread context base address is obtained in a similar manner.

[0119] HTP 300 generally includes one or more processor cores 305, which may be any type of processor core, such as a RISC-V processor core, an ARM processor core, etc., all of which are examples and not limiting. Core control circuitry 310 and core control memory 315 are provided for each processor core 305 and are Figure 5 305. For example, when multiple processor cores 305 are implemented in one or more HTPs 300, corresponding multiple core control circuits 310 and core control memories 315 are also implemented, wherein each core control circuit 310 and core control memory 315 is used to control the corresponding processor core 305. In addition, one or more of the HTPs 300 may also include a data path control circuit system 395, which is used to control the access size (e.g., load request of the memory 125) on the first interconnection network 150 to manage potential congestion of the data path.

[0120] The core control circuitry 310 in turn includes control logic and thread selection circuitry 330 and network interface circuitry 335. The core control memory 315 includes a plurality of registers or other memory circuitry, conceptually divided and referred to herein as a thread memory (or thread control memory) 320 and a network response memory 325. For example and without limitation, the thread memory 320 includes a plurality of registers for storing information related to thread state and execution, while the network response memory 325 includes a plurality of registers for storing information related to data packets transmitted to and from the first memory 125 over the first interconnection network 150, such as requests to read or store data from the first memory 125.

[0121] refer to Figure 6 , the thread memory 320 includes a plurality of registers, including: a thread ID pool register 322 (for example, but not limited to, storing a predetermined number of thread IDs that are available and are usually filled with identification numbers 0 to 31 for a total of 32 thread IDs when the system 100 is configured); a thread state (table) register 324 (storing thread information, such as valid, idle, paused, waiting for instructions, first (normal) priority, second (low) priority, temporary change of priority when resources are not available); a program counter register 326 (for example, storing the address or virtual address where the thread will start next in the instruction cache 340); a general register 328 for storing integer and floating point data; and Thread memory 320 includes a thread return count register 332 (which tracks the number of pending threads that will return to complete execution); a return variable parameter buffer 334 ("RAB", e.g., a header RAB that is a header of a linked list of return variable parameter buffers); a thread return register 336 (e.g., stores a return address, a call identifier, any thread identifiers associated with the calling thread); a custom atomic transaction identifier register 338; an event received mask register 342 (used to specify which events to "listen" to, as discussed in more detail below); an event status register 344; and a data cache register 346 (which typically provides 4-8 cache lines of cache memory for each thread). All of the different registers of thread memory 320 are indexed using the assigned thread ID of a given or selected thread.

[0122] refer to Figure 7, the network response memory 325 includes a plurality of registers, such as a memory request (or command) register 348 (e.g., a command to read, write, or perform a custom atomic operation); a thread ID and transaction identifier ("transaction ID") register 352 (where the transaction ID is used to track any request for memory and to associate each such transaction ID with the thread ID of the thread that generated the request for memory 125); a request cache line index register 354 (used to specify which cache line in the data cache registers 346 is to be written to when data is received from memory for a given thread (thread ID); a register byte register 356 (specifies the number of bytes to be written to the general registers 328); and a general register index and type register 358 (indicates which general register 328 is to be written to, and whether it is sign-extended or floating point).

[0123] As described in more detail below, the HTP 300 will receive the work descriptor packet. In response, the HTP 300 will find an idle or empty context and initialize a context block, assign a thread ID to the execution thread (generally referred to herein as a "thread") if the thread ID is available, and place the thread ID in the execution (i.e., "ready to run") queue 345. The threads in the execution (ready to run) queue 345 are selected for execution, typically in a round-robin or "bucket" selection process, where a single instruction for a first thread is provided to the execution pipeline 350 of the processor core 305, followed by a single instruction for a second thread, followed by a single instruction for a third thread, followed by a single instruction for the next thread, and so on, until all threads in the execution (ready to run) queue 345 have had corresponding instructions provided to the execution pipeline 350, at which point thread selection begins again with the next instruction for the first thread of the execution (ready to run) queue 345 provided to the execution pipeline 350, followed by the next instruction for the second thread, and so on, in a cycle through all threads of the execution (ready to run) queue 345. This execution will continue for each such thread until execution for that thread has been completed, for example by executing a thread return instruction, at which point a response packet (having the results of that thread execution) is transmitted back to the source of the work descriptor packet, i.e., the source of the work descriptor call packet. Additionally, in a representative embodiment and as discussed in more detail below, different levels of priority are optionally provided to the execution (ready-to-run) queue 345, illustrated as a first priority queue 355 and a second (lower) priority queue 360, where execution of threads in the first priority queue 355 occurs more frequently than execution of threads in the second (lower) priority queue 360.

[0124] Therefore, HTP 300 is an "event driven" processor, and will automatically start thread execution (provided that thread ID is available, but without any other requirements for starting execution) after receiving a work descriptor packet, that is, the arrival of the work descriptor packet automatically triggers the start of local thread execution, without reference to memory 125 or additional requests for memory 125. This is extremely useful because the response time for starting to execute many threads (e.g., thousands of threads) in parallel is quite low. HTP 300 will continue thread execution until the thread execution is completed, or wait for a response, in which case the thread will enter a "pause" state, as discussed in more detail below. Several different pause states are discussed in more detail below. After receiving the response, the thread returns to an active state, in which the thread resumes execution, wherein its thread ID returns to the execution (ready to run) queue 345. In conjunction with the thread state information stored in thread memory 320, control of this thread execution is performed by hardware through control logic and thread selection circuit system 330.

[0125] In addition to the host processor 110 generating work descriptor packets, the HTP 300 may also generate and transmit work descriptor packets to initiate work as one or more computing threads on another computing resource (e.g., another HTP 300 or any HTF circuit 200). Such work descriptor packets are "call" work descriptor packets and generally include a source identifier or address of the host processor 110 or HTP 300 that is generating the call work descriptor packet, a thread ID (e.g., a 16-bit call identifier (ID)) for identifying a return or associating a return with the original call, a 64-bit virtual kernel address (as a program counter, used to locate the first instruction to begin executing a thread, typically stored in the instruction cache 340 of the HTP 300 (or HTF circuit 200), which may also be a virtual address space), and one or more call variable parameters (e.g., up to four call variable parameters).

[0126] Similarly, when a thread has completed, the HTP 300 or HTF circuit 200 generates another work descriptor packet, referred to as a "return" work descriptor packet, which is generally created when the HTP 300 or HTF circuit 200 executes the last instruction of the thread (referred to as a return instruction), wherein the return work descriptor packet is assembled by the data packet encoder circuit 380, as discussed below. The return packet will be addressed back to the source (using the identifier or address provided in the call work descriptor packet), the thread ID (or call ID) from the call work descriptor packet (to allow the source to correlate the return with the call issued, particularly when multiple calls have been generated by the source and are pending at the same time), and one or more return values ​​(as a result), for example up to four return values.

[0127] Figure 8is a detailed block diagram of a representative embodiment of HTP 300. For ease of illustration and discussion, it should be noted that Figure 8 Not all registers of thread memory 320 and network response memory 325 are described. Figure 8 , the core control circuit 310 includes a control logic and thread selection circuit system 330 and a network interface circuit system 335. The control logic and thread selection circuit system 330 includes, for example but not limited to, a circuit system formed by a combination of any one of a plurality of various logic gates (e.g., NAND, NOR, AND, OR, EXCLUSIVE OR, etc.) and various state machine circuits (control logic 331), and multiplexers (e.g., input multiplexer 387, thread selection multiplexer 385). The network interface circuitry 335 includes an AF input queue 365 that receives data packets (including work descriptor packets) from the first interconnect network 150; an AF output queue 370 that transmits data packets (including work descriptor packets) to the first interconnect network 150; a packet decoder circuit 375 that decodes incoming data packets from the first interconnect network 150, obtains data (in specified fields) and transmits the data provided in the packets to the thread memory 320 and the associated registers of the network response memory 325 (in conjunction with the thread ID assigned to the thread by the control logic and thread selection circuitry 330, which also provides or forms an index to the thread memory 320, as discussed in more detail below); and a packet encoder circuit 380 that encodes outgoing data packets (e.g., requests for memory 125, using a transaction ID from the thread ID and transaction identifier (“transaction ID”) register 352) for transmission on the first interconnect network 150. The packet decoder circuit 375 and the packet encoder circuit 380 may each be implemented as a state machine or other logic circuitry.

[0128] When the work descriptor packet arrives, the control logic and thread selection circuit system 330 assigns the available thread ID to the thread of the work descriptor packet from the thread ID pool register 322, wherein the assigned thread ID is used as the index of other registers in the thread memory 320, and then the corresponding data from the work descriptor packet, usually the program counter and one or more variable parameters, are filled in the register. For example, but not limited to, when preparing to start the thread execution instruction, the control logic and thread selection circuit system 330 autonomously initialize the remainder of the thread context state, such as loading data cache registers 346 and loading thread return registers 336. For another example, the execution thread has a main memory stack space and a main memory context space. The context space is only used when the state of the thread needs to be written to the memory to be accessed by the host. Each HTP 300 processor core 305 is initialized with a core stack base address and a core context base address, wherein the base address points to a stack block and a context space block. The thread stack base address is obtained by obtaining the core stack base address and adding the thread ID multiplied by the thread stack size. The thread context base address is obtained in a similar manner.

[0129] The thread ID is provided with a valid status (indicating that it is ready to execute) and is pushed to the first priority queue 355 of the execution (ready to run) queue 345, since threads are typically assigned first (or normal) priority. The selection circuitry (e.g., multiplexer 385) of the control logic and thread selection circuitry 330 selects the next thread ID in the execution (ready to run) queue 345 for use as an index (program counter register 326 and thread status register 324) of the thread memory 320 to select an instruction from the instruction cache 340, which is then provided to the execution pipeline 350 for execution. The execution pipeline then executes the instruction.

[0130] Upon completion of execution of the instruction, the same triplet of information (thread ID, valid state, and priority) may be returned to the execution (ready to run) queue 345 for continued selection for loop execution, under the control of the control logic and thread selection circuitry 330, depending on various conditions. For example, if the last instruction for the selected thread ID was a return instruction (indicating that the thread execution is complete and a return data packet is being provided), the control logic and thread selection circuitry 330 returns the thread ID to the available thread ID pool in the thread ID pool register 322 so that it can be used by another different thread. Also, for example, the valid indicator may change, such as to a suspended state (e.g., at this time, the thread may be waiting for information to be returned from or written to memory 125 or waiting for another event), and in this case, the thread ID (now having a suspended state) is not returned to the execution (ready to run) queue 345 until the state changes back to valid.

[0131] Continuing with the previous example, when the last instruction for the selected thread ID is a return instruction, the return information (thread ID and return variable parameters) is then pushed to the network command queue 390, which is typically implemented as a first-in, first-out (FIFO), through the execution pipeline 350. The thread ID is used as an index into the thread return register 336 to obtain the return information, such as the transaction ID and source (caller) address (or other identifier), and the packet encoder circuit (on the first interconnect network 150) then generates an outgoing return data packet.

[0132] Continuing with the latter example, the thread's instruction may be a load instruction, i.e., a read request to memory 125, which is then pushed to the network command queue 390 through the execution pipeline 350. The packet encoder circuit then generates an outgoing data packet (on the first interconnect network 150) with a request (either a read or a write request) to memory 125, including the size of the request and the assigned transaction ID (from the thread ID and transaction ID register 352, which is also used as an index to the network response memory 325), the address of the HTP 300 (such as the return address of the requested information). When a packet is subsequently received from the first interconnect network 150 and decoded, the transaction ID is used as an index to the network response memory 325, the thread ID of the thread that made the request is obtained, and the location of the data returned in the response by the write in the data cache register 346 is also provided, the transaction ID is then returned to the thread ID and transaction ID register 352 for reuse, and the state of the corresponding thread ID is set to valid again and the thread ID is pushed to the execution (ready to run) queue 345 again to resume execution.

[0133] A storage request to memory 125 is similarly performed, where the outgoing packet also has the data to be written to memory 125, the assigned transaction ID, the source address of HTP 300, and the return packet is a reply with the transaction ID. The transaction ID is then also returned to the thread ID and transaction ID register 352 for reuse, and the state of the corresponding thread ID is set to valid again and the thread ID is pushed to the execution (ready to run) queue 345 again to resume execution.

[0134] Fig. 9 is a flow chart of a representative embodiment of a method for self-scheduling and thread control for HTP 300, which provides a useful overview, wherein HTP 300 has been filled with instructions in instruction cache 340 and a predetermined number of thread IDs in thread identifier pool register 322. The method starts at step 400 immediately after receiving a work descriptor packet. At step 402, the work descriptor packet is decoded, and at step 404, the information received in the work descriptor packet is filled into each register of thread memory 320, thereby initializing the context block. When thread ID is available in step 406, at step 408, thread ID is assigned (and if thread ID is unavailable in step 406, then at step 410, thread will wait until thread ID becomes available). At step 412, initially valid state is assigned to thread (with any initially assigned priority (e.g., first or second priority)), and at step 414, thread ID is provided to execution (ready to run) queue 345. Next, at step 416, a thread ID in the execution (ready to run) queue 345 is selected for execution (at a predetermined frequency, discussed in more detail below). At step 418, using the thread ID, the thread memory 320 is accessed and the program count (or address) is obtained. At step 420, the instruction corresponding to the program count (or address) is obtained from the instruction cache 340 and provided to the execution pipeline 350 for execution.

[0135] When the thread execution is completed in step 422, that is, when the instruction being executed is a return instruction, in step 424, the thread ID is returned to the thread ID pool register 322 for reuse by another thread, in step 426, the register of the thread memory 320 associated with the thread ID can be cleaned up (optionally), and in returning to step 434, the thread control for the thread can end. When the thread execution is not completed in step 422, and when the thread state remains valid in step 428, the thread ID (and its valid state and priority) is returned to the execution (ready to run) queue 345, returning to step 414 to continue execution. When the thread state is no longer valid (that is, the thread is suspended) in step 428, wherein the suspended state of the thread ID is indicated in the thread memory 320, in step 430, the execution of the thread is suspended until the state of the thread ID returns to valid and the thread ID (and its valid state and priority) is returned to the execution (ready to run) queue 345 in step 432, returning to step 414 to continue execution.

[0136] Similarly, the HTP 300 may generate calls to create threads on local or remote computing elements, thereby creating threads on other HTPs 300 or HTF circuits 200. Such calls are also created as outgoing data packets, and more specifically, as outgoing work descriptor packets on the first interconnection network 150. For example, the instruction of the current thread being executed may be a "filament creation" instruction (stored as a possible instruction in the instruction cache 340) to enable multiple threads to execute on various computing resources. As discussed in more detail below, such a fiber creation instruction specifies (using an address or a virtual address (node ​​identifier)) what computing resource will execute the thread, and will also provide associated variable parameters. When the fiber creation instruction is executed in the execution pipeline 350, the fiber creation command is pushed into the network command queue 390, and the next instruction is executed in the execution pipeline 350. The command is pulled out of the network command queue 390 and the data packet encoder circuit 380 has the information needed to create and send a work descriptor packet to the designated destination HTF 200 or HTP 300 .

[0137] If the created thread will have return variable parameters, such instructions will also allocate and reserve associated memory space, for example, in the return variable parameter buffer 334. If the space in the return variable parameter buffer 334 is not enough, the instruction will be suspended until the return variable parameter buffer 334 is available. The number of created shreds or threads is limited only by the amount of space to store the response variable parameters. The created thread without return variable parameters can be exempted from retaining the return variable parameter space, thereby avoiding a possible pause state. This mechanism ensures that the return from the completed thread always has a location to store its variable parameters. When the return is returned to the HTP 300 as a data packet on the first interconnection network 150, those packets are decoded, as discussed above, where the return data is stored in the associated reserved space in the return variable parameter buffer 334 of the thread memory 320, which is indexed by the thread ID associated with the shred creation instruction. Because many registers can be used for return variable parameters, the return variable parameter buffer 334 can be provided as a linked list of all created threads or a return variable parameter buffer or register allocated for the thread ID. Notably, this mechanism can make it possible to create thousands of threads extremely quickly, effectively minimizing the time involved in transitioning from single thread execution to high thread count parallelism.

[0138] As discussed in more detail below, various types of fiber join instructions are utilized to determine when all created threads are completed, and may be instructions with or without waiting. The count of the number of created threads is stored in the pending fiber return count register 332, which is decremented when the HTP 300 receives the thread return. The join operation can be implemented by copying the return into the register associated with the created thread ID. If the join instruction is a wait instruction, it will remain in a paused state until the return of the thread ID of the specified created thread arrives. During this period, other instructions are executed by the execution pipeline 350 until the paused state of the join instruction becomes an effective state and the join instruction returns to the execution (ready to run) queue 345.

[0139] The thread return instruction may also be used as an instruction following the shred creation instruction, rather than an add instruction. When the count in the pending shred return count register 332 reaches zero and the last thread return packet is received, the thread return instruction may also be executed and indicates that the shred creation operation has been completed and all returns have been received, thereby allowing the thread ID, return variable parameter buffer 334, and linked list to be released for other purposes. In addition, a work descriptor return packet (e.g., with result data) may also be generated and transmitted to a source called the primary thread (e.g., an identifier or address of a source that generates a call).

[0140] The join all instruction need not return variable parameters, only cause the pending shred to return an acknowledgment of the decrement of the count in count register 332. When the count reaches zero, the thread restarts because the join all is now complete.

[0141] Communication between processing elements is required to facilitate the processing of parallel algorithms. Representative embodiments provide an efficient means for threads of a set of processing resources to communicate using individual event messages, which may also contain data (e.g., variable parameters or results). Event messaging allows any host processor 110 with hardware-maintained cache coherence and any accelerator processor (e.g., HTP 300) with software-maintained cache coherence to efficiently participate in event messaging.

[0142] Event messaging supports both point-to-point and broadcast event messages. Each processing resource (HTP 300) can determine when a received event operation is complete and the processing resource should be notified. Event reception modes include simple (a single received event completes the operation), collect (a counter is used to determine when enough events have been received to complete the operation), and broadcast (an event received on a specific channel completes the event). In addition, events can be sent with an optional 64-bit data value.

[0143] HTP 300 has a set of event receiving states consisting of a 2-bit receiving mode, a 16-bit counter / channel number and a 64-bit event data value, which is stored in an event status register 344. HTP 300 can have multiple sets of event receiving states for each thread context, wherein each set of event receiving states is indexed with an event number. Therefore, events can be for a specific thread (thread ID) and an event number. The events sent can be point-to-point messages with a single destination thread, or broadcast messages sent to all threads in a set of processing resources belonging to the same process. When such events are received, a paused or sleeping thread can be reactivated to resume processing.

[0144] This use of the event status register 344 is much more efficient than a standard Linux-based host processor that can send and receive events through an interface that allows the host processor 110 to periodically poll for completed receive events. Threads waiting for event messages can be suspended until the receive operation is complete, i.e., the HTP 300 can suspend execution of threads pending completion of receive events, rather than wasting resources by polling, thereby allowing other threads to execute during these intervals. Each HTP 300 also maintains a list of processing resources that should participate in receiving events to avoid process safety issues.

[0145] The point-to-point message will specify the event number and destination (e.g., node number, which HTP 300, which core and which thread ID). On the receiving side, the HTP 300 will be configured or programmed with one or more event numbers stored in the event status register 344. If the HTP 300 receives event information with the event number, it is triggered and transitions from a pause state to an active state to resume execution, such as executing an event received instruction (e.g., EER below). Then, the instruction will determine whether the correct event number is received, and if so, any associated 64-bit data is written to the general register 328 for use by another instruction. If the event received instruction is executed and the correct event number is not received, it will be paused until the specific event number is received.

[0146] Event Listen (EEL) instructions may also be utilized, wherein an event mask is stored in the event received mask register 342, indicating one or more events that will be used to trigger or wake up the thread. When event information with any of those specified events arrives, the receiving HTP 300 will know which event number was triggered, e.g., which other process may have completed, and will receive event data from those completed events. Event Listen instructions may also have wait and non-wait variations, as discussed in more detail below.

[0147] For event messaging in collect mode, the receiving HTP 300 will collect (wait) a set of receive events before triggering, setting the count in the event status register 344 to the required value, decrementing when the required event message is received, and triggering when the count decrements to zero.

[0148] In broadcast mode, the sender processing resource can transmit messages to any thread within the node. For example, a sending HTP 300 can transmit a series of point-to-point messages to each other HTP 300 within the node, and each receiving HTP 300 then passes the message to each internal core 305. Each core control circuit 310 will go through its thread list to determine if it corresponds to the event number that has been initialized to receive, and determine which channel may have been specified on the first interconnect network 150.

[0149] This broadcast mode is particularly useful when thousands of threads can be executed in parallel, with the last thread to execute transmitting a broadcast event message indicating completion. For example, a first count of all threads that need to be completed can be maintained in the event status register 344, and a second count of all threads that have been executed can be maintained in the memory 125. As each thread executes, the second count is also extracted and incremented atomically, such as by atomic operations of the memory 125 (and compared to the first count), and its mode is set to receive broadcast messages by executing an EER instruction that waits until it receives a broadcast message. The last thread to execute treats the extracted value of the second count as the required first count minus one, indicating that it is the last thread to execute, and therefore sends a broadcast message, which is an extremely fast and efficient way to indicate the completion of a large number of parallel processes.

[0150] As mentioned above, while the HTP 300 can utilize standard RISC-V instructions, a significantly extended instruction set is provided to utilize all system 100 computing resources, as discussed in more detail below. Threads created from the host processor 110 are generally referred to as main threads, and threads created from the HTP 300 are generally referred to as shreds or shred threads, all of which execute identically on the destination HTP 300 and HTF 200 without passing through the memory 125.

[0151] Loading instructions:

[0152] HTP 300 has a relatively small number of read / write buffers for each thread, also referred to as data cache registers 346. The buffer (data cache register 346) temporarily stores shared memory data for use by the own thread. The data cache register 346 is managed by a combination of hardware and software. The hardware automatically allocates buffers and reclaims data when needed. By using RISC-V instructions, the software decides which data should be cached (read and write data), and when the data cache register 346 should be invalidated (if clean) or written back to the memory (if dirty). The RISC-V instruction set provides a FENCE instruction, and acquires and releases indicators on atomic instructions.

[0153] Standard RISC-V load instructions automatically use read data cache registers 346. Standard loads check to determine if the required data is in the existing data cache registers 346. If so, the data is obtained from the data cache registers 346, and the execution thread can continue to execute without pausing. If the required data is not in the data cache registers 346, the HTP 300 looks for an available data cache register 346 (needs to retrieve data from the buffer) and reads 64 bytes from memory into the data cache registers 346. The execution thread is paused until the memory read is completed and the load data is written to the RISC-V registers.

[0154] Read buffering has two main benefits: 1) larger accesses to the memory controller 120 are more efficient, and 2) access to the buffer allows the execution thread to not stall. However, there are situations where the use of the buffer creates problems. An example is a gather operation, where accesses would normally result in thrashing of the data cache register 346. For this reason, a set of special load instructions are provided that force the load instruction to check for a cache hit, but on a cache miss, only issue a memory request for the requested operand and do not place the obtained data in the data cache register 346, but instead place the data in one of the general registers 328.

[0155] These load instructions provide "probabilistic" caching based on expected access frequency for frequently used data versus infrequently or rarely used data. This is particularly important when used with sparse data sets, which, if placed in the data cache registers 346, will also be overwritten more frequently with other data that is needed, effectively polluting the data cache registers 346. The load instructions (NB or NC) allow frequently used data to remain in the data cache registers 346, while infrequently used (sparse) data that would normally be cached is actually designated for non-cached storage in the general registers 328.

[0156] This type of instruction has an NB suffix (non-buffered) or, equivalently, an NC suffix (non-cacheable):

[0157] LB.NB RA,40(SP).

[0158] The NB(NC) load instruction is intended for runtime libraries written into an assembly.

[0159] In Table 1, the following load instructions are added as 32-bit instructions, where Imm is the immediate field, RA is the register name, rs1 is the source index, rd is the destination index, and the bits in fields 14-12 and 6-0 specify the instruction.

[0160] Table 1:

[0161]

[0162] Bandwidth to memory is often the primary factor limiting application performance. Representative embodiments provide a means of informing HTP 300 of the size of memory load requests that should be issued to memory 125. Representative embodiments reduce waste of memory and bandwidth of first interconnect network 150 by accessing memory data that is not used by the application.

[0163] There is another optimization, wherein the application knows the size of the data structure being accessed, and can specify the amount of data to be loaded into the data cache register 346. As an example, if the algorithm uses a structure of size 16 bytes, and the structure is scattered in the memory, it will be optimal to issue a 16-byte memory read and place the data into the data cache register 346. A representative embodiment defines a set of memory load instructions, which provide the size of the operand to be loaded into the HTP 300 register and the size of the access to the memory in the case of a load miss data cache register 346. If the memory access crosses the cache line boundary, the actual load of the memory 125 can be less than the size specified by the instruction. In this case, the access size is reduced to ensure that the response data is written to a single cache line of the data cache register 346.

[0164] When the requested data is smaller than a cache line, the load instruction may also request additional data that is not currently needed by the HTP 300 but may be needed in the future and is worth obtaining at the same time (e.g., as a prefetch), thereby optimizing the read size access to the memory 125. This instruction may also overwrite (as discussed in more detail below, see Fig.12 ) any reduction in access size that may have been used for bandwidth management.

[0165] Thus, representative embodiments minimize bandwidth waste by requesting only memory data that is known to be needed. The result is improved application performance.

[0166] A set of load instructions has been defined that allow the specification of the amount of data to be accessed. The data is written to the buffer and invalidated by an eviction, FENCE, or atomic with a specified fetch. The load instruction provides a hint as to how much additional data (in 8-byte increments) will be accessed from memory and written to the store buffer. The load will only access additional data to the next 64-byte boundary. The load instruction specifies the number of additional 8-byte elements to load using the operation suffixes RB0-RB7:

[0167] LD.RB7 RA,40(SP)

[0168] The instruction format is shown in Table 2. The number of 8-byte data elements loaded into the buffer is specified as bits 6 and 4:3 of the 32-bit instruction. These load instructions can be used in assembly write routines, or ideally by a compiler. It is expected that initially only hand-written assemblies will utilize these instructions.

[0169] Table 2:

[0170]

[0171] Storage Instructions

[0172] The HTP 300 has a small amount of memory buffers for temporary storage of shared memory data. The memory buffers allow multiple writes to the memory to be merged into a smaller number of memory write requests. This has two benefits: 1) fewer write requests are more efficient for the first interconnection network 150 and the memory controller 120, and 2) the HTP 300 suspends the thread that is performing the memory store until the data is stored in the HTP 300 memory buffer or stored at the memory controller 120. Storage to the HTP 300 memory buffer is very fast and will not usually cause the thread to be temporarily executed. When the buffer is written to the memory controller 120, the thread is then suspended until completion is received to ensure the consistency of the memory 125.

[0173] Standard RISC-V storage instructions write data to the HTP 300 memory buffer. However, there are known situations where data is written directly to the memory and not to the memory buffer. One such situation is a scatter operation. A scatter operation will typically write only a single data value to the memory buffer. Writing to the buffer causes the buffer to jitter and forces other stored data that would benefit from write merging back to the memory. A set of storage instructions is defined for the HTP 300 to indicate that write buffering should not be used. These instructions write data directly to the memory 125, causing the execution thread to pause until the write is complete.

[0174] Unbuffered instructions are expected to be used in manually assembled libraries and are indicated with an NB suffix:

[0175] ST.NB RA,40(SP)

[0176] Add the following storage instructions as shown in Table 3.

[0177] Table 3:

[0178]

[0179]

[0180] Custom atomic store and clean lock (CL) instructions:

[0181] The custom atomic operation sets a lock on the provided address when the atomic operation is observed by the memory controller. The atomic operation is performed on the associated HTP 300. The HTP 300 should inform the memory controller when the lock should be cleaned. This should be on the last store operation performed by the HTP 300 for the custom atomic operation (or on the fiber termination instruction if no store is required). The HTP 300 indicates that the lock will be cleaned by executing a specific store operation. Store and clean lock instructions.

[0182] The following instruction sequence may be used to implement a custom atomic DCAS operation.

[0183] / / a0-atomic address

[0184] / / a1-64-bit memory value a0

[0185] / / a2-DCAS comparison value 1

[0186] / / a3-DCAS comparison value 2

[0187] / / a4-DCAS exchange value 1

[0188] / / a5-DCAS exchange value 2

[0189] atom_dcas:

[0190]

[0191] fail:

[0192] li a1,1

[0193] eft.cl a1,(a0) / / AMO failure response (and clear memory lock)

[0194] ATOMIC_FLOAT_ADD:

[0195] fadd.d a2,a1,a2 / / a1 contains the memory value, a2 contains the value to be added

[0196] fsd.cl a2,0(a0) / / a0 contains the memory address, clear the lock and terminate the atomic

[0197] eft / / Evict all lines from the buffer and terminate the atomic thread

[0198] The storage directive indicating that a lock should be cleaned is:

[0199] SB.CL RA,40(SP)

[0200] SH.CL RA,40(SP)

[0201] SW.CL RA,40(SP)

[0202] SD.CL RA,40(SP)

[0203] FSW.CL RA,40(SP)

[0204] FSD.CL RA,40(SP)

[0205] The formats of these storage instructions are shown in Table 4.

[0206] Table 4:

[0207]

[0208] Fiber creation instructions:

[0209] A fiber create (“EFC”) instruction initiates a thread on an HTP 300 or HTF 200 .

[0210] EFC.HTP.A4

[0211] EFC.HTF.A4

[0212] This instruction is called on HTP 300 (or HTF 200), starting execution at the address in register a0. (Optionally, the suffix .DA may be used. The instruction suffix DA indicates that the target HTP 300 is determined by the virtual address in register a1. If the DA suffix is ​​not present, the HTP 300 on the local system 100 is targeted.) The suffixes A1, A1, A2, and A4 specify the number of additional variable parameters to be passed to the HTP 300 or HTF 200. The variable parameter count is limited to the values ​​0, 1, 2, or 4 (e.g., the packet should fit in 64B). The additional variable parameters come from the register states (a2-a5).

[0213] It should be noted that if the return buffer is not available when executing the EFC instruction, the EFC instruction will wait until the return variable parameter buffer is available to start execution. Once the EFC instruction successfully creates a shred, the thread continues at the instruction immediately after the EFC instruction.

[0214] It should also be noted that threads created by host processor 110 are allowed to execute EFC instructions and create shreds. Shreds created by EFC instructions are not allowed to execute EFC instructions and will force an exception, optionally. The format of these shred creation instructions is shown in Table 5.

[0215] Table 5:

[0216]

[0217]

[0218] Thread return instruction:

[0219] The thread return (ETR) instruction passes variable parameters back to the parent thread that initiated the current thread (either through host processor 110 thread creation or HTP 300 shred creation). Once the thread has completed the return instruction, the thread is terminated.

[0220] ETR.A2

[0221] This instruction performs a return to the HTP 300 or host processor 110. The ac suffix specifies the number of additional variable parameters to be passed to the HTP or host. The variable parameter count can be a value of 0, 1, 2, or 4. The variable parameters come from register states (a0-a3). The format of these thread return instructions is shown in Table 6.

[0222] Table 6:

[0223]

[0224] Fiber adding instruction:

[0225] The Fiber Join (EFJ) instruction checks to see if the created fiber has returned. The instruction has two variants, Join Wait and No-Wait. The Wait variant will suspend thread execution until the fiber has returned. Join No-Wait does not suspend thread execution, but provides a success / failure status. For both variants, if the instruction is executed without an outstanding fiber returning, an exception is generated.

[0226] Write the variable arguments from the return fiber (up to four) to registers a0-a3.

[0227] EFJ

[0228] EFJ.NW

[0229] The format of these fiber join instructions is shown in Table 7.

[0230] Table 7:

[0231]

[0232] Fibers add all instructions:

[0233] The fiber joins all instructions (EFJ.ALL) pending until all outstanding fibers have returned. The instruction may be called with zero or more pending fiber returns. No instruction status or exceptions are generated. Any return variable parameters from fiber returns are ignored.

[0234] EFJ.ALL

[0235] Table 8 shows the format of these fiber join all instructions.

[0236] Table 8:

[0237]

[0238] Atomic return instruction:

[0239] The EMD Atomic Return instruction (EAR) is used to complete the thread of execution of a custom atomic operation and possibly provide a response back to the source that issued the custom atomic request.

[0240] The EAR instruction can send zero, one, or two 8-byte variable parameter values ​​back to the issuing computational element. The number of variable parameters to be sent back is determined by the ac2 suffix (A1 or A2). No suffix means zero variable parameters, A1 implies a single 8-byte variable parameter, and A2 implies two 8-byte variable parameters. If necessary, variable parameters are obtained from X registers a1 and a2.

[0241] The EAR instruction can also clear the memory row lock associated with the atomic instruction. EAR uses the value in the a0 register as the address to issue a clear lock operation. If the instruction has the suffix CL, a clear lock operation is issued.

[0242] The following DCAS example uses the EAR directive to send success or failure back to the request processor.

[0243] / / a0-atomic address

[0244] / / a1-64-bit memory value a0

[0245] / / a2-DCAS comparison value 1

[0246] / / a3-DCAS comparison value 2

[0247] / / a4-DCAS exchange value 1

[0248] / / a5-DCAS exchange value 2

[0249] atom_dcas:

[0250]

[0251]

[0252] fail:

[0253] li a1,1

[0254] ear.cl.a1 / / AMO failure response (and clear memory lock)

[0255] The instruction has two variants that allow the EFT instruction to also clean up the memory lock associated with the atomic operation. The format of the supported instructions is shown in Table 9.

[0256] Figure 9:

[0257]

[0258] First and second priority instructions:

[0259] The second (or low) priority instruction changes the current thread, which has a first priority, to the second lowest priority. The instruction is typically used when a thread is polling on an event (ie, a barrier) to occur.

[0260] ELP

[0261] The format of the ELP instruction is shown in Table 10.

[0262] Table 10:

[0263]

[0264] The first (or high) priority instruction changes the current thread, which has a second (or low) priority, to a first (or high or normal) priority. The instruction is typically used when a thread is polling and an event has occurred (ie, a barrier).

[0265] ENP

[0266] The format of the ELP instruction is shown in Table 11.

[0267] Table 11:

[0268]

[0269]

[0270] Floating-point atomic memory operations:

[0271] Floating point atomic memory operations are performed by the HTP 300 associated with the memory controller 120. For both 32 and 64 bit data types, the floating point operations performed are MIN, MAX, and ADD.

[0272] The aq bit and rl bit in the instruction specify whether all written data should be visible to other threads before issuing the atomic operation (aq), and whether all previously written data should be visible to this thread after the atomic completion (rl). In other words, the aq bit forces all write buffers to write back to memory, and the rl bit forces all read buffers to fail. Note that rs1 is an X register value, while rd and rs2 are F register values.

[0273] AMOFADD.S rd,rs2,(rs1)

[0274] AMOFMIN.S rd,rs2,(rs1)

[0275] AMOFMAX.S rd,rs2,(rs1)

[0276] AMOFADD.D rd,rs2,(rs1)

[0277] AMOFMIN.D rd,rs2,(rs1)

[0278] AMOFMAX.D rd,rs2,(rs1)

[0279] The formats of these floating-point atomic memory operation instructions are shown in Table 12.

[0280] Table 12:

[0281]

[0282] Custom atomic memory operations:

[0283] The custom atomic operations are performed by the HTP 300 associated with the memory controller 120. The operations are performed by executing RISC-V instructions.

[0284] Up to 32 custom atomic operations are available within the memory controller 120 of the system 100. Custom atomics are system-level resources that are available to any process attached to the system 100.

[0285] The aq and rl bits in the instruction specify whether all written data should be visible to other threads before issuing the atomic operation (rl), and whether all previously written data should be visible to this thread after the atomic completion (aq). In other words, the rl bit forces all write buffers to write back to memory, and the aq bit forces all read buffers to fail.

[0286] Custom atoms use the a0 register to specify a memory address. The number of source variable parameters is given by the suffix (A0, A1, A2, or A4) and is obtained from registers a1-a4. The number of result values ​​returned from memory can be 0 to 2 and is defined by the custom memory operation. The result values ​​are written to registers a0-a1.

[0287] AMOCUST0.A4

[0288] The following custom atomic instructions are defined as shown in Table 13.

[0289] Table 13:

[0290]

[0291] The ac field is used to specify the number of variable parameters (0, 1, 2 or 4). The following Table 14 shows the encoding.

[0292] Table 14:

[0293] ac coding suffix Variable argument count 0 No suffix 0 1 .A1 1 2 .A2 2 3 .A4 4

[0294] Eight custom atomic instructions are defined, each with 4 variable argument count variations, resulting in a total of 32 possible custom atomic operators.

[0295] Event Management:

[0296] System 100 is an event-driven architecture. Each thread has a set of events that can be monitored using event received mask register 342 and event status register 344. Event 0 is reserved for returning from the created fiber (HTP 300 or HTF 200). The remainder of the event can be used for event messaging, thread to thread, broadcast or collect. Thread to thread allows a thread to send an event to a specific destination thread on the same or different nodes. Broadcast allows a thread to send a specified event to a subset of threads on its node. The receiving thread should specify which specified broadcast events are expected. Collecting refers to the ability to specify the number of events to be received before the event becomes active.

[0297] The event trigger bit can be cleared (using the EEC instruction), and all events can be listened to (using the EEL instruction). The listening operation can suspend the thread until the event has been triggered, or in a non-wait mode (.NW), allow the thread to poll periodically while other execution is in progress.

[0298] A thread can send an event to a specific thread using an event send instruction (EES), or broadcast an event to all threads within a node using an event broadcast instruction (EEB). The broadcasted event is a specified event, where the transmitting thread specifies the event name (16-bit identifier), and the receiving thread filters the received broadcast events for the pre-specified event identifier. Once received, the event should be explicitly cleaned up (EEC) to avoid receiving the same event again. It should be noted that all event trigger bits are clear when the thread starts executing.

[0299] Event mode directives:

[0300] The Event Mode (EEM) instruction sets the operating mode of an event. Event 0 is reserved for thread return events, and the rest of the events can be in one of three receive modes: simple, broadcast, or collect.

[0301] In simple mode, a received event immediately causes the trigger bit to be set and the received message count to be incremented by one. Each newly received event causes the received event count to be incremented. An receive event instruction (EER) causes the received event count to be decremented by one. When the count transitions back to zero, the event trigger bit is cleared.

[0302] In broadcast mode, the channel of the received event is compared with the broadcast channel of the event number. If the channels match, the event trigger bit is set. The EER instruction causes the trigger bit to be cleared.

[0303] In collect mode, received events cause the event trigger count to be decremented by one. When the count reaches zero, the event trigger bit is set. The EER instruction causes the trigger bit to be cleared.

[0304] The EEM instruction prepares the event number for the selected operating mode. In simple mode, the 16-bit event counter is set to zero. For broadcast mode, the 16-bit event channel number is set to the value specified by the EEM instruction. For collection mode, the 16-bit event count is set to the value specified by the EEM instruction. Each of the three modes uses the same 16-bit value in a different way.

[0305]

[0306] The format of the event mode instruction is shown in Table 15.

[0307] Table 15:

[0308]

[0309] Event destination directives:

[0310] The Event Destination (EED) instruction provides an identifier to an event within an execution thread. The identifier is unique among all execution threads within a node. The identifier can be used with the Event Send instruction to send an event to a thread using the EES instruction. The identifier is an opaque value that contains the information needed to send an event from a source thread to a specific destination thread.

[0311] The identifier can also be used to obtain a unique value for transmitting a broadcast event. The identifier contains space for an event number. The input register rs1 specifies the event number to be encoded in the destination thread identifier. After the instruction is executed, the output rd register contains the identifier.

[0312] EED rd,rs1

[0313] The format of the event destination instruction is shown in Table 16.

[0314] Table 16:

[0315]

[0316] The event destination instruction may also be used by a process to obtain its own address, which may then be used for other broadcast messages, for example, to enable the process to receive other event messages as a destination, for example, for receiving return messages when the process is the main thread.

[0317] Event sending instructions:

[0318] The Event Send (EES) instruction sends an event to a specific thread. Register rs1 provides the destination thread and event number. Register rs2 provides the optional 8 bytes of event data.

[0319] EES rs1

[0320] EES.A1 rs1,rs2

[0321] The rs2 register provides the target HTP 300 for the event send operation. Register rs1 provides the event number to be sent. The legal values ​​for rs1 are 2 to 7. The format of the event send instruction is shown in Table 17.

[0322] Table 17:

[0323]

[0324] Event broadcast instructions:

[0325] The Event Broadcast (EEB) instruction broadcasts an event to all threads within a node. Register rs1 provides the broadcast channel (0 to 65535) to be sent. Register rs2 provides the optional 8 bytes of event data.

[0326] EEBrs1

[0327] EEB.A1 rs1,rs2

[0328] The format of the event broadcast instruction is shown in Table 18.

[0329] Table 18:

[0330]

[0331] Event monitoring instructions:

[0332] The Event Listener (EEL) instruction allows a thread to monitor the status of received events. The instruction can operate in one of two modes: wait and non-wait. The wait mode will suspend the thread until an event is received, and the non-wait mode provides received events when the instruction is executed.

[0333] EEL rd,rs1

[0334] EEL.NW rd,rs1

[0335] Register rs1 provides a mask of available events as the output of the snoop operation. If no event is available, the non-wait mode will return a value of zero in rs1. The format of the event snoop instruction is shown in Table 19.

[0336] Table 19:

[0337]

[0338] Event receiving instructions:

[0339] The Event Receive (EER) instruction is used to receive an event. Receive event contains confirmation that the event was observed and receives optional 8 bytes of event data. Register rs1 provides the event number. Register rd contains the optional 8 bytes of event data.

[0340] EERrs1

[0341] EER.A1 rd,rs1

[0342] The format of the event reception instruction is shown in Table 20.

[0343] Table 20:

[0344]

[0345] Fig.10 is a detailed block diagram of a representative embodiment of the control logic of the HTP 300 and the thread selection control circuitry 405 of the thread selection circuitry 330. As mentioned above, a second or low priority queue 360 ​​is provided, and a thread ID is selected from either the first (or high) priority queue 355 or the second or low priority queue 360 ​​using a thread selection multiplexer 385 under the control of the thread selection control circuitry 405. Threads in the second priority queue 360 ​​are pulled from the queue and executed at a lower rate than threads in the first priority queue 355.

[0346] As mentioned above, a pair of instructions ENP and ELP is used to transition a thread from a first priority level to a second priority level (ELP) and from the second priority level to the first priority level (ENP).

[0347] Threads in parallel applications often have to wait for other threads to complete their priority levels to resume execution (i.e., a barrier operation). The wait operation is accomplished through communication between threads. This communication can be supported by events that wake up the suspended thread or by the waiting thread polling on a memory location. When a thread polls, it wastes processing resources that could be used by the thread, which must finish its work to allow all threads to resume productive execution. The second or low priority queue 360 ​​allows the waiting thread to enter a low priority mode that will reduce the overhead of the polling thread. This is used to reduce the thread execution overhead of the polling thread, so that the thread that must complete the productive work consumes most of the available processing resources.

[0348] The configuration register is used to determine the number of high priority threads to be run for each low priority thread, such as Fig.10 442, multiplexer 444, and adder 446) until it is equal to zero (logic block 448), at which time the select input of the thread selection multiplexer 385 switches to select a thread from the second or low priority queue 360.

[0349] Thus, threads in the second priority queue 360 ​​are pulled from the queue and executed at a lower rate than threads in the first priority queue 355. Configuration registers (e.g., in thread memory 320) are used to determine the number of high priority threads to run for each low priority thread. A pair of instructions, ENP and ELP, are used to transition a thread from a first (or normal) priority to a second low priority (ELP), and from a second low priority to a first normal priority (ENP).

[0350] Fig.11 A block diagram of a representative embodiment and a representative data packet of a portion of a first interconnection network 150. In a representative embodiment, the first interconnection network 150 includes a network bus structure 152 (a plurality of wires or pipelines) wherein a first plurality of network lines 154 are dedicated to addressing (or routing) data packets (158) and are used to set up data paths through various crossbar switches, and a remaining second plurality of network lines 156 are dedicated to transmitting data packets (data loads, illustrated as from 1621 to 1622) on the paths established by the addressing pipelines (the first plurality of network lines 154). N A series or sequence of "N" data packets (containing operand data, variable parameters, results, etc.) is typically provided to or from each computing resource as channels, a first channel for receiving data and a second channel for transmitting data. A single first addressed (or routed) data packet (illustrated as addressed (or routed) data packet 1581) may be used to establish a route to a first designated destination and may be followed (typically for a number of clock cycles to allow switches to be set) by one or more data packets 162 to be transmitted to the first designated destination, up to a predetermined number of data packets 162 (e.g., up to N data packets). While the predetermined number of data packets 162 are being routed, another second addressed (or routed) data packet (illustrated as addressed (or routed) data packet 1582 may be transmitted and used to establish a route to a second designated destination for transmitting data destined for this second designated destination (illustrated as data packet 162 N+1 )'s other subsequent one or more data packets 162.

[0351] Fig.12 is a detailed block diagram of a representative embodiment of a data path control circuitry 395 of the HTP 300. As mentioned above, one or more of the HTPs 300 may also include a data path control circuitry 395 for controlling the size of accesses (e.g., memory 125 load requests) on the first interconnect network 150 to manage potential congestion, thereby providing adaptive bandwidth.

[0352] Application performance is often limited by the bandwidth available to the processor from memory. Performance limitations can be mitigated by ensuring that only data required by the application is brought into the HTP 300. The data path control circuitry 395 automatically (i.e., without user intervention) reduces the size of requests to main memory 125 to reduce utilization of the processor interface and memory 125 subsystem.

[0353] As mentioned above, the computing resources of the system 100 may have many applications using sparse data sets, with frequent access to small data segments distributed throughout the data set. As a result, if a considerable amount of data is accessed, most of it may not be used, thereby wasting bandwidth. For example, a cache line may be 64 bytes, but all of it will not be utilized. At other times, it would be beneficial to use all available bandwidth, such as for efficient power usage. The data path control circuit 395 provides dynamic adaptive bandwidth on the first interconnect network 150, adjusting the size of the data path load to optimize the performance of any given application, such as adjusting the data path load down to 8 to 32 bytes (as an example) based on the utilization of the receive (e.g., response) channel of the first interconnect network 150 returned to the HTP 300.

[0354] The data path control circuitry 395 monitors the utilization level on the first interconnect network 150 and as utilization increases reduces the size of the memory 125 load (i.e., read) requests from the network interface circuitry 335. In a representative embodiment, the data path control circuitry 395 performs a time-averaged weighting of the utilization levels of the response channels of the first interconnect network 150 (time-averaged utilization block 364). If utilization is above the threshold (and the load request size is greater than a minimum) after a fixed period of time (adjustment interval timer 362) using threshold logic circuit 366 (having multiple comparators 482 and selection multiplexers 484, 486), the size of the load request is reduced by load request access size logic circuit 368 (typically by a power of 2 (e.g., 8 bytes) from threshold logic circuit 366, using negative increment 492) so that: (a) fewer data packets 162 will be included in the series of data packets 162, thereby allowing bandwidth to be used for routing data packets to another location or for another process; or (b) memory 125 is utilized more efficiently (e.g., not requesting 64 bytes when only 16 bytes will be utilized). If utilization is below the threshold (and the load request size is less than a maximum) after a fixed period of time using threshold logic circuit 366, the size of the load request is increased by load request access size logic circuit 368, also typically by a power of 2 (e.g., 8 bytes), using positive increment 488. The minimum and maximum values ​​for the size of a load request are user configurable, however, the minimum size is typically the size of the load instruction issued (e.g., the maximum operand size of the HTP 300, such as 8 bytes), and the maximum size is the cache line size (e.g., 32 or 64 bytes). In an alternative embodiment, the data path control circuit 395 may be located at the memory controller 120 to accommodate bandwidth pressure from multiple HTPs 300.

[0355] Fig.13 Detailed block diagram of a representative embodiment of the system call circuitry 415 and host interface 115 of the HTP 300. The representative system 100 embodiment allows, for example, only user mode computing elements of the HTP 300 to perform system calls, breakpoints, and other privileged operations without running an operating system, so as to open files, print, etc. To do so, any of these system operations originate from the HTP 300 executing user mode instructions. The processor's instruction execution identifies that the processor must forward the request to the host processor 110 for execution. The system request from the HTP 300 has the form of a system call work descriptor packet sent to the host processor 110, and in response, the HTP 300 can receive a system call return work descriptor packet.

[0356] The system call work descriptor packet, assembled and transmitted by the data packet encoder circuit 380, contains a system call identifier (e.g., thread ID, core 305 number, virtual address indicated by the program counter, system call variable parameters or arguments (which are typically stored in general purpose registers 328), and return information. The packet is sent to the host interface 115 (SRAM FIFO 464), which writes and queues the system call work descriptor packet in a main memory queue, such as the DRAM FIFO 466 illustrated in the host processor 110 main memory, increments the write pointer, and the host interface 115 then further sends an interrupt to the host processor 110 for the host processor 110 to poll the memory for the system call work descriptor packet. The host processor's operating system accesses the queue (DRAM FIFO 466) entry, performs the requested operation and places the return work descriptor data in the main memory queue (DRAM FIFO 468), and may also signal the host interface 115. The host interface 115 monitors the return queue (DRAM FIFO 464). 468), and when the entry exists, moves the data to the output queue (SRAM output queue 472), formats a return work descriptor packet with the provided work descriptor data, and sends the return work descriptor packet to the HTP 300 that initiated the system call packet.

[0357] The packet decoder 375 of the HTP 300 receives the return work descriptor packet and places the return variable parameters in the general registers 328 as if the local processor (HTP 300) were operating alone. This transparent execution as viewed by an application running on the user mode HTP 300 results in the ability to use the same programming environment and runtime libraries used when the processor has a local operating system, and is highly suitable for a wide variety of scenarios using inserted breakpoints, such as programming software debugging.

[0358] However, the host interface 115 typically has limited FIFO space, which can be a problem when utilizing multiple HTPs 300, each with a large number of cores (e.g., 96), each of which can run a large number of threads (e.g., 32 / core). To avoid adding a large amount of memory to the host interface 115, the total number of system calls that can be submitted is limited using a system call credit mechanism for each HTP 300 and each processor core 305 within the HTP 300.

[0359] Each processor core 305 includes a first register 452 that maintains a first credit count as part of a system call circuitry 415. The system call circuitry 415 provided in accordance with the HTP 300 includes a second register 458 that includes a second credit count as a pool of available credits. When a system call work descriptor packet is generated, if there are sufficient available credits in the first register 452, then (multiplexer 454) the system call work descriptor packet may be selected and transmitted, and if not, the system call work descriptor packet may be queued in a system call work descriptor (system call) packet table 462, possibly with other system call work descriptor packets from other processor cores 305 of a given HTP 300. If there are sufficient available credits in the second register 458, providing an additional credit pool for bursts of system calls and shared among all processor cores 305 of the HTP 300, then the next system call work descriptor packet may be transmitted, otherwise it is kept in the table.

[0360] As those system call work descriptor packets are processed by the host interface 115 and read from the FIFO 464, the host interface 115 generates an acknowledgement back to the system call circuitry 415, causing the per-core credit count in register 456 to be incremented, which in turn may increment the first credit count in the first register 452 for each processor core 305.

[0361] Alternatively, register 456 may be utilized equivalently to first register 452, without requiring a separate first register 452 per core, and instead maintaining the first count in register 456, also per core 305. As another alternative, all system call work descriptor packets may be queued on a per core 305 basis in system call work descriptor packet table 462, and transmitted when a core has a sufficient first credit count in its corresponding register 456 or sufficient available credits in second register 458.

[0362] A mechanism for thread state monitoring is also provided to collect the state of the thread set running on the HTP 300 in hardware, which allows the programmer to have visibility to the work of the application. Through this feature, the host processor 110 can periodically access and store information for subsequent use, such as when generating a user distribution report. Through the visibility provided, the programmer can make changes to the application to improve its performance. For example, the host processor can transmit a monitoring request, and wherein in response to the monitoring request from the host processor, the control logic and thread selection circuit can be used to provide a program count or instruction to the host processor, thereby generating a command to the command queue, to copy and transmit all data from the thread control memory corresponding to the selected thread identifier to the host processor, thereby monitoring the thread state. For example, in response to the monitoring request from the host processor, the control logic and thread selection circuit can be used to directly generate a command to the command queue, so that the interconnection network can copy and transmit all data from the thread control memory corresponding to the selected thread identifier to the host processor, thereby monitoring the thread state.

[0363] All thread state changes can be monitored and statistics on the amount of time in each state maintained. The processor (110 or 300) collecting the statistics provides a means for a separate second processor (110 or 300) to access and store the data. The data is collected while the application is running so that a report can be provided to an application analyst showing the amount of time in each state reported on a periodic basis, which provides detailed visibility on the running application for subsequent use by the application analyst.

[0364] According to a representative embodiment that can be implemented in hardware or software, all information about threads is stored in various registers of thread memory 320, and can be copied and saved in another location on a regular basis. Counters can be used to capture the amount of time spent by any given thread in a selected state, such as a pause state. For example, the host processor 110 can record or capture the current state of all threads and thread counters (the amount of time spent in the state), or the difference (Δ) between the state and the count over time, and write it to a file or otherwise save it in memory. And, for example, a program or thread can be a barrier, where all threads must be completed before anything else can start, and it helps to monitor which threads are in what state when they pass through various barriers or when they change state. The illustrated code (below) is an example of an emulator code that will be executed as hardware or can be transferred to hardware:

[0365] InStateCount[N]-6b InStateTimeStamp[N]-64b InStateTotalTime[N]-64b

[0366]

[0367]

[0368] Many advantages of representative embodiments are apparent. Representative devices, systems, and methods provide a computing architecture capable of providing high performance and energy-efficient solutions for compute-intensive cores, for example, to compute Fast Fourier Transforms (FFTs) and Finite Impulse Response (FIR) filters for sensing, communication, and analytical applications, such as synthetic aperture radar, 5G base stations, and graph analytics applications, such as graph clustering using spectral techniques, machine learning, 5G networking algorithms, and large die codes, but not limited thereto.

[0369] As used herein, a "processor core" may be any type of processor core and may be implemented as one or more processor cores configured, designed, programmed, or otherwise adapted to perform the functionality discussed herein. As used herein, a "processor" 110 may be any type of processor and may be implemented as one or more processors configured, designed, programmed, or otherwise adapted to perform the functionality discussed herein. Processor 110 or 300, as the term processor is used herein, may include the use of a single integrated circuit ("IC"), or may include the use of multiple integrated circuits or other components connected, arranged, or grouped together, such as a controller, a microprocessor, a digital signal processor ("DSP"), an array processor, a graphics or image processor, a parallel processor, a multi-core processor, a custom IC, an application specific integrated circuit ("ASIC"), a field programmable gate array ("FPGA"), an adaptive computing IC, associated memory (e.g., RAM, DRAM, and ROM), and other ICs and components (whether analog or digital). Thus, as used herein, the term processor or controller should be understood to equivalently mean and include a single IC, or an arrangement of custom ICs, ASICs, processors, microprocessors, controllers, FPGAs, adaptive computing ICs, or some other grouping of integrated circuits that together with associated memory, such as microprocessor memory or additional RAM, DRAM, SDRAM, SRAM, MRAM, ROM, FLASH, EPROM, or EEPROM, perform the functions discussed herein. 2PROM. As discussed herein, the processor 110 or 300 and associated memory may be adapted or configured (via programming, FPGA interconnection, or hard wiring) to perform the method of the present invention. For example, the method may be programmed and stored in the processor 300 through its associated memory (and / or memory 125) and other equivalent components as a program instruction or other code set (or equivalent configuration or other program) for continuous execution when the processor 110 or 300 is operational (i.e., powered on and functioning). Equivalently, when the processor 300 may be implemented in whole or in part as an FPGA, a custom IC, and / or an ASIC, the FPGA, custom IC, or ASIC may also be designed, configured, and / or hardwired to implement the method of the present invention. For example, the processor 110 or 300 may be implemented as an arrangement of analog and / or digital circuits, controllers, microprocessors DSPs, and / or ASICs, collectively referred to as "processors" or "controllers", which are hardwired, programmed, designed, adapted, or configured to implement the method of the present invention, including possibly in conjunction with the memory 125 to implement the method of the present invention.

[0370] Depending on the selected embodiment, memory 125 may include a data repository (or database) and may be implemented in any number of forms, including implementation in any computer or other machine-readable data storage medium, memory device, or other storage or communication device for information storage or communication now known or that becomes available in the future, including but not limited to a memory integrated circuit ("IC"), or a memory portion of an integrated circuit (e.g., a processor or resident memory within a processor IC), whether volatile or non-volatile, removable or non-removable, including but not limited to RAM, FLASH, DRAM, SDRAM, SRAM, MRAM, FeRAM, ROM, EPROM, or E-RAM. 2 The memory 125 may be a PROM, or any other form of memory device, such as a magnetic hard drive, an optical drive, a disk or tape drive, a hard drive, other machine-readable storage or memory medium, such as a floppy disk, CDROM, CD-RW, digital versatile disk (DVD) or other optical storage, or any other type of memory, storage medium, or data storage device or circuit that is or becomes known. The memory 125 may be used to store various lookup tables, parameters, coefficients, other information and data, programs or instructions (of the software of the present invention), and other types of tables, such as database tables.

[0371] As indicated above, the processor 110 or 300 is hardwired or programmed using the software and data structures of the present invention, for example, to perform the methods of the present invention. Thus, the systems and related methods of the present invention include various instructions that can be implemented as software that provides such programming or other instructions discussed above (e.g., instruction sets and / or metadata implemented in a non-transitory computer-readable medium). In addition, metadata can also be used to define various data structures of a lookup table or database. By way of example but not limitation, such software can be in the form of source or object code. The source code can also be compiled into some form of instructions or object code (including assembly language instructions or configuration information). The software, source code, or metadata of the present invention can be implemented as any type of code, such as C, C++, Matlab, SystemC, LISA, XML, Java, Brew, SQL and its variants (e.g., SQL 99 or proprietary versions of SQL), DB2, Oracle, or any other type of programming language that performs the functionality discussed herein, including various hardware definition or hardware modeling languages ​​(e.g., Verilog, VHDL, RTL) and resulting database files (e.g., GDSII). Thus, "construction," "program construct," "software construct," or "software" as used equivalently herein means and refers to any programming language of any class with any syntax or feature markers that provides or can be interpreted as providing the specified associated functionality or method when instantiated or loaded into a processor or computer and executed, including, for example, the processor 300.

[0372] The software, metadata or other source code, and any resulting bit files (object code, databases, or lookup tables) of the present invention may be embodied in any tangible, non-transitory storage medium (e.g., any of a computer or other machine-readable data storage medium) as computer-readable instructions, data structures, program modules, or other data, such as discussed above with respect to memory 125 (e.g., a floppy disk, CDROM, CD-RW, DVD, magnetic hard drive, optical drive, or any other type of data storage device or medium, as mentioned above).

[0373] The present disclosure should be considered as an example of the principles of the present invention, and it is not intended that the present invention be limited to the specific embodiments shown. In this regard, it should be understood that the application of the present invention is not limited to the construction details and component arrangements set forth above and below, shown in the drawings, or described in the examples. The systems, methods, and devices according to the present invention can have other embodiments and can be practiced and implemented in various ways.

[0374] Although the present invention has been described with respect to its specific embodiments, these embodiments are merely illustrative and do not limit the present invention. In the description herein, many specific details are provided, such as examples of electronic components, electronic and structural connections, materials, and structural changes, to provide a thorough understanding of embodiments of the present invention. However, those skilled in the relevant art will recognize that embodiments of the present invention may be practiced without one or more of the specific details, or with other devices, systems, assemblies, components, materials, parts, etc. In other cases, in order to avoid confusing various aspects of embodiments of the present invention, well-known structures, materials, or operations are not specifically shown or described in detail. In addition, various drawings are not drawn to scale and should not be considered to be restrictive.

[0375] References throughout this specification to "one embodiment," "an embodiment," or a specific "embodiment" mean that a particular feature, structure, or characteristic described in conjunction with the embodiment is included in at least one embodiment of the invention and not necessarily in all embodiments, and further, does not necessarily refer to the same embodiment. Furthermore, the particular features, structures, or characteristics of any specific embodiment of the invention may be combined in any suitable manner and in any appropriate combination with one or more other embodiments (including the use of selected features without the corresponding use of other features). Furthermore, many modifications may be made to adapt specific applications, situations, or materials to the basic scope and spirit of the invention. It should be understood that other variations and modifications of the embodiments of the invention described and illustrated herein are possible in light of the teachings herein and are to be considered part of the spirit and scope of the invention.

[0376] For the purpose of describing the numerical ranges herein, each inserted value with the same precision between them is explicitly included. For example, for the range 6 to 9, the values ​​7 and 8 are included in addition to 6 and 9, and for the range 6.0 to 7.0, the values ​​6.0, 6.1, 6.2, 6.3, 6.4, 6.5, 6.6, 6.7, 6.8, 6.9 and 7.0 are explicitly included. In addition, each inserted sub-range in the range in any combination is contemplated, which is within the scope of the present disclosure. For example, for the range 5 to 10, the sub-ranges 5 to 6, 5 to 7, 5 to 8, 5 to 9, 6 to 7, 6 to 8, 6 to 9, 6 to 10, 7 to 8, 7 to 9, 7 to 10, 8 to 9, 8 to 10 and 9 to 10 are contemplated, which are within the scope of the disclosed range.

[0377] It will also be appreciated that one or more of the elements depicted in the drawings may also be implemented in a more independent or integrated manner, or even removed or rendered inoperable in certain circumstances, as may be useful depending on the specific application. The integrated formation of combinations of components is also within the scope of the present invention, especially for embodiments in which the separation or combination of discrete components is unclear or difficult to discern. In addition, the term "coupled" as used herein, including its various forms ("coupling" or "couplable"), refers to and includes any direct or indirect electrical, structural or magnetic coupling, connection or attachment, or the adaptation or ability for such direct or indirect electrical, structural or magnetic coupling, connection or attachment, including integrally formed components and components coupled via or through another component.

[0378] By signal, in this context is meant a parameter that "represents" or "is representative of" a given metric, where a metric is a measure of the state of at least part of a regulator or its input or output. A parameter is considered to represent a metric if the parameter is sufficiently directly related to the metric that adjusting the parameter satisfactorily adjusts the metric. A parameter may be considered an acceptable representation of a metric if it represents multiple or a portion of a metric.

[0379] In addition, any signal arrows in the accompanying drawings / diagrams should be considered as merely exemplary and non-restrictive, unless otherwise specifically noted. The combination of the components of the steps will also be considered within the scope of the present invention, especially when the ability to separate or combine is unclear or foreseeable. The disjunctive term "or" as used herein and throughout the appended claims is generally intended to mean "and / or", unless otherwise indicated, it has both combined and disjunctive meanings (and is not limited to the "XOR" meaning). As used in the description herein and throughout the appended claims, unless the context clearly stipulates otherwise, "a / an" and "said" include multiple references. And, unless the context clearly stipulates otherwise, as used in the description herein and throughout the appended claims, the meaning of "in" includes "in" and "on".

[0380] The foregoing description of the illustrated embodiments of the invention, including the description in the Summary of the Invention or the Abstract, is not intended to be exhaustive or to limit the invention to the precise form disclosed herein. Based on the foregoing, it will be observed that a large number of changes, modifications, and substitutions can be expected and implemented without departing from the spirit and scope of the novel concepts of the invention. It should be understood that it is not intended to limit the particular methods and apparatus shown herein or that limitations should not be inferred. Of course, the present invention is intended to encompass all such modifications as falling within the scope of the claims by the appended claims.

Claims

1. A processor capable of being coupled to an interconnect network (150) in a system having a memory (125) and a host processor (110), comprising: A plurality of processor cores (305), each processor core of the plurality of processor cores being configured to execute a plurality of instructions; as well as A plurality of core control circuits (310), each of the plurality of core control circuits being coupled to a corresponding processor core of the plurality of processor cores, each core control circuit comprising: interconnect network interface circuitry (335) capable of coupling to the interconnect network to receive operation descriptor packets and to receive and decode monitoring requests from the host processor; a thread control memory (320) comprising a plurality of registers including a thread identifier pool register (322) storing a plurality of thread identifiers, a program counter register (326) storing an initial program counter, a data cache register (346) storing cached data, and a general register (328) storing at least one received first variable parameter and non-cached data; an execution queue (345) coupled to the thread control memory; a network command queue (390) storing one or more request or return messages for transmission by the interconnection network interface circuitry over the interconnection network; and a control logic and thread selection circuit (330) coupled to the execution queue and the network command queue, the control logic and thread selection circuit being operable to, in response to receiving the work descriptor packet having the initial program counter and the at least one received first variable parameter, automatically assign an available thread identifier of the plurality of thread identifiers to a corresponding execution thread of a plurality of execution threads, automatically place each assigned thread identifier of the plurality of thread identifiers in the execution queue, and automatically and periodically cycle through selecting each of the plurality of thread identifiers in the execution queue for execution by the corresponding processor core of a single instruction of the corresponding execution thread of the plurality of instructions, the corresponding processor core using The at least one received first variable parameter stored in the general register automatically starts execution of the single instruction corresponding to the initial program count, the control logic and thread selection circuit are further used to automatically and repeatedly loop through the selection of each of the plurality of thread identifiers in the execution queue so that when the thread identifier of each execution thread is in the execution queue, the corresponding processor core executes the single instruction of the execution thread, and is used to generate a command to the network command queue in response to the monitoring request from the host processor to copy and transfer all data corresponding to a selected thread identifier of the plurality of thread identifiers from the thread control memory to the host processor, thereby monitoring thread status.

2. The processor of claim 1 , wherein in response to the monitoring request from the host processor, the control logic and thread selection circuit (330) is used to provide a program counter or instruction to the corresponding processor core (305), thereby generating the command to the network command queue to copy and transfer all data corresponding to the selected thread identifier from the thread control memory to the host processor, thereby monitoring the thread state.

3. The processor of claim 1 , wherein in response to the monitoring request from the host processor, the control logic and thread selection circuit (330) is used to directly generate the command to the network command queue so that the interconnect network interface circuit system copies and transmits all data corresponding to the selected thread identifier from the thread control memory to the host processor to monitor the thread state.

4. The processor of claim 1 , wherein the interconnect network interface circuitry ( 335 ) is operable to decode the received work descriptor data packet into the initial program counter of the execution thread and the at least one received first variable parameter, and to generate a return work descriptor packet in response to the corresponding processor core executing a return instruction.

5. The processor of claim 2, wherein the control logic and thread selection circuit (330) is further configured to automatically initiate execution of the single one of the plurality of instructions by the corresponding processor core in response to a received event data packet.

6. The processor of claim 5, wherein the interconnect network interface circuitry (335) is further for receiving an event data packet, and for decoding the received event data packet into an event identifier and a received second variable parameter.

7. The processor of claim 2, wherein the interconnect network interface circuitry (335) is further for storing the initial program counter and the received first variable parameter in the thread control memory for each of the plurality of execution threads using the assigned thread identifier as an index into the thread control memory.

8. The processor of claim 2, wherein the interconnect network interface circuitry (335) is further operable to generate and receive point-to-point event data messages and broadcast event data messages.

9. A processor according to claim 1, wherein the corresponding processor core (305) is used to execute a shred creation instruction, and wherein the corresponding core control circuit is further used to generate one or more work descriptor data packets to another processor or a hybrid thread fabric circuit for executing corresponding multiple execution threads, each of the one or more work descriptor data packets having a program counter and one or more variable parameters or memory addresses.

10. The processor of claim 9, wherein the control logic and thread selection circuit (330) is further operable to reserve a predetermined amount of memory space in the thread control memory to store return variable parameters.

11. The processor of claim 1 , wherein the control logic and thread selection circuit (330) is further operable to determine an event number corresponding to a received event data packet and to use an event mask stored in an event mask register to respond to the received event data packet.

12. The processor of claim 1, wherein the core control circuit (310) further comprises: Network Response Storage (325); as well as An instruction cache (340) is coupled to the control logic and thread selection circuitry.

13. The processor of claim 1 , wherein the control logic and thread selection circuit ( 330 ) is further configured to assign an active state or a paused state to each assigned thread identifier of the plurality of thread identifiers, and to return the active state thread identifier to the execution queue for the corresponding processor core to continue executing a single instruction for a duration during which the active state remains unchanged, and to pause thread execution by not returning the paused state thread identifier to the execution queue until the state of the thread identifier returns to active.

14. The processor of claim 1, wherein the thread control memory (320) further comprises registers selected from the group consisting of: a thread status register (324); a pending shred return count register (332); a thread return buffer or register (336); a return variable parameter linked list register; a custom atomic transaction identifier register (338); an event received mask register (342); an event status register (344); and combinations thereof.

15. The processor of claim 13, wherein the control logic and thread selection circuitry (330) is further operable to assign a halt state to a thread identifier of the plurality of thread identifiers in response to the corresponding processor core executing a memory load instruction or a memory store instruction.

16. The processor of claim 13 , wherein the control logic and thread selection circuit ( 330 ) are further configured to change the state of a thread identifier among the multiple thread identifiers from the paused state to the valid state in response to a received event data packet to resume execution of a single instruction of the corresponding execution thread by the corresponding processor core, or to change the state of a thread identifier among the multiple thread identifiers from the paused state to the valid state in response to an event number of a received event data packet to resume execution of a single instruction of the corresponding execution thread by the corresponding processor core.

17. The processor of claim 1, wherein the control logic and thread selection circuit (330) is further configured to terminate execution of a selected thread and return a corresponding thread identifier of the selected thread from the plurality of thread identifiers to the thread identifier pool register (322) in response to the corresponding processor core executing a return instruction.

18. The processor of claim 17, wherein the control logic and thread selection circuitry (330) is further for clearing the register of the thread control memory (320) indexed by the corresponding assigned thread identifier of the selected thread in response to the corresponding processor core executing a return instruction.

19. The processor of claim 1, wherein the execution queue (345) further comprises: a first priority queue (355) storing one or more thread identifiers of first priority execution threads; as well as A second priority queue (360) stores one or more thread identifiers of second priority execution threads.

20. The processor of claim 19, wherein the control logic and thread selection circuitry further comprises: A thread selection control circuit system (405) is coupled to the execution queue, the thread selection control circuit system is used to select a thread identifier from the one or more thread identifiers of a first priority execution thread from the first priority queue at a first frequency for the processor core to execute the next single instruction of the corresponding execution thread, and is used to select a thread identifier from the one or more thread identifiers of a second priority execution thread from the second priority queue at a second frequency for the processor core to execute the next single instruction of the corresponding execution thread, the second frequency being lower than the first frequency.

21. A processor capable of being coupled to an interconnect network (150) in a system having a memory (125) and a host processor (110), comprising: A plurality of processor cores (305), each processor core of the plurality of processor cores being configured to execute a plurality of instructions; as well as A plurality of core control circuits (310), each of the plurality of core control circuits being coupled to a corresponding processor core of the plurality of processor cores, each core control circuit comprising: interconnect network interface circuitry (335) capable of coupling to the interconnect network to receive a work descriptor packet, decode the received work descriptor packet into an initial program count of an execution thread and received variable parameters, and receive and decode a monitor request from the host processor; a thread control memory (320) coupled to the interconnect network interface circuitry and comprising a plurality of registers, the plurality of registers comprising a thread identifier pool register (322) storing a plurality of thread identifiers, a thread status register (324), a program counter register (326) storing the initial program count, a data cache register (346) storing cached data, and general purpose registers (328) storing the received variable parameters and non-cached data; an execution queue (345) coupled to the thread control memory; a network command queue (390) storing one or more request or return messages for transmission by the interconnection network interface circuitry over the interconnection network; a control logic and thread selection circuit (330) coupled to the execution queue, the network command queue and the thread control memory, the control logic and thread selection circuit being used to automatically assign available thread identifiers from the plurality of thread identifiers to corresponding execution threads from the plurality of execution threads in response to receiving the work descriptor packet, automatically place each assigned thread identifier from the plurality of thread identifiers in the execution queue, automatically cyclically select each of the plurality of thread identifiers in the execution queue for execution of a single instruction of the corresponding execution thread from the plurality of instructions by the corresponding processor core, and automatically start execution of the corresponding processor core corresponding to the initial program counter using the received variable parameters stored in the general purpose registers the single instruction of the control logic and thread selection circuitry being further operable to automatically and repeatedly loop through the selection of each of the plurality of thread identifiers in the execution queue so that when the thread identifier of each execution thread is in the execution queue, the corresponding processor core executes a single instruction of the execution thread, accesses the thread control memory using the assigned thread identifier as an index to select the program counter of the corresponding execution thread, and generates a command to the network command queue in response to the monitoring request from the host processor so that the interconnect network interface circuitry copies and transmits all data corresponding to a selected thread identifier of the plurality of thread identifiers from the thread control memory to the host processor to monitor thread status; and An instruction cache (340) is coupled to the corresponding processor core and the control logic and thread selection circuitry to receive the initial program count and provide a corresponding single instruction of the plurality of instructions to the corresponding processor core for execution.

22. A processor capable of being coupled to an interconnect network (150) in a system having a memory (125) and a host processor (110), comprising: A plurality of processor cores (305), each processor core of the plurality of processor cores being configured to execute a plurality of instructions; as well as A plurality of core control circuits (310), each of the plurality of core control circuits being coupled to a corresponding processor core of the plurality of processor cores, each core control circuit comprising: interconnect network interface circuitry (335) capable of coupling to the interconnect network to receive a call work descriptor packet, decode the received work descriptor packet into an initial program count of an execution thread and received variable parameters, encode the work descriptor packet for transmission to other processing elements, and receive and decode a monitor request from the host processor; a thread control memory (320) coupled to the interconnect network interface circuitry and comprising a plurality of registers, the plurality of registers comprising a thread identifier pool register (322) storing a plurality of thread identifiers, a thread status register (324), a program counter register (326) storing the initial program count, a data cache register (346) storing cached data, and general purpose registers (328) storing the received variable parameters and non-cached data; an execution queue (345) coupled to the thread control memory; a network command queue (390) storing one or more request or return messages for transmission by the interconnection network interface circuitry over the interconnection network; a network response memory (325) coupled to the interconnect network interface circuitry; an instruction cache (340) coupled to the corresponding one of the plurality of processor cores to receive the program counter and provide a corresponding single one of the plurality of instructions to the corresponding processor core for execution; and a control logic and thread selection circuit (330) coupled to the execution queue, the network command queue, the thread control memory and the instruction cache, the control logic and thread selection circuit being used to automatically assign available thread identifiers and initial valid states from the plurality of thread identifiers to corresponding execution threads from a plurality of execution threads in response to receiving the work descriptor packet, automatically place each assigned thread identifier from the plurality of thread identifiers in the execution queue, automatically cyclically select each of the thread identifiers for execution of a single instruction of the corresponding execution thread from the plurality of instructions by the corresponding processor core, the corresponding processor core automatically starting execution of all instructions corresponding to its initial program count using the received variable parameters stored in the general purpose registers, The control logic and thread selection circuit are further used to automatically and repeatedly loop through the selection of each of the plurality of thread identifiers in the execution queue so that the corresponding processor core executes a single instruction of the execution thread when the thread identifier of each execution thread is in the execution queue, access the thread control memory using the assigned thread identifier as an index to select the initial program count of the corresponding execution thread, and generate instructions to the corresponding processor core (305) in response to the monitoring request from the host processor to generate a command to the command queue to copy and transfer all data corresponding to the selected thread identifier from the thread control memory to the host processor to monitor thread status.

Images