Superscalar field programmable gate array (FPGA) vector processor

By designing a vector processor with a vector register file paradigm on an FPGA and combining it with the time and space SIMD paradigm, the challenge of building a high-performance vector processor on an FPGA was solved, enabling high-frequency and efficient neural network operations.

CN121844294APending Publication Date: 2026-04-10MICROSOFT TECHNOLOGY LICENSING LLC
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202480058782.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2023-10-23
Filing Date
2024-10-08
Publication Date
2026-04-10

AI Technical Summary

Technical Problem

Building high-performance vector processors on FPGAs presents challenges. Existing solutions typically focus on accelerating matrix multiplication operations, which limits the performance of neural network accelerators. Furthermore, the vector memory-memory paradigm and dataflow paradigm require changes to the software stack, making implementation difficult.

Method used

By adopting the vector register file paradigm, a vector processor is designed on the FPGA, using multiple vector processor channels and execution units. Combining the time and space SIMD paradigm, multi-port access is achieved through the vector register file library, simulating multiple execution units being busy, thereby reducing execution time and resource requirements.

Benefits of technology

It achieves high-frequency vector processing, reduces the FPGA resources required for data processing, improves the performance of neural network operations, supports superscalar execution, and optimizes resource utilization.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN121844294A_ABST
    Figure CN121844294A_ABST
Patent Text Reader

Abstract

The present disclosure relates to a vector processor implemented on programmable hardware (e.g., a field programmable gate array (FPGA) device). The vector processor includes a plurality of vector processor channels, wherein each vector processor channel includes a vector register file having a plurality of register file libraries and a plurality of execution units. Implementations described herein include features for optimizing resource availability on programmable hardware units and enabling superscalar execution when coupled with Time Single Instruction Multiple Data (SIMD).
Need to check novelty before this filing date? Find Prior Art

Description

BACKGROUND

[0001] In recent years, there has been an increase in the use of programmable hardware to perform various computing tasks. Now, many computing applications commonly use programmable block arrays to perform various tasks. The logic, memory, and arithmetic blocks of these programmable blocks provide a useful alternative to application specific integrated circuits with more specialized or specific sets of tasks. For example, field programmable gate arrays (FPGAs) provide programmable blocks that can be individually programmed and provide significant flexibility for performing various tasks.

[0002] Neural networks, particularly large language models (LLMs), are becoming increasingly popular in data centers. These networks include matrix multiplication and other operations. Current solutions often focus on accelerating the matrix multiplication operations, while other operations can limit the performance of neural network accelerators. Vector processing is a natural choice for accelerating these operations.

[0003] However, building a high-performance vector processor on an FPGA is challenging due to the limited number of building blocks available on the FPGA. For example, because programmable hardware units typically have rigid element structures, such as lookup tables, adder chains, and specialized memories that follow different protocol and timing constraint sets, implementing a typical processor on programmable hardware is often inefficient and results in undesirable latency for issuing and processing various instructions. Thus, current solutions have changed the software abstraction for vector processors by using a vector memory-memory paradigm or a dataflow paradigm, which requires changes to the software stack including compilers, making adoption challenging. SUMMARY

[0004] This summary is provided to introduce some concepts in a simplified form that will be further described below in the detailed description. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter.

[0005] Some embodiments relate to a processor implemented on programmable hardware. The processor includes a vector controller configured to receive an instruction for execution on the processor and provide control signals to a plurality of vector processor lanes; and a plurality of vector processor lanes, where each vector processor lane of the plurality of vector processor lanes includes: a vector register file having a plurality of register file banks to store data; a load unit having a plurality of first-in-first-out queues, the load unit to load data into the plurality of register file banks; a plurality of execution units configured to perform operations on data in each register file bank of the plurality of register file banks and execute instructions from an instruction issue queue; and a store unit having a plurality of first-in-first-out queues, the store unit to read data from each register file bank of the plurality of register file banks.

[0006] Some embodiments relate to a method. The method includes receiving an instruction for execution using a vector processor. The method includes providing, in response to determining to issue the instruction, a control signal for the instruction to each vector processor lane of a plurality of vector processor lanes in the vector processor, where each vector processor lane performs operations on a subset of data in parallel. The method includes performing, using a plurality of execution units in each vector processor lane, operations on data in a plurality of register file banks within each vector processor lane.

[0007] Additional features and advantages will be set forth in the description that follows, and in part will be apparent from the description, or can be learned by practice of the teachings herein. Features and advantages of the present disclosure will be realized and attained by the instrumentalities and combinations particularly pointed out in the appended claims. Features of the present disclosure will become more fully apparent from the following description and appended claims, or can be learned by the practice of the present disclosure as set forth hereinafter. BRIEF DESCRIPTION OF DRAWINGS

[0008] Figure 1 FIGURE illustrates an example hardware environment including an example vector processor on a programmable hardware component, in accordance with embodiments of the present disclosure.

[0009] Figure 2 FIGURE illustrates an example vector processor, in accordance with embodiments of the present disclosure.

[0010] Figure 3 FIGURE illustrates an example vector processor lane, in accordance with embodiments of the present disclosure.

[0011] Figure 4 FIGURE illustrates an example timing diagram for a register file read by a vector processor, in accordance with embodiments of the present disclosure.

[0012] Figure 5An example timing diagram is shown for a write to a register file library performed by a vector processor according to an embodiment of the present disclosure.

[0013] Figure 6 An example vector controller according to an embodiment of the present disclosure is illustrated.

[0014] Figure 7 An example method executed by a vector processor according to an embodiment of the present disclosure is illustrated.

[0015] Figure 8A and Figure 8B An example timing diagram of a read from a register file library performed according to an embodiment of the present disclosure is illustrated.

[0016] Figure 9A and Figure 9B An example timing diagram of writing to a register file library according to an embodiment of the present disclosure is illustrated.

[0017] Figure 10A and Figure 10B An example timing diagram of a loading unit according to an embodiment of the present disclosure is shown.

[0018] Figure 11A and Figure 11B An example timing diagram of a storage unit according to an embodiment of the present disclosure is illustrated. Detailed Implementation

[0019] This disclosure generally relates to vector processor coverage on programmable hardware units, such as FPGAs. FPGAs offer a limited variety of logic, arithmetic, and memory primitives, rigid interconnect architectures, and fixed structural topologies. Achieving both high clock frequencies and high resource utilization is challenging and requires careful consideration of device primitives and constraints when designing microarchitectures. Current device vendor design guidelines call for aggressively pipelined logic implementations. To avoid costly routing delays, both data paths and control logic should be feedforward, pause-free, and low-fanout, with minimized logic depth and sparse reset usage. Connections between logic placed in distant areas of the device must be pipelined using one or more feedforward stages. Placement constraints and high-latency connection requirements must be considered early in the design process and frequently to ensure that the final design can be efficiently placed and routed at high frequencies on the target device. To minimize resource utilization and logic depth at each pipeline stage, and therefore the critical path, all modules should be designed and implemented to be efficiently composed of available building blocks from the device.

[0020] Even minor deviations from FPGA device primitive constraints, such as exceeding the number of inputs to a single lookup table (LUT), exceeding the fan-out drive strength of any module output, or using memory control signals not natively implemented in available SRAM, can significantly increase resource utilization and critical path latency. In high-density designs with high resource utilization and high wiring congestion, tiny microarchitectural inefficiencies in the replication logic can rapidly reduce the design's maximum frequency.

[0021] Neural networks, particularly Large Language Models (LLMs), are becoming increasingly popular in data centers. These networks involve matrix multiplication and other operations. Current solutions typically focus on accelerating matrix multiplication operations, while other operations can limit the performance of neural network accelerators. Vector processing is a natural choice for accelerating these operations.

[0022] However, building high-performance vector processors on FPGAs is challenging due to the limited number of building blocks available on FPGAs. Current solutions have shifted the processing paradigm to a vector memory-memory paradigm or dataflow paradigm. The changes required to the software stack, including application code, compiler, and firmware, make adopting current solutions challenging.

[0023] Most FPGA SRAMs provide only one read port and one write port per SRAM. The systems and methods of this disclosure provide vector processors on programmable hardware units (e.g., FPGAs) using a vector register file paradigm, where instructions manipulate data stored in registers of a register file, and the results of the operations are also written to the register file. Instructions can be provided to move data between the register file and memory. The systems and methods of this disclosure use a library-based vector register file, where each library is accessed sequentially (read from or written to), to simulate a multi-port vector register file that can keep multiple execution units busy. In some embodiments, the execution units use a combination of temporal and spatial single-instruction multiple-data (SIMD) paradigms to process instructions over multiple cycles and utilize multiple parallel execution units. In some embodiments, the execution units use temporal SIMD to process instructions over multiple cycles.

[0024] The implementations described herein include features and functionalities for optimizing resource availability on programmable hardware units and for implementing multi-release, superscalar execution when coupled with a time-based SIMD paradigm for vector execution using a vector register file processing paradigm. This disclosure includes numerous practical applications that provide benefits and / or solve problems associated with vector processors in programmable hardware unit environments. Examples of these applications and benefits will be discussed in further detail below.

[0025] A vector processor refers to a processing architecture that includes scalar units for handling the fetching, decoding, and forwarding of instructions. A vector processor also includes vector units for issuing and executing instructions. A vector unit may include a control unit and a data path. The data path includes one or more channels.

[0026] The vector processor may also include multiple channels with a feedforward design configured to receive control signals to drive the channels to perform various vector operations. Each channel includes a library of vector register files (e.g., multiple register file libraries), wherein each register file library is accessed sequentially (read from or written to). Each channel may include multiple execution units thereon to perform operations on data in the multiple register file libraries or a set of related operations. All channels in the vector processor operate synchronously with each other and are controlled using a single vector control unit. In some embodiments described herein, the vector processor issues instructions to the execution units to perform one or more operations according to an instruction set.

[0027] Vector processors use temporal SIMD to time-division multiplex the register file library within the vector processor to keep multiple hardware execution units busy. Spatial SIMD refers to distributing the execution of a single instruction across different execution units in space. For example, to implement spatial SIMD, a vector processor uses multiple execution units configured to perform the same function and are stacked in parallel configuration with a shared control unit. The vector processor uses multiple execution units to execute the same instruction on different slices of data. The elements of the vector are distributed across the channels of the vector processor, allowing each channel to perform computations on different elements of the vector. Temporal SIMD refers to executing a single instruction on multiple data in the time domain. To implement temporal SIMD in a vector processor, the vector processor executes different vector elements at different pipeline stages. Each instruction operates on a fixed number of vector elements, which are accessed sequentially in time. The duration of instruction execution is called the vector chime (chained execution time).

[0028] The vector processor also includes a vector processor controller that receives instructions for execution by at least one execution unit of the vector processor. In some embodiments, upon receiving an instruction, the vector processor controller places the instruction in a release queue that classifies instructions before they are released. The vector processor controller also applies one or more hazard trackers to the released instructions to determine whether the instructions can be safely released. If the hazard tracker(s) identify a potential problem, the vector processor controller can reset the instruction to the earliest instruction that was not successfully released.

[0029] In some implementations, the software used by the vector processor leverages the heterogeneity present in the hardware execution units to reduce both the execution time and FPGA resources required for data processing in the vector processor. In some implementations, the software integrates multiple stages of a multi-stage operation, allowing the vector processor to process multiple stages simultaneously, averaging the hardware requirements of the multi-stage operation and allowing more execution units to be busy at the same time.

[0030] One technical advantage of the systems and methods disclosed herein includes the use of a standard vector register file processing paradigm in the vector processor. Another technical advantage of the systems and methods disclosed herein includes high-frequency vector processor implementation. Another technical advantage of the systems and methods disclosed herein is the reduction in execution time when the vector processor processes data. Another technical advantage of the systems and methods disclosed herein includes reduced FPGA resources required for data processing. Another technical advantage of the systems and methods disclosed herein includes configurability of the vector processor design. Another technical advantage of the disclosure includes superscalar execution (execution of more than one instruction simultaneously) on the vector processor. Superscalar execution allows for reduced execution time and reduced FPGA resources.

[0031] As illustrated in the foregoing discussion, this disclosure uses a variety of terms to describe the features and advantages of the system described herein. Additional details regarding the meanings of some example terms are now provided.

[0032] For example, as used herein, a “hazard” is an instance of a problem in an instruction pipeline where an instruction cannot be executed without risking incorrect computation results. A hazard can refer to a data hazard, such as when data is not yet available, or when available data may be overwritten before it is used. An example data hazard includes the read-after-write hazard. Data reads must reflect data that has been written. Another example data hazard includes the write-after-read hazard. Writes must not overwrite data before it is read. Another example data hazard includes the write-after-write hazard. Data from a first write should not overwrite data from a second write. Another example type of data hazard mentioned herein is a memory hazard. As another example, a hazard can refer to a structural hazard, such as a hazard caused by the lack of availability of hardware resources (e.g., execution units or memory ports). In some implementations, an availability hazard refers to any hazard caused by the lack of availability of data and / or hardware resources.

[0033] As used herein, a “register file” refers to a hardware unit to which data can be written or read according to the embodiments described herein. In some embodiments, a register file includes multiple register file libraries on which data instances can be written and stored during one or more cycles (e.g., clock cycles). In some embodiments, data is copied across register file libraries to simulate multiple input / output ports. A register file contains multiple registers.

[0034] Additional details about the vector processor example will now be provided in conjunction with the illustrative drawings depicting the example implementation.

[0035] Now for reference Figure 1 The illustration shows an example hardware environment 100 for hardware component 104. In some embodiments, hardware component 104 is a neural network accelerator, which includes a matrix multiplication unit, a vector processor 102, on-chip memory, and a control plane that coordinates the execution of the accelerator. Example operations performed by hardware component 104 include SoftMax, layer normalization, and Gaussian error linearization unit (GELU).

[0036] In some implementations, vector processor 102 refers to a vector processor on hardware component 104, such as an FPGA component on an FPGA device having multiple components. As an example, vector processor 102 on hardware component 104 refers to a vector processor implemented on an FPGA overlay and optimized for utilizing various logic modules on the FPGA device. In some implementations, the features described herein in conjunction with example vector processors can be similarly applied to processors using digital signal processing blocks, memories for storage, and other processing components.

[0037] Vector processor 102 may include multiple components thereon, including, for example, a scalar unit 106 and a vector unit 108, which includes a vector controller 110 and multiple vector processor channels 112. The vector processor channels 112 operate synchronously with each other. In some embodiments, a vector processor channel 112 receives a portion of a vector, and each vector processor channel 112 performs the same operation on that portion of the vector in parallel. Additional information about each of these components 106 to 112 will be discussed in further detail below.

[0038] Now for reference Figure 2The illustration shows an example vector processor 102. Vector processor 102 includes a scalar unit 106. Scalar unit 106 refers to the hardware of vector processor 102 that handles tasks related to fetching and decoding instructions and executing scalar instructions. In some embodiments, scalar unit 106 forwards vector instructions to vector unit 108 for execution. In some embodiments, scalar unit 106 expands instructions before forwarding them to vector unit 108. In some embodiments, scalar unit 106 includes multiple components, including a scalar register file, memory unit, arithmetic unit, multiplexer, decoder, and other hardware.

[0039] Vector processor 102 also includes vector controller 110. Vector controller 110 may refer to the control unit of vector processor 102 and is configured to send control commands to a plurality of vector processor channels 112 (e.g., vector processor channels 1120, 1121 to 112). m (in m A pipeline of positive integers. Vector controller 110 manages the forwarding of instructions to the corresponding vector processor channel 112. In some embodiments, vector controller 110 manages an instruction issuance queue for instructions. In some embodiments, vector controller 110 includes multiple hazard trackers configured to track and detect different types of hazards associated with the issuance and forwarding of instructions by vector controller 110. Examples of hazard trackers include structural availability hazard trackers (e.g., execution unit occupancy hazard trackers, register file availability hazard trackers), data availability hazard trackers, and memory hazard trackers(s). Although scalar unit 106 is illustrated as a separate unit from vector controller 110, in some embodiments, scalar unit 106 and vector controller 110 are combined together.

[0040] Vector processor 102 includes multiple vector processor channels 112 (e.g., vector processor channels 1120, 1121 to 112). m Multiple vector processor channels 112 include hardware for executing or otherwise performing instructions issued by vector controller 110. Vector processor 102 may include any number of vector processor channels 112. In some embodiments, the number of vector processor channels 112 is responsive to programmable hardware (e.g., hardware component 104). Figure 1 The number of vector processor channels 112 is selected based on performance requirements and available space on the programmable hardware. The number of vector processor channels 112 is selected based on throughput requirements and the available area for vector processor 102. When selecting the number of vector processor channels 112 for vector processor 102, the data path width of the vector processor channel multiplied by the number of vector processor channels multiplied by the number of register files must equal the vector length.

[0041] Each vector processor channel in vector processor channel 112 includes a vector register file 114 and multiple execution units 118 (e.g., 118...). a 118 b Until 118 r ,in r (For positive integers). Vector register file 114 includes multiple register file libraries 116 (e.g., 1160, 1161 up to 116). N ,in N (A positive integer), used to store data instances and feed multiple execution units 118. In some embodiments, the number of execution units 118 is equal to the number of register file libraries 116 (e.g., r equal N In some implementations, the number of execution units 118 differs from the number of register file libraries 116 (e.g., r Greater than N ,or r Less than N In some implementations, the number of execution units 118 and the number of register file libraries 116 are selected in response to operations performed by the vector processor 102.

[0042] In some implementations, register file 116 simultaneously feeds multiple execution units 118. Multiple register files 116 are used to simulate additional write and / or read ports. As described above, vector processor channel 112 can have a simple feedforward data path. Control signals from vector controller 110 can drive vector processor channel 112 to perform vector operations. Execution unit 118 is configured to perform an operation or a set of related operations.

[0043] As described above, vector processor 102 can use temporal and spatial SIMD paradigms. For example, multiple vector processor channels 112 can implement the spatial paradigm by using different channels to process the same instruction across multiple data elements. The temporal SIMD paradigm can be implemented by requiring instructions to process different elements of vector processor 102 over multiple clock cycles (or simply "cycles"). In some implementations, the length of time for processing instructions is referred to as the clock or chain execution time. While one or more instructions are being executed within a specific clock cycle, other independent instructions can be issued to other idle execution units 118.

[0044] Execution unit 118 reads from or writes to each register file 116 at a fixed number of cycles, cycling to the next register file 116 in the series, thereby freeing up the original files for reading or writing by different hardware execution units 118. While one execution unit 118 (e.g., execution unit 118a) is reading from or writing to a register file 116 (e.g., register file 1161), other execution units 118 (e.g., execution unit 118b) can read from or write to other register files 116 (e.g., register file 1160). Although register files 116 must be read or written to in sequence, the order in which execution units 118 read from or write to register files 116 is determined by the instruction stream and the availability of execution units 118. By allowing each execution unit 118 to cycle through the register file library 116 in a fixed, cyclical order, the vector processor 102 uses only 1R1W memory primitives to create multiple virtual read ports and write ports. The vector processor 102 can create virtual (2N)R(N)W memory (where N is a positive integer) using the only 1R1W memory available on the FPGA by doubling the number of read ports through duplication. This utilization of execution unit 118 enables superscalar execution within the vector processor 102 because multiple instructions can be executed simultaneously.

[0045] Now for reference Figure 3 The diagram illustrates the use of vector processor 102 ( Figure 1 and Figure 2 Example vector processor channel 112 used together. Each vector processor channel 112 includes multiple execution units 118 (e.g., execution unit 118). a 118 b Until 118 r ) and multiple register file libraries 116 (e.g., register file libraries 1160, 1161 up to 116) N In some implementations, each register file 116 stores one or more 32-bit elements of a vector register. In some implementations, multiple register files 116 simultaneously feed multiple hardware execution units 118.

[0046] Each hardware execution unit 118 reads from and writes to each register file 116 in a sequential order, creating multiple virtual register file read and write ports. While the register files 116 must be read or written to in a sequential order, the order in which execution units 118 read from or write to the register files 116 is determined by the instruction stream and the availability of execution units 118. Execution units 118 are heterogeneous, and each execution unit can only execute a subset of vector instructions. In some implementations, execution units 118 perform different operations (e.g., one execution unit 118 performs addition, while different execution units 118 perform multiplication). Multiple instances of the same execution unit 118 can be provided for common instructions. For example, a subset of execution units 118 performs the same operation (e.g., two execution units 118 perform addition, and two different execution units 118 perform multiplication), while other execution units 118 perform different operations (e.g., one execution unit 118 performs logarithmic operation, and another execution unit 118 performs square root).

[0047] Each hardware execution unit 118 has a first multiplexer 120 that selects which register file 116 to read from at any given time. Multiplexer 120 multiplexes data from all register files 116 into the execution pipeline. The execution pipeline is a simple feedforward unit. In some implementations, the output of the execution pipeline is buffered in an optional padding delay (used to ensure all execution units 118 have the same execution delay) before being provided to the execution unit 118. Each hardware execution unit 118 has a second multiplexer 122 (for binary operations only), which is also selected among the register files 116 and has additional inputs that can hold scalar values. For example, the second multiplexer 122 includes a flip-flop that holds constant values ​​from the control unit for vector-scalar and vector-constant operations. The two multiplexers in execution unit 118 have independent signals controlled by execution unit 118. However, the mobile execution unit 118 has only one multiplexer that can be selected among the register file libraries 116, and this one multiplexer of the mobile execution unit 118 includes an additional input field that can store scalar values.

[0048] Each register file 116 also has a multiplexer on its write port, which selects which hardware execution unit 118 can write to the register file 116 in any cycle. Vector controller 110 ( Figure 1The system ensures that the read and write multiplexer select signals for each hardware execution unit 118 and each register file library 116 are correctly set and synchronized with each other. The vector controller 110 ensures that instructions are issued only after all dependencies have been resolved, and ensures that issued instructions do not pause the data path.

[0049] In some implementations, each register file 116 is 32 bits wide and has two read ports and one write port. Each register file 116 stores E 32-bit values ​​(where E is a positive integer) from the vector register. The control logic for reading the multiplexer 120 ensures that the multiplexer 120 cycles through all register files 116, starting from register file 1160 and ending at register file 116. N The process ends with the register file library 116 switching every E cycles, as the read address provided to register file library 116 reads E source values ​​from register file library 116. The write multiplexer select signal is controlled via a shift register, which shifts the select signal to the next multiplexer every E cycles, ensuring that the same execution unit 118 can write to all N register files 116.

[0050] Connected to the read port of each register file library 116 are E 32-bit wide dual-clock FIFOs used to store values ​​from the vector register file to on-chip shared memory. Each vector processor channel 112 also includes a load unit 124 and a storage unit 126. In some embodiments, the load unit 124 and storage unit 126 are connected to an on-chip 4096-bit wide shared memory. Each register file library 116 is 32 bits wide (e.g., each register file library 116 can write to or read two bfloat16 values ​​or a single single-precision value from the vector register in a single clock cycle). In some embodiments, the number of load units 124 in each channel is equal to the number of register file libraries 116 (e.g., E...). Multiple first-in-first-out (FIFO) queues (N), and each register file 116 is associated with a FIFO queue. For example, the multiple FIFO queues are dual-clock, thirty-two-bit wide. In some implementations, the memory cells 126 include a number equal to the number of register files 116 (e.g., E). Multiple FIFO queues (N). In some embodiments, the number of FIFO queues in the loading unit 124 and the storage unit 126 is equal to the number (N).

[0051] Loading a vector involves writing vector values ​​from on-chip shared memory into each FIFO queue of the FIFO queue in the loading unit 124 of each vector processor channel 112. Once the data is in the FIFO queue of the loading unit 124, each vector processor channel 112 cycles through each register file library 116 and each FIFO queue in the FIFO queue of the loading unit 124 connected to that register file library 116, extracting values ​​from the FIFO queues and loading the values ​​into the register file library 116.

[0052] To store data back to the temporary SRAM, each vector processor channel 112 cycles through its register file 116, reading portions of the vector elements stored in the register file 116. Selective writing ensures that the FIFO queues of memory cell 126 guarantee that the appropriate data is written to each FIFO queue of memory cell 126. Data can be directed to the appropriate FIFO queue by providing a write enable signal at the correct time. Back-to-back load or store instructions can be served without any delay because once the data used for loading or storing is written to or read from a FIFO, there is no other data from the same load or store in the same FIFO, and the vector processor 102 does not have to wait for the load or store instruction to complete. The parameters of the vector processor can be changed independently of the on-chip shared memory parameters because the width of each load / store FIFO is multiplied by the number of such FIFOs (E...). N) is a constant.

[0053] Once all FIFO queues have been written to memory cell 126, the temporary register memory can retrieve data from the FIFO queues of memory cell 126 and write it to memory. For both loading and storing, once the register file 116 within vector processor channel 112 has finished reading from or writing to a set of FIFO queues in its loading unit 124 and / or memory cell 126, the register file 116 is ready to read a new set of values ​​from the same set of FIFO queues in loading unit 124 or write a new set of values ​​to the same set of FIFO queues in memory cell 126. Therefore, as long as the on-chip temporary registers can serve these requests at the same rate, vector processor channel 112 can serve back-to-back loading and storing without any pauses.

[0054] The number (e.g., N) of register files 116 and / or the number (e.g., r) of execution units 118 in vector processor channels 112 are configurable and can be changed. For example, for a fixed vector length defined in the processor ISA, the number of vector processor channels 112 multiplied by the number of register files 116 in each vector processor channel 112 and the number of vector elements in each register file 116 must equal the vector length. Another example involves changing the number of hardware execution units 118 for each operation type. For example, the resource utilization and expected performance of a given vector processor channel configuration can be used to determine the number of hardware execution units 118 used in each vector processor channel 112 for each operation type to provide a performance level with minimal resources used (e.g., the number of execution units 118 used).

[0055] In some implementations, some execution units 118 are not used for certain operations. For example, if a combination of hardware execution units 118 for inverse, square root, exponentiation, and logarithmic operations is provided in the vector processor 102, such a combination of hardware execution units 118 is frequently used in SoftMax operations, which require calculating the exponent for each value, but is rarely used in layer normalization operations, where the inverse and square root are calculated only after significant data aggregation.

[0056] In some embodiments, vector processor 102 includes a configuration of eight (nominal 32-bit) vector processor channels 112 with eight vector register files 116, each storing two 32-bit vector elements, coupled to seven execution units 118 (three bfloat16 multiplication / addition execution units 118, one single-precision addition execution unit 118, one execution unit 118 for approximating bfloat16 transcendental functions, one execution unit 118 for discard / compare operations, and one execution unit 118 for move / transform operations), resulting in an optimal performance-FPGA resource tradeoff for the performance requirements and resource constraints on hardware component 104. In some embodiments, vector processor 102 includes a configuration of sixteen vector processor channels 112, each channel having eight vector register files 116, and each register file 116 having one 32-bit vector element.

[0057] In some implementations, vector processor 102 includes a configuration of four vector processor channels 112 that can issue 32-cycle instructions every four cycles, thereby balancing time and space SIMD. For example, a 4096-bit vector is divided among the four vector processor channels 112, such that the lowest 1024 bits (e.g., 32) are distributed across the four vector processor channels 112. E L, where L is a positive integer, is allocated to the first vector processor channel 112 (e.g., vector processor channel 1120), and the next 1024 bits (e.g., 32... E L) is assigned to the second vector processor channel 112 (e.g., vector processor channel 1121), and so on.

[0058] Input data is partitioned across vector processor channels 112, with each channel 112 placed adjacent to an on-chip shared memory storing the corresponding data element loaded or stored by vector processor 102. In some embodiments, the on-chip shared memory is 4096 bits wide to match the width of a high-bandwidth memory (HBM) system on an FPGA. In some embodiments, small networks are placed between vector processor channels 112 to move data across channels, as required by some variations supporting move and transformation operations in vectors. When the data order is rearranged in the vector, a temporary copy of the data is stored in memory and then read out in the correct order required by register file library 116, thus allowing move and transformation operations to have predictable latency and adhering to the sequential read and write order both within and across register files, which can be controlled by vector controller 110 ( Figure 1 ) is used to correctly schedule instructions.

[0059] Now for reference Figure 4 The diagram illustrates the use of vector processor 102 ( Figures 1 to 3 The two execution units 118 in ) Figure 2 and Figure 3 (For example, the first execution unit 118) a Second execution unit 118 b Example timing diagram 400 for register file reads performed. In the illustrated example, vector processor 102 has four vector processor channels 112 with eight register file libraries 116 (e.g., N equals 8), where each register file library 116 includes 128 bits of vector data, with the lowest 128 bits of the vector data in vector processor channel 112 in register file library 0, and the remaining bits are sequentially placed into the remaining register file libraries 116 in 128-bit blocks. First execution unit 118 a An addition operation is performed with a destination register (r0) and a source destination (r1 and r2), and a multiplication operation is performed with a destination register r3 and a source destination (r4 and r5).

[0060] The x-axis 402 of timing diagram 400 illustrates the different register files 116 (e.g., register files 1160, 1161, 1162 up to 116). NThe y-axis 404 of timing diagram 400 illustrates different times corresponding to different clock cycles (e.g., T0, T1, T2, etc.). At time T0, the first execution unit 118a performs a read from the first register file 1160 to perform an addition operation on the data stored in the first register file 1160. The first execution unit 118a reads from positions 4 and 8 of the first register file 1160 in the first cycle, from positions 5 and 9 in the second cycle (e.g., time T1), from positions 6 and 10 in the third cycle (e.g., time T2), and from positions 7 and 11 in the fourth cycle (e.g., time T3).

[0061] At time T4, the first execution unit 118 a The operation on the first register file library 1160 is completed, and the reading of the next register file library 116 in the sequence (e.g., register file library 1161) begins. First execution unit 118 a In the first cycle (e.g., time T4), it reads from the second file library 1161 at positions 4 and 8. At the same time T4, the second execution unit 118... b The execution unit 118 begins reading from the first register file 1160 to perform a multiplication operation on the data stored in the first register file 1160. Although the register file 116 must be read in order, the order in which the execution unit 118 reads from the register file 116 is determined by the instruction stream and the availability of the execution unit 118. The illustrated example shows the second execution unit 118. b Next, it reads from the first register file library 1160; however, in execution unit 118... a Subsequently, if other execution units (e.g., execution unit 118) c ) can be used instead of the second execution unit 118 b Or if the instruction stream directs to other execution units (e.g., execution unit 118) c Next, register file library 1160 is read, meaning another execution unit may have already read register file library 1160. At time T5, the first execution unit 118... a Read from positions 5 and 9 of the second register file 1161, and from the second execution unit 118. b Read from positions 17 and 21 of the first register file 1160. At time T6, the first execution unit 118... a Read from positions 6 and 10 of the second register file library 1161, and from the second execution unit 118. b Read from positions 18 and 22 of the first register file 1160. At time T7, the first execution unit 118... a Read from positions 7 and 11 of the first register file library 1161, and from the second execution unit 118.b Read from positions 19 and 23 of the first register file 1160.

[0062] At time T8, the first execution unit 118 a Complete the operation on the second register file library 1161, and the second execution unit 118 b Complete the operation on the first register file library 1160. First execution unit 118 a It begins reading from the third register file library 1162, and the second execution unit 118 b Reading begins from the second register file library 1161. This process continues until each execution unit (e.g., the first execution unit 118) is reached. a Second execution unit 118 b The registers are read from the register file library 116 in the vector processor channel 112 in sequential order.

[0063] Vector processor 102 uses time-division multiplexing of register file library 116 in vector processor 102 to keep multiple hardware execution units 118 in vector processor 102 busy.

[0064] Now for reference Figure 5 The illustration shows the vector processor 102 ( Figures 1 to 3 The two execution units 118 in ) Figure 2 and Figure 3 (For example, the first execution unit 118) a Second execution unit 118 b Example timing diagram 500 shows an example write to register file library 116. In the illustrated example, E=4 (there are four vector elements in each register file library 116). Figure 5 The example shown in the figure corresponds to Figure 4 The example is illustrated in the figure. In some implementations, Figure 5 The time plotted on the y-axis at 504 is used as Figure 4 This occurs as a continuation of the timing diagram 400 shown in the figure.

[0065] The x-axis 502 of timing diagram 500 illustrates different register file libraries 116 (e.g., register file libraries 1160, 1161, 1162 up to 116). N The y-axis 504 of timing diagram 500 illustrates different times corresponding to different clock cycles (e.g., T0, T1, T2, etc.). At time T0, the first execution unit 118a writes to the first register file 1160. For example, a first instruction is issued, and in response to the issuance of the first instruction, the first execution unit 118a... aWrite to position 0 of the first register file library 1160. First execution unit 118 a Write to position 0 of the first register file library 1160 in the first cycle (e.g., time T0), write to position 1 in the second cycle (e.g., time T1), write to position 2 in the third cycle (e.g., time T2), and write to position 3 in the fourth cycle (e.g., time T3).

[0066] At time T4, the first execution unit 118 a The operation on the first register file 1160 is completed, and writing to the next register file 116 in the sequence (e.g., register file 1161) begins. First execution unit 118 a In the first cycle (e.g., time T4), a write is made to position 0 of the second file library 1161. At the same time T4, the second execution unit 118... b Writes begin to the first register file 1160 at position 12. For example, a second instruction is issued, and in response to the issuance of the second instruction, the second execution unit 118... b Writing begins at position 12 to the first register file 1160. Although the register file 116 must be written in sequence, the order in which execution unit 118 writes to the register file 116 is determined by the instruction stream and the availability of execution unit 118. The illustrated example shows the first execution unit 118. a First, write to the first register file library 1160, and then to the second execution unit 118. b Next, the first register file 1160 is written. However, if another execution unit is available instead of the first execution unit 118, this will be done. a Or the second execution unit 118 b Or if the instruction stream directs to other execution units (e.g., execution unit 118) d If the data is written to register file library 1160, then another execution unit (e.g., execution unit 118) will also write the data. d It may have already been written to register file library 1160, or in execution unit 118. a Then it is written. At time T5, the first execution unit 118 a Write to position 1 of the second register file library 1161, and the second execution unit 118 b Write to position 13 of the first register file 1160. At time T6, the first execution unit 118... a Write to position 2 of the second register file library 1161, and the second execution unit 118 b Write to location 14 of the first register file 1160. At time T7, the first execution unit 118... aWrite to position 3 of the first register file 1161, and the second execution unit 118 b Write to position 15 of the first register file 1160.

[0067] At time T8, the first execution unit 118 a Complete the operation on the second register file library 1161, and the second execution unit 118 b Complete the operation on the first register file library 1160. First execution unit 118 a Writes begin to the third register file library 1162, and the second execution unit 118... b Writing to the second register file 1161 begins. This process continues until the first execution unit 118. a Second execution unit 118 b All register files 116 in the vector processor channel 112 are written sequentially, and the instruction stream is processed by multiple execution units 118.

[0068] Vector processor 102 uses time-division multiplexing of register file library 116 in vector processor 102 to keep multiple hardware execution units 118 in vector processor 102 busy.

[0069] Now for reference Figure 6 The diagram illustrates an example architecture of vector controller 110. Vector controller 110 identifies memory hazards and issues memory read and write commands for load and store as early as possible. Vector controller 110 also detects and avoids register file data hazards (e.g., read-after-write or write-after-write). Vector controller 110 also tracks the availability of hardware execution unit 118 and register file write ports to detect and avoid structural hazards. In some implementations, vector controller 110 issues instructions at a rate at which the data path can consume instructions, thereby preventing instruction bubbles and pauses during issuance.

[0070] Vector controller 110 includes instruction queue 128, whose buffer is provided by scalar unit 106 ( Figure 1 and Figure 2 Non-speculatory instructions are issued. Instructions are expanded to a wider format. Instructions are forwarded to instruction queue 130 stored in SRAM. Memory instructions (load, store, barrier) are also forwarded to the memory instruction queue, which feeds a memory hazard tracker 132 responsible for tracking memory hazards, issuing load and store commands to the on-chip shared memory, and receiving completion responses from the scratchpad.

[0071] Instructions in instruction queue 130 are speculatively read into three hazard trackers: register file data hazard tracker 134, execution unit structural hazard tracker 136, and register file write port structural hazard tracker 138. Hazard trackers are populated with the same delay. If all three hazard trackers agree that no hazard exists, the head pointer of the instruction queue is incremented. If any hazard tracker indicates a hazard exists, the read pointer is reset to the head pointer value, and the release logic is reset to the last instruction that was not successfully released. Simultaneously, speculative instructions in the hazard tracker pipeline are compressed.

[0072] If there is no danger, the instruction proceeds through the vector control unit pipeline to the hardware execution unit allocator 140, which allocates physical hardware execution units 118 capable of processing the instruction in a cyclic manner. Figure 2 and Figure 3 The instruction continues downward through the vector control unit to the corresponding hardware execution unit control unit 142, which generates cycle-by-cycle control signals for the data path of the hardware execution unit 118.

[0073] Simultaneously, the instruction is also sent to the register file write multiplexer control unit 144, which takes into account the latency of the hardware execution pipeline and reserves the write multiplexer for the corresponding hardware execution pipeline. This ensures that the data generated by the execution unit 118 is written to the vector register file 114 in the appropriate clock cycle. Figure 2 Note that there are no structural hazards on the register file write port at this point, as these have been addressed by the register file write port structural hazard tracker. Once an instruction has been issued to execution unit 118, no further control feedback is required, and each execution unit 118 proceeds with a predictable, fixed delay.

[0074] The vector controller 110 holds a view of the future state of the vector data path, ensuring that the vector data path never stalls or generates backpressure. Control signals from the respective execution control units 142 are forwarded via a deeply pipelined fan-out tree to the respective vector processor channels 112 of the vector data path. Figure 2 and Figure 3This architecture is purely feedforward and allows the individual vector processor channels 112 to be placed far apart from each other in the final FPGA layout without incurring long wiring delays that could limit clock frequency. This minimizes the impact of the vector processors 102 on the overall physical efficiency of the system placement and routing. Furthermore, the vector processor channels 112 can be freely placed on the 4096-bit wide-area shared memory (physically distributed across a large area of ​​the device) that the vector processor channels 112 read from and write to, taking advantage of spatial locality. Finally, the vector processor channels 112 can be placed in the physical region most suitable for the overall system layout.

[0075] The register file data hazard tracker 134 detects and prevents read-after-write and write-after-write hazards. Each instruction can have up to two source operands and one destination operand. The register file data hazard tracker 134 assigns a valid bit to each register, which is reset when an instruction to write to that register is issued and set again once the instruction is completed.

[0076] If the other two hazard trackers (execution unit structural hazard tracker 136 and register file write port structural hazard tracker 138) agree that the instruction can be issued, register file hazard tracker 134 clears the valid state of the destination register. When the instruction is scheduled to begin writing to the register file, the valid state of the destination register is set again, thus allowing subsequent dependent instructions to be issued.

[0077] Because the valid state is read only at the beginning of the register file hazard tracker 134 pipeline and updated at the end, this method cannot track real data hazards existing in an N-instruction window, where N is the number of instructions that can be in flight in the hazard tracker pipeline. The register file data hazard tracker 134 also uses the preceding N instructions to check each instruction to verify that no real data hazard exists; this information is added to the AND reduction performed on the valid state obtained from the state table to complete the process of checking all data hazards.

[0078] The hardware execution unit structural hazard tracker 136 counts the number of available hardware execution units 118 for each type (e.g., available hardware execution units 118 for performing operations). When an instruction is issued, it decrements the count for the corresponding execution unit type; if the count reaches zero, an instruction requiring a hardware execution unit 118 of that given type cannot be issued. In some implementations, the latency of the logic in the execution unit structural hazard tracker 136 is accounted for by creating two sets of counters. A first counter holds a speculative count of available execution units 118 and is read and decremented when an instruction arrives at the head of the hazard tracker pipeline. A second set of counters holds the actual number of available execution units 118 and decrements this count only when all three hazard trackers agree that an instruction can be issued. If it is determined that an instruction cannot be issued, the value of the speculative counter is reset to the value of the actual counter. When an instruction is expected to complete and release the execution unit 118, the corresponding speculative and actual availability counters are appropriately incremented, thereby allowing future instructions using those hardware execution units 118 to be issued.

[0079] The register file write port structural hazard tracker 138 uses a shift register to store the reservation status of the register file write port. This shift register stores not only whether the write port is reserved, but also the ID of the register that must be written to. This information is used to control the register file write address and to enable the register status in the register file data hazard tracker. Taps on this shift register are used to read and write reservations for the write port. The read tap is used by the head of the hazard tracker pipeline to query whether the register file write port is reserved for a future clock cycle corresponding to the delay of the hardware execution unit 118 responsible for processing the current instruction. The write tap is used to reserve the write port when all three hazard trackers determine that the instruction can be successfully issued.

[0080] Because instructions read the state of the write port several cycles before the state update, previous instructions can reserve write ports for the same cycle after the state has been checked. Vector controller 110 does not allow any pair of hardware execution units 118 to have a delay difference less than the delay of register file write port hazard tracker 138, to prevent previous instructions from reserving write ports for the same cycle after the state has been checked, and ensures that write taps for a given type of hardware execution unit 118 are followed by read taps for the same hardware execution unit 118, without intermediate read or write taps for any other execution unit 118 type.

[0081] In some implementations, the memory hazard tracker 132 uses a memory table to store bits indicating pending loads and pending stores for each possible memory address. If a load detects that a previous store may not have completed writing its data to a given memory address, the load operation is paused until the hazard is resolved, and vice versa for the store. The on-chip shared memory provides early acknowledgments to the memory hazard tracking unit that data is being read from or written to the scratchpad memory bank, allowing the hazard to be resolved several cycles earlier. Once the load operation is complete, the data is written to load unit 124 in the vector data path (…). Figure 3 The FIFO queue of the execution unit structural hazard tracker 136 is incremented to indicate that the data is ready. Similarly, once the data is retrieved from the storage unit 126 ( Figure 3 The hazard is read from the FIFO queue, and the counter corresponding to the memory cell is incremented. The memory hazard tracker 132 can use multiple cycles to track a single hazard without sacrificing performance.

[0082] The memory hazard tracker 132 is responsible for tracking memory hazards and issuing load and store commands. When a load command is completed and data is placed in the FIFO queue of the load unit 124, the corresponding load instruction can be issued by the execution unit structural hazard tracker 136. This allows the data path and its associated control to be decoupled from the commands sent to the memory system, thereby hiding the latency of the memory system.

[0083] Now for reference Figure 7 The illustration shows the vector processor 102 ( Figures 1 to 6 Example method 700 is executed. Action reference for method 700. Figures 1 to 6 This will be discussed below.

[0084] In some embodiments, method 700 is executed by vector processor 102, which includes: a vector controller 110 configured to receive instructions for execution on vector processor 102 and provide control signals to a plurality of vector processor channels 112; and a plurality of vector processor channels 112, wherein each of the plurality of vector processor channels 112 includes: a vector register file 114 having a plurality of register files 116 for storing data; a loading unit 124 having a plurality of first-in-first-out queues for loading data into the plurality of register files 116; a plurality of execution units 118 configured to perform operations on data in each of the plurality of register files 116 and execute instructions from an instruction issue queue; and a storage unit 126 having a plurality of first-in-first-out queues for reading data from each of the plurality of register files 116.

[0085] In some implementations, the programmable hardware component 104 is a field-programmable gate array (FPGA) device, and the processor is a vector processor 102 implemented as an overlay on the FPGA device.

[0086] In some implementations, the number of vector processor channels 112 is selected in response to the performance requirements of the programmable hardware component 104 and the available space on the programmable hardware component 104.

[0087] In some implementations, method 700 is executed by vector processor channel 112 of vector processor 102. In some implementations, each of the plurality of vector processor channels 112 receives a subset of data, and each vector processor channel performs the same operation on that subset of data in parallel.

[0088] At 702, method 700 includes receiving instructions for execution using a vector processor. For example, vector controller 110 receives instructions for execution using vector processor 102. In some embodiments, vector controller 110 receives instructions from scalar unit 106.

[0089] In 704, method 700 includes testing for hazards during instruction issuance. Vector controller 110 tests for hazards during instruction issuance. In some embodiments, the hazard includes a data hazard. In some embodiments, the hazard includes a structural hazard. In some embodiments, the hazard includes a memory hazard. In some embodiments, the hazard includes any combination of data hazards, structural hazards, or memory hazards.

[0090] In 706, the method includes determining whether to issue an instruction. The vector controller 110 determines whether to issue an instruction in response to a hazard test.

[0091] At 708, the method includes holding an instruction in response to determining that no instruction should be issued. In some implementations, the vector controller 110 holds the instruction in response to identifying one or more hazards during a hazard test, thereby preventing the issuance of the instruction. Method 700 may return to 704 to continue testing the hazard, and the vector controller 110 may continue to hold the instruction until the hazard is resolved.

[0092] At 710, method 700 includes providing control signals for the instruction to each of a plurality of vector processor channels in the vector processor in response to determining a release instruction. Vector controller 110 provides control signals for the instruction to each vector processor channel 112 in vector processor 102 in response to determining the release instruction. Each vector processor channel 112 performs an operation on a subset of data in parallel. In some embodiments, each vector processor channel 112 performs the operation synchronously.

[0093] At 712, method 700 includes performing operations on data in multiple register files within each vector processor channel using multiple execution units in each vector processor channel. In some embodiments, the multiple execution units 118 read data from multiple register files 116 in an order starting from a first register file, and write data to multiple register files 116 in an order starting from the first register file, until data is read from or written to each of the multiple register files 116. In some embodiments, control signals and the availability of the multiple execution units 118 determine the order in which data is read from the multiple register files 116 for each of the multiple execution units 118. In some embodiments, each execution unit 118 includes a multiplexer 120 to select a register file from the multiple register files 116 to read data from during a period (e.g., time T0, time T1, etc.).

[0094] In some implementations, one of the plurality of execution units 118 reads portions of data from each register file library 116 in a fixed number of cycles, and the execution unit performs operations on portions of data in each register file library during the fixed number of cycles until all data has been read from each of the plurality of register file libraries 116.

[0095] In some embodiments, each of the plurality of register files 116 includes a portion of data provided by a plurality of first-in-first-out queues of the loading unit 124, and each register file stores portions of data in sequential order. In some embodiments, the number of the plurality of first-in-first-out queues of the loading unit 124 is equal to the number of the plurality of register files 116, and each register file 116 is associated with a single first-in-first-out queue of the loading unit 124.

[0096] In some implementations, one of the plurality of execution units 118 reads portions of data from each register file library 116 in a fixed number of cycles, and performs operations on portions of data in each register file library during the fixed number of cycles until all data has been read from each of the plurality of register file libraries 116.

[0097] In some implementations, each execution unit 118 performs different operations on the data (e.g., addition, multiplication, logarithmic operation, square root). In some implementations, a subset of execution units 118 performs the same operation on the data. In some implementations, a subset of execution units 118 performs the same operation on the data, while other execution units 118 perform different operations on the data.

[0098] In some embodiments, the number of register files 116 is selected in response to the length of the vector, and the size of each register file is determined using the number of register files 116 and the length of the vector. In some embodiments, the number of execution units 118 is different from the number of register files 116. In some embodiments, the number of execution units 118 is equal to the number of register files 116. In some embodiments, the number of execution units 118 is selected in response to the type of operation required by the instruction.

[0099] In some implementations, each register file 116 writes data to a first-in-first-out (FIFO) queue of the memory unit 126 associated with that register file 116. In some implementations, the number of FIFO queues of the memory unit 126 is equal to the number of register files 116, and each register file 116 is associated with a single FIFO queue of the memory unit 126.

[0100] In some implementations, method 700 further includes writing data from multiple first-in-first-out queues of storage unit 126 into memory.

[0101] An example use case of method 700 includes: a first execution unit (e.g., execution unit 118) among a plurality of execution units 118 a During the first cycle (e.g., time T0), a first portion of data is read from the first register file (e.g., register file 1160) in a plurality of register file files 116, and an operation is performed on the first portion of the data; by the first execution unit (e.g., execution unit 118) a During the second cycle (e.g., time T4), a second portion of data is read from a second register file (e.g., register file 1161) in a plurality of register files 116, and an operation is performed on the second portion of the data; and in response to a subsequent instruction being issued, a subsequent execution unit (e.g., execution unit 118) reads a second portion of data from a second register file (e.g., register file 1161) in a second cycle (e.g., time T4), ... b During the second cycle (e.g., time T4), a first portion of data is read from the first register file (e.g., register file 1160), and a second operation is performed on the first portion of the data during the second cycle, while the first execution unit (e.g., execution unit 118) reads the first portion of data.a The instruction is being read from the second register file library. Execution units in the plurality of execution units 118 (e.g., other execution units) continue to read data from the plurality of register files 116 during subsequent cycles until the instruction stream has been processed.

[0102] Now for reference Figure 8A The illustration shows the process of accessing register file library 116 ( Figure 2 and Figure 3 Example timing diagram 800 for reading. Timing diagram 800 illustrates the read address for each register file library 116. In the illustrated example, each library element (E) equals four, and the vector processor 102 ( Figure 2 and Figure 3 Each vector processor channel 112 has four register files 116 (corresponding to CHIME=16). Each register file contains four elements of a vector. The architecture vector register address is multiplied by four to obtain the starting address of the physical vector in the file, and then the process loops through the next four elements in the file. Figure 8B The diagram illustrates the process of accessing register file library 116 ( Figure 2 and Figure 3 Example timing diagram 802 for reading, where each library element (E) equals one.

[0103] Figure 9A The diagram illustrates the process of directing data to register file library 116 ( Figure 2 and Figure 3 Example timing diagram 900 is written. In timing diagram 900, each library element (E) equals four, and vector processor 102 ( Figure 2 and Figure 3 Each vector processor channel 112 has eight register files 116. Figure 9A The diagram shows three of the eight register file libraries 116. Figure 9B The illustration shows an example timing diagram 902 for writing to register file library 116, where each library element (E) equals one.

[0104] Figure 10A The illustration shows loading unit 124 ( Figure 3 Example timing diagram 1000. In timing diagram 1000, each library element (E) equals four, and vector processor 102 ( Figure 2 and Figure 3 Each vector processor channel 112 has eight register files 116. Figure 10A The diagram shows three of the eight register file libraries 116. Figure 10B The illustration shows loading unit 124 ( Figure 3 Example timing diagram 1002, where each library element (E) equals one.

[0105] Figure 11A The illustration shows storage cell 126 ( Figure 3 Example timing diagram 1100. In timing diagram 1100, each library element (E) equals four, and vector processor 102 ( Figure 2 and Figure 3 Each vector processor channel 112 has eight register files 116. Figure 11A The diagram shows three of the eight register file libraries 116. Figure 11B The illustration shows the reading unit 126 ( Figure 3 Example timing diagram 1102, where each library element (E) equals one.

[0106] The techniques described herein can be implemented in hardware, software, firmware, or any combination thereof, unless specifically described as being implemented in a particular manner. Any features described as modules, components, etc., can also be implemented together in an integrated logic device or separately as discrete but interoperable logic devices. If implemented in software, these techniques can be implemented at least in part by a non-transitory processor-readable storage medium comprising instructions that, when executed by at least one processor, perform one or more of the methods described herein. Instructions can be organized into routines, programs, objects, components, data structures, etc., which can perform specific tasks and / or implement specific data types, and can be combined or distributed as desired in various implementations.

[0107] Computer-readable media can be any available medium that can be accessed by a general-purpose or special-purpose computer system. A computer-readable medium storing computer-executable instructions is a non-transitory computer-readable storage medium (device). A computer-readable medium carrying computer-executable instructions is a transmission medium. Therefore, by way of example and not limitation, embodiments of this disclosure may include at least two distinctly different types of computer-readable media: non-transitory computer-readable storage media (devices) and transmission media.

[0108] As used herein, a nontransitory computer-readable storage medium (device) may include RAM, ROM, EEPROM, CD-ROM, solid-state drive (“SSD”) (e.g., RAM-based), flash memory, phase-change memory (“PCM”), other types of memory, other optical disk storage devices, magnetic disk storage devices or other magnetic storage devices, or any other medium that may be used to store desired program code in the form of computer-executable instructions or data structures and that may be accessed by a general-purpose or special-purpose computer.

[0109] The steps and / or actions of the methods described herein may be interchanged without departing from the scope of the claims. In other words, the order and / or use of specific steps and / or actions may be modified without departing from the scope of the claims unless the proper operation of the described methods requires a specific order of steps or actions.

[0110] The term "determine" encompasses a wide range of actions, and therefore "determine" can include calculation, estimation, processing, derivation, investigation, searching (e.g., looking in a table, database, or other data structure), ascertainment, etc. Furthermore, "determine" can include receiving (e.g., receiving information), accessing (e.g., accessing data in memory), etc. Additionally, "determine" can include parsing, selecting, choosing, building, etc.

[0111] The terms "comprising" and "having" are intended to be inclusive, meaning that additional elements may exist in addition to those listed. Additionally, it should be understood that references to "one embodiment" or "embodiment" in this disclosure are not intended to exclude the existence of additional implementations that also incorporate the described features. For example, where compatible, any element or feature described herein with respect to an embodiment may be combined with any element or feature of any other embodiment described herein.

[0112] This disclosure may be embodied in other specific forms without departing from its spirit or characteristics. The described embodiments are to be regarded as illustrative rather than restrictive. Therefore, the scope of this disclosure is indicated by the appended claims rather than the foregoing description. Variations falling within the meaning and scope of equivalents of the claims will be included within their scope.

Claims

1. A processor (102), said processor being implemented on programmable hardware (104), comprising: A vector controller (110) is configured to receive instructions for execution on the processor (102) and to provide control signals to a plurality of vector channels (112); as well as The plurality of vector processor channels (112), wherein each of the plurality of vector processor channels (112) includes: A vector register file (114) having multiple register file libraries (116) for storing data; The loading unit (124) has multiple first-in-first-out queues and loads the data into the multiple register file libraries (116). Multiple execution units (118) configured to perform operations on the data in each of the multiple register files (116) and execute the instructions from the instruction dispatch queue; and The storage unit (126) has multiple first-in-first-out queues, and the storage unit reads the data from each of the multiple register file libraries (116).

2. The processor of claim 1, wherein each of the plurality of vector processor channels receives a subset of the data, and each vector processor channel performs the same operation on the subset of the data in parallel.

3. The processor of claim 1, wherein the control signal and the availability of the plurality of execution units determine the order in which the plurality of execution units are read from the plurality of register files, and The plurality of execution units read the data from the plurality of register files in an order starting from the first register file library, and write the data into the plurality of register files in the same order starting from the first register file library.

4. The processor of claim 1, wherein each of the plurality of register file libraries includes a portion of the data, and each register file library stores the portion of the data in a sequential order, and Each execution unit includes a multiplexer to select one of the plurality of register file libraries during a cycle to read the data from it.

5. The processor of claim 1, wherein a first execution unit of the plurality of execution units reads the data from the plurality of register files in a first cycle in an order starting from a first register file, and in response to a subsequent instruction being issued, a subsequent execution unit of the plurality of execution units reads the data from the plurality of register files in an order starting from the first register file for the subsequent instruction, and The execution unit in the plurality of execution units continues to read the data from the plurality of register files during subsequent cycles until the instruction stream is processed.

6. The processor of claim 1, wherein each execution unit performs a different operation on the data.

7. The processor of claim 1, wherein a subset of the execution units performs the same operation on the data.

8. The processor of claim 1, wherein the number of the plurality of first-in-first-out queues of the loading unit is equal to the number of the plurality of register file libraries, and the number of the plurality of first-in-first-out queues of the storage unit is equal to the number of the plurality of register file libraries.

9. The processor of claim 1, wherein the number of the plurality of execution units is different from the number of the plurality of register file libraries.

10. The processor of claim 1, wherein the number of the plurality of register files is selected in response to the length of the vector, and the size of each register file is determined using the number of the plurality of register files and the length of the vector, and The number of execution units is selected in response to the type of operation required by the instructions, and the number of vector processor channels is selected in response to the performance requirements of the programmable hardware and the available space on the programmable hardware.

11. The processor of claim 1, wherein the programmable hardware is a field-programmable gate array (FPGA) device, and the processor is a vector processor implemented as an overlay on the FPGA device.

12. A method comprising: Receive (702) instructions for execution using vector processor (102); In response to determining that the instruction is to be issued, a control signal (710) for the instruction is provided to each of the plurality of vector processor channels (112) in the vector processor (102), wherein each vector processor channel (112) performs an operation on a subset of data in parallel; and The operation described in (712) is performed on the data in the multiple register file libraries (116) within each vector processor channel (12) using multiple execution units (118) in each vector processor channel (12).

13. The method of claim 12, further comprising: The test addresses the hazards associated with issuing the instruction, including one or more of the following: data hazards, structural hazards, or memory hazards. In response to the fact that a hazard was not identified during the test, the instruction is determined to be issued; as well as In response to identifying one or more hazards during a hazard test, it is determined to maintain the instruction until the one or more hazards are resolved.

14. The method of claim 12, wherein performing the operation using the plurality of execution units further comprises: One of the plurality of execution units reads portions of data from each of the plurality of register files in a fixed number of cycles, and performs operations on the portions of data in each register file during the fixed number of cycles, until all data has been read from each of the plurality of register files.

15. The method of claim 12, wherein the control signal and the availability of the plurality of execution units determine the order of reading from the plurality of register files for each of the plurality of execution units, and The plurality of execution units read the data from the plurality of register files in an order starting from the first register file library, and write the data into the plurality of register files in the same order starting from the first register file library.