Vector computing device

Abstract

A vector processor is disposed on a chip and contains a multi-port shared memory, a unit for performing horizontal operations, and interconnected scalar devices, each of which is configured to be capable of receiving a vector element from a matrix multiplication device. A scalar device contains scalar modules, two demultiplexers, and a unit for performing complex arithmetic operations. The technical result is an increase in the operating speed of a vector processor and a decrease in the chip area thereof.

Description

VECTOR COMPUTING DEVICE AREA OF TECHNOLOGY

[0001] The claimed technical solution generally relates to the field of microprocessors, and in particular to a vector computing device. LEVEL OF TECHNOLOGY

[0002] Today, the modern world performs a vast number of data-driven operations. With the development and implementation of neural networks in all areas of life, the need has arisen for devices capable of effectively working with neural networks. General-purpose processors, due to their architecture, are not suited for high-speed machine learning processes, such as training neural networks. Therefore, the development of vector processors has become especially important in recent years.

[0003] A vector processor is a type of processor that can process multiple data elements simultaneously, providing higher performance than scalar processors for some operations. Such a processor can perform operations on a vector of data elements in parallel. Vector processors are particularly useful for tasks such as neural network computation, image processing, and other computations that require processing large amounts of data in parallel.

[0004] Thus, a vector computing device is known from the prior art (see Internet: htps: / / cloud.google.com / tpu / docs / system-architecture-tpu-vm). This vector processor, developed by Google™, is designed to increase data throughput (increase computational performance) for machine learning processes.

[0005] The vector processor elements are located on the integrated circuit chip. To speed up computations, the computing units are segmented into several tracks, each of which is designed to perform a computational operation on a vector element, thus enabling parallelization of operations. Within each track is a vector Memory, which may include multiple memory banks, each with multiple memory address locations. In more detail, each lane includes a multidimensional data / file register configured to store multiple vector elements and an arithmetic logic unit (ALU) configured to perform arithmetic operations on vector elements accessed from and stored in the data register. All processor elements are configured using VLIW instructions.

[0006] The disadvantages of this solution include low performance due to architectural features related to the organization of the tracks, as well as the method of configuring elements when performing arithmetic operations. Furthermore, due to the proposed parallelization of operations across multiple lines, some computational operations requiring intermediate results are executed in multiple processor cycles (passes), which correspondingly increases the execution time and, consequently, reduces performance. Furthermore, the presented solution does not support configuring computational units via a dataflow. Another drawback is the low bandwidth of vector memory due to the need to connect the lines to the computational units via data buses.

[0007] Accordingly, the goal of this technical solution is to create a high-performance vector computing device. Furthermore, this solution should enable streaming configuration of the processor's computing elements, including vector memory, as well as reduce die area and improve energy efficiency, thanks to the specific features of the processor's architecture. DISCLOSURE OF THE INVENTION

[0008] The proposed technical solution proposes a new approach to the architecture of a vector processor that provides high performance.

[0009] The technical problem of increasing the efficiency and speed of a vector processor by increasing the number of operations per processor cycle is solved.

[0010] The technical result achieved by solving this problem is an increase in the operating speed of the vector processor.

[0011] An additional technical result that appears when solving the above problem is a reduction in the area of ​​the vector processor crystal.

[0012] The specified technical results are achieved through the implementation of a vector processor located on a crystal, containing: • scalar devices, each of which is configured to obtain a vector element from a matrix multiplication device and consists of: o at least two scalar modules forming at least one scalar line, configured to perform arithmetic operations on a vector element; o at least one first demultiplexer configured to redirect data from the first scalar module in the line to a multiport shared memory or a second scalar module in the line; o a complex arithmetic operations unit connected to at least one scalar module in the line and configured to perform mathematical functions on a vector element; o at least one second demultiplexer configured to redirect data from the multiport memory to a scalar module or a matrix multiplication device; • a multi-port shared memory connected to at least an external memory, a second demultiplexer of each scalar device and a matrix multiplication device, configured to buffer and store the results of intermediate calculations over the elements of the vector of each scalar device; • a horizontal operations unit associated with scalar devices, configured to: perform arithmetic operations with combined elements of a vector; and transmit the result of performing an arithmetic operation with combined elements of a vector to scalar devices.

[0013] In one particular implementation, scalar devices are arranged on a crystal in the form of a tile.

[0014] In another particular embodiment, each scalar device comprises at least four scalar modules.

[0015] In another particular embodiment, at least four scalar modules form at least two scalar lines.

[0016] In another particular embodiment, the scalar line is at least two scalar modules connected in series.

[0017] In another particular embodiment, each of the at least two scalar modules is a configurable stream processor containing: an adder unit, a product unit, a function calculation module, and local memory.

[0018] In another particular embodiment, the data switching performed by each first demultiplexer in each scalar device, from the first scalar module in the line, depends on the executed processor operation.

[0019] In another particular embodiment, the processor elements are interconnected via the AXI4-Stream interface.

[0020] In another particular embodiment, the mathematical functions on a vector element are at least: • exhibitor; • natural logarithm; • reciprocal number; • inverse square root; • mathematical functions using LUT tables.

[0021] In another particular embodiment, the arithmetic operations performed by the horizontal operations block are at least: • addition of vector elements obtained from scalar devices; • finding the maximum element among the obtained scalar elements; • finding the minimum element among the obtained scalar elements; • finding the inverse value of scalar elements; • finding the inverse square root of scalar elements.

[0022] In another particular embodiment, the executable operation performed by the processor is specified by at least one configuration stream.

[0023] In another particular implementation, each configuration defines the parameters and type of vector processing.

[0024] In another particular implementation, one configuration is parallelized across each of the scalar devices.

[0025] In another particular embodiment, the processor elements are connected to each other by a bus (interconnect).

[0026] In another particular embodiment, the bus is configured to provide communication between vector processor devices and distribution of configurations and data.

[0027] In another particular implementation, the multiported shared memory contains: • write port for each scalar module; • reading port for each scalar module; • ports for communication with external memory; • ports for communication with the permutation device. BRIEF DESCRIPTION OF DRAWINGS

[0028] The features of the claimed technical solution and a detailed description are given below in the form of attached drawings.

[0029] Fig. 1 illustrates an example of a block diagram of a vector processor with one scalar line.

[0030] Fig. 2 illustrates an example of a block diagram of a vector processor with two scalar lines.

[0031] Fig. 3 illustrates the structural diagram of one scalar line.

[0032] Fig. 4 illustrates an example of the arrangement of elements on a crystal. IMPLEMENTATION OF THE INVENTION

[0033] Below we will describe the concepts and terms necessary for understanding this technical solution.

[0034] A vector processor is a processor in which the operands of some commands can be ordered data arrays—vectors. This differs from scalar processors, which can only operate on one operand at a time.

[0035] The proposed technical solution offers a new approach to creating a high-performance vector processor. Furthermore, the proposed solution enables streaming configuration of the processor's computational elements, which further increases processor performance by eliminating the need to send a set of instructions for each identical command executed. Thus, the proposed architecture of the proposed solution enables streaming operations on data without the need to reconfigure the processor elements after each operation, further enhancing processor performance.

[0036] The term "instructions" as used in this application may refer generally to software instructions or software commands that are written in a given programming language to perform a specific function, such as, for example, configuring the computing elements of a vector processor to perform an operation on a vector, etc. The instructions may be implemented in a variety of ways, including, for example, streaming machine instruction configuration, object-oriented methods, etc. The instructions that perform the processes described in this solution may be transmitted over wired channels from control devices.

[0037] Fig. 1 shows a block diagram of a vector processor 100. Said processor 100 includes a plurality of scalar devices 110, each of which consists of scalar modules 210-1 and 210-2, a first demultiplexer 220, a complex arithmetic operations unit 230, a second demultiplexer 240, a multiport shared memory 120, and a horizontal operations unit 130.

[0038] The architecture of the declared vector processor is primarily aimed at solving massively parallel computing tasks, including the tasks of computing neural networks and artificial intelligence, processing video streams, etc. However, it is worth noting that the declared vector processor should not be limited to machine learning tasks or computations based on neural networks, but can also be used for computations related to various areas of technology that require such processors, such as image processing, etc.

[0039] As indicated above, the claimed vector processor 100 (Vector Engine (VE)) is designed to perform a number of mathematical operations on vectors, for example, 128 words long in FP24 format, with the ability to buffer input and intermediate values. The main application of the processor 100 is the execution of computations of convolutional layers of neural networks after multiplying matrices of features and weights in the matrix multiplication unit (Matrix Engine (ME)). The vector processor is mainly part of a neural network accelerator and is designed to complete the computation of neural network layers. The vector processor 100 is controlled by means of configuration streams received from a control unit (external with respect to the processor). Also, the vector processor 100 is connected to external memory for receiving / sending data to / from it. In yet another particular embodiment, the vector processor 100 is configured to interact with a permutation unit (external with respect to the processor 100).

[0040] The elements of vector processor 100 are located on an integrated circuit (IC) chip. In one particular embodiment, the IC chip may be part of another IC chip, such as a neural network accelerator, which also contains other components associated with processor 100 (a matrix multiplier ME, a control unit CU, a permutation unit, etc.).

[0041] Scalar units (SUs) 110 are scalar computing units, each configured to process one 24-bit word (scalar) out of the total number of words in the vector. Thus, the number of SUs determines the vector length that processor 100 can process in a single pass. In one particular embodiment, processor 100 is divided into 128 scalar SUs, each of which processes one 24-bit word (scalar) out of the 128 words in the vector. Accordingly, in other particular embodiments, processor 100 may comprise a greater or lesser number of SUs, which are part of processor 100.

[0042] The software devices are laid out on the crystal in the form of tiles. An example of the arrangement of software devices in the form of tiles is shown in Fig. 4. On the left, one device 110 is shown, containing four devices 210; on the right, 128 software devices. Each of the devices 110 is independent. In one particular embodiment, the devices 110 may be combined into clusters, for example, 16, 8, etc. The devices are connected to a common data bus (interconnect). The devices 110 are independent and operate in parallel. That is, in accordance with the configuration supplied to the bus from the control unit, said configuration is parallelized across each device, for example, across 128 devices, and each device independently performs an operation with each vector element (scalar). The bus also facilitates interaction with external elements of processor 100.

[0043] In turn, for performing arithmetic operations on a vector element, each software device consists of at least two scalar modules 210 (Vector Subunit (VS)) forming at least one scalar line (210- 1 - 210-2), configured with the possibility of performing arithmetic operations on a vector element; at least one first demultiplexer 220, configured with the possibility of redirecting data from the first scalar module 210-1 (VS) in the line, to the multiport common memory 120 or the second scalar module in the line 210-2; a complex arithmetic operations unit 230 (Special Math Operations (SMO)), connected to at least one scalar module in the line, for example, module 210-2, and configured with the possibility of performing mathematical functions on a vector element;at least one second demultiplexer configured to redirect data from the multiport memory 120 to a scalar module, such as module 210-1 or a matrix multiplier (ME);

[0044] It is worth noting that a line is understood to mean a group of sequentially connected modules 210 within each device 110. Thus, in one particular embodiment, the number of lines within each device 110 depends on the number of matrix multiplication units contained in a higher-level device (for example, a neural network accelerator). Thus, in another particular embodiment, each device 110 may contain two lines formed by four modules 210, as shown in Fig. 2: Line 0: (MEO ->) module 210-1 - module 210-2 -> memory 120. Line 1: (ME1 -) module 210-3 -> module 210-4 - memory 120. In yet another particular embodiment, a line may contain only one module 210. Such an embodiment is possible when executing a mathematical apparatus that does not require complex calculations.

[0045] Devices 210 in different lines operate independently and are connected to each other only through memory 120. This means that the lines are identical, and modules 210 in the same position in the line have identical functionality. For example, the second modules 210 in the line (210-2 and 210-4) are connected to block 230 and, through it, to horizontal operations block 130.

[0046] Returning to Fig. 1, scalar modules 210 provide for the execution of arithmetic operations on data from the matrix multiplication unit (ME) or memory 120. Both modules 210-1 and 210-2 in the line are identical, each consisting of the following elements shown in Fig. 3: an adder unit 310, a product unit 320, a local memory 330, and an interface unit for communication with the complex mathematical operations unit 230. Fig. 3 shows an example of one line consisting of two scalar modules 210, such as, for example, modules 210-1 and 210-2. Modules 210 have two inputs (the first from the matrix multiplication unit (ME), the second from the memory of the vector processor) and two outputs (to the memory of the vector processor). The output of the first module 210 in the line is fed to a demultiplexer, such as demultiplexer 220 controlled by the instruction of this module.Thus, if the instruction specifies the OUTPUT VM operation (writing to the vector processor memory), the data from module 210-1 is redirected to be written to memory 120; if the operation is OUTPUT VS, the data is fed to the input of the next module 210 in the line, for example, module 210-2.

[0047] This principle of organizing the architecture of processor 100 ensures a reduction in the number of instructions required to execute an operation in processor 100 and provides the possibility of streaming execution of operations. Thus, such a principle also makes it possible to achieve an increase in the performance of processor 100 by reducing the number of calls to processor 100 for its configuration. That is, the serial method of connecting modules 210 in a line allows for a reduction in the number of operations required by the processor to perform a calculation that requires an intermediate result. Thus, as an example, consider the operation (A + B) * C. When processing such an operation in parallel, it will be necessary to configure the module to execute the first operation, namely, performing the operation (A + B), storing this result in memory, reading the result from memory for multiplication by C. Accordingly, in the claimed solution, due to the presence of a serial line of arithmetic units (modules 210), module 210- 1 will perform the operation A+B and pass the result to module 210-2 for multiplication. This architecture allows for sequential execution of operations within the computational pipeline. Support for the processor's streaming configuration (DataFlow) further increases the speed of processor 100 by eliminating the need for multiple configurations of computational elements to perform similar operations. With a streaming configuration, the operation to be executed is specified once and is performed for the entire incoming data packet. This example is provided to more accurately illustrate the essence of the claimed technical solution and should not limit the complexity of the operations performed. Modules 210-1 and 210-2 can be used to perform more complex operations, such as exp[([(A+B)*C]+D)*E], where [(A+B)*C] will be executed by the first module in the pipeline, and the remainder by the second.

[0048] Accordingly, a similar situation will apply to modules 210-3 and 210-4, shown in Fig. 2, which form the second line. As noted above, the lines are identical. The output of the last module 210 in the line is always written to memory 120 (in VE MEM). Each module 210 also has read access to memory 120. It is worth noting that the presence of the second and subsequent lines (third, fourth, etc.) provides increased parallel computing, but increases the area and power consumption of processor 100.

[0049] Each of the arithmetic units comprising module 210 may be a configurable stream processor and supports its own unique instruction set. For adder unit 310, this primarily includes addition and subtraction, while for product unit 320, it includes product. Furthermore, as noted above, to execute functions on operands, the last module in the line is connected to block 230 via a communication interface unit with a complex mathematical operations unit. In more detail, adder unit 310 is capable of executing simple arithmetic operations, such as summation, finding the minimum and maximum number, accumulation, comparison operations (greater than / less than, equality), etc. Block 320 is capable of executing operations such as multiplication, exponentiation, etc.

[0050] Accordingly, as can be seen from the diagram, the second module 210 in the line is connected, via an interface block (SMO IO) to the block of complex mathematical operations 230 (Special Math Operations, SMO), which is common to the io lines. The specified block 230 is intended for the execution of rare and complex arithmetic operations, in accordance with the specified configuration 230.

[0051] Said block 230 enables the execution of rare and complex arithmetic operations. Since block 230 does not have its own instructions, SMO IO (the interface block of the last module 210 in the line, shown in Fig. 3) adds the number of the executed operation to the USER bits of the AXI packet before sending it to block 230. Only the last (e.g., second) modules 210 in the line are connected to block 230. Accordingly, the first modules 210 in the line do not have external connections and cannot be configured to execute operations. Said block 230 is described in more detail below.

[0052] It's also worth noting that, although this solution describes examples for one or two lines consisting of two or four modules 210, the number of lines may be greater or lesser. The number of modules 210 in each line may also vary and depends on the complexity of the operation being performed. For example, in one particular embodiment, each line may contain three or more modules 210. Accordingly, in such a line design, the first and last modules will have connections identical to the first and second modules in the line described above, while the remaining modules in the line will serve as intermediate modules and provide the required computing power.

[0053] Continuing to expand on modules 210, each arithmetic unit can be configured to perform only one of its operations at a time. The current operation of each unit is specified by an instruction for module 210. Arithmetic units can be connected in a chain, the composition and order of which are dynamically specified by an instruction for module 210. The chain specifies only the source of the first operand for each arithmetic unit. However, addition 310 and multiplication 320 units also use a second operand for their binary operations, which can be taken either from VS MEM (memory 330) or read from memory 120 (VE MEM). Furthermore, the presence of local memory 330 allows for storing intermediate calculation results directly within module 210 if the calculation result does not exceed the size of local memory 330. This approach also allows for using the result for subsequent calculations without accessing shared memory 120.

[0054] Returning to block 230 (Special Math Operations (SMO)) provides execution of complex rare arithmetic operations that are not executed in ii arithmetic units of modules 210, namely: exponential EXP; natural logarithm LN; reciprocal number 1 / x (RCP); reciprocal square root 1 / ^x (RSQRT); mathematical functions using LUT tables, ReLU and other arbitrary activations; horizontal operations via the XLU unit.

[0055] One block 230 services both lines of modules 210, receiving data from the last modules 210 in the lines, for example, modules 210-2 and 210-4. Block 230 has a separate 24-bit input AXI bus for each module 210. Data for processing by block 230 is formed and sent via the SMO IO device inside modules 210. The data path to block 230 depends on which operation needs to be executed. Block 230 does not have its own instructions, so the operation to be executed is specified by its number in the USER bits of the incoming AXI packet. The operation number of block 230 is added to the AXI packet by the SMO IO device based on the current instruction of module 210. Block 230 is connected to the horizontal operations block 130. It is worth noting that in one particular embodiment, block 230 is configured to, for example, by means of a task scheduler, sequentially send the operations required for execution in block 130 in accordance with their priority. That is,block 230 sends some operations, which independently come from modules 210, to block 130 in turn.

[0056] The horizontal operations unit 130 (Cross-Lane Unit (XLU)) is connected to the scalar devices 110 through the units 230 of each scalar device and is configured to: perform arithmetic operations with the combined elements of a vector; and transmit the result of performing an arithmetic operation with the combined elements of a vector to the scalar devices.

[0057] All horizontal operations executed by block 130 accept an entire vector, for example, 128 24-bit words. Each 24-bit scalar is sent from block 230 to one of, for example, 128 scalar modules 210. Block 130 executes the operation only after all 128 scalars have arrived. Data is fed to block 130 from independent devices 110, for example, from 128 devices 110, via the data bus (interconnect). That is, all devices 110 independently send the results of their calculations to block 130 via the data bus.

[0058] Block 130 supports the following operations: ADD TREE; ADD TREE+RCP; ADD TREE+RSQRT; MAX TREE; MAX TREE+RCP; MAX TREE+RSQRT; MIN TREE; MIN TREE+RCP; MIN TREE+RSQRT. Where: ADD TREE - add 128 numbers, MAX TREE - find the maximum of 128 numbers, MIN TREE - find the minimum of 128 numbers, RCP = 1 / х, RSQRT = 1 / x.

[0059] The result of block 130 operations is a single 24-bit scalar. A copy of the resulting 24-bit number is sent to block 230 of each of the 128 devices 110.

[0060] Multiport shared memory 120 (VMEM) is the main memory for buffering intermediate neural network computation results. It's worth noting that the term "shared memory" refers to memory accessible by all elements. Physically, memory 120 is divided into multiple "chunks" corresponding to the number of units 110. This memory can store data in various formats used to configure various nodes, such as integer permutation indices for the permutation unit and tables for calculating activation functions. Memory 120 stores the results of modules 210, from where they are fed back to modules 210, to matrix multipliers as input data (features), to the permutation unit, or to external memory, such as HBM memory.Thus, the memory 120 is connected at least to the external memory (EM), the second demultiplexer of each scalar device and the matrix multiplication device and is configured to buffer and store the results of intermediate calculations over the vector elements of each scalar device.

[0061] In one particular embodiment, the memory 120 has the following ports: a write port for each scalar module 210; a read port for each scalar module 210; ports for communication with an external memory (EMM); ports for communication with a shuffler.

[0062] In another particular embodiment, the memory 120 may be a shared memory with 6 ports for reading and 6 ports for writing. In total, therefore, the memory 120 may have 12 ports: four ports for writing from each module 210; four ports for reading into each module 210, wherein the ports for reading into the first modules 210 in a line are divided through the second demultiplexers 240 with the data supply to the ME devices in the same line; two ports (reading and writing) for memory controller devices (DMA); two ports (reading and writing) for the shuffler permutation device. Accordingly, to organize the memory 120 with a different number of lines (one, three, etc.), the number of ports for writing from each module 210 and for reading into each module 210 can be changed (decreased / increased) in accordance with the number of modules

[0063] All 120 memory ports are configurable. Port configuration, except for the shuffler ports, specifies the access address sequence; access addresses on the shuffler ports are generated by the shuffler module itself.

[0064] All elements within the VE processor 100 use the AXI4-Stream interface to exchange data with each other and with external devices, as well as to receive instructions from the Control Unit (CU). AXI enables automatic control of data transfer across an arbitrarily long chain of devices without distortion or loss. For example, if a device in the chain cannot accept new data, it notifies the sending device by resetting the ready signal. In turn, the sending device cannot send data without a ready signal from the receiver, meaning it cannot replace the data it sent and stops sending the ready signal to its sender. Thus, transmission is halted throughout the chain, all the way to the data source (e.g., the SHMEM read port), until the cause of the stoppage at the very first device is resolved.

[0065] The length of the input scalar sequence is not specified and can be arbitrarily long. The end of the sequence is indicated by the signal last=l : this signal arrives at the beginning of the device chain along with the last data packet of the sequence and is transmitted through the entire chain.

[0066] Processor 100 is configured to exchange data with devices that handle vector data: a matrix multiplier (ME) and a shuffler. Thus, as part of a device such as a neural network accelerator, vector processor 100, which may be part of said accelerator, receives a vector consisting of 128 scalars in FP24 format. As noted above, the vector may come, for example, from a matrix multiplier. Each scalar is transmitted to its own device 110. The index of device 110 is equal to the index of the scalar in the vector. Transmission is performed via an independent 24-bit AXI bus (interconnect bus). Upon receipt of all 128 scalars, a vector is formed on the receiving side (the index of the scalar in the vector is equal to the index of the transmitting device 110).

[0067] Memory 120 exchanges data with external memory (HBM - High Bandwidth Memory). A pair of memory controllers (DMA - Direct Memory Access) are dedicated to this purpose: for writing to the HBM and a pair of DMA devices are dedicated to reading from the HBM.

[0068] The operation of vector processor 100 is controlled by a set of configuration streams (instructions) for each element within it. Each configuration defines the processing parameters for a certain number of vectors on a single element, for example, device 110. During operation, configurations are replaced as their processing is completed. The operation of each device 110 is generally independent; synchronization is achieved through data dependencies. Internal communications between devices are implemented using the AXI4-Stream protocol.

[0069] To configure the devices included in processor 100, the control unit (CU) may have multiple instruction queues (two or more). Each instruction queue may be transmitted via its own bus. Let's consider a variant with four instruction queues (Threads). Accordingly, the control unit will be connected to processor 100 via four buses for transmitting instructions to vector processor 100. Each bus is 64 bits wide. All instructions are divided into 32-bit words. For example, an instruction for module 210 consists of 20 words (640 bits). The words of one instruction are transmitted sequentially along one of the four buses. Configuration of device 110 occurs after receiving the last word of the current instruction.

[0070] Thus, let us consider as an example a processor 100 with two lines. Instructions of different elements of processor 100 will be distributed among 4 queues of the CU (control unit) as follows: Thread O (queue O): modules 210-1 and 210-2, memory ports 120 for reading to module 210-1 / 210-2 and writing from module 210-2, demultiplexer 240 (in memory port instruction 120), demultiplexer 220 (in module 210-1 instruction).

[0071] Thread l (queue!): modules 210-3 and 210-4, memory ports 120 for reading to module 210-3 / 210-4 and writing from module 210-4, second demultiplexer 240 (in memory port instruction 120), second demultiplexer 220 (in module 210-4 instruction).

[0072] Thread_2 (queue_2): memory ports 120 for reading / writing to / from external memory (HBM).

[0073] Thread_3 (queue 3): memory ports 120 for reading / writing to the shuffler.

[0074] All instructions are written into their queues in program order (in the order of the instructions in the HBM).

[0075] 128 identical devices 110 (scalar devices) are configured by a single instruction, duplicated 128 times. For example, 128 modules 210-1 accept the same instruction. The same applies to instructions for memory ports 120; for example, the first memory write port 120 in each device 110 is configured by the same instruction. The instruction is distributed to all devices 110 in parallel along the Broadcast device tree. However, due to the wide dispersion of devices 110 across the core die, a single instruction reaches different devices 110 in a different number of cycles. After executing the instruction, a "done" signal is transmitted from each device 110: first, "done" is collected from each of the 128 devices 110, and only then is the common "done" signal sent to the CU.

[0076] The instruction of module 210-1 also configures the demultiplexer 220 to select where the data from module 210-1 is transferred: to the input of module 210-2 or to be written to memory 120. Similarly, the instruction of module 210-3 configures the second demultiplexer 220. The demultiplexers 240 (each for its own line) are configured by the instructions of the read ports of memory 120. Each of the two states of the demultiplexers 240 corresponds to a separate instruction of the memory port 120: one for reading into modules 210, the other for reading into ME.

[0077] Another specific embodiment also provides support for looping operations of modules 210 via a dedicated configuration field. Blocks 230 (SMO) and 130 (XLU) do not have their own configurations.

[0078] Let us consider one of the operating scenarios for processor 100. A vector, say, 128 numbers long, is input to processor 100. This vector comes from a matrix multiplication (ME) unit, which can be implemented, for example, as a systolic array. All 128 numbers are then distributed across 128 devices 110 via the data bus (interconnect) in accordance with their index in the vector. In each ME unit, an arithmetic operation is performed on the vector element, depending on the configuration. It is worth noting that, depending on the complexity of the operation, and therefore the configuration, unit 230 in the ME unit line, as well as the second module in line 210-2, may or may not be involved. In accordance with the executable configuration, the calculation result from module 210-1, via demultiplexer 220, is either transferred to memory 120 or transferred to the next module 210 in line / block 230 / block 130, etc.Additionally, in one particular embodiment, when initially operating module 230, the configuration may include instructions that translate a number. vectors from ME through module 210-1 to module 210-2 for subsequent execution of the operation in block 230 and / or in block 130. After execution of the arithmetic operations by each of the devices 110, the results are transferred to memory 120. As indicated above, memory 120 acts as a synchronizing device for 128 asynchronous devices 110, and stores the execution results, for example, in the form of a vector. Then, if further operations with the calculated vector are required, via demultiplexer 240, the said result can be fed either to the matrix multiplier (in the form of a vector), or to each of the PO devices (in the form of scalars in accordance with the index in the vector).

[0079] Thus, the submitted application materials described various embodiments of a vector processor that ensure high operating speed by organizing computing elements as scalar FPGAs with lines that in turn contain sequential arithmetic units (modules 210). This, accordingly, enables operations on vector elements that require actions with intermediate results, i.e., enables pipelined operations. Furthermore, the claimed solution enables streaming configuration of the processor's computing elements, including vector memory, which further improves processor performance by eliminating the need to send commands for each identical operation to all processor elements.

[0080] The specific selection of elements of the processor 100 for the implementation of various software, hardware and / or architectural solutions may vary while maintaining the required functionality provided.

[0081] The submitted application materials disclose preferred embodiments of the technical solution and should not be construed as limiting other, specific embodiments within the scope of the requested legal protection that are obvious to specialists in the relevant technical field. Therefore, the scope of this technical solution is limited only by the scope of the attached claims.

Claims

FORMULA 1. An on-chip vector processor containing: • scalar devices, each of which is configured to obtain a vector element from a matrix multiplication device and consists of: o at least two scalar modules forming at least one scalar line, configured to perform arithmetic operations on a vector element; o at least one first demultiplexer configured to redirect data from the first scalar module in the line to a multiport shared memory or a second scalar module in the line; o a complex arithmetic operations unit connected to at least one scalar module in the line and configured to perform mathematical functions on a vector element; o at least one second demultiplexer configured to redirect data from the multiport memory to a scalar module or a matrix multiplication device; • a multi-port shared memory connected to at least an external memory, a second demultiplexer of each scalar device and a matrix multiplication device, configured to buffer and store the results of intermediate calculations over the elements of the vector of each scalar device; • a horizontal operations block associated with scalar devices, configured to: perform arithmetic operations with combined elements of a vector; and transmit the result of performing an arithmetic operation with combined elements of a vector to scalar devices.

2. A vector processor according to claim 1, characterized in that the scalar devices are located on the crystal in the form of a tile.

3. The vector processor according to claim 1, characterized in that each scalar device contains at least four scalar modules.

4. The vector processor according to claim 3, characterized in that at least four scalar modules form at least two scalar lines.

5. The vector processor according to claim 1, characterized in that the scalar line is at least two scalar modules connected in series.

6. The vector processor according to claim 1, characterized in that each of the at least two scalar modules is a configurable stream processor containing: a summation unit, a product unit, a function calculation module, and local memory.

7. The vector processor according to claim 1, characterized in that the data switching performed by each first demultiplexer in each scalar device, from the first scalar module in the line, depends on the executed processor operation.

8. A vector processor according to claim 1, characterized in that the mathematical functions over the vector element are at least: • exhibitor; • natural logarithm; • reciprocal number; • inverse square root; • mathematical functions using LUT tables.

9. The vector processor according to claim 1, characterized in that the arithmetic operations performed by the horizontal operations block are at least: • addition of vector elements obtained from scalar devices; • finding the maximum element among the obtained scalar elements; • finding the minimum element among the obtained scalar elements; • finding the inverse value of scalar elements; • finding the inverse square root of scalar elements.

10. The vector processor according to claim 1, characterized in that the executable operation performed by the processor is specified by at least one configuration stream.

11. The vector processor according to claim 10, characterized in that each configuration determines the parameters and type of vector processing.

12. The vector processor according to paragraph 10, characterized in that one configuration is parallelized to each of the scalar devices.

13. A vector processor according to claim 1, characterized in that the processor elements are connected to each other by a bus.

14. The vector processor according to claim 13, characterized in that the bus is configured to provide communication between devices of the vector processor and distribution of configurations and data.

15. The vector processor according to claim 1, characterized in that the multi-port shared memory contains: • write port for each scalar module; • reading port for each scalar module; • ports for communication with external memory; • ports for communication with the permutation device.

Images