XCPU supercomputer chip with three-dimensional architecture

Abstract

The invention relates to the technical field of chips, in particular to an XCPU supercomputing chip with a three-dimensional structure. The multi-layer printed circuit board is located below the packaging substrate, a silicon through hole block is arranged between the multi-layer printed circuit board and the packaging substrate, and a plurality of intermediate layers are stacked above the packaging substrate through the silicon through hole block; each intermediate layer comprises a substrate, the substrate is in a long strip shape, a silicon through hole block is arranged above the substrate, an XCPU super-computing chip is installed in the middle section of the silicon through hole block, a plurality of memory chips are arranged on the two sides of the XCPU super-computing chip, supporting blocks are arranged at the two ends of the silicon through hole block, and a silicon through hole block is arranged above the supporting blocks and makes contact with the upper intermediate layer. The XCPU supercomputing chip has a good acceleration effect on a convolution kernel thread pipeline algorithm and a deconvolution kernel thread pipeline algorithm, the XCPU supercomputing chip is a chip with superstrong computing power and accurate calculation, and standardized construction of domestic hardware, software and data can be promoted through compatibility with the ecological environment.

Description

Technical Field

[0001] The present invention relates to the field of chip technology, and in particular to an XCPU supercomputer chip with a three-dimensional architecture. Background Art

[0002] The XCPU supercomputing chip is a three-dimensional architecture eXtreme Computing Performance Unit system integrated chip specially designed to solve the training and application of big data artificial intelligence algorithms (such as convolution kernel thread pipeline algorithm and deconvolution kernel thread pipeline algorithm). It consists of an FPGA-based programmable area, multiple I / O devices surrounding the area, and an integrated 64-bit RISC-V processor. It is a highly integrated artificial intelligence processor on the chip and is suitable for various practical application fields involving imaginary data.

[0003] like Figure 1 As shown in the figure, (a) is the FPGA routing model, and (b) is the connection block F C =2, (c) is the exchange block F S =3, the RISC-V_FPGA in the XCPU supercomputing chip is an interconnected network of gates, registers, and wires with programmable connections. Programmable connections make it possible to make almost any type of digital electronic circuit. Generally speaking, basic blocks refer to programmable lookup tables (LUTs), registers, wires, I / O blocks (IOBs), and programmable connections (switching blocks) used to connect basic blocks and wires. LUT is a simple way to replicate complex logic functions (consisting of AND, OR, NOT, XOR, etc.) with a small number of inputs (usually 5 or 6). The LUT is combined with a register that captures and temporarily saves the output of the logic function to form a configurable logic block CLB, the basic building block of RISC-V_FPGA. The other parts are wires and programmable connections that enable the way to connect CLBs to perform more complex functions such as mathematical functions, and programmable I / O blocks (IOBs) to connect outside the chip. There are usually other functional blocks, such as memory and multiplier accumulators (MACs) and digital signal processors (DSPs), which provide a more efficient way to implement some common functions than building CLBs from scratch. The main reason for using RISC-V_FPGA to solve problems that traditional computers cannot solve is that RISC-V_FPGA is faster, which is mainly achieved in two ways: pipelines and parallel computing units.

[0004] Existing XCPU chips cannot train and calculate big data artificial intelligence models faster. XCPU supercomputing chips have poor computing power, low computing accuracy and poor compatibility. Summary of the invention

[0005] The technical problem to be solved by the present invention is: in order to solve the problems existing in the prior art in the above-mentioned background technology, a three-dimensional XCPU supercomputer chip is provided.

[0006] The technical solution adopted by the present invention to solve its technical problems is: an XCPU supercomputer chip with a three-dimensional structure, including a multi-layer printed circuit board and a packaging substrate, the multi-layer printed circuit board is located below the packaging substrate, a silicon through via block is provided between the multi-layer printed circuit board and the packaging substrate, and a plurality of intermediate layers are stacked above the packaging substrate through the silicon through via block; the intermediate layer includes a substrate, the substrate is in a long strip shape, a silicon through via block is provided above the substrate, the XCPU supercomputer chip is installed in the middle section of the silicon through via block, a plurality of memory chips are provided on both sides of the XCPU supercomputer chip, support blocks are provided at both ends of the silicon through via block, and a silicon through via block is provided above the support block to contact the upper intermediate layer.

[0007] Furthermore, the cross-section of the substrate in the vertical direction is in a "gate" shape.

[0008] Furthermore, partition columns are provided on the four lower corners of the substrate, and a plurality of intermediate layer connection marks are provided around the periphery of the XCPU supercomputer chip, with gaps between the partition columns and the intermediate layer connection marks.

[0009] Furthermore, a through silicon via block is provided above the substrate, and a plurality of stacked XCPU supercomputing chips are installed in the middle section of the through silicon via block, and a plurality of memory chips are provided on both sides of the XCPU supercomputing chip.

[0010] Furthermore, the XCPU supercomputing chip includes a programmable area of ​​FPGA, multiple I / O devices surrounding the area, and an integrated RISC-V processor, which is used to supervise the loading, selection and running of RISC-V_FPGA configurations and other tasks.

[0011] Furthermore, the XCPU supercomputing chip uses dynamically controlled switch block connections, and new settings of the switch block configuration will be changed by loading the new bitstream into the local switch block part of the j-tag chain or by loading in parallel from adjacent registers.

[0012] Furthermore, the XCPU supercomputing chip uses dynamically controlled configurable switch blocks and wires separated from fixed configuration time wires. The switch block is controlled by a set of registers, one of which is used to maintain the current connection configuration and the other is used to save the next configuration.

[0013] Furthermore, the XCPU supercomputing chip uses actual non-blocking crossbar switches implemented by RISC-V_FPGA hardware, and the non-blocking crossbar switches are connected in series.

[0014] Furthermore, the XCPU supercomputing chip supports multiple independent concurrent operation memory access channels.

[0015] Furthermore, the XCPU supercomputing chip supports multiple independent concurrent operation PCI channels to achieve fast I / O.

[0016] Furthermore, the XCPU supercomputing chip uses 100GigE that supports multiple independent concurrent operations to achieve fast I / O.

[0017] Furthermore, the convolution kernel thread pipeline algorithm used by the XCPU supercomputing chip is a neural network algorithm.

[0018] Furthermore, neural network convolution, neural network sampling operations, fully connected neural networks or recurrent neural networks are implemented in the RISC-V_FPGA hardware of the XCPU supercomputing chip.

[0019] Furthermore, the deconvolution kernel thread pipeline algorithm used by the XCPU supercomputing chip is divided into the Fermi-Dirac probability distribution function model and the Bose-Einstein probability distribution function model.

[0020] Furthermore, the deconvolution kernel thread pipeline algorithm is implemented in the RISC-V_FPGA hardware of the XCPU supercomputing chip.

[0021] Furthermore, the XCPU supercomputing chip is a parallel computing architecture that uses a systolic array or a gate array.

[0022] The beneficial effects of the present invention are that 1) the XCPU supercomputing chip of the present invention is a system on chip with RISC-V_FPGA resources, and uses 28nm, 14nm and below 14nm processes to build a core thread pipeline configuration, including multiple digital signal processors (DSPs) to perform calculations and processing, and multiple pipelines are run in parallel, which can train and calculate big data artificial intelligence models faster;

[0023] 2) The acceleration effect of the XCPU supercomputing chip on the convolution kernel thread pipeline algorithm is as follows: the clock cycles required to read and execute a pass in all 7 nested for loops of the first part of the convolution operation are approximately (number of intermediary layers * 12500); the clock cycles required to read and execute a pass in all 5 nested for loops of the second part of the sampling operation are approximately (number of intermediary layers * 8700); the clock cycles required to read and execute a pass in all 3 nested for loops of the third part of the fully connected neural network operation are approximately (number of intermediary layers * 3250); the clock cycles required to read and execute a pass in all 12 nested for loops of the fourth part of the recurrent neural network operation are approximately (number of intermediary layers * 29000);

[0024] 3) The acceleration effect of the XCPU supercomputing chip on the deconvolution kernel thread pipeline algorithm is as follows: the clock cycles for reading and executing a pass in all 4 nested for loops are approximately (number of interposer layers * 3450);

[0025] 4) The XCPU supercomputing chip has super computing power and precise calculations. Through its compatibility with the ecological environment, it can promote the standardization of domestic hardware, software and data. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0027] Figure 1 It is a structural schematic diagram of the RISC-V_FPGA routing model in the XCPU supercomputing chip of the present invention;

[0028] Figure 2 It is a schematic diagram of the structure of the RISC-V_FPGA in the XCPU supercomputer chip of the present invention;

[0029] Figure 3 is a cross-sectional view of the manufacturing structure of Example 1 of the present invention;

[0030] Figure 4 is a cross-sectional view of the manufacturing structure of Example 2 of the present invention;

[0031] Figure 5 is a top view of the manufacturing structure of Example 3 of the present invention;

[0032] Figure 6 The present invention is Figure 5 A cross-sectional view after the dotted line portion is cut open;

[0033] Figure 7 is a cross-sectional view of the manufacturing structure of Example 4 of the present invention;

[0034] Figure 8 This is a schematic diagram of the XCPU supercomputing chip stack accessing all memories in Example 4 of the present invention;

[0035] Fig. 9 It is a schematic diagram of the processor field of the XCPU supercomputer chip of the present invention for for loop operation;

[0036] Fig.10 It is a schematic diagram of the basic structure of the neural network in the convolution kernel thread pipeline algorithm of the present invention;

[0037] Fig.11 It is a schematic diagram of weighted connections between neural network nodes in the convolution kernel thread pipeline algorithm of the present invention;

[0038] Fig.12 It is a flow chart of convolutional neural network modeling of the present invention;

[0039] Fig.13 It is a schematic diagram of a deep neural network in the RISC-V_FPGA in the XCPU supercomputing chip of the present invention;

[0040] Fig.14 is a schematic diagram of a non-blocking crossbar switch of the present invention;

[0041] Fig.15 is a schematic diagram of a non-blocking crossbar switch between layers of a deep neural network according to the present invention;

[0042] Fig.16 It is a schematic diagram of the pipeline steps of the for loop operation in the convolution kernel thread pipeline algorithm executed by the XCPU supercomputing chip of the present invention;

[0043] Fig.17 It is a pipeline step process timetable of the for loop operation in the convolution kernel thread pipeline algorithm executed by the XCPU supercomputing chip of the present invention;

[0044] Fig.18 It is a schematic diagram of the operation steps of the for loop in the deconvolution kernel thread pipeline algorithm executed by the XCPU supercomputing chip of the present invention;

[0045] Fig.19 It is a schematic diagram of the pipeline steps of the for loop operation in the deconvolution kernel thread pipeline algorithm executed by the XCPU supercomputer chip of the present invention;

[0046] Fig. 20 It is a pipeline step process timetable of the for loop operation in the deconvolution kernel thread pipeline algorithm executed by the XCPU supercomputer chip of the present invention;

[0047] Fig.21 It is a comparison table of the five algorithms of the present invention using CPU accelerated calculation and efficient XCPU calculation speed;

[0048] In the figure: 1. Multilayer printed circuit board, 2. Package substrate, 3. Through silicon via block, 4. Substrate, 5. XCPU supercomputer chip, 6. Memory chip, 7. Support block, 8. Spacer column, 9. Intermediary layer connection imprint, 10. Non-blocking cross switch. DETAILED DESCRIPTION

[0049] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, which only illustrate the basic structure of the present invention in a schematic manner, and therefore only show the components related to the present invention.

[0050] Embodiment 1:

[0051] like Figures 1 to 3 , 9 to 21, a three-dimensional architecture XCPU supercomputer chip is shown in order to achieve higher throughput, such as Figure 3As shown, a through silicon via block 3 is used to support a 3D structure with up to 10 layers. The bottom interposer is different from other interposers and is directly connected to the package substrate 2. The bottom interposer includes a multi-layer printed circuit board 1 and the package substrate 2. The multi-layer printed circuit board 1 is located below the package substrate 2. A through silicon via block 3 is provided between the multi-layer printed circuit board 1 and the package substrate 2. A plurality of interposers are stacked above the package substrate 2 through the through silicon via block 3.

[0052] The intermediary layer includes a substrate 4, which is in a long strip shape. A through silicon via block 3 is arranged above the substrate 4. An XCPU supercomputer chip 5 is installed in the middle section of the through silicon via block 3. A number of memory chips 6 are arranged on both sides of the XCPU supercomputer chip 5. Support blocks 7 are arranged at both ends of the through silicon via block 3. A through silicon via block 3 is arranged above the support block 7 to contact the upper intermediary layer.

[0053] Among them, the XCPU supercomputing chip 5 includes a programmable area of ​​FPGA, multiple I / O devices surrounding the area, and an integrated RISC-V processor, which is used to supervise the loading, selection and running of RISC-V_FPGA configurations and other tasks.

[0054] XCPU supercomputing chip 5 supports multiple independent concurrent operation memory access channels.

[0055] The XCPU supercomputing chip 5 supports multiple independent concurrent operation PCI channels to achieve fast I / O.

[0056] The XCPU supercomputing chip 5 uses 100GigE that supports multiple independent concurrent operations to achieve fast I / O.

[0057] Convolution is an important operation in analytical mathematics. The simple definition is as follows: Let f(x), g(x) be Two integrable functions on This integral defines a new function h(x), called the convolution of functions f and g, denoted by h(x) = (f·g)(x). Deconvolution is the process of finding a solution to a convolution equation of the form (f·g)(x) + ε = h(x). In artificial intelligence algorithms, h represents the observed value Y, f·g represents the model value P, and ε represents the loss.

[0058] The convolution kernel thread pipeline algorithm is a neural network algorithm that is mainly used for classification, perception, understanding, discovery, prediction and creation. It can be used for data analysis of behavior prediction, large linear models of data analysis, speech and sound recognition, text-based applications, time series, video detection speech recognition systems, image and video recognition and tagging, self-driving cars, text summarization, sentiment analysis and other aspects.

[0059] The convolution kernel thread pipeline algorithm used by XCPU supercomputing chip 5 is a neural network algorithm, which divides the data into N parts. The mathematical framework of the deep learning algorithm for each part is as follows:

[0060] Suppose the mini-batch sample set randomly drawn from the data is:

[0061] {(x i ,y i ), i=1,…,M} (1)

[0062] The empirical risk function of the sample set is defined as:

[0063]

[0064] Where θ=[θ 1 ,θ 2 ,…,θ p ] T is the parameter vector (3)

[0065] Let the gradient vector be:

[0066] g=[g 1 , g 2 , …, g p ] T (4)

[0067] in,

[0068] Let the accumulated square gradient vector of the iter+1th iteration be:

[0069]

[0070] in,

[0071] Then the solution for the iter+1th iteration is:

[0072]

[0073] in,

[0074] Among them, δ is a constant greater than 0, ∈ is the learning rate, and ρ is the decay rate.

[0075] Implement neural network convolution, neural network sampling operations, fully connected neural networks or recurrent neural networks in the RISC-V_FPGA hardware of the XCPU supercomputing chip 5.

[0076] The deconvolution kernel thread pipeline algorithm used by the XCPU supercomputing chip 5 is divided into the Fermi-Dirac probability distribution function model and the Bose-Einstein probability distribution function model.

[0077] Among them, the Fermi-Dirac distribution function model is as follows:

[0078] Under the constraints on the spatial manifold, find the minimum value of the unit log-likelihood function J(θ, α):

[0079]

[0080] The model and signal manifold space can be expressed as:

[0081]

[0082]

[0083] Where f(x|θ,α)=θ μ δ μ (x-α δ )+o(...) (13)

[0084] The gradient vector and Hessian matrix are:

[0085]

[0086]

[0087] The iterative solution to the nonlinear optimization problem is:

[0088]

[0089]

[0090] Among them, the Bose-Einstein distribution function model is as follows:

[0091] Under the constraints on the spatial manifold, find the minimum value of the unit log-likelihood function J(θ, α):

[0092]

[0093] The model and signal manifold space can be expressed as:

[0094]

[0095]

[0096] where f(x|θ,α)=θ μ δ μ (x-α δ )+o(...) (21)

[0097] The gradient vector and Hessian matrix are:

[0098]

[0099]

[0100] The iterative solution to the nonlinear optimization problem is:

[0101]

[0102]

[0103] The deconvolution kernel thread pipeline algorithm is implemented in the RISC-V_FPGA hardware of the XCPU supercomputing chip (5).

[0104] The XCPU supercomputer chip with a three-dimensional architecture and extreme computing performance unit is a system-on-chip with RISC-V_FPGA resources, which can calculate and train artificial intelligence models faster. The problems it aims to solve include the training and application of convolution kernel thread pipeline algorithms and deconvolution kernel thread pipeline algorithms.

[0105] The training data size is usually 100 million rows and 100 columns, each column is 8 bytes long, and the total input data is 800 billion bytes. The maximum workload supported by the memory on the interposer layer is 128GB.

[0106] The XCPU supercomputing chip can use RISC-V_FPGA resources to perform all the calculations required by the algorithm as quickly as possible, thereby realizing a pipeline field configuration built using a 28nm process.

[0107] A chip with multiple RISC-V_FPGA resources is made using 28nm process, including a large number of DSPs for computational processing, so more pipelines can be run in parallel.

[0108] for Figure 2The reconfigurable parts in the RISC-V_FPGA mainly include: (1) LUT - lookup table; (2) CLB - reconfigurable logic block; (3) block RAM memory block - small block of high-speed local memory; (4) other RAM blocks - larger but possibly slower memory blocks; (5) DSP blocks including MAC - digital signal processing blocks containing multipliers, adders and accumulators; (6) internal routing resources - configurable connection structures connecting LUTs, CLBs, etc.; (7) non-blocking crossbar resources. In addition: (8) QSFP 28 - fiber or copper adapter connector; (9) 100GigE - 100GB Ethernet; (10) PCIe16x - multi-channel high-speed input and output; (10) Security block - the security block has the required ID number, encryption and security handshake to support license use and prevent reverse engineering of configuration bitstreams and RISC-V_FPGA configuration; (11) RISC V 64-bit core - runs the part of the code that is not accelerated by the RISC-V_FPGA configurable logic; (12) L3 cache - used by the RISC V 64-bit core (the core has L1 and L2 caches as part of the core); (13) Custom instruction registers - support custom RISC-V instructions to select and control accelerator functions in the RISC-V_FPGA configurable part; (14) Main memory - DDR of DRAM chips (128GB) on the interposer 4 / QDDR+ memory interface, smaller but faster QDDR+DRAM can also be used; (15) USB - used for debugging and setting functions; (16) JTAG Loader - used to load RISC-V_FPGA configuration; (17) Clock Generator - drives the clock of RISC-V_FPGA and RISC V core; (18) GPIO 26X general input and output - used to implement other interfaces, indicators, etc.; (19) SSD interface PCIe 4x - used for directly connected M.2 solid-state disk modules.

[0109] The RISC-V 64-bit core controls operations by connecting to each non-blocking crossbar 10, RISC-V_FPGA node instance, off-chip memory interface, etc. Some controls can be achieved by setting RISC-V 64-bit core set values ​​in registers of the RISC-V_FPGA for individual nodes or groups of nodes.

[0110] For most operations of AI algorithms, all nodes are performing the same operation as a non-blocking crossbar switch, so one register load step is sufficient to control the configuration of all nodes in that step. This will greatly reduce the time it takes for the RISC-V 64-bit core to configure the RISC-V_FPGA for each step.

[0111] Operations that require control by the RISC-V 64-bit core include: (1) controlling the order of steps; (2) loading and adjusting stored weight factors; (3) reading data from connected memory to the input layer; (4) selecting which node is connected to which node through a non-blocking crossbar switch; (5) selecting the correct weight for the MAC step based on the connections between nodes; (6) timing node operations to transfer data between nodes; (7) writing outputs back to memory; and (8) controlling the updates required at the end of each training.

[0112] To facilitate the manufacture of the system on chip, the RISC-V_FPGA chip can be installed on the interposer, and the interposer, XCPU and memory chip are set on the package substrate. The interposer is a passive semiconductor material (such as silicon) and uses four-layer 65nm technology with through holes and interconnect traces; the interposer can also have chips to implement some logic, such as PCIe interface, 100GigE interface or RISC-V core and 128GB DRAM. Alternatively, these functions can be implemented in hard logic on the chip or even in the configurable part of the RISC-V_FPGA.

[0113] The chip size is expected to be about 40mm x 40mm, and the package size is about 75mm x 75mm, so that the design can leave enough space for the memory chip on the interposer. The number of components required for other chip manufacturing is shown in Table 1;

[0114] Table 1: Component requirements for manufacturing XCPU supercomputing chips

[0115] 2M CLB 3MB Distributed RAM 50MB Block RAM 3,000 FIFO block RAM (48KB each) 10,000 DSP 4 100GigE 2 PCle 16x 1 RISC-V 64-bit core 4 DDR4 interface 1 USB 1 Jtag (loading and debugging) 26 GPIO 1 SSD interface 3Ghz Clock Speed

[0116] The code that can be written in RISC-V_FPGA has the following characteristics: (1) Each position of reading and processing all elements of an array or table row is suitable for implementation as a pipeline structure in RISC-V_FPGA, because speeding up the heavily used processes will bring the greatest benefit. Some of these loop iterations can run RISC-V_FPGA code copies in parallel in one RISC-V_FPGA, but only one copy can be run for other RISC-V_FPGAs;

[0117] (2) Because multiple logic operations occur in one run, running pipeline operations in RISC-V_FPGA logic can greatly speed up the calculation;

[0118] (3) For highly pipelined code logic, data can be processed at a rate close to the maximum I / O rate of the main memory;

[0119] (4) Therefore, for the convolution kernel thread pipeline algorithm and the deconvolution kernel thread pipeline algorithm, most of the processing work can be completed in the pipeline, thus saving time;

[0120] (5) It is also possible to reconfigure the RISC-V_FPGA between multiple steps, although this takes more time. Therefore, this should only be done if speed is not a factor, such as when a new model is needed;

[0121] (6) Assuming that these steps are run sequentially, the logic of these steps can be written directly in RISC-V_FPGA.

[0122] In particular, if Fig. 9 For the for loop part, in addition to running as much as possible in the pipeline, we can also run the code in parallel and configure multiple RISC-V_FPGA copies to perform the same set of operations in the for loop without affecting other parts. This can be seen as a processor farm for the operation steps in the for loop.

[0123] The following will explain the specific acceleration effects of using RISC-V_FPGA for the convolution kernel thread pipeline algorithm and the deconvolution kernel thread pipeline algorithm.

[0124] Convolution kernel thread pipeline algorithm application and pipeline example:

[0125] The core of the convolution kernel thread pipeline algorithm is the neural network. There are three common neural networks: fully connected neural networks, convolutional neural networks (CNNs), and recurrent neural networks (RNNs). Neural networks with more than three layers are usually called deep neural networks (DNNs). Fig.10 It is a schematic diagram of a neural network, which consists of the number of layers, the number of nodes in each layer, the connection method between two adjacent layers, the activation function and the loss function.

[0126] like Fig.11 As shown, for a deep neural network, we need to manage the weight matrix W, the bias matrix b, and the input matrix X.

[0127] Specifically, for layer L, we need to manage the following:

[0128] 1. Weight matrix, where p is the number of attributes of the L-th layer input data, and q is the number of nodes in the L-th layer;

[0129] 2. Input matrix, where n is the number of records and p is the number of attributes of the L-th layer input data;

[0130] 3. Deviation matrix;

[0131] 4. The forward propagation of the Lth layer is Where f is the activation function;

[0132] 5. For the i-th row and j-th column, we have:

[0133] The basic process is: when the error > ε, execute: (1) select n records from the original data; (2) forward propagation; (3) calculate the error; (4) back propagation; (5) update the weight matrix and bias matrix; (6) end.

[0134] The acceleration method of the training process includes the preparation part and the iteration part. The preparation part includes: (1) loading the data types of all attributes; (2) generating the index table S for the first attribute; (3) loading the neural network; (4) allocating memory for the neural network. For each iteration, we perform the following operations: (1) selecting n indexes from the index table S; (2) generating an index matrix according to the data types of the n records; (3) using the pipeline method to perform forward propagation from one layer to the next layer; (4) We build nxq pipeline blocks and calculate in each pipeline block

[0135] Convolutional neural network is a deep learning algorithm. Fig.12 As shown in the figure, it can take in an input image, assign importance (learnable weights and biases) to the various objects in the image, and is able to distinguish between all objects. Compared to other classification algorithms, convolutional neural networks require much less preprocessing and are able to learn all features after enough training.

[0136] A recurrent neural network is an artificial neural network that uses sequence data or time series data. The deep learning algorithm of this neural network is usually used to process ordered data or time data, such as language translation, natural language processing (NLP), etc.

[0137] like Fig.14 As shown, the deep neural network description in RISC-V_FPGA:

[0138] (1) For the fully connected layer, each node has an input buffer (also called a fifo, memory, or pipeline);

[0139] (2)MAC (multiplication-accumulation) can be implemented as a DSP slice in RISC-V_FPGA;

[0140] (3) Output buffer (also called FIFO, memory or pipe);

[0141] (4) a table for storing weights, etc.;

[0142] (5) Through this arrangement, the entire neural network completes a processing in a pipeline form;

[0143] (6) Data will be transmitted over the network, with one output transmitted per major cycle;

[0144] (7) Data between layers does not need to be stored in off-chip memory.

[0145] The method of transmitting a deep neural network in RISC-V_FPGA is as follows: In the above example, since each node has only one MAC, it takes the same number of loops as the number of input connections to the node to complete the accumulation. This means that a simple non-blocking crossbar switch such as Fig.14 We then transfer the data so that we only need 2 connections per node instead of 16 per hidden node. In each small loop, every output and input will be activated, but the non-blocking crossbars of different loops will be set to different connections.

[0146] Taking deep neural networks as an example, in order to make all propagation proceed in an effective way, such as Fig.15 As shown, we need to build a non-blocking crossbar between each layer, which can be reconfigured to create all the connection paths by sorting different connection patterns. In this simple example, there are 64 different connections, and we do 8 at a time. If the number of nodes reaches 100, there are 10,000 different connection patterns that need to be created.

[0147] When a traditional computer processes a 5-layer, 8-node deep neural network, it takes about 25,500 clock cycles to fully propagate the training value to the next layer, that is, 102,000 clock cycles for full propagation through the neural network, not including the time it takes to load the data into the first layer. The RISC-V_FPGA implementation using a non-blocking crossbar switch uses a single-bit switch for each crosspoint, which requires 64*32 clock cycles (for 32-bit values). This provides ample time for the deep pipeline processing that actually takes place in the node. Once the pipeline is full, one layer of input can be processed every 2048 clock cycles, which is about 50 times faster than a traditional computer.

[0148] If the layer has more nodes, the acceleration effect of RISC-V_FPGA will be more obvious. In fact, for a layer of 100 nodes, a traditional computer will need about 16 million clock cycles to complete the processing of each layer through a deep neural network. Using a non-blocking crossbar switch, RISC-V_FPGA can fully process a layer every 32,000 clock cycles, achieving a speedup of about 500 times.

[0149] To achieve this, what we have done is to include crosspoint connections in the RISC-V_FPGA fabric switch blocks, connections that are like those when the RISC-V_FPGA load is configured, but connections that can be dynamically reconfigured by run-time control signals. Fig.15The signals in the RISC-V components are generated, but the control signals can also be generated by the RISC-V_FPGA logic. This makes it possible to change the routing structure required to achieve all the connections required for a non-blocking crossbar switch.

[0150] The XCPU supercomputing chip (5) uses dynamically controlled switch block connections, and new settings of the switch block configuration will be changed by loading the new bitstream into the local switch block part of the j-tag chain or by loading in parallel from adjacent registers.

[0151] This set of dynamically configurable switch blocks and wires can be separated from the wires and connections that are fixed at configuration time, or preferably integrated into the wire connection structure by providing the option of dynamically controlling the crosspoints of all or some of the switch block connections. The switch blocks are controlled by a set of registers, one of which is used to hold the current connection configuration and another register to hold the next configuration. Of course, not all switch blocks need to be dynamically configured.

[0152] The fixed or dynamic mode of each crosspoint in the switch block can be controlled by an adjacent CLB which will also control the lowest level dynamic control information. New settings for the switch block configuration will be changed by loading a new bitstream into the local switch block portion of the j-tag chain or by loading in parallel from adjacent registers. Several of these bitstreams will be stored in the local RISC-V_FPGA fabric memory to facilitate fast changes.

[0153] Another approach is to implement a 32x32 (or some other size) actual non-blocking crossbar switch directly in the RISC-V_FPGA hardware, which can be connected together to form larger non-blocking crossbar switches to handle more complex neural network models.

[0154] The XCPU supercomputing chip 5 uses an actual non-blocking cross switch implemented by RISC-V_FPGA hardware, and the non-blocking cross switches are connected in series.

[0155] The RISC-V 64-bit core controls operations by connecting to each non-blocking crossbar, RISC-V_FPGA node instance, off-chip memory interface, etc. Some control can be achieved by setting the RISC-V 64-bit core set values ​​in the registers of the RISC-V_FPGA for individual nodes or groups of nodes.

[0156] In the above example, for most operations, all nodes are performing the same operation as a non-blocking crossbar, so one register load step is sufficient to control the configuration of all nodes in that step. This will greatly reduce the time that the RISC-V64 bit core takes to configure the RISC-V_FPGA for each step.

[0157] The specific acceleration effect of the convolution kernel thread pipeline algorithm using the RISC-V_FPGA supercomputing chip with a three-dimensional architecture is as follows.

[0158] 1. Convolution operation

[0159] 1. Input

[0160] 1) The input data is an unstructured four-dimensional matrix M b×m×n×c ,in

[0161] b Batch size

[0162] m Image width

[0163] n Image length

[0164] c Image channels, usually 3 (RGB images)

[0165] Data type float32

[0166] 2) The filter is a four-dimensional matrix F p×p×c×l ,in

[0167] p Filter size

[0168] c Image Channel

[0169] The filter depth, which is the number of channels in the output matrix

[0170] Data type float32

[0171] 3) Step size s is an integer

[0172] s The width and length of the output matrix are determined by s

[0173] Data Type Integer

[0174] 4) Deviation vector B 1×l ,in

[0175] The filter depth, which is the number of channels in the output matrix

[0176] Data type float32

[0177] 2. Output

[0178] The output is a four-dimensional matrix O b×x×y×l ,in

[0179] b Batch size

[0180] x Output matrix width, x = INT((m-p+1) / s)

[0181] y Output matrix length, y = INT((n-p+1) / s)

[0182] The filter depth, which is the number of channels in the output matrix

[0183] 3. Resources (The following is a hypothetical example to help understand the data)

[0184] 1) Dimensions

[0185] b=15000

[0186] m=n=500

[0187] c=3

[0188] p=3

[0189] l=16

[0190] s=1

[0191] 2) Input

[0192] Input data: 11.7G

[0193] 3) Output

[0194] Output data: 11.65G

[0195] 4) Total

[0196] Input + Output = 23.35G

[0197] 4. Calculation

[0198]

[0199] 5. Activation Function

[0200]

[0201] Therefore, the CPU needs about (number of interposer layers * 12500) clock cycles to read and execute one pass through all 7 nested for loops of the first part of the convolution operation.

[0202] 2. Sampling Operation

[0203] 1. Input

[0204] 1) The input data is an unstructured four-dimensional matrix M b×m×n×c ,in

[0205] b Batch size

[0206] m Image width

[0207] n Image length

[0208] c Image channels, usually 3 (RGB images)

[0209] Data type float32

[0210] 2) Step size s is an integer

[0211] s The width and length of the output matrix are determined by s

[0212] Data Type Integer

[0213] 2. Output

[0214] The output is a four-dimensional matrix O b×x×y×l ,in

[0215] b Batch size

[0216] x Output matrix width, x = INT((m-p+1) / s)

[0217] y Output matrix length, y = INT((n-p+1) / s)

[0218] c Image Channel

[0219] 3. Resources (The following is a hypothetical example to help understand the data)

[0220] 1) Dimensions

[0221] b=15000

[0222] m=n=500

[0223] c=3

[0224] s=2

[0225] 2) Input

[0226] Input data: 11.7G

[0227] 3) Output

[0228] Output data: 6G

[0229] 4) Total

[0230] Input + Output = 17.7G

[0231] 4. Calculation

[0232]

[0233] Therefore, the CPU takes approximately (number of interposer layers * 8700) clock cycles to read and execute one pass through all five nested for loops in the second part of the sampling operation.

[0234] 3. Fully Connected Neural Network Operation

[0235] 1. Input

[0236] 1) The input data is a structured two-dimensional matrix M b×m ,in

[0237] b Batch size

[0238] m is the number of input data features

[0239] Data type float32

[0240] 2) The weight matrix is ​​a two-dimensional matrix N m×p ,

[0241] b Input data feature number

[0242] p Output data feature number

[0243] Data type float32

[0244] 3) Deviation vector B 1×p ,in

[0245] p Output data feature number

[0246] Data type float32

[0247] 2. Output

[0248] 1) The output data is a two-dimensional matrix O b×p ,in

[0249] b Batch size

[0250] p Output data feature number

[0251] 3. Resources

[0252] 1) Dimensions

[0253] b=100000

[0254] m=2048

[0255] p=500

[0256] 2) Input

[0257] Input data: 6.2G

[0258] 3) Output

[0259] Output data: 6.15G

[0260] 4) Total

[0261] Input data + output data = 12.35G

[0262] 4. Calculation

[0263]

[0264] Therefore, the CPU takes approximately (number of interposer layers * 3250) clock cycles to read and execute one pass through all 3 nested for loops of the third part of the fully connected neural network operation.

[0265] 4. Recurrent Neural Network Operation

[0266] 0. Predefined functions

[0267]

[0268]

[0269]

[0270]

[0271] 1. Input

[0272] 1) The input data is matrix D b×t×n

[0273] 2) The weight matrix is ​​defined as follows

[0274] matrix W f1 ; / / Forget weight matrix 1

[0275] matrix W f2 ; / / Forget weight matrix 2

[0276] matrix b f ; / / Forget bias vector

[0277] matrixW i1 ; / / Input weight matrix 1

[0278] matrixW i2 ; / / Input weight matrix 2

[0279] matrix b i ; / / Input bias vector

[0280] matrix W c1 ; / / State weight matrix 1

[0281] matrix W c2 ; / / State weight matrix 2

[0282] matrix b c; / / State deviation vector

[0283] matrix W o1 ; / / Output weight matrix 1

[0284] matrix W o2 ; / / Output weight matrix 2

[0285] matrix b o ; / / Output deviation vector

[0286] where all weight matrices 1 are of size m×m, all weight matrices 2 are of size m×m, and all bias vectors are of size m.

[0287] 3) Initial state

[0288] matrixh -1 ; / / Matrix size is b×m, all variables are 0

[0289] matrixC -1 ; / / Matrix size is b×m, all variables are 0

[0290] 2. Output

[0291] 1) The output data is three matrix series

[0292] Series 1: h 0 ,h 1 , ..., h t-1

[0293] Series 2: C 0 , C 1 , ..., C t-1

[0294] Series 3: O 0 , O 1 , ..., O t-1

[0295] 3. Calculation

[0296]

[0297] Therefore, the CPU takes approximately (number of interposer layers * 29000) clock cycles to read and execute one pass through all 12 nested for loops of the fourth part of the recurrent neural network operation.

[0298] Example of acceleration effect:

[0299] The following uses the inner for loop of the sampling operation in the second part as an example of pipeline process to illustrate the acceleration effect.

[0300]

[0301] In the pipeline flow, there are many operation blocks. The rising edge of the clock signal triggers the data element to move to the next stage (action block) of the pipeline, usually an extraction, holding, calculation or storage step.

[0302] Taking this inner for loop as an example, the code executed in each iteration of the for loop can be decomposed into a series of small steps, where a, b, c, d, e are values ​​that can be calculated once by the for loop, and a, b, c, d, e are registers that store the data unit values ​​of the current time step. a = j*s (initial 3), b = k*s (initial 5), c' = O[i][j][k][q] (initial 12), d = a+u, e = b+u, E = M[i][d][e][q].

[0303] In this case, the input is the elements of M[i][d][e][q], and the array is Fig.16 The steps in Figure 1 show the progression of elements u = 0 to 9 through the pipeline. The values ​​of the M array are 10, 20, 15, 25, 20, 33, 20, 21. For values ​​u = 0 to 8, the output is written to the O[i][j][k][q] array.

[0304] The timetable of each process in the pipeline is as follows Fig.17 , the boxed part shows the progress of data through the pipeline steps when u = 2 and u = 5. In reality, each functional block has several small steps to perform the calculation, but this does not reduce the throughput, it just makes the pipeline have more steps.

[0305] The use of a non-blocking crossbar can significantly speed up many steps of a neural network because it can do a large number of propagations in parallel and perform more operations on the RISC-V_FPGA chip without the need to introduce off-chip memory.

[0306] When the pipeline is filled, the RISC-V_FPGA approach can execute one data element per clock cycle per pipeline, so only the memory access time for the data is limited.

[0307] Most computations are done in large numbers of very small pipeline steps, and in many cases if the operation is logically correct it can be run in multiple pipelines running in parallel, so multiple data elements can be processed, with the only thing limiting you is memory bandwidth and access time.

[0308] In fact, most memory accesses can be avoided because the pipeline arrangement can usually move most of the data in RISC-V_FPGA directly from one step to the next.

[0309] The number of clock cycles to fill the pipeline is the number of steps in the pipeline, which is typically a small fraction of the number of data elements to be processed.

[0310] For the two distribution models described above, there are parts of the for loop that are executed multiple times and can be distributed to multiple blocks of the RISC-V_FPGA for parallel operation. For example:

[0311]

[0312]

[0313] As Fig.18 shown, there are many operation blocks in the pipeline process. The rising edge of the clock signal triggers the data element to move to the next stage of the pipeline (action block), which is usually an extraction, hold, calculation, or storage step.

[0314] Taking the inner for loop in the above code for(int m = 0; m < nfit; m++){pvec0[m][k]+= hv * xwoe0[m];} as an example, the code executed in each iteration can be decomposed into a series of small steps, where A, B, C, D, and E are registers storing the values of data units at the current time step. A = xwoe0[m], B = pvec0[m][k], C = hv * B, D = B, E = D + B; pvec0[m][k]= E.

[0315] In this example, the inputs are the xwoe0[m] and pvec0[m][k] arrays. The following steps show the process of elements m = 0 to 9 in the pipeline. Among them, as Fig.19 shown, the values of the pvec0 array are 10, 20, 30, 40, 50, 60, 70, 80, 90…, the values of the xwoe0 array are 1, 2, 3, 4, 5, 6, 7, 8, 9…, and the value of hv is 2. When it completes the process of element m = 3, it is noted that the process of element m = 6 has already started, and the data of elements m = 4 and m = 5 have been partially processed. The delay step only saves the values of the pvec0() data elements to synchronize with the correct elements in the xwoe0() array.

[0316] As Fig. 20 shown, the part in the box shows the process of data being processed through the pipeline at m = 3. In fact, each functional block has several small steps for calculation, but this does not reduce the throughput, only adding more steps to the pipeline.

[0317] The number of clock cycles for a normal computer to read and execute a for loop is about (number of interposer layers * 3450). In RISC-V_FPGA, when the pipeline is filled, one data field can be processed per clock cycle, so the limitation only exists in the memory access time of the data.

[0318] like Fig.21 As shown in the figure, the computational advantage of the XCPU supercomputing chip algorithm, n is the number of times the sub-loop is called during the CPU accelerated calculation process.

[0319] In addition, the systolic array and the gate array are the design and implementation methods of two different parallel computing architectures in the claims.

[0320] A systolic array is a special form of parallel computing with a deeply pipelined network of processing elements (PEs) that communicate locally and have a regular layout. It has the characteristics of low global data transfer and high clock frequency, which is suitable for large-scale parallel design on FPGA. Systolic arrays can be used for various matrix calculations such as matrix multiplication, convolution, correlation, etc. To implement a systolic array in FPGA, it is necessary to design PEs, interconnection network, and data flow. PEs can be hardwired or software-configured for specific operations such as multiplication and accumulation. The interconnection network can be programmable or fixed, depending on the application and FPGA resources. The data flow can be synchronous or asynchronous, depending on the timing and synchronization requirements.

[0321] A systolic array is a homogeneous network of tightly coupled data processing units, each of which is called a cell or node. Each node independently computes a partial result based on data received from its upstream neighbors, stores the result internally and passes it downstream. Systolic arrays are typically hardwired for specific operations, such as matrix multiplication, convolution, or sequence alignment.

[0322] The XCPU supercomputing chip 5 is a parallel computing architecture that uses a systolic array or a gate array.

[0323] A gate array is a method of designing and manufacturing application-specific integrated circuits (ASICs) using prefabricated chips whose components are connected into logic devices (e.g., NAND gates, flip-flops, etc.) by adding metal interconnect layers in a custom sequence at the factory. Gate arrays are often used to implement custom logic functions or digital circuits.

[0324] Some of the main differences between systolic arrays and gate arrays are:

[0325] (1) Systolic arrays are based on data flow, while gate arrays are based on logic functions;

[0326] (2) Systolic arrays rely on synchronous data transmission, while gate arrays can operate asynchronously;

[0327] (3) Systolic arrays are usually fixed and identical, while gate arrays can be programmable and heterogeneous;

[0328] (4) Systolic arrays have lower latency and higher throughput, while gate arrays have higher flexibility and lower cost.

[0329] A special type of gate array is the Field Programmable Gate Array (FPGA), which allows the user to program the logic functions and interconnections after the chip is manufactured. FPGAs can also implement a systolic array architecture for parallel computing. FPGAs offer advantages such as reconfigurability, scalability, and rapid prototyping.

[0330] Embodiment 2:

[0331] The difference from Example 1 is that: Figure 4 As shown, the vertical cross-section of the substrate 4 is in the shape of a "gate".

[0332] like Figure 4 As shown, a multi-level interposer is used to support a 3D structure of up to 10 layers (10 modules). The bottom interposer is different from other interposers and is directly connected to the package substrate 2.

[0333] Embodiment 3:

[0334] The difference from Example 1 is that: Figures 5-6 As shown, there are partition columns 8 on the four lower corners of the substrate 4, and a plurality of intermediate layer connection traces 9 are arranged around the periphery of the XCPU supercomputer chip 5. There are gaps between the partition columns 8 and the intermediate layer connection traces 9. In the figure, M represents 32 Dram memory chips 6.

[0335] Among them, a silicon interposer is used to connect the imprint 9 and the spacer 8, supporting a 3D structure with up to 10 layers. This type has 4 spacer columns 8 that are longer than the interposer connection. When the interposer stack is reflowed, the solder balls will not be too flat.

[0336] Embodiment 4:

[0337] The difference from Example 1 is that: Figures 7-8 As shown, a through silicon via block 3 is provided above the substrate 4, and a plurality of stacked XCPU supercomputer chips 5 are installed in the middle section of the through silicon via block 3, and a plurality of memory chips 6 are provided on both sides of the XCPU supercomputer chip 5.

[0338] Among them, the XCPU chip with 10 layers of connected silicon through-vias supports a 3D structure of up to 10 layers of XCPU chips. This type is suitable for situations where a large number of XCPU routing is required.

[0339] like Figure 8As shown, using the form of stacked intermediate layers, access to all memories in the XCPU stack module can be achieved.

[0340] Based on the above ideal embodiments of the present invention, the relevant staff can make various changes and modifications without departing from the technical concept of the present invention through the above description. The technical scope of the present invention is not limited to the contents of the specification, and its technical scope must be determined according to the scope of the claims.

Claims

1. A three-dimensional XCPU supercomputer chip, characterized by: The invention comprises a multi-layer printed circuit board (1) and a packaging substrate (2), wherein the multi-layer printed circuit board (1) is located below the packaging substrate (2), a through silicon via block (3) is provided between the multi-layer printed circuit board (1) and the packaging substrate (2), and a plurality of intermediate layers are provided above the packaging substrate (2) through the through silicon via block (3); The intermediary layer comprises a substrate (4), the substrate (4) is in the shape of an elongated strip, a through silicon via block (3) is arranged above the substrate (4), an XCPU supercomputer chip (5) is installed in the middle section of the through silicon via block (3), a plurality of memory chips (6) are arranged on both sides of the XCPU supercomputer chip (5), support blocks (7) are arranged at both ends of the through silicon via block (3), and a through silicon via block (3) is arranged above the support block (7) to contact with the upper intermediary layer.

2. The XCPU supercomputer chip with a three-dimensional architecture according to claim 1, characterized in that: The cross-sectional shape of the substrate (4) in the vertical direction is in the shape of a "gate".

3. The XCPU supercomputer chip with a three-dimensional architecture according to claim 1, characterized in that: The four lower corners of the substrate (4) are each provided with a partition column (8), and a plurality of intermediate layer connection marks (9) are provided around the periphery of the XCPU supercomputer chip (5), with a gap between the partition column (8) and the intermediate layer connection marks (9).

4. The XCPU supercomputer chip with a three-dimensional architecture according to claim 1, characterized in that: A through silicon via block (3) is arranged above the substrate (4), a plurality of stacked layers of XCPU supercomputing chips (5) are installed in the middle section of the through silicon via block (3), and a plurality of memory chips (6) are arranged on both sides of the XCPU supercomputing chip (5).

5. The XCPU supercomputer chip with a three-dimensional architecture according to claim 1, characterized in that: The XCPU supercomputing chip (5) includes a programmable area of ​​FPGA, multiple I / O devices surrounding the area, and an integrated RISC-V processor, which is used to supervise the loading, selection and operation of RISC-V_FPGA configuration and other tasks.

6. The XCPU supercomputer chip with a three-dimensional architecture according to claim 1, characterized in that: The XCPU supercomputing chip (5) has a practical dynamically controlled switch block connection, and the new setting of the switch block configuration will be changed by loading the new bit stream into the local switch block part of the j-tag chain or by loading in parallel from adjacent registers.

7. The three-dimensional XCPU supercomputer chip according to claim 1, characterized in that: The XCPU supercomputing chip (5) uses dynamically controlled configurable switch blocks and wires separated from fixed configuration time wires, and the switch block is controlled by a group of registers, one of which is used to maintain the current connection configuration, and the other register is used to save the next configuration.

8. The XCPU supercomputer chip with a three-dimensional architecture according to claim 1, characterized in that: The XCPU supercomputing chip (5) uses an actual non-blocking cross switch implemented by RISC-V_FPGA hardware, and the non-blocking cross switches are connected in series.

9. The XCPU supercomputer chip with a three-dimensional architecture according to claim 1, characterized in that: The XCPU supercomputing chip (5) supports multiple independent concurrent operation memory access channels.

10. The three-dimensional XCPU supercomputer chip according to claim 1, characterized in that: The XCPU supercomputing chip (5) supports multiple independent concurrent operation PCI channels to achieve fast I / O.

11. The XCPU supercomputer chip with a three-dimensional architecture according to claim 1, characterized in that: The XCPU supercomputing chip (5) adopts 100GigE that supports multiple independent concurrent operations to achieve fast I / O.

12. The three-dimensional XCPU supercomputer chip according to claim 1, characterized in that: The convolution kernel thread pipeline algorithm used by the XCPU supercomputing chip (5) is a neural network algorithm.

13. The XCPU supercomputer chip with a three-dimensional architecture according to claim 1, characterized in that: The neural network convolution, neural network sampling operation, fully connected neural network or recurrent neural network are implemented in the RISC-V_FPGA hardware of the XCPU supercomputing chip (5).

14. The XCPU supercomputer chip with a three-dimensional architecture according to claim 1, characterized in that: The deconvolution kernel thread pipeline algorithm used by the XCPU supercomputer chip (5) is divided into a Fermi-Dirac probability distribution function model and a Bose-Einstein probability distribution function model.

15. The XCPU supercomputer chip with a three-dimensional architecture according to claim 1, characterized in that: The deconvolution kernel thread pipeline algorithm is implemented in the RISC-V_FPGA hardware of the XCPU supercomputing chip (5).

16. The three-dimensional XCPU supercomputer chip according to claim 1, characterized in that: The XCPU supercomputing chip (5) is a parallel computing architecture, which adopts a systolic array or a gate array.

Images