Reconfigurable integrated circuit suitable for artificial intelligence large language model reasoning

Abstract

The application discloses a reconfigurable integrated circuit suitable for artificial intelligence large language model (LLM) reasoning, which comprises a special integrated circuit for realizing a transformer function and an on-chip configuration storage unit, wherein the special integrated circuit for realizing the transformer function realizes a specific configuration according to the on-chip configuration storage before operation.

Description

Technical Field

[0001] This invention relates to the field of integrated circuit technology, and in particular to a reconfigurable integrated circuit suitable for reasoning using large language models in artificial intelligence. Background Technology

[0002] Traditional FPGA chips comprise three generations: First-generation FPGA chips implemented a basic programmable logic array (PLA), including on-chip memory (Configuration Memory). Logic functions were implemented within the PLB (Configurable Logic Block), and routing was handled in a SwitchBox. Second-generation FPGA chips integrated the PLA and SoC, including a CPU and memory system. Third-generation FPGA chips integrated the PLA, SoC, GPU, and NPU to create an AI heterogeneous computing acceleration platform. This platform was designed to accelerate traditional AI neural networks (such as CNNs and RNNs). Following its release, large language models based on Transformers experienced rapid development. ChatGPT achieved phenomenal human-computer dialogue. DeepSeek significantly reduced the hardware cost of implementing large language models. Numerous inference chips developed domestically and internationally for large language models aim to achieve energy efficiency far exceeding that of GPUs. These inference chips utilize a SoC architecture, adding Transformer-specific instructions to the instruction set to accelerate large language model inference. Summary of the Invention

[0003] This invention provides a large language model inference chip based on FPGA architecture, which realizes reconfigurable large language model inference based on on-chip storage.

[0004] The differences between FPGA-based and SoC-based large language model inference chips are as follows: SoC-based chips convert software instructions into dynamic control signals within the chip through compilation, enabling a degree of reconfigurable configuration, such as configuring the precision of matrix multiplication. In contrast, FPGA-based chips utilize on-chip memory for reconfigurable configuration signals, possessing more configuration bits to achieve more complex configurations, such as interconnections between different functional modules. FPGA-based chips have largely static reconfigurable configuration signals; the chip starts operating after configuration, eliminating the need for compilation. Compilation is not only complex but also incurs runtime costs if dynamic control signals fail to arrive in time. FPGA-based chips can also have dynamic reconfigurable configuration signals, such as through RuntimeReconfiguration, requiring more configuration bits for more complex reconfiguration. Attached Figure Description

[0005] Figure 1 This section presents a reconfigurable circuit architecture diagram suitable for reasoning with large language models.

[0006] Figure 2 This demonstrates the power-on configuration process for a reconfigurable circuit suitable for reasoning with large language models.

[0007] Figure 3 This demonstrates an SRAM-based on-chip configuration memory.

[0008] Figure 4 This demonstrates an on-chip configuration memory based on edge-triggered triggers.

[0009] Figure 5 Demonstrate a reconfigurable adder circuit based on on-chip configuration memory.

[0010] Figure 6 Demonstrates the mechanisms of single-head attention and multi-head attention.

[0011] Figure 7 Demonstrate a multiplication circuit that performs both single-precision and double-precision operations simultaneously.

[0012] Figure 8 Demonstrate a multiplication circuit that can be reconfigured to single-precision or double-precision based on on-chip configuration memory.

[0013] Figure 9 This paper demonstrates a reconfigurable attention circuit based on on-chip configuration memory.

[0014] Figure 10 Demonstrates a reconfigurable hybrid expert circuit based on on-chip configuration memory.

[0015] Specific implementation measures

[0016] The following detailed description, in conjunction with the accompanying drawings and specific embodiments, provides a reconfigurable circuit suitable for large language model reasoning proposed in this invention. The advantages and features of this invention will become clearer from the following description. It should be noted that the accompanying drawings are all in a very simplified form and use non-precise proportions, and are only used to facilitate and clarify the illustration of the embodiments of this invention.

[0017] The following detailed description, in conjunction with the accompanying drawings and specific embodiments, provides a reconfigurable circuit suitable for large language model reasoning proposed in this invention. The advantages and features of this invention will become clearer from the following description. It should be noted that the accompanying drawings are all in a very simplified form and use non-precise proportions, used only to facilitate and clarify the illustration of the embodiments of this invention. Figure 1 shows the architecture diagram of the reconfigurable circuit suitable for large language model reasoning. As shown in Figure 1, the reconfigurable circuit suitable for large language model reasoning includes on-chip configuration storage, configuration control circuitry, and an integrated circuit for implementing large language model reasoning. Its inputs include signals from the on-chip storage, and different configurations are implemented according to the on-chip storage.

[0018] Figure 2 This demonstrates the power-on configuration process of a reconfigurable circuit suitable for reasoning with large language models. For example... Figure 2 As shown, after the chip powers on, the configuration control circuit uploads configuration signals from the chip interface (SPI, BPI, or JTAG, etc.) to the on-chip configuration memory. After configuration is complete, the configuration control circuit switches the chip mode from configuration mode (OPMODE=0) to operating mode (OPMODE=1). In operating mode, based on the configuration signals in the on-chip memory, the chip performs large language model inference with a specific configuration.

[0019] Figure 3 This demonstrates an on-chip configuration memory based on SRAM, where each memory cell includes a loop consisting of two inverters and two NMOS transmission gates. In configuration mode, the configuration control circuitry communicates via bit lines. WL i Addressing is performed by writing configuration signals to each configuration memory cell via the word line BL. In operating mode, each memory cell is enabled by the operating mode signal OPMODE through an AND logic gate, and outputs... R i Configure signals.

[0020] Figure 4 This demonstrates an on-chip configuration memory based on edge-triggered flip-flops, where the flip-flops form a chain. In configuration mode, configuration control circuitry writes configuration signals into the flip-flops. In operating mode, each flip-flop is enabled by the operating mode signal OPMODE through an AND logic gate, outputting... R i Configure signals.

[0021] Figure 5 This demonstrates a reconfigurable adder circuit based on on-chip configuration memory, which performs operations of varying precision depending on the contents of the on-chip configuration memory. The first... i Adder FA i Carry output Cout i Through a logic gate and the firsti+1 Adder FA i+1 Carry input Cin i Connected to the bypass input of the logic gate. R i It comes from on-chip memory, which can be an edge-triggered flip-flop (FF), a level-sensitive latch, an SRAM cell, a DRAM cell, or a flash cell. When R i When =1, the AND gate is conducting, the first... i Adder FA i and the i+1 Adder FA i+1 On a carry chain. When R i When =0, the AND gate is not conducting, and the carry chain is in the th order. i Adder FA i and the i+1 Adder FA i+1 The operations are disconnected between these parameters. This allows for addition or multiplication operations of different precisions, such as INT4, INT8, FP4, FP8, etc.

[0022] Figure 6 This paper demonstrates a reconfigurable circuit that implements a multi-head attention mechanism based on on-chip configuration memory. The multi-head attention mechanism uses... h indivual d v A bit-wide attention function replaces one d model = HD v Attention functions of varying bit widths. Each attention function includes matrix multiplication, optional scaling and masking, SoftMax, and a second matrix multiplication. Reconfigurable circuitry implements matrix multiplication, SoftMax, and interconnections between different modules with varying bit widths.

[0023] Figure 7 This demonstrates a multiplication circuit that performs both single-precision and double-precision operations simultaneously. The circuit implements a double-precision multiplication operation using four single-precision multiplication operations, as shown in the following formula: A × B = (A... upper 2 N +A lower )×(B upper 2 N +B lower )=A upper ×B upper 2 2N+(A upper ×B lower +A lower ×B upper )2 N +A lower ×B lower Where A and B are 2 N -bit INT integers or FP floating-point numbers, including N-bit high-order bits A upper , B upper and N-bit low bits A lower , B lower This circuit simultaneously outputs the results of double-precision multiplication operations. P=(P upper ,P lower ) and the result of four single-precision multiplication operations P 0 , P 1 , P 2 , P 3 .

[0024] Figure 8 Demonstrates a multiplication circuit that can be reconfigured to single-precision or double-precision based on on-chip configuration memory, where configuration signals from the on-chip configuration memory... R d The input of the adder is enabled by a logic gate. R d When the value is 0, the circuit only outputs the results of four single-precision multiplication operations. P 0 , P 1 , P 2 , P 3 .exist R d When =1, the circuit outputs the result of a double-precision multiplication operation. P=(P upper , P lower ) .

[0025] Figure 9 This demonstrates a reconfigurable attention circuit based on on-chip configuration memory, which includes two multiplication circuits. These two multiplication circuits can be... Figure 7 or Figure 8The circuit shown is implemented as follows. Its output includes single-precision and double-precision multiplication results. Other modules (Scale & Mask, SoftMax) also accept single-precision and double-precision inputs and provide single-precision and double-precision outputs. The input KQV of this attention circuit can be single-precision or double-precision, determined by a configuration signal from the on-chip configuration storage. R K , R Q , R V Single-precision or double-precision inputs are selected via a multiplexer. The single-precision and double-precision outputs of this attention-multiplexed input are determined by configuration signals from on-chip configuration storage. R h The output is selected using a multiplexer.

[0026] Figure 10 Demonstrates a reconfigurable hybrid expert circuit based on on-chip configuration memory, where each expert... Expert i The output is generated by configuration signals from the on-chip configuration storage. R i Enabled via an AND logic gate.

[0027] The above description is merely a description of preferred embodiments of the present invention and is not intended to limit the scope of the present invention in any way. Any changes or modifications made by those skilled in the art based on the above disclosure shall fall within the protection scope of the claims.

Claims

1. A reconfigurable integrated circuit suitable for reasoning in large language model (LLM) artificial intelligence, characterized in that, The integrated circuit includes an application-specific integrated circuit (ASIC) that implements the Transformer function and an on-chip configuration memory unit. The ASIC that implements the Transformer function implements a specific configuration based on the on-chip configuration memory before operation.

2. The integrated circuit of claim 1, wherein attention functions of different precisions are implemented according to on-chip configuration memory.

3. The integrated circuit as described in claim 2, wherein addition of different bit widths is implemented according to on-chip configured memory.

4. The integrated circuit of claim 2, wherein multiplication of different precisions is implemented according to on-chip configured storage.

5. The integrated circuit as described in claim 4, wherein a double-precision multiplication is implemented by four single-precision multiplications, and the output of the single-precision or double-precision multiplication result is selected according to the on-chip configuration memory.

6. The integrated circuit of claim 1, wherein a multi-head attention mechanism comprising a different number of heads is implemented based on on-chip configuration memory.

7. The integrated circuit of claim 1, wherein a multi-head latent attention mechanism comprising a different number of heads is implemented based on on-chip configuration memory.

8. The integrated circuit of claim 1, wherein a mixed expert model with a different number of experts is implemented according to on-chip configuration memory.

9. The integrated circuit of claim 1, wherein the on-chip configuration memory outputs a configuration signal by an operating mode enable signal.

10. The integrated circuit of claim 1, wherein the on-chip configuration memory is implemented by SRAM.

11. The integrated circuit of claim 1, wherein the on-chip configuration memory is implemented by an edge-triggered flip-flop.

Images