A general-purpose Transformer acceleration method and system based on floating-point instruction set extension

Abstract

This invention proposes a general-purpose Transformer acceleration method and system based on floating-point instruction set extensions, comprising: constructing a multiplier, which includes a series of sequentially connected adders; encoding data using a 4-Booth encoding algorithm; performing operations on the encoded data through the series of adders to obtain the output of the multiplier; constructing a floating-point register, specifically by adding a floating-point register group; the write port logic of the floating-point register group corresponds the input result register index with its respective register number to generate a write enable signal; the enabled general-purpose register writes data into the register; the read port is implemented using a multiplexer, and the selection signal of the multiplexer is the index of the read operand register; the RISC-V processor accelerates the operation of the Transformer neural network based on the constructed multiplier and floating-point registers.

Description

Technical Field

[0001] This invention belongs to the field of computer Transformer neural network acceleration technology, and particularly relates to a general Transformer acceleration method and system based on floating-point instruction set extension. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] The open-source processor architecture RISC-V has seen rapid development in recent years. RISC-V, the fifth generation of the RISC instruction set, features a compact and efficient architecture while employing a modular design philosophy, allowing users to flexibly combine different modules to meet various application scenarios. The Hummingbird E203 is a completely domestically developed processor core based on the RISC-V instruction set architecture. It adopts a modular design, offering excellent scalability and flexibility, making it suitable for various embedded system applications. The Hummingbird E203 core features high performance and low power consumption, meeting the performance requirements of different scenarios.

[0004] The Transformer is a neural network architecture based on an attention mechanism, consisting of an encoder and a decoder. In natural language processing, Transformer models are widely used in tasks such as text classification, machine translation, and sentiment analysis. It has also demonstrated powerful application capabilities in areas such as speech recognition, computer vision, and reinforcement learning. Compared to traditional neural network structures such as CNNs and RNNs, the Transformer, as a novel network architecture, has a strong advantage in terms of late-mover advantage, and can replace traditional convolutional neural networks and recurrent neural networks in various tasks.

[0005] Currently, RISC-V (taking E203 as an example) suffers from problems such as low computing power and is mostly used in low-power and embedded environments. At the same time, since E203 does not contain a floating-point instruction set, but instead uses integer instructions to simulate floating-point instructions, the computing speed of RISC-V is greatly reduced, making it unable to effectively perform forward inference on neural networks. At present, there is still a great lack of work on combining Transformer neural network inference with RISC-V. Summary of the Invention

[0006] To overcome the shortcomings of the prior art, this invention provides a general Transformer acceleration method based on floating-point instruction set extension, which improves the computing power of RISC-V cores by adding floating-point instruction sets and attempting to run neural networks based on the optimized BERT architecture on them.

[0007] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions:

[0008] Firstly, a general Transformer acceleration method based on floating-point instruction set extensions is disclosed, including:

[0009] A multiplier is constructed, which consists of multiple stages of adders connected in sequence. The 4-Booth encoding algorithm is used to encode the data. The encoded data is then processed by the multiple stages of adders to obtain the output of the multiplier.

[0010] The specific steps for constructing a floating-point register are as follows: a floating-point register group is generated, the write port logic of the floating-point register group corresponds the input result register index with its respective register number, generates a write enable signal, the enabled general-purpose register writes data into the register, and the read port is implemented using a multiplexer, the selection signal of the multiplexer is the index of the read operand register;

[0011] The RISC-V processor runs Transformer neural networks based on a built-in multiplier and floating-point register.

[0012] As a further technical solution, the multi-stage adder of the multiplier adopts an addition tree structure, so that the number of bits of the adder used to add two adjacent numbers is minimized.

[0013] As a further technical solution, the steps for constructing a floating-point register include:

[0014] Extended register set: Add a floating-point register;

[0015] Modify the decoding process: Add a decoding process for floating-point instructions;

[0016] Modify the dispatch module;

[0017] Modify the calculation and processing module;

[0018] Add a floating-point CSR register;

[0019] The decoder identifies the floating-point instruction and passes its characteristics to the dispatch module. The dispatch module dispatches the floating-point instruction to the floating-point calculation and processing module according to the instruction type. The calculation and processing module performs the floating-point operation and updates the result and status information to the floating-point CSR register.

[0020] As a further technical solution, decoding and recognizing floating-point instructions involves the following steps:

[0021] Generate type information and operand register indexes for different instructions;

[0022] Determine the target of each instruction, compare the corresponding part of the instruction with the specific value, and if they are equal, pull the corresponding signal high; otherwise, keep it low.

[0023] After obtaining the corresponding judgment signal value, a specific instruction is selected;

[0024] After receiving the specific instructions, it is necessary to determine the specific operations that the instructions will perform.

[0025] Different floating-point instructions will point to different register sets;

[0026] Not all instructions in the floating-point instruction set use the floating-point register set. Some instructions move data between different register sets or access memory, read data addresses from the integer general-purpose register set, and finally write the data back to the floating-point register set.

[0027] Different enable signals are used to indicate whether these structures and specific pointers exist in floating-point instructions.

[0028] As a further technical solution, it also includes: using different enable signals to indicate whether these structures and the specific pointers of rs1, rs2, rs3, and rd exist in the floating-point instructions.

[0029] As a further technical solution, it also includes: assigning the decoded index value to the index of the corresponding register of the FPU;

[0030] After determining whether a floating-point instruction has rs1, rs2, rs3, rd and their corresponding index values, the register group type corresponding to that floating-point instruction is determined.

[0031] As a further technical solution, the specific steps for determining the register set type corresponding to the floating-point instruction are as follows:

[0032] Except for flw, fsw, and fmv.wx, the rs1 parameter of floating-point instructions all point to the floating-point register group;

[0033] In floating-point instructions with the rs2 parameter, rs2 always points to the floating-point register set;

[0034] In floating-point instructions with the rs3 parameter, rs3 always points to the floating-point register set;

[0035] Except for fmv.xw and flt, the rd parameter of floating-point instructions with the rd parameter points to the floating-point register set.

[0036] As a further technical solution, after decoding is completed, the decoding result of the instruction and the operands read from the register set will be dispatched to different arithmetic units for execution;

[0037] All instructions are transferred to the ALU for execution, and the decoding results of floating-point instructions are added to the input / output interfaces.

[0038] As a further technical solution, the decoding results of floating-point instructions are added to the input / output interface, specifically:

[0039] First, the parameter rs3 is used for the first time in floating-point instructions, and index signals, judgment signals and specific values ​​of rs3 are added to input and output;

[0040] In addition, a discrimination signal needs to be added to the input and output to determine the type of register group to be pointed to. If it is a floating-point register group, the actual dispatch function is arranged to be executed in the ALU. The group information generated by decoding is used to determine the type of register group to be pointed to and dispatched to the corresponding unit in the ALU to execute the instruction.

[0041] The addition of grouping information from the fonecycle and fdivsqrt modules ensures that floating-point instructions can be sent to the correct cell for execution.

[0042] The remaining two non-operational long instructions, flw and fsw, are added to the same execution process as sw and lw, and are also added to the AGU's information bus for dispatch to the AGU for processing, and then sent to the LSU for execution.

[0043] As a further technical solution, in the fonecycle module, the processing of all floating-point instructions is completed within one cycle, including floating-point multiplication;

[0044] The fdivsqrt module contains only two instructions: fdiv and fsqrt. These are multi-cycle instructions, requiring 24 clock cycles to execute each time.

[0045] Secondly, a general-purpose Transformer acceleration system based on floating-point instruction set extensions is disclosed, including:

[0046] The multiplier construction module is configured to: construct a multiplier, which includes a series of adders connected in sequence, and encode the data using a 4-Booth encoding algorithm; the encoded data is then processed by the series of adders to obtain the output of the multiplier;

[0047] The floating-point register construction module is configured to construct floating-point registers. Specifically, it generates a floating-point register set. The write port logic of the floating-point register set maps the input result register index to its corresponding register number, generating a write enable signal. The enabled general-purpose register writes data into the register. The read port uses a multiplexer, with the selection signal of the multiplexer being the index of the operand register to be read.

[0048] The acceleration module is configured to accelerate the operation of the Transformer neural network based on the constructed multiplier and floating-point register.

[0049] The above one or more technical solutions have the following beneficial effects:

[0050] This invention improves the Coremark score of the RISC-V kernel by optimizing the multiplier. Simultaneously, it extends the floating-point instruction set operators of the original kernel and implements the relevant underlying software libraries for the floating-point instruction set, enabling the kernel to directly support the use of floating-point instructions.

[0051] Existing processors have room for optimization in their multiplier components; this solution optimizes the serial multiplier into a parallel multiplier. The original E203 core lacks a floating-point operation section, while neural networks involve a large number of floating-point operations. This solution designs a floating-point instruction set to improve the operating speed of neural networks.

[0052] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0053] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0054] Figure 1 Schematic diagram of a five-stage production line;

[0055] Figure 2 A block diagram of the RISC-V architecture;

[0056] Figure 3 This is a schematic diagram of the BERT network, a derivative of Transformer.

[0057] Figure 4 A schematic diagram of the multi-head attention mechanism of Transformer;

[0058] Figure 5 A schematic diagram of parallel encoding of 32-bit multiplier rs1;

[0059] Figure 6 This is a diagram illustrating the Booth 4 encoding process.

[0060] Figure 7 This is a schematic diagram of a multiplier structure;

[0061] Figure 8 A schematic diagram illustrating the specific implementation process of the arithmetic operator;

[0062] Figure 9 Schematic diagram of floating-point registers and general-purpose integer registers;

[0063] Figure 10 Comparison chart of predicted and standard values ​​for refrigerator power model. Detailed Implementation

[0064] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0065] It should be noted that the terminology used herein is for the purpose of describing particular implementations only and is not intended to limit the exemplary implementations of the present invention.

[0066] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0067] Definitions:

[0068] About RISC-V processors:

[0069] The RISC-V hardware architecture also inherits the traditional five-stage pipeline of MIPs. In the pipeline, the lifecycle of an instruction is divided into the following steps:

[0070] Instruction fetch: This stage involves the process of retrieving instructions from memory.

[0071] Decoding: At this stage, the instruction fetched from memory is translated to determine the required operands and register indices, and then the operands are retrieved from the general-purpose register set.

[0072] Execution: Once the instruction is decoded and the required operation type and operands are determined, the execution phase begins. In this phase, the actual computation of the instruction is performed; for example, addition or subtraction is performed.

[0073] Memory access: Memory access instructions are one of the core parts of the instruction set. They involve the process of reading data from memory or writing data to memory.

[0074] Write-back: The write-back stage records the output of the instruction operation back to the general-purpose register set. For arithmetic instructions, this result comes directly from the calculation in the execution stage; for memory read instructions, the result comes from the data retrieved in the memory access stage.

[0075] like Figure 1As shown, since the next instruction can immediately enter the fetch stage immediately after the previous instruction completes the fetch and enters the decode stage, and so on, it can effectively hide execution latency by processing different stages of other instructions in parallel while the execution of one instruction is delayed, thereby increasing the number of instructions that can be completed per unit time. At the same time, the five-stage pipeline can also improve the utilization of processor resources, make it possible to increase the processor clock frequency, and improve the instruction pipeline structure, providing support for advanced features such as out-of-order execution and branch prediction.

[0076] Figure 2 The RISC-V pipeline architecture demonstrated in this example does not employ a traditional fixed-stage pipeline. Specifically, the instruction fetch stage (executed in the IFU) constitutes the first stage of the pipeline. Subsequently, within the same clock cycle, the second stage of the pipeline completes instruction decoding (handled by the EXU), execution (also handled by the EXU), and write-back (handled by the WB). Following this, in the third stage of the pipeline after the EXU, the memory access stage (handled by the LSU) arrives. However, it is important to note that the write-back result from the LSU must also be returned to the general-purpose register set via the WB module. Therefore, the processor design in this example implements a variable-length pipeline architecture.

[0077] RISC-V instruction set:

[0078] The RISC-V architecture includes basic instructions and extended instructions. Based on register width and address space, it is divided into three different integer instruction sets (denoted by I): 32-bit, 64-bit, and 128-bit. The integer instruction set includes arithmetic, logic, branch, and memory access instructions, which can already implement a complete software stack. If some CPUs have more functional requirements, extended instructions can be assembled on top of the basic instructions. The RISC-V architecture has a rich variety of extended instructions, covering a wide range of computational needs. These extended instructions mainly include, but are not limited to: multiplication and division operations, modulo and remainder operations (M extension), single-precision floating-point operations (F extension), double-precision floating-point operations (D extension), quad-precision floating-point operations (Q extension), and atomic operations (A extension), etc.

[0079] A key feature of the RISC-V instruction set is its modular design. This means that users can selectively add necessary extended instruction sets, starting from the base instruction set, to build a customized instruction set architecture based on specific application needs. Based on such an architecture, CPUs that meet specific performance and functional requirements can be designed. This flexibility and scalability are among the key reasons for RISC-V's popularity, allowing hardware designers to optimize chip performance and power consumption according to the characteristics of the target application.

[0080] This implementation example primarily uses the basic instruction set RV32I and the floating-point extended instruction set RV32F. The basic instruction set (RV32I) is the simplest instruction set in RISC-V, and it includes the following instruction classes:

[0081] Data transfer instructions: used to transfer data between registers and between registers and memory.

[0082] Arithmetic logic instructions include addition, subtraction, multiplication, division, and bit operations.

[0083] Control flow instructions include branching, jumping, and returning.

[0084] System call instructions: used to execute functions at the operating system or other system level.

[0085] The RISC-V Floating-Point Extended Instruction Set (RV32F) is a new instruction set added to the basic RISC-V instruction set to assist in scenarios requiring floating-point operations. It's a RISC-V variant based on RV32I, adding support for single-precision floating-point operations. In addition to the instruction types included in RV32I, RV32F adds floating-point instructions, including single-precision floating-point (32-bit) arithmetic instructions. These instructions conform to the IEEE 754 single-precision floating-point standard. The design of RV32F allows processor implementers to choose whether to include a floating-point instruction set based on application requirements. This enables RISC-V processors to be optimized for different application scenarios while maintaining hardware and software simplicity.

[0086] The RV32F processor can support both integer and floating-point operations on the same hardware platform, providing developers with flexibility. For example, an RV32F processor designed for general-purpose computing might include the full set of floating-point instructions, while a processor designed for specific applications, such as graphics processing or digital signal processing, might selectively implement a set of floating-point instructions as needed.

[0087] Non-invasive load identification (NILM):

[0088] Non-intrusive load monitoring (NILM) involves installing sensors at the entrance of a household to collect data. By analyzing the collected total load data and applying relevant decomposition algorithms to identify and decompose the load, detailed electricity consumption information for each electrical appliance in the user's home is obtained, thus achieving the purpose of load monitoring. Studies have shown that by providing residents with specific energy consumption information for their household appliances, it can help them improve their electricity consumption habits and save up to 20% on electricity costs, demonstrating its broad application scenarios.

[0089] Transformer neural network:

[0090] The core idea of ​​the Transformer network is to process traditional sequence-to-sequence data through a self-attention mechanism. Unlike traditional convolutional neural networks (CNNs), recurrent neural networks (RNNs), and long short-term memory networks (LSTMs), the Transformer can capture long-distance dependencies in the sequence. By assigning an attention weight to each token in the sequence, the model can dynamically focus on different parts of the input sequence, thereby better understanding the meaning of the entire sequence. At the same time, the Transformer adopts parallel computing, which greatly improves computational efficiency.

[0091] like Figure 3 As shown, the Transformer-derived network BERT consists of multiple modules that work together to model long-term sequences. The entire network comprises convolutional layers, pooling layers, position embedding layers, layer normalization layers, linear layers, deconvolutional layers, activation layers (tanh or ReLU), feedforward neural networks, and multi-attention mechanisms.

[0092] Convolutional layers are the most fundamental part of convolutional neural networks (CNNs). Their function is to extract features from the input data by performing a convolution operation between a set of filters (convolutional kernels) and the input matrix, thereby obtaining the output feature map. These convolutional kernels are obtained through training the neural network.

[0093] Pooling layers play a crucial role in convolutional neural networks. They downsample the feature maps output by convolutional layers, thereby reducing the spatial dimensionality of the data. Pooling layers can extract key features while maintaining the forward propagation efficiency of the model.

[0094] After pooling, the result needs to be added to the position encoding information, that is...

[0095] x out =x in +position_embedding

[0096] The position encoding information, position_embedding, is determined by the formula shown below.

[0097]

[0098] Next, the data containing location information is fed into a normalization layer. The normalization layer transforms the input into data with a mean of 0 and a variance of 1. The normalization formula is as follows:

[0099]

[0100] Where: w and b are the weight matrix and bias matrix obtained from training, mean is the mean of the input matrix, std is the standard deviation of the input matrix, and eps is a minimum value set to prevent the denominator from being zero when the standard deviation is zero.

[0101] The core of the Transformer model is the multi-head attention mechanism, and its computational diagram is as follows: Figure 4 As shown, the calculation formula is:

[0102]

[0103] Where: Q is the query vector, K is the key vector, V is the value vector, and d k This is the scaling factor. The Transformer can capture complex dependencies between different positions in the input sequence, thus exhibiting superior performance when processing sequence data. This embodiment uses a multi-head attention mechanism with two heads, which splits the input matrix into two sets of data, performs attention mechanism operations on each set separately, and then concatenates the results before outputting.

[0104] Example 1

[0105] This embodiment discloses a processor acceleration method for running Transformer neural networks, including:

[0106] Since the main program executed by neural networks is matrix operation, the core optimization strategy to accelerate the computation speed of neural networks is to optimize the kernel's numerical matrix operations:

[0107] Step 1: Modify the original iterative adder into an addition tree structure based on the Booth-4 encoding algorithm. First, use the radix-4 Booth encoding algorithm to calculate the partial product. Then, use an iterative method to accumulate the partial product in each cycle using the adder. After multiple cycles of iteration, the final product is obtained.

[0108] In this step, the multiplier uses a parallel adder tree structure, and the specific steps are as follows:

[0109] (1) The radix 4-Booth encoding algorithm is adopted. The difference is that the original encoding is changed to encoding the 32-bit multiplier rs1 in parallel instead of encoding once per cycle.

[0110] Simultaneously, the 32-bit multiplier rs2 is sign-extended. After sign-extending unsigned multiplication, it can be uniformly treated as a signed number for operation. The maximum product of the radix-4-Booth encoded result is x2. Therefore, obtaining the rs2 result of x1 and x2 facilitates subsequent operations. See Appendix. Figure 5 As shown.

[0111] (2) Obtain the corresponding 16 33-bit rs2 results add_tree_op0-add_tree_op15 based on the radix-4 Booth encoding booth_code0-booth_code15; see the appendix for the Booth 4 encoding process. Figure 6 As shown.

[0112] (3) Obtain the final adder array to be implemented. Considering the resource consumption and operation speed of the adder, it is expected to use a fewer bit adder to implement the 4-Booth encoding adder. Therefore, the adder tree structure is adopted to minimize the number of bits used in the adder when adding two adjacent numbers. The calculation formula of each level of adder is as follows.

[0113] First level: add_tree_fir_(i)=add_tree_op(2*i)+add_tree_op(2*i+1), i=0,1,2,3,4,5,6,7.

[0114] Second level: add_tree_sec_(i)=add_tree_fir_(2*i)+add_tree_fir_(2*i+1), i=0,1,2,3.

[0115] The third level: add_tree_thi_(i)=add_tree_sec_(2*i)+add_tree_sec_(2*i+1), i=0,1.

[0116] The fourth level: add_tree_fou_(i)=add_tree_thi_(2*i)+add_tree_thi_(2*i+1), i=0.

[0117] A schematic diagram of the multiplier structure is shown below. Figure 7 As shown below. The four types of adders used will be explained in more detail below:

[0118] The first-stage 34-bit adder needs to perform the following addition operations:

[0119] Simply add the first 31 bits of add_tree_op(2*i) to add_tree_op(2*i+1), and the lowest 2 bits of add_tree_op(2*i) go directly to the next adder. For signed multiplication, the highest 2 bits S of add_tree_op(2*i) need to be extended by the sign bit (bit 32), while for unsigned multiplication, the highest 2 bits S can be set to 0.

[0120] The addition operations that the second-stage 36-bit adder needs to perform are:

[0121] We only need to add the first 32 bits of add_tree_fir_(2*i) to add_tree_fir_(2*i+1), and the lowest 4 bits of add_tree_fir_(2*i) directly enter the next stage adder. For signed multiplication, the highest 4 bits S of add_tree_fir(2*i) need to be extended by the sign bit (bit 35), while for unsigned multiplication, the highest 4 bits S are set to 0. The difference from the first stage is that, according to the maximum value derivation, the second stage adder cannot generate a carry, so a 36-bit adder is chosen here instead of a 37-bit adder.

[0122] The addition operations that the third-level 40-bit adder needs to perform are:

[0123] The first 32 bits of `add_tree_sec_(2*i)` are added to `add_tree_sec_(2*i+1)`. The lowest 8 bits of `add_tree_sec_(2*i)` directly enter the next stage adder. For signed multiplication, the highest 8 bits `S` of `add_tree_sec_(2*i)` need to be extended by the sign bit (bit 39), while for unsigned multiplication, the highest 8 bits `S` are set to 0. Similarly, based on the maximum value derivation, the third-stage adder cannot generate a carry; therefore, a 40-bit adder is chosen here instead of a 41-bit adder.

[0124] The addition operations that the fourth-stage 48-bit adder needs to perform are:

[0125] The first 32 bits of `add_tree_thi_(2*i)` are added to `add_tree_thi_(2*i+1)`. The lowest 16 bits of `add_tree_thi_(2*i)` directly enter the next stage adder. For signed multiplication, the highest 16 bits `S` of `add_tree_thi_(2*i)` need to be extended by the sign bit (bit 47), while for unsigned multiplication, the highest 16 bits `S` are set to 0. Similarly, based on the maximum value derivation, the fourth-stage adder cannot generate a carry, therefore a 48-bit adder is chosen here instead of a 49-bit adder.

[0126] The radix-4 Booth encoding described above reduces the number of addition and shift operations required by grouping the processed data. By extending the sign bit of 32-bit operands and extending the sign bit of unsigned numbers, they can be uniformly operated on as signed numbers. Then, the operands are grouped into three-bit groups and encoded in parallel, increasing the chip area to improve the operation speed of the core adder.

[0127] The input to this multiplier typically consists of two parts: the multiplicand (the number to be multiplied) and the multiplicand (the number to be multiplied by the multiplicand). In Booth's algorithm, the focus is usually on the binary representation of the multiplicand, particularly the consecutive zeros.

[0128] The output is the result of the multiplication, that is, the product of the multiplicand and the multiplier. In the addition tree structure of the Booth coding algorithm, the output is the accumulated result of the partial products, which is the final multiplication result.

[0129] During the computation process, the output of the addition tree is usually fed into the accumulator for subsequent operations, such as further addition operations or other arithmetic operations.

[0130] The data processed is partial products. In Booth's algorithm, the decision to perform an addition, subtraction, or no operation is made by comparing adjacent bits of the multiplier, and then partial products are generated. These partial products are then accumulated in an addition tree structure to obtain the final multiplication result.

[0131] Specifically, the Booth-4 algorithm processes data in the following steps:

[0132] Encode every four consecutive bits of the multiplier to generate the corresponding opcode (addition, subtraction, or no operation).

[0133] Generate a partial product based on the opcode.

[0134] The partial products are accumulated using an addition tree structure.

[0135] The final result of the multiplication is obtained.

[0136] Step Two: Extend the Floating-Point Instruction Set: See Appendix for the overall process. Figure 8 As shown.

[0137] RV32D and RV32F are floating-point extension instructions in the RISC-V instruction set architecture, used to support floating-point operations. RV32D is a double-precision floating-point extension instruction, while RV32F is a single-precision floating-point extension instruction. This implementation example adds a single-precision floating-point instruction set, hoping to reduce the number of instructions compiled by the compiler by increasing the number of floating-point operators, thereby improving the processor's operating efficiency.

[0138] With the addition of floating-point extension instructions to the Hummingbird e203 RISC-V core, a single `fmul.s` instruction can replace complex operations. Compared to software-simulated floating-point operations using integer instructions, this significantly reduces the runtime for multiplication operations.

[0139] 2-1) Expand the register group exu_fregfile to add a floating-point register.

[0140] When using the RV32F instruction set extension, specific floating-point registers are required. The number of floating-point registers is also limited and needs to be determined based on the specific application scenario. Furthermore, the floating-point register f0 is no longer a constant 0 but a variable register like other registers. First, a floating-point register group fregfile is added, similar to regfile, and then the generate syntax is used to generate the floating-point register group. The write port logic of the floating-point register group module maps the input result register index to its corresponding register number, generating a write enable signal. The enabled general-purpose register writes data into the register. The read port is implemented using a multi-row parallel selector; the selector signal is the index of the operand register to read.

[0141] Compared to the general-purpose integer register set, the floating-point register set differs in the following ways:

[0142] a. The floating-point register set X0 can be read and written, while the integer general-purpose register set x0 is set to 0 by hardware and cannot be written to. Therefore, when generating the floating-point register set, the portion of the general-purpose registers that forces x0 to be 0 must be deleted.

[0143] b. From the structural diagram ( Figure 9 As can be seen from the diagram, the inputs and outputs of the integer general-purpose register group and the floating-point register group are not the same. In floating-point instructions, instructions such as `fmadd` require the `rs3` parameter, while integer instructions do not. Therefore, the input and output of the floating-point register group should be modified accordingly, and an `rs3` input / output channel must be added. The relevant structure is as follows... Figure 9 As shown.

[0144] 2-2) Modify the decoding stage exu_decode to add a decoding process for floating-point instructions.

[0145] The original code file exu_decode.v only contains the decoding process for the integer instruction set and part of the 16-bit reduced instruction set; the decoding process for the floating-point instruction set is missing.

[0146] The decoding module is entirely composed of combinational logic, and its main logic decodes instructions according to the instruction encoding rules of the RISC-V architecture. It generates type information for different instructions, operand register indices, etc.

[0147] RISC-V's instruction set has the advantage of modularity; the combination of different modules can represent a new instruction. Each instruction is determined by three parts: opcode, func3, and funct7. The corresponding part of the instruction is compared with a specific value; if they are equal, the corresponding signal is pulled high; otherwise, it remains low.

[0148] After obtaining the corresponding judgment signal values, these three signals can be used to select a specific instruction. Taking the floating-point instruction fsgnj.s as an example, to correctly translate this instruction, it is necessary to observe the three signals rv32_op_fp, rv32_func7_0010000, and rv32_func3_000 simultaneously. Only when all three signals are high can the instruction be identified as fsgnj.s. The absence of any one signal may lead to confusion with other instructions. For example, fsgnj.s and fsgnjn.s differ only in rv32_func3, and cannot be distinguished by the remaining two signals.

[0149] After receiving the specific instructions, it is necessary to determine the specific operations that the instructions will perform. This includes determining the pointers to rs1, rs2, rs3, and rd, or whether the ALU needs to be invoked. The FPU_op signal is used to define the instructions that require computation as op instructions, indicating that the added FPU processing module needs to be used.

[0150] Furthermore, different floating-point instructions point to different register sets. Not all instructions in the floating-point instruction set use the floating-point register set; some move data between different register sets (such as the `fmv.wx` and `fmv.xw` instructions), or access memory, reading data addresses from the integer general-purpose register set and finally writing the data back to the floating-point register set (such as the `fsw` instruction). Therefore, different enable signals are needed to indicate whether these structures and the specific pointers pointed to by `rs1`, `rs2`, `rs3`, and `rd` are present in the floating-point instructions.

[0151] Except for flwsp and fswsp, all floating-point instruction sets (including the reduced instruction set) use rs1.

[0152] Except for flw, fmv.xw, fmv.wx, fsqrt, and flwsp, all floating-point instruction sets (including the reduced instruction set) use rs2.

[0153] In the floating-point instruction set (including the reduced instruction set), only fmadd and fnmadd use rs3.

[0154] The floating-point instruction set (including the reduced instruction set) uses rd except for fsw and fswsp.

[0155] For the indexing of floating-point registers, since the register indexing location of floating-point instructions is the same as that of other instructions, the decoded index value can be directly assigned to the corresponding index of the FPU register. After determining whether the floating-point instruction has rs1, rs2, rs3, rd and the corresponding index values, it is necessary to determine the register set type corresponding to the floating-point instruction, whether it is an integer general-purpose register set or a floating-point register set.

[0156] Floating-point instructions (including the reduced instruction set) with the rs1 parameter, except for flw, fsw, and fmv.wx, all point to the floating-point register group.

[0157] For floating-point instructions (including the reduced instruction set) with the rs2 parameter, rs2 points to the floating-point register set.

[0158] For floating-point instructions (including the reduced instruction set) with the rs3 parameter, rs3 points to the floating-point register set.

[0159] Floating-point instructions (including the reduced instruction set) with the rd parameter, except for fmv.xw and flt, all point to the floating-point register set.

[0160] Multiplexing it to the corresponding rv32_need_rs1, rv32_need_rs3, rv32_need_rs3, and rv32_need_rd signals will output it to the corresponding register group.

[0161] After the decoding process is complete, the information bus required by the FPU needs to be generated. The purpose of this information bus is to transmit the decoded information to the embedded FPU module, allowing the FPU module to understand the operation required by the currently executing instruction based on the bus content. The bus information is selected and output by dec_info. A classic parallel multiplexer multiplexes different bus signals onto the output signal (dec_info), and the selection signal is the previously mentioned op signal, which indicates whether to use the module in the ALU. It effectively groups the signals. The decoding part ends. Due to the excellent structured nature of RISC-V instructions, the immediate value decoding part of the instruction does not require major modifications and can remain unchanged.

[0162] 2-3) Modify the dispatch module exu_disp.

[0163] After decoding, the decoded result of the instruction and the operands read from the register set are dispatched to different arithmetic units for execution. Before any floating-point instructions are added, all instructions are dispatched to the ALU and delivered through the interface between the ALU and the delivery module. If the instruction is a long instruction, it will be further dispatched by the ALU to the corresponding long instruction arithmetic unit; for example, the long instructions Load and Store are further dispatched to the LSU for execution through the ALU submodule AGU.

[0164] Since all instructions must pass through the ALU, the actual dispatch function occurs within the ALU. `exu_decode` has already grouped the instructions according to their execution unit during the decoding process and provided corresponding indicator signals (op signals). Therefore, the purpose of the `exu_disp` module is to pass signals to the ALU and transmit long instruction signal information to the OITF. Continuing the design philosophy of the Hummingbird E203, all instructions are transmitted to the ALU for execution; therefore, only the decoding results of floating-point instructions need to be added at the input / output interface.

[0165] Since the parameter rs3 is used for the first time in floating-point instructions, an index signal, a judgment signal, and a specific value for rs3 need to be added to the input and output.

[0166] Furthermore, discrimination signals need to be added to the input and output to determine the type of register set being pointed to; if it points to a floating-point register set, then... The actual dispatch function is also executed in the ALU. The grouping information generated by the decoding module is used to determine the instruction and dispatch it to the appropriate unit in the ALU for execution. Because of the addition of the `fonecycle` and `fdivsqrt` modules, their grouping information needs to be added to ensure that floating-point instructions are sent to the correct unit for execution.

[0167] The remaining two non-operational long instructions, flw and fsw, are added to the same execution process as sw and lw, and also added to the AGU's information bus for dispatch to the AGU for processing, and further sent to the LSU for execution. The LSU is the main control unit of the processor's memory subsystem, used to access different memory units such as DTCM and ITCM.

[0168] 2-4) Modify the calculation and processing module exu_alu.

[0169] To implement floating-point instruction operations, the `fonecycle` and `fdivsqrt` modules were added to the original ALU architecture. These modules are integrated from the open-source floating-point code HardFloat from UC Berkeley and can handle all floating-point instructions. The idea of ​​dividing these instructions into two parts is inspired by the division of single-cycle and multi-cycle instructions in the Hummingbird e203 RISC-V core—in the ALU, integer multiplication and division are separated because they use multi-cycle shift multipliers, meaning the instructions cannot be completed in a single cycle. Conversely, in the `fonecycle` module, all floating-point instructions, including floating-point multiplication (`fmul`), can be processed in a single cycle. The `fdivsqrt` module contains only two instructions, `fdiv` and `fsqrt`, which are multi-cycle instructions, requiring 24 clock cycles to execute.

[0170] This example uses the Fonecycle as an example to introduce the specific process of floating-point instruction processing.

[0171] Besides the input and output values ​​frs1, frs2, frs3, and farethical_res, the exception_flags signal outputs an exception flag. Different exception flags indicate possible exceptions during floating-point calculations. This flag is written to the FCSR register along with the output of the Fonecycle. The FCSR register will be explained in a later report. As a five-bit output, exception_flags represents five different exceptions in floating-point calculations: illegal operation, division by zero, overflow, underflow, and inaccuracy. The corresponding bit is set high for each exception. When both underflow and inaccuracy exceptions occur simultaneously, the value of exception_flags is 00011.

[0172] roundingMode:

[0173] This input signal indicates the rounding mode for floating-point numbers. According to the IEEE-754 standard, floating-point operations require specifying the rounding mode.

[0174] (1) Static rounding mode: Three bits are used as the rounding mode field in the encoding of floating-point instructions. Different rounding mode encodings are shown in the figure. The RISC-V architecture supports five legal rounding modes. In addition, if the rounding mode encoding is 101 or 110, it is an illegal mode; if the rounding mode encoding is 111, it means that dynamic rounding mode is used.

[0175] (2) Dynamic rounding mode: If dynamic rounding mode is used, the rounding mode field in the FCSR register is used. The FCSR register contains a rounding mode field and only supports five valid rounding modes. If the rounding mode field in the FCSR register is specified as an invalid rounding mode, subsequent floating-point instructions will generate an illegal instruction exception.

[0176] The value in fonecycle_i_info comes from FPU_info in decode. The fonecycle module decodes the data transmitted on the bus to obtain the specific floating-point instructions to be executed, and assigns the selection signal to ftype. ftype is used to control the specific processing operations performed by the FPU module.

[0177] After the FPU module finishes execution, the result will be returned to its top-level module, ALU. wbck_o_wdat is the final write-back result.

[0178] `fdivsqrt` is a module integrated into Berkeley's floating-point arithmetic code. Because `fdiv` and `fsqrt` instructions require multiple cycles to complete, they need special handling. Otherwise, while these instructions are still executing, new instructions might be fetched, decoded, and executed, leading to conflicts and errors. In `exu_decode`, these two instructions are not categorized into the overall FPU signal bus. Instead, like `muldiv`, they are given a separate bus, outputting to the independent `fdivsqrt` module within the ALU.

[0179] fdivsqrt_i_info: This is the bus information for fdivsqrt, and this module is parallel to the FPU. Because it only implements the two floating-point long instructions fatqrt and fdiv, only one bit is needed for this bus.

[0180] The instructions processed by fdivsqrt are long instructions with multiple cycles. During execution, attention should be paid to: pipeline conflicts, long instructions, and OITF handling.

[0181] Pipeline conflicts fall into two categories: resource conflicts and data conflicts. Both types of conflicts can cause pipeline blockages. The Hummingbird e203 employs two different methods to handle resource conflicts and data conflicts respectively.

[0182] Data conflict, as the name suggests, is a conflict caused by the correlation of data.

[0183] The Hummingbird e203 employs a clever method to handle data conflicts: it divides all instructions into two categories and data dependencies into three categories, and processes them through long instruction concatenation and pipeline flushing.

[0184] Resource conflicts typically occur when instructions are dispatched to different execution units. A resource conflict arises when an instruction takes a long time to execute, and then other instructions are subsequently dispatched to the same hardware module. Subsequent instructions must wait for the previous instruction to complete its operation and release the hardware module before they can be executed.

[0185] The Hummingbird e203 uses a strict valid-ready handshake interface. Once a resource conflict occurs in a module, it will output a ready=0 signal. Even if the other side has valid=1, the handshake cannot be completed. Therefore, the previous module cannot allocate instructions and will enter a waiting state until ready=1.

[0186] Long instructions and OITF processing

[0187] The Hummingbird E203 categorizes all instructions to be executed into two types:

[0188] Single-cycle execution instructions: The delivery and write-back functions of the Hummingbird E203 are both at the second stage of the pipeline, and single-cycle execution instructions complete the delivery and write-back at this stage. Therefore, it does not involve the conflict mentioned above.

[0189] Multi-cycle instruction execution: These instructions typically require multiple clock cycles to complete execution and write back, hence they are also called "long-pipeline instructions with delayed delivery," or simply long instructions. To deliver long instructions after many clock cycles, data dependencies need to be detected. The Hummingbird E203 uses a module called OITF (Outstanding Instructions Track FIFO) to detect RAW and WAW dependencies associated with long instructions. WAR dependencies are not detected because the e203 uses an in-order dispatch and in-order write-back microarchitecture; the source operands are already read from the register bank during dispatch, so the write-back operation to the regfile does not occur before reading the source operands from the regfile.

[0190] OITF is essentially a FIFO. At the dispatch point, each time a long instruction is dispatched, an entry is allocated in the OITF. This entry stores the source operand register index and result register index of the long instruction. At the write-back point, after each long instruction is written back in sequence, the entry of this instruction in the OITF is removed, and the FIFO is exited. In essence, OITF stores information about long instructions that have been dispatched but have not yet been written back.

[0191] When each instruction is dispatched, the source and destination operand register indices are compared with the entries in the OITF to determine if the instruction has RAW or WAW dependencies with long instructions that have already been dispatched but not yet written back. If a dependency exists, dispatching the current instruction is paused. If there are no RAW or WAW dependencies, and the instruction is a multi-cycle long instruction, it is written to the OITF. If the OITF is full, dispatching is still stopped, and the instruction is written and dispatched only after the OITF has freed up space. The added fsqrt and fdiv instructions are precisely these multi-cycle instructions, therefore they should be input as long instruction signals to the OITF to check their dependencies and determine whether the pipeline needs to be blocked.

[0192] 2-5) Add a floating-point CSR register.

[0193] The RISC-V architecture specifies that if single-precision or double-precision floating-point instructions are supported, a floating-point control status register (fcsr) needs to be added, such as... Figure 9 As shown. It is a 32-bit read / write register used for selecting dynamic rounding modes in floating-point arithmetic operations and storing exception flags.

[0194] The fcsr register consists of user-reserved bits, frm, and fflags. Although the RISC-V architecture only defines one floating-point control register (fcsr), the different fields frm and fflags of this register, as well as the register itself fcsr, are all assigned independent CSR addresses.

[0195] For example, the `frm` field of the `fcsr` register can be read and written individually via address 0x002, and `fflags` can be read and written via address 0x001. However, it's important to note that these three read / write modes should be independent of each other. When writing data to `fflags` individually, care must be taken not to overwrite other data bits, as the data in `frm` may be used in subsequent instructions. Therefore, these three write modes cannot be simply reused; selection signals must be set to use different write modes. Setting the `sel_fcsr`, `sel_frm`, and `sel_fflags` signals allows the determination of the specific write mode, preserving the original values ​​that do not need to be overwritten and writing the new values.

[0196] The reading of the CSR register is also done by a multiplexer to select the specific output result. The selection signal, such as rd_mtvec, is obtained by judging the access address - sel_mtvec = (csr_idx == 12'h305); the value in the mtvec register will be assigned to the read_csr_dat output.

[0197] In floating-point instructions, the purpose of `fcsr` is to write back the exception flags during floating-point operations. While regular CSR register reads and writes can be performed using dedicated instructions, writing `fflags`, in addition to using dedicated CSR instructions, should be automatically written back after each FPU operation as part of the result. The exception flag data `exception_flags` from the `fonecycle` is added to the CSR write data signal `wbck_csr_dat`.

[0198] The write enable signal of the CSR is pulled high by the fonecycle's identification signal, so that the CSR register can be read and written by the fonecycle even when there is no CSR instruction.

[0199] 2-6) Modification of the misa register.

[0200] The misa register is used to indicate the architectural features supported by the current processor.

[0201] The highest two bits of the misa register are used to indicate the architecture bitness supported by the current processor.

[0202] If the two highest bits are 1, it indicates a 32-bit architecture (RV32). If the two highest bits are 2, it indicates a 64-bit architecture (RV64). If the two highest bits are 3, it indicates a 128-bit architecture (RV128).

[0203] The lower 26 bits of the misa register are used to indicate the different modular instruction subsets in the RISC-V ISA supported by the current processor. In the RISC-V architecture documentation, the misa register is defined as a read-write register, allowing certain processor designs to be dynamically configured with specific features. However, in the Hummingbird e203 implementation, the misa register is a read-only register. It consistently reflects the ISA modular subsets supported by different processor cores. For example, if the Hummingbird e203 core supports RV32IMAC, this is reflected in the register with the highest two bits set to 1, and the values ​​of the corresponding I / M / A / C fields in the lower 26 bits set to high.

[0204] After adding the floating-point instructions, you need to pull the corresponding bit in the MISA register high; otherwise, the debugger will report an error indicating that the current processor does not support the F instruction set. When pulling the bit corresponding to the F instruction set high, be careful not to confuse it with the double-precision floating-point instruction set.

[0205] Decoder module: The decoder module is responsible for translating the machine code of instructions into control signals and operands.

[0206] Dispatcher module: The dispatch module sends instructions to the corresponding execution unit according to the type of the decoded instruction.

[0207] Execution Unit: The execution unit is responsible for performing specific arithmetic or logical operations.

[0208] Floating-Point Control and Status Register (fcsr): The floating-point CSR register is used to store the status and control information of floating-point operations.

[0209] The logical relationship is as follows: the decoding module identifies the floating-point instruction and passes its characteristics to the dispatch module. The dispatch module dispatches the floating-point instruction to the floating-point calculation and processing module according to the instruction type. The calculation and processing module performs the floating-point operation and updates the result and status information to the floating-point CSR register.

[0210] Relationship with acceleration: Decoding module: Acceleration is reflected in faster recognition and decoding of floating-point instructions. Dispatch module: Acceleration is reflected in more efficient instruction routing and reduced instruction execution latency. Computation module: This is the core of acceleration, significantly improving floating-point performance through hardware acceleration of floating-point operations. Floating-point CSR register: Acceleration is reflected in fast access to and updating of floating-point operation status information.

[0211] Decoding module: Input is the instruction in machine code form. Dispatch module: Input is the decoded instruction characteristics and control signals. Calculation processing module: Input is operands and opcodes. Floating-point CSR register: Input is floating-point arithmetic status information.

[0212] Processing steps: Decoding module: Parses instructions, extracts opcodes and operands, and generates control signals. Dispatch module: Routes instructions to the corresponding execution units based on instruction type. Calculation module: Performs floating-point operations, such as addition, multiplication, and division. Floating-point CSR register: Stores and updates status information such as exceptions and rounding modes in floating-point operations.

[0213] The outputs are as follows: Decoding module: outputs control signals and operands. Dispatch module: outputs instruction dispatch information, instructing the execution unit to begin execution. Calculation processing module: outputs calculation results and possible status information (such as overflow, underflow, etc.). Floating-point CSR register: outputs the status information of the current floating-point operation for use by subsequent instructions.

[0214] Step 3: Neural Network Training Optimization

[0215] To make the neural network run faster on processor cores with optimized adders, the neural network was optimized to a certain extent by modifying the original BERT neural network structure. In the original network model structure, the positional encoding was obtained through self-learning, meaning the model learned the positional information related to the sequence during training. Since the application scenario is non-intrusive load identification, the input data of the neural network is all users' electricity consumption, which is presented in chronological order. This data inherently contains their own positional information. Therefore, the original self-learning positional encoding method was modified to a fixed sine and cosine encoding method. This reduces the computational complexity and memory requirements of the model. Furthermore, because sine and cosine encoding is periodic, it means it can better generalize to sequence lengths not seen during training. For the BERT model, this means that when the model processes data exceeding the length of the longest sequence during training, it can more effectively utilize positional information.

[0216] For BERT neural network models (such as...) Figure 3 As shown, firstly, the PyTorch library function `nn.Conv1d()` is used to perform a one-dimensional convolution on the input. This convolution process uses a kernel size of 5 and padding of 2. The convolution process expands the input matrix data from 32×480 (note that this neural network is trained in groups of 32 to speed up training, leveraging the GPU's strength in tensor computation; the input used during deployment is 1x480) into a 32×480×256 three-dimensional matrix. The weights inside the convolution kernel are obtained through neural network training. This one-dimensional convolution process on the 1×480 array is repeated 256 times to obtain the convolution output.

[0217] The next step is to use PyTorch's built-in `nn.LPPool1d()` function to perform a one-dimensional pooling operation on the convolution result, effectively reducing the size of the parameter matrix and thus reducing the number of parameters in the connected layers. After pooling, the original 32×480×256 matrix is ​​reduced to a 32×240×256 three-dimensional matrix.

[0218] The next step is to add the pooled result to the position encoding information, that is...

[0219] x out =x in +position_embedding

[0220] The position encoding information, position_embedding, is determined by the formula shown below.

[0221]

[0222] Next, the data containing location information is fed into a normalization layer. This normalization layer transforms the input into data with a mean of 0 and a variance of 1, accelerating the model's convergence. The data normalization formula and code are as follows:

[0223]

[0224] Where: w and b are the weight matrix and bias matrix obtained from training, mean is the mean of the input matrix, std is the standard deviation of the input matrix, and eps is a minimum value set to prevent the denominator from being zero when the standard deviation is zero.

[0225] After the normalization layer, the code enters the Transformer module. The Transformer module consists of two repeating encoders: a multi-head attention mechanism module and two fully connected networks. The operation of the multi-head attention mechanism is illustrated below. Figure 4 As shown.

[0226] Since a multi-head attention mechanism with 2 heads is used, the input matrix passed to the Transformer module is divided into two sets of data, attention operations are performed on each set separately, and the results are concatenated and output.

[0227] The first fully connected layer performs a linear operation between the result of the multi-head attention mechanism and the matrix of the initial input Transformer module, while the second fully connected layer performs a linear operation between the output of the previous fully connected layer and the position-fully connected feedforward network, which includes two linear transformations and a non-linear activation process (using the ReLU non-linear activation function).

[0228] To improve the model's accuracy, two consecutive Transformer modules were used. After the data passed through the two Transformer modules, it entered the deconvolution step. The deconvolution used the nn.ConvTranspose1d() function built into PyTorch, which is the inverse operation of the convolution operation nn.Conv1d().

[0229] After the deconvolution step is completed, the final output of the neural network is obtained by passing through two linear layers and one non-linear activation layer tanh(). Their weight matrices and bias matrices are obtained by training the neural network.

[0230] After modifying the BERT network structure, the network model was retrained and tested. The predicted power values ​​of the refrigerator were compared with the standard values. Figure 10 As shown in the figure, orange represents the predicted value and blue represents the standard value. It can be seen that the predicted value of the refrigerator power is close to the standard value, indicating that the network has a good inference effect.

[0231] Functional simulation test:

[0232] To test the functionality of the muldiv module under the ALU, the open-source risc-v-test instruction set self-test suite was used for functional verification. This primarily verified the functionality of the mul, mulh, mulhu, and mulhsu instructions, as well as the div and divu instructions within the Open Instruction Set Architecture (ISA). Test files targeting these instructions were run, and all instructions passed the tests.

[0233] Coremark's performance metric is typically expressed as Coremark / MHz. This metric indicates how many times a processor can run the Coremark program per second at a specific clock frequency (MHz). A higher number indicates better processor performance. Coremark / MHz provides a unified standard, allowing for direct performance comparisons between processors of different architectures and frequencies.

[0234] After the adder function was tested and found to be working correctly, the optimization effect of the adder was tested and observed. The open-source SDK toolchain was used to generate a coremark.verilog file to simulate the optimization effect and obtain the optimized performance parameters. Similar to the ISA simulation steps, the generated coremark test script was executed. The final score was 3.135 CoreMark / MHz, which is about 47.8% higher than the original benchmark score of 2.12 CoreMark / MHz.

[0235] FPGA Platform Verification: After data is output from the radix-4 Booth encoding stage, it needs to pass through four stages of adders. The critical path is at least: 34-bit adder - 36-bit adder - 40-bit adder - 48-bit adder, making this combinational logic circuit large. The RISC-V core used operates at a 16MHz clock frequency, and the circuit needs to meet timing constraints. Therefore, an FPGA platform is used for verification, allowing for comparison of optimized resource consumption. The FPGA verification platform uses the XC7A100T-2FGG484 (Zhengdian Atom Da Vinci development board).

[0236] Since the RISC-V core used in this study requires two external crystal oscillators—one for the normally open domain (32.768 kHz) and the other for the main domain (16 MHz)—and the development board only has one clock, the 16 MHz clock needs to be divided. The original RISC-V core code and the optimized RISC-V core code were synthesized separately, and the resource consumption was compared, as shown in Table 1.

[0237] Table 1 Hardware Resource Consumption

[0238] parameter Before optimization After optimization LUT 11274 12561 LUTRAM 16 16 FF 11243 11243 BRAM 32 32 IO 80 80 BUFG 4 4 MMCM 1 1 WNS 24.864ns 24.454ns WHS 0.36ns 0.029ns WPWS 7.0ns 7.0ns Total On-Chip Power 0.23W 0.23W

[0239] As can be seen, in terms of resources, only the LUT usage increased from 11274 to 12561, an increase of 11.4%. Both timing margins decreased slightly, with WNS decreasing from 24.864ns to 24.454ns and WHS decreasing from 0.36ns to 0.029ns. However, both are positive values ​​and still meet the timing requirements.

[0240] The bitstream files (.bit and .bin) were generated and burned onto the development board. The development board was then used to execute the CoreMark test program, and the CoreMark score was sent back to the host computer via serial port. The CPU clock speed was 16MHz, and after running 500 rounds of the CoreMark program, the final score was 3.135CoreMark / MHz, which was verified to be correct by the FPGA.

[0241] Similarly, the Dhrystone and Whetstone programs were also burned onto the board for benchmarking: the Dhrystone score increased from 1.28 DMIPS / MHz to 1.43 DMIPS / MHz, an increase of 11.7%; the Whetstone score increased from 0.04 MWIPS / MHz to 0.051 MWIPS / MHz, an increase of 27.5%.

[0242] The optimization of the RISC-V core multiplier enables it to better perform forward inference of neural networks, increasing the coremark to 3.135 coremark / MHz within an acceptable range of chip resource overhead. This optimization has proven effective in both host computer simulation and FPGA implementation. It also prepares for subsequent floating-point instruction set expansion and neural network optimization and operation. The test results are summarized in Table 2.

[0243] Table 2 Summary of Test Results

[0244]

[0245]

[0246] Floating-point instruction set extension test

[0247] Feasibility test

[0248] We use open-source self-check test cases to check whether the modified code runs successfully or fails. riscv-test is an open-source project maintained by the developers of the RISC-V architecture and contains test programs to test whether a processor conforms to the instruction set architecture definition. These assembly test programs use macro definitions to organize program points and test the instructions defined in the instruction set architecture. If the execution result of an instruction is equal to the expected result, the program continues execution; if it is not equal to the expected result, the program directly uses the jump instruction to jump to the TEST FAIL address; assuming all test points pass, the program will continue execution until the TEST PASS address. Below, we perform feasibility tests on the relevant instruction set.

[0249] Floating-point speedup test results:

[0250] The execution time and efficiency of the RISC-V kernel with software-simulated floating-point instructions and the hardware floating-point execution time after adding the floating-point instruction set were tested and compared. Various floating-point instructions were executed in both kernels, and the number of cycles required to execute the corresponding instructions were observed and compared. The time saved by software and hardware floating-point operations is shown in Table 3.

[0251] Table 3 Comparison of time saved in floating-point operations by software and hardware.

[0252] instruction Software floating point Hardware floating point Save time per calculation fmul 145 32 113 fdiv 224 58 166 fadd 120 32 88 fsub 149 32 117 fmadd 225 35 190 fnmadd 230 35 195 fsqrt 1224 56 1186

[0253] Except for the fsqrt instruction, which has a longer cycle count, the other instructions have significantly improved execution efficiency, saving 75%-96% of the execution time for the same computational work.

[0254] It's worth noting that, regardless of whether the `sqrt` command is used in the floating-point or integer GCC toolchain, the synthesis result is always simulated using integer instructions. To further save time, assembly code needs to be embedded in the C code to force the square root operation to be executed using `fsqr.s`.

[0255] Neural Network Operation and Testing

[0256] Since the neural network needs to be deployed on a RISC-V core, the neural network model needs to be written in C, and the Transformer module and the entire neural network need to be built. The C code is then burned into the Hummingbird E203 processor, runs on the RISC-V core, and the processing results are sent back to the host computer via serial port for display. The results are completely consistent with the standard results. At this point, the functional verification is successful.

[0257] Performance verification:

[0258] The synthesis report contains four code files: ① Original project ② Project after score optimization ③ Project after floating-point instruction extension ④ Project after score optimization and floating-point instruction extension. Some of the synthesis results are listed in Table 5.

[0259] Table 5 Comparison of Resource Consumption

[0260]

[0261] Although the timing margin was further reduced in the final project, it was still able to operate stably at a clock frequency of 16MHz.

[0262] To compare the speedup effect of hardware floating-point operations compared to software-simulated floating-point operations, a comparison was made using CORE=e205f compiled and flashed (hardware floating-point instructions) and CORE=e203 compiled and flashed (software-simulated floating-point instructions). To make the runtime more intuitive, code for counting operation cycles and time was added to the C program. Finally, the runtime and number of cycles before optimization (software-simulated floating-point operations), after optimization (hardware floating-point operation instructions), and using hardware fsqrt to implement square root operations (the sqrt() function of the math library will use software square root operations after assembly, which requires 1224 clock cycles; if inline assembly syntax is used to force the execution of fsqrt, the runtime cycle can be further shortened) are displayed in Table 6.

[0263] Table 6 Comparison of Running Speed

[0264] parameter Before optimization After optimization Optimized (including fsqrt) Total ticks 647280 371185 358536 Total times 0.040476 0.023196 0.022412

[0265] Before optimization, the number of clock cycles required to execute the test routine was 647,280, while after optimization, the number of clock cycles required was only 358,536, which is 55.4% of the original. Therefore, the total time was reduced from 0.040476 to 0.022412.

[0266] To address the need for non-invasive NIL M load recognition, the BERT neural network used was optimized and tested, improving its recognition accuracy and precision. Running the neural network with the optimized kernel reduced the runtime from 165s to 91.4s.

[0267] Example 2

[0268] The purpose of this embodiment is to provide a general-purpose Transformer acceleration system based on floating-point instruction set extensions, including:

[0269] The multiplier construction module is configured to: construct a multiplier, which includes a series of adders connected in sequence, and encode the data using a 4-Booth encoding algorithm; the encoded data is then processed by the series of adders to obtain the output of the multiplier;

[0270] The floating-point register construction module is configured to construct floating-point registers. The specific steps are: adding a floating-point register group to generate the floating-point register group. The write port logic of the floating-point register group maps the input result register index to its corresponding register number, generating a write enable signal. The enabled general-purpose register writes data into the register. The read port is implemented using a multiplexer, and the selection signal of the multiplexer is the index of the operand register to be read.

[0271] The accelerated execution module is configured to run the Transformer neural network based on the constructed multiplier and floating-point registers.

[0272] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A general Transformer acceleration method based on floating-point instruction set extensions, characterized by: include: Construct a multiplier, which consists of multiple stages of adders connected in sequence, and encode the data using the 4-Booth encoding algorithm; The encoded data is processed sequentially by a multi-stage adder to obtain the output of the multiplier; the multi-stage adder of the multiplier adopts an addition tree structure to minimize the number of bits used in the adder when adding two adjacent numbers. The specific steps for constructing a floating-point register are as follows: a floating-point register group is generated, the write port logic of the floating-point register group corresponds the input result register index with its respective register number, generates a write enable signal, the enabled general-purpose register writes data into the register, and the read port is implemented using a multiplexer, the selection signal of the multiplexer is the index of the read operand register; Extended register set: Add a floating-point register; Modify the decoding process: Add a decoding process for floating-point instructions; Modify the dispatch module; Modify the calculation and processing module; Add a floating-point CSR register; The decoder identifies the floating-point instruction and passes its characteristics to the dispatch module. The dispatch module dispatches the floating-point instruction to the floating-point calculation and processing module according to the instruction type. The calculation and processing module performs the floating-point operation and updates the result and status information to the floating-point CSR register. Decoding and recognizing floating-point instructions involves the following steps: Generate type information and operand register indexes for different instructions; Determine the target of each instruction, compare the corresponding part of the instruction with the specific value, and if they are equal, pull the corresponding signal high; otherwise, keep it low. After obtaining the corresponding judgment signal value, a specific instruction is selected; After receiving the specific instructions, it is necessary to determine the specific operations that the instructions will perform. Different floating-point instructions will point to different register sets; Not all instructions in the floating-point instruction set use the floating-point register set. Some instructions move data between different register sets or access memory, read data addresses from the integer general-purpose register set, and finally write the data back to the floating-point register set. Different enable signals are used to indicate whether these structures exist in the floating-point instructions and the specific pointers of rs1, rs2, rs3, and rd; The RISC-V processor runs Transformer neural networks based on a built-in multiplier and floating-point register.

2. The general-purpose Transformer acceleration method based on floating-point instruction set extension as described in claim 1, characterized in that it further... include: The decoded index value is assigned to the index of the corresponding register in the FPU; After determining whether a floating-point instruction has rs1, rs2, rs3, rd and their corresponding index values, the register group type corresponding to that floating-point instruction is determined.

3. The general Transformer acceleration method based on floating-point instruction set extension as described in claim 1, characterized in that, The specific steps to determine the register set type corresponding to this floating-point instruction are as follows: Except for flw, fsw, and fmv.wx, the rs1 parameter of floating-point instructions all point to the floating-point register group; In floating-point instructions with the rs2 parameter, rs2 always points to the floating-point register set; In floating-point instructions with the rs3 parameter, rs3 always points to the floating-point register set; Except for fmv.xw and flt, the rd parameter of floating-point instructions with the rd parameter points to the floating-point register set.

4. The general-purpose Transformer acceleration method based on floating-point instruction set extension as described in claim 1, characterized in that, After decoding is complete, the decoded result of the instruction and the operands read from the register set will be dispatched to different arithmetic units for execution; All instructions are transferred to the ALU for execution, and the decoding results of floating-point instructions are added to the input / output interfaces.

5. The general Transformer acceleration method based on floating-point instruction set extension as described in claim 1, characterized in that, Add the decoded results of floating-point instructions to the input / output interface, specifically: First, the parameter rs3 is used for the first time in floating-point instructions, and index signals, judgment signals and specific values ​​of rs3 are added to input and output; In addition, a discrimination signal needs to be added to the input and output to determine the type of register group to be pointed to. If it is a floating-point register group, the actual dispatch function is also arranged to be executed in the ALU. The group information generated by decoding is used to make a judgment and dispatch it to the corresponding unit in the ALU to execute the instruction. The addition of grouping information from the fonecycle and fdivsqrt modules ensures that floating-point instructions can be sent to the correct cell for execution. The remaining two non-operational long instructions, flw and fsw, are added to the same execution process as sw and lw, and are also added to the AGU's information bus for dispatch to the AGU for processing, and then sent to the LSU for execution.

6. A general-purpose Transformer acceleration system based on floating-point instruction set extensions, characterized in that: include: The multiplier construction module is configured to: construct a multiplier, which includes a series of interconnected multi-level adders; encode the data using a 4-Booth encoding algorithm; perform operations on the encoded data through the multi-level adders to obtain the output of the multiplier; the multi-level adders of the multiplier adopt an addition tree structure to minimize the number of bits used in the adders when adding adjacent numbers. The floating-point register construction module is configured to: construct floating-point registers. The specific steps are: generate a floating-point register group; the write port logic of the floating-point register group corresponds the input result register index with its respective register number to generate a write enable signal; the enabled general-purpose register writes data into the register; the read port is implemented using a multiplexer; the selection signal of the multiplexer is the index of the read operand register. Extended register set: Add a floating-point register; Modify the decoding process: Add a decoding process for floating-point instructions; Modify the dispatch module; Modify the calculation and processing module; Add a floating-point CSR register; The decoder identifies the floating-point instruction and passes its characteristics to the dispatch module. The dispatch module dispatches the floating-point instruction to the floating-point calculation and processing module according to the instruction type. The calculation and processing module performs the floating-point operation and updates the result and status information to the floating-point CSR register. Decoding and recognizing floating-point instructions involves the following steps: Generate type information and operand register indexes for different instructions; Determine the target of each instruction, compare the corresponding part of the instruction with the specific value, and if they are equal, pull the corresponding signal high; otherwise, keep it low. After obtaining the corresponding judgment signal value, a specific instruction is selected; After receiving the specific instructions, it is necessary to determine the specific operations that the instructions will perform. Different floating-point instructions will point to different register sets; Not all instructions in the floating-point instruction set use the floating-point register set. Some instructions move data between different register sets or access memory, read data addresses from the integer general-purpose register set, and finally write the data back to the floating-point register set. Different enable signals are used to indicate whether these structures exist in the floating-point instructions and the specific pointers of rs1, rs2, rs3, and rd; The acceleration module is configured to accelerate the operation of the Transformer neural network based on the constructed multiplier and floating-point register.

Images