Universal Transform acceleration method and system based on floating point instruction set extension

Abstract

The invention provides a universal Transform acceleration method and system based on floating point instruction set extension, and the method comprises the steps: constructing a multiplier which comprises a plurality of stages of adders and a 4-Booth coding algorithm which are connected in sequence, and coding data; the coded data is operated through a plurality of stages of adders in sequence to obtain the output of a multiplier; the construction of the floating-point number register comprises the following specific steps: adding a floating-point number register group, generating the floating-point number register group, enabling input result register indexes to correspond to respective register numbers by writing port logic of the floating-point number register group, generating a writing enable signal, writing data into the register by an enabled general register, and writing the data into the floating-point number register group. The read port is realized by using a multi-path parallel selector, and a selection signal of the multi-path parallel selector is an index of a read operand register; and the RISC-V processor accelerates the operation of the Transform neural network based on the constructed multiplier and the floating-point number register.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer Transformer neural network acceleration, and in particular relates to a general Transformer acceleration method and system based on floating-point instruction set extension. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] The open source processor architecture RISC-V has developed rapidly in recent years. RISC-V is the fifth generation of the RISC series instruction set. It has a compact and powerful architecture and adopts a modular design concept, which enables users to flexibly choose different modules to combine to meet different usage scenarios. Hummingbird E203 is a completely domestically produced processor core based on the RISC-V instruction set architecture. It adopts a modular design, has good scalability and flexibility, and is suitable for various embedded system applications. The Hummingbird E203 core has the characteristics of high performance and low power consumption, which can meet the performance requirements in different scenarios.

[0004] Transformer is a neural network architecture based on the attention mechanism, which consists of an encoder and a decoder. In the field of natural language processing, the Transformer model is widely used in tasks such as text classification, machine translation, and sentiment analysis. At the same time, it has also demonstrated strong application capabilities in fields such as speech recognition, computer vision, and reinforcement learning. Compared with traditional neural network structures such as CNN and RNN, Transformer, as a new type of network structure, has a strong latecomer advantage and can replace traditional convolutional neural networks and recurrent neural networks in a variety of tasks.

[0005] Currently, RISC-V (taking E203 as an example) has problems such as low computing power and is mostly used in low-power and embedded environments. At the same time, since E203 does not contain a floating-point instruction set, but uses integer instructions to simulate floating-point instructions, the computing speed of RISC-V is greatly reduced, and it is impossible to effectively perform forward reasoning on neural networks. At present, the work of combining Transformer neural network reasoning with RISC-V is still very lacking. Summary of the invention

[0006] In order to overcome the shortcomings of the above-mentioned prior art, the present invention provides a general Transformer acceleration method based on floating-point instruction set extension, improves the computing power of the RISC-V core, adds a floating-point instruction set to it, and attempts to run a neural network based on the optimized BERT architecture on it.

[0007] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions:

[0008] In the first aspect, a general Transformer acceleration method based on floating-point instruction set extension is disclosed, including:

[0009] A multiplier is constructed, the multiplier includes a plurality of adders connected in sequence, and a 4-Booth encoding algorithm is used to encode data; the encoded data is sequentially operated by the plurality of adders to obtain an output of the multiplier;

[0010] Constructing floating-point registers, the specific steps are: generating a floating-point register group, the write port logic of the floating-point register group corresponds the input result register index to the respective register number, generating a write enable signal, and the enabled general register writes the data into the register, and the read port is implemented using a multi-way parallel selector, and the selection signal of the multi-way parallel selector is the index of the read operand register;

[0011] The RISC-V processor runs the Transformer neural network based on the built-in multipliers and floating-point registers.

[0012] As a further technical solution, the multi-stage adder of the multiplier adopts an addition tree structure, so that the number of adder bits used for adding two adjacent numbers is minimized.

[0013] As a further technical solution, the steps of constructing a floating point register include:

[0014] Extended register group: add floating point registers;

[0015] Modify the decoding process: add the decoding process of floating-point instructions;

[0016] Modify the dispatch module;

[0017] Modify the calculation processing module;

[0018] Added CSR register for floating point numbers;

[0019] The decoding module identifies the floating-point instruction and passes its characteristics to the dispatch module. The dispatch module dispatches the floating-point instruction to the floating-point calculation processing module according to the instruction type. The calculation processing module performs floating-point operations and updates the results and status information to the floating-point CSR register.

[0020] As a further technical solution, decoding and identifying floating-point instructions, the specific steps include:

[0021] Generate type information and operand register indexes for different instructions;

[0022] Determine the direction of each instruction, compare the corresponding part of the instruction with the specific value, if they are equal, pull the corresponding signal high, if not, keep it low;

[0023] After obtaining the corresponding judgment signal value, a specific instruction is selected;

[0024] After receiving the specific instruction, it is necessary to determine the specific operation implemented by the instruction;

[0025] Different floating-point instructions will point to different register groups;

[0026] Not all instructions in all floating-point instruction sets use floating-point registers. Some instructions move data between different registers, or access memory, read data addresses from integer general registers, and write data back to floating-point registers.

[0027] Different enable signals are used to indicate whether these structures and their specific directions are present in floating-point instructions.

[0028] As a further technical solution, it also includes: using different enable signals to indicate whether these structures exist in the floating-point instruction and the specific directions of rs1, rs2, rs3, and rd.

[0029] As a further technical solution, it also includes: assigning the decoded index value to the index of the corresponding register of the FPU;

[0030] After determining whether the floating-point instruction has rs1, rs2, rs3, rd and the corresponding index value, the register group type corresponding to the floating-point instruction is determined.

[0031] As a further technical solution, the register group type corresponding to the floating-point instruction is determined, and the specific steps are as follows:

[0032] The rs1 of floating-point instructions with rs1 parameters points to the floating-point register group except flw, fsw, and fmv.wx;

[0033] The rs2 of floating-point instructions with rs2 parameters all point to the floating-point register group;

[0034] The rs3 of floating-point instructions with rs3 parameters all point to the floating-point register group;

[0035] Except for fmv.xw and flt, the rd of floating-point instructions with rd parameter points to the floating-point register group.

[0036] As a further technical solution, after decoding is completed, the decoding result of the instruction and the operands read from the register group will be dispatched to different operation units for execution;

[0037] All instructions are transferred to the ALU for execution, and the decoding results of floating-point instructions are added to the input and output interfaces.

[0038] As a further technical solution, the decoding result of the floating-point instruction is added at the input and output interface, specifically:

[0039] First, the parameter rs3 is used for the first time in the floating-point instruction, and the index signal, judgment signal and specific value of rs3 are added to the input and output;

[0040] In addition, a discrimination signal should be added to the input and output to determine the type of register group pointed to. If it is pointing to a floating-point register group, the actual dispatch function is arranged to be executed in the ALU. The judgment is made through the grouping information generated by decoding, and the instruction is dispatched to the corresponding unit in the ALU to execute this instruction.

[0041] Adding the grouping information of the fonecycle module and the fdivsqrt module allows floating-point instructions to be sent to the correct unit for execution;

[0042] The remaining two non-operational long instructions, flw and fsw, are added to the same execution process of sw and lw, and are also added to the AGU information bus, dispatched to the AGU for processing, and further sent to the LSU for execution.

[0043] As a further technical solution, in the fonecycle module, the processing of all floating-point instructions is completed in one cycle, including floating-point multiplication;

[0044] In the fdivsqrt module, there are only two instructions, fdiv and fsqrt, which are multi-cycle instructions and require 24 clock cycles each time they are executed.

[0045] Second, a general Transformer acceleration system based on floating-point instruction set extension is disclosed, including:

[0046] The multiplier building module is configured to: build a multiplier, the multiplier includes a multi-stage adder connected in sequence, and a 4-Booth encoding algorithm is used to encode data; the encoded data is sequentially operated by the multi-stage adder to obtain the output of the multiplier;

[0047] The floating point register construction module is configured to: construct a floating point register, specifically, to: generate a floating point register group. The write port logic of the floating point register group corresponds the input result register index to the respective register number, generates a write enable signal, and the enabled general register writes the data into the register. The read port is implemented using a multi-way parallel selector, and the selection signal of the multi-way parallel selector is the index of the read operand register;

[0048] The accelerated operation module is configured to accelerate the operation of the Transformer neural network based on the constructed multipliers and floating-point registers.

[0049] One or more of the above technical solutions have the following beneficial effects:

[0050] The technical solution of the present invention improves the coremark score of the kernel by optimizing the multiplier of the RISC-V kernel. At the same time, the original kernel is expanded with floating-point instruction set operators and the related underlying software library for implementing the floating-point instruction set, so that the kernel can support the direct use of floating-point instructions.

[0051] The existing processors have room for optimization in the multiplier components. This solution optimizes the serial multiplier into a parallel multiplier. The original E203 core lacks the floating-point operation part, and there are a lot of floating-point operations in the operation of the neural network. This solution designs a floating-point instruction set to improve the operation speed of the neural network.

[0052] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0054] Figure 1 Schematic diagram of the five-stage pipeline structure;

[0055] Figure 2 This is the RISC-V architecture block diagram;

[0056] Figure 3 This is a schematic diagram of the structure of the Transformer-derived network BERT;

[0057] Figure 4 Schematic diagram of Transformer's multi-head attention mechanism;

[0058] Figure 5 Schematic diagram of parallel encoding of 32-bit multiplier rs1;

[0059] Figure 6 Schematic diagram of Booth 4 encoding process;

[0060] Figure 7 It is a schematic diagram of the multiplier structure;

[0061] Figure 8 Schematic diagram of the specific implementation process of the operation operator;

[0062] Fig. 9 Schematic diagram of floating point registers and general integer registers;

[0063] Fig.10 Comparison chart of refrigerator power model prediction value and standard value. DETAILED DESCRIPTION

[0064] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0065] It should be noted that the terms used herein are for describing specific embodiments only and are not intended to be limiting of exemplary embodiments according to the present invention.

[0066] In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0067] Glossary:

[0068] About RISC-V processors:

[0069] The RISC-V hardware structure also inherits the traditional MIPs five-stage pipeline. The life cycle of an instruction in the pipeline is divided into the following steps:

[0070] Instruction Fetch: This stage involves the process of fetching instructions from memory.

[0071] Decoding: In this stage, the instruction just fetched from memory is translated to determine the required operands and register indices, and the operands are retrieved from the general register set.

[0072] Execution: Once the instruction is decoded and the required operation type and operands are determined, the execution phase begins. Here, the actual operation of the instruction is performed, for example, an addition or subtraction operation is performed.

[0073] Memory Access: Memory access instructions are one of the core parts of the instruction set, which involves the process of reading data from the memory or writing data to the memory.

[0074] Writeback: The writeback stage records the output result of the instruction operation back to the general register group. For operation instructions, this result comes directly from the calculation in the execution stage; for memory read instructions, the result comes from the data extracted in the memory access stage.

[0075] like Figure 1As shown in the figure, after the previous instruction completes the "fetch" and enters the "decode" stage, the next instruction can immediately enter the "fetch" stage, and so on. When an instruction is delayed in execution, it can process other instructions in different stages in parallel, effectively hiding the execution delay and increasing the number of instructions that can be completed per unit time. At the same time, the five-stage pipeline can also improve the utilization of processor resources, provide the possibility of increasing the processor clock frequency, and improve the instruction pipeline structure to provide support for advanced features such as out-of-order execution and branch prediction.

[0076] Figure 2 The RISC-V pipeline structure shown, the processor pipeline used in this embodiment example is not a fixed number of stages in the traditional sense. Specifically, the instruction fetch stage (executed in the IFU) constitutes the first stage of the pipeline. Subsequently, in the same clock cycle, the second stage of the pipeline completes the decoding (processed by EXU), execution (also processed by EXU), and write back (WB is responsible for) operations of the instruction. Then, in the third stage of the pipeline after the EXU, it comes to the memory access stage (responsible for the LSU). However, it should be noted that the write back result of the LSU must also be returned to the general register group through the WB module. Therefore, the processor design of this embodiment example implements a variable-length pipeline architecture.

[0077] RISC-V instruction set:

[0078] Including: basic instructions and extended instructions, among which, according to the register bit width and address space, there are three different integer instruction sets (represented by I): 32, 64, and 128 bits. The integer instruction set includes arithmetic, logic, branch, memory access (access memory) instructions, etc., which can already implement a complete software stack. If some CPUs have more functional requirements, extended instructions can be assembled on the basis of basic instructions. The extended instructions of the RISC-V architecture are rich and varied, covering a variety of computing needs. These extended instructions mainly include but are not limited to: multiplication and division operations, modulus and remainder operations (M extension), single-precision floating-point operations (F extension), double-precision floating-point operations (D extension), quad-precision floating-point operations (Q extension), and atomic operations (A extension).

[0079] A notable feature of the RISC-V instruction set is its modular design, which means that users can start from the basic instruction set and selectively add the required extended instruction set to build a customized instruction set architecture based on specific application requirements. Based on such an architecture, a CPU that meets specific performance and functional requirements can be designed. This flexibility and scalability is one of the important reasons why RISC-V is popular, as it allows hardware designers to optimize the performance and power consumption of the chip based on the characteristics of the target application.

[0080] This implementation example mainly uses the basic instruction set RV32I and the floating-point extension instruction set RV32F. The basic instruction set (RV32I) is the simplest instruction set of RISC-V, which includes the following types of instructions:

[0081] Data transfer instructions: used to transfer data between registers and between registers and memory.

[0082] Arithmetic logic instructions: including addition, subtraction, multiplication, division and bit operations.

[0083] Control flow instructions: including branch, jump, return, etc.

[0084] System call instructions: used to execute operating system or other system-level functions.

[0085] The RISC-V floating-point extension instruction set RV32F is a new instruction set added to the RISC-V basic instruction set to provide assistance in scenarios where floating-point operations are required. It is a RISC-V variant that adds single-precision floating-point operation support based on RV32I. In addition to the instruction types contained in the above RV32I, RV32F also adds floating-point instructions including single-precision floating-point (32-bit) arithmetic instructions. These instructions follow the IEEE 754 single-precision floating-point standard. The design of RV32F allows processor implementers to choose whether to include a floating-point instruction set based on application requirements. This enables the RISC-V processor to be optimized for different application scenarios while maintaining the simplicity of hardware and software.

[0086] RV32F processors can support integer and floating-point operations on the same hardware platform, which provides flexibility for developers. For example, an RV32F processor for general-purpose computing may include a full floating-point instruction set, while a processor for specific applications, such as graphics processing or digital signal processing, may selectively implement a set of floating-point instructions based on demand.

[0087] Non-Intrusive Load Identification (NILM):

[0088] Non-intrusive Load Monitoring (NILM) refers to installing information collection sensors at the entrance of a household, analyzing the total load data collected, and using relevant decomposition algorithms to identify and decompose the load, ultimately obtaining detailed power consumption information of each electrical device in the user's home to achieve the purpose of load monitoring. Studies have shown that by providing residents with feedback on the specific energy consumption information of household electrical devices, it can help them improve their electricity usage habits and save up to 20% of electricity costs, which has a wide range of application scenarios.

[0089] Transformer Neural Network:

[0090] The core idea of ​​the Transformer network is to process traditional sequence-to-sequence data through the self-attention mechanism. Different from traditional convolutional neural networks (CNN), recurrent neural networks (RNN) and long short-term memory networks (LSTM), Transformer can capture long-distance dependencies in sequences. By assigning an attention weight to each token in the sequence, the model can dynamically focus on different parts of the input sequence to better understand the meaning of the entire sequence. At the same time, Transformer uses parallel computing to greatly improve computing efficiency.

[0091] like Figure 3 As shown in the figure, the derivative network of Transformer, BERT, consists of multiple modules that work together to model long time series. The entire network consists of a convolutional layer (conv), a pooling layer (pool), a position vector encoding layer (positionembedding), a normalization layer (layernorm), a linear layer (linear), a deconvolution layer (deconv), an activation layer (tanh or RELU), a feedforward neural network (feedforward), and a multi-attention mechanism layer (multi-attention).

[0092] The convolution layer is the most basic part of the convolutional neural network (CNN). The function of the convolution layer is to extract features from the input data, and use a set of filters (convolution kernels) to perform convolution operations with the input matrix to obtain the output feature map. These convolution kernels are obtained through neural network training.

[0093] The pooling layer plays an important role in convolutional neural networks. It downsamples the feature maps output by the convolutional layer to reduce the spatial dimension of the data. The pooling layer can extract key features while maintaining the forward propagation efficiency of the model.

[0094] After pooling, the pooled result needs to be added to the position encoding information, that is,

[0095] x out =x in +position_embedding

[0096] The position encoding information position_embedding is determined by the following formula:

[0097]

[0098] Then the data containing the position information is placed in the normalization layer. The normalization layer can convert the input into data with a mean of 0 and a variance of 1. The normalization formula is as follows:

[0099]

[0100] Among them: w and b are the weight matrix and bias matrix obtained by training, mean is the mean of the input matrix, std is the standard deviation of the input matrix, and eps is the minimum value set to prevent the denominator from being zero when the standard deviation is zero.

[0101] The core of the Transformer model is the multi-head attention mechanism, and its operation diagram is as follows Figure 4 As shown, the calculation formula is:

[0102]

[0103] Where: Q is the query vector (Query), K is the key vector (Keys), V is the value vector (Values), d k is the scaling factor. Transformer can capture the complex dependencies between different positions in the input sequence, thus showing excellent performance when processing sequence data. This embodiment uses a multi-head attention mechanism with 2 heads, which divides the input matrix into two groups of data and performs attention mechanism operations respectively, and then splices the operation results and outputs them.

[0104] Embodiment 1

[0105] This embodiment discloses a processor acceleration method for running a Transformer neural network, including:

[0106] Since the main program of the neural network is matrix operation, in order to accelerate the operation speed of the neural network, the core optimization strategy is to optimize the kernel digital matrix operation:

[0107] Step 1: Modify the original iterative adder into an addition tree structure based on the Booth-4 coding algorithm. First, use the radix 4-Booth coding algorithm to calculate the partial products. Then use the iterative method to accumulate the partial products using the adder in each cycle. After multiple cycles of iteration, the final product is obtained.

[0108] In this step, the multiplier adopts a parallel addition tree structure, and the specific steps are:

[0109] (1) The radix 4-Booth encoding algorithm is used, except that the original one-cycle encoding is changed to parallel encoding of the 32-bit multiplier rs1;

[0110] At the same time, the sign bit of the 32-bit multiplier rs2 is extended. After the unsigned multiplication is extended, it can be uniformly treated as a signed number for operation; the maximum product of the base 4-Booth encoding result is x2, so the rs2 result of x1 and x2 is obtained to facilitate the subsequent operation, see the attached Figure 5 shown.

[0111] (2) According to the radix 4-Booth encoding booth_code0-booth_code15, the corresponding 16 33-bit rs2 results add_tree_op0-add_tree_op15 are obtained; the Booth 4 encoding process is shown in the attached Figure 6 shown.

[0112] (3) The final addition array to be implemented is obtained. Taking into account the resource consumption and operation speed of the adder, it is expected to use an adder with fewer bits to implement the adder based on 4-Booth encoding. Therefore, an addition tree structure is used to minimize the number of adders used to add two adjacent numbers. The calculation formulas for each level of adder are as follows.

[0113] First level: add_tree_fir_(i)=add_tree_op(2*i)+add_tree_op(2*i+1), i=0,1,2,3,4,5,6,7.

[0114] Second level: add_tree_sec_(i)=add_tree_fir_(2*i)+add_tree_fir_(2*i+1), i=0,1,2,3.

[0115] The third level: add_tree_thi_(i)=add_tree_sec_(2*i)+add_tree_sec_(2*i+1), i=0,1.

[0116] The fourth level: add_tree_fou_(i)=add_tree_thi_(2*i)+add_tree_thi_(2*i+1), i=0.

[0117] The schematic diagram of the multiplier structure is as follows Figure 7 As shown. Below, the four adders used are explained in more detail:

[0118] The first stage 34-bit adder needs to complete the addition operation:

[0119] It is only necessary to add the first 31 bits of add_tree_op(2*i) to add_tree_op(2*i+1), and the lowest 2 bits of add_tree_op(2*i) directly enter the next level adder. For signed multiplication, the highest 2 bits S of add_tree_op(2*i) need to be extended according to the sign bit (bit 32), and for unsigned multiplication, the highest 2 bits S can be set to 0.

[0120] The second-stage 36-bit adder needs to complete the following addition operations:

[0121] Just add the first 32 bits of add_tree_fir_(2*i) to add_tree_fir_(2*i+1), and the lowest 4 bits of add_tree_fir_(2*i) directly enter the next level adder. For signed multiplication, the highest 4 bits S of add_tree_fir(2*i) need to be extended by the sign bit (bit 35), while for unsigned multiplication, the highest 4 bits S can be set to 0. The difference from the first level is that after the maximum value derivation, the second level adder cannot generate a carry, so the adder here chooses a 36-bit adder instead of a 37-bit adder.

[0122] The third-level 40-bit adder needs to complete the following addition operations:

[0123] The first 32 bits of add_tree_sec_(2*i) can be added to add_tree_sec_(2*i+1), and the lowest 8 bits of add_tree_sec_(2*i) directly enter the next level adder. For signed multiplication, the highest 8 bits S of add_tree_sec(2*i) need to be extended by the sign bit (bit 39), while for unsigned multiplication, the highest 8 bits S can be set to 0. Similarly, after the maximum value derivation, the third-level adder cannot generate a carry, so the adder here chooses a 40-bit adder instead of a 41-bit adder.

[0124] The fourth-level 48-bit adder needs to complete the following addition operations:

[0125] The first 32 bits of add_tree_thi_(2*i) can be added to add_tree_thi_(2*i+1), and the lowest 16 bits of add_tree_thi_(2*i) directly enter the next level adder. For signed multiplication, the highest 16 bits S of add_tree_thi_(2*i) need to be extended by the sign bit (bit 47), while for unsigned multiplication, the highest 16 bits S can be set to 0. Similarly, after the maximum value derivation, the adder of the fourth level cannot generate a carry, so the adder here chooses a 48-bit adder instead of a 49-bit adder.

[0126] The above-mentioned radix 4-Booth encoding reduces the required addition and shift operations by grouping the processed data. The 32-bit operands are sign-extended, and the unsigned numbers can be uniformly operated as signed numbers after the sign bit extension. The operands are then grouped into three bits and encoded in parallel, thereby increasing the operating speed of the core adder by increasing the chip area.

[0127] The input to the multiplier usually consists of two parts: Multiplicand: This is the number that needs to be multiplied. Multiplicand: This is the number to be multiplied by the multiplicand. In Booth's algorithm, the binary representation of the multiplier is usually of interest, especially the consecutive zeros.

[0128] The output is: The output is the result of the multiplication, that is, the product of the multiplicand and the multiplier. In the addition tree structure of the Booth coding algorithm, the output is the accumulation of the partial products, which is the final multiplication result.

[0129] During the calculation process, the output of the addition tree is usually sent to the accumulator for subsequent operations, such as more addition operations or other arithmetic operations.

[0130] The data processed are partial products. In Booth's algorithm, the adjacent bits of the multipliers are compared to determine whether to perform addition, subtraction, or no operation, and then generate partial products. These partial products are then accumulated in an addition tree structure to obtain the final multiplication result.

[0131] Specifically, the Booth-4 algorithm processes data in the following steps:

[0132] Each consecutive four bits of the multiplier are encoded to generate the corresponding operation code (addition, subtraction, or no operation).

[0133] Generates partial products according to the opcode.

[0134] The partial products are accumulated through an addition tree structure.

[0135] Finally, we get the multiplication result.

[0136] Step 2: Expand the floating-point instruction set: See the attached Figure 8 shown.

[0137] RV32D and RV32F are floating-point extension instructions in the RISC-V instruction set architecture, which are used to support floating-point operations. Among them, RV32D is a double-precision floating-point extension instruction, and RV32F is a single-precision floating-point extension instruction. In this implementation example, a single-precision floating-point instruction set is added, hoping to reduce the number of instructions compiled by the compiler by increasing the floating-point operation operator, thereby improving the operating efficiency of the processor.

[0138] After the floating-point extension instruction is added to the Hummingbird e203 RISC-V core, a single fmul.s instruction can be used to replace the complicated operations. Compared with software simulation of floating-point operations, the use of integer instructions to simulate floating-point operations greatly reduces the running time of processing multiplication operations.

[0139] 2-1) Extend the register set exu_fregfile and add floating point registers.

[0140] When using the RV32F instruction set extension, specific floating-point registers are required. The number of floating-point registers is also limited, and the number of registers required needs to be determined based on the specific application scenario. In addition, the floating-point register f0 is no longer a constant 0 but a variable register like other registers. First, add the floating-point register group fregfile in the same way as regfile, and use the generate syntax to generate the floating-point register group. The write port logic of the floating-point register group module matches the input result register index with the respective register number, generates a write enable signal, and enables the general register to write the data into the register. The read port is implemented using a multi-row parallel selector, and the selection signal of the multiplexer is the index of the read operand register.

[0141] Compared with the integer general register set, the floating-point register set has the following differences:

[0142] a. The floating-point register group X0 can be read and written, while the integer general register group x0 is set to 0 by hardware and cannot be written. Therefore, when generating a floating-point register group, the part of the general register that forces x0 to be 0 must be deleted.

[0143] b. From the structure diagram ( Fig. 9 ) It can also be seen that the input and output of the integer general register group and the floating-point register group are not the same. In floating-point instructions, there are instructions such as fmadd that require the rs3 parameter, while for integer instructions, the rs3 parameter is not used. Therefore, the input and output of the floating-point register group should also be changed accordingly, and an rs3 input and output channel must be added. The relevant structure is as follows Fig. 9 shown.

[0144] 2-2) Modify the decoding link exu_decode and add the decoding process of floating-point instructions.

[0145] The original code file exu_decode.v only contains the decoding process of the integer instruction set and part of the 16-bit reduced instruction set, and the decoding process of the floating-point instruction set is missing.

[0146] The decoding module is composed entirely of combinational logic, and its main logic decodes according to the instruction encoding rules of the RISC-V architecture, generating type information of different instructions, operand register index, etc.

[0147] RISC-V's instruction composition has the advantage of modularity. The splicing of different modules can represent a new instruction. The direction of each instruction is determined based on the three parts of opcode, func3 and funct7. The corresponding part of the instruction is compared with the specific value. If they are equal, the corresponding signal is pulled high, and if they are not equal, the signal is kept low.

[0148] After getting the corresponding judgment signal value, these three signals can be used to select specific instructions. Take the floating point instruction fsgnj.s as an example. To correctly translate the instruction, you need to observe the three signals rv32_op_fp, rv32_func7_0010000 and rv32_func3_000 at the same time. Only when the three signals are high at the same time can we say that the instruction is fsgnj.s. The lack of any signal may cause confusion with other instructions. For example, fsgnj.s and fsgnjn.s are different only in rv32_func3, and the remaining two signals cannot be used to distinguish them.

[0149] After getting the specific instruction, you need to determine the specific operation implemented by the instruction, such as the direction of rs1, rs2, rs3, rd, or whether the ALU needs to be called, etc. Use the FPU_op signal to define the instruction that needs to be calculated as an op instruction, which means that the added FPU processing module is needed here.

[0150] In addition, different floating-point instructions will point to different register groups. Not all instructions in all floating-point instruction sets use floating-point register groups. Some will implement data movement between different register groups (such as fmv.wx and fmv.xw instructions), or access memory, read data addresses from integer general register groups, and finally write data back to floating-point register groups (such as fsw instructions). Therefore, different enable signals are needed here to indicate whether these structures exist in floating-point instructions and the specific directions of rs1, rs2, rs3, and rd.

[0151] In the floating-point instruction set (including the reduced instruction set), rs1 is used except for flwsp and fswsp.

[0152] In the floating-point instruction set (including the reduced instruction set), rs2 is used except for flw, fmv.xw, fmv.wx, fsqrt, and flwsp.

[0153] In the floating-point instruction set (including the reduced instruction set), only fmadd and fnmadd use rs3.

[0154] In the floating-point instruction set (including the reduced instruction set), rd is used except for fsw and fswsp.

[0155] For the index of floating-point registers, the register index position of floating-point instructions is the same as other instructions. Therefore, the decoded index value can be directly assigned to the index of the corresponding register of FPU. After determining whether the floating-point instruction has rs1, rs2, rs3, rd and the corresponding index value, it is necessary to determine the register group type corresponding to the floating-point instruction, whether it is an integer general register group or a floating-point register group.

[0156] Except for flw, fsw, and fmv.wx, rs1 of all floating-point instructions (including reduced instruction sets) with rs1 parameters points to the floating-point register group.

[0157] rs2 of floating-point instructions (including reduced instruction sets) with rs2 parameters all point to the floating-point register group.

[0158] rs3 of floating-point instructions (including reduced instruction sets) with rs3 parameters all point to the floating-point register group.

[0159] Except for fmv.xw and flt, the rd of floating-point instructions (including reduced instruction set) with rd parameter points to the floating-point register group.

[0160] Multiplex it to the corresponding rv32_need_rs1, rv32_need_rs3, rv32_need_rs3, rv32_need_rd signals and output them to the corresponding register group.

[0161] After the decoding process is completed, the information bus required by the FPU needs to be generated. The function of this information bus is to transmit the decoding information to the implanted FPU module, so that the FPU module can understand the operations required for the currently executed instruction based on the bus content. The bus information is selected and output by dec_info. Through a classic parallel multiplexer, different bus signals are multiplexed into the output signal (dec_info), and the selection signal is the op signal mentioned earlier, which represents whether the module in the ALU is to be used, and it can be used to group the signals well. The decoding part is over. Due to the excellent structured characteristics of RISC-V instructions, the immediate decoding part of the instruction does not need to be changed significantly, and can be kept as it is.

[0162] 2-3) Modify the dispatch module exu_disp.

[0163] After decoding, the decoding result of the instruction and the operands read from the register group will be dispatched to different operation units for execution. Before any floating-point instructions are added, all instructions will be dispatched to the ALU and delivered through the interface between the ALU and the delivery module. If the instruction is a long instruction, it will be further sent by the ALU to the corresponding long instruction operation unit: for example, the Load and Store instructions belonging to long instructions will be further dispatched to the LSU for execution through the ALU sub-module AGU.

[0164] Since all instructions need to go through the ALU, the actual dispatch function occurs inside the ALU. exu_decode has grouped the instructions according to the operation units that execute the instructions during the decoding process, and gave the corresponding indication signal (op signal). Therefore, the exu_disp module is used to pass signals to the ALU and transmit the information of long instruction signals to oitf. Continuing the design concept of the Hummingbird e203, all instructions are transferred to the ALU for execution, so here we only need to add the decoding results of floating-point instructions at the input and output interface.

[0165] Since the parameter rs3 is used for the first time in the floating-point instruction, the index signal, judgment signal and specific value of rs3 must be added to the input and output.

[0166] In addition, a discrimination signal should be added to the input and output to determine the type of register group pointed to, if it is a floating-point register group. The actual dispatch function is also arranged to be executed in the ALU. The judgment is made through the grouping information generated by the decoding module, and the instruction is dispatched to the corresponding unit in the ALU to execute this instruction. Since the fonecycle module and fdivsqrt module are newly added, their grouping information needs to be added so that the floating-point instruction can be sent to the correct unit for execution.

[0167] The remaining two non-operational long instructions, flw and fsw, are added to the same execution process of sw and lw, and are also added to the AGU information bus, dispatched to the AGU for processing, and further sent to the LSU for execution. The LSU is the main control unit of the memory subsystem in the processor, used to access different memory units such as DTCM and ITCM.

[0168] 2-4) Modify the calculation processing module exu_alu.

[0169] In order to realize the functions of floating-point instruction operation, the fonecycle module and fdivsqrt module are added on the basis of the original ALU architecture. These modules are integrated based on the open source floating-point operation code HardFloat of the University of Berkeley, and can realize the processing of all floating-point instructions. The idea of ​​dividing these instructions into two parts is to refer to the division of single-cycle instructions and multi-cycle instructions in the Hummingbird e203 RISC-V core. In the ALU, integer multiplication and division are distinguished because they use multi-cycle shift multipliers and the instructions cannot be completed in one cycle. Correspondingly, in the fonecycle module, the processing of all floating-point instructions can be completed in one cycle, including floating-point multiplication (fmul). In the fdivsqrt module, there are only two instructions, fdiv and fsqrt, which are multi-cycle instructions and require 24 clock cycles to execute each time.

[0170] This embodiment takes fonecycle as an example to introduce the specific process of floating-point instruction processing.

[0171] In addition to the calculation input and output values ​​of frs1, frs2, frs3, and farithematic_res, the exception_flags signal outputs the exception flag. Different exception flags indicate possible exceptions that may occur during floating-point calculations. This flag will be written into the fcsr register along with the output of fonecycle. What is the fcsr register will be explained in the following report. As a five-bit output, exception_flags represents five different exceptions in floating-point calculations - illegal operation, division by 0, overflow, underflow, and inaccuracy. When different exceptions occur, the corresponding bit will be set high. When both underflow and inaccuracy occur at the same time, the value of exception_flags is 00011.

[0172] roundingMode:

[0173] This input signal indicates the rounding mode of floating point numbers. According to the IEEE-754 standard, floating point operations require the specification of a rounding mode (running mode).

[0174] (1) Static rounding mode: There are 3 bits in the encoding of floating-point instructions as the rounding mode field. Different rounding mode encodings are shown in the figure. The RISC-V architecture supports five legal rounding modes. In addition, if the rounding mode is encoded as 101 or 110, it is an illegal mode; if the rounding mode is encoded as 111, it means that the dynamic rounding mode is used.

[0175] (2) Dynamic rounding mode: If the dynamic rounding mode is used, the rounding mode field in the fcsr register is used. The fcsr register contains the rounding mode field and only supports five legal rounding modes. If the rounding mode field in the fcsr register specifies an illegal rounding mode, subsequent floating-point instructions will generate an illegal instruction exception.

[0176] The value in fonecycle_i_info comes from FPU_info in decode. The fonecycle module decodes the data transmitted by the bus to obtain the specific floating-point instructions to be executed, selects the signal and assigns it to ftype, and uses ftype to control the specific processing operations performed by the FPU module.

[0177] After the FPU module is finally executed, the result will be returned to its top-level module ALU. wbck_o_wdat is its final write-back result.

[0178] fdivsqrt is a module integrated by Berkeley's floating-point arithmetic code. The two instructions fdiv and fsqrt require special processing because they require multiple cycles to complete. Otherwise, new instructions will be fetched, decoded, and executed while they are still being executed, resulting in conflicts and errors. In exu_decode, these two instructions are not classified into the overall FPU signal bus. Instead, like muldiv, a separate bus is opened and output to the independent module fdivsqrt in the ALU.

[0179] fdivsqrt_i_info: It is the bus information of fdivsqrt, which is parallel to FPU. Because it only implements the only two floating-point long instructions, faqrt and fdiv, the bus only needs one bit.

[0180] The instructions processed by fdivsqrt are all multi-cycle long instructions. During execution, you need to pay attention to the following issues: pipeline conflicts, long instructions, and OITF processing.

[0181] Pipeline conflicts include resource conflicts and data conflicts, both of which will cause pipeline blockage. Hummingbird e203 uses two methods to handle resource conflicts and data conflicts respectively.

[0182] Data conflict, as the name suggests, is a conflict caused by data relevance.

[0183] The Hummingbird e203 uses a clever method to handle data conflicts: all instructions are divided into two categories, data dependencies are divided into three categories, and they are handled by splicing long instructions and flushing the pipeline.

[0184] Resource conflicts usually occur when instructions are dispatched to different execution units for execution. When an instruction takes a long time to execute, and other instructions are dispatched to the same hardware module for processing, resource conflicts will occur. Subsequent instructions need to wait for the previous instruction to complete the operation and release the hardware module before they can be executed.

[0185] The interface of Hummingbird e203 adopts a rigorous valid-ready handshake interface. Once a resource conflict occurs in a module, it will output a ready=0 signal. Even if the other side has valid=1, the handshake cannot be completed. Therefore, the previous level module cannot allocate instructions and will enter a waiting state until ready=1.

[0186] Long instructions and OITF handling

[0187] Hummingbird E203 divides all instructions that need to be executed into two categories:

[0188] Single-cycle execution instructions: The delivery and write-back functions of the Hummingbird E203 are both in the second stage of the pipeline, and the single-cycle execution instructions complete the delivery and write-back at this stage, so it does not involve the conflicts mentioned above.

[0189] Multi-cycle execution instructions: usually require multiple clock cycles to complete execution and write back, so they are also called "late-delivered long pipeline instructions", or long instructions for short. In order to deliver long instructions after many clock cycles, data dependencies need to be detected first. The Hummingbird E203 uses a module called OITF (Outstanding Instructions Track FIFO long instruction tracking queue) to detect RAW and WAW dependencies related to long instructions. The reason why WAR dependencies are not detected is that the e203 is a micro-architecture that dispatches and writes back in order. The source operands have been read from the register group during dispatch, so the write-back regfile operation does not occur before reading the regfile source operands.

[0190] OITF is essentially a FIFO. At the dispatch point, each time a long instruction is dispatched, an entry is allocated in OITF, in which the source operand register index and result register index of the long instruction are stored. At the write-back point, each time a long instruction is written back in sequence, the entry of this instruction in OITF is removed and the FIFO is exited. OITF essentially saves the information of long instructions that have been dispatched but not yet written back.

[0191] When each instruction is dispatched, the source operand and destination operand register index of the instruction will be compared with each table entry in OITF to determine whether the instruction has RAW and WAW correlation with the long instruction that has been dispatched and not yet written back. If a correlation occurs, the dispatch of the current instruction is suspended. If there is no RAW and WAW correlation, and the instruction is a multi-cycle long instruction, write the instruction into OITF. If OITF is full, the dispatch will still be stopped and wait for OITF to release space before writing and dispatching. The added fsqrt and fdiv instructions are exactly the kind of instructions that require multiple cycles to execute, so they should be input as long instruction signals to oitf to detect their correlation and determine whether the pipeline needs to be blocked.

[0192] 2-5) Add CSR register for floating point numbers.

[0193] The RISC-V architecture stipulates that if single-precision floating-point instructions or double-precision floating-point instructions are supported, a floating-point control status register (fcsr) needs to be added, such as Fig. 9 It is a 32-bit read / write register used to select dynamic rounding mode for floating-point arithmetic operations and to store exception flags.

[0194] The fcsr register consists of user reserved bits, frm, and fflags. Although the RISC-V architecture only defines one floating-point control register (fcsr), the different fields frm and fflags of the register and the register itself fcsr are assigned independent CSR addresses.

[0195] For example, through address 0x002, you can read and write the frm field of the fcsr register separately, and through address 0x001, you can read and write fflags. However, it is worth noting that these three read and write methods should not affect each other. When writing data to fflags alone, be careful not to overwrite other data bits. The data in frm may be used in the next instruction, so these three write modes cannot be simply reused. You must set the selection signal to use different write modes. The sel_fcsr, sel_frm, and sel_fflags signals are set to determine the specific write mode, keep the original value that does not need to be overwritten, and write the new value.

[0196] The reading of the CSR register is also done by a multiplexer to select the specific output result. The selection signal, such as rd_mtvec, is obtained by judging the access address - sel_mtvec = (csr_idx == 12'h305); the value in the mtvec register will be assigned to the read_csr_dat output.

[0197] The purpose of fcsr in floating-point instructions is to write back the exception flags in floating-point operations. There are special instructions to read and write ordinary CSR registers, but in addition to using special CSR instructions, the writing of fflags should also be automatically written back after each FPU performs an operation as part of the operation result. Add the exception flag data exception_flags of fonecycle to the write data signal wbck_csr_dat of CSR.

[0198] Use the fonecycle identification signal to pull the CSR write enable signal high, so that the CSR register can be read and written by fonecycle when there is no CSR instruction.

[0199] 2-6) Modification of misa register.

[0200] The MISA register is used to indicate the architectural features supported by the current processor.

[0201] The highest two bits of the MISA register are used to indicate the number of architecture bits supported by the current processor.

[0202] If the highest two bits are 1, it indicates that the current architecture is 32-bit (RV32). If the highest two bits are 2, it indicates that the current architecture is 64-bit (RV64). If the highest two bits are 3, it indicates that the current architecture is 128-bit (RV128).

[0203] The lower 26 bits of the misa register are used to indicate the different modular instruction subsets in the RISC-V ISA supported by the current processor. The misa register is defined as a readable and writable register in the RISC-V architecture document, allowing certain processor designs to dynamically configure certain features. However, in the implementation of the Hummingbird e203, the misa register is a read-only register. Constantly reflects the ISA modular subsets supported by different models of processor cores. For example, if the Hummingbird e203 core supports RV32IMAC, it is reflected in this register, the highest two bits are 1, and the value of the corresponding domain of I / M / A / C in the lower 26 bits is high.

[0204] After adding the floating-point instruction, you need to pull the bit of the misa register corresponding to the floating-point instruction high, otherwise the debugger will report an error, indicating that the current processor does not support the use of the F instruction set. Pull the bit corresponding to the F instruction set high, and be careful not to confuse it with the double-precision floating-point instruction set.

[0205] Decoder module: The decoder module is responsible for translating the machine code of the instruction into control signals and operands.

[0206] Dispatcher: The dispatch module sends instructions to the corresponding execution unit according to the decoded instruction type.

[0207] Execution Unit: The execution unit is responsible for performing specific arithmetic or logical operations.

[0208] Floating-Point Control and Status Register (fcsr): The floating-point CSR register is used to store the status and control information of floating-point operations.

[0209] The logical relationship is: the decoding module recognizes the floating-point instruction and passes its characteristics to the dispatch module. The dispatch module dispatches the floating-point instruction to the floating-point calculation processing module according to the instruction type. The calculation processing module performs floating-point operations and updates the results and status information to the floating-point CSR register.

[0210] Relationship with acceleration: Decoding module: Acceleration is reflected in faster recognition and decoding of floating-point instructions. Dispatch module: Acceleration is reflected in more efficient instruction routing and reduced instruction execution latency. Computation processing module: This is the core of acceleration, which significantly improves the performance of floating-point operations through hardware acceleration of floating-point operations. Floating-point CSR register: Acceleration is reflected in fast access and update of floating-point operation status information.

[0211] Decoding module: Input is instructions in machine code form. Dispatching module: Input is decoded instruction features and control signals. Calculation processing module: Input is operands and operation codes. Floating point CSR register: Input is floating point operation status information.

[0212] Processing process: Decoding module: parses instructions, extracts opcodes and operands, and generates control signals. Dispatching module: routes instructions to corresponding execution units according to instruction types. Computation processing module: performs floating-point operations such as addition, multiplication, and division. Floating-point CSR register: stores and updates status information such as exceptions and rounding modes of floating-point operations.

[0213] The outputs are: Decoding module: outputs control signals and operands. Dispatch module: outputs instruction dispatch information, instructing the execution unit to start execution. Calculation processing module: outputs calculation results and possible status information (such as overflow, underflow, etc.). Floating point CSR register: outputs the status information of the current floating point operation for use by subsequent instructions.

[0214] Step 3: Neural Network Training Optimization

[0215] In order to make the neural network run faster on the processor core after the adder is optimized, the neural network is optimized to a certain extent and the original BERT neural network structure is modified. In the original network model structure, the position encoding is obtained through self-learning, that is, the model learns the position information related to the sequence by itself during the training process. Since the application scenario is non-intrusive load identification, the input data of the neural network are all the user's electricity consumption, and the user's electricity consumption is presented in chronological order. The data itself contains its own position information, so the original self-learning position encoding method is modified to a fixed sine-cosine encoding method, which can reduce the computational complexity and memory requirements of the model. At the same time, since the sine-cosine encoding is periodic, it means that it can be better generalized to sequence lengths that have not been seen during training. For the BERT model, this means that when the model processes data that exceeds the length of the longest sequence during training, it can use the position information more effectively.

[0216] For BERT neural network models (such as Figure 3 As shown in the figure, first use the nn.Conv1d() function of the pytorch library to perform one-dimensional convolution on the input. The kernel size used in this convolution process is 5, and the padding is 2. The convolution process can expand the input 32×480 (note that this neural network is trained in groups of 32 at the same time, and the advantage of the graphics card's ability to perform tensor calculations can be used to speed up training. The input used in deployment is 1x480) matrix data to a 32×480×256 three-dimensional matrix, where the weight data inside the convolution kernel is obtained through neural network training. Repeat this operation 256 times for the one-dimensional convolution process of the 1×480 array to get the convolution output result.

[0217] The next step is to use the nn.LPPool1d() function provided by PyTorch to perform a one-dimensional pooling operation on the result after convolution, effectively reducing the size of the parameter matrix, thereby reducing the number of parameters in the connection layer. After the pooling operation, the original 32×480×256 can be reduced to a three-dimensional matrix of size 32×240×256.

[0218] The next step is to add the pooled result to the positional encoding information, i.e.

[0219] x out =x in +position_embedding

[0220] The position encoding information position_embedding is determined by the following formula:

[0221]

[0222] Then the data containing the location information is placed in the normalization layer, which can convert the input into data with a mean of 0 and a variance of 1, accelerating the convergence of the model. The data normalization formula and code are as follows:

[0223]

[0224] Among them: w and b are the weight matrix and bias matrix obtained by training, mean is the mean of the input matrix, std is the standard deviation of the input matrix, and eps is the minimum value set to prevent the denominator from being zero when the standard deviation is zero.

[0225] After the normalization layer, the Transformer module is entered. The Transformer module consists of two repeated encoders: a multi-head attention mechanism module and two fully connected networks. The operation of the multi-head attention mechanism is shown below: Figure 4 shown.

[0226] Since a multi-head attention mechanism with 2 heads (Head=2) is used, the input matrix passed into the Transformer module is divided into two groups of data for separate attention operations, and the operation results are concatenated and output.

[0227] The first fully connected layer performs a linear operation on the result of the multi-head attention mechanism and the matrix of the initial input Transformer module, while the second fully connected layer performs a linear operation on the output of the previous fully connected layer and the position fully connected feedforward network, which includes two linear transformations and a nonlinear activation process (using the RELU nonlinear activation function).

[0228] In order to improve the accuracy of the model, two serial Transformer modules are used continuously. After passing through the two Transformer modules, the data enters the deconvolution step. Deconvolution uses the nn.ConvTranspose1d() function that comes with pytorch, which is the inverse operation of the convolution operation nn.Conv1d() function.

[0229] After the deconvolution step is completed, the final output result of the neural network can be obtained through two linear layers and a nonlinear activation layer tanh(). Their weight matrix and bias matrix are obtained by neural network training.

[0230] After modifying the BERT network structure, the network model was retrained and tested, and the power model prediction value of the refrigerator was compared with the standard value. Fig.10 As shown, orange is the predicted value and blue is the standard value. It can be found that the predicted value of refrigerator power is close to the standard value, indicating that the network has a good reasoning effect.

[0231] Functional simulation test:

[0232] Test whether the muldiv module under ALU functions normally, and use the open source risc-v-test instruction set self-test set to verify the function, mainly verifying whether the mul instruction, mulh instruction, mulhu instruction, mulhsu instruction, div instruction, and divu instruction in the open instruction set architecture (ISA) function normally. Run the test files for the above instructions respectively, and the test results show that the above instructions all pass.

[0233] Coremark's performance indicator is usually expressed as Coremark / MHz, which refers to how many times a processor can run the Coremark program per second at a specific clock frequency (MHz). The higher this number, the stronger the processor's performance. Coremark / MHz provides a unified measurement standard that allows direct performance comparisons between processors of different architectures and frequencies.

[0234] After the adder function execution process is tested and confirmed, the optimization effect of the adder is tested and observed. The open source SDK tool chain is used to generate the coremark.verilog file to simulate the optimization effect to obtain the optimized performance parameters. Similar to the ISA simulation steps, the generated coremark test script is executed, and the final running score is 3.135CoreMark / MHz, which is about 47.8% higher than the original project's benchmark score of 2.12CoreMark / MHz.

[0235] FPGA platform verification: After the data is output from the base 4-Booth encoding, it needs to pass through a four-stage adder. Its critical path is at least: 34-bit adder-36-bit adder-40-bit adder-48-bit adder. The combinational logic circuit is huge. The RISC-V core used works at a clock frequency of 16MHz. It is necessary to ensure that the circuit can meet the timing constraints. Therefore, the FPGA platform is used for verification, and the consumption of resources after optimization can be compared. The FPGA verification platform uses XC7A100T-2FGG484 (Zhengdian Atom Da Vinci Development Board).

[0236] Since the RISC-V core used this time requires two external crystal oscillators, namely the 32.768kHz required by the normally-on domain and the 16MHz clock required by the main domain, and since the development board has only one clock, the 16MHz clock needs to be divided. The original RISC-V core code and the optimized RISC-V core code are synthesized separately to observe and compare the resource consumption, as shown in Table 1.

[0237] Table 1 Hardware resource consumption

[0238] parameter Before optimization After optimization LUT 11274 12561 LUTRAM 16 16 FF 11243 11243 BRAM 32 32 IO 80 80 BUFG 4 4 MMCM 1 1 WNS 24.864ns 24.454ns WHS 0.36ns 0.029ns WPWS 7.0ns 7.0ns Total On-Chip Power 0.23W 0.23W

[0239] It can be seen that in terms of resources, only the usage of LUTs increased from 11274 to 12561, an increase of 11.4%. Both timing margins decreased slightly, with WNS decreasing from 24.864ns to 24.454ns and WHS decreasing from 0.36ns to 0.029ns, but they are all positive values ​​and still meet the timing requirements.

[0240] Generate bitstream files .bit and .bin files, burn them into the development board, use the development board to execute the coremark test program, and send the output coremark running results back to the host computer through the serial port. The CPU main frequency is 16MHz, run 500 rounds of coremark program, and the final score is 3.135CoreMark / MHz, and the FPGA verification is correct.

[0241] Similarly, the Dhrystone and Whetstone programs were burned into the board for running: the Dhrystone running score increased from 1.28DMIPS / MHz to 1.43DMIPS / MHz, an increase of 11.7%; the Whetstone running score increased from 0.04MWIPS / MHz to 0.051MWIPS / MHz, an increase of 27.5%.

[0242] The optimization of the RISC-V core multiplier enables it to better perform the forward reasoning of the neural network, and improves the coremark to 3.135 coremark / MHz within the acceptable range of chip resource overhead. This optimization has been proven to have a good effect in the host computer simulation and FPGA implementation. It also prepares for the subsequent floating-point instruction set expansion and neural network optimization and operation. The test results are summarized in Table 2.

[0243] Table 2 Summary of test results

[0244]

[0245]

[0246] Floating point instruction set extension test

[0247] Feasibility test

[0248] Use open source self-test cases (Self-Check Testcase) to detect whether the modified code runs successfully or fails. riscv-test is an open source project maintained by RISC-V architecture developers, which contains some test programs to test whether the processor complies with the instruction set architecture definition. This type of assembly test program uses certain macro definitions to organize program points to test the instructions defined in the instruction set architecture. If the execution result of the instruction is equal to the expected result, the program continues to execute; if it is not equal to the expected result, the program directly uses the jump instruction to jump to the TEST FAIL address; assuming that all test points pass, the program will continue to execute to the TEST PASS address. Next, the feasibility test of the relevant instruction set is performed.

[0249] Floating point acceleration test results:

[0250] The execution time and efficiency of the RISC-V core with software emulation of floating-point instructions and the hardware floating-point execution after adding the floating-point instruction set were tested and compared. Various floating-point instructions were executed in the two cores respectively, and the number of cycles required to execute the corresponding instructions was observed and compared. The time saving comparison of software and hardware floating-point operations is shown in Table 3.

[0251] Table 3 Comparison of time saved by software and hardware floating-point operations

[0252] instruction Software floating point Hardware floating point Save time per operation fmul 145 32 113 fdiv 224 58 166 fadd 120 32 88 fsub 149 32 117 fmadd 225 35 190 fnmadd 230 35 195 fsqrt 1224 56 1186

[0253] Except for the fsqrt instruction which takes longer cycles to run, the running efficiency of other instructions has been greatly improved. The same computing power can save 75%-96% of the running time.

[0254] It is worth noting that sqrt is simulated with integer instructions in both the floating point gcc toolchain and the integer gcc toolchain. If you want to save time further, you need to embed assembly in the C code and force the square root operation to be executed using fsqr.s.

[0255] Neural network operation and testing

[0256] Since the neural network needs to be deployed in the RISC-V core, the neural network model needs to be written in C language to build the Transformer module and the entire neural network. The C language code is burned into the Hummingbird E203 processor, runs in the RISC-V core, and the processing results are sent back to the host computer through the serial port for display, which is completely consistent with the standard results. At this point, the function verification has passed.

[0257] Performance Verification:

[0258] The comprehensive report contains four codes, namely: ① original project ② project after score optimization ③ project after floating-point instruction extension ④ project after score optimization and floating-point instruction extension. The following is a list of some of the comprehensive results as shown in Table 5.

[0259] Table 5 Comparison of resource consumption

[0260]

[0261] Although the timing margin of the final project further decreased, it was still able to run stably at a clock frequency of 16MHz.

[0262] In order to compare the acceleration effect of hardware floating-point operations compared with software simulated floating-point operations, the comparison is made by burning after compiling CORE=e205f (hardware floating-point instructions) and burning after compiling CORE=e203 (software simulated floating-point instructions). In order to make the running time more intuitive, the code for statistical operation cycles and time is added to the C program. Finally, the running time and number of cycles before optimization (software simulated floating-point operations), after optimization (hardware floating-point operation instructions) and using hardware fsqrt to implement square root operations (the sqrt() function of the math library will use software open operations after assembly, which requires 1224 clock cycles. If the inline assembly syntax is used to force the execution of fsqrt, the running cycle can be further shortened) are printed out through the serial port through printf, as shown in Table 6.

[0263] Table 6 Running speed comparison

[0264] parameter Before optimization After optimization After optimization (including fsqrt) Total ticks 647280 371185 358536 Total times 0.040476 0.023196 0.022412

[0265] Before optimization, the number of clock cycles required to execute the test routine was 647280, while after optimization, the number of clock cycles required was only 358536, and the required clock cycles were reduced to 55.4% of the original. Therefore, the total time was reduced from the original 0.040476 to 0.022412.

[0266] In response to the needs of non-intrusive load identification NIL M, the BERT neural network used was optimized and tested to improve its recognition precision and accuracy. The running time of the neural network was reduced from 165s to 91.4s using the optimized kernel.

[0267] Embodiment 2

[0268] The purpose of this embodiment is to provide a general Transformer acceleration system based on floating-point instruction set extension, including:

[0269] The multiplier building module is configured to: build a multiplier, the multiplier includes a multi-stage adder connected in sequence, and a 4-Booth encoding algorithm is used to encode data; the encoded data is sequentially operated by the multi-stage adder to obtain the output of the multiplier;

[0270] The floating-point register construction module is configured to: construct a floating-point register, specifically, add a floating-point register group, and generate a floating-point register group. The write port logic of the floating-point register group corresponds the input result register index to the respective register number, generates a write enable signal, and the enabled general register writes the data into the register. The read port is implemented using a multi-way parallel selector, and the selection signal of the multi-way parallel selector is the index of the read operand register;

[0271] The accelerated running module is configured to run the Transformer neural network based on the constructed multipliers and floating-point registers.

[0272] Although the above describes the specific implementation mode of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.

Claims

1. A general Transformer acceleration method based on floating-point instruction set extension, characterized in that: include: A multiplier is constructed, the multiplier includes a plurality of adders connected in sequence, and a 4-Booth encoding algorithm is used to encode data; the encoded data is sequentially operated by the plurality of adders to obtain an output of the multiplier; Constructing floating-point registers, the specific steps are: generating a floating-point register group, the write port logic of the floating-point register group corresponds the input result register index to the respective register number, generating a write enable signal, and the enabled general register writes the data into the register, and the read port is implemented using a multi-way parallel selector, and the selection signal of the multi-way parallel selector is the index of the read operand register; The RISC-V processor runs the Transformer neural network based on the built-in multipliers and floating-point registers.

2. The general Transformer acceleration method based on floating point instruction set extension as claimed in claim 1, characterized in that: The multi-stage adder of the multiplier adopts an addition tree structure, so that the number of adder bits used for adding two adjacent numbers is minimized.

3. The general Transformer acceleration method based on floating point instruction set extension as claimed in claim 1, characterized in that: The steps to construct a floating point register include: Extended register group: add floating point registers; Modify the decoding process: add the decoding process of floating-point instructions; Modify the dispatch module; Modify the calculation processing module; Added CSR register for floating point numbers; The decoding module identifies the floating-point instruction and passes its characteristics to the dispatch module. The dispatch module dispatches the floating-point instruction to the floating-point calculation processing module according to the instruction type. The calculation processing module performs floating-point operations and updates the results and status information to the floating-point CSR register.

4. The general Transformer acceleration method based on floating point instruction set extension as claimed in claim 3, characterized in that decoding Identify floating-point instructions. The specific steps include: Generate type information and operand register indexes for different instructions; Determine the direction of each instruction, compare the corresponding part of the instruction with the specific value, if they are equal, pull the corresponding signal high, if not, keep it low; After obtaining the corresponding judgment signal value, a specific instruction is selected; After receiving the specific instruction, it is necessary to determine the specific operation implemented by the instruction; Different floating-point instructions will point to different register groups; Not all instructions in all floating-point instruction sets use floating-point registers. Some instructions move data between different registers, or access memory, read data addresses from integer general registers, and write data back to floating-point registers. Different enable signals are used to indicate whether these structures and their specific directions are present in floating-point instructions.

5. The general Transformer acceleration method based on floating point instruction set extension as claimed in claim 1, characterized in that: include: Different enable signals are used to indicate whether these structures exist in the floating-point instruction and the specific directions of rs1, rs2, rs3, and rd.

6. The general Transformer acceleration method based on floating point instruction set extension as claimed in claim 5, characterized in that: include: Assign the decoded index value to the index of the corresponding register of FPU; After determining whether the floating-point instruction has rs1, rs2, rs3, rd and the corresponding index value, the register group type corresponding to the floating-point instruction is determined.

7. The general Transformer acceleration method based on floating point instruction set extension as claimed in claim 5, characterized in that: Determine the register group type corresponding to the floating-point instruction. The specific steps are: The rs1 of floating-point instructions with rs1 parameters points to the floating-point register group except flw, fsw, and fmv.wx; The rs2 of floating-point instructions with rs2 parameters all point to the floating-point register group; The rs3 of floating-point instructions with rs3 parameters all point to the floating-point register group. Except for fmv.xw and flt, the rd of floating-point instructions with rd parameter points to the floating-point register group.

8. The general Transformer acceleration method based on floating point instruction set extension as claimed in claim 1, characterized in that: After decoding, the decoding result of the instruction and the operands read from the register group will be dispatched to different operation units for execution; All instructions are transferred to the ALU for execution, and the decoding results of floating-point instructions are added to the input and output interfaces.

9. The general Transformer acceleration method based on floating point instruction set extension as claimed in claim 1, characterized in that: Add the decoding results of floating-point instructions at the input and output interface, specifically: First, the parameter rs3 is used for the first time in the floating-point instruction, and the index signal, judgment signal and specific value of rs3 are added to the input and output; In addition, a discrimination signal should be added to the input and output to determine the type of register group pointed to. If it is pointing to a floating-point register group, the actual dispatch function is also arranged to be executed in the ALU. The judgment is made through the grouping information generated by decoding, and the instruction is dispatched to the corresponding unit in the ALU to execute this instruction. Adding the grouping information of the fonecycle module and the fdivsqrt module allows floating-point instructions to be sent to the correct unit for execution; The remaining two non-operational long instructions, flw and fsw, are added to the same execution process of sw and lw, and are also added to the AGU information bus, dispatched to the AGU for processing, and further sent to the LSU for execution.

10. A general purpose Transformer acceleration system based on floating point instruction set extension, characterized in that: include: The multiplier building module is configured to: build a multiplier, the multiplier includes a multi-stage adder connected in sequence, and a 4-Booth encoding algorithm is used to encode data; the encoded data is sequentially operated by the multi-stage adder to obtain the output of the multiplier; The floating point register construction module is configured to: construct a floating point register, specifically, to: generate a floating point register group. The write port logic of the floating point register group corresponds the input result register index to the respective register number, generates a write enable signal, and the enabled general register writes the data into the register. The read port is implemented using a multi-way parallel selector, and the selection signal of the multi-way parallel selector is the index of the read operand register; The accelerated operation module is configured to accelerate the operation of the Transformer neural network based on the constructed multipliers and floating-point registers.

Images