RISC-V-based AI processor design method for small convolution kernel acceleration

Abstract

The invention relates to the technical field of RISC-V instruction set processors, in particular to an RISC-V-based AI processor design method for small convolution kernel acceleration, which comprises the following steps: determining the acceleration demand of a 3 * 3 convolution kernel in a convolutional neural network (CNN), selecting an open architecture RV32I of RISC-V as a basis, defining a core architecture of an AI processor, comprising a calculation unit, a register file and a control unit, the processing of the small convolution kernel is optimized; designing and realizing a vector arithmetic logic unit (VALU) and a convolution operation unit (CUU); the VALU has parallel computing capability and can execute addition, subtraction and multiplication vector operations to support instruction set extension; a convolution arithmetic unit CUU processes 3 * 3 convolution operations, including dedicated hardware logic to perform element-by-element multiplication and accumulation operations, thereby accelerating the convolution calculation process.

Description

Technical Field

[0001] The present invention relates to the technical field of RISC-V instruction set processors, and in particular to a RISC-V-based AI processor design method for accelerating small convolution kernels. Background Art

[0002] RISC-V is a new, simple, open and free instruction set architecture. It was invented in 2010 by developers such as Professor Krste Asanovic, Andrew Waterman and Yunsup Lee of the University of California, Berkeley, and has been strongly supported by David Patterson, a leader in the field of computer architecture. As a new generation of reduced instruction set, although RISC-V has a short history of development, it has taken into account both the amount and speed of data transmission, and has obvious advantages in architectural simplicity, operating efficiency, cost and flexibility. RISC-V is highly expected in the AIoT era and has become one of the instruction architectures for AI processor design.

[0003] RISC-V's processor design uses the Harvard structure as its memory architecture. The Harvard structure is a computer memory design that separates program instruction storage and data storage. In the Harvard structure, the program memory and data memory are two independent memories, each with its own independent address space and data bus.

[0004] Although traditional CPUs can be used to perform various computing tasks, AI CPUs have higher efficiency and performance when processing AI-related computing tasks. In the field of AI, CPUs are mainly responsible for some general computing tasks in data preprocessing, model training, and reasoning. Since AI tasks usually require the processing of large amounts of data, AI CPUs usually have multiple cores and threads, which can handle multiple tasks at the same time, thereby improving overall performance. However, for more complex AI tasks, such as the training and reasoning of deep learning models, more powerful computing power is usually required. In this case, dedicated hardware such as graphics processing units (GPUs), tensor processing units (TPUs), and neural network processors (NPUs) are often used to accelerate AI computing. These processors have higher parallel processing capabilities and optimized AI algorithms, which can significantly improve the execution efficiency of AI tasks.

[0005] FPGA technology and related development platforms have greatly reduced the cost of processor design and can obtain specific experimental data in the research stage before tape-out. The present invention uses the QuartusⅡ development environment on the CycloneⅢ chip, through hardware description language, and circuit design from gate level to RTL level to complete the design of a processor that has an acceleration effect on a specific AI algorithm. Summary of the invention

[0006] The purpose of the present invention is to provide a RISC-V-based AI processor design method for accelerating small convolution kernels. Through the design of dedicated vector instructions and convolution operation instructions, the operation efficiency when processing 3×3 convolution kernels is significantly improved. The implementation of vector register stacks and vector arithmetic logic units (VALUs) enables multiple data elements to be calculated in parallel, reducing the number of instructions and data transmission delays, and accelerating the execution of convolution operations.

[0007] In order to achieve the above technical objectives and the above technical effects, the present invention is implemented through the following technical solutions:

[0008] A RISC-V-based AI processor design method for accelerating small convolution kernels includes the following steps:

[0009] S1: Determine the acceleration requirements of 3×3 convolution kernels in convolutional neural networks (CNNs), select RISC-V's open architecture RV32I as the basis, and define the core architecture of the AI ​​processor, including computing units, register files, and control units to optimize the processing of small convolution kernels.

[0010] S2: Design a vector register stack, which actually has 32 96-bit registers. The register stack supports fast and parallel data access to ensure that data can be loaded and stored in an efficient manner during the convolution operation. When designing the register stack, its layout needs to be optimized so that it can well support the operation process of 3×3 convolution, with the focus on reducing data transmission delays and improving the parallelism and efficiency of the overall calculation.

[0011] S3: Introduces vector instructions and convolution operation instructions to enhance the computing power of the processor; including vector addition, subtraction and multiplication operations, and designs a special instruction format for convolution operations to ensure programming simplicity and efficient execution. In addition, these new instructions are encoded to ensure their compatibility with existing RISC-V instructions, and the instruction set is expanded by assigning specific opcodes and function bits to extended instructions.

[0012] S4: Design and implementation of the vector arithmetic logic unit (VALU) and convolution unit (CUU); the VALU has parallel computing capabilities and can perform addition, subtraction, and multiplication of vector operations to support instruction set extensions; the convolution unit CUU handles 3×3 convolution operations and contains dedicated hardware logic to perform element-by-element multiplication and accumulation operations, thereby accelerating the convolution calculation process.

[0013] S5: Design a control unit for parsing and executing extended instructions to ensure that control signals accurately schedule the operations of VALU and CUU; ensure that data transmission between the register file, VALU, and CUU has sufficient bandwidth and low latency when executing extended instructions to avoid performance bottlenecks and improve overall efficiency.

[0014] S6: Use simulation tools to perform functional simulation and verification on the new processor design to ensure that each hardware unit and new instructions can run correctly in a variety of scenarios. Write comprehensive test cases to verify the correctness and performance of the new design, evaluate its speed and efficiency improvement when performing 3×3 convolution calculations, find out the improvements compared to the traditional design, and make corresponding optimization adjustments.

[0015] S7: In the optimization and iteration phase, the hardware design and instruction set are further optimized based on the simulation and verification results to ensure high efficiency and reliability when processing deep learning and image processing tasks, and ultimately achieve the best performance.

[0016] Furthermore, the step S2 specifically includes:

[0017] Design a vector register stack containing 32 96-bit registers to support the storage of sufficient convolution kernel data and intermediate operation results. Each vector register consists of multiple elements for parallel operation. According to the encoding format of RISC-V vector instructions, the vector register capacity is determined to be 32. The register addresses from top to bottom are v1, v2, v3…v32. When processing three-dimensional vectors, the depth of each vector register is set to 3, storing the vector components in the x, y, and z directions respectively; each register should support fast read and write operations to meet the needs of parallel computing. Optimize the register layout to support the data flow requirements of 3×3 convolution operations, facilitating parallel loading, calculation, and storage of data.

[0018] Furthermore, the step S3 specifically includes:

[0019] Expand the RISC-V instruction set and add vector addition, subtraction, and multiplication (dot product) instructions to support common vector operations. Design special instructions for convolution operations to simplify programming complexity and accelerate execution efficiency.

[0020] Encode the new instructions to ensure compatibility with existing RISC-V instructions. Assign specific `opcode` and `func3` fields to each type of instruction.

[0021] Vector addition instruction: `opcode` is 1010111, `func3` is 000. Element-wise addition is performed on corresponding components of two vectors, and the result is stored in the destination register.

[0022] Vector subtraction instruction: `opcode` is 1010111, `func3` is 001. Implements element-by-element subtraction of two vectors, and the result is stored in the destination register.

[0023] Vector multiplication (dot product) instruction: `opcode` is 1010111, `func3` is 100. Calculates the dot product of two vectors and outputs a scalar, which is suitable for deep learning and graphics processing;

[0024] Convolution operation instruction: `opcode` is 1010111, `func3` is 010; it calculates the convolution of two 3×3 arrays and outputs a scalar, which is suitable for deep learning and graphics processing.

[0025] Furthermore, the step S4 specifically includes:

[0026] Implement VALU to handle parallel vector operations. VALU receives operands from vector registers to perform specified addition, subtraction, and multiplication operations and quickly returns the results. Design the logic of the arithmetic unit to fully utilize hardware parallelism and accelerate operations.

[0027] Design and implement the convolution operation unit, including the input data buffer and convolution kernel buffer, which are used to store and process the data of 3×3 convolution. Ensure that the CUU can efficiently perform element-by-element product and sum operations and send the results to registers.

[0028] Furthermore, the step S5 specifically includes:

[0029] Design the vector instruction control unit to parse and manage the execution of vector instructions, schedule control signals to drive VALU and register file operations, and ensure that vector instructions can be executed efficiently and in parallel. Define control logic to coordinate the execution order and data flow of extended instructions. Optimize data path design to support high-bandwidth data transmission and minimize waiting time. Ensure smooth data flow between register files, VALUs, and CUUs without bottlenecks.

[0030] The convolution unit has a convolution kernel buffer and an input data buffer, which are dedicated to storing the matrix to be processed and the convolution kernel weights. The data flow to the VALU or dot product operator is determined by instruction decoding. The parallel processing of the convolution operation unit allows multiple data windows to be processed in parallel, and element-by-element multiplication and summation are performed with the convolution kernel, and the final result is stored in a dedicated buffer.

[0031] Furthermore, the step S6 specifically includes:

[0032] Use the Quartus II simulation tool to verify functionality by observing output waveforms and expected results. Focus on verifying the correctness of vector instructions and convolution operations. Write extensive test cases to ensure that new instructions and hardware units perform consistently in multiple scenarios. Compare the computing performance of the AI ​​processor with the traditional RISC-V architecture to evaluate its acceleration effect when performing 3×3 convolution kernel calculations. Identify performance bottlenecks through test results and make corresponding adjustments and optimizations.

[0033] Furthermore, the step S7 specifically includes:

[0034] Based on simulation and test feedback, the hardware design and instruction set are adjusted to further improve computing performance and efficiency. Through multiple iterations, the efficient performance and reliability of AI processors in deep learning and image processing applications are ensured.

[0035] Beneficial effects of the present invention:

[0036] The present invention enhances the computing power of the processor when performing convolution operations by introducing dedicated vector instructions and convolution operation instructions. When performing convolution operations in convolutional neural networks (CNNs), traditional CPUs usually need to complete multiplication and accumulation operations through multiple cycles and multiple instructions, and their efficiency is limited by instruction set redundancy and data transmission delay. The technical solution realizes efficient parallel computing capabilities by designing dedicated vector register stacks and vector arithmetic logic units (VALUs), and can process multiple data elements at the same time. Vector instructions such as addition, subtraction and dot products can be directly executed at the hardware level, reducing the number of instructions, thereby improving execution speed and overall computing efficiency. Using vector registers, convolution operation data can be loaded, stored and calculated in parallel, significantly reducing data transmission delays and execution time.

[0037] The present invention realizes efficient utilization of resources through optimization of hardware design. The design of the vector register stack supports fast parallel data access and operation, effectively reduces register read and write delays, and improves the parallelism and efficiency of calculations. In the convolution operation, the specific hardware logic (convolution operation unit CUU) can specifically handle 3×3 convolution operations, avoiding repeated calls and power consumption increases of general computing units. In addition, the convolution unit is connected to the register stack through a high-bandwidth internal bus to ensure smooth data transmission and reduce the frequency of memory access, thereby further reducing system power consumption. This optimized design is particularly beneficial for power-constrained embedded devices, making the processor suitable for application in edge computing and AIoT devices.

[0038] In deep learning tasks, especially image recognition, small convolution kernels (such as 3×3) improve the efficiency of model training due to their small number of parameters. Smaller parameters mean that the model complexity is reduced, and the model is easier to converge during training, and is less likely to have overfitting problems. The present invention significantly accelerates the execution of the convolution kernel by designing special instructions for the 3×3 convolution kernel, while maintaining a high nonlinear expression capability and being able to extract more complex features. In addition, by superimposing multiple layers of small convolution kernels, local and global features are extracted layer by layer, and the generalization ability of the model is also improved. This enhanced feature expression capability comes from more efficient hardware implementation, which directly converts the speed increase of convolution operations into better feature extraction effects.

[0039] The present invention fully considers the compatibility with the existing RISC-V instruction set when designing new instructions, and ensures the seamless integration of new and old instructions by assigning specific opcode and func fields to extended instructions. This compatibility not only protects existing software assets, but also facilitates the integration and promotion of new instructions. At the same time, the openness and modular design of the RISC-V instruction set make future instruction expansion and hardware improvements more flexible and cost-effective. During the functional simulation and testing process of the processor, through implementation and verification on the FPGA platform, the design risk and development cost are further reduced, laying the foundation for the large-scale deployment of new technologies in diverse AI applications.

[0040] Of course, any product implementing the present invention does not necessarily need to achieve all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for describing the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0042] Figure 1 This is a schematic diagram of the RISC-V structure before expansion;

[0043] Figure 2 (a) is the schematic diagram of the vector register stack; (b) is the schematic diagram of the vector register;

[0044] Figure 3 This is the schematic diagram of the vector register stack;

[0045] Figure 4 This is a schematic diagram of a RISC-V-based AI processor for accelerating small convolution kernels;

[0046] Figure 5 RISC-V-based computer RTL circuit diagram for accelerating AI;

[0047] Figure 6 It is an accelerated AI processor based on RISC-V;

[0048] Figure 7 A schematic diagram showing the simulation results. DETAILED DESCRIPTION

[0049] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0050] Example 1

[0051] The RISC-V-based AI processor design method for accelerating small convolution kernels described in this embodiment includes the following steps:

[0052] S1: Determine the acceleration requirements of 3×3 convolution kernels in convolutional neural networks (CNNs), select RISC-V's open architecture RV32I as the basis, and define the core architecture of the AI ​​processor, including computing units, register files, and control units to optimize the processing of small convolution kernels.

[0053] S2: Design a vector register stack, which actually has 32 96-bit registers. The register stack supports fast and parallel data access to ensure that data can be loaded and stored in an efficient manner during the convolution operation. When designing the register stack, its layout needs to be optimized so that it can well support the operation process of 3×3 convolution, with the focus on reducing data transmission delays and improving the parallelism and efficiency of the overall calculation.

[0054] S3: Introduces vector instructions and convolution operation instructions to enhance the computing power of the processor; including vector addition, subtraction and multiplication operations, and designs a special instruction format for convolution operations to ensure programming simplicity and efficient execution. In addition, these new instructions are encoded to ensure their compatibility with existing RISC-V instructions, and the instruction set is expanded by assigning specific opcodes and function bits to extended instructions.

[0055] S4: Design and implementation of the vector arithmetic logic unit (VALU) and convolution unit (CUU); the VALU has parallel computing capabilities and can perform addition, subtraction, and multiplication of vector operations to support instruction set extensions; the convolution unit CUU handles 3×3 convolution operations and contains dedicated hardware logic to perform element-by-element multiplication and accumulation operations, thereby accelerating the convolution calculation process.

[0056] S5: Design a control unit for parsing and executing extended instructions to ensure that control signals accurately schedule the operations of VALU and CUU; ensure that data transmission between the register file, VALU, and CUU has sufficient bandwidth and low latency when executing extended instructions to avoid performance bottlenecks and improve overall efficiency.

[0057] S6: Use simulation tools to perform functional simulation and verification on the new processor design to ensure that each hardware unit and new instructions can run correctly in a variety of scenarios. Write comprehensive test cases to verify the correctness and performance of the new design, evaluate its speed and efficiency improvement when performing 3×3 convolution calculations, find out the improvements compared to the traditional design, and make corresponding optimization adjustments.

[0058] S7: In the optimization and iteration phase, the hardware design and instruction set are further optimized based on the simulation and verification results to ensure efficiency and reliability in processing deep learning and image processing tasks, and ultimately achieve the best performance.

[0059] In this embodiment, step S2 specifically includes:

[0060] Design a vector register stack containing 32 96-bit registers to support the storage of sufficient convolution kernel data and intermediate operation results. Each vector register consists of multiple elements for parallel operations. According to the encoding format of RISC-V vector instructions, the vector register capacity is determined to be 32. The register addresses from top to bottom are v1, v2, v3...v32. When processing three-dimensional vectors, the depth of each vector register is set to 3, storing the vector components in the x, y, and z directions respectively; each register should support fast read and write operations to meet the needs of parallel computing. Optimize the register layout to support the data flow requirements of 3×3 convolution operations, facilitating parallel loading, calculation, and storage of data. The specific register layout is as follows: Figure 3 shown.

[0061] In this embodiment, step S3 specifically includes:

[0062] Expand the RISC-V instruction set and add vector addition, subtraction, and multiplication (dot product) instructions to support common vector operations. Design special instructions for convolution operations to simplify programming complexity and accelerate execution efficiency.

[0063] Encode the new instructions to ensure compatibility with existing RISC-V instructions. Assign specific `opcode` and `func3` fields to each type of instruction.

[0064] Vector addition instruction: `opcode` is 1010111, `func3` is 000. Element-wise addition is performed on corresponding components of two vectors, and the result is stored in the destination register.

[0065] Vector subtraction instruction: `opcode` is 1010111, `func3` is 001. Implements element-by-element subtraction of two vectors, and the result is stored in the destination register.

[0066] Vector multiplication (dot product) instruction: `opcode` is 1010111, `func3` is 100. Calculates the dot product of two vectors and outputs a scalar, suitable for deep learning and graphics processing.

[0067] Convolution operation instruction: `opcode` is 1010111, `func3` is 010; it calculates the convolution of two 3×3 arrays and outputs a scalar, which is suitable for deep learning and graphics processing.

[0068] In this embodiment, step S4 specifically includes:

[0069] Implement VALU to handle parallel vector operations. VALU receives operands from vector registers to perform specified addition, subtraction, and multiplication operations and quickly returns the results. Design the logic of the arithmetic unit to fully utilize hardware parallelism and accelerate operations.

[0070] Design and implement the convolution operation unit, including the input data buffer and convolution kernel buffer, which are used to store and process the data of 3×3 convolution. Ensure that the CUU can efficiently perform element-by-element product and sum operations and send the results to registers.

[0071] In this embodiment, step S5 specifically includes:

[0072] Design the vector instruction control unit to parse and manage the execution of vector instructions, schedule control signals to drive VALU and register file operations, and ensure that vector instructions can be executed efficiently and in parallel. Define control logic to coordinate the execution order and data flow of extended instructions. Optimize data path design to support high-bandwidth data transmission and minimize waiting time. Ensure smooth data flow between register files, VALUs, and CUUs without bottlenecks.

[0073] The convolution unit has a convolution kernel buffer and an input data buffer, which are dedicated to storing the matrix to be processed and the convolution kernel weights. The data flow to the VALU or dot product operator is determined by instruction decoding. The parallel processing of the convolution operation unit allows multiple data windows to be processed in parallel, and element-by-element multiplication and summation are performed with the convolution kernel, and the final result is stored in a dedicated buffer.

[0074] In this embodiment, step S6 specifically includes:

[0075] Use the Quartus II simulation tool to verify functionality by observing output waveforms and expected results. Focus on verifying the correctness of vector instructions and convolution operations. Write extensive test cases to ensure that new instructions and hardware units perform consistently in multiple scenarios. Compare the computing performance of the AI ​​processor with the traditional RISC-V architecture to evaluate its acceleration effect when performing 3×3 convolution kernel calculations. Identify performance bottlenecks through test results and make corresponding adjustments and optimizations.

[0076] In this embodiment, step S7 specifically includes:

[0077] Based on simulation and test feedback, the hardware design and instruction set are adjusted to further improve computing performance and efficiency. Through multiple iterations, the efficient performance and reliability of AI processors in deep learning and image processing applications are ensured.

[0078] Example 2

[0079] AI processor design for accelerating small convolution kernels

[0080] Instruction extension based on RV32I basic instruction architecture

[0081] Among the common AI algorithms, convolutional neural networks (CNNs) are widely used. Due to their wide application range, they are widely used in deep learning scenarios such as image recognition. At the same time, this algorithm also has a huge computing power consumption problem that many AI algorithms have. There can be multiple convolutional layers during the training process, and each convolutional layer requires a convolution kernel to participate in the convolution operation. Taking a 3×3 convolution kernel as an example, its calculation formula is:

[0082]

[0083] It can be seen from the above formula that the convolution operation mainly includes the vector dot product operation composed of multiplication and addition. Taking the 3×3 convolution kernel as an example, 9 multiplication and 8 addition operations are required to obtain a component result of the output matrix. Therefore, any convolutional neural network algorithm implemented in any high-level language requires the high-frequency use of vector multiplication and vector addition operations. This inevitably poses a great challenge to the performance of the processor, so GPUs are often required to assist ordinary CPUs in the operating environment. Designing an AI processor to provide instruction-level support for specific operations will greatly alleviate the pressure of artificial intelligence algorithms on the CPU. The CPU needs to complete the basic instruction set and expand the vector instructions and convolution operation instructions in a targeted manner to support Formula 1, which can fundamentally speed up the execution of the convolutional neural network algorithm.

[0084] The RV32I basic instruction architecture includes six common 32-bit instructions, namely R, I, S, B, U, and J. Based on the demand for convolution kernel acceleration, combined with some widely used RISC-V instruction sets, the basic processor structure is constructed. The schematic diagram is as follows:

[0085] Small convolution kernel selection

[0086] Common convolution kernel sizes include 1×1, 3×3, 5×5, 7×7, and sometimes 11×11. If we extract features in the convolution layer, we choose the most commonly used 3×3 convolution kernel as the target for AI processor acceleration for the following reasons:

[0087] (1) Small number of parameters. The receptive fields of two 3×3 convolution kernels and one 5×5 convolution kernel are the same, and the receptive fields of three 3×3 convolution kernels and one 7×7 convolution kernel are the same (the receptive field is the feature that can extract the number of surrounding neighbors). Assuming that the input and output channels are both C, and the parameters required by the 7×7 convolution kernel are used, the calculation process is:

[0088] 7×7×C×C=49C 2 (2)

[0089] Using the parameters required by the 3×3 convolution kernel, the calculation process is:

[0090] 3×3×C×C+3×3×C×C+3×3×C×C=27C 2 (3)

[0091] It can be seen that under the same receptive field, three 3×3 convolutions require much fewer parameters than one 7×7 convolution, which undoubtedly reduces the complexity of the model and speeds up the training. When training a model, the smaller the number of parameters, the easier it is for the model to converge and it is not easy to overfit. Therefore, a 3×3 convolution kernel can effectively reduce the number of model parameters and improve the training efficiency and generalization ability of the model. Moreover, although the receptive field is the same, the 3×3 convolution has a higher degree of nonlinearity and can represent more complex functions.

[0092] (2) Better feature extraction capability. A 3×3 convolution kernel has better feature extraction capability than a 5×5 convolution kernel. In a convolutional neural network, a smaller convolution kernel can extract finer-grained features. For example, in an image recognition task, a 3×3 convolution kernel can capture information about 8 pixels around a pixel, while a 5×5 convolution kernel can only capture information about 16 pixels. Therefore, using a 3×3 convolution kernel can better capture local features in the input data, thereby improving the accuracy of the model.

[0093] (3) It can improve computing efficiency. In convolutional neural networks, convolution operations are one of the most computationally intensive operations. Using a 3×3 convolution kernel can improve computing efficiency, thereby speeding up model training and inference. For example, when using a GPU to train a model, using a 3×3 convolution kernel can reduce the amount of data transmitted, thereby improving computing efficiency.

[0094] (4) Better model generalization ability. In convolutional neural networks, the generalization ability of a model refers to its ability to adapt to unknown data. The 3×3 convolution kernel can gradually extract local and global features in an image by stacking multiple layers of convolution.

[0095] Implementation of small convolution kernel acceleration processor

[0096] 1. Design and implementation of vector register stack

[0097] The 3×3 convolution kernel is designed as the acceleration target. For this purpose, it is necessary to design instructions for executing dot products and corresponding dedicated registers and operators. Vector registers will be a necessary choice. In order to provide 3×3 matrices at the same time, a register stack that can accommodate larger vectors is designed. The schematic diagram is as follows Figure 2 As shown, the circuit design schematic is as follows Figure 3 The actual capacity of the vector register stack is 32 vector registers, each of which can store 96 bits.

[0098] 2. Vector Extension Instruction Set

[0099] Neural network convolution operations mainly involve operations on vector and matrix data. While designing independent operation instructions, this processor also designs and supports some vector expansion operations, mainly including three vector instructions: addition, subtraction, and multiplication. Its specific technologies include:

[0100] 1. Vector registers are used to store and operate vector data. Each vector register consists of multiple elements and is used for parallel operations. According to the encoding format of RISC-V vector instructions, the vector register capacity is determined to be 32. The register addresses from top to bottom are v1, v2, v3...v32. When processing three-dimensional vectors, the depth of each vector register is set to 3, storing the vector components in the x, y, and z directions respectively.

[0101] 2. Vector arithmetic logic unit, which can perform vector operations, including vector addition, vector subtraction and vector dot product; VALU supports parallel computing, that is, it can process multiple elements in a vector at the same time, improving computing speed and efficiency. VALU receives vector operands from vector registers, performs the operations specified by the instructions, and returns the results to the vector registers.

[0102] 3. Vector instruction control unit, used to parse and manage the execution of vector instructions; specifically handle vector instructions, generate and schedule control signals to drive the operation of VALU and vector register files; ensure that vector instructions are executed efficiently in parallel.

[0103] The design vector instruction format is as follows:

[0104]

[0105] The specific coding is as follows:

[0106] 1. 32-bit vector addition instruction:

[0107] The vector addition instruction is used to add the components of two vectors element by element. That is, if vector registers A and B store vectors (A1, A2, A3) and (B1, B2, B3) respectively, the vector addition instruction will calculate (A1+B1, A2+B2, A3+B3) and store the result in the target vector register. The instruction supports parallel processing of data and improves the calculation speed. The design instruction format is as follows:

[0108]

[0109] 2. 32-bit vector subtraction instruction:

[0110] The vector subtraction instruction is used to perform element-by-element subtraction of the components of two vectors. That is, if vector registers A and B store vectors (A1, A2, A3) and (B1, B2, B3) respectively, the vector subtraction instruction will calculate (A1-B1, A2-B2, A3-B3) and store the result in the target vector register. The instruction also supports parallel processing of data to improve computing efficiency. The design instruction format is as follows:

[0111]

[0112] 3. 32-bit vector multiplication instructions:

[0113] Here, vector multiplication refers to the dot product or scalar product of vectors. The vector dot product operation instruction is used to calculate the dot product of two vectors. The dot product operation multiplies the corresponding components of the two vectors and sums them. The result is a scalar value; that is, vector registers A and B store vectors (A1, A2, A3) and (B1, B2, B3) respectively, then the vector dot product instruction will calculate A1*B1+A2*B2+A3*B3 and store the result in a scalar register; it is used for machine learning and graphics processing. The design instruction format is as follows:

[0114]

[0115] From the above machine instruction format, we can see that the OP code 01010111 is all instructions related to vector operations, and the specific instruction decoding control is determined by the func3 field. (Note: The all-0 part is the third operand, which can be replaced by the register address according to the instruction requirements in actual applications.)

[0116] When func3 is encoded as 000, it is vector addition;

[0117] When func3 is encoded as 001, it is vector subtraction;

[0118] When func3 is encoded as 100, it is vector multiplication;

[0119] The relationship between the vector operator VALU, the instruction bus, the data bus, and other components can be seen in Figure 4 .

[0120] 3. Design and implementation of dot product instruction

[0121] The 3×3 convolution kernel and convolution data involved in this processor design are input into the corresponding matrix in the form of arrays. At the same time, in order to avoid interference with other instructions and to increase a certain computing rate, a sufficiently large buffer is created to store the results of the convolution operation. In order to reduce redundant design, the vector register stack is shared with VALU, and the current data is sent to VALU or the dot product operator through control methods such as instruction decoding. Since the convolution instruction is independent of other instructions, but in order to integrate with the whole machine, it is encoded with the help of the encoding format of the vector instruction. The 32-bit instruction is shown in the following table.

[0122]

[0123] Convolution operation unit, used for convolution operation in deep learning; contains convolution kernel buffer (used to store convolution kernel weights), input data buffer (used to store data to be processed) and operation logic for performing convolution operation; convolution operation unit can process multiple data windows in parallel, multiply and sum the elements with the convolution kernel, and store the output results in the buffer. This unit is designed to accelerate the forward propagation calculation in convolutional neural networks.

[0124] 4. Accelerate AI Processor Implementation and Testing

[0125] 1. Implementation of the processor

[0126] Combined with the basic RV32I instruction set, the processor designed for convolution kernel acceleration can also handle normal programs and scalar operations, and a processor containing multiple operators has been designed. The processor can use one machine instruction to complete the calculation of a 3×3 convolution kernel, achieving the purpose of maximally accelerating small-scale convolution operations.

[0127] For convolution kernels larger than 3×3, since this processor supports vector instructions, the convolution operation can be translated into vector multiplication instructions and vector addition instructions at the machine instruction level through compilation to achieve the purpose of maximum acceleration. Compared with the traditional CPU's component-by-component cyclic multiplication and addition, it can also achieve the purpose of significant speed-up.

[0128] The overall design of AI processor is as follows Figure 4 As shown, the test program can be executed.

[0129] Integrate the instruction memory, data memory and AI processor to complete the whole machine design. The RTL diagram is shown in Figure 5 .

[0130] The detailed circuit diagram of the processor is shown in the attached Figure 6 As shown;

[0131] Example 3

[0132] Test Method

[0133] The simulation tool provided by the Quartus II platform is used for simulation, and the correctness of the results is judged by observing the output results of the waveform graph and combining them with the expected results.

[0134] The simulation results show that when funct is the corresponding instruction function code, the corresponding instruction operation is executed, such as Figure 7 .

[0135] The calculation result of the vector instruction is that the elements of the two vectors are (9, 8, 7) and (1, 2, 3), respectively. The convolution kernel weight is a 3*3 matrix with all 1s, and the convolved data is a 3*3 matrix with all 1s:

[0136] · Vector addition: When the function code funct is [0], a vector addition operation is performed, and the elements of two vectors are added correspondingly to output a new vector (result_x, result_y, result_z) = (10, 10, 10).

[0137] · Vector subtraction: When the function code funct is [1], a vector subtraction operation is performed. The elements of two vectors are subtracted correspondingly to output a new vector (result_x, result_y, result_z) = (8, 6, 4).

[0138] · Vector multiplication (dot product): When the function code funct is [4], a vector dot product operation is performed. The elements of two vectors are correspondingly multiplied and then added, and a scalar data reslut=2E (in hexadecimal) is output.

[0139] Convolution operation: When the function code funct is [2], the convolution operation is performed. The two matrices are operated according to the convolution formula and a scalar data conv_result=9 is output.

[0140] (2) Comparison of acceleration effects

[0141] Since convolution operations generally need to be decomposed into multiplication and addition during the compilation phase, we take vector addition with the worst acceleration ratio and convolution operation with the largest acceleration ratio as examples to compare the amount of program code running on the accelerated AI processor and the conventional RISC-V architecture processor.

[0142]

[0143]

[0144] If RISC-V vector addition instructions are used, only one instruction is needed, which is "vadd vd, vs2, vs1". The vector in the vs1 register is added to the vector in the vs2 register, and the result is sent to the vd register. Because RISC-V supports parallel data operations, the three elements of the vector do not need to be input in a loop like assembly, and the elements do not need to be added in a loop. In this regard, RISC-V vector instructions save computing time and improve computing efficiency; at the same time, it also saves hardware resources and effectively avoids resource waste.

[0145] The preferred embodiments of the present invention disclosed above are only used to help illustrate the present invention. The preferred embodiments do not describe all the details in detail, nor do they limit the invention to the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present invention, so that those skilled in the art can understand and use the present invention well. The present invention is limited only by the claims and their full scope and equivalents.

Claims

1. A RISC-V-based AI processor design method for accelerating small convolution kernels, characterized in that: The following steps are involved: S1: Determine the acceleration requirements of 3×3 convolution kernels in convolutional neural networks, select RISC-V's open architecture RV32I as the basis, and define the core architecture of the AI ​​processor, including computing units, register files, and control units, to optimize the processing of small convolution kernels; S2: Design a vector register stack, which actually has 32 96-bit registers. The register stack supports fast and parallel data access to ensure that data can be loaded and stored in an efficient manner during the convolution operation. When designing the register stack, its layout needs to be optimized to support the 3×3 convolution operation process well, with the focus on reducing data transmission latency and improving the parallelism and efficiency of the overall calculation. S3: Introduce vector instructions and convolution operation instructions to enhance the computing power of the processor; including vector addition, subtraction and multiplication operations, and design a special instruction format for convolution operations to ensure the simplicity of programming and the efficiency of execution; in addition, encode these new instructions to ensure their compatibility with existing RISC-V instructions, and implement the expansion of the instruction set by assigning specific opcodes and function bits to the extended instructions; S4: Design and implementation of vector arithmetic logic unit and convolution unit; VALU has parallel computing capability and can perform vector addition, subtraction and multiplication operations to support instruction set extension; convolution unit CUU handles 3×3 convolution operations and contains dedicated hardware logic to perform element-by-element multiplication and accumulation operations to accelerate the convolution calculation process; S5: Design a control unit for parsing and executing extended instructions to ensure that control signals accurately schedule the operations of VALU and CUU; ensure that data transfer between register files, VALU, and CUU has sufficient bandwidth and low latency when executing extended instructions to avoid performance bottlenecks and improve overall efficiency; S6: Use simulation tools to perform functional simulation and verification on the new processor design to ensure that each hardware unit and new instructions can run correctly in various scenarios; write comprehensive test cases to verify the correctness and performance of the new design, evaluate its speed and efficiency improvement when performing 3×3 convolution calculations, find out the improvements compared to the traditional design and make corresponding optimization adjustments; S7: In the optimization and iteration phase, the hardware design and instruction set are further optimized based on the simulation and verification results; To ensure efficiency and reliability when processing deep learning and image processing tasks, ultimately achieving optimal performance.

2. The RISC-V-based AI processor design method for small convolution kernel acceleration as claimed in claim 1, characterized in that: The step S2 specifically includes: A vector register stack containing 32 96-bit registers is designed to support the storage of sufficient convolution kernel data and intermediate operation results; each vector register consists of multiple elements for parallel operations; the vector register capacity is determined to be 32 according to the encoding format of RISC-V vector instructions, and the register addresses from top to bottom are v1, v2, v3...v32 respectively. When processing three-dimensional vectors, the depth of each vector register is set to 3, storing the vector components in the x, y, and z directions respectively; each register should support fast read and write operations to meet the needs of parallel computing; the register layout is optimized to support the data flow requirements of 3×3 convolution operations, facilitating parallel loading, calculation and storage of data.

3. The RISC-V-based AI processor design method for small convolution kernel acceleration as claimed in claim 1, characterized in that: The step S3 specifically includes: Expand the RISC-V instruction set and add vector addition, subtraction, and multiplication instructions to support common vector operations; design special instructions for convolution operations to simplify programming complexity and speed up execution efficiency; Encode the new instructions to ensure compatibility with existing RISC-V instructions; assign specific `opcode` and `func3` fields to each type of instruction; Vector addition instruction: `opcode` is 1010111, `func3` is 000; element-by-element addition is performed on corresponding components of two vectors, and the result is stored in the destination register; Vector subtraction instruction: `opcode` is 1010111, `func3` is 001; it implements element-by-element subtraction of two vectors, and the result is stored in the destination register; Vector multiplication instruction: `opcode` is 1010111, `func3` is 100; calculates the dot product of two vectors and outputs a scalar, which is suitable for deep learning and graphics processing; Convolution operation instruction: `opcode` is 1010111, `func3` is 010; it calculates the convolution of two 3×3 arrays and outputs a scalar, which is suitable for deep learning and graphics processing.

4. The RISC-V-based AI processor design method for small convolution kernel acceleration according to claim 1, characterized in that: The step S4 specifically includes: Implement VALU to handle parallel vector operations; VALU receives operands from vector registers to perform specified addition, subtraction, and multiplication operations and quickly returns the results; design the logic of the arithmetic unit to fully utilize hardware parallelism and accelerate operations; Design and implement the convolution operation unit, including the input data buffer and the convolution kernel buffer, which are used to store and process the data of 3×3 convolution; ensure that the CUU can efficiently perform element-by-element product and sum operations and send the results to the register.

5. The RISC-V-based AI processor design method for small convolution kernel acceleration according to claim 1, characterized in that: The step S5 specifically includes: Design the vector instruction control unit to parse and manage the execution of vector instructions, schedule control signals to drive VALU and register file operations, and ensure that vector instructions can be executed efficiently and in parallel; define control logic to coordinate the execution order and data flow of extended instructions; optimize data path design to support high-bandwidth data transmission and minimize waiting time; ensure smooth data flow between register files, VALUs, and CUUs without bottlenecks; The convolution unit has a convolution kernel buffer and an input data buffer, which are dedicated to storing the matrix to be processed and the convolution kernel weights. The data flow to the VALU or dot product operator is determined by instruction decoding. The parallel processing of the convolution operation unit allows multiple data windows to be processed in parallel, and element-by-element multiplication and summation are performed with the convolution kernel. The final result is stored in a dedicated buffer.

6. The RISC-V-based AI processor design method for small convolution kernel acceleration according to claim 1, characterized in that: The step S6 specifically includes: With the help of the Quartus II simulation tool, functional verification is performed by observing the output waveform and expected results. The correctness of vector instructions and convolution operations is verified in particular. A wide range of test cases are written to ensure that new instructions and hardware units perform consistently in a variety of scenarios. The computing performance of the AI ​​processor is compared with the traditional RISC-V architecture to evaluate its acceleration effect when performing 3×3 convolution kernel calculations. Performance bottlenecks are identified through test results, and corresponding adjustments and optimizations are made.

7. The RISC-V-based AI processor design method for small convolution kernel acceleration according to claim 1, characterized in that: The step S7 specifically includes: Based on simulation and test feedback, the hardware design and instruction set are adjusted to further improve computing performance and efficiency. Through multiple iterations, the efficient performance and reliability of the AI ​​processor in deep learning and image processing applications are ensured.

Images