A Chisel-based parameterized hardware neural network and its construction method

Abstract

The present invention discloses a method for constructing a parameterized hardware neural network based on Chisel, the method comprising: using the Tensorflow deep learning framework to train a neural network, and after the neural network is trained, using a written Python script to derive type parameters, configuration parameters, and weight parameters of each layer of the neural network; passing the type parameters, configuration parameters, and weight parameters of each layer of the derived neural network into a hardware neural network layer module constructor written in Chisel, obtaining a corresponding hardware neural network layer and performing a simulation test; connecting the obtained hardware neural network layers using a <> operator to form an overall hardware neural network and perform a simulation test. The hardware neural network disclosed by the present invention is optimized in terms of resource occupancy, reduces hardware resource requirements, reduces module latency, and uses fixed-point numbers as data representation to reduce data complexity and hardware circuit power consumption.

Description

Technical Field

[0001] The invention belongs to the field of computer technology and relates to a parameterized hardware neural network based on a novel hardware description language Chisel and a construction method thereof. Background Art

[0002] Artificial intelligence is in the third wave of development. Driven by new technologies such as the Internet, distributed computing, and the Internet of Things, the development of this field has entered a new stage of overall advancement of various disciplines and chain breakthroughs in related technologies. Artificial neural network ANN is a direction of artificial intelligence and is applied in the fields of medicine, economy, transportation, etc. It has strong learning ability. At the same time, the ANN algorithm framework is already very mature. These frameworks use highly abstract APIs for calling, and do not need to pay attention to the underlying complex algorithms. The framework will also combine the specific hardware environment to accelerate the operation at the hardware level to improve the training speed of the model. The ease of use and efficiency of the framework have greatly reduced the learning threshold, and more and more application projects have adopted neural network technology.

[0003] In the embedded field, processor performance is relatively weak, and most of them use a single-core architecture. Software runs on general-purpose processors for serial processing, and the processor takes time to decode the binary instructions of the software and read and write data. Neural networks require matrix operations between layers, and the parallelism is not high when running on software, and the reasoning speed is slow, so it cannot be used in high-real-time embedded systems.

[0004] FPGAs are capable of parallel processing and support repeated erasure, making them ideal for studying the hardware implementation of neural networks. At the same time, as FPGAs have been on the market for longer and longer, companies are increasingly relying on third-party IP, which can often be highly parameterized to create complex digital circuits. Therefore, it makes sense to have highly parameterizable ANNs that can be used in different applications. Traditional FPGA development uses hardware description languages ​​for memory transfer level design. Hardware description languages ​​support parameterization mechanisms and can iterate through integers, but due to the low level of abstraction of the language itself, the code generated using parameterization is poorly readable. Chisel is a domain-specific language based on Scala. Using Chisel can ensure that the coding style is similar to that of traditional HDL, and high-level languages ​​can be used for hardware development. This can improve the degree of code reuse during hardware development and the maintainability of the project. Summary of the invention

[0005] In order to solve the deficiencies in the prior art, the purpose of the present invention is to provide a parameterized hardware neural network construction method and system based on a new hardware description language Chisel.

[0006] Aiming at the classic MLP model and CNN model in ANN, the present invention combines the existing deep learning framework Tensorflow and the new hardware description language Chisel to design a hardware neural network module that can be parameterized and configured, and develops a test framework for verification.

[0007] The present invention proposes a Chisel-based parameterized hardware neural network construction method, the method comprising the following steps:

[0008] Step 1: Use the Tensorflow deep learning framework to train a neural network, which includes a multi-layer perceptron model neural network and a convolutional neural network model. The connection order of each layer in the neural network can be defined as needed; after the neural network is trained, use the written Python script to export the type parameters, configuration parameters, and weight parameters of each layer of the neural network;

[0009] Step 2: Pass the type parameters, configuration parameters, and weight parameters of each layer of the neural network derived in step 1 into the hardware neural network layer module constructor written in Chisel to obtain the corresponding hardware neural network layer and perform simulation testing;

[0010] Step 3: According to the connection order between the layers in the neural network defined as needed in step 1, the hardware neural network layers obtained in step 2 are connected using the <> operator to form the overall hardware neural network and perform simulation testing.

[0011] In step 1, the python script converts the weight parameters into a fixed-point representation, and customizes the number of fixed-point digits, decimal places, and integer places; the type parameters include fully connected layer, convolutional layer, expansion layer, pooling layer, and output layer; the configuration parameters include the number of neurons in the layer, the input data size, and the output data size; the weight parameters include neuron weights and neuron offsets.

[0012] In step 2, the hardware neural network module constructor will first determine which neural network layer to construct based on the type parameter, then set the corresponding hardware neural network module based on the configuration parameters, and finally assign the weight parameters to the registers in the hardware neural network module; the hardware neural network module includes fully connected layer, convolutional layer, expansion layer, pooling layer, and output layer hardware modules.

[0013] In step 2, a test framework is developed for each type of hardware neural network layer obtained, and the test framework is used to perform simulation tests on the hardware neural network layer to obtain the output signal and delay of the hardware module.

[0014] In step 3, a test framework is developed for the obtained overall hardware neural network, and the test framework is used to perform simulation tests on the overall hardware neural network to obtain the accuracy and latency of the overall hardware neural network.

[0015] The present invention uses Chisel for design and implements a hardware neural network that can be parameterized and configured. It includes a dense layer, a flatten layer, a convolution layer, a pooling layer, and an output layer. Each module is composed of the most basic hardware operators, and the language characteristics of Chisel are used to generate the corresponding hardware operation tree according to the configuration parameters. The specific functions of each module are as follows:

[0016] The basic unit in the fully connected layer and the convolutional layer is the neuron, which is an abstraction of the perceptron algorithm model. Its function is to multiply the input data and the weight data in sequence, then accumulate the multiplication results, and finally perform the neuron activation function operation on the accumulated results to obtain the neuron output;

[0017] The fully connected layer contains one or more neurons. This layer will calculate the input data with each neuron in the layer separately, and merge the output of each neuron in the layer as its own output; Figure 2 In the example, the weight ROM stores the weight parameters corresponding to each neuron and is directly connected to the corresponding neuron, and is directly read when performing calculations; the offset ROM stores the offset parameters corresponding to each neuron and is directly connected to the corresponding neuron, and is directly read when performing calculations;

[0018] Convolutional layer, the number of neurons contained in it is the product of the length of the convolution kernel and the width of the convolution kernel. For a given input data, the convolution kernel will take data from the input data equal to the number of neurons for calculation at a time; Figure 3 In the process, the weight rom stores the weight parameters corresponding to each neuron, and is directly connected to the corresponding neuron, and is directly read when performing calculations; the offset rom stores the offset parameters corresponding to each neuron, and is directly connected to the corresponding neuron, and is directly read when performing calculations; after the calculation is completed, data will continue to be taken until all the data is processed, and finally the intermediate calculation results will be collected as its own output;

[0019] The expansion layer does not contain any neurons. The input of the expansion layer is multiple sets of data. This layer converts multiple sets of data into one layer as its own output. The function of the expansion layer is mainly to adjust the data format to facilitate transmission to the next layer of the network.

[0020] The pooling layer does not contain any neurons. For a given input data, the pooling layer will take data with the same pooling kernel size from the data each time, calculate the mean or maximum value in the data, and then continue to take data after the operation is completed until all the data is processed. Finally, the intermediate operation results are collected as their own output;

[0021] The output layer does not contain any neurons. For a given input data, the output layer will select the data with the largest value in the input data and use the subscript position of the largest data as its own output.

[0022] Based on the implemented hardware unit, the present invention proposes an optimization scheme for resource occupation. By optimizing resource occupation and pipeline control, the hardware resources required for the convolution layer are reduced, and the delay of the fully connected layer and the convolution layer hardware module is reduced; fixed-point numbers are used as the data representation form in the circuit, which reduces the complexity of the data, reduces the power consumption of the hardware circuit, and saves hardware resources. Due to parameterized generation, users can decide the number of fixed-point digits, the number of decimal digits, and the number of integer digits by themselves; the convolution layer is added with input and output caches, the number of neurons in the convolution layer is reduced, and the area of ​​the hardware circuit is reduced; handshake signals are added for coordination between modules, and each hardware module has a handshake signal for input data and a handshake signal for output data. For the handshake signal of input data: when the module is in an idle state, the signal will be set to Ready, waiting for other modules to send data; when the module is in an operating state, the signal will be set to Busy, and data sent by other modules will not be accepted. For the handshake signal of output data: when the module is in an operating state, the signal will be set to Busy, and data will not be sent to other modules; when the module is in an idle state, the signal will be set to Ready, and data will not be sent to other modules.

[0023] The present invention develops a test tool using the characteristics of Chisel. In the process of building a hardware neural network, the test tool is used to perform functional simulation tests on each hardware neural network layer in the network and the overall hardware neural network to ensure that the functions of each hardware neural network module are correct. The tool generates parameter reports and test samples for each hardware neural network module, and the parameter report describes the interface and operation cycle of the model. The test framework supports the use of high-level languages ​​for hardware simulation testing, and when the entire hardware neural network is tested, the accuracy of the network can be calculated.

[0024] The present invention also provides a Chisel-based parameterized hardware neural network constructed by the above-mentioned construction method; the hardware neural network includes a multilayer perceptron model and a convolutional neural network model; the multilayer perceptron model includes two fully connected layers and an output layer, and the convolutional neural network model includes a convolutional layer, a pooling layer, an expansion layer, a fully connected layer and an output layer.

[0025] The Chisel-based parameterized hardware neural network proposed in the present invention can be applied in the Internet of Things, embedded systems, and edge computing.

[0026] The present invention also proposes a system, comprising: a memory and a processor; a computer program is stored in the memory, and when the computer program is executed by the processor, the Chisel-based parameterized hardware neural network construction method described in the present invention is implemented.

[0027] The present invention also proposes a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the Chisel-based parameterized hardware neural network construction method described in the present invention is implemented.

[0028] The beneficial effects of the present invention include: the hardware neural network constructed by the present invention is optimized in terms of resource occupation, reduces the need for hardware resources, reduces module delay, and uses fixed-point numbers as data representation to reduce data complexity and hardware circuit power consumption. Taking the handwritten digit recognition network as an example, using the hardware modules and tools implemented by the present invention, the final network accuracy is 93.6%, the hardware module network frequency is 103MHz, the power consumption is 8.758W, and it takes 1.436μs to complete the recognition of an image. The implemented hardware network has practical application value and can meet high real-time and low power consumption application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is a neuron hardware design diagram in the parameterized hardware neural network based on Chisel of the present invention.

[0030] Figure 2 It is the hardware design diagram of the fully connected layer in the parameterized hardware neural network based on Chisel of the present invention.

[0031] Figure 3 It is a hardware design diagram of the convolution layer in the Chisel-based parameterized hardware neural network of the present invention.

[0032] Figure 4 It is a convolutional layer operation sequence diagram in the Chisel-based parameterized hardware neural network of the present invention.

[0033] Figure 5It is a structural diagram of a convolutional neural network in a Chisel-based parameterized hardware neural network of the present invention.

[0034] Figure 6 This is a diagram of the simulation results of the constructed hardware convolutional neural network in the Chisel-based parameterized hardware neural network of the present invention.

[0035] Figure 7 It is a pipeline diagram of the convolution layer in the Chisel-based parameterized hardware neural network of the present invention. DETAILED DESCRIPTION

[0036] The present invention is further described in detail with reference to the following specific examples and drawings. The process, conditions, experimental methods, etc. for implementing the present invention, except for the contents specifically mentioned below, are all common knowledge and common common sense in the art and are not particularly limited by the present invention.

[0037] The multi-layer perceptron model is the foundation of deep learning. The model is composed of multiple fully connected layers, each of which is composed of multiple neurons. A single neuron is also called a perceptron. The invention of the perceptron was inspired by the structure of biological neuron cells and the way cells are connected.

[0038] Convolutional neural networks are very good at image tasks. Similar to the multi-layer perceptron model, the convolutional neural network model consists of a certain number of network layers in sequence. The first layer is called the convolution layer, which reads the input image and performs convolution operations to obtain the feature map. The pooling layer performs a reduction operation on the feature map to reduce the area of ​​the feature map. This process is also called downsampling. The image after the reduction operation is input to the fully connected layer for classification.

[0039] Therefore, the present invention constructs a hardware neural network including a multi-layer perceptron model and a convolutional neural network model by establishing a fully connected layer, an expansion layer, a convolutional layer, a pooling layer and an output layer.

[0040] Chisel is a domain-specific language based on Scala. Using Chisel can ensure that the coding style is similar to traditional HDL, and high-level languages ​​can be used for hardware development. This can improve the code reuse degree and project maintainability during hardware development. After writing the Chisel code, a FIRRTL file will be generated, and FIRRTL will be compiled into Verilog code. You can also directly set the backend of the FIRRTL compiler for simulation verification. The most important point is that as long as the syntax conditions of Chisel are met and the compilation does not report an error, the generated hardware module must be synthesizable. At the same time, Chisel code supports logic gate style, data flow style, and functional description style, and has the same experience as traditional hardware development languages. Chisel can be used to test-drive development (TDD) of hardware. The built-in PeekPokeTester is similar to Junit, which can perform unit testing on the designed hardware modules. After the test cases are written, they can be run directly and waveforms can be obtained without writing testbench files. The characteristics of the Scala language can also be used within the test to generate test samples more flexibly.

[0041] Therefore, the present invention adopts Chisel for development and design, which allows hardware developers to quickly get started and apply neural network hardware modules to their SoCs. By exporting the corresponding configuration parameters and weight parameters of the model trained by the deep learning framework, the corresponding hardware neural network module can be generated using the parameters, and the hardware module has lower power consumption.

[0042] The Chisel-based parameterized hardware neural network construction method of the present invention includes the following contents:

[0043] 1. Neuron hardware design:

[0044] In the present invention, the neuron first multiplies the input signal by the corresponding weight, then accumulates the products, and finally outputs the accumulated result after being operated by the ReLU function. The structure is as follows: Figure 1 As shown. The neuron calculation in the present invention uses 16-bit fixed-point numbers, and the fixed-point number multiplication uses the DSP (Digital Signal Process) in the FPGA. The multiplication results are summed through the addition tree. Since the result of the multiplier is a 32-bit fixed-point number, the result needs to be shifted to ensure that the output data is also 16 bits. The number of neuron inputs is N, corresponding to N multipliers calculating the product of the input and the weight, and the number of layers is log2 N The adder tree is used for accumulation, 1 shifter and 1 ReLU activation unit. The multiplier obtains the result in 1 cycle, and the adder tree log2 NThe operation cycle of a neuron is log2 N +3.

[0045] Neurons can be configured with the number of neuron inputs inputNo, the fixed-point format dtype, and the number of decimal places in the fixed-point number frac_bit. The parameter details are shown in Table 1. Neurons contain excitation signals neuron_rst and neuron_clk, input data neuron_dataIn with inputNo data width dtype, input weights neuron_weights with inputNo data width dtype, an input offset neuron_bias with a data width of dtype, and an output data neuron_dataOut with a data width of dtype. The neuron interface is shown in Table 2.

[0046] Table 1 Neuron configurable parameters

[0047]

[0048] Table 2 Neuron ports

[0049]

[0050] 2. Fully connected layer hardware design:

[0051] The fully connected layer module in the present invention is as follows Figure 2 As shown in the figure, the trained weights and offsets are stored in ROM (Read Only Memory), and the input and output implement handshake signals for control between layers. The ROM is implemented using the distributed storage unit of FPGA, which can return data in a single cycle. The fully connected layer has M inputs, N outputs, the offset memory is 16×N, the weight memory is 16×N×M, and the processing cycle is log2 M +3.

[0052] The configurable parameters of the fully connected layer are: fixed-point format (dtype), number of decimal places (frac_bit), input data size (inputNo), output data size (outputNo), initial value of bias value (bias), initial value of weight value (weights). See Table 3 for parameter details.

[0053] The fully connected layer contains the excitation signals dense_clk and dense_rst. The input handshake signals are dense_dataIn_valid and dense_dataIn_ready. The input data is dense_dataIn_data, which has inputNo data of width dtype. The output handshake signals are dense_dataOut_valid and dense_dataOut_ready. The output data is dense_dataOut_data, which has outputNo data of width dtype. For detailed interface signals, see Table 4.

[0054] Table 3 Fully connected layer configuration parameters

[0055]

[0056]

[0057] Table 4 Fully connected layer ports

[0058]

[0059] 3. Convolutional layer hardware design:

[0060] In the design of the convolution layer, the present invention adds a cache and related timing control signals to reduce the number of neurons. The configurable parameters of the convolution layer are shown in Table 5, and the supported parameters are: fixed-point format (dtype), fractional digits (frac_bit), number of convolution kernels (filterBatch), vertical step size (strideHeight), horizontal step size (strideWidth), convolution kernel width (filterWidth), convolution kernel height (filterHeight), input image width (dataWidth), input image height (dataHeight), weight initial value (weights), and compilation initial value (bias).

[0061] For a given input image width, horizontal stride, and convolution kernel width, the output width of the convolution layer can be calculated as shown in Equation 1. For a given input image height, vertical stride, and convolution kernel height, the output height of the convolution layer can be calculated as shown in Equation 2. The convolution layer output size calculation process is shown in Equation 3.

[0062]

[0063]

[0064] outputSize=filterBatch×outputHeight×outputWidth#(3)

[0065] Table 5 Convolutional layer configuration parameters

[0066]

[0067] The convolution layer contains excitation signals conv_clk and conv_rst. The input handshake signals are conv_dataIn_valid and conv_dataIn_ready. The input data is conv_dataIn_data, which has dataWidth width of dtype. The output handshake signals are conv_dataOut_valid and conv_dataOut_ready. The output data signal is conv_dataOut_data, which has outputSize width of dtype. The interface of the convolution layer module is shown in Table 6.

[0068] Table 6 Convolutional layer interface

[0069]

[0070] The internal structure of the convolutional layer is as follows: Figure 3 The convolution line reads the image from the cache, reads the parameters from the memory, and writes the result to the cache. The signal sequence is as follows Figure 4 shown.

[0071] 4. Pooling layer design:

[0072] The function of the pooling layer in the present invention is to first segment the image according to the pooling kernel size, and perform a reduction operation on each segmented image. The two common reduction operations are taking the maximum value and taking the average value. The output result needs log2 N cycles, where N is the size of the segmented image. The width of the pooling layer input data and the size of the output feature map are shown in Equation 4 and Equation 5. The pooling layer performs reduction operations, and there is no need to configure the number of decimal places and read weights and offsets. The configurable parameter list is shown in Table 7. The fixed-point format (dtype), input feature map width (dataWidth), input feature map height (dataHeight), number of pooling kernels (filterBatch), pooling kernel width (filterWidth), and pooling kernel height (filterHeight) of the pooling layer module can be configured.

[0073] inputSize=dataWidth×dataHeight×filterBatch#(4)

[0074]

[0075] Table 7 Pooling layer configuration parameters

[0076]

[0077] The pooling layer interface is shown in Table 8. The excitation signals of the pooling layer are pooling_clk and pooling_rst. The input data handshake signals are pooling_dataIn_valid and pooling_dataIn_ready. The input data signal is pooling_dataIn_data, which contains inputSize data with a width of dtype. The output data handshake signals are pooling_dataOut_valid and pooling_dataIn_ready. The output data signal is pooling_dataOut_data, which contains outputSize data with a width of dtype.

[0078] Table 8 Pooling layer interface

[0079]

[0080] 5. Output layer design:

[0081] Since Softmax involves exponential operations, division operations, and accumulation operations, it is very expensive to implement it in hardware, so the design of the present invention uses comparators instead. After the output layer obtains the input, it performs a reduction operation through the comparator tree, log2 N The maximum value of the input value is obtained in cycles, and the index of the maximum value is output. The configurable parameters are shown in Table 9, and the fixed-point format (dtype) and the number of inputs (inputNo) can be configured.

[0082] Table 9 Output layer configuration parameters

[0083]

[0084] The output layer excitation signals are output_clk and output_rst. The input data signal is output_dataIn_data, which contains inputNo data of width dtype. The handshake signals of the input data are output_dataIn_valid and output_dataIn_ready. The output data signal is output_dataOut_data, which contains a width of log2 inputNo The handshake signals are output_dataOut_valid and output_dataOut_ready. The interface is shown in Table 10.

[0085] Table 10 Output layer interface

[0086]

[0087] 6. Convolutional Neural Networks:

[0088] The present invention uses the Tensorflow deep learning framework to train a convolutional neural network for the handwritten digit recognition task. The network architecture is shown in the attached figure. Figure 5 The parameters of each layer of the convolutional neural network are shown in Table 11. Since the image of handwritten digits is black and white, the input layer has only one channel and the size of the image is 28×28 pixels. The output of the convolutional layer is 24×24×3. The output of the pooling layer is 12×12×3. The expansion layer sends the output of the pooling layer to the fully connected layer for classification.

[0089] Table 11 Parameters of each layer of convolutional neural network

[0090]

[0091]

[0092] The hardware neural network architecture of the present invention is shown in Table 12. The number of cycles required for each module will be described below. The convolution layer uses pipeline technology. Figure 7 In the convolution layer, a single convolution row requires 1800 multipliers and 72 adder trees of height 5. For the operation of one row of output results: first, 5 cycles are required to fill the input buffer, then 9 cycles of convolution row operation are performed, and finally 1 cycle is spent to write the result to the output buffer. A total of 130 clock cycles are required.

[0093] The reduction operation in the maximum pooling layer is completed by the MaxMux module, which generates a comparator tree with a height of 2 and a number of comparators of 3. It takes 2 cycles to perform the pooling operation, and the output size is 12 × 12 × 3. 432 comparator trees with a height of 2 are required, and the total number of comparators used is 1296.

[0094] The fully connected layer requires 12 cycles to complete the operation. The fully connected layer requires 4320 multipliers and 10 addition trees of height 9.

[0095] The height of the comparator tree in the output layer is 4, and the operation cycle is 4. For a picture, after entering the image cache, it takes a total of 148 clock cycles to complete the recognition of the picture.

[0096] Table 12 Hardware convolutional neural network module architecture

[0097]

[0098] Example

[0099] This invention proposes a Chisel-based parameterized hardware neural network construction method. The following is its code implementation part (important parts):

[0100] Users can use the existing deep learning framework to train a neural network model for a specific task, export the configuration parameters and weight parameters, and then use the system of the present invention to generate a corresponding hardware neural network model, thereby meeting low power consumption and high real-time application scenarios.

[0101] Code 1 is the code implementation of the neuron hardware module, which is similar to Figure 1 Corresponding. The neuron first calculates the product of the input data and the input weight, then accumulates the products, and finally adds the offset and performs the ReLU activation function. Since the neuron is called internally, the handshake signal is not implemented.

[0102]

[0103]

[0104] Code 1

[0105] Code 2 is the code implementation of the fully connected layer hardware module. Figure 2 Corresponding. A neuron list is created inside the fully connected layer, the input data is connected to each neuron in the list, and the output of the neuron is connected to its own output signal. The module calculates the operation delay inside and changes the handshake signal according to the delay.

[0106]

[0107]

[0108] Code 2

[0109] Code 3 is the code implementation of the convolutional layer hardware module, which is similar to Figure 3 Corresponding. A convolutional row consisting of several neurons, an input buffer, and an output buffer are created inside the convolutional layer. The input data is connected to the input buffer. When the buffer is equal to the convolution kernel, the data is given to the convolution row for operation, and the operation result is stored in the output buffer. The module calculates the operation delay inside and changes the handshake signal according to the delay.

[0110]

[0111]

[0112]

[0113] Code 3

[0114] Code 4 is the code implementation of the convolutional neural network. The network structure is similar to Figure 5 Corresponding, the specific parameter configuration corresponds to Table 12. The convolutional neural network hardware module contains a convolution layer, a pooling layer, an expansion layer, a fully connected layer and an output layer, and the layers are connected by <> symbols.

[0115]

[0116]

[0117]

[0118] Code 4

[0119] Code 5 is a Python script that extracts the offset and weight parameters of the convolutional layer and fully connected layer in the Tensorflow neural network and saves them in a csv file.

[0120]

[0121]

[0122] Code 5

[0123] Code 6 uses the written hardware test framework to test the convolutional neural network hardware module. The test framework reads the weight parameters parsed by the Python script and generates the corresponding neural network hardware module. Then it reads the image and label, feeds the neural network, obtains the network output, calculates the network accuracy, and saves the results in a csv file.

[0124]

[0125]

[0126]

[0127]

[0128] Code 6

[0129] The present invention studies the multi-layer perceptron model and the convolutional neural network model, focusing on the analysis of the operations involved in the forward reasoning process of the neural network. On this basis, a highly parameterized hardware module is designed. The specific contents are as follows: 1. A parameterizable hardware neural network is implemented. It includes a fully connected layer (Dense Layer), an unfolded layer (FlattenLayer), a convolution layer (Convolution Layer), a pooling layer (Pooling Layer) and an output layer (OutputLayer). A handshake signal is added to the module for coordination to support the transmission of multiple data. 2. The hardware neural network module reduces the hardware resources required for the convolution layer and reduces the delay of the fully connected layer and convolution layer hardware modules by optimizing resource occupancy and pipeline control; fixed-point numbers are used as the data representation form in the circuit, which reduces the complexity of the data, reduces the power consumption of the hardware circuit, and saves hardware resources; the convolution layer adds input and output caches to reduce the number of neurons in the convolution layer; handshake signals are added for coordination between modules, and each hardware module has a handshake signal for input data and a handshake signal for output data. 3. Tools for generating parameter reports and test samples for each module are developed. The parameter report describes the model's interface and operation cycle, and the test framework supports hardware simulation testing using high-level languages.

[0130] The present invention can be used in high real-time, low-power embedded neural network reasoning scenarios. The performance of neural network forward reasoning can be greatly improved. The neural network structure in the present invention is very flexible. Users can customize the neural network including the fully connected layer, the expansion layer, the convolution layer, the pooling layer, and the output layer. After training, the corresponding hardware neural network is generated through the parameter extraction tool and the hardware generator provided by the present invention.

[0131] The protection content of the present invention is not limited to the above embodiments. Without departing from the spirit and scope of the present invention, changes and advantages that can be thought of by those skilled in the art are included in the present invention and are protected by the attached claims.

Claims

1. A method for constructing a parameterized hardware neural network based on Chisel, characterized in that: The method comprises the following steps: Step 1: Use the Tensorflow deep learning framework to train a neural network, which includes a multi-layer perceptron model neural network and a convolutional neural network model. After the neural network is trained, use the written Python script to export the type parameters, configuration parameters, and weight parameters of each layer of the neural network; In step 1, the python script converts the weight parameters into a fixed-point representation, and customizes the number of fixed-point digits, decimal places, and integer places; the type parameters include fully connected layer, convolutional layer, expansion layer, pooling layer, and output layer; the configuration parameters include the number of neurons in the layer, the input data size, and the output data size; the weight parameters include neuron weights and neuron offsets; The hardware neural network module reduces the hardware resources required for the convolution layer and reduces the latency of the fully connected layer and convolution layer hardware modules by optimizing resource occupancy and pipeline control; adopts fixed-point numbers as the data representation form in the circuit, reduces the complexity of the data, reduces the power consumption of the hardware circuit, and saves hardware resources; adds input and output caches to the convolution layer to reduce the number of neurons in the convolution layer; adds handshake signals for coordination between modules, and each hardware module has a handshake signal for input data and a handshake signal for output data; Step 2: Pass the type parameters, configuration parameters, and weight parameters of each layer of the neural network derived in step 1 into the hardware neural network layer module constructor written in Chisel to obtain the corresponding hardware neural network layer and perform simulation testing; Step 3: Use the <> operator to connect the hardware neural network layers obtained in step 2 to form the overall hardware neural network and perform simulation testing.

2. The Chisel-based parameterized hardware neural network construction method according to claim 1, characterized in that: The convolutional neural network structure includes a fully connected layer, a convolutional layer, an expansion layer, a pooling layer, and an output layer. The multi-layer perceptron neural network structure includes a fully connected layer and an output layer. The connection order of the network structure is defined as needed.

3. The Chisel-based parameterized hardware neural network construction method according to claim 1, characterized in that: In step 2, the hardware neural network module constructor will first determine which neural network layer to construct based on the type parameter, then set the corresponding hardware neural network module based on the configuration parameters, and finally assign the weight parameters to the registers in the hardware neural network module; the hardware neural network module includes fully connected layer, convolutional layer, expansion layer, pooling layer, and output layer hardware modules.

4. The Chisel-based parameterized hardware neural network construction method according to claim 1, characterized in that: In step 2, a test framework is developed for each type of hardware neural network layer obtained, and the test framework is used to perform simulation tests on the hardware neural network layer to obtain the output signal and delay of the hardware module.

5. The Chisel-based parameterized hardware neural network construction method according to claim 1, characterized in that: In step 3, a test framework is developed for the obtained overall hardware neural network, and the test framework is used to perform simulation tests on the overall hardware neural network to obtain the accuracy and latency of the overall hardware neural network.

6. The Chisel parameterized hardware neural network construction method according to claim 3, characterized in that: The fully connected layer and the convolutional layer contain neurons, which are the basic units in the fully connected layer and the convolutional layer, and are an abstraction of the perceptron algorithm model. The function of the neurons is to multiply the input data and the weight data in sequence, and then accumulate the multiplication results, and finally perform a neuron activation function operation on the accumulated results to obtain the output of the neuron; The fully connected layer includes one or more neurons. The layer will perform operations on the input data with each neuron in the layer respectively, and merge the output of each neuron in the layer as its own output; The number of neurons in the convolution layer is the product of the length of the convolution kernel and the width of the convolution kernel. For a given input data, the convolution kernel will take out data equal to the number of neurons from the input data for calculation at one time. After the calculation, the above operation is repeated until all the data is processed. Finally, the intermediate calculation results are collected as their own output; The expansion layer does not contain any neurons. The input of the expansion layer is multiple sets of data. The layer converts the multiple sets of data into one layer as its own output. The expansion layer is used to adjust the data format to facilitate transmission to the next layer of the network. The pooling layer does not contain any neurons. For a given input data, the pooling layer will take data with the same pooling kernel size from the data each time, calculate the mean or maximum value in the data, and repeat the above operation after the operation is completed until all the data is processed. Finally, the intermediate operation results are collected as their own output; The output layer does not contain any neurons. For given input data, the output layer will select the data with the largest value in the input data and use the subscript position of the largest data as its own output.

7. The Chisel-based parameterized hardware neural network construction method according to claim 1, characterized in that: For the handshake signal of input data: when the module is in idle state, the signal will be set to Ready and wait for other modules to send data; when the module is in operation state, the signal will be set to Busy and will not accept data sent by other modules; for the handshake signal of output data: when the module is in operation state, the signal will be set to Busy and will not send data to other modules; when the module is in idle state, the signal will be set to Ready and send data to other modules.

8. The Chisel-based parameterized hardware neural network construction method according to claim 1, characterized in that: A Chisel-based simulation test tool was developed; during the construction process, the tool was used to simulate and verify the generated hardware neural network layer and hardware neural network to ensure the correct function of each hardware neural network module; the simulation test tool generated parameter reports and test samples for each hardware neural network module; the parameter report described the interface and operation cycle of the model, and supported the use of high-level language features to write test samples for hardware simulation.

9. A Chisel-based parameterized hardware neural network constructed by the construction method according to any one of claims 1 to 8; the hardware neural network includes a multilayer perceptron model and a convolutional neural network model; the multilayer perceptron model includes two fully connected layers and an output layer, and the convolutional neural network model includes a convolutional layer, a pooling layer, an expansion layer, a fully connected layer and an output layer.

10. Application of the Chisel-based parameterized hardware neural network as claimed in claim 9 in the Internet of Things, embedded systems, and edge computing.

11. A hardware system for implementing the construction method according to any one of claims 1 to 8, characterized in that: include: A memory and a processor; a computer program is stored in the memory, and when the computer program is executed by the processor, the method according to any one of claims 1 to 8 is implemented.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images