An ai hardware accelerator for neural network computation

Abstract

An AI hardware accelerator for neural network calculation comprises: an input module for obtaining configuration data and calculation data; a control module for outputting configuration parameters and state enable signals according to the configuration data; an address generation module for calculating data access addresses and control signals required for current time calculation according to the configuration parameters and the state enable signals; an SRAM read-write selection module for reading the calculation data from an on-chip memory according to the data access addresses and the control signals; a bit shift register module for performing padding supplement and bit width alignment operations on the calculation data; a calculation array module for performing calculation on the supplemented and aligned data and outputting calculation results; a backhaul module for being responsible for format adaptation, temporary storage and write-back control of the calculation results; and an output module for outputting final calculation results. Effects: the AI hardware accelerator can provide overall performance of neural network calculation, and can optimize on-chip storage and reduce off-chip access.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence hardware acceleration technology, and in particular to an AI hardware accelerator for neural network computing. Background Technology

[0002] Traditional CNN accelerators generally suffer from the following constraints in their design: First, padding operations are inefficient and have high hardware overhead, often relying on software preprocessing or dedicated hardware logic. They typically only support simple modes such as zero padding, making it difficult to achieve efficient pipelining with computation units. Second, data storage and access bottlenecks are prominent, manifested in frequent off-chip accesses, rigid storage layouts, high data alignment overhead, and redundant weight loading, limiting data reuse and parallel computing potential. Third, address generation and control logic are complex and inflexible, often based on fixed addressing modes and multi-layered loops, making it difficult to adapt to dynamically changing padding requirements and diverse network structures, resulting in low hardware utilization. Finally, the system framework has high coupling and insufficient scalability, with computational and storage resources tightly bound together, severe coupling between control flow and data flow, and a static configuration mechanism lacking unified abstraction, making it costly to support new operators and difficult to adapt to rapidly evolving algorithm requirements.

[0003] Disadvantages of existing technology:

[0004] (1) Traditional AI accelerators often adopt fixed data flow and separate storage design, which has the problem of "memory wall". Frequent access to off-chip storage leads to low energy efficiency. At the same time, the architecture has poor adaptability and is difficult to flexibly accommodate the computing needs of different types of neural network layers such as convolution, pooling, and fully connected. It cannot dynamically adapt to deployment scenarios of multi-size convolution kernels and multi-network models.

[0005] (2) Traditional filling operations require additional off-chip storage or hardware resources to store the filling data, which increases storage and transmission overhead; and the filling mode is fixed and cannot flexibly support various scenarios such as zero filling and edge filling, thus limiting adaptability. Summary of the Invention

[0006] The present invention provides an AI hardware accelerator for neural network computing, which can improve the overall performance of AI hardware accelerators in performing neural network computing, and can optimize on-chip storage and reduce off-chip access.

[0007] To achieve the above objectives, the present invention provides an AI hardware accelerator for neural network computation, the key of which is: a control module is provided, the control module is connected to an input module, and the input module is connected to an SRAM read / write selection module; the control module is also connected to the SRAM read / write selection module and an address generation module respectively, the address generation module is connected to the SRAM read / write selection module, the SRAM read / write selection module is also connected to a computation array module via a shift register module, the computation array module is connected to a feedback module, and the feedback module is connected to the SRAM read / write selection module and an output module respectively;

[0008] The input module is used to: acquire configuration data and calculation data, then transmit the configuration data to the control module, and transmit the calculation data to the SRAM read / write selection module;

[0009] The control module is used to: output configuration parameters and corresponding status enable signals to the address generation module according to the configuration data, and trigger the address generation process;

[0010] The address generation module is used to: calculate the data access address and control signal required for calculation at the current moment according to the configuration parameters and the status enable signal, and pass them to the SRAM read / write selection module;

[0011] The SRAM read / write selection module is used to: read computational data from the corresponding on-chip memory according to the data access address and control signal, and then pass the read computational data to the displacement register module;

[0012] The displacement register module is used to: fill and supplement the calculated data, perform bit width alignment operations, and then pass the supplemented and aligned data to the calculation array module;

[0013] The computing array module is used to: perform calculations on the supplemented and aligned data, and output the calculation results to the feedback module;

[0014] The return module is responsible for the format adaptation, temporary storage, and write-back control of the calculation results. The return module controls the write-back by determining whether the current layer calculation is the last layer calculation. If it is not the last layer calculation, it drives the SRAM read / write selection module to alternately write the calculation results of the current layer back to the corresponding on-chip memory so that they can be used as the input for the next layer calculation. If it is the last layer calculation, it outputs the calculation results of the current layer to the output module so that they can be used as the final calculation result.

[0015] The output module is used to output the final calculation result.

[0016] Through the above design, this invention obtains the configuration data and computation data required for the current neural network computation from external storage in one go through the input module. Then, the computation data and intermediate data during the computation process are stored in the on-chip memory, which significantly reduces the number of external storage accesses of the AI ​​accelerator chip and improves the system energy efficiency ratio. At the same time, the corresponding neural network configuration data can be obtained from external storage according to the usage requirements, so that the AI ​​accelerator provided by this invention can meet different neural network deployment requirements and adapt to computing tasks in different scenarios, with high flexibility and adaptability.

[0017] The beneficial effects of this invention are:

[0018] (1) Improve computing efficiency: By designing a ping-pong SRAM read / write selection structure and online control signal generation data flow scheduling, the number of off-chip storage accesses is reduced, alleviating the "memory wall" problem and significantly improving the energy efficiency ratio of AI hardware accelerators; the m×m PE computing array realizes highly parallel computing, completing m×m multiplication and accumulation operations in a single clock cycle, thus improving computing speed.

[0019] (2) Enhanced flexibility: With 128-bit configurable parameters, it supports dynamic switching of various parameters such as kernel size, padding type, and activation function, adapting to the deployment requirements of various neural network models such as LeNet5 and VggNet16.

[0020] (3) Reduce padding overhead: Based on the extended window design of the shift register module, multiple modes such as zero padding and edge padding can be achieved without additional off-chip storage, reducing data transmission latency and storage resource consumption. Attached Figure Description

[0021] Figure 1 This is a diagram of the AI ​​hardware accelerator architecture used for neural network computation in the embodiment.

[0022] Figure 2 This is a schematic diagram illustrating the activation value storage method in the embodiment;

[0023] Figure 3 This is a schematic diagram of the weight storage method in the embodiment;

[0024] Figure 4 This is a schematic diagram of the computing array module design in the embodiment;

[0025] Figure 5 This is a flowchart of the counter accumulation process at each level in the address generation module of the embodiment;

[0026] Figure 6 This is a schematic diagram illustrating the calculation method of the computing array module in the embodiment;

[0027] Figure 7This is a schematic diagram of the sliding process of calculating data in the embodiment;

[0028] Figure 8 This is a schematic diagram illustrating the mapping rules for configuration information fields in the embodiment;

[0029] Figure 9 This is a control logic block diagram of the SRAM read / write selection module in the embodiment;

[0030] Figure 10 This is a block diagram illustrating the hardware implementation logic of the activation data in this embodiment.

[0031] Figure 11 The example demonstrates feature map filling and calculation of sliding relationship graphs.

[0032] Figure 12 This is a block diagram of the return logic in the embodiment;

[0033] Figure 13 This is a block diagram illustrating the pooling implementation logic in the embodiment;

[0034] Figure 14 This is a diagram showing the working state transition of the control module in the embodiment;

[0035] Figure 15 This is a diagram of the LeNet5 network architecture in Example 1;

[0036] Figure 16 This is a diagram of the VggNet16 network architecture in Example 2. Detailed Implementation

[0037] The present invention will be further described in detail below with reference to the accompanying drawings and specific examples. The following embodiments or drawings are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0038] like Figure 1 As shown, an AI hardware accelerator for neural network computation includes a control module. The control module is connected to an input module via a decoding module. The input module is connected to an SRAM read / write selection module via a data receiving module. The control module is also connected to both the SRAM read / write selection module and an address generation module. The address generation module is connected to the SRAM read / write selection module. The SRAM read / write selection module is further connected to a computation array module via a shift register module. The computation array module is connected to a feedback module, which is connected to both the SRAM read / write selection module and an output module.

[0039] The input module is used to: acquire configuration data and calculation data, then transmit the configuration data to the control module, and transmit the calculation data to the SRAM read / write selection module;

[0040] The decoding module is used to: decode the configuration data, and then transmit the decoded configuration data to the control module;

[0041] The data receiving module is used to write the calculated data into the SRAM read / write selection module.

[0042] The control module is used to: output configuration parameters and corresponding status enable signals to the address generation module according to the configuration data, and trigger the address generation process;

[0043] The address generation module is used to: calculate the data access address and control signal required for calculation at the current moment according to the configuration parameters and the status enable signal, and pass them to the SRAM read / write selection module;

[0044] The SRAM read / write selection module is used to: read computational data from the corresponding on-chip memory according to the data access address and control signal, and then pass the read computational data to the displacement register module;

[0045] The displacement register module is used to: fill and supplement the calculated data, perform bit width alignment operations, and then pass the supplemented and aligned data to the calculation array module;

[0046] The computing array module is used to: perform calculations on the supplemented and aligned data, and output the calculation results to the feedback module;

[0047] The return module is responsible for the format adaptation, temporary storage, and write-back control of the calculation results. The return module controls the write-back by determining whether the current layer calculation is the last layer calculation. If it is not the last layer calculation, it drives the SRAM read / write selection module to alternately write the calculation results of the current layer back to the corresponding on-chip memory so that they can be used as the input for the next layer calculation. If it is the last layer calculation, it outputs the calculation results of the current layer to the output module so that they can be used as the final calculation result.

[0048] The output module is used to output the final calculation result.

[0049] The configuration data consists of parameter information for each layer of the network, including convolutional kernel size, padding type, and total number of network layers; the calculation data includes weight data and activation data used in the calculation.

[0050] AI hardware accelerators can be used for gas identification tasks or image classification tasks:

[0051] In the gas identification task, the activation data includes the original gas input data and the intermediate gas feature data calculated by each network layer, i.e., the calculation results of each network layer; the output module is used for the gas identification results.

[0052] In the image classification task, the activation data includes the original image input data and the intermediate image feature data calculated by each network layer, i.e., the calculation results of each network layer; the output module is used for the image classification results.

[0053] The overall computational flow of an AI hardware accelerator for accelerating on-chip convolutional neural network computation is as follows:

[0054] P1: The input module obtains relevant computational data and network configuration information from outside the chip and inputs it into the chip.

[0055] P2: The control module reads the configuration data decoded by the decoding module and calculates state enable signals such as convolution enable; the address generation module receives the current enable signal and configuration parameters, calculates the data access address required for the current layer convolution calculation such as activation / weight read address, and valid control signals.

[0056] P3: The SRAM read / write selection module reads activation / weight data from the corresponding on-chip memory based on the address and control signals output by the address generation module. After the data is filled and aligned by the shift register module, it is sent to the computation array module to perform convolution calculation.

[0057] P4: The return module receives the calculated data and returns the calculation results to the storage.

[0058] P5: The SRAM read / write selection module determines whether pooling is required for the returned data based on the network configuration. If pooling is not required for the current layer, this step is skipped.

[0059] P6: Wait for the signal indicating the end of pooling or the signal indicating the end of feedback, and perform statistics to determine whether all calculations of the current layer have been completed. If the calculation of the current layer has been completed, jump to P7; otherwise, jump to P3 to continue the calculation.

[0060] P7: Determine the current computation layer. If it is not the last layer of the convolution, jump to P2 to perform the next layer's computation. If it is the last layer, end all operations and output the current computation result.

[0061] The above process fully presents the overall data flow of on-chip convolutional data network acceleration computation, covering the entire computation chain from data import to result output. The highly parallel convolutional data stream corresponds to stage P3 in the aforementioned process, and its core hardware implementation components are the storage array and the computing array. For stage P4, the computing array outputs Cb×Ma data per cycle, while the bandwidth of the storage unit in a single clock cycle cannot handle this scale of data. Therefore, a dedicated cache needs to be designed to write the computation results back to storage in batches over multiple clock cycles. Stage P5 requires an independent pooling module to perform data pooling operations. Here, the accelerator adopts a "store-first, retrieve-later" logic: the computation results are first completely returned to the storage unit, and then the data is read from storage to perform pooling operations, rather than directly pooling the output results of the computing array and then storing them back. The remaining four stages, P1, P2, P6, and P7, are all controlled by the control module. These four stages can be mapped through the state machine built into the main control module, with each state corresponding to a specific operation process.

[0062] The address generation module is provided with a convolution address generation unit and a pooling address generation unit. The SRAM read / write selection module is also connected to a pooling module, which is connected to the return module. The pooling module is provided with a max pooling unit and an average pooling unit.

[0063] The pooling module is used to perform pooling calculations. If the current layer is configured as a convolution + pooling layer, the calculation data first undergoes convolution calculation through the calculation array module. The convolution calculation result is then sent back to the SRAM read / write selection module via the return module and used as input data for the pooling module. The pooling module then performs pooling calculations, and the calculated pooling result is passed to the return module as the calculation result of the current layer.

[0064] The address generation module is configured from the inside out with the following components: a convolution kernel x-direction counter kx_cnt, an input channel counter n_cnt, a convolution kernel y-direction counter ky_cnt, a weight block counter w_cnt, a row counter r_cnt, and an activation value column block counter a_cnt.

[0065] Among them, the convolution kernel X-direction counter kx_cnt is used to traverse the width of the convolution kernel, the input channel counter n_cnt is used to traverse the input channels, the convolution kernel Y-direction counter ky_cnt is used to traverse the height of the convolution kernel, the weight block counter w_cnt is used to traverse the weight blocks, the row counter r_cnt is used to traverse the rows of the input feature map, and the activation value column block counter a_cnt is used to traverse the column blocks of the input feature map;

[0066] The data access address generation logic of the address generation module is as follows:

[0067] First, the convolution kernel x-direction counter kx_cnt increments in each calculation cycle, traversing the horizontal position of the convolution kernel in the current input channel. After the convolution kernel x-direction counter kx_cnt completes one cycle, the input channel counter n_cnt increments, switching to the next input channel. After all input channels are processed, the convolution kernel y-direction counter ky_cnt increments, moving the convolution window down one row. After the convolution kernel y-direction counter ky_cnt completes the vertical traversal of the entire convolution kernel, the weight block counter w_cnt increments, switching to the next set of output channels. After all output channels at the current spatial position are calculated, i.e., the weight block counter w_cnt completes a full cycle from 0 to w_max-1, the row counter r_cnt increments, moving the convolution window to the next row. Finally, the activation value column block counter a_cnt increments after all rows are processed, switching to the next m-column activation data block. The cascading accumulation calculation process between the counters is as follows: Figure 5 As shown.

[0068] These counters are updated in an inward-outward order: after the X-direction counter of the convolution kernel completes a traversal of one convolution kernel width, the channel counter increments by one; after the channel counter has traversed all channels, the Y-direction counter of the convolution kernel increments by one; and so on, until all dimensions have been traversed. The main read address of the activation data is calculated jointly by the activation value column block counter, row counter, convolution kernel Y-direction counter, channel counter, and channel offset, following the data layout order in storage. Simultaneously, based on the padding type and the current row position, the module determines whether boundary padding is needed and generates the corresponding left and right padding addresses and their valid signals. The read address of the weight data uses a linear incrementing method, incrementing by one for each weight data read, and is used in conjunction with the weight block counter to select different weight storage units. The entire address generation process is controlled by multiple conditional logics to ensure the correctness and continuity of addresses at the start, end, and boundary conditions of calculation, thereby providing accurate address information for subsequent data reading and calculation.

[0069] The values ​​of these counters are calculated through a five-stage pipeline: first, the column block base offset is calculated, then the convolution kernel vertical offset is added, then the current row offset is added, then the number of channels is multiplied, and finally the channel index is added to generate the precise memory access address.

[0070] Among them, kx_cnt and w_cnt indirectly determine the update timing of n_cnt, ky_cnt, r_cnt, and a_cnt through state machine control. kx_cnt determines whether n_cnt is incremented, that is, the next channel is switched only after all x positions within a kernel window are completed; w_cnt determines whether r_cnt is incremented, that is, the output row is moved only after the weights of all output channels are read. kx_cnt and w_cnt together constitute a multi-layered nested control for convolution calculation.

[0071] During address generation, consider two possible padding scenarios:

[0072] Zero padding: Generates an invalid address signal to avoid memory access and directly provides a zero value.

[0073] Edge padding: Address remapping is performed to generate valid addresses, and the actual boundary data is read.

[0074] The address generation module calculates the addresses of activation data and weight data. This module is directly connected to the main control module and calculates the target address required at each moment in real time based on its internal counter. Details are as follows:

[0075] Activation value address generation:

[0076] First, when the module begins calculation, the control signal `cal_state_en` from the control module changes from 0 to 1, indicating the start of calculation. The address generation module implements loop control through a set of nested counters and begins generating addresses. The generation of activation value addresses can be viewed as a series of nested loops. From the innermost to the outermost loop, the loop variables include:

[0077] kx_cnt: The x-position of the convolution kernel, ranging from 0 to ker_x-1.

[0078] n_cnt: Input channel count, from 0 to channel number - 1.

[0079] ky_cnt: The position of the convolution kernel in the y-direction, from 0 to ker_y-1.

[0080] w_cnt: Weight block count, from 0 to w_max-1. Each weight block contains 32 convolutional kernels. w_max is the number of weight blocks.

[0081] r_cnt: The row count of the activation graph, from 0 to activate_row - ker_y + padding_h. Note that the initial value of r_cnt is 0 - padding_h to support padding.

[0082] a_cnt: Activation value block count, from 0 to a_max-1. Each activation value block contains 32 columns, and a_max is the number of activation value blocks.

[0083] The data dependency table is shown in Table 1 below:

[0084] Table 1 Data Dependency Relationship Table

[0085]

[0086] Without fill, the specific formula can be expressed as:

[0087] Data access address = (a_cnt × number of active rows + ky_cnt + r_cnt) × number of channels + n_cnt.

[0088] However, if padding is present, it will affect the r_cnt signal, so this address needs to be adjusted according to the padding type. Currently, the chip supports two padding methods, which can be determined based on the padding_type signal in bits 82-83 of the configuration information:

[0089] When padding_type=0: it means no padding is needed, and the address calculation is as shown above.

[0090] When padding_type=1: it indicates invalid padding, that is, when the convolution kernel exceeds the range of the activation map, the read activation value is 0, and the address invalid machine rd_aa_vld is 0.

[0091] When `padding_type=2`: This indicates valid padding, meaning that boundary values ​​are read when the convolution kernel exceeds the activation map's range. In this case, the address needs adjustment.

[0092] Where row_num = number of channels × number of active rows, row_num represents the total number of data in a full-width block.

[0093] If ky_cnt+r_cnt<0 and a_cnt==0, then the address is n_cnt, which is the boundary of the first row.

[0094] Upper boundary: If ky_cnt+r_cnt<0 and a_cnt≠0, then the address is a_cnt×row_num+n_cnt, which is the first row boundary of the current block.

[0095] Lower boundary: If ky_cnt+r_cnt≥activate_row, then the address is (a_cnt+1)×row_num-channel number+n_cnt. Note that it should actually be the last row boundary of the current block, the first row boundary of the next block minus the channel number, which is the last row boundary of the current block.

[0096] Therefore, when `padding_type=1` (i.e., padding with 0), the top and bottom padding is implemented as follows: When generating the read address, the module determines whether the row to be read is within the valid range of the input feature map. This determination is achieved by calculating the current row position, which is the vertical offset of the convolution kernel plus the offset of the currently read row. If the calculated row position is less than 0 (indicating an upper padding region) or greater than or equal to the actual number of rows in the input feature map (indicating a lower padding region), the module generates an invalid read signal and sets the read address to 0. This means that when the external circuit receives these signals, it will not actually read data from memory but will directly use 0 as padding, thus achieving the effect of top and bottom padding with 0.

[0097] When `padding_type=2` (i.e., copying edge values), the module determines whether the currently read row is outside the valid range. If the current row is in the upper padding area, the module generates an address based on the index of the currently processed data block: if it's the first data block, it generates an address pointing to the first row of the input feature map; otherwise, it generates an address pointing to the first row of the current data block. This allows the value of the corresponding edge row to be copied as upper padding. If the current row is in the lower padding area, the module generates an address pointing to the starting address of the next data block minus one row of data, which is actually the address of the last row of the current data block, thus copying the value of the last row as lower padding. In either case, the module keeps the read signal valid, ensuring that the external circuitry can read the correct edge value from the generated address for padding.

[0098] In addition, two padding addresses, pad_l_addr and pad_r_addr, are generated for padding the left and right boundaries. These two addresses are located to the left and right of the currently active value block, respectively. When the current address is at the boundary of a block, data from the adjacent block needs to be read for padding.

[0099] Weight address generation:

[0100] The weight address is incremented only when addr_cal_vld is 1, until it reaches 8191 or a weight block is read and completed (condition3 is 1, indicating the completion of a weight block calculation). Specifically, each clock cycle, if addr_cal_vld is 1, rd_addr_wb is incremented by 1, unless it has reached 8191 or a weight block calculation is completed, in which case it is reset to 0. Simultaneously, there is a w_sram_cnt counter used to record the switching of weight SRAM. When rd_addr_wb reaches 8191, it switches to the next SRAM. Here, 10 groups of W_SRAM with a depth of 8192 are set.

[0101] In addition to address generation, the address generation module is also responsible for precise pipeline alignment and timing adjustment of control signals, ensuring that address, enable, and data validity signals arrive synchronously at the correct time points in subsequent computation modules. The address generation module transforms static configuration parameters provided by the main control module, such as matrix size and convolution kernel parameters, into dynamic address sequences and control instructions. These instructions include data read addresses, validity flags, and boundary handling information, collectively controlling the execution of subsequent computation units to achieve efficient acceleration of convolution computation. Through this design, the main control module only needs to provide configuration once, and the address generation module can automatically complete the address scheduling for the entire computation process.

[0102] like Figure 4 As shown, the computing array module consists of m×m arrayed computing units, each of which includes a multiplier, an adder, an accumulator register, an activation function unit, and a scaling unit.

[0103] The data scheduling of the computing array module is allocated by the control module: within each clock cycle, the weight data read by the computing array module is broadcast to all column computing units of the array in the column direction, while the activation data is broadcast to all row computing units of the array in the row direction; each computing unit first multiplies the received activation data and weight data through a multiplier, and then determines, according to the control signal, whether to add the multiplication result to the value in the accumulation register to continue accumulation, or to overwrite the value in the accumulation register with the multiplication result to start a new accumulation; during the multiplication and accumulation operation, the number of accumulations is controlled by a loop counter. When the preset total number of accumulations for the convolution window is reached, the calculation of a complete convolution kernel is completed. The total number of accumulations is determined by the convolution kernel size and the number of input channels, and a valid output signal is generated; after the multiplication and accumulation calculation is completed, the accumulation result in the accumulation register is first sent to the activation function unit for activation. This activation function unit is used to select ReLU activation, LeakyReLU activation, or pass-through operation according to the configuration parameters. The activated result is scaled by a scaling unit, which performs a right shift operation according to a set scaling factor to achieve quantization output. Finally, the scaled result is output as the calculation result of the current calculation unit. In the next clock cycle, the accumulator register is cleared or a new round of accumulation begins, thereby achieving parallel processing of the convolution calculation.

[0104] Through the above design, the present invention effectively reduces off-chip data access by utilizing on-chip cache and data reuse, thereby reducing latency and power consumption. The input data is reorganized and some redundancy is added to enable the computing unit and data reading to work in coordination, thereby improving computing efficiency. Multiple multiply-accumulate units are allocated in each PE computing unit to construct a structured PE computing unit, making the circuit area compact.

[0105] In this embodiment, convolution calculation is performed using a fixed output data stream. Parallel computation is carried out along the M and C directions. Taking a parallelism of 32 as an example, 32 convolution kernels and 32 columns of activation values ​​are processed in parallel at once. Therefore, the cycle required to complete the entire layer is Kx×Ky×N×[M / 32]×[C / 32]×(R-Ky+1). Each clock cycle reads 32 activation data and 32 weight data, so a computation array is constructed using 32×32 PE computation units.

[0106] Building upon convolution calculations, the computation unit also handles quantization scaling and nonlinear activation operations. Floating-point data is preprocessed at the software level, calculating the corresponding shift number based on the quantization formula. This shift value is transmitted as a configuration parameter to the AI ​​hardware accelerator chip via the host computer. During calculation, both the activation value and weight inputs are 8-bit integers. Their multiplication produces a 16-bit intermediate result. In the subsequent accumulation stage, the system expands the accumulated result to 32 bits to maintain computational precision and reduce quantization errors. For final output, the system right-shifts and scales the 32-bit accumulated result according to the configured shift number `scaling_size`. Specifically, the shift operation unit right-shifts the 32-bit data by `scaling_size` bits, then truncates the lower 8 bits as the calculation result for subsequent modules. Notably, the chip uses a truncation method that directly discards the lower bits.

[0107] The nonlinear layer primarily performs leaky_relu and Relu operations. The choice between Relu and leaky_relu is based on the input parameters configured at the top level. The Relu implementation checks the sign bit: when the sign bit is 1, the output is 0; when the sign bit is 0, the data is output. For leaky_relu, the parameter value is fixed at 0.125. When the sign bit is 1, the data is shifted right by three bits; when the sign bit is 0, the data is output. The specific calculation method is as follows... Figure 6 As shown.

[0108] In the data sliding process within the PE computation array, the activation value vector A_32 is input from the left side of the array, and the weight vector W_32 is input from the top. Each clock cycle, the activation value vector moves one column to the right, and the weight vector moves one row down. Each PE unit receives a pair of activation values ​​and weights each clock cycle, performs a multiplication operation, and determines whether to add the multiplication result to the internal accumulator based on the control signal sel_n. After completing the multiplication and accumulation of a set of convolution kernel parameters, an output valid signal is generated, and the accumulated result is processed by activation functions such as ReLU or LeakyReLU, and then scaled before being output. Throughout the computation process, the counter sel_cnt controls the number of accumulations. After completing a set of accumulations, the counter is reset to zero, and a new round of accumulation begins. The computation end signal is triggered by the rising edge of the data valid signal data_vld_n, indicating that the current computation task is complete. The data sliding process is as follows: Figure 7 As shown.

[0109] The control module is a state machine, used to generate corresponding control signals in various states to control the activation of other modules. The control module includes the following states:

[0110] IDLE: IDLE is the initial state. In the IDLE state, the input module will continuously send frame requests to the host computer. When it receives the signal that the data from the host computer has entered the chip, it will jump to the RX state to perform network calculations and store the data on the chip.

[0111] RX: RX is the initial data transmission state. The initial data includes the configuration information of all layers of the network, the initial image data, and the weight information of the first layer of the network. All of these are continuously transmitted into the chip in the form of 128-bit serial transmission. Therefore, in the RX state, the input data is classified, spliced, and saved. When the signal that the data transmission is complete is received, it jumps to the CALC state to perform data calculation.

[0112] CALC: CALC is the normal operating computation state. In CALC state, the address generation module generates access addresses for activation data and weight data, and then reads the corresponding m activation data and weight data from the SRAM read / write selection module. After data preprocessing by the shift register module, the weight data is broadcast row-wise to the m×m computation array, and the activation data is broadcast column-wise to the computation array for parallel convolution computation. After Kx×Ky×N clock cycles, the data enters the feedback module. If the current layer is the last layer, it jumps to the TX state to output the result; if the current layer is not the last layer, the feedback module buffers the data of m channels and generates the corresponding address, and then stores the address and data back to the SRAM read / write selection module as the computation input for the next layer; after all computations of the current layer are completed, it jumps to the DEC state to determine the layer type of the next layer.

[0113] DEC: DEC is an intermediate transition state. After one layer of the network has completed its computation, it will wait for several cycles to decode the configuration information of the next layer. If the next layer is a convolutional layer, it will jump to the RXW state to receive the weight information used by the next layer; if the next layer is a pooling layer, it will jump to the PL state to perform pooling operations.

[0114] PL: PL is the pooling state. At this time, the pooling address generation unit generates the pooling address and reads the corresponding activation data from the SRAM read / write selection module into the pooling module to perform the corresponding pooling operation. After the pooling is completed, it jumps to the RXW state and waits to receive the weight information required for the next layer calculation.

[0115] RXW: The RXW state is similar to the RX state. The chip will initiate a weight request to the outside world, but only the weight information is transmitted. The RXW state is used to update the weight values ​​between layers. After the transmission is completed, there will be a transmission end signal. When the transmission end signal is received, it will jump to the CALC state.

[0116] TX: The TX state is used for on-chip to off-chip result transmission. In the TX state, the output module serially outputs the network inference result, i.e. the final calculation result, to the off-chip in 8-bit form. After all the results have been transmitted, it returns to the IDLE state to wait for the next frame of data to be processed.

[0117] The state transition diagram of the control module is as follows: Figure 14 As shown, the ping-pong signal and the layer selection signal change when the single-layer calculation end signal or the pooling end signal is valid.

[0118] The on-chip memory includes a weighted data storage unit WRAM, a first active data storage unit ARAM0, and a second active data storage unit ARAM1; WRAM, ARAM0, and ARAM1 are each provided with two ports, A and B.

[0119] like Figure 9 As shown, the control logic of the SRAM read / write selection module is as follows:

[0120] When the control module is in RX state and RXW state, the SRAM read / write selection module only receives data from outside the chip, and only one port A of the SRAM read / write selection module is active for writing.

[0121] When in the RX state, the initial image data and the first layer weight data are stored sequentially into the first active data storage unit ARAM0 and the weight data storage unit WRAM according to the address generated by the address generation module.

[0122] When in the RXW state, only the weight data storage unit WRAM that stores weight data is updated, and the activation data is not processed.

[0123] When in CALC state, the activation data is read from port A of the first activation data storage unit ARAM0 according to the address. Due to the ping-pong operation, the calculation result is stored back into the second activation data storage unit ARAM1. The next time convolution or pooling is calculated, the functions performed by the first activation data storage unit ARAM0 and the second activation data storage unit ARAM1 are exchanged, that is, the activation data is read from the second activation data storage unit ARAM1 and the result is stored back into the first activation data storage unit ARAM0. This process is repeated in turn.

[0124] Through the above design, this invention achieves efficient reuse and pipelined execution of computational data by designing a collaborative architecture of multiple on-chip memories. This allows activation data and weight data from outside the chip to be read into the chip at once, effectively reducing on-chip and off-chip interactions and improving data processing speed. Considering the resources and transmission speed of the I / O interface, all data input into the chip is transmitted through the same 128-bit bus channel. This invention reads all layer configuration information and activation data into the chip at once, and also stores the results of inter-layer calculations within the chip. In subsequent calculations, only the weight data of the next layer needs to be read for computation. Therefore, weight data can be stored in a single layer. After each layer's data is calculated, the weight data of the next layer is updated for calculation, eliminating the need to load all weight data onto the chip. This way, on-chip storage only needs to meet the storage requirements of the layer with the largest data volume. Table 2 visually illustrates the operations performed by each SRAM in various states.

[0125] Table 2 SRAM Read / Write Selection Function Table

[0126]

[0127] Since convolution is computed in parallel across 32 channels, and its results occupy multiple channels, to improve the backhaul speed, both ports A and B are opened simultaneously to store the results back into ARAM0 or ARAM1. Pooling, on the other hand, is computed serially, occupying one channel at a time, so its results are stored back into ARAM0 or ARAM1 only through port A. Furthermore, because convolution and pooling operations are performed sequentially, there are no conflicts between their results during backhaul.

[0128] Based on the preceding analysis, activation data is typically read from port A of the ARAM. However, if there is an operation involving concatenating left and right computation blocks, requiring the reading of two or three addresses at once for convolution or padding of block edges, the data for the left and right computation blocks is read from port B of the ARAM according to their corresponding addresses. Since the weight data is independent of the left and right data blocks, the A and B ports of the WRAM have clearly defined functions: port A is used for storing data, and port B is used for reading data.

[0129] The convolution operation of a Convolutional Neural Network (CNN) is essentially a large matrix operation. Frequent reading and writing of such a large matrix can put enormous pressure on storage space. This invention employs a data block storage technique to divide the weight data and input image of each layer into smaller parts. In this embodiment, both activation data and weight data are int8 format. They are first converted and quantized in precision by software before being transmitted to the AI ​​hardware accelerator.

[0130] Activation data storage method such as Figure 2 As shown. The activation matrix At0 is a three-dimensional matrix, which needs to be transformed into a two-dimensional matrix to be stored in memory. Assuming that the size of At0 is R(N,R,C), it is first divided into blocks along the column direction according to the parallelism. In this embodiment, the parallelism is 32, resulting in R(N,R×C',32). Then, it is expanded along the channel direction to obtain a two-dimensional matrix of size R(N×R×C',32), where C' is [C / 32] and [] is rounded up. Here, N is the number of channels, R is the number of rows in the activation matrix, and C is the number of columns in the activation matrix.

[0131] In practice, the input data is divided into blocks of 32 columns, padding with zeros to ensure data alignment if there are fewer than 32 columns. Each row of 32 data points from a channel is grouped together, maintaining the original order, and stored in the same memory address. The next adjacent memory address stores the next row of 32 data points from the next channel. This process continues until all input data for N channels is stored, then the data for the next row is stored, until the entire computation block is stored, at which point the data for the next computation block is stored.

[0132] During the off-chip to on-chip transmission process, in order to save I / O resources and ensure data alignment, the 32 data are split into groups of 16 and placed in two adjacent storage spaces in the original order of the 32 data. When the accelerator reads the activation data, it will read them in sequence and concatenate the two adjacent data to restore them into a group of 32. The weight data is also processed in the same way during transmission.

[0133] Saving Weight Data for Each Layer: During the process of saving the weight data for each layer to the SRAM read / write selection module, it is also necessary to transform the four-dimensional data into two-dimensional storage. Let the weight matrix be Wt0, with a size of R(M,N,Kx,Ky). First, expand it along the channel direction to obtain a three-dimensional matrix Wt1, with a size of R(M,Kx×N,Ky). Then, expand it along the row direction to obtain a two-dimensional matrix Wt2 with a size of R(M,Kx×N×Ky). Transpose Wt2 and divide it into blocks according to the parallelism. Assuming the parallelism is 32, the final result is a two-dimensional matrix of size R(Kx×N×Ky×M', 32), where M' is [M / 32]. Here, M is the number of convolution kernels, Kx is the number of columns in the weight matrix, and Ky is the number of rows in the weight matrix.

[0134] In practice, convolutional kernels are stored in groups of 32. If a layer has M convolutional kernels, there are [M / 32] groups of convolutional kernels; if there are fewer than 32, zeros are used. The weights are stored as follows: Figure 3 As shown, the process first processes the first channel of each group. The first row stores the first number of each of the 32 convolutional kernels sequentially, until the third row stores the third number. Then, the first row of the second and third channels is processed. This continues until the data for the first row of all channels has been processed. Only then are the data for each channel in the next row processed. Once all the convolutional kernels for a group have been stored, the convolutional kernel data for the next group is saved.

[0135] The storage outside the accelerator also follows the above rules, except that the 32 data items are split into 16 data items per group and stored in two adjacent addresses.

[0136] In this invention, activation data and its intermediate values ​​involve both read and write operations. To improve computational efficiency and ensure data accuracy, all activation values ​​are stored in two SRAM memories: one for reading and one for writing—a ping-pong operation. This mechanism ensures that, except for the initial input and final output, all intermediate data flows within the chip, eliminating the need for repeated access to external storage. This significantly alleviates I / O bandwidth pressure, achieving functional decoupling and avoiding data conflicts. For weights, after each data read from external storage and stored on-chip, only a read operation is performed on the chip; therefore, a single SRAM is sufficient to meet storage requirements.

[0137] Specifically, this design employs a grouped ping-pong structure in the active SRAM (a_sram01–a_sram04 and a_sram11–a_sram14) to store intermediate feature maps, and controls read / write switching via a ping-pong enable signal, enabling parallel execution of current layer computation and next layer data backhaul. Simultaneously, 10 independent weight SRAMs (w_sram0–w_sram9) are used to load and store weight data layer by layer. During computation, when the current network layer number is less than the total network layer number, the intermediate results output by the PE computation array are written back to the corresponding active SRAM via signals such as data_back_en and back_data_a / b. The final result is only output to off-chip via the data_to_out_en signal when the last layer is completed. This mechanism ensures that all intermediate data, except for the first layer input and the last layer output, flows within the chip, eliminating the need for repeated access to off-chip storage, thus greatly alleviating I / O bandwidth pressure.

[0138] The first active data storage unit ARAM0 is divided into four active bank regions, a_sram01–a_sram04, and the second active data storage unit ARAM1 is divided into a_sram11–a_sram14. Each active bank has a bit width of 64 bits (8×8 bits) and is used to store eight 8-bit integers. Therefore, a complete 256-bit active data, i.e., 32 8-bit integers, requires parallel access from the four active banks. These two sets of RAMs are switched via a ping-pong enable signal to achieve ping-pong operation. In this embodiment, the bit width of a single data item is 8 bits. For an active bank, one address stores eight data items.

[0139] For the Weighted Data Storage Unit (WRAM), one address stores 32 data items, or 256 bits. The WRAM contains 10 independent weight RAMs (w_sram0~w_sram9), each with a width of 256 bits (32×8 bits), used to store 32 8-bit integers. The depth of the weight RAMs depends on the maximum on-chip storage capacity. When the required storage depth exceeds the maximum theoretical depth of a single weight RAM, it can be achieved through data concatenation or block partitioning, such as splitting into multiple banks. The specific degree of parallelism is determined based on the bit width of the weights or active data.

[0140] The on-chip configuration information of the AI ​​hardware accelerator is a 128-bit binary sequence. The decoding module is used to parse the received configuration data into a series of independent configuration parameters for controlling neural network computation, according to a predefined bit field mapping rule. The mapping rule is as follows: Figure 8As shown, each network layer has corresponding configuration information. After the chip sends a frame request, it loads all the received configuration information for each network layer into the on-chip memory. If the on-chip network type needs to be changed, the configuration information only needs to be modified according to the mapping rules and read into the chip, providing high flexibility.

[0141] In the 128-bit binary sequence, fields 0-7 represent the current number of computational layers and are used for layer counting; fields 92-99 represent the maximum number of layers in the current network. These two fields together determine whether it is the last layer. The maximum number of layers is preset to 256, which can well support the current mainstream convolutional neural networks, such as ResNet-50 / 101 and VGG.

[0142] Fields 8-23 represent the height H of the input image or feature map, fields 24-39 represent the width W of the input image or feature map, and fields 40-55 represent the number of channels N of the input image or feature map. Considering the on-chip memory, the maximum supported single input image size of int8 type is 512×512.

[0143] Fields 56-71 represent the number of convolutional kernels, and fields 72-75 represent the size of the convolutional kernels. The kernel size is encoded using 4 bits: 0001 represents a 1x1 kernel, 0010 a 3x3 kernel, 0011 a 5x5 kernel, 0100 a 7x7 kernel, 0101 a 3x1 kernel, 0110 a 5x1 kernel, 0111 a 7x1 kernel, and others represent 0x0 kernels (no convolution operation). The current design defaults to a stride of 1. Considering on-chip memory, the maximum total data size (int8 type) for a single convolutional kernel layer is 10×512×512.

[0144] Fields 76 and 77 indicate the layer type, with 00 indicating a convolutional layer and 01 indicating a pooling layer;

[0145] Fields 78 and 79 indicate the pooling type: 00 indicates max pooling and 01 indicates average pooling.

[0146] Fields 80-81 indicate the non-linearity type: 00 indicates no non-linearity operation after convolution, 01 indicates a ReLU layer, and 10 indicates a Leaky_reLU layer.

[0147] Fields 82 and 83 indicate the padding type: 00 indicates no padding, 01 indicates padding with 0 values, and 10 indicates padding with edge values. The width or height of the padding is determined by the current convolution kernel size.

[0148] Fields 84-91 represent the scaling size, which is used to quantize the multiplicative summed value into the form of int8. In combination with the current chip design, the designed scaling size should not be greater than 16.

[0149] Fields 92-99 represent the maximum number of network layers. Defining the depth of this storage array in advance is to prevent infinite layers from causing state machine dead loops and logic freezes.

[0150] Fields 100-127 are reserved to prepare for future functional expansion of chips.

[0151] The shift register module is used to implement data alignment, padding and window sliding functions before CNN convolution calculation, solve the data calculation in the case of block overflow, and prepare the activation value a_32 and weight w_32 data for subsequent calculation.

[0152] The activation value data is implemented in the hardware logic as follows: Figure 10 As shown.

[0153] The shift register module reads data from the SRAM read / write selection module: when the ping-pong enable signal is 0, it reads data from ports A and B of the first group of SRAMs (SRAMs 01 to 04), storing the data from port A into four 8-bit ports (ports A1 to 4), and the data from port B into four other 8-bit ports (ports B1 to 4). When the ping-pong enable signal is 1, it reads data from the corresponding ports of the second group of SRAMs (SRAMs 11 to 14). Next, the four 8-bit data from ports A1 to 4 are concatenated into a 32-bit active window register. Then, based on the padding type, padding width, address calculation valid signal, read activation valid signal, left and right padding address valid signals, and the 5th bit of the active column, a 40-bit extended window is constructed. This 40-bit window is formed by adding the maximum convolution size of 7 and a supplementary bit to the original 32-bit window.

[0154] When constructing a 40-bit extended window, it is essential to ensure that three control signals are simultaneously valid: the compute state enable signal, the address compute valid signal, and the read activation valid signal. Window construction will only proceed if all three signals are valid. Otherwise, the 40-bit window will be set to zero. This excludes the unpadded type, which only requires the compute state enable signal and the address compute valid signal to be simultaneously valid for 40-bit window construction.

[0155] Next, based on the different fill types, there are three cases:

[0156] The first case: No fill type.

[0157] If the right padding address valid signal is valid, the 32-bit active window register is concatenated with the 8-bit data of port B1 to form a 40-bit window, with the first 32 bits being the active window and the last 8 bits being the data of port B1.

[0158] If the right padding address valid signal is invalid, the 32-bit active window register is concatenated with 8 zeros to form a 40-bit window, with the first 32 bits being the active window and the last 8 bits being zeros.

[0159] The second case: zero-padding type.

[0160] Depending on the padding width and the status of the left and right padding address valid signals, there are several sub-cases:

[0161] When the padding width is 1:

[0162] If both the left and right padding addresses are invalid, add one zero to the left of the 32-bit active window and seven zeros to the right.

[0163] If the left padding address is valid and the right padding address is invalid, then add the 7th bit (single element) of port 4 of the 32-bit active window to the left and add 7 zeros to the right.

[0164] If the left padding address is invalid and the right padding address is valid, add a zero to the left of the 32-bit active window and add the lower 7 bits of port 1 of B to the right.

[0165] If both the left and right padding addresses are valid, then add the 7th bit of port 4 to the left of the 32-bit active window and the lower 7 bits of port 1 to the right.

[0166] When the padding width is 2:

[0167] If both the left and right padding addresses are invalid, add two zeros to the left and six zeros to the right of the 32-bit active window.

[0168] If the left padding address is valid and the right padding address is invalid, then add the 6th and 7th bits (two elements) of port 4 of the 32-bit active window to the left and add 6 zeros to the right.

[0169] If the left padding address is invalid and the right padding address is valid, then add two zeros to the left of the 32-bit active window and add the lower 6 bits of port 1 of B to the right.

[0170] If both the left and right padding addresses are valid, then add the 6th and 7th bits of port 4 to the left of the 32-bit active window and the lower 6 bits of port 1 to the right.

[0171] When the padding width is 3:

[0172] If both the left and right padding addresses are invalid, add three zeros to the left and five zeros to the right of the 32-bit active window.

[0173] If the left padding address is valid and the right padding address is invalid, then add bits 5 to 7 (three elements) of port 4 of the 32-bit active window to the left and add 5 zeros to the right.

[0174] If the left padding address is invalid and the right padding address is valid, add three zeros to the left of the 32-bit active window and add the 5th bit of port 1 of B to the right.

[0175] If both the left and right padding addresses are valid, then add bits 5 to 7 of port 4 to the left of the 32-bit active window and bit 5 of port 1 to the right.

[0176] The third case: edge padding type.

[0177] This padding type depends not only on the padding width (1, 2, or 3) and the left and right padding address valid signals, but also on the 5th bit value of the active column and the active block count.

[0178] When the padding width is 1:

[0179] If both the left and right padding addresses are invalid, the determination needs to be made based on the 5th value of the active column:

[0180] If the 5th element of the active column is not zero, then copy the first element (the 0th element) of the window to the left of the 32-bit active window, and copy the 5th element of the active column of the window to the right of the active window minus 1.

[0181] If the 5th element of the active column is zero, then the first element of the window is copied to the left of the 32-bit active window, and the 31st element of the window is copied to the right.

[0182] If the left padding address is valid and the right padding address is invalid, the same judgment is made based on the 5th value of the active column:

[0183] If the 5th bit of the active column is not zero, then the 7th bit of port 4 of the B port is used on the left side of the 32-bit active window, and the 5th bit of the active column of the copy window on the right is reduced by 1.

[0184] If the 5th bit of the active column is zero, then the 7th bit of port 4 of the 32-bit active window is used on the left and the 31st bit of the window is copied on the right.

[0185] If the left padding address is invalid and the right padding address is valid, then the first element of the window is copied to the left of the 32-bit active window, and the lower 7 bits of port 1 of B are used on the right.

[0186] If both the left and right padding addresses are valid, then the 7th bit of port 4 (B port 4) is used on the left side of the 32-bit active window, and the lower 7 bits of port 1 (B port 1) are used on the right side.

[0187] When the padding width is 2:

[0188] If both the left and right padding addresses are invalid, the determination needs to be made based on the 5th value of the active column:

[0189] If the 5th position of the active column is not zero, then the first element of the window is copied twice consecutively on the left side of the 32-bit active window, and the 5th position of the active column of the window is reduced by 1 consecutively on the right side.

[0190] If the 5th element of the active column is zero, then the first element of the window is copied twice on the left side of the 32-bit active window, and the 31st element of the window is copied twice on the right side.

[0191] If the left padding address is valid and the right padding address is invalid, the same judgment is made based on the 5th value of the active column:

[0192] If the 5th bit of the active column is not zero, then the 6th and 7th bits (two elements) of port 4 of the 32-bit active window are used on the left side, and the 5th bit of the active column of the window is copied twice on the right side by minus 1.

[0193] If the 5th bit of the active column is zero, then the 6th and 7th bits of port 4 of the 32-bit active window are used on the left side, and the 31st bit of the window is copied twice on the right side.

[0194] If the left padding address is invalid and the right padding address is valid, it is necessary to check whether the active block count is equal to the maximum active block count minus 2, and whether the 5th bit of the active column is 1.

[0195] If the above special conditions are met, the first element of the window will be copied twice consecutively on the left side of the 32-bit active window, and the 0th bit of port 1 of B will be copied twice consecutively on the right side.

[0196] If the conditions are not met, the first element of the window is copied twice consecutively on the left side of the 32-bit active window, and the lower 6 bits of port 1 of B are used on the right side.

[0197] If both the left padding address and the right padding address are valid, also check if the active block count is equal to the maximum active block count minus 2, and if the 5th bit of the active column is 1:

[0198] If satisfied, then use bits 6 and 7 of port 4 on the left side of the 32-bit active window, and copy bit 0 of port 1 twice on the right side.

[0199] If the conditions are not met, then the 6th and 7th bits of port 4 of B are used on the left side of the 32-bit active window, and the lower 6 bits of port 1 of B are used on the right side.

[0200] When the padding width is 3:

[0201] If both the left and right padding addresses are invalid, the determination needs to be made based on the 5th value of the active column:

[0202] If the 5th element of the active column is not zero, then the first element of the window is copied three times consecutively on the left side of the 32-bit active window, and the 5th element of the active column of the window is reduced by 1 consecutively on the right side.

[0203] If the 5th element of the active column is zero, then the first element of the window is copied three times consecutively on the left side of the 32-bit active window, and the 31st element of the window is copied three times consecutively on the right side.

[0204] If the left padding address is valid and the right padding address is invalid, the same judgment is made based on the 5th value of the active column:

[0205] If the 5th bit of the active column is not zero, then the 5th to 7th bits (three elements) of port 4 of the 32-bit active window are used on the left side, and the 5th bit of the active column of the window is copied three times consecutively on the right side minus 1.

[0206] If the 5th bit of the active column is zero, then bits 5 to 7 of port 4 of the 32-bit active window are used on the left side, and bit 31 of the window is copied three times consecutively on the right side.

[0207] If the left padding address is invalid and the right padding address is valid, it is necessary to check whether the active block count is equal to the maximum value of the active block minus 2, and the value of the 5th position of the active column:

[0208] If the active block count equals the maximum active block count minus 2 and the 5th bit of the active column is 1, then the first element of the window is copied three times consecutively on the left side of the 32-bit active window, and the 0th bit of port 1 of B is copied three times consecutively on the right side.

[0209] If the active block count equals the maximum active block count minus 2 and the 5th position of the active column is 2, then the first element of the window is copied three times consecutively on the left side of the 32-bit active window, and the 0th to 1st positions of port B1 are used on the right side, and the 1st position of port B1 is repeated once.

[0210] In other cases, the first element of the window is copied three times consecutively on the left side of the 32-bit active window, and the 5th bit of port 1 of B is used on the right side.

[0211] If both the left padding address and the right padding address are valid, also check if the active block count is equal to the maximum active block count minus 2, and the value of the 5th position in the active column:

[0212] If the active block count equals the maximum active block count minus 2 and the 5th bit of the active column is 1, then bits 5 to 7 of port 4 are used on the left side of the 32-bit active window, and bit 0 of port 1 is copied three times consecutively on the right side.

[0213] If the active block count equals the maximum active block count minus 2 and the 5th bit of the active column is 2, then bits 5 to 7 of port 4 are used on the left side of the 32-bit active window, bits 0 to 1 of port 1 are used on the right side, and bit 1 of port 1 is repeated once.

[0214] In other cases, bits 5 to 7 of port 4 are used on the left side of the 32-bit activation window, and bit 5 of port 1 is used on the right side.

[0215] Finally, based on the value of the convolution kernel horizontal position counter (0 to 6), the corresponding 32-bit activation data is truncated from a 40-bit window. Simultaneously, based on the value of the weight SRAM counter signal (w_sram_cnt) after a 4-clock-cycle delay, data is output from the B ports of the 10 weight RAMs, each 32 bits. After a 1-clock-cycle register delay, w_sram0_doutb is stored in w_sram9_doutb. A corresponding 32-bit weight data is selected as the current output. After a first-level output register delay, the valid signal, 32-bit activation data, and 32-bit weight data are output to the subsequent computation units. The feature map filling and calculation sliding relationship is as follows: Figure 11 As shown.

[0216] The data return module acts as a buffer. Since the PE array calculates 32 channels × 32 data, it's impossible to transfer all data back to SRAM within a single clock cycle; therefore, data return is designed as a buffer. During non-final layer calculations, the return module automatically time-division multiplexes data return based on the number of channels, returning a maximum of two channels at a time via ports A and B, and automatically generating corresponding return addresses. Simultaneously, the module sends a signal to the main control module to notify it of return completion. During the final layer calculation, data is directly output to the external interface and does not participate in the return process. The return logic is as follows: Figure 12 As shown.

[0217] The backhaul module executes the following logic based on the input signals in each clock cycle: First, all input signals, including computation status enable, current layer number, total layer number, channel number, PE output valid, PE calculation result, PE calculation end, weight block maximum value, number of active rows, number of active columns, height padding, width padding, convolution kernel width, and convolution kernel height, are processed by a first-level register to generate corresponding register signals to ensure timing stability. Then, the module calculates the size of the next output feature map and the activation block parameters based on the registered signals. When the module detects that the computation status enable is valid, the PE output is valid, and the current layer is the last layer, it uses the data in column 0 of the PE output result as the final output data and sets the output enable signal, directly outputting it to the external interface. If it is not the last layer and the PE output is valid, backhaul enable is activated, and the number of channels to be backhauled in this round is determined based on the current weight block and channel number. Then, the backhaul counter sequentially selects data from two channels in the data buffer register as backhaul data, simultaneously generating corresponding backhaul addresses with initial values ​​of 0 and 1 respectively. During normal backhaul, the address increments by 2 per cycle. If the number is odd and it is the last pair of channels, the address is incremented by 1. When a layer of backhaul ends, the address is reset to the initial value. During the backhaul, the weight block counter increments after each round of channel backhaul until it reaches the maximum value of the weight block, while the line output counter increments when the PE output is valid to track the output line position, and the activation block counter increments after each line of output is completed. In addition, the module generates an intermediate layer backhaul end flag by detecting the PE calculation end signal and layer information. The flag is triggered by the falling edge of the signal and enabled by the final calculation end, and is set when the last layer of PE calculation ends or when a non-last layer backhaul ends.

[0218] The buffer register stores the current PE output result when the PE output is valid. If the active column is not an integer multiple of 32, only the valid part is stored.

[0219] For pooling layers in convolution calculations, a pooling module and a pooling address generation unit were designed. Based on the layer type in the configuration information, if the current layer does not require pooling, the data is directly returned to the active SRAM for subsequent operations. If the current layer requires pooling, the calculated data is read out, pooled, and then returned. The chip is designed for configurable networks with 2×2 max pooling with a stride of 2 and 2×2 average pooling with a stride of 2. The pooling type is parsed according to the configuration information: when pooling_type=0, 2×2 max pooling with a stride of 2 is performed; when pooling_type=1, 2×2 average pooling with a stride of 2 is performed. The 2×2 max pooling and average pooling operation flow with a step size of 2 is as follows: First, select the corresponding SRAM output group according to the pooling state enable and ping-pong enable signals. When the ping-pong enable is 0, the output ports a and b from sram01 to sram04 are concatenated into 32-bit a_com and b_com respectively; otherwise, the output ports a and b from sram11 to sram14 are concatenated. Next, with the pooling state enabled and the input valid signal valid, a_com and b_com (or a_com and all zeros) are concatenated into 64-bit act_window_1. For max pooling (pooling type 0), act_window_2_r is obtained by taking the maximum value of every pair of elements in act_window_1. For average pooling (pooling type 0), ... 1. This is obtained by summing these elements pairwise. Then, `act_window_2` is processed according to the block count and the remaining elements. When the flag is 0, `act_window_2` is stored in `act_window_3`; when the flag is 1, `act_window_2` is stored in `act_window_4`. When the flag is 1, the output is valid. During max pooling, the output data is the maximum value of `act_window_3` and `act_window_4`, completing the max pooling of the 2×2 window. During average pooling, the output data is the sum of `act_window_3` and `act_window_4` divided by 4, completing the average pooling of the 2×2 window. Simultaneously, the output address increments when the output is valid. The end enable signal is generated when the input valid signal changes from valid to invalid. The logic of the pooling module is as follows: Figure 13 As shown.

[0220] The core function of the flag is to alternately store the two intermediate results required for the 2×2 window. When flag is 0, the intermediate result after the initial processing of the current `act_window_2` is stored in `act_window_3` to temporarily store the first part of the data in the 2×2 window. When flag is 1, the current `act_window_2` is stored in `act_window_4` to temporarily store the second part of the data in the 2×2 window. When flag is 1, the output is valid. At this time, `act_window_3` has already stored the first part of the result, and `act_window_4` has stored the second part of the result. The two together form a complete 2×2 window of data. Max pooling takes the maximum value of the two, and average pooling takes the sum of the two and divides it by 4, which completes one 2×2 window pooling calculation.

[0221] The address generation process begins when the pooling enable signal pl_state_en experiences a rising edge. The pooling address module first calculates the number of data blocks per row, a_count, based on the number of activated columns in the input, and aligns it upwards to even-numbered blocks, a_max. During address generation, four nested counters—channel counter n_cnt, row counter r_cnt, data block counter a_cnt, and kernel position counter kx_cnt—traverse all data positions sequentially: first the channel dimension, then the row dimension, and finally the data block dimension. Both the row and block dimension traversals are performed with a step size of 2.

[0222] The first read address, pl_addr_aa, is calculated as: (data block index × number of active rows + current row offset + core position) × number of channels + channel index, implemented using a four-stage pipeline. The second read address, pl_addr_ab, is calculated by adding a fixed offset (number of channels × number of active rows) to the first address. Its validity requires that the preceding address's valid signal remain valid after a four-cycle delay, and that the data block count has not reached its upper limit. In a special case, when the number of active columns is not divisible by 32 and the address is in the last data block, the second address is invalid.

[0223] Implementation details: When the pooling address module allocates data addresses, it receives input signals such as pooling status enable, active row, active column, and number of channels, and temporarily stores them in the input register. Next, the active column is divided into units of 32, the number of blocks is calculated, the active column is divided by 32 and rounded up, and the maximum block value is determined based on whether the number of blocks is even; if even, it equals the number of blocks, and if odd, it is the number of blocks plus one. Simultaneously, the offset of the second type of pooled data address is calculated, which is the number of channels multiplied by the number of active rows.

[0224] The start read enable signal is triggered when the current pooling state is enabled and the previous state is disabled. The kx counter increments by 1 when the valid register for first-type pooling data is 1 and its own value is less than 1, otherwise it is cleared to zero; the active row correction value is calculated based on the active row, which is 0 when the active row is 0, decremented by 2 for even-numbered active rows, and decremented by 3 for odd-numbered active rows.

[0225] The update of related counters is controlled by four conditional signals: Condition 0 is that the Type I pooled data valid register is 1 and the kx counter is 1; Condition 1 is that Condition 0 is true and the n counter equals the channel number minus 1; Condition 2 is that Condition 1 is true and the r counter equals the active row correction value; Condition 3 is that Condition 2 is true and the block counter equals the block maximum value minus 2. The n counter is cleared when Condition 1 is true, incremented by 1 when Condition 0 is true, otherwise it retains its original value; the r counter is cleared when Condition 2 is true, incremented by 2 when Condition 1 is true, otherwise it retains its original value; the block counter is cleared when Condition 3 is true, incremented by 2 when Condition 2 is true, otherwise it retains its original value. When the end-read enable signal equals Condition 3, the Type I pooled data valid register is cleared when the end-read enable is 1, and set to 1 when the start-read enable is 1, otherwise it retains its original value.

[0226] The first type of pooled data address is obtained through a series of calculations: first, calculate the product of the block counter and the active row, add the value of the r counter, then add the value of the kx counter, multiply the result by the number of channels, and then add the value of the n counter to finally obtain the first type of pooled data address.

[0227] The address and valid signal of the second type of pooled data are determined based on the delay of the valid register of the first type of pooled data by 4 clock cycles and the delay of the block counter by 4 clock cycles: if the former is 1 and the latter is less than the maximum block value minus 2, the address is the address of the first type of pooled data plus the offset, and the valid signal is 1; if the former is 1 and the number of blocks is odd and the latter is equal to the maximum block value minus 2, the address is 0 and the valid signal is 0; if the former is 1 and the number of blocks is even and the latter is equal to the maximum block value minus 2, the address is the address of the first type of pooled data plus the offset, and the valid signal is 1; otherwise, the address is 0 and the valid signal is 0. The pooled address counting method is shown in Table 3.

[0228] Table 3 Pooling Address Calculation Table

[0229]

[0230] The data receiving module's core logic is to distinguish data types based on the input selection signal and perform corresponding storage, conversion, and address generation operations, ultimately outputting configuration data, activation data, or weighted data and their corresponding control signals. The specific process is as follows:

[0231] The module receives four types of input signals: receive data select signal, receive data enable signal, 128-bit receive data, and configuration select signal. These inputs are first registered for one clock cycle, forming corresponding register signals for use by subsequent timing logic.

[0232] The data receiving module is mainly divided into three processing paths:

[0233] Configuration data processing path: When the receive data enable is valid and the selection signal is in configuration mode, the module stores the received 128-bit data into the corresponding configuration data memory (16 groups in total) according to an internal configuration address counter (0~15). The configuration address is incremented each time a valid receive occurs; otherwise, it is cleared to zero. During output, based on the value of the configuration selection register, the module selects the corresponding group of 128-bit data from the configuration memory as the configuration output.

[0234] Activation data and weighted data conversion path: When the receive data enable is active and the selection signal is in active or weighted mode, the module concatenates two consecutive 128-bit receive data into a single 256-bit data using an alternating enable signal, and then calls a format conversion function to convert it into 32-bit data for output. This path simultaneously generates a data validity signal and an address counter.

[0235] For activation data, when the selection signal is in activation mode and splicing enable is valid, an activation data valid signal is generated, and the activation address counter is incremented, ranging from 0 to 8191, and then reset to zero when full.

[0236] For weighted data, when the selection signal is in weighted mode and the splicing enable is active, a weighted data valid signal is generated, and the weight address counter is incremented, ranging from 0 to 8191. Every time the weight address reaches 8191, a weight SRAM counter is incremented by one, which is used to distribute the weighted data to the 10 weight RAMs, i.e., SRAM0 to SRAM9, in turn. Each block generates a write enable signal when the corresponding count occurs.

[0237] Receive status indication path: Based on the change of the receive enable signal, the module generates a receive start enable signal (currently enabled and invalid in the previous clock cycle) and a receive end enable signal (currently enabled and invalid in the previous clock cycle). All output signals, including configuration output, converted 32-bit data, activation / weight address, write enable for each SRAM block, data valid signal, and receive status signal, are sent out after synchronization through the first-level output register.

[0238] The core logic of the output module is driven by internal registers and counters. When both the transmission status enable register and the data output enable register are 1, the output result enable register is set to 1 in the next cycle until the counter reaches 31, at which point it is reset to 0; otherwise, it retains the previous state. The counter increments from 0 each cycle when the output result enable is valid, automatically resetting to zero when it reaches 31; otherwise, it remains 0. The temporary result register is used to temporarily store input data: when both the transmission status enable and data output enable are valid, it stores the currently input data; if the transmission end enable is valid, it is cleared to zero; otherwise, it retains its original value. The transmission end valid flag is set to 1 when the calculation end flag is 1, or cleared to zero when the counter reaches 31 and the calculation is not yet finished. The transmission end enable register is set to 1 when the transmission status is enabled, the transmission end valid flag is 1, and the counter reaches 31; otherwise, it is set to 0.

[0239] When the output result enable is active, the corresponding bit data is retrieved from the temporary result register based on the current value of the counter and output; otherwise, 0 is output. Finally, the output result, output result enable, and transmission end enable signals are each output after a one-cycle delay.

[0240] The main technical features of this invention are as follows:

[0241] (1) Dynamic data stream scheduling: A scheduling system driven by "online control signal generation" is constructed, with the multi-state machine of the main control module as the core. The data stream strategy is dynamically switched according to the network layer type, such as convolution / pooling / fully connected. The convolutional layer adopts a fixed output data stream to maximize data reuse. When switching between layers, a real-time address is generated through multi-level counter linkage, which gets rid of the adaptation limitation of fixed data stream and realizes precise matching of the computing needs of different layers.

[0242] (2) Ping-pong style on-chip storage optimization: Design a collaborative architecture of dual-group active SRAM and hierarchical weight storage. The two groups of active SRAM achieve parallel "computation-back" through switching signals to improve backhaul speed; weight data is loaded layer by layer using 10 independent SRAMs, reserving storage resources for the largest layer only, without full loading; all data is stored in blocks with 32-bit granularity and zero-padded for alignment to ensure the continuity and efficiency of data access.

[0243] (3) High-density parallel computing array: A 32×32 dimension PE computing array is constructed. Each PE computing unit integrates multiply-accumulate, activation, and quantization functional units to form an integrated "computation-activation-scaling" processing link. The synchronous distribution of 32 activation values ​​and weight values ​​is achieved through a broadcast mechanism, and 1024 multiply-accumulate operations are completed in a single clock cycle, which greatly improves the parallel computing density and energy efficiency ratio.

[0244] (4) Fully Configurable Control: A 128-bit binary configuration word is used as the control core. Key information such as the decoding module parsing layer index, convolution kernel size (1×1 to 7×7), operator type, and quantization parameters can be configured. Based on the configuration parameters, the operator type and computation logic can be dynamically switched, adapting to the deployment requirements of different network models such as LeNet5 and VggNet16 without modifying the hardware structure.

[0245] (5) Storage block adaptation: Both activation values ​​and weight data are stored in blocks with a 32-bit granularity. When the activation value is less than 32 columns, zeros are added for alignment. The weights are grouped into groups of 32 convolution kernels, so that the data granularity of the filling process is completely matched with the storage blocks, thus avoiding the extra storage overhead caused by block overflow from the root.

[0246] (6) Dual-port SRAM collaboration: A true dual-port SRAM design is adopted, with port A responsible for main data reading and port B dedicated to synchronous reading of adjacent block data or boundary data. During fill calculation, the adjacent block data required for splicing is directly obtained through port B, without the need for additional storage of fill content, and is directly used for subsequent window construction.

[0247] (7) Shift register dynamic expansion window: A 40-bit expansion window construction mechanism based on the shift register module is designed. Based on the kernel size and the same layer type, 8 bits of padding data such as zero / boundary copy data are added by logical splicing on the basis of the 32-bit original activation data to form an expansion window adapted to the convolution sliding calculation without the need for additional storage of padding data.

[0248] (8) Multi-mode configuration adaptation: The processing logic of three modes—no padding, zero padding, and edge padding—is dynamically switched through the same-level type field in the 128-bit configuration word. No padding is performed directly in the expansion window. Zero padding is performed directly in the expansion window. Edge padding reuses the boundary elements of the original data. Multi-scenario adaptation can be achieved without modifying the hardware structure.

[0249] (9) Dynamic address adaptation: The address generation module adjusts the address calculation logic in real time according to the filling mode. When zero filling occurs, an invalid address is generated and a zero value is provided directly. When edge filling occurs, boundary data is read through address remapping to ensure accurate coordination between filling calculation and on-chip memory addressing without the need for additional address space allocation.

[0250] Example 1: The neural network configured in Example 1 is a LeNet5 network architecture, used to perform gas identification tasks. The parameter configuration of this LeNet5 network architecture is shown in Table 4.

[0251] Table 4 LeNet5 network architecture configuration

[0252]

[0253] Experiments have shown that the LeNet5 network architecture achieves an accuracy of 0.96 before quantization and 0.88 after quantization on the gas dataset.

[0254] Example 2: The neural network configured in Example 2 is a VggNet16 network architecture, used to perform image classification tasks. The parameter configuration of this VggNet16 network architecture is shown in Table 5.

[0255] Table 5 VggNet16 Network Architecture Configuration

[0256]

[0257] The CIFAR10 dataset contains 60,000 samples, each of which is a 32×32 pixel RGB image (i.e., a color image). Each RGB image is further divided into three channels: R channel, G channel, and B channel. These 60,000 samples are divided into 50,000 training samples and 10,000 test samples.

[0258] The VggNet16 network achieved an accuracy of 0.87 before quantization and 0.82 after quantization on the CIFAR10 dataset.

[0259] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. An AI hardware accelerator for neural network computation, characterized in that: The system includes a control module connected to an input module, which in turn is connected to an SRAM read / write selection module. The control module is also connected to both the SRAM read / write selection module and an address generation module. The address generation module is connected to the SRAM read / write selection module, which is further connected to a computation array module via a shift register module. The computation array module is connected to a feedback module, which is connected to both the SRAM read / write selection module and an output module. The input module is used to: acquire configuration data and calculation data, then transmit the configuration data to the control module, and transmit the calculation data to the SRAM read / write selection module; The control module is used to: output configuration parameters and corresponding status enable signals to the address generation module according to the configuration data, and trigger the address generation process; The address generation module is used to: calculate the data access address and control signal required for calculation at the current moment according to the configuration parameters and the status enable signal, and pass them to the SRAM read / write selection module; The SRAM read / write selection module is used to: read computational data from the corresponding on-chip memory according to the data access address and control signal, and then pass the read computational data to the displacement register module; The displacement register module is used to: fill and supplement the calculated data, perform bit width alignment operations, and then pass the supplemented and aligned data to the calculation array module; The computing array module is used to: perform calculations on the supplemented and aligned data, and output the calculation results to the feedback module; The return module is responsible for the format adaptation, temporary storage, and write-back control of the calculation results. The return module controls the write-back by determining whether the current layer calculation is the last layer calculation. If it is not the last layer calculation, it drives the SRAM read / write selection module to write the calculation result of the current layer back to the corresponding on-chip memory so that it can be used as the input for the next layer calculation. If it is the last layer calculation, it outputs the calculation result of the current layer to the output module so that it can be used as the final calculation result. The output module is used to output the final calculation result.

2. The AI ​​hardware accelerator for neural network computation according to claim 1, characterized in that: The configuration data consists of parameter information for each layer of the network, including but not limited to kernel size, padding type, and total number of network layers. The calculation data includes, but is not limited to, the weight data and activation data used in the calculation; AI hardware accelerators are used for gas identification or image classification tasks. In the gas identification task, the activation data includes the original gas input data and the intermediate gas feature data calculated by each network layer; the output module is used for the gas identification results. In the image classification task, the activation data includes the original image input data and the intermediate image feature data calculated by each network layer; The output module is used for image classification results.

3. The AI ​​hardware accelerator for neural network computation according to claim 1, characterized in that: The address generation module is provided with a convolution address generation unit and a pooling address generation unit. The SRAM read / write selection module is also connected to a pooling module, which is connected to the return module. The pooling module is provided with a max pooling unit and an average pooling unit. The pooling module is used to perform pooling calculations. If the current layer is configured as a convolution + pooling layer, the calculation data first undergoes convolution calculation through the calculation array module. The convolution calculation result is then sent back to the SRAM read / write selection module via the return module and used as input data for the pooling module. The pooling module then performs pooling calculations, and the calculated pooling result is passed to the return module as the calculation result of the current layer.

4. An AI hardware accelerator for neural network computation according to claim 1, characterized in that: The address generation module is configured from the inside out with the following components: a convolution kernel x-direction counter kx_cnt, an input channel counter n_cnt, a convolution kernel y-direction counter ky_cnt, a weight block counter w_cnt, a row counter r_cnt, and an activation value column block counter a_cnt. Among them, the convolution kernel X-direction counter kx_cnt is used to traverse the width of the convolution kernel, the input channel counter n_cnt is used to traverse the input channels, the convolution kernel Y-direction counter ky_cnt is used to traverse the height of the convolution kernel, the weight block counter w_cnt is used to traverse the weight blocks, the row counter r_cnt is used to traverse the rows of the input feature map, and the activation value column block counter a_cnt is used to traverse the column blocks of the input feature map; The data access address generation logic of the address generation module is as follows: First, the convolution kernel x-direction counter kx_cnt increments in each calculation cycle, traversing the horizontal position of the convolution kernel in the current input channel. After the convolution kernel x-direction counter kx_cnt completes one cycle, the input channel counter n_cnt increments, switching to the next input channel. After all input channels are processed, the convolution kernel y-direction counter ky_cnt increments, moving the convolution window down one row. After the convolution kernel y-direction counter ky_cnt completes the vertical traversal of the entire convolution kernel, the weight block counter w_cnt increments, switching to the next set of output channels. After all output channels at the current spatial position are calculated, i.e., the weight block counter w_cnt completes a full cycle from 0 to w_max-1, the row counter r_cnt increments, moving the convolution window to the next row. Finally, the activation value column block counter a_cnt increments after all rows are processed, switching to the next m-column activation data block. The formula for generating the data access address in the address generation module is as follows: Data access address = (a_cnt × number of active rows + ky_cnt + r_cnt) × number of channels + n_cnt.

5. An AI hardware accelerator for neural network computation according to claim 1, characterized in that: The computing array module consists of m×m arrayed computing units, each of which includes a multiplier, an adder, an accumulator register, an activation function unit, and a scaling unit. The data scheduling of the computing array module is allocated by the control module: within each clock cycle, the weight data read by the computing array module is broadcast to all column computing units of the array in the column direction, while the activation data is broadcast to all row computing units of the array in the row direction; each computing unit first multiplies the received activation data and weight data through a multiplier, and then determines whether to add the multiplication result to the value in the accumulation register through an adder or to overwrite the value in the accumulation register with the multiplication result according to the control signal; during the multiplication and accumulation operation, the number of accumulations is controlled by a loop counter, and when the preset total number of accumulations of the convolution window is reached, that is, when the calculation of a complete convolution kernel is completed, an output valid signal is generated; after the multiplication and accumulation calculation is completed, the accumulation result in the accumulation register is first sent to the activation function unit for activation, the activated result is scaled by the scaling unit, and finally, the scaled result is output as the calculation result of the current computing unit.

6. An AI hardware accelerator for neural network computation according to claim 1, characterized in that: The control module is a state machine, used to generate corresponding control signals in various states to control the activation of other modules. The control module includes the following states: IDLE: IDLE is the initial state. In the IDLE state, the input module sends a frame request to the host computer. When it receives the signal that the data has entered the chip from the host computer, it jumps to the RX state to perform network calculations and store the data on the chip. RX: RX is the initial data transmission state. The initial data includes the configuration information of all layers of the network, the initial image data, and the weight information of the first layer of the network. All of these are continuously transmitted into the chip in the form of 128-bit serial transmission. When the signal that the data transmission is complete is received, it jumps to the CALC state to perform data calculation. CALC: CALC is the normal operating computation state. In CALC state, the address generation module generates access addresses for activation data and weight data, and then reads the corresponding m activation data and weight data from the SRAM read / write selection module. After data preprocessing by the shift register module, the weight data is broadcast row-wise to the m×m computation array, and the activation data is broadcast column-wise to the computation array for parallel convolution computation. After Kx×Ky×N clock cycles, the data enters the feedback module. If the current layer is the last layer, it jumps to the TX state to output the results. If the current layer is not the last layer, the return module buffers the data of m channels and generates the corresponding address, and then stores the address and data back into the SRAM read / write selection module as the calculation input for the next layer; after all the calculations of the current layer are completed, it jumps to the DEC state to determine the layer type of the next layer. DEC: DEC is an intermediate transition state. After one layer of the network has completed its computation, it will wait for several cycles to decode the configuration information of the next layer. If the next layer is a convolutional layer, it will jump to the RXW state to receive the weight information used by the next layer; if the next layer is a pooling layer, it will jump to the PL state to perform pooling operations. PL: PL is the pooling state. At this time, the pooling address generation unit generates the pooling address and reads the corresponding activation data from the SRAM read / write selection module into the pooling module to perform the corresponding pooling operation. After the pooling is completed, it jumps to the RXW state and waits to receive the weight information required for the next layer calculation. RXW: The RXW state is used for updating inter-layer weight values. When a transmission end signal is received, it jumps to the CALC state. TX: The TX state is used for on-chip to off-chip result transmission. In the TX state, the output module serially outputs the final calculation result to the off-chip in 8-bit form. After all the results have been transmitted, it returns to the IDLE state to wait for the next frame of data to be processed.

7. An AI hardware accelerator for neural network computation according to claim 6, characterized in that: The on-chip memory includes a weighted data storage unit WRAM, a first active data storage unit ARAM0, and a second active data storage unit ARAM1; The control logic of the SRAM read / write selection module is as follows: When the control module is in RX state and RXW state, the SRAM read / write selection module only receives data from outside the chip. When in the RX state, the initial image data and the first layer weight data are stored sequentially into the first active data storage unit ARAM0 and the weight data storage unit WRAM according to the address generated by the address generation module. When in the RXW state, only the weight data storage unit WRAM that stores weight data is updated, and the activation data is not processed. When in CALC state, the activation data is read from port A of the first activation data storage unit ARAM0 according to the address, and the calculation result is stored back into the second activation data storage unit ARAM1. The next time convolution or pooling is calculated, the functions performed by the first activation data storage unit ARAM0 and the second activation data storage unit ARAM1 are exchanged, that is, the activation data is read from the second activation data storage unit ARAM1, and the result is stored back into the first activation data storage unit ARAM0.

8. An AI hardware accelerator for neural network computation according to claim 1, characterized in that: The control module is connected to the input module via the decoding module, and the input module is connected to the SRAM read / write selection module via the data receiving module. The decoding module is used to decode the configuration data, and the data receiving module is used to write the calculated data into the SRAM read / write selection module.

9. An AI hardware accelerator for neural network computation according to claim 8, characterized in that: The on-chip configuration information of the AI ​​hardware accelerator is a 128-bit binary sequence. The decoding module is used to parse the received configuration data into configuration parameters for controlling neural network computation according to a predefined bit field mapping rule. In the 128-bit binary sequence, fields 0 to 7 represent the current calculation layer number and are used for layer counting; Fields 8 to 23 represent the height H of the input image or feature map, fields 24 to 39 represent the width W of the input image or feature map, and fields 40 to 55 represent the number of channels N of the input image or feature map; Fields 56-71 represent the number of convolution kernels, and fields 72-75 represent the size of the convolution kernels. The size of the convolution kernels is encoded in 4 bits: 0001 represents a 1x1 convolution kernel, 0010 represents a 3x3 convolution kernel, 0011 represents a 5x5 convolution kernel, 0100 represents a 7x7 convolution kernel, 0101 represents a 3x1 convolution kernel, 0110 represents a 5x1 convolution kernel, 0111 represents a 7x1 convolution kernel, and others represent 0x0 convolution kernels, i.e., no convolution operation. Fields 76 and 77 indicate the layer type, with 00 indicating a convolutional layer and 01 indicating a pooling layer; Fields 78 and 79 indicate the pooling type: 00 indicates max pooling and 01 indicates average pooling. Fields 80-81 indicate the non-linearity type: 00 indicates no non-linearity operation after convolution, 01 indicates a ReLU layer, and 10 indicates a Leaky_reLU layer. Fields 82 and 83 indicate the padding type: 00 indicates no padding, 01 indicates padding with 0 values, and 10 indicates padding with edge values. The width or height of the padding is determined by the current convolution kernel size. Fields 84-91 represent the scaling size, used to quantize the multiplicative summed value into int8 form; Fields 92-99 represent the maximum number of network layers. Defining the depth of this storage array in advance is to prevent infinite layers from causing state machine dead loops and logic freezes. Fields 100-127 are reserved spaces to allow for future feature expansion.

Images