A neural network accelerator with adaptive fpga memory resources

Abstract

The application discloses a neural network accelerator with adaptive FPGA storage resources, comprising an instruction control module, an input feature map array, a weight value array and a multiply-accumulate array, wherein the instruction control module decides the cache mode and the calculation mode according to the data and the capacity of the FPGA storage resources; the input feature map array caches the data of the corresponding address in the input feature map data according to the instruction of the instruction control module; the weight value array caches the data of the corresponding address in the weight value data according to the instruction of the instruction control module; the multiply-accumulate array executes the multiply-accumulate operation according to the data in the input feature map array and the weight value array and outputs the convolution operation result according to the instruction of the instruction control module. According to the capacity of the selected FPGA internal storage resources and the capacity of the network layer data, different convolution operation modes are selected, the deployment of the convolution neural network acceleration method on the FPGA is realized, and then the acceleration function is completed.

Description

Technical Field

[0001] This invention belongs to the field of computers, particularly deep learning, and specifically relates to a neural network accelerator that adapts to FPGA storage resources. By properly controlling instructions and selecting different convolution operation modes based on the selected FPGA's internal storage capacity and the network layer's data capacity, this method implements convolutional neural network acceleration on the FPGA, thereby achieving acceleration. Background Art

[0002] In recent years, deep neural network technology has been widely used, particularly in applications such as image processing, object detection, and speech recognition. Deep neural networks offer exceptional accuracy, analyzing vast amounts of data to achieve more precise results. The deeper the deep neural network, the stronger its simulation capabilities, and the more accurate its reasoning.

[0003] Convolutional neural networks are a type of deep neural network. While they offer high accuracy, the computational complexity of the convolutional neural network algorithm increases with the number of neural network layers, placing increasing demands on hardware memory capacity. Once hardware resources are selected and internal storage space is determined, the number of parameters in different network layers can vary for the same algorithm model. Using the same computational approach, the input feature image data or weight parameter data for a large network layer may be too large to be cached in the internal storage. If the hardware's internal data cache is too small, the performance of convolutional neural network acceleration will be affected. Furthermore, for the same algorithm model, a fixed network layer convolution operation can be deployed on FPGAs with abundant storage resources, but cannot be deployed on FPGAs with less storage due to resource constraints. This directly hinders the implementation of convolutional neural network acceleration methods on FPGAs. Summary of the Invention

[0004] The purpose of the present invention is to provide a neural network accelerator with adaptive FPGA storage resources, reasonably control instructions, select different convolution operation modes according to the capacity of the selected FPGA internal storage resources and the capacity of the network layer data, realize the deployment of the convolutional neural network acceleration method on the FPGA, and thus complete the acceleration function.

[0005] To achieve the above object, the technical solution adopted by the present invention is:

[0006] A neural network accelerator with adaptive FPGA storage resources, comprising: an instruction control module, an input feature map array, a weight array, and a multiply-accumulate array, wherein:

[0007] The instruction control module is used to decide the caching method and calculation method of the input feature map data and the weight data according to the input feature map data to be processed and the capacity of the input feature map array, the weight data to be processed and the capacity of the weight array, and issue corresponding instructions to the input feature map array, the weight array and the multiply-accumulate array according to the decision results;

[0008] The input feature map array is used to obtain data at corresponding addresses in the input feature map data for caching according to the instructions issued by the instruction control module;

[0009] The weight array is used to obtain data corresponding to addresses in the weight data for caching according to the instructions issued by the instruction control module;

[0010] The multiplication-accumulation array is used to obtain data in the input feature map array and the weight array according to the instructions issued by the instruction control module, perform multiplication-accumulation operations, and output convolution operation results.

[0011] Several optional methods are also provided below, but they are not intended to be additional limitations on the above-mentioned overall solution. They are merely further supplements or optimizations. Under the premise that there are no technical or logical contradictions, each optional method can be combined separately for the above-mentioned overall solution, or multiple optional methods can be combined.

[0012] Preferably, the caching method is as follows:

[0013] Cache mode 1: the input feature map data is completely cached into the input feature map array, and the weight data is completely cached into the weight array;

[0014] Cache mode 2: The input feature map data is completely cached into the input feature map array, and the weight data is split and cached into the weight array in sequence;

[0015] Cache mode 3: the input feature map data is split and cached into the input feature map array in sequence, and the weight data is completely cached into the weight array;

[0016] Cache mode 4: the input feature map data is split and cached in the input feature map array in sequence, and the weight data is split and cached in the weight array in sequence.

[0017] Preferably, the instruction control module determines a caching method for the input feature map data and the weight data based on the input feature map data to be processed and the capacity of the input feature map array, the weight data to be processed and the capacity of the weight array, and performs the following operations:

[0018] If the input feature map data is less than or equal to the capacity of the input feature map array, and the weight data is less than or equal to the capacity of the weight array, then any one of cache mode 1, cache mode 2, cache mode 3 and cache mode 4 is selected to cache the input feature map data and weight data;

[0019] Alternatively, if the input feature map data is less than or equal to the capacity of the input feature map array, and the weight data is greater than the capacity of the weight array, then any one of cache mode 2 and cache mode 4 is selected to cache the input feature map data and weight data;

[0020] Alternatively, if the input feature map data is larger than the capacity of the input feature map array, and the weight data is less than or equal to the capacity of the weight array, then any one of cache mode 3 and cache mode 4 is selected to cache the input feature map data and weight data;

[0021] Alternatively, if the input feature map data is larger than the capacity of the input feature map array and the weight data is larger than the capacity of the weight array, cache mode 4 is selected to cache the input feature map data and the weight data.

[0022] Preferably, the instruction control module determines a calculation method for the input feature map data and the weight data based on the input feature map data to be processed and the capacity of the input feature map array, the weight data to be processed and the capacity of the weight array, and performs the following operations:

[0023] If the cache mode of the decision is cache mode 1, the calculation method of the decision is: instructing the multiplication and accumulation array to sequentially obtain input feature map data and weight data from the input feature map array and the weight array to perform multiplication and accumulation operations;

[0024] Alternatively, if the cache mode of the decision is cache mode 2, the calculation method of the decision is: commanding the multiplication-accumulation array to first sequentially obtain input feature map data and weight data from the input feature map array and the weight array to perform multiplication-accumulation operations, and then, after the weight data in the weight array is updated, continue to sequentially obtain input feature map data and weight data from the input feature map array and the weight array to perform multiplication-accumulation operations;

[0025] Alternatively, if the cache mode of the decision is cache mode 3, the calculation method of the decision is: instructing the multiplication-accumulation array to first sequentially obtain input feature map data and weight data from the input feature map array and the weight array to perform multiplication-accumulation operations, and then, after the input feature map data in the input feature map array is updated, continue to sequentially obtain input feature map data and weight data from the input feature map array and the weight array to perform multiplication-accumulation operations;

[0026] Alternatively, if the decision cache mode is cache mode 4, the decision calculation method is: command the multiplication and accumulation array to first obtain the input feature map data and weight data from the input feature map array and the weight array in turn to perform multiplication and accumulation operations, and then continue to obtain the input feature map data and weight data from the input feature map array and the weight array in turn to perform multiplication and accumulation operations after the input feature map data in the input feature map array is updated or the weight data in the weight array is updated.

[0027] Preferably, the input feature map data is split in rows, or the input feature map data is split in a manner such that N pixels constitute a whole.

[0028] Preferably, the weight data is split in a manner that the weight data of Q / 2 filters are a whole, or the weight data is split in a manner that R weight data are a whole, where Q is the total number of filters corresponding to the weight data.

[0029] The present invention provides a neural network accelerator with adaptive FPGA storage resources. The instruction control module includes multiple instructions. Different instructions can be issued based on the relationship between the input feature image data and weight parameter data capacity of different network layers and the internal storage capacity of the hardware to implement convolution operations. This reduces the coupling relationship between the input feature image data and weight parameter data capacity of the network layer and the internal storage capacity of the hardware, thereby improving the deployment and implementation of the FPGA. At the same time, when hardware storage resources are limited, adopting different convolution calculation methods for different network layers can further improve the acceleration performance of the accelerator. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 A schematic diagram of the structure of a neural network accelerator with adaptive FPGA storage resources according to the present invention;

[0031] Figure 2 A schematic diagram of a cache method for inputting feature image data and weight data of the present invention;

[0032] Figure 3 Schematic diagram of the method for splitting the input feature map data of the present invention;

[0033] Figure 4 Schematic diagram of the weight data splitting method of the present invention;

[0034] Figure 5 A schematic diagram of the allocation of input feature map array and weight array of the present invention;

[0035] Figure 6 This is a schematic diagram of the convolution operation method when controlling the cache in cache mode 1 of the present invention;

[0036] Figure 7 This is a schematic diagram of the convolution operation method when controlling the cache in cache mode 2 of the present invention;

[0037] Figure 8 This is a schematic diagram of the convolution operation method when controlling the cache in cache mode 3 of the present invention;

[0038] Figure 9 This is a schematic diagram of the convolution operation method when controlling the cache in cache mode 4 of the present invention. DETAILED DESCRIPTION

[0039] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0040] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in the specification of the present invention herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention.

[0041] In order to overcome the defects of neural network accelerators in the prior art, this embodiment provides a neural network accelerator with adaptive FPGA storage resources, which fully utilizes the operation mode of the instruction control accelerator, controls the multiplication and accumulation array to adopt different operation modes through different instructions, and adapts to the FPGA storage resources, so that the convolutional neural network is not limited by the internal storage resources of the FPGA, thereby achieving the purpose of acceleration.

[0042] Specifically, such as Figure 1 As shown, the neural network accelerator with adaptive FPGA storage resources in this embodiment includes: an instruction control module, an input feature map array, a weight array, and a multiplication and accumulation array.

[0043] The instruction control module is used to decide the caching method and calculation method of the input feature map data and weight data based on the input feature map data to be processed and the capacity of the input feature map array, the weight data to be processed and the capacity of the weight array, and issue corresponding instructions to the input feature map array, weight array and multiplication-accumulation array according to the decision results.

[0044] The input feature map array is used to obtain data at corresponding addresses in the input feature map data for caching according to the instructions issued by the instruction control module.

[0045] The weight array is used to obtain data at corresponding addresses in the weight data for caching according to the instructions issued by the instruction control module.

[0046] A multiply-accumulate array is configured to retrieve data from the input feature image array and weight array according to instructions issued by the instruction control module, perform multiply-accumulate operations, and output the convolution operation results. The data retrieved by the multiply-accumulate array is essentially the input feature image array and weight array, and the input feature image data and weight data are sent to the multiply-accumulate array in a sliding window manner based on the convolution kernel size.

[0047] This embodiment uses the instruction control module to control the convolution operation mode of the current network layer. The convolution operation mode of the current network layer is selected based on the capacity of the FPGA internal cache for input feature image data and weight data. In this process, different operation modes can be selected for the same convolutional network layer of the same convolutional neural network algorithm. This enables this embodiment to use the instruction control module to issue different instructions, allowing different convolution operation modes to be used at different network layers, thereby achieving adaptive FPGA storage resources and realizing the purpose of deployment and acceleration on the FPGA.

[0048] In this embodiment, the instruction control module determines how to cache the input feature map data and weight data. The instruction control module sends instructions to the input feature map array and the weight array. The input feature map array and the weight array obtain the corresponding data from the outside and cache them in their own arrays based on the received instructions.

[0049] At the same time, the instruction control module sends specific computational steps to the multiply-accumulate array. More precisely, the instruction control module controls the specific operations performed by the multiply-accumulate array. The instruction control module sends a single instruction, and the multiply-accumulate array executes the task accordingly. It's easy to understand that the instruction control module sends different instructions to the multiply-accumulate array, the weight array, and the input feature map array; each has its own dedicated instruction.

[0050] like Figure 2 As shown, the caching methods set in this embodiment include:

[0051] Cache mode 1: The input feature map data (feature image) is completely cached into the input feature map array, and the weight data is completely cached into the weight array.

[0052] Cache mode 2: The input feature map data is completely cached into the input feature map array, and the weight data is split and cached into the weight array in sequence.

[0053] Cache mode 3: The input feature map data is split and cached in the input feature map array in sequence, and the weight data is completely cached in the weight array.

[0054] Cache mode 4: the input feature map data is split and cached in the input feature map array in sequence, and the weight data is split and cached in the weight array in sequence.

[0055] It should be noted that the cache mode set in this embodiment is a conventional and general cache mode, and can be increased or decreased as needed in other embodiments.

[0056] Based on the four caching methods mentioned above, the instruction control module performs the following operations when deciding the caching method for input feature map data and weight data:

[0057] If the input feature map data is less than or equal to the capacity of the input feature map array, and the weight data is less than or equal to the capacity of the weight array, then any one of cache mode 1, cache mode 2, cache mode 3 and cache mode 4 is selected to cache the input feature map data and weight data.

[0058] Alternatively, if the input feature map data is less than or equal to the capacity of the input feature map array, and the weight data is greater than the capacity of the weight array, then any one of cache mode 2 and cache mode 4 is selected to cache the input feature map data and weight data.

[0059] Alternatively, if the input feature map data is larger than the capacity of the input feature map array, and the weight data is less than or equal to the capacity of the weight array, then any one of cache mode 3 and cache mode 4 is selected to cache the input feature map data and weight data.

[0060] Alternatively, if the input feature map data is larger than the capacity of the input feature map array and the weight data is larger than the capacity of the weight array, cache mode 4 is selected to cache the input feature map data and the weight data.

[0061] That is, in this embodiment, when the input feature map array can fully store the input feature map data, the input feature map data can still be split according to other factors (such as calculation pressure, rate), etc., to improve calculation flexibility. The weight data is understood in the same way.

[0062] Based on the four cache methods mentioned above, the instruction control module performs the following operations when deciding how to calculate the input feature map data and weight data:

[0063] If the decision cache mode is cache mode 1, the decision calculation method is: command the multiplication and accumulation array to obtain input feature map data and weight data from the input feature map array and the weight array in turn to perform multiplication and accumulation operations.

[0064] Alternatively, if the caching method for the decision is caching method 2, the calculation method for the decision is as follows: command the multiply-accumulate array to first sequentially obtain input feature map data and weight data from the input feature map array and the weight array for multiply-accumulate operations, and then, after the weight data in the weight array is updated, continue to sequentially obtain input feature map data and weight data from the input feature map array and the weight array for multiply-accumulate operations.

[0065] Alternatively, if the caching method for the decision is caching method 3, the calculation method for the decision is as follows: command the multiply-accumulate array to first sequentially obtain input feature map data and weight data from the input feature map array and the weight array for multiply-accumulate operations, and then, after the input feature map data in the input feature map array is updated, continue to sequentially obtain input feature map data and weight data from the input feature map array and the weight array for multiply-accumulate operations.

[0066] Alternatively, if the caching method for the decision is caching method 4, the calculation method for the decision is as follows: command the multiply-accumulate array to first sequentially obtain input feature map data and weight data from the input feature map array and the weight array for multiply-accumulate operations, and then, after the input feature map data in the input feature map array is updated or the weight data in the weight array is updated, continue to sequentially obtain input feature map data and weight data from the input feature map array and the weight array for multiply-accumulate operations.

[0067] Taking caching method 2 as an example, where the weight data needs to be split and cached successively, in this case, multiple multiply-accumulate operations need to be performed according to the order of data update in the weight array. For example, the weight data is split into two parts. After completing the multiply-accumulate operation of the first part of the weight data and the input feature map data, the instruction control module triggers the weight array to read and cache the second part of the weight data. After the weight array updates its cached data to the second part of the weight data, the instruction control module triggers the multiply-accumulate array to read the data in the input feature map array and the weight array for multiply-accumulate operations to complete the final convolution operation.

[0068] As Figure 3 shown, where the input feature map data is split row by row (in the figure, C represents the number of channels of the input feature map data, m represents the number of rows of the input feature map data, n is the number of rows used to split the input feature map data, 1 <= n < m), or the input feature map data is split in a way that N pixel points are taken as a whole (in the figure, M is the total number of pixel points of the input feature map data, N is the number of pixel points used to split the input feature map data, 1 <= N < M).

[0069] As Figure 4As shown, the weight data is split in a way that the weight data of Q / 2 filters are a whole (in the figure, C represents the number of channels of the weight data, and Q is the total number of filters corresponding to the weight data), or the weight data is split in a way that R weight data are a whole, and the weight data of each filter is split in units of R.

[0070] It should be noted that the input feature map data and weight data can be split equally or not, depending on the actual situation.

[0071] To increase storage flexibility, such as Figure 5 As shown, the input feature image data and weight parameter data can be cached in different locations in the same array, or cached in different arrays.

[0072] For ease of understanding, the following description is based on a specific example.

[0073] Taking the case where the space for caching input feature image data (i.e., the capacity of the input feature map array) in the hardware internal FPGA storage resource is 1MB, and the space for caching weight data (i.e., the capacity of the weight array) is 2MB as an example, the convolutional neural network acceleration method is as follows:

[0074] Case 1: When the input feature image data of the network layer is 512KB (less than 1MB) and the weight data is 256KB (less than 2MB), the instruction control module decides the cache mode to be cache mode 1, such as Figure 6 As shown. The input feature image data and weight data are cached in the input feature image array and weight array, respectively, within the hardware. The multiplication-accumulation array then sequentially retrieves the corresponding input feature image data and weight data from the input feature image array and weight array, performing multiplication-accumulation operations, and finally outputs the calculated result data. In this case, the instruction control module can also select caching mode 2 through 4. The detailed calculation process is described in Cases 2 through 4 below.

[0075] Case 2: When the input feature image data of the network layer is 512KB (less than 1MB) and the weight data is 3MB (greater than 2MB), the instruction control module decides to use cache mode 2, such as Figure 7As shown. The input feature image data is obtained and cached in the input feature image array inside the hardware. At the same time, 1.5MB (this embodiment takes equal division as an example) of weight data is first obtained and cached in the weight array. Then the multiplication and accumulation array obtains the corresponding input feature image data and weight data from the input feature image array and the weight array in turn to perform multiplication and accumulation operations, and outputs the calculated result data. When the 1.5MB weight data in the weight array is used up, the remaining 1.5MB weight data is obtained and cached in the weight array, and then the above-mentioned convolution operation process is repeated. In addition, in this case, the instruction control module can also choose the cache mode as cache mode 4, and the detailed calculation process is described in the following situation 4.

[0076] Case 3: When the input feature image data of the network layer is 1.6MB (greater than 1MB) and the weight data is 256KB (less than 2MB), the instruction control module decides to use cache mode 3, such as Figure 8 As shown. First, 0.8MB of input feature image data is obtained and cached in the input feature image array inside the hardware, and at the same time, all weight data is obtained and cached in the weight array. Then the multiplication and accumulation array obtains the corresponding input feature image data and weight data from the input feature image array and the weight array in turn to perform multiplication and accumulation operations, and outputs the calculated result data. When the input feature image data in the input feature image array is used up, the remaining 0.8MB of input feature image data is obtained and cached in the input feature image array, and then the above-mentioned convolution operation process is repeated. In addition, in this case, the instruction control module can also choose the cache mode as cache mode 4, and the detailed calculation process is described in the following situation 4.

[0077] Case 4: When the input feature image data of the network layer is 1.6MB (greater than 1MB) and the weight data is 3MB (greater than 2MB), the instruction control module decides to use cache mode 4, such as Figure 9As shown. First, 0.8MB of input feature image data is obtained and cached in the input feature image array inside the hardware. At the same time, the first 1.5MB of weight data is obtained and cached in the weight array. Then the multiplication and accumulation array obtains the corresponding input feature image data and weight data from the input feature image array and the weight array in turn to perform multiplication and accumulation operations, and outputs the calculated result data. When the 1.5MB of weight data in the weight array is used up, the remaining 1.5MB of weight data is obtained and cached in the weight array, and then the above-mentioned convolution operation process is repeated. When all the weight data and the first 0.8MB of input feature image data have completed the convolution operation, the remaining 0.8MB of input feature image data is obtained and cached in the input feature image array inside the hardware. At the same time, the first 1.5MB of weight data is obtained again and cached in the weight array, and then the above-mentioned convolution operation process is repeated, and the remaining 1.5MB of weight data is obtained and cached in the weight array, until the remaining 0.8MB of data and all the weight data complete the corresponding convolution operation.

[0078] In summary, the adaptive FPGA storage resource neural network accelerator of this embodiment reasonably controls instructions and selects different convolution operation modes according to the selected FPGA internal storage resource capacity and network layer data capacity, thereby implementing the deployment of the convolutional neural network acceleration method on the FPGA and completing the acceleration function.

[0079] The technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above-mentioned embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0080] The above-described embodiments merely illustrate several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that a person skilled in the art would be able to make numerous modifications and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be determined by the appended claims.

Claims

1. A neural network accelerator with adaptive FPGA storage resources, characterized in that: The neural network accelerator with adaptive FPGA storage resources includes: an instruction control module, an input feature map array, a weight array, and a multiplication-accumulation array, wherein: The instruction control module is used to decide the caching method and calculation method of the input feature map data and the weight data according to the input feature map data to be processed and the capacity of the input feature map array, the weight data to be processed and the capacity of the weight array, and issue corresponding instructions to the input feature map array, the weight array and the multiply-accumulate array according to the decision results; The input feature map array is used to obtain data at corresponding addresses in the input feature map data for caching according to the instructions issued by the instruction control module; The weight array is used to obtain data corresponding to addresses in the weight data for caching according to the instructions issued by the instruction control module; The multiplication-accumulation array is used to obtain data in the input feature map array and the weight array according to the instructions issued by the instruction control module, perform multiplication-accumulation operations, and output convolution operation results.

2. The neural network accelerator with adaptive FPGA storage resources according to claim 1, characterized in that: The caching method is as follows: Cache mode 1: the input feature map data is completely cached into the input feature map array, and the weight data is completely cached into the weight array; Cache mode 2: The input feature map data is completely cached into the input feature map array, and the weight data is split and cached into the weight array in sequence; Cache mode 3: the input feature map data is split and cached into the input feature map array in sequence, and the weight data is completely cached into the weight array; Cache mode 4: the input feature map data is split and cached in the input feature map array in sequence, and the weight data is split and cached in the weight array in sequence.

3. The neural network accelerator with adaptive FPGA storage resources according to claim 2, characterized in that: The instruction control module determines a cache mode for the input feature map data and the weight data based on the input feature map data to be processed and the capacity of the input feature map array, the weight data to be processed and the capacity of the weight array, and performs the following operations: If the input feature map data is less than or equal to the capacity of the input feature map array, and the weight data is less than or equal to the capacity of the weight array, then any one of cache mode 1, cache mode 2, cache mode 3 and cache mode 4 is selected to cache the input feature map data and weight data; Alternatively, if the input feature map data is less than or equal to the capacity of the input feature map array, and the weight data is greater than the capacity of the weight array, then any one of cache mode 2 and cache mode 4 is selected to cache the input feature map data and weight data; Alternatively, if the input feature map data is larger than the capacity of the input feature map array, and the weight data is less than or equal to the capacity of the weight array, then any one of cache mode 3 and cache mode 4 is selected to cache the input feature map data and weight data; Alternatively, if the input feature map data is larger than the capacity of the input feature map array and the weight data is larger than the capacity of the weight array, cache mode 4 is selected to cache the input feature map data and the weight data.

4. The neural network accelerator with adaptive FPGA storage resources according to claim 2, characterized in that: The instruction control module determines a calculation method for the input feature map data and the weight data based on the input feature map data to be processed and the capacity of the input feature map array, the weight data to be processed and the capacity of the weight array, and performs the following operations: If the cache mode of the decision is cache mode 1, the calculation method of the decision is: instructing the multiplication and accumulation array to sequentially obtain input feature map data and weight data from the input feature map array and the weight array to perform multiplication and accumulation operations; Alternatively, if the cache mode of the decision is cache mode 2, the calculation method of the decision is: commanding the multiplication-accumulation array to first sequentially obtain input feature map data and weight data from the input feature map array and the weight array to perform multiplication-accumulation operations, and then, after the weight data in the weight array is updated, continue to sequentially obtain input feature map data and weight data from the input feature map array and the weight array to perform multiplication-accumulation operations; Alternatively, if the cache mode of the decision is cache mode 3, the calculation method of the decision is: instructing the multiplication-accumulation array to first sequentially obtain input feature map data and weight data from the input feature map array and the weight array to perform multiplication-accumulation operations, and then, after the input feature map data in the input feature map array is updated, continue to sequentially obtain input feature map data and weight data from the input feature map array and the weight array to perform multiplication-accumulation operations; Alternatively, if the decision cache mode is cache mode 4, the decision calculation method is: command the multiplication and accumulation array to first obtain the input feature map data and weight data from the input feature map array and the weight array in turn to perform multiplication and accumulation operations, and then continue to obtain the input feature map data and weight data from the input feature map array and the weight array in turn to perform multiplication and accumulation operations after the input feature map data in the input feature map array is updated or the weight data in the weight array is updated.

5. The neural network accelerator with adaptive FPGA storage resources according to claim 2, characterized in that: The input feature map data is split in rows, or the input feature map data is split in a manner that N pixels constitute a whole.

6. The neural network accelerator with adaptive FPGA storage resources according to claim 2, characterized in that: The weight data is split in a manner that the weight data of Q / 2 filters are a whole, or the weight data is split in a manner that R weight data are a whole, where Q is the total number of filters corresponding to the weight data.

Images