A convolutional neural network hardware acceleration device with dynamic cache space allocation

Abstract

The present invention discloses a convolutional neural network hardware acceleration device with dynamic cache space allocation, including: a parameter storage module for caching the structural parameters and space allocation parameters of the convolutional neural network and each convolution layer thereof; an input cache module for caching the weight data of the convolution kernel and the input feature map data according to the space allocation parameters; a weight data conversion module for obtaining the weight data of the convolution kernel and performing format conversion; a feature map data conversion module for obtaining the input feature map data and performing format conversion; an operation array module for performing multiplication and accumulation operations on the weight data and the input feature map data after format conversion, and outputting the result as the output feature map data of the corresponding layer; an output cache module for caching the output feature map data, and the output feature map data is used as the input feature map data of the next convolution layer. The device reduces the amount of external input data in the calculation process, is suitable for various convolutional neural network hardware accelerations, and is easy to deploy.

Description

Technical Field

[0001] The present invention belongs to the field of computer technology, and in particular relates to a convolutional neural network hardware acceleration device with dynamic cache space allocation. Background Art

[0002] In recent years, artificial intelligence has been widely used in many fields, bringing a lot of convenience to people's lives. In the field of image processing, compared with traditional machine learning algorithms, deep convolutional neural networks have higher accuracy, even surpassing human accuracy.

[0003] Generally, the more layers and parameters a convolutional neural network has, the more accurate the inference results will be. But at the same time, the more network parameters there are, the more cache bandwidth will be consumed. In the convolutional neural network structure, the calculations within the layers are independent and unrelated, while the calculations between the layers are similar to the pipeline structure, which is not efficient to implement using a general-purpose processor. Due to the special calculation mode of convolutional neural networks, they are particularly suitable for hardware acceleration.

[0004] However, convolutional deep neural networks have a huge amount of data, so how to reduce the external storage access bandwidth of convolutional deep neural network hardware accelerators has always been a hot research direction in the field of artificial intelligence. How to be compatible with various deep neural network algorithms, use the limited internal cache of the hardware accelerator, maintain a simple hardware structure of pipeline and parallelism, and reduce the amount of external data access is the difficulty of hardware acceleration at present. Summary of the invention

[0005] The purpose of the present invention is to address the above-mentioned problems and propose a convolutional neural network hardware acceleration device with dynamic cache space allocation, which reduces the overall amount of data input from the outside during the neural network calculation process, greatly improves the hardware acceleration performance, and is suitable for hardware acceleration of various convolutional neural networks and is easy to deploy.

[0006] To achieve the above object, the technical solution adopted by the present invention is:

[0007] The present invention proposes a convolutional neural network hardware acceleration device with dynamic cache space allocation, comprising a parameter storage module, an input cache module, a weight data conversion module, a feature map data conversion module, a calculation array module and an output cache module, wherein:

[0008] A parameter storage module is used to cache the structural parameters of the convolutional neural network and each convolutional layer thereof and the spatial allocation parameter K. The spatial allocation parameter K is determined by judging the size of the convolution kernel of the corresponding convolutional layer and the input feature map data;

[0009] An input cache module, comprising N cache units, and used to obtain the weight data of the convolution kernel and the input feature map data of the corresponding address from the data bus according to the structural parameters, and then cache the weight data of the convolution kernel and the input feature map data according to the spatial allocation parameter K, wherein K cache units are used to cache the weight data of the convolution kernel, and NK cache units are used to cache the input feature map data;

[0010] The weight data conversion module is used to obtain the weight data of the convolution kernel from the input buffer module and perform format conversion;

[0011] A feature map data conversion module is used to obtain input feature map data from the input cache module and perform format conversion;

[0012] The operation array module is used to perform multiplication and accumulation operations on the weight data after format conversion and the input feature map data, and output the convolution operation result as the output feature map data of the corresponding layer;

[0013] The output cache module is used to cache the output feature map data, and the output feature map data is used as the input feature map data of the next layer.

[0014] Preferably, the operation array module performs convolution operation at a layer granularity, and the amount of data processed by each convolution operation is: performing convolution operation on the input feature map data of the current convolution layer with a size of C*R*CN and a convolution kernel with a size of W*H*CN*G;

[0015] Among them, C represents the width of the input feature map data, R represents the height of the input feature map data, CN represents the number of channels, W represents the width of the convolution kernel, H represents the height of the convolution kernel, and G represents the number of convolution kernel groups.

[0016] Preferably, each cache unit is a random access memory with a storage space of M.

[0017] Preferably, the spatial allocation parameter K is determined by judging the size of the convolution kernel of the corresponding convolution layer and the input feature map data, as follows:

[0018] If the size of the convolution kernel W*H*CN*G ≤ (N-1)*M, then the convolution operation of this layer is performed, and K is set to N-1;

[0019] If the size of the convolution kernel W*H*CN*G>(N-1)*M, and the size of the input feature map data C*R*CN≤(N-1)*M, then the convolution operation of this layer is performed, and K is set to 1;

[0020] If the size of the convolution kernel W*H*CN*G>(N-1)*M, the size of the input feature map data C*R*CN>(N-1)*M, and W*H*CN*G<C*R*CN, then for this layer of convolution operation, K is set to N-1;

[0021] If the size of the convolution kernel W*H*CN*G>(N-1)*M, the size of the input feature map data C*R*CN>(N-1)*M, and W*H*CN*G>C*R*CN, then the convolution operation of this layer is performed, and K is set to 1;

[0022] If the size of the convolution kernel W*H*CN*G>(N-1)*M, the size of the input feature map data C*R*CN>(N-1)*M, and W*H*CN*G=C*R*CN, then for this layer of convolution operation, K is set to 1 or N-1;

[0023] Among them, C represents the width of the input feature map data, R represents the height of the input feature map data, CN represents the number of channels, W represents the width of the convolution kernel, H represents the height of the convolution kernel, and G represents the number of convolution kernel groups.

[0024] Preferably, determining the size of the convolution kernel of the corresponding layer and the input feature map data is completed by a parameter storage module or an external CPU.

[0025] Compared with the prior art, the present invention has the following beneficial effects:

[0026] (1) This application adopts a hardware structure of a multiplexed input cache module, dynamically allocates input cache according to the size of the convolution kernel or input feature map data in the current calculation convolution layer, reduces the overall amount of data input from the outside during the convolutional neural network calculation process, and improves hardware acceleration performance;

[0027] (2) This application maintains the original structure of the convolutional neural network and only ensures that the convolutional neural network operation consumes as little external cache bandwidth as possible by adjusting the weight data of the cached convolution kernel or the input cache size of the input feature map data, thereby significantly improving the operation efficiency without affecting the data storage structure and calculation method. It is suitable for hardware acceleration of various convolutional neural networks and is easy to deploy. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 It is a schematic diagram of the structure of the convolutional neural network hardware acceleration device of the present invention;

[0029] Figure 2 A schematic diagram of a storage method of an input cache module of the present invention;

[0030] Figure 3 A schematic diagram of the input cache module of the present invention preferentially storing weight data of convolution kernels;

[0031] Figure 4 A schematic diagram of preferentially storing input feature map data for the input cache module of the present invention. DETAILED DESCRIPTION

[0032] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0033] It should be noted that, unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application.

[0034] The present application provides a convolutional neural network hardware acceleration device with dynamic cache space allocation, which solves the problem that the current convolutional neural network consumes a lot of external cache bandwidth and the general processor has low implementation efficiency.

[0035] like Figure 1-4 As shown, a convolutional neural network hardware acceleration device with dynamic cache space allocation includes a parameter storage module, an input cache module, a weight data conversion module, a feature map data conversion module, a calculation array module and an output cache module, wherein:

[0036] A parameter storage module is used to cache the structural parameters of the convolutional neural network and each convolutional layer thereof and the spatial allocation parameter K. The spatial allocation parameter K is determined by judging the size of the convolution kernel of the corresponding convolutional layer and the input feature map data;

[0037] An input cache module, comprising N cache units, and used to obtain the weight data of the convolution kernel and the input feature map data of the corresponding address from the data bus according to the structural parameters, and then cache the weight data of the convolution kernel and the input feature map data according to the spatial allocation parameter K, wherein K cache units are used to cache the weight data of the convolution kernel, and NK cache units are used to cache the input feature map data;

[0038] The weight data conversion module is used to obtain the weight data of the convolution kernel from the input buffer module and perform format conversion;

[0039] A feature map data conversion module is used to obtain input feature map data from the input cache module and perform format conversion;

[0040] The operation array module is used to perform multiplication and accumulation operations on the weight data after format conversion and the input feature map data, and output the convolution operation result as the output feature map data of the corresponding layer;

[0041] The output cache module is used to cache the output feature map data, and the output feature map data is used as the input feature map data of the next layer.

[0042] Among them, Figure 1 As shown in the figure, during the convolution calculation process, the convolution neural network hardware acceleration device obtains the input feature map data and convolution kernel through the data bus, and outputs the final calculation result data through the data bus after the convolution calculation is completed. Data transmission is realized through the bus, which has higher data transmission stability and integrity. The application of each module is as follows:

[0043] The parameter storage module can be used to store any convolutional neural network compilation information in the prior art, such as storing YOLO and MobileNet series as neural networks to be accelerated and optimized during this convolution operation. And by setting different configuration parameters, the hardware acceleration device of this application can be adapted to different convolutional neural network algorithms to ensure the universality of the hardware acceleration device.

[0044] The input cache module can obtain the weight data of the convolution kernel and the input feature map data of the corresponding address according to the size of the input feature map data, the convolution kernel size, the calculation step, the input feature map, and the starting address of the convolution kernel external cache in the corresponding layer structure parameters, and then dynamically allocate cache space for the obtained convolution kernel weight data and input feature map data according to the spatial allocation parameter K of the corresponding layer.

[0045] The weight data conversion module and the feature map data conversion module obtain the weight data of the convolution kernel and the input feature map data from the input buffer module, and convert them into the format required by the operation array module, such as arranging them in the channel priority calculation order, that is, first arranging all the channel data of a pixel point. The operation array module includes a number of calculation units, each of which is used for all the convolution operations required for an output feature map pixel point. When the integer multiple of the number of calculation units in the operation array is not satisfied, it is necessary to fill in zeros to form an integer multiple, and then arrange all the channel data of other pixels. Among them, the weight data conversion module can perform the format conversion of the corresponding number of weight data according to the number of convolution kernels and the number of cycles required for one calculation in the corresponding layer structure parameters; the feature map data conversion module can perform the format conversion of the corresponding input feature map data according to the number of feature maps and the number of cycles required for one calculation set in the corresponding layer structure parameters.

[0046] The operation array module performs multiplication and accumulation operations on the weight data after format conversion and the input feature map data, and outputs the convolution operation result as the output feature map data of the current layer, such as performing operations based on the number of clock beat cycles and the number of loops required for one calculation in the corresponding layer structure parameters.

[0047] The output cache module is used to cache the output feature map data and store the complete data in the external cache through the data bus, such as storing it in the external cache according to the size of the output feature map in the corresponding layer structure parameters and the starting address of the output feature map external cache; the output feature map data of the middle layer can be used as the input feature map data of the next layer, and the output feature map data of the final result layer can be used as the final output result data.

[0048] In one embodiment, in order to ensure a simple hardware structure and make full use of the internal cache space, the operation array module performs convolution operations at a layer granularity, and the amount of data processed by each convolution operation is: convolution operations are performed on the input feature map data of the current convolution layer with a size of C*R*CN and a convolution kernel with a size of W*H*CN*G;

[0049] Among them, C represents the width of the input feature map data, R represents the height of the input feature map data, CN represents the number of channels, W represents the width of the convolution kernel, H represents the height of the convolution kernel, and G represents the number of convolution kernel groups.

[0050] It is easy to understand that if the current layer is not the last layer of the convolutional neural network, it means that the output feature map data is not the final convolution calculation result of the entire convolutional neural network, then the output feature map data of the current layer will be used as the input feature map data of the next layer of convolution operation; if the current layer is the last layer of the convolutional neural network, it means that the output feature map data is the final convolution calculation result of the entire convolutional neural network, then the output feature map data of the current layer will be output through the data bus. Figure 1 As shown, the operation array module also includes a plurality of computing units, each of which is used for all convolution operations required for an output feature map pixel.

[0051] In one embodiment, each cache unit is a random access memory with a storage space of M.

[0052] In one embodiment, the spatial allocation parameter is determined by judging the size of the convolution kernel of the corresponding convolution layer and the input feature map data, as follows:

[0053] If the size of the convolution kernel W*H*CN*G ≤ (N-1)*M, then the convolution operation of this layer is performed, and K is set to N-1;

[0054] If the size of the convolution kernel W*H*CN*G>(N-1)*M, and the size of the input feature map data C*R*CN≤(N-1)*M, then the convolution operation of this layer is performed, and K is set to 1;

[0055] If the size of the convolution kernel W*H*CN*G>(N-1)*M, the size of the input feature map data C*R*CN>(N-1)*M, and W*H*CN*G<C*R*CN, then for this layer of convolution operation, K is set to N-1;

[0056] If the size of the convolution kernel W*H*CN*G>(N-1)*M, the size of the input feature map data C*R*CN>(N-1)*M, and W*H*CN*G>C*R*CN, then the convolution operation of this layer is performed, and K is set to 1;

[0057] If the size of the convolution kernel W*H*CN*G>(N-1)*M, the size of the input feature map data C*R*CN>(N-1)*M, and W*H*CN*G=C*R*CN, then for this layer of convolution operation, K is set to 1 or N-1;

[0058] Among them, C represents the width of the input feature map data, R represents the height of the input feature map data, CN represents the number of channels, W represents the width of the convolution kernel, H represents the height of the convolution kernel, and G represents the number of convolution kernel groups.

[0059] Specifically, if the size of the convolution kernel of the convolution layer is smaller than or equal to the size of the N-1 cache unit spaces of the input cache module, that is, W*H*CN*G≤(N-1)*M, the input cache module can use N-1 cache units to cache the weight data of the convolution kernel, and use 1 cache unit to cache the input feature map data. The total amount of data that needs to be read from the outside is C*R*CN+W*H*CN*G; compared with the input cache module using N-1 cache units to cache the input feature map data, the amount of data input from the outside can be reduced by (W*H*CN*G)*((C*R*CN) / ((N-1)*M)-1).

[0060] If the size of the convolution kernel of the convolution layer is larger than the space size of the N-1 cache units of the input cache module, that is, W*H*CN*G>(N-1)*M, and the size of the input feature map data is smaller or equal to the space size of the N-1 cache units of the input cache module, that is, C*R*CN≤(N-1)*, then the input cache module can use N-1 cache units to cache the input feature map data, and use 1 cache unit to cache the convolution kernel data. The total amount of data that needs to be read from the outside is C*R*CN+W*H*CN*G; compared with the input cache module using N-1 cache units to cache the input feature map data, the amount of data input from the outside can be reduced by (C*R*CN)*((W*H*CN*G) / ((N-1)*M)-1).

[0061] If the size of the convolution kernel of the convolution layer is larger than the space size of the N-1 cache units of the input cache module, and the size of the input feature map data is larger than the space size of the N-1 cache units of the input cache module, that is, W*H*CN*G>(N-1)*M, C*R*CN>(N-1)*M, and the size of the input feature map data of the convolution layer is larger than the size of the convolution kernel, that is, W*H*CN*G<C*R*CN, then the input cache module can use N-1 cache units to cache the weight data of the convolution kernel, and use 1 cache unit to cache the input feature map data. The total amount of data that needs to be read from the outside is (W*H*CN*G)*(C*R*CN) / ((N-1)*M)+W*H*CN*G; compared with the input cache module using N-1 cache units to cache the input feature map data, the amount of data input from the outside can be reduced by C*R*CN-W*H*CN*G.

[0062] If the size of the convolution kernel of the convolution layer is larger than the space size of the N-1 cache units of the input cache module, the size of the input feature map data is larger than the space size of the N-1 cache units of the input cache module, that is, W*H*CN*G>(N-1)*M, C*R*CN>(N-1)*M, and the size of the input feature map data of the convolution layer is smaller than the size of the convolution kernel, that is, W*H*CN*G>C*R*CN, then the input cache module uses N-1 cache units to cache the input feature map data, and uses 1 cache unit to cache the weight data of the convolution kernel. The total amount of data that needs to be read from the outside is (W*H*CN*G)*(C*R*CN) / ((N-1)*M)+C*R*CN; compared with the input cache module using N-1 cache units to cache the weight data of the convolution kernel, the amount of data input from the outside can be reduced by W*H*CN*GC*R*CN.

[0063] If the size of the convolution kernel of the convolution layer is larger than the space size of the N-1 cache units of the input cache module, and the size of the input feature map data is larger than the space size of the N-1 cache units of the input cache module, that is, W*H*CN*G>(N-1)*M, C*R*CN>(N-1)*M, and if the size of the input feature map data of the convolution layer is equal to the size of the convolution kernel, that is, W*H*CN*G=C*R*CN, then the input cache module can use N-1 cache units to cache the convolution kernel The weight data of the input feature map data is cached using 1 cache unit. The total amount of data that needs to be read from the outside is (W*H*CN*G)*(C*R*CN) / ((N-1)*M)+W*H*CN*G. Alternatively, the input cache module can use N-1 cache units to cache the input feature map data. The total amount of data that needs to be read from the outside is (W*H*CN*G)*(C*R*CN) / ((N-1)*M)+C*R*CN. The amount of data input from the outside is the same in both methods.

[0064] Usually, the input feature map data size corresponding to the front convolution layer in the convolutional neural network is larger than the convolution kernel size, while the input feature map data size corresponding to the back convolution layer is smaller than the convolution kernel size. If the convolutional neural network includes an even number of convolution layers, the front convolution layer and the back convolution layer are divided in half. Dynamically adjust the cache space of the input cache unit of the convolution kernel weight data or input feature map data to ensure that the external cache bandwidth consumed by the convolutional neural network operation is as small as possible, significantly improving the operation efficiency.

[0065] In one embodiment, the size of the convolution kernel and the input feature map data of the corresponding layer is determined by the parameter storage module or the external CPU. The allocation information of the N cache units of the input cache module (such as the spatial allocation parameter) can be completed by the compilation module on the external CPU side. After the compilation module performs comparison processing, the allocation information of the N cache units of the input cache module of each layer is configured to the parameter storage module of the hardware acceleration device; the parameter storage module can also automatically compare the relative size of the convolution kernel and the input feature map data of each convolution layer to generate the allocation information of the N cache units of the input cache module of the corresponding layer.

[0066] The convolutional neural network hardware acceleration device with dynamic cache space allocation dynamically allocates input cache by the size of the convolution kernel or input feature map data in the current calculation convolution layer, thereby reducing the overall amount of data input from the outside during the convolutional neural network calculation process and improving the hardware acceleration performance; and maintaining the original structure of the convolutional neural network, only dynamically allocating the input cache space size for storing the weight data of the convolution kernel or the input feature map data, ensuring that the convolutional neural network operation consumes as little external cache bandwidth as possible, significantly improving the operation efficiency, and not affecting the data storage structure and calculation method.

[0067] The following is a detailed description through specific embodiments.

[0068] Before performing convolutional neural network accelerated calculations, the compiled convolutional neural network information is read from the external SDRAM cache into the parameter storage module, and the allocation information of the N cache units of the input cache module is determined according to the relative size of the convolution kernel of each convolutional layer and the input feature map data.

[0069] According to the allocation information of the N cache units of the input cache module, the weight data of the convolution kernel and the input feature map data dynamically use the cache of the N cache units of the input cache module, that is, whoever has a smaller amount of data uses more cache units.

[0070] like Figure 3As shown in the figure, when a certain layer of convolution operation has input feature map data that is larger than the convolution kernel, the input cache module uses N-1 cache units to cache the weight data of the convolution kernel and uses 1 cache unit to cache the input feature map data.

[0071] like Figure 4 As shown in the figure, when a certain layer of convolution operation is performed and the input feature map data is smaller than the convolution kernel, the input cache module uses N-1 cache units to cache the input feature map data and uses 1 cache unit to cache the weight data of the convolution kernel.

[0072] For example, there is a layer of convolution operation, the input feature map data size C*R*CN is 208*208*256 bytes, the convolution kernel size W*H*CN*M is 3*3*256*256 bytes, and the input cache module is composed of N cache units, N is 16, and each cache unit is a random access memory with a storage space of M, M is 4*1024 bytes. 208*208*256>3*3*256*256, for this layer of convolution operation, the input cache module uses N-1 (i.e. 15) cache units to cache the weights of the convolution kernel, and the total amount of data that needs to be read from the outside is 208*208*256*(3*3*256*256 / (15*4*1024))+3*3*256*256=111345664 bytes. However, if the input cache module uses N-1 (i.e., 15) cache units to cache the input feature map data, the total amount of data that needs to be read from the outside is 3*3*256*256*(208*208*256 / (15*4*1024))+208*208*256=117833728 bytes. The total amount of data that needs to be read from the outside by the dynamic allocation cache solution can be reduced by 117833728-111345664=6488064 bytes.

[0073] For example, there is a layer of convolution operation, the input feature map data size C*R*CN is 13*13*1024 bytes, the convolution kernel size W*H*CN*M is 3*3*1024*4096 bytes, and the input cache module is composed of N cache units, N is 16, and each cache unit is a random access memory with a storage space of M, M is 4*1024. 3*13*1024<3*3*1024*4096. For this layer of convolution operation, the input cache module uses N-1 (i.e. 15) cache units to cache the input feature map data. The total amount of data that needs to be read from the outside is 3*3*1024*4096*(13*13*1024 / (15*4*1024))+13*13*1024=113419264 bytes. However, if the input cache module uses N-1 (i.e. 15) cache units to cache the weights of the convolution kernel, the total amount of data that needs to be read from the outside is 13*13*1024*(3*3*1024*4096 / (15*4*1024))+3*3*1024*4096=125303808 bytes. The total amount of data that needs to be read from the outside by the dynamic allocation cache solution is reduced by 125303808-113419264=11884544 bytes.

[0074] Combined with the above two-layer convolution operation example, the total amount of data that needs to be read from the outside by the dynamic allocation cache scheme is 111345664+113419264=224764928 bytes. If the input cache module uses N-1 (i.e. 15) cache units to cache the weights of the convolution kernel, the total amount of data that needs to be read from the outside for the two layers is 111345664+125303808=236649472 bytes; the total amount of data that needs to be read from the outside by the dynamic allocation cache scheme is reduced by 236649472-224764928=11884544 bytes.

[0075] If the input cache module uses N-1 (i.e. 15) cache units to cache the input feature map data, the total amount of data that needs to be read from the outside for the two layers is 117833728+113419264=231252992 bytes; the total amount of data that needs to be read from the outside by the dynamic allocation cache scheme is reduced by 231252992-224764928=6488064 bytes.

[0076] It should be noted that the N cache units of the input cache module can also be arbitrarily allocated according to actual needs. In order to make the description concise, not all possible allocations of the cache units of the input cache module in the above embodiments are described. However, as long as there is no contradiction in the allocation of the cache units of these input cache modules, they should be considered to be within the scope of this specification.

[0077] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0078] The above-described embodiments only express the more specific and detailed embodiments described in this application, but they cannot be understood as limiting the scope of the patent application. It should be pointed out that for ordinary technicians in this field, several modifications and improvements can be made without departing from the concept of this application, which all belong to the protection scope of this application. Therefore, the protection scope of the patent application shall be based on the attached claims.

Claims

1. A convolutional neural network hardware acceleration device with dynamic cache space allocation, characterized in that: The convolutional neural network hardware acceleration device with dynamic cache space allocation includes a parameter storage module, an input cache module, a weight data conversion module, a feature map data conversion module, a calculation array module and an output cache module, wherein: The parameter storage module is used to cache the structural parameters of the convolutional neural network and each convolutional layer thereof and the spatial allocation parameter K, wherein the spatial allocation parameter K is determined by judging the size of the convolution kernel of the corresponding layer and the input feature map data; The input cache module includes N cache units, and is used to obtain the weight data of the convolution kernel and the input feature map data of the corresponding address from the data bus according to the structural parameters, and then cache the weight data of the convolution kernel and the input feature map data according to the spatial allocation parameter K, wherein K cache units are used to cache the weight data of the convolution kernel, and NK cache units are used to cache the input feature map data; The weight data conversion module is used to obtain the weight data of the convolution kernel from the input buffer module and perform format conversion; The feature map data conversion module is used to obtain input feature map data from the input buffer module and perform format conversion; The operation array module is used to perform multiplication and accumulation operations on the weight data after format conversion and the input feature map data, and output the convolution operation result as the output feature map data of the corresponding layer; The output cache module is used to cache output feature map data, and the output feature map data is used as input feature map data of the next layer; The space allocation parameter K is determined by judging the size of the convolution kernel of the corresponding layer and the input feature map data, as follows: If the size of the convolution kernel W*H*CN*G ≤ (N-1)*M, then the convolution operation of this layer is performed, and K is set to N-1; If the size of the convolution kernel W*H*CN*G>(N-1)*M, and the size of the input feature map data C*R*CN≤(N-1)*M, then the convolution operation of this layer is performed, and K is set to 1; If the size of the convolution kernel W*H*CN*G>(N-1)*M, the size of the input feature map data C*R*CN>(N-1)*M, and W*H*CN*G<C*R*CN, then for this layer of convolution operation, K is set to N-1; If the size of the convolution kernel W*H*CN*G>(N-1)*M, the size of the input feature map data C*R*CN>(N-1)*M, and W*H*CN*G>C*R*CN, then the convolution operation of this layer is performed, and K is set to 1; If the size of the convolution kernel W*H*CN*G>(N-1)*M, the size of the input feature map data C*R*CN>(N-1)*M, and W*H*CN*G=C*R*CN, then for this layer of convolution operation, K is set to 1 or N-1; Among them, C represents the width of the input feature map data, R represents the height of the input feature map data, CN represents the number of channels, W represents the width of the convolution kernel, H represents the height of the convolution kernel, G represents the number of convolution kernel groups, and M represents the storage space of the cache unit.

2. The convolutional neural network hardware acceleration device with dynamic cache space allocation according to claim 1, characterized in that: The operation array module performs convolution operation at a layer granularity, and the amount of data processed by each convolution operation is: convolution operation is performed on the input feature map data of the current convolution layer with a size of C*R*CN and a convolution kernel with a size of W*H*CN*G; Among them, C represents the width of the input feature map data, R represents the height of the input feature map data, CN represents the number of channels, W represents the width of the convolution kernel, H represents the height of the convolution kernel, and G represents the number of convolution kernel groups.

3. The convolutional neural network hardware acceleration device with dynamic cache space allocation according to claim 1, characterized in that: Each of the cache units is a random access memory with a storage space of M.

4. The convolutional neural network hardware acceleration device with dynamic cache space allocation according to claim 1, characterized in that: The determination of the size of the convolution kernel of the corresponding layer and the input feature map data is completed by the parameter storage module or the external CPU.

Images