Hardware computing dynamic allocation cache device

Abstract

The application discloses a hardware computing dynamic allocation cache device, comprising a data storage unit, a dynamic allocation cache unit and an operation unit, wherein the dynamic allocation cache unit comprises a dynamic allocation control module, a B_cache and an S_cache, wherein: the data storage unit is used for storing eigenvalues and weight values; the dynamic allocation control module is used for executing different cache strategies for the B_cache and the S_cache according to the data amount of the eigenvalues and the weight values and the capacity of the B_cache and the S_cache; and the operation unit is used for obtaining the data in the B_cache and the S_cache to perform a multiply-accumulate operation and output an operation result. The application can reduce the high total amount of calculation and high bandwidth requirement caused by repeated reading of data due to the difference between the weight and the input characteristic parameter amount.

Description

Technical Field

[0001] This invention belongs to the field of hardware computing cache allocation technology, and specifically relates to a hardware computing dynamic allocation cache device. Background Technology

[0002] In recent years, artificial intelligence has emerged to meet the ever-increasing needs of people's production technology and lives. Currently, deep convolutional neural networks based on artificial intelligence technology contain millions of floating-point parameters, and the calculation process requires a large amount of computing resources and memory, thus placing increasingly higher demands on computing power.

[0003] In terms of chips supporting neural network computation, traditional general-purpose CPUs can no longer meet the demands of high-performance, complex calculations. To overcome these structural bottlenecks of general-purpose processors, GPUs (Graphical Processing Units) have achieved great success in image processing by employing a multi-core processor architecture. Compared to GPU computation, deploying models on small terminal chips faces significant challenges. At the other end of the hardware architecture spectrum, corresponding to CPUs and GPUs, is the dedicated application-specific integrated circuit (ASIC). From the perspective of programmability and energy efficiency, ASICs offer the highest energy efficiency. However, their functionality cannot be changed, failing to meet the demands of constantly evolving neural network architectures. While general-purpose FPGAs and GPUs offer powerful computational efficiency, their power consumption is often high. Although general-purpose FPGAs offer excellent versatility and energy efficiency, their structure is not optimized for deep learning neural networks, failing to fully leverage the efficiency of parallel hardware architectures.

[0004] Existing technologies, such as Chinese patent application number CN201910559200.3, disclose a method, apparatus, device, and storage medium for generating chip-based computing functions, proposing a scheme for dynamically allocating hardware resources based on different computational data. This allocation method requires the simultaneous use of input parameter value information, candidate computational function modules, target computational function modules, and computing devices. These modules impose significant constraints on general-purpose FPGAs with limited resources. The reason FPGAs excel in image processing-related deep convolutional neural network applications is that they can transform convolutional operations into parallel multiply-accumulate operations, thus accelerating computation. Network model layer structures vary, while existing FPGA hardware cache allocation is mostly fixed-capacity. When the layer structure changes, the number of channels and the amount of data exceed the cache capacity, due to the fixed-capacity allocation, when the amount of weight data or feature value data exceeds the specified cache capacity, increased computation time and insufficient bandwidth due to repeated data readings can occur. Summary of the Invention

[0005] The purpose of this invention is to provide a hardware computing dynamic allocation cache device to reduce the high total computation and high bandwidth requirements caused by repeated data readings due to differences in weights and input feature parameters.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0007] A hardware computing dynamic allocation cache device includes a data storage unit, a dynamic allocation cache unit, and a processing unit. The dynamic allocation cache unit includes a dynamic allocation control module, a B-cache, and an S-cache.

[0008] The data storage unit is used to store feature values ​​and weight values;

[0009] The dynamic allocation control module is used to execute different caching strategies for the B_cache and S_cache based on the amount of data of feature values ​​and weight values, as well as the capacity of the B_cache and S_cache.

[0010] The computation unit is used to obtain data from B_cache and S_cache, perform multiplication and accumulation operations, and output the operation results.

[0011] If the capacity of the B cache is greater than the capacity of the S cache, then the caching strategy is as follows:

[0012] If, among the four factors—the amount of data in the feature value, the amount of data in the weight value, the capacity of the B-cache, and the capacity of the S-cache—the amount of data in the feature value is the largest, or the amount of data in the weight value is the largest, then the smaller of the two, the feature value and the weight value, will be allocated to the B-cache for storage, and the larger of the two, the feature value and the weight value, will be allocated to the S-cache for storage.

[0013] Alternatively, if the capacity of the B-cache is the largest among the four factors—the amount of data in the feature value, the amount of data in the weight value, the capacity of the B-cache, and the capacity of the S-cache—then the feature value with the larger data volume and the weight value with the smaller data volume will be allocated to the B-cache for storage, and the feature value with the smaller data volume and the weight value will be allocated to the S-cache for storage.

[0014] Several alternative methods are provided below, but they are not intended as additional limitations on the overall solution above. They are merely further additions or optimizations. Provided there are no technical or logical contradictions, each alternative method can be combined individually with respect to the overall solution above, or multiple alternative methods can be combined with each other.

[0015] Preferably, the maximum amount of data processed by the arithmetic unit in one multiplication-accumulation operation is: the input feature map size (I h *I w*N), kernel size (W) h *W w *N*M); where I h I represents the height of the input feature map image. w W represents the width of the input feature map image. h W represents the height of the convolution kernel. w N represents the width of the convolution kernel, N represents the number of channels, and M represents the number of convolution kernel groups.

[0016] The size of the input feature map to be processed is (I′) h *I′ w *N′), when N′>N, the B_buffer or S_buffer uses multiple consecutive address spaces to store the feature value of N′ channels for a pixel; when N′≤N, each address space of the B_buffer or S_buffer stores the feature value of N channels for a pixel. h I′ represents the height of the input feature map image to be processed. w N represents the width of the input feature map image to be processed, and N′ represents the number of channels to be processed;

[0017] Take the kernel size to be processed as (W′) h *W′ w *N′*M′), when M′>M, the B_cache or S_cache uses multiple contiguous address spaces to store the weight values ​​of M′ groups of convolutional kernels for one point. If M′≤M, each address space of the B_cache or S_cache stores the weight values ​​of M groups of convolutional kernels for one point, W′ h W' represents the height of the convolution kernel to be processed. w M' represents the width of the convolution kernel to be processed, and M' represents the number of convolution kernel groups to be processed.

[0018] Preferably, the computational unit includes M groups of parallel MAC units, each MAC unit performing a multiplication and accumulation operation on the feature value and the weight value of the corresponding convolution kernel.

[0019] Preferably, if, among the four factors—the amount of data in the feature value, the amount of data in the weight value, the capacity of the B-cache, and the capacity of the S-cache—the amount of data in the feature value is the largest, or the amount of data in the weight value is the largest, then the smaller of the two (feature value and weight value) is allocated to the B-cache for storage, and the larger of the two (feature value and weight value) is allocated to the S-cache for storage. Specifically:

[0020] If the amount of data for the feature values ​​> the amount of data for the weight values ​​> the capacity of the B_cache > the capacity of the S_cache, then the weight values ​​will be allocated to the B_cache and the feature values ​​will be allocated to the S_cache.

[0021] Alternatively, if the amount of feature data > the capacity of B_cache > the amount of weight data > the capacity of S_cache, then the weight values ​​are allocated to B_cache and the feature values ​​are allocated to S_cache.

[0022] Alternatively, if the amount of feature data > the capacity of B_cache > the capacity of S_cache > the amount of weight data, then the weight values ​​are allocated to B_cache and the feature values ​​are allocated to S_cache.

[0023] Alternatively, if the amount of data for weight values ​​> the amount of data for feature values ​​> the capacity of B_cache > the capacity of S_cache, then the weight values ​​are allocated to S_cache and the feature values ​​are allocated to B_cache.

[0024] Alternatively, if the amount of data for weight values ​​> the capacity of B_cache > the amount of data for feature values ​​> the capacity of S_cache, then the weight values ​​are allocated to S_cache and the feature values ​​are allocated to B_cache.

[0025] Alternatively, if the amount of data for weight values ​​> the capacity of B_cache > the capacity of S_cache > the amount of data for feature values, then the weight values ​​are allocated to S_cache and the feature values ​​are allocated to B_cache.

[0026] Preferably, if the capacity of the B-cache is the largest among the four factors—the amount of data in the feature values, the amount of data in the weight values, the capacity of the B-cache, and the capacity of the S-cache—then the feature value with the larger data volume and the weight value with the smaller data volume are allocated to the B-cache for storage, and the feature value with the smaller data volume and the weight value are allocated to the S-cache for storage. Specifically:

[0027] If the capacity of B_cache > the amount of data for feature values ​​> the amount of data for weight values ​​> the capacity of S_cache, then the weight values ​​will be allocated to S_cache and the feature values ​​will be allocated to B_cache.

[0028] Alternatively, if the capacity of B_cache > the amount of data for weight values ​​> the amount of data for feature values ​​> the capacity of S_cache, then the weight values ​​are allocated to B_cache and the feature values ​​are allocated to S_cache.

[0029] Alternatively, if the capacity of B_cache > the amount of feature data > the capacity of S_cache > the amount of weight data, then the weight values ​​are allocated to S_cache and the feature values ​​are allocated to B_cache.

[0030] Alternatively, if the capacity of B_cache > the capacity of S_cache > the amount of data for feature values ​​> the amount of data for weight values, then the weight values ​​are allocated to S_cache and the feature values ​​are allocated to B_cache.

[0031] Alternatively, if the capacity of B_cache > the amount of data for weight values ​​> the capacity of S_cache > the amount of data for feature values, then the weight values ​​are allocated to B_cache and the feature values ​​are allocated to S_cache.

[0032] Alternatively, if the capacity of B_cache > the capacity of S_cache > the amount of data for weight values ​​> the amount of data for feature values, then the weight values ​​are allocated to B_cache and the feature values ​​are allocated to S_cache.

[0033] The hardware computing dynamic allocation cache device provided by the present invention has the following advantages compared with the prior art: (1) it reduces the problem of increased computing time caused by repeated data reading; (2) it reduces the problem of insufficient bandwidth caused by repeated data reading; (3) the allocation strategy only depends on the relationship between the amount of data and the cache capacity, and has stronger versatility. Attached Figure Description

[0034] Figure 1 This is a schematic diagram of the hardware computing dynamic allocation cache device of the present invention;

[0035] Figure 2 This is a schematic diagram illustrating the first specific logic of the allocation strategy of this invention.

[0036] Figure 3 This is a schematic diagram illustrating the second specific logic of the allocation strategy of the present invention.

[0037] Figure 4 This is a schematic diagram illustrating the storage method of the weight values ​​in this invention;

[0038] Figure 5 This is a schematic diagram illustrating the storage method of the feature values ​​of the present invention;

[0039] Figure 6 This diagram illustrates the amount of data for each feature value and the amount of data for each weight value in a VGG16 network. Detailed Implementation

[0040] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0041] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to limit the invention.

[0042] To address the issue in existing technologies where, due to fixed capacity allocation, the amount of weighted data or feature value data exceeds the specified cache capacity, leading to increased computation time and insufficient bandwidth caused by repeated data readings, this embodiment provides a hardware computing dynamic allocation cache device.

[0043] like Figure 1 As shown, the hardware computing dynamic allocation cache device in this embodiment includes a data storage unit, a dynamic allocation cache unit, and a computing unit, wherein the dynamic allocation cache unit includes a dynamic allocation control module, a B-cache, and an S-cache.

[0044] The data storage unit is used to store feature values ​​and weight values. It should be noted that the data storage unit is used to store the parameters required for the calculation. This embodiment is based on a convolutional neural network, and the example will use feature values ​​and weight values ​​as the parameters required for the calculation.

[0045] The dynamic allocation control module is used to execute different caching strategies for the B_cache and S_cache based on the amount of feature values ​​and weight values, as well as the capacity of the B_cache and S_cache. In other words, it is used to control the balanced calculation of the computing unit, detect the amount of input weight parameters, feature parameters, B_cache capacity and S_cache capacity, determine which data caching strategy to adopt, and confirm and control the generation of data output.

[0046] The computation unit is used to retrieve data from the B-cache and S-cache, perform multiplication and accumulation operations, and output the results. In this embodiment, the computation unit includes M parallel MAC units. Each MAC unit performs multiplication and accumulation operations on the feature value and the weight value of the corresponding convolution kernel, ensuring that the convolution algorithm has the maximum parallelism and significantly improving computational efficiency.

[0047] This embodiment primarily provides a dynamically allocated cache unit for execution on an FPGA computing device, which performs one or more data reception buffering operations between one or more arithmetic units and data storage units. One or more types of data from one or more data storage units communicate with one or more arithmetic units through the dynamically allocated cache unit. The dynamically allocated cache unit of this embodiment includes a dynamic allocation control module and a data cache module. The data cache module consists of a B-cache and an S-cache with different capacities. The dynamic allocation control module controls the allocation of one or more types of data to the data cache module, the allocation of data to one or more arithmetic units, and the control module also controls the communication of output data from one or more arithmetic units to the data bus.

[0048] To facilitate expanded control, the dynamically allocated cache unit in this embodiment can include multiple data cache modules, which are controlled by the dynamic allocation control module. Therefore, when forming a caching strategy, the dynamic allocation control module can operate and respond to the data requirements of one or more computing units to control the allocation of one or more data cache modules. The dynamic allocation control module can also operate and respond to the maximum allowable bandwidth limit of one or more computing units to control the allocation of one or more data cache modules. Furthermore, the dynamic allocation control module can also operate and respond to the data requirements of one or more computing units to control the allocation of one or more data cache modules or the allocation of one or more types of data.

[0049] The aforementioned computational unit includes one or more multiply-accumulate operation array units and an output image buffer unit. The computational unit is implemented as a neural network operation. The data required for the neural network operation is allocated to one or more data buffer modules via a dynamic allocation control module based on the computational load of the one or more computational units. The generated data from the neural network operation is exchanged and communicated via the dynamic allocation control module, taking into account the computational requirements of the one or more computational units. The data required for the neural network operation is issued to the data buffer modules for dynamic caching based on different instructions from the dynamic allocation control module. The dynamic allocation buffer unit can be executed on at least one of the one or more computational units.

[0050] This embodiment uses the fact that the capacity of B_cache is greater than the capacity of S_cache to illustrate the caching strategy of this embodiment. In other embodiments, the names of B_cache and S_cache can be changed; this is just a name for ease of description. Based on this, the main idea of ​​the caching strategy of this embodiment includes the following two points:

[0051] First point: If, among the four factors of the amount of data in the feature value, the amount of data in the weight value, the capacity of the B-cache, and the capacity of the S-cache, the amount of data in the feature value is the largest or the amount of data in the weight value is the largest, then the smaller of the two, the feature value and the weight value, will be allocated to the B-cache for storage, and the larger of the two, the feature value and the weight value, will be allocated to the S-cache for storage.

[0052] like Figure 2 As shown, the specific logic of this caching strategy is as follows:

[0053] If the amount of data for the feature values ​​> the amount of data for the weight values ​​> the capacity of the B-cache > the capacity of the S-cache, then the weight values ​​will be allocated to the B-cache and the feature values ​​will be allocated to the S-cache.

[0054] Alternatively, if the amount of feature data > the capacity of B_cache > the amount of weight data > the capacity of S_cache, then the weight values ​​are allocated to B_cache and the feature values ​​are allocated to S_cache.

[0055] Alternatively, if the amount of feature data > the capacity of B_cache > the capacity of S_cache > the amount of weight data, then the weight values ​​are allocated to B_cache and the feature values ​​are allocated to S_cache.

[0056] Alternatively, if the amount of data for weight values ​​> the amount of data for feature values ​​> the capacity of B_cache > the capacity of S_cache, then the weight values ​​are allocated to S_cache and the feature values ​​are allocated to B_cache.

[0057] Alternatively, if the amount of data for weight values ​​> the capacity of B_cache > the amount of data for feature values ​​> the capacity of S_cache, then the weight values ​​are allocated to S_cache and the feature values ​​are allocated to B_cache.

[0058] Alternatively, if the amount of data for weight values ​​> the capacity of B_cache > the capacity of S_cache > the amount of data for feature values, then the weight values ​​are allocated to S_cache and the feature values ​​are allocated to B_cache.

[0059] The capacity of the B cache is greater than that of the S cache. In order to increase data reusability, this caching strategy puts the data with smaller volume into the B cache, that is, into the B cache with a large cache capacity, and puts the data with larger volume into the small cache. This can ensure that the data in the large cache has a higher reuse rate.

[0060] Second point: If, among the four factors of the amount of data in the feature value, the amount of data in the weight value, the capacity of the B-cache, and the capacity of the S-cache, the capacity of the B-cache is the largest, then the feature value or the weight value with the larger amount of data will be allocated to the B-cache for storage, and the feature value or the weight value with the smaller amount of data will be allocated to the S-cache for storage.

[0061] like Figure 3 As shown, the specific logic of this caching strategy is as follows:

[0062] If the capacity of B_cache > the amount of feature data > the amount of weight data > the capacity of S_cache, then the weight values ​​are allocated to S_cache and the feature values ​​are allocated to B_cache.

[0063] Alternatively, if the capacity of B_cache > the amount of data for weight values ​​> the amount of data for feature values ​​> the capacity of S_cache, then the weight values ​​are allocated to B_cache and the feature values ​​are allocated to S_cache.

[0064] Alternatively, if the capacity of B_cache > the amount of feature data > the capacity of S_cache > the amount of weight data, then the weight values ​​are allocated to S_cache and the feature values ​​are allocated to B_cache.

[0065] Alternatively, if the capacity of B_cache > the capacity of S_cache > the amount of data for feature values ​​> the amount of data for weight values, then the weight values ​​are allocated to S_cache and the feature values ​​are allocated to B_cache.

[0066] Alternatively, if the capacity of B_cache > the amount of data for weight values ​​> the capacity of S_cache > the amount of data for feature values, then the weight values ​​are allocated to B_cache and the feature values ​​are allocated to S_cache.

[0067] Alternatively, if the capacity of B_cache > the capacity of S_cache > the amount of data for weight values ​​> the amount of data for feature values, then the weight values ​​are allocated to B_cache and the feature values ​​are allocated to S_cache.

[0068] It's easy to understand that during data allocation, if the amount of data is exactly equal to the capacity of the cache, then the two are matched; otherwise, the allocation follows the same rules. Specifically: if either the amount of data for the weight value or the amount of data for the feature value is equal to the capacity of the B_cache, then the weight value or feature value with the same amount of data as the B_cache is allocated to the B_cache, and the other weight value or feature value is allocated to the S_cache; if both the amount of data for the weight value and the amount of data for the feature value are equal to the capacity of the B_cache, then either the weight value or the feature value is allocated to the B_cache, and the other weight value or feature value is allocated to the S_cache.

[0069] If either the data size of the weight value or the data size of the feature value is equal to the capacity of the S_cache, then the weight value or feature value whose data size is equal to the capacity of the S_cache is allocated to the S_cache, and the other weight value or feature value is allocated to the B_cache; if both the data size of the weight value and the data size of the feature value are equal to the capacity of the S_cache, then either the weight value or the feature value is allocated to the S_cache, and the other weight value or feature value is allocated to the B_cache.

[0070] Let the maximum amount of data processed by the arithmetic unit in this embodiment for one multiplication-accumulation operation be: the input feature map size (I h *I w *N), kernel size (W) h *W w *N*M); where I h I represents the height of the input feature map image. w W represents the width of the input feature map image. h W represents the height of the convolution kernel. w N represents the width of the convolution kernel, N represents the number of channels, and M represents the number of convolution kernel groups.

[0071] like Figures 4-5 As shown, therefore, for B_cache and S_cache, when storing weights, there are a total of W. h *W wAn *N*M address space, where each address space stores the weight data of M sets of convolutional kernels for one point. When storing input features, there are a total of I... h *I w An address space of *N, where each address space stores the data of N channels for one pixel.

[0072] When the amount of data to be cached is greater than the cache capacity, if the size of the input feature map to be processed is (I′) h *I′ w *N′), when N′>N, the B_buffer or S_buffer uses multiple consecutive address spaces to store the feature value of N′ channels for a pixel; when N′≤N, each address space of the B_buffer or S_buffer stores the feature value of N channels for a pixel. h I′ represents the height of the input feature map image to be processed. w N represents the width of the input feature map image to be processed, and N′ represents the number of channels to be processed.

[0073] If the kernel size to be processed is (W′) h *W′ w *N′*M′), when M′>M, the B_cache or S_cache uses multiple contiguous address spaces to store the weight values ​​of M′ groups of convolutional kernels for one point. If M′≤M, each address space of the B_cache or S_cache stores the weight values ​​of M groups of convolutional kernels for one point, W′ h W' represents the height of the convolution kernel to be processed. w M' represents the width of the convolution kernel to be processed, and M' represents the number of convolution kernel groups to be processed.

[0074] The following examples further illustrate the workflow of the hardware computing dynamic allocation cache device of this application in convolutional neural network applications.

[0075] Example 1

[0076] like Figure 6 As shown in the figure, the total number of eigenvalue parameters and the total number of weight parameters in each layer of the VGG16 network are displayed. It can be clearly observed from the figure that the total number of eigenvalue parameters in the first seven layers of the network, including the seventh layer, far exceeds the total number of weight parameters. As the network structure changes and the network parameters are adjusted, the total number of weight parameters far exceeds the total number of eigenvalue parameters starting from the eighth layer.

[0077] When the large buffer unit stores the input features, there are a total of I in the B_ buffer. h1 *I w The address space is N, with each address space storing N channels of data for one pixel; when storing weights, the B_cache contains a total of W... h *W wAn *N*M address space, where each address space stores the weight data of M sets of convolutional kernels for one point.

[0078] When the small buffer unit stores the input features, there are a total of I in the S_buffer. h2 *I w The S_buffer contains N address spaces, each storing N channels of data for one pixel; when storing weights, the S_buffer contains a total of W... h *W w An *N*M address space, where each address space stores the weight data of M sets of convolutional kernels for one point.

[0079] Therefore, when the total number of input features (i.e., the amount of feature data) is greater than the total number of weights (i.e., the amount of weight data), and both are greater than the capacity of the large cache (i.e., the capacity of the B_ cache) and the capacity of the small cache (i.e., the capacity of the S_ cache), the dynamic allocation control module issues an instruction, and the B_ cache stores the weight data (i.e., the weight values), with the size of the weights to be processed being W′. h *W′ w *N′*M′, where W′ w W′ represents the width of the convolution kernel to be processed. h The S_cache represents the height of the convolutional kernel to be processed, N′ represents the number of channels to be processed, and M′ represents the number of groups of convolutional kernels to be processed. If M′>M, the large cache uses multiple consecutive addresses to store the weight data of M′ groups of convolutional kernels for one point. The S_cache stores feature data (i.e., feature values), and the size of the input feature to be processed is I′. h *I′ w *N′, where I′ w I′ represents the width of the image to be processed. h The height of the image to be processed is represented by N, and the number of channels to be processed is represented by N′. If N′>N, the small buffer uses multiple contiguous address spaces to store the data of N′ channels for one pixel.

[0080] When the total number of input features is greater than the total number of weights, and the total number of weights is less than the large cache capacity, but the total number of features is greater than the small cache capacity, the dynamic allocation control module issues an instruction: the B_cache stores the weight data, and the size of the weights to be processed is W. h *W w *N*M, where W h W represents the height of the convolution kernel. w Let N represent the width of the convolutional kernel, N represent the number of channels, and M represent the number of convolutional kernel groups. The large cache uses multiple contiguous addresses to store the weight data of M convolutional kernel groups for a single point. The S_cache stores the feature data, and the size of the input feature to be processed is I′. h *I′ w *N′, where I′ w I′ represents the width of the image to be processed.h The height of the image to be processed is represented by N, and the number of channels to be processed is represented by N′. If N′>N, the small buffer uses multiple contiguous address spaces to store the data of N′ channels for one pixel.

[0081] When the total number of input features is less than the total number of weights, and both are greater than the total cache size of the large cache and the small cache, the dynamic allocation control module issues an instruction: the B_cache stores the feature data, and the size of the input feature to be processed is I′. h *I′ w *N′, where I′ w I′ represents the width of the image to be processed. h The height of the image to be processed is represented by N, and the number of channels to be processed is represented by N′. If N′>N, the large cache uses multiple contiguous address spaces to store the data for the number of N′ channels of a single pixel; the S_ cache stores the weight data, and the weight size to be processed is W′. h *W′ w *N′*M′, where W′ w W′ represents the width of the convolution kernel to be processed. h N' represents the height of the convolutional kernel to be processed, N' represents the number of channels to be processed, and M' represents the number of groups of convolutional kernels to be processed. If M' > M2, the small cache uses multiple consecutive addresses to store the weight data of M' groups of convolutional kernels for one point.

[0082] When the total number of input features is less than the total number of weights, and the total number of input features is less than the large cache capacity, while the total number of weights is greater than the small cache capacity, the dynamic allocation control module issues an instruction: the B_cache stores the feature data, and the size of the input features to be processed is I. h *I w *N, where I h I represents the height of the image. w Let N represent the width of the image and N represent the number of channels. The large cache uses multiple contiguous address spaces to store the N-channel data for a single pixel; the S_ cache stores weight data, and the weights to be processed have a size of W′. h *W′ w *N′*M′, where W′ w W′ represents the width of the convolution kernel to be processed. h N' represents the height of the convolutional kernel to be processed, N' represents the number of channels to be processed, and M' represents the number of groups of convolutional kernels to be processed. If M' > M2, the small cache uses multiple consecutive addresses to store the weight data of M' groups of convolutional kernels for one point.

[0083] When the total number of input features is greater than the total number of weights, and the total number of input features is less than the large cache capacity, while the total number of weights is greater than the small cache capacity, the dynamic allocation control module issues an instruction: the B_cache stores the feature data, and the size of the input features to be processed is I. h *Iw *N, where I h I represents the height of the image. w The width of the image is represented by , and N represents the number of channels. The large cache uses multiple contiguous address spaces to store the N-channel data for a single pixel. The S_ cache stores weight data, and the weights to be processed have a size of W′. h *W′ w *N′*M′, where W′ w W′ represents the width of the convolution kernel to be processed. h N' represents the height of the convolutional kernel to be processed, N' represents the number of channels to be processed, and M' represents the number of groups of convolutional kernels to be processed. If M' > M2, the small cache uses multiple consecutive addresses to store the weight data of M' groups of convolutional kernels for one point.

[0084] When the total number of input features is greater than the total number of weights, and the total number of input features is less than the large cache capacity, and the total number of weights is less than the small cache capacity, the dynamic allocation control module issues an instruction: the B_cache stores the feature data, and the size of the input features to be processed is I. h *I w *N, where I h I represents the height of the image. w The width of the image is represented by , and N represents the number of channels. The large cache uses multiple contiguous address spaces to store the N-channel data for a single pixel. The S-cache stores weight data, and the weights to be processed have a size of W. h *W w *N*M, where W h W represents the height of the convolution kernel. w Let N represent the width of the convolution kernel, N represent the number of channels, and M represent the number of convolution kernel groups. Then, the small cache uses multiple consecutive addresses to store the weight data of M convolution kernel groups for one point.

[0085] When the total number of input features is less than the total number of weights, and the total number of input features is greater than the small cache capacity, while the total number of weights is less than the large cache capacity, the dynamic allocation control module issues an instruction: the B_ cache stores the weight data, and the size of the weights to be processed is W. h *W w *N*M, where W h W represents the height of the convolution kernel. w Let N represent the width of the convolutional kernel, N represent the number of channels, and M represent the number of convolutional kernel groups. The large cache uses multiple contiguous addresses to store the weight data of M convolutional kernel groups for a single point. The S_cache stores the feature data, and the size of the input feature to be processed is I′. h *I′ w *N′, where I′ w I′ represents the width of the image to be processed. hThe height of the image to be processed is represented by N, and the number of channels to be processed is represented by N′. If N′>N, the small buffer uses multiple contiguous address spaces to store the data of N′ channels for one pixel.

[0086] When the total number of input features is less than the total number of weights, and the total number of input features is less than the small cache capacity, and the total number of weights is less than the large cache capacity, the dynamic allocation control module issues an instruction: the B_ cache stores the weight data, and the size of the weights to be processed is W. h *W w *N*M, where W h W represents the height of the convolution kernel. w Let N represent the width of the convolutional kernel, N represent the number of channels, and M represent the number of convolutional kernel groups. The large cache uses multiple contiguous addresses to store the weight data of M convolutional kernel groups for a single point. The S_cache stores the feature data, and the size of the input feature to be processed is I. h *I w *N, where I h I represents the height of the image. w The width of the image is represented by N, and the number of channels is represented by N. The small buffer uses multiple contiguous address spaces to store the N-channel data of a single pixel.

[0087] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0088] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the appended claims.

Claims

1. A hardware computing dynamic allocation cache device, characterized in that, The hardware computing dynamic allocation cache device includes a data storage unit, a dynamic allocation cache unit, and a computing unit, wherein the dynamic allocation cache unit includes a dynamic allocation control module, a B-cache, and an S-cache, wherein: The data storage unit is used to store feature values ​​and weight values; The dynamic allocation control module is used to execute different caching strategies for the B_cache and S_cache based on the amount of data of feature values ​​and weight values, as well as the capacity of the B_cache and S_cache. The computation unit is used to obtain data from B_cache and S_cache, perform multiplication and accumulation operations, and output the operation results. If the capacity of the B cache is greater than the capacity of the S cache, then the caching strategy is as follows: If, among the four factors—the amount of data in the feature value, the amount of data in the weight value, the capacity of the B-cache, and the capacity of the S-cache—the amount of data in the feature value is the largest, or the amount of data in the weight value is the largest, then the smaller of the two, the feature value and the weight value, will be allocated to the B-cache for storage, and the larger of the two, the feature value and the weight value, will be allocated to the S-cache for storage. Alternatively, if the capacity of B_cache is the largest among the four factors: the amount of data in the feature value, the amount of data in the weight value, the capacity of B_cache, and the capacity of S_cache, then the feature value with the larger amount of data and the weight value with the smaller amount of data will be allocated to B_cache for storage, and the feature value with the smaller amount of data and the weight value will be allocated to S_cache for storage. The maximum amount of data processed by the arithmetic unit in one multiplication-accumulation operation is: the size of the input feature map. kernel size ;in, Indicates the height of the input feature map image. This represents the width of the input feature map image. Indicates the height of the convolution kernel. Indicates the width of the convolution kernel. Indicates the number of channels. Indicates the number of convolution kernel groups; Take the size of the input feature map to be processed as ,when In this case, the B-cache or S-cache uses multiple contiguous address spaces to store a single pixel. The characteristic value of the number of channels, when At that time, each address space of the B-cache or S-cache stores one pixel. The characteristic value of the number of channels, This represents the height of the input feature map image to be processed. This represents the width of the input feature map image to be processed. Indicates the number of channels to be processed; Take the kernel size to be processed as ,when At that time, the B-cache or S-cache uses multiple contiguous address spaces to store a single point. The weights of the convolution kernels, if Each address space of the B-cache or S-cache stores one point. The weights of the convolutional kernels, Indicates the height of the convolution kernel to be processed. This represents the width of the convolution kernel to be processed. This indicates the number of convolution kernel groups to be processed.

2. The hardware computing dynamic allocation cache device as described in claim 1, characterized in that, The computational unit includes M groups of parallel MAC units, each of which performs a multiplication and accumulation operation on the feature value and the weight value of the corresponding convolution kernel.

3. The hardware computing dynamic allocation cache device as described in claim 1, characterized in that, If, among the four factors—the amount of data in the feature value, the amount of data in the weight value, the capacity of the B-cache, and the capacity of the S-cache—the amount of data in the feature value is the largest, or the amount of data in the weight value is the largest, then the smaller of the two (feature value and weight value) will be allocated to the B-cache for storage, and the larger of the two (feature value and weight value) will be allocated to the S-cache for storage. Specifically: If the amount of data for feature values ​​> the amount of data for weight values ​​> the capacity of B_cache > the capacity of S_cache, then the weight values ​​will be allocated to B_cache and the feature values ​​will be allocated to S_cache. Alternatively, if the amount of feature data > the capacity of B_cache > the amount of weight data > the capacity of S_cache, then the weight values ​​are allocated to B_cache and the feature values ​​are allocated to S_cache. Alternatively, if the amount of feature data > the capacity of B_cache > the capacity of S_cache > the amount of weight data, then the weight values ​​are allocated to B_cache and the feature values ​​are allocated to S_cache. Alternatively, if the amount of data for weight values ​​> the amount of data for feature values ​​> the capacity of B_cache > the capacity of S_cache, then the weight values ​​are allocated to S_cache and the feature values ​​are allocated to B_cache. Alternatively, if the amount of data for weight values ​​> the capacity of B_cache > the amount of data for feature values ​​> the capacity of S_cache, then the weight values ​​are allocated to S_cache and the feature values ​​are allocated to B_cache. Alternatively, if the amount of data for weight values ​​> the capacity of B_cache > the capacity of S_cache > the amount of data for feature values, then the weight values ​​are allocated to S_cache and the feature values ​​are allocated to B_cache.

4. The hardware computing dynamic allocation cache device as described in claim 1, characterized in that, If, among the four factors—the amount of data in the feature values, the amount of data in the weight values, the capacity of the B-cache, and the capacity of the S-cache—then the capacity of the B-cache is the largest, then the feature value with the larger data volume and the weight value with the smaller data volume are allocated to the B-cache for storage, and the feature value with the smaller data volume and the weight value are allocated to the S-cache for storage. Specifically: If the capacity of B_cache > the amount of feature data > the amount of weight data > the capacity of S_cache, then the weight values ​​are allocated to S_cache and the feature values ​​are allocated to B_cache. Alternatively, if the capacity of B_cache > the amount of data for weight values ​​> the amount of data for feature values ​​> the capacity of S_cache, then the weight values ​​are allocated to B_cache and the feature values ​​are allocated to S_cache. Alternatively, if the capacity of B_cache > the amount of feature data > the capacity of S_cache > the amount of weight data, then the weight values ​​are allocated to S_cache and the feature values ​​are allocated to B_cache. Alternatively, if the capacity of B_cache > the capacity of S_cache > the amount of data for feature values ​​> the amount of data for weight values, then the weight values ​​are allocated to S_cache and the feature values ​​are allocated to B_cache. Alternatively, if the capacity of B_cache > the amount of data for weight values ​​> the capacity of S_cache > the amount of data for feature values, then the weight values ​​are allocated to B_cache and the feature values ​​are allocated to S_cache. Alternatively, if the capacity of B_cache > the capacity of S_cache > the amount of data for weight values ​​> the amount of data for feature values, then the weight values ​​are allocated to B_cache and the feature values ​​are allocated to S_cache.

Images