A storage device and a convolutional neural network processing device

Abstract

The patent discloses a storage device and a convolutional neural network processing device, and belongs to the field of convolutional neural networks, which is used for improving the efficiency of data transmission, improving the convolution calculation speed and reducing the power consumption. The main technical scheme of the present application is that the storage device comprises a first memory, a second memory and a third memory. The first memory is used for storing the dot multiplication operation results obtained by the dot multiplication processing module in the convolutional neural network processing device. The second memory is used for storing the dot multiplication operation results transmitted from the first memory. The second memory is also used for transmitting the dot multiplication operation results without accumulation objects to the third memory when the storage time is greater than the preset time threshold. The third memory is used for storing the dot multiplication operation results transmitted from the second memory.

Description

Technical Field

[0001] This invention relates to the field of convolutional neural networks, and more particularly to a storage device and a convolutional neural network processing device. Background Technology

[0002] A Convolutional Neural Network (CNN) is a type of feedforward neural network whose artificial neurons can respond to surrounding units within a certain coverage area, making it suitable for processing large images. CNNs are widely used in image recognition, speech recognition, and other fields, but they require a very large amount of computation.

[0003] Existing convolutional neural network storage devices typically equip each computation unit with a storage unit of the same size as the output feature map. The product results from each computation unit need to be accumulated, that is, the product results within each storage unit need to be accumulated, and the product results from multiple storage units also need to be accumulated. This can lead to multiple product results needing to be accumulated at the same time, causing conflicts and blocking problems.

[0004] Existing convolutional neural network processing devices mostly use systolic arrays. To achieve correct matrix operations, data entering the array needs to be formatted and passed continuously in a certain order. This presents the following problems: some calculation results need to be summed together. If summing is performed simultaneously, data contention can occur, leading to computational blocking and reduced computing power. Furthermore, if sparse data is compressed, this systolic operation cannot be implemented. Existing processors capable of handling sparse data directly determine the storage location of calculation results based on coordinates. However, since the computation time cannot be determined, the accumulated results need to be stored at the same output address, consuming more storage space. Moreover, searching for products or partial sums within and across multiple storage units can lead to conflicts and blocking when multiple numbers need to be accumulated simultaneously. Summary of the Invention

[0005] This invention provides a storage device and a convolutional neural network processing device to solve the conflict and blocking problem in the convolution calculation process.

[0006] This invention provides a storage device for storing the computation results of a convolutional neural network processing device. The device includes: a primary memory, a secondary memory, and a tertiary memory.

[0007] The primary memory is used to store the dot product operation results obtained by the dot product processing module in the convolutional neural network processing device.

[0008] The secondary memory is used to store the dot product operation results transmitted from the primary memory; the secondary memory is also used to send the dot product operation results with a storage time greater than a preset time threshold and no accumulation object to the tertiary memory;

[0009] The third-level memory is used to store the dot product operation results transmitted from the second-level memory.

[0010] The primary memory includes multiple registers; each register stores the result of dot product operations for different convolution kernels.

[0011] The primary memory is also used to store the results of dot product operations belonging to the same convolution kernel into the same register in the secondary memory.

[0012] The secondary memory includes at least one register, and the secondary memory is also used to store the dot product operation results from the primary memory into each register.

[0013] The secondary memory is also used to retrieve the accumulation object of the dot product operation result from the tertiary memory when there is no accumulation object in the secondary memory.

[0014] Each register of the secondary memory includes a target cache space, which is used to store the result of dot product operations that are less than or equal to a preset time threshold and have no accumulated objects.

[0015] This invention provides a convolutional neural network processing device, comprising: an input processing module, a dot product processing module, and an accumulation processing module; the accumulation processing module includes the storage device described in the above embodiments.

[0016] The input processing module is connected to the dot product processing module, and the dot product processing module is connected to the accumulation processing module.

[0017] The input processing module includes: a weight element processing unit and a feature data processing unit;

[0018] The weight element processing unit is used to input the rearranged weight elements into the dot product processing module; the feature data processing unit is used to input the feature data to be processed into the dot product processing module according to the channel and coordinate of the feature data.

[0019] The weight element processing unit is also used to compress the rearranged weight elements.

[0020] The feature data processing unit is also used to compress the feature data to be processed.

[0021] The dot product processing module includes: a multiplier array and a multiplier storage unit;

[0022] The multiplier array includes multiple multipliers, which are used to perform dot product operations on the weight elements and the feature data to be processed, and obtain the dot product operation result;

[0023] The multiplier storage unit includes multiple registers, each of which is used to store the dot product result obtained by a multiplier.

[0024] The accumulation processing module further includes an accumulator, which is used to accumulate the result of the dot product operation.

[0025] Beneficial effects:

[0026] The convolutional neural network processing device provided by this invention can improve data transmission efficiency, increase computing speed, and reduce power consumption. Moreover, it is compatible with compressed data formats. When weight data or input feature data is sparse, data compression can further improve computing speed and reduce power consumption.

[0027] Each multiplier is equipped with a storage unit that is only responsible for storage, while the three-level storage device is responsible for accumulation. This allows multiple multipliers to perform parallel calculations while the accumulation order of the product results is reasonably arranged. It makes good use of limited storage space to reduce conflicts and blocking, solves the problem of frequent conflicts and contradictions in partial addition and accumulation, improves the overall efficiency of convolution, and increases the utilization rate of sparse data. Attached Figure Description

[0028] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0029] Figure 1 This is a diagram of a convolutional neural network processing device according to an embodiment of the present invention;

[0030] Figure 2 This is a diagram of a storage device according to an embodiment of the present invention. Detailed Implementation

[0031] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0032] Example 1

[0033] In this embodiment, a storage device is provided for storing the computation results of a convolutional neural network processing device. The device includes: a primary memory, a secondary memory, and a tertiary memory, such as... Figure 2 As shown,

[0034] The primary memory is used to store the dot product operation results obtained by the dot product processing module in the convolutional neural network processing device; the primary memory is connected to the storage unit of the convolutional neural network processing device that stores the operation results.

[0035] The secondary memory is used to store the dot product operation results transmitted from the primary memory; the secondary memory is also used to send the dot product operation results that have been stored for longer than a preset time threshold and have no accumulation objects to the tertiary memory; the two-level storage structure, wherein the primary memory obtains and stores the operation results of a row of multipliers in the dot product processing module, and at the same time transmits them to the secondary memory according to their convolution kernel number, accumulates them in the secondary memory, and puts the accumulated result into the corresponding storage location. The multi-level storage reduces the complexity of wiring, and at the same time, some accumulation calculations are performed in the secondary memory. Before being stored in the upper-level memory, the operation results that can be added together are accumulated together in advance according to the pipeline, reducing the probability of blocking, and reducing the probability of reading the operation results and the accumulated part back and forth between the memory, thereby improving the overall performance and energy efficiency ratio.

[0036] The tertiary memory is used to store the dot product operation results transmitted from the secondary memory. The tertiary memory can relieve the pressure on the secondary memory and store the dot product operation results transmitted from the secondary memory that are greater than a preset time threshold and have no accumulated objects.

[0037] Example 2

[0038] This embodiment provides a storage device for storing the computation results of a convolutional neural network processing device. The device includes: a primary memory, a secondary memory, and a tertiary memory.

[0039] The primary memory is used to store the dot product operation results obtained by the dot product processing module in the convolutional neural network processing device.

[0040] The secondary memory is used to store the dot product operation results transmitted from the primary memory; the secondary memory is also used to send the dot product operation results with a storage time greater than a preset time threshold and no accumulation object to the tertiary memory;

[0041] The third-level memory is used to store the dot product operation results transmitted from the second-level memory.

[0042] The primary memory includes multiple registers; each register stores the dot product results of different convolution kernels, which facilitates the transfer of the results of operations with the same convolution kernel to the same register in the secondary memory.

[0043] The primary memory is also used to store the dot product results belonging to the same convolution kernel into the same register of the secondary memory. The secondary memory includes at least one register, and is also used to store the dot product results from the primary memory into each register. In some optional embodiments, the results of each row of multipliers in the dot product processing module belong to different channels. The set of dot product results from the first row of multipliers is passed to the primary memory; the dot product result of each multiplier is stored in a register of the primary memory; then the dot product results of the first row of multipliers stored in the primary memory are passed to the secondary memory, and the dot product results of the second row of multipliers are passed to the primary memory; at this time, the dot product results of the second row of multipliers currently stored in the primary memory are... When inputting to the secondary memory, the dot product results belonging to the same convolution kernel can be input to the same register in the secondary memory. Since only the dot product result of one line of multipliers is input to the primary memory at a time, after the dot product result in the primary memory is sent to the secondary memory, the dot product result of the next line of multipliers is sent. Only then will the dot product results of different channels be added. And there are at most two dot product results of two lines of multipliers belonging to the same convolution kernel. This means that the dot product result input to the secondary memory will be accumulated at most once. This architecture solves the problem of accumulation conflict and blocking caused by too many operation results that need to be added at the same time in each layer.

[0044] The secondary memory is also used to retrieve the accumulation object of the dot product operation result from the tertiary memory when there is no accumulation object in the secondary memory. Each register of the secondary memory includes a target cache space, which is used to store dot product operation results that are less than or equal to a preset time threshold and have no accumulation object. This allows for reasonable allocation of storage space and saves on-chip resources.

[0045] Example 3

[0046] In one embodiment of the present invention, a convolutional neural network processing device is provided, comprising: an input processing module, a dot product processing module, and an accumulation processing module; the accumulation processing module includes the storage device described in embodiments 1-2 above.

[0047] The input processing module is connected to the dot product processing module, and the dot product processing module is connected to the accumulation processing module.

[0048] The input processing module includes: a weight element processing unit and a feature data processing unit;

[0049] The weight element processing unit is used to input the rearranged weight elements into the dot product processing module; the feature data processing unit is used to input the feature data to be processed into the dot product processing module according to the channel and coordinate of the feature data.

[0050] In some optional embodiments, each channel of each convolutional kernel is unfolded into a row vector, and the row vectors of the same channel belonging to different convolutional kernels are input to form a rearranged kernel matrix. The weight elements of the rearranged kernel matrix are input to the multiplier array in the dot product processing module; the weight elements of each row of multipliers belong to different convolutional kernels of the same channel.

[0051] The set of feature data to be processed is input into the multiplier array. In this embodiment, the arrangement of the feature data to be processed is determined according to its channels, coordinates and the storage method of the weight elements in the array.

[0052] The weight element processing unit is also used to compress the rearranged weight elements; the feature data processing unit is also used to compress the feature data to be processed. The processing device provided in this embodiment is suitable for processing sparse data, and the connections between the hardware components are suitable for compressed data.

[0053] The dot product processing module includes: a multiplier array and a multiplier storage unit;

[0054] The multiplier array includes multiple multipliers, which are used to perform dot product operations on the weight elements and the feature data to be processed, and obtain the dot product operation result;

[0055] The multiplier storage unit includes multiple registers, each of which is used to store the dot product result obtained by a multiplier.

[0056] The accumulation processing module further includes an accumulator, which is used to accumulate the result of the dot product operation.

[0057] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0058] Each module in the above-mentioned device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0059] In one embodiment, a computer device is provided, which may be a server. The computer device includes a processor, memory, a network interface, and a database connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage medium. The network interface of the computer device is used for communication with external terminals via a network connection.

[0060] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0061] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A storage device, characterized in that, The storage device is used for storing the results of operations performed by a convolutional neural network processing device, and the storage device includes: a primary memory, a secondary memory, and a tertiary memory; The primary memory is used to store the dot product operation results obtained by the dot product processing module in the convolutional neural network processing device. The secondary memory is used to store the dot product operation results transmitted from the primary memory; the secondary memory is also used to transmit the dot product operation results that have been stored for a longer than a preset time threshold and have no accumulation object to the tertiary memory; the primary memory obtains and stores the operation results of a row multiplier in the dot product processing module, and transmits them to the secondary memory according to their convolution kernel number, accumulates them in the secondary memory, and places the accumulated result into the corresponding storage location; the secondary memory is also used to retrieve the accumulation object of the dot product operation result from the tertiary memory when the dot product operation result transmitted from the primary memory has no accumulation object in the secondary memory. The third-level memory is used to store the dot product operation results transmitted from the second-level memory.

2. The storage device according to claim 1, characterized in that, The primary memory includes multiple registers; each register stores the result of dot product operations for different convolution kernels.

3. The storage device according to claim 1, characterized in that, The primary memory is also used to store the results of dot product operations belonging to the same convolution kernel into the same register of the secondary memory.

4. The storage device according to claim 1, characterized in that, The secondary memory includes at least one register, and the secondary memory is also used to store the dot product operation results from the primary memory into each register.

5. The storage device according to claim 1, characterized in that, Each register of the secondary memory includes a target cache space, which is used to store the result of dot product operations that are less than or equal to a preset time threshold and have no accumulated objects.

6. A convolutional neural network processing device, characterized in that, include: The system includes an input processing module, a dot product processing module, and an accumulation processing module; the accumulation processing module includes a storage device as described in any one of claims 1-5. The input processing module is connected to the dot product processing module, and the dot product processing module is connected to the accumulation processing module.

7. The processing apparatus according to claim 6, characterized in that, The input processing module includes: a weight element processing unit and a feature data processing unit; The weight element processing unit is used to input the rearranged weight elements into the dot product processing module; the feature data processing unit is used to input the feature data to be processed into the dot product processing module according to the channel and coordinate of the feature data.

8. The processing apparatus according to claim 7, characterized in that, The weight element processing unit is also used to compress the rearranged weight elements.

9. The processing apparatus according to claim 7, characterized in that, The feature data processing unit is also used to compress the feature data to be processed.

10. The processing apparatus according to claim 6, characterized in that, The dot product processing module includes: a multiplier array and a multiplier storage unit; The multiplier array includes multiple multipliers, which are used to perform dot product operations on the weight elements and the feature data to be processed, and obtain the dot product operation result; The multiplier storage unit includes multiple registers, each of which is used to store the dot product result obtained by a multiplier.

11. The processing apparatus according to claim 6, characterized in that, The accumulation processing module further includes an accumulator, which is used to accumulate the result of the dot product operation.

Images