Convolution computing chip and method

Abstract

Embodiments of the present application relates to the technical field of integrated circuits, and provide a convolution computing chip and a method. The convolution computing chip comprises: a feature map cache, used for caching a feature map; a row sliding register group, used for acquiring a current data row in the feature map and shifting data therein to obtain at least two first data rows, a plurality of second data rows having the same shift step among a plurality of first data rows corresponding to the feature map forming a sub-feature map, and a plurality of sub-feature maps respectively corresponding to a plurality of weights; and a computing unit, used for acquiring the plurality of first data rows and a weight corresponding to a sub-feature map to which each first data row belongs, and respectively performing a multiply-add operation on each first data row and the weight corresponding to the sub-feature map to which the first data row belongs among the plurality of weights to obtain a current computing result of the sub-feature map to which the first data row belongs, computing results of the plurality of sub-feature maps forming a convolution computing result of the feature map. The convolution computing chip provided in the present application can improve the performance of convolution computing chips.

Description

Convolution calculation chip and method

[0001] This application claims priority to the Chinese patent application No. 202410829682.0, filed on June 24, 2024, and entitled “Convolution calculation chip and method”, the entire content of which is incorporated herein by reference. TECHNICAL FIELD

[0002] Embodiments of the present application relate to the field of integrated circuit technology, and in particular to a convolution calculation chip and method. BACKGROUND

[0003] In a neural network in the field of artificial intelligence (AI), convolution refers to an operation of performing convolution on a feature map (FM) using a feature matrix, i.e., a convolution kernel. The elements in the convolution kernel are weights, and the feature map can include an input matrix or an output matrix in the neural network. Specifically, the convolution kernel slides on the feature map multiple times, and each sliding corresponds to a different position in the feature map. After each sliding, the convolution kernel is multiplied by the data at the corresponding position in the feature map and summed, i.e., multiplied and added, until all the multiplications and additions corresponding to the multiple slidings are completed, and the final convolution result forms a matrix, achieving feature extraction on the feature map.

[0004] In related technologies, convolution calculation can be implemented by a convolution calculation chip, such as a neural-network processing unit (NPU), a graphics processing unit (GPU), a convolution calculation module, etc. The convolution calculation chip usually includes a buffer, a matrix multiplier, an accumulator, and a register group. Taking a 3x3 convolution kernel as an example: when the convolution kernel slides on the feature map multiple times, the 9 weights in the convolution kernel correspond to 9 different sub-feature maps. The convolution calculation chip stores these 9 sub-feature maps in different regions of the buffer to read the data with consecutive storage addresses in each sub-feature map at each time, and performs multiplication and addition operation, i.e., matrix multiplication, on the read data of the sub-feature map and the weight corresponding to the sub-feature map in the buffer, and then adds the multiplication and addition result to the accumulated result saved in the register group at the previous time (the initial accumulated value is 0) through the accumulator, and updates the accumulated result to the register group to participate in the accumulation at the next time. In this way, 9 times of reading the sub-feature maps corresponding to the 9 weights and 9 times of accumulation can complete the convolution on the feature map and output the convolution result.

[0005] However, the convolution calculation chip needs to read and write the cache a lot to obtain multiple sub-feature maps corresponding to multiple weights respectively when performing convolution calculation, and accordingly, the read-write bandwidth and data transmission power consumption of the cache are large, especially in the scene where multiple convolution kernels exist and / or multiple convolutions are performed, which is more significant, and is easy to reduce the performance of the convolution calculation chip. SUMMARY

[0006] To solve the above technical problems, the present application provides a convolution calculation chip and method. The convolution calculation chip obtains at least two first data rows respectively belonging to different sub-feature maps by shifting the current data row of the feature map through the row sliding register group, which can participate in the multiplication and addition operation of different weights. In this way, the read result of the feature map cache can be used for multiple operations, thereby greatly reducing the read-write bandwidth and data transmission power consumption of the feature map cache, and improving the performance of the convolution calculation chip.

[0007] In a first aspect, the embodiments of the present application provide a convolution calculation chip, comprising: a feature map cache configured to cache a feature map; a row sliding register group coupled to the feature map cache and configured to obtain a current data row in the feature map, shift data in the current data row, and obtain at least two first data rows, wherein a plurality of second data rows in the plurality of first data rows obtained by shifting a plurality of data rows of the feature map have the same shift step, and the plurality of second data rows are used to form a sub-feature map, and the plurality of sub-feature maps correspond to a plurality of weights respectively; and a calculation unit coupled to the row sliding register group and configured to obtain the plurality of first data rows from the row sliding register group, obtain a weight corresponding to a sub-feature map to which each first data row belongs, and perform multiplication and addition operation on each first data row and the weight corresponding to the sub-feature map to which the first data row belongs to obtain a current calculation result of the sub-feature map, and the calculation results of the plurality of sub-feature maps are used to form a convolution calculation result of the feature map.

[0008] In the embodiment of the present application, the row sliding register group in the convolution calculation chip obtains the current data row of the feature map in the feature map cache, shifts the data in the current data row, and obtains at least two first data rows. Based on this, each data row of the plurality of data rows of the feature map can obtain at least two first data rows through shifting, thereby forming a plurality of first data rows, and a plurality of second data rows with the same shift step in the plurality of first data rows are used to form a sub-feature map, so that different shifts of the plurality of data rows in the feature map can form a plurality of sub-feature maps, and the plurality of sub-feature maps correspond to a plurality of weights respectively. Correspondingly, the plurality of first data rows can be used for multiplication and addition operation with the weights corresponding to the sub-feature map to which the first data row belongs, to obtain the current calculation result of the sub-feature map, so that the calculation results of the plurality of sub-feature maps form the convolution calculation result of the feature map. In this way, the convolution calculation chip can perform a plurality of operations corresponding to a plurality of weights based on the read and write result of the feature map in the feature map cache, thereby greatly reducing the read and write times of the feature map cache, and also greatly reducing the read and write bandwidth and data transmission power consumption of the feature map cache, and improving the performance of the convolution calculation chip.

[0009] According to the first aspect, the chip further includes: a weight cache coupled to the calculation unit, configured to cache at least one of the plurality of weights; a row sliding register group, specifically configured to divide each first data row corresponding to the current data row into a plurality of sub-data rows; and the calculation unit is specifically configured to obtain, in a plurality of clock cycles, a plurality of sub-data rows of the first data row corresponding to the current data row from the row sliding register group, and obtain the weights corresponding to the sub-feature map to which the first data row belongs from the weight cache.

[0010] In the embodiment of the present application, the row sliding register group divides each first data row corresponding to the current data row into a plurality of sub-data rows, which can ensure that the calculation unit uses the weights corresponding to one first data row to perform multiplication and addition operation on a plurality of sub-data rows in a plurality of clock cycles, that is, one weight can remain unchanged in a plurality of consecutive clock cycles for corresponding multiplication and addition operation. In this way, one weight can be used repeatedly, and accordingly, the read bandwidth and power consumption of the weight cache can be greatly reduced, and the frequency of weight change can be reduced, so that the flip rate of the input signal in the calculation unit can be greatly reduced, thereby greatly reducing the power consumption of the matrix multiplier.

[0011] According to the first aspect, or any one of the implementation manners of the first aspect, the chip further includes: a plurality of row backup register groups respectively coupled to the feature map cache, configured to obtain a plurality of data rows from the feature map cache, wherein each row backup register group is configured to cache one data row of the plurality of data rows; and the row sliding register group is coupled to the plurality of row backup register groups, and is specifically configured to obtain one data row of the plurality of data rows from each row backup register group.

[0012] In the embodiments of the present application, the data rows in the feature map are backed up by the plurality of row backup register groups. Thus, the data rows that need to be reused can be directly obtained from the row backup register groups, without the need to repeatedly read from the feature map cache. Therefore, the read bandwidth and power consumption of the feature map cache can be further reduced.

[0013] According to the first aspect, or any one of the implementations of the above first aspect, the row sliding register group is specifically configured to: respectively shift the data in the plurality of data rows by the same shift step to obtain a third data row corresponding to each data row in the plurality of data rows, the third data row corresponding to the data row being one of the at least two first data rows corresponding to the data row; and the calculation unit includes: a multiply-add unit coupled to the row sliding register group, configured to obtain the current third data row of each data row in the plurality of data rows from the row sliding register group, and obtain the weight corresponding to the current sub-feature map, and perform multiply-add operation based on the current third data row, the weight corresponding to the current sub-feature map, and the existing one-row accumulation result corresponding to the plurality of sub-feature maps obtained from the row accumulation register group, to obtain the current one-row accumulation result corresponding to the plurality of sub-feature maps, wherein the current sub-feature map is the sub-feature map to which the current third data row belongs, the one-row accumulation result corresponding to the plurality of sub-feature maps corresponds to the one-row convolution calculation result of the feature map, and the final accumulation result corresponding to the plurality of sub-feature maps is the convolution calculation result of the feature map; and the plurality of row accumulation register groups are respectively coupled to the multiply-add unit, wherein each row accumulation register group is configured to obtain the current one-row accumulation result corresponding to the plurality of sub-feature maps corresponding to the one-row convolution calculation result of the feature map from the multiply-add unit, and update the existing one-row accumulation result corresponding to the plurality of sub-feature maps corresponding to the one-row convolution calculation result of the feature map using the current one-row accumulation result corresponding to the plurality of sub-feature maps corresponding to the one-row convolution calculation result of the feature map.

[0014] In the embodiments of the present application, in combination with the plurality of row backup register groups, the plurality of row accumulation register groups are used to respectively cache the accumulation results of the plurality of data rows. In the process of obtaining the accumulation results of the plurality of data rows, each data row has a second sub-data row using the same weight. Thus, one weight can be reused multiple times. In this way, the read bandwidth and power consumption of the weight cache can be further reduced, one weight can remain unchanged within the time of calculating multiple data rows, and the signal flipping rate of the matrix multiplier is greatly reduced, thereby greatly reducing the calculation power consumption of the matrix multiplier.

[0015] In a second aspect, the embodiments of the present application provide a convolution calculation method applied to a convolution calculation chip, the method comprising: caching a feature map; obtaining a current data row in the feature map, shifting data in the current data row to obtain at least two first data rows, a plurality of second data rows in the plurality of first data rows obtained by shifting the plurality of data rows in the feature map have the same shift step, and the plurality of second data rows are used to form a sub-feature map, and a plurality of sub-feature maps correspond to a plurality of weights respectively; obtaining the plurality of first data rows, obtaining the weight corresponding to the sub-feature map to which each first data row in the plurality of first data rows belongs, and performing multiplication and addition operation on each first data row and the weight corresponding to the sub-feature map to which the first data row belongs in the plurality of weights to obtain a current calculation result of the sub-feature map to which the first data row belongs, and the calculation results of the plurality of sub-feature maps are used to form a convolution calculation result of the feature map.

[0016] According to the second aspect, before obtaining the weight corresponding to the sub-feature map to which each first data row in the plurality of first data rows belongs, the method further comprises: caching at least one of the plurality of weights; after shifting the data in the current data row to obtain the at least two first data rows, the method further comprises: for each first data row corresponding to the current data row, dividing the first data row into a plurality of sub-data rows; obtaining the plurality of first data rows, and obtaining the weight corresponding to the sub-feature map to which each first data row in the plurality of first data rows belongs, comprises: obtaining the plurality of sub-data rows of the first data row corresponding to the current data row in a plurality of clock cycles, and obtaining the weight corresponding to the sub-feature map to which the first data row belongs.

[0017] According to the second aspect, or any one of the implementation modes of the second aspect, the method further comprises: obtaining the plurality of data rows from the cached feature map, and caching one data row in the plurality of data rows respectively; and obtaining one data row in the plurality of data rows respectively to obtain the current data row in the feature map.

[0018] According to a second aspect, or any possible implementation mode of the second aspect, the obtaining the current data row in the feature map, and shifting the data in the current data row to obtain at least two first data rows comprises: shifting the data in the plurality of data rows by the same shift step to obtain a third data row corresponding to each data row in the plurality of data rows, the third data row corresponding to each data row being one of the at least two first data rows corresponding to the data row; obtaining the current third data row of each data row in the plurality of data rows, and obtaining the weight corresponding to the current sub-feature map; performing multiplication and addition operation on the current third data row, the weight corresponding to the current sub-feature map, and the obtained previous row accumulation result corresponding to the plurality of sub-feature maps from the row accumulation register group, to obtain a current row accumulation result corresponding to the plurality of sub-feature maps, wherein the current sub-feature map is the sub-feature map to which the current third data row belongs, the row accumulation result corresponding to the plurality of sub-feature maps corresponds to the one row convolution calculation result of the feature map, and the final accumulation result corresponding to the plurality of sub-feature maps is the convolution calculation result of the feature map; obtaining the current row accumulation result corresponding to the plurality of sub-feature maps corresponding to the one row convolution calculation result of the feature map, and updating the previous row accumulation result corresponding to the plurality of sub-feature maps corresponding to the one row convolution calculation result of the feature map using the current row accumulation result corresponding to the plurality of sub-feature maps corresponding to the one row convolution calculation result of the feature map.

[0019] The second aspect and any possible implementation mode of the second aspect correspond to the first aspect and any possible implementation mode of the first aspect respectively. The technical effects of the second aspect and any possible implementation mode of the second aspect can refer to the technical effects of the first aspect and any possible implementation mode of the first aspect, which will not be described herein.

[0020] In a third aspect, the embodiments of the present application provide a computer readable medium for storing a computer program, the computer program comprising instructions for executing the method in the second aspect or any possible implementation mode of the second aspect.

[0021] In a fourth aspect, the embodiments of the present application provide a computer program comprising instructions for executing the method in the second aspect or any possible implementation mode of the second aspect. BRIEF DESCRIPTION OF DRAWINGS

[0022] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the description of the embodiments of the present application. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative labor.

[0023] Figure 1 is a process diagram of 3x3 convolution calculation of 16 input channels and 16 output channels;

[0024] Figure 2 is a process diagram of 3x3 convolution kernel corresponding convolution calculation;

[0025] Figure 3 is a structural block diagram of a convolution calculation chip;

[0026] Figure 4 is a timing diagram of convolution calculation of a convolution calculation chip;

[0027] Figure 5 is a structural block diagram of an electronic device applying a convolution calculation chip according to an embodiment of the present application;

[0028] Figure 6 is a structural block diagram of a convolution calculation chip according to an embodiment of the present application;

[0029] Figure 7 is a timing diagram of row sliding accumulation according to an embodiment of the present application;

[0030] Figure 8 is a process diagram of 3x3 convolution calculation of a convolution calculation chip according to an embodiment of the present application;

[0031] Figure 9 is a timing diagram of row sliding accumulation in 3x3 convolution according to an embodiment of the present application;

[0032] Figure 10 is a flow diagram of a convolution calculation method according to an embodiment of the present application;

[0033] Figure 11 is a structural block diagram of a convolution calculation chip according to an embodiment of the present application;

[0034] Figure 12 is a structural block diagram of a convolution calculation chip according to an embodiment of the present application;

[0035] Figure 13 is a process diagram of 3x3 convolution calculation of a convolution calculation chip according to an embodiment of the present application;

[0036] Figure 14 is a structural block diagram of a convolution calculation chip according to an embodiment of the present application;

[0037] Figure 15 is a structural block diagram of a convolution calculation chip according to an embodiment of the present application;

[0038] Figure 16 is a process diagram of 3x3 convolution calculation of a convolution calculation chip according to an embodiment of the present application;

[0039] Figure 17 is a timing diagram of row sliding accumulation in 3x3 convolution according to an embodiment of the present application. DETAILED DESCRIPTION

[0040] With reference to the drawings, the technical solutions in the embodiments of the present application will be clearly and completely described below. Obviously, the described embodiments are only some of the embodiments of the present application, but not all of them. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts should fall within the scope of the present application.

[0041] The term "and / or" used in the present application is only used to describe an association relationship of associated objects, and indicates that there can be three relationships, for example, A and / or B can represent three cases of A existing alone, A and B existing together, and B existing alone.

[0042] The terms "first" and "second" and the like in the description and claims of the embodiments of the present application are used to distinguish different objects, and are not used to describe a specific order of the objects. For example, the first target object and the second target object are used to distinguish different target objects, and are not used to describe a specific order of the target objects.

[0043] In the embodiments of the present application, the words "exemplary" or "for example" are used to mean serving as an example, instance, or illustration. Any embodiment or design presented as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or advantageous than other embodiments or design solutions. Rather, the use of "exemplary" or "for example" is intended to present concepts in a particular manner.

[0044] In the description of the embodiments of the present application, unless otherwise specified, the meaning of "a plurality of" is two or more. For example, a plurality of processing units refers to two or more processing units; a plurality of systems refers to two or more systems.

[0045] In order to facilitate the understanding of the embodiments, first, some technical terms and background technologies related to the embodiments will be introduced:

[0046] Register: a limited high-speed access memory directly integrated into a chip, which includes some NAND gate combinations and is divided into general registers and special registers. The core of the chip accesses the register at the fastest speed.

[0047] Static random-access memory (SRAM): a random-access memory. The so-called "static" means that as long as the power supply is maintained, the data stored in the memory can be constantly maintained.

[0048] Input channel and output channel: A feature map has four dimensions of batch size, channel, width and height, where the number of channels can be one or more. When a convolution calculation chip uses a convolution kernel to perform convolution calculation on a feature map, it inputs data in the feature map by channel, so the channels of the feature map can be referred to as input channels. That is, the input channels are independent channels in a convolutional neural network for inputting data, and one input channel can correspond to one dimension of input data. For example, when processing image data, the number of input channels usually corresponds to the number of color channels of the image, such as an RGB image having three input channels and a grayscale image having only one input channel. The number of input channels can reflect the dimension and complexity of the input data. Correspondingly, a convolution kernel contains multiple input channels and output channels, the input channels of the convolution kernel correspond to the input channels of the feature map, and are used for convolution calculation of the convolution kernel and the data in the feature map corresponding to the input channels. The output channels of the convolution kernel correspond to the output channels of the feature map, and are used for outputting the convolution calculation results corresponding to the input channels. That is, the output channels are independent channels in a convolutional neural network for outputting data. The number of output channels can reflect the dimension and complexity of the output data.

[0049] With the rapid development of AI technology, convolution calculation for neural networks has been widely applied. A feature matrix performs convolution calculation on an input matrix, thereby realizing extraction of a certain feature of the input data, and the obtained output matrix is used as the input of the next convolution calculation, so that AI algorithms can be realized through multiple convolution calculations. Among them, the feature matrix, i.e., the convolution kernel, contains the weights of each pixel point, and the input matrix / output matrix is the feature map. For ease of understanding, the convolution calculation is described below with reference to FIG. 1.

[0050] For example, FIG. 1 is a process diagram of 3x3 convolution calculation with 16 input channels and 16 output channels. As shown in FIG. 1, each of the 16 input channels of the feature map 0-15 corresponds to an input channel of a 3x3 convolution kernel to perform convolution calculation, and each of the 16 output channels 0-15 corresponds to a 3x3 convolution kernel to output the result of the convolution calculation corresponding to the convolution kernel. Specifically, W(CHout, Ch in, i, j) represents the weight value of the position coordinate (i, j) in the 3x3 convolution kernel corresponding to the input channel CHin and the output channel CHout; and F(Chin, x, y) represents the pixel value of the pixel point with coordinate (x, y) in the feature map input to the input channel Ch in.

[0051] The convolution kernel slides to the right and performs multiplication and addition operation with the pixel points in the feature map corresponding to the sliding position of the convolution kernel to output F'(CHout, x, y), that is, the convolution operation on the feature map is completed. Wherein, F'(CHout, x, y) is the output feature map, that is, the pixel value of the pixel point with coordinates (x, y) and output channel CHout in the convolution result, that is, the multiplication and addition operation satisfies the following formula:

[0052] Wherein, the initial value of n is the coordinate of the first row and the first column of the convolution kernel in the feature map in the process of sliding the convolution kernel on the feature map; the value of n depends on the size of the feature map.

[0053] The convolution calculation chip usually implements convolution calculation based on the im2col method. Taking a 3x3 convolution kernel, 16 input channels and 16 output channels as an example, the convolution calculation based on the im2col method is described. For example, FIG. 2 is a process diagram of the convolution calculation corresponding to the 3x3 convolution kernel. As shown in FIG. 2, the convolution calculation chip arranges the pixel points of the feature map in a column in the order from left to right and from top to bottom, and expands each pixel point into a row by channel, so that 16 feature maps represented by a list, that is, in the form of a row, are obtained, thereby obtaining an input matrix with a width of 16 and a height of (FM_Wx(FM_H-2)-2), that is, the input feature map shown in FIG. 2. Wherein, the row formed by different pixel points of each input channel can be regarded as a feature map with only the dimension reflected by the input channel. For example, the first row and the first column under the first input channel is a feature map in the form of F(Chin=0, 0, 0), the first row and the first column under the first input channel is a feature map in the form of F(Chin=1, 0, 0), …, the first row and the first column under the 16th input channel is a feature map in the form of F(Chin=15, 0, 0), wherein FM_W indicates the width of the feature map, and FM_H indicates the height of the feature map. Each weight in the convolution kernel is expanded into a matrix of output channel x input channel. When the convolution kernel slides on the input feature map multiple times, 9 sub-feature maps corresponding to the 9 weights in the weight matrix can be formed: sub-feature map 1 corresponding to the weight (i=0, j=0) in the weight matrix, sub-feature map 2 corresponding to the weight (i=0, j=1) in the weight matrix, …, sub-feature map 4 corresponding to the weight (i=1, j=0) in the weight matrix, …, sub-feature map 9 corresponding to the weight (i=2, j=2) in the weight matrix. Each weight in the convolution kernel is multiplied by the sub-feature map corresponding to the weight to obtain 9 intermediate matrices, and the 9 intermediate matrices are accumulated to obtain an output matrix, that is, the convolution result of the input feature map.

[0054] It can be understood that, after the processing of the above-mentioned embodiment of FIG. 2, the feature map contains pixel points distributed according to the input channels of the feature map, and the convolution processing of the feature map is to perform convolution calculation on the pixel points under each input channel of the feature map. The convolution calculation performed by the pixel points under different input channels is different in that the data in the calculated feature map reflects different dimensions. For example, for three input channels processing a color image, the image data input to the R channel reflects the red dimension of the color image, the image data input to the G channel reflects the green dimension of the color image, and the image data input to the B channel reflects the green dimension of the color image. Similarly, the data output by different output channels reflects different dimensions of the output data. The number of input channels and the number of output channels can be set according to specific application requirements, and the embodiments of the present application do not limit this. The data of each input channel can be expanded into a row form and spliced into a feature map in the manner of the above-mentioned embodiment of FIG. 2 for parallel processing, and the output results obtained after parallel processing can be output in parallel through the corresponding output channels. In order to facilitate description and understanding, the structure and function of the convolution calculation chip are described in the case of convolution of a feature map of one input channel, and the hardware structure and function corresponding to the remaining input channels can be referred to the case of one input channel, which will not be described again.

[0055] For example, FIG. 3 is a structural diagram of a convolution calculation chip. As shown in FIG. 3, in the related art, the convolution calculation chip includes a feature map buffer (SRAM), a weight buffer (SRAM), an accumulator and a register group coupled with a matrix multiplier (CUBE) respectively to implement the convolution calculation provided in the embodiment of FIG. 2. When a 3x3 convolution kernel slides and convolves in an input feature map, 9 weights in the convolution kernel can form 9 different sub-feature maps respectively corresponding to different regions in the input feature map when the convolution kernel slides in the input feature map. Based on this, the convolution calculation chip reads and writes the input feature map multiple times through an external bus coupled with the feature map buffer to obtain 9 sub-feature maps corresponding to the 9 weights respectively: sub-feature maps ① to ⑨, and caches the sub-feature maps ① to ⑨ in different regions of the feature map buffer (FM buffer) to continuously read the data in the input feature map. In this way, for example, FIG. 4 is a timing diagram of the convolution calculation of the NPU. As shown in FIG. 4, the NPU reads M data of one sub-feature map of the 9 sub-feature maps from the feature map buffer into the matrix multiplier (CUBE) and inputs the weights corresponding to the sub-feature map among the 9 weights A to I in the weight buffer into the matrix multiplier to perform multiplication and addition calculation every beat, that is, every clock cycle. The multiplication and addition result is added to the accumulated result saved in the register group in the last beat through the accumulator (the initial accumulated result is 0), and the current accumulated result is updated to the register group for accumulation in the next beat. In this way, the data of the 9 sub-feature maps is sequentially read, and the final convolution result is obtained after 9 times of accumulation. Then, the calculation output is output through the bus, that is, the convolution result of the input feature map is output.

[0056] However, the NPU provided in the embodiment of FIG. 3 needs to read and write the input feature map multiple times to obtain the 9 sub-feature maps: sub-feature maps ① to ⑨ stored in the feature map buffer when performing convolution calculation, resulting in large read and write bandwidth and data transmission power consumption of the feature map buffer. In addition, as shown in FIG. 4, the sub-feature map and the weight input into the matrix multiplier change every clock cycle, that is, every beat, for example, changing from sub-feature map ① to ⑨ and from weight A to I in 9 clock cycles, resulting in high flip rate of the smallest calculation unit in the matrix multiplier, that is, the partial product integration (PPI), and large power consumption of the matrix multiplier.

[0057] Therefore, the embodiment of the present application provides a convolution calculation chip to solve the above problems. The row sliding register group in the chip obtains a current data row of a feature map in a feature map cache, shifts the data in the current data row to obtain at least two first data rows. Based on this, each data row of the plurality of data rows of the feature map can obtain at least two first data rows through shifting to form a plurality of first data rows, and a plurality of second data rows with the same shift step in the plurality of first data rows are used to form a sub-feature map, so that different shifts of the plurality of data rows of the feature map can form a plurality of sub-feature maps, and the plurality of sub-feature maps correspond to a plurality of weights respectively. Correspondingly, the plurality of first data rows can be used for multiplication and addition operation with the weights corresponding to the sub-feature map to which the first data row belongs to obtain the current calculation result of the sub-feature map, so that the convolution calculation result of the feature map is formed by the calculation results of the plurality of sub-feature maps. In this way, the convolution calculation chip can perform a plurality of operations corresponding to a plurality of weights based on one reading and writing result of the feature map in the feature map cache, thereby greatly reducing the reading and writing times of the feature map cache, and also greatly reducing the reading and writing bandwidth and data transmission power consumption of the feature map cache, and improving the performance of the convolution calculation chip.

[0058] For example, in the case of an n x k convolution kernel, a data row of the feature map transmitted to the sliding register group and the result obtained by shifting the data row can be used for matrix calculation of k weights respectively, that is, one reading result of the feature map cache can be used k times, and the reading and writing bandwidth and power consumption of the feature map cache can be reduced by k times, thereby greatly improving the performance of the convolution calculation chip.

[0059] Next, the convolution calculation chip of the embodiment of the present application will be described in conjunction with the drawings.

[0060] The convolution calculation chip provided by the embodiment of the present application can be applied to an electronic device. For example, the electronic device can be a mobile terminal, a computer, a server, a smart large screen, a wearable smart device, a tablet computer, etc. For the convenience of description and reasonable layout, the structure of the convolution calculation chip and the electronic device will be described in detail below. For example, FIG. 5 is a structural block diagram of an electronic device applying the convolution calculation chip according to an embodiment of the present application. As shown in FIG. 5, the electronic device 100 can include a convolution calculation chip 500, a communication interface 101 and an external memory 102 coupled by a bus 504.

[0061] The convolution calculation chip 500 can include:

[0062] a feature map cache 501 for caching a feature map;

[0063] The row sliding register group 502 is coupled to the feature map cache 501, and is configured to obtain a current data row in the feature map, and shift data in the current data row to obtain at least two first data rows. A plurality of second data rows obtained by shifting a plurality of data rows of the feature map have a same shift step, and are used to form a sub-feature map. A plurality of sub-feature maps correspond to a plurality of weights respectively.

[0064] The calculation unit 503 is coupled to the row sliding register group 502, and is configured to obtain the plurality of first data rows from the row sliding register group 502, and obtain a weight corresponding to a sub-feature map to which each first data row belongs. The calculation unit 503 is further configured to multiply and add each first data row with the weight corresponding to the first data row in the plurality of weights to obtain a current calculation result of the sub-feature map to which the first data row belongs. Calculation results of the plurality of sub-feature maps are used to form a convolution calculation result of the feature map.

[0065] The above coupling may, for example, be through a circuit connection, or through software interaction, etc. In an example, the components in the convolution calculation chip 500 may be coupled through a bus 504. For example, the bus 504 is a transmission line bundle including wires, and may be a bus system including a data bus, an address bus and a control bus for transmitting data, data addresses and control signals respectively, according to the type of information transmitted.

[0066] In order to facilitate understanding and reasonable layout, the structure of the convolution calculation chip 500 and the functions implemented by the components in the convolution calculation chip 500 will be described in detail in combination with FIGS. 6 to 17.

[0067] The communication interface 101 is any kind of electronic device such as a transceiver, and is configured to communicate with other devices or communication networks, such as an Ethernet, a radio access network (RAN), a wireless local area network (WLAN), etc.

[0068] The external memory 102 is connected to the communication interface 101 through the bus 504. In an example, the external memory 102 may be coupled to one of the on-chip memory 506 and the feature map cache 501 in the convolution calculation chip 500 through an external storage interface (not shown in the figure) or the bus 504. The external memory 102 is configured to store the feature map obtained through the communication interface 101, and transmit the feature map to the convolution calculation chip. For example, the external memory 102 can be a double data rate synchronous dynamic random access memory (DDR). In an example, the external memory 102 can also be configured to store the convolution calculation result output by the calculation unit 503 to reduce the storage pressure of the on-chip memory 506.

[0069] FIG. 5 shows a structural schematic diagram of the electronic device 100 and the convolution calculation chip 500. It should be understood that the electronic device 100 and the convolution calculation chip 500 shown in FIG. 5 are only one example of the electronic device and the convolution calculation chip respectively, and the electronic device 100 and the convolution calculation chip 500 can have more or fewer components than shown in the figure, can combine two or more components, or can have a different component configuration. The functions of various components shown in FIG. 5 can be implemented in hardware, software, or a combination of hardware and software including one or more signal processing and / or application specific integrated circuits.

[0070] For example, FIG. 6 is one of structural block diagrams of a convolution calculation chip provided by an embodiment of the present application. As shown in FIG. 6, the convolution calculation chip includes an input bus, a feature map cache, a weight cache, a row sliding register group, a matrix multiplier, an accumulator, a row accumulation register group, and an output bus. Specifically, in the convolution calculation chip:

[0071] The input bus is configured to transmit the feature map to the feature map cache and transmit the convolution kernel to the weight cache.

[0072] The feature map cache is configured to cache the feature map.

[0073] The row sliding register group is configured to obtain a current data row of the feature map in the feature map cache, and sequentially cache data in the current data row in a plurality of registers of the row sliding register group, the plurality of registers including a plurality of registers connected to each other in pairs except registers at the beginning and the end.

[0074] Similar to the embodiment of FIG. 2, the feature map in the feature map cache of FIG. 6 is stored in the form of being unfolded into a row, and the current data row of the feature map is a row of the feature map in the form of a matrix. In one example, the interval between each two connected registers in the plurality of registers, i.e., the minimum interval, can be the same or different. For example, in FIG. 6, the minimum intervals of the plurality of registers are the same, and each is 0 registers, i.e., the two connected registers are adjacent. For example, the data in the current data row includes F(0, 0), F(0, 1), …, F(B-1, M) in sequence, and the two connected registers in the plurality of registers are adjacent, and from right to left, they are register 0, register 1, …, register R in sequence. The row sliding register group sequentially caches the data in the current data row from the first register on the right: F(0, 0) is cached in register 0, F(0, 1) is cached in register 1, …, F(B-1, M-1) is cached in register R. Wherein, (B-1) x M is the total number of data contained in the current data row.

[0075] It can be understood that the minimum interval in the plurality of registers can be 0, one or more registers, for example, register 1 and register 2 are connected, and there are 2 registers between them; register 3 and register 4 are connected, and there are 2 registers between them, etc. The interval between each two connected registers in the plurality of registers can be set according to application requirements, and the embodiments of the present application do not limit this. In addition, the data in the data row can be sequentially cached from the first register on the right or the first register on the left in the plurality of registers, and the caching direction can be set according to application requirements, and the embodiments of the present application do not limit this.

[0076] In an optional embodiment, the row sliding register group is specifically used for:

[0077] According to a 0th shift step of moving 0 bits to the previously stored adjacent register, and a shift step of moving 1 bit to the previously stored adjacent register after the 0th time, the data in the current data row cached in each register in the plurality of registers is shifted multiple times, and each shift obtains a first data row;

[0078] Among the plurality of first data rows obtained by shifting the plurality of data rows of the feature map, a plurality of second data rows with the same shift step are used to form a sub-feature map, and the plurality of sub-feature maps correspond to the plurality of weights respectively.

[0079] Similar to the embodiment of FIG. 2, when the convolution kernel in the embodiment slides on the feature map, the data corresponding to different regions in the input feature map in the plurality of weights in the convolution kernel can form a plurality of different sub-feature maps respectively, that is, the plurality of weights in the convolution kernel correspond to the plurality of sub-feature maps one by one. For example, the weight (0, 0) of the convolution kernel in the weight cache shown in FIG. 6 corresponds to the sub-feature map 1, the weight (0, 1) corresponds to the sub-feature map 2, the weight (0, 2) corresponds to the sub-feature map 3, …, the weight (n-1, k-3) corresponds to the sub-feature map n×k-2, the weight (n-1, k-2) corresponds to the sub-feature map n×k-1, and the weight (n-1, k-1) corresponds to the sub-feature map n×k.

[0080] For example, FIG. 7 is a timing diagram of row sliding accumulation provided by an embodiment of the present application. As shown in FIG. 7, the feature map includes a first data row F0, …, an n-th data row Fn, and a (n+1)-th data row Fn+1. The data row F0 includes data F0(0, 0), F0(0, 1), …, F0(B-1, M-1). The data row Fn includes data Fn(0, 0), Fn(0, 1), …, Fn(B-1, M-1). The data row Fn+1 includes data Fn+1(0, 0), Fn+1(0, 1), …, Fn+1(B-1, M-1). n-1 ; data row F0 includes data F0(0, 0), F0(0, 1), …, F0(B-1, M-1); data row Fn includes data Fn(0, 0), Fn(0, 1), …, Fn(B-1, M-1); data row Fn+1 includes data Fn+1(0, 0), Fn+1(0, 1), …, Fn+1(B-1, M-1). n-1 ; data row F0 includes data F0(0, 0), F0(0, 1), …, F0(B-1, M-1); data row Fn includes data Fn(0, 0), Fn(0, 1), …, Fn(B-1, M-1); data row Fn+1 includes data Fn+1(0, 0), Fn+1(0, 1), …, Fn+1(B-1, M-1). n-1 ; data row F0 includes data F0(0, 0), F0(0, 1), …, F0(B-1, M-1); data row Fn includes data Fn(0, 0), Fn(0, 1), …, Fn(B-1, M-1); data row Fn+1 includes data Fn+1(0, 0), Fn+1(0, 1), …, Fn+1(B-1, M-1). n-1 ; data row F0 includes data F0(0, 0), F0(0, 1), …, F0(B-1, M-1); data row Fn includes data Fn(0, 0), Fn(0, 1), …, Fn(B-1, M-1); data row Fn+1 includes data Fn+1(0, 0), Fn+1(0, 1), …, Fn+1(B-1, M-1). n-1(B-1, M-1). The row sliding register group obtains data in the feature map row by row from the first row, and sequentially caches data in the current data row in a plurality of registers connected in pairs and adjacent to each other except the first and last registers from the first register on the right. The plurality of registers are sequentially register 0, register 1, …, register R from right to left.

[0081] Based on the above caching function of the row sliding register group, in the case that the current data row is the first data row F0:

[0082] The row sliding register group sequentially performs the following caching: F0(0, 0) is cached in register 0, F0(0, 1) is cached in register 1, …, F0(B-1, M-1) is cached in register R. At this time, the shift step is 0, that is, the first time of shifting, and the first first data row of the sub-feature map 1 is obtained.

[0083] The row sliding register performs the first time of shifting on the first data row F0: the data cached in register 0 is shifted to the right by one bit, that is, discarded, the data cached in register 1 is shifted to the right by one bit, that is, moved to the adjacent register 0 previously cached, …, the data cached in register R is shifted to the right by one bit, that is, moved to the adjacent register R-1 previously cached. In this way, the first time of shifting result of the first data row F0 is obtained: F0(0, 1) is cached in register 0, F0(0, 2) is cached in register 1, …, F0(B-1, M-1) is cached in register R-1, register R is empty or caches data 0, and the first first data row of the sub-feature map 2 is obtained.

[0084] The row sliding register performs the second time of shifting on the first data row F0: the data cached in register 0 is shifted to the right by one bit, that is, discarded, the data cached in register 1 is shifted to the right by one bit, that is, moved to the adjacent register 0 previously cached, …, the data cached in register R is shifted to the right by one bit, that is, moved to the adjacent register R-1 previously cached. In this way, the second time of shifting result of the first data row F0 is obtained: F0(0, 2) is cached in register 0, F0(0, 2) is cached in register 1, …, F0(B-1, 3) is cached in register R-2, register R-1 and register R are empty or cache data 0, and the first first data row of the sub-feature map 3 is obtained.

[0085] In this way, at least two first data rows of the first data row F0 are obtained through the above multiple times of shifting on the first data row F0. That is, one first data row of the current data row in the feature map is obtained through one time of shifting, and at least two first data rows of the current data row are obtained through multiple times of shifting. One first data row of the current data row corresponds to one weight, different first data rows of the current data row belong to different sub-feature maps respectively, and correspondingly, different weights correspond to different first data rows of the current data row respectively.

[0086] The row sliding register group is used for sliding the remaining data rows F1 to F n-1 The above-mentioned caching and shifting are respectively performed on the first data row F0, and the difference is that the processed data is different, so as to obtain at least two first data rows corresponding to each data row in the feature map respectively. Among the plurality of first data rows corresponding to all data rows in the feature map, the plurality of second data rows with the same shifting step are used to form a sub-feature map, so as to obtain the remaining data rows of the sub-feature map 1, that is, the remaining first data rows, and the remaining data rows of each of the sub-feature maps 2 to sub-feature maps n x k. For example, the row sliding register group performs the 0th shifting on the data of the cached data row F1, that is, the cached data row F1 is the second first data row of the sub-feature map 1, and the first shifting on the data row F1 is the second first data row of the sub-feature map 2.

[0087] Among them, the shifting step of the first data row obtained by moving the data row F1 by 0 bits and the first data row obtained by moving the data row F0 by 0 bits is 0, and these two first data rows are the two second data rows used to form the sub-feature map 1. The shifting step of the second first data row of the sub-feature map 1 and the first first data row of the sub-feature map 1 is 0, that is, the two second data rows used to form the sub-feature map 1. The shifting step of the first data row obtained by moving the data row F1 by 1 bit and the first data row obtained by moving the data row F0 by 1 bit is 1, and these two first data rows are the two second data rows used to form the sub-feature map 2.

[0088] In an optional example, the convolution calculation chip can further include a weight cache coupled to the calculation unit;

[0089] The weight cache is used for caching at least one weight in the convolution kernel;

[0090] The row sliding register group is specifically used for dividing each first data row corresponding to the current data row into a plurality of sub-data rows;

[0091] The calculation unit is specifically used for obtaining, in a plurality of clock cycles, a plurality of sub-data rows of the first data row corresponding to the current data row from the row sliding register group, and obtaining the weight corresponding to the first data row from the weight cache.

[0092] For example, as shown in FIG. 7, the row sliding register group can transmit M data in one clock cycle, and each first data row of the current data row, for example, data row F0, can be divided into B sub-data rows, and each sub-data row contains M data. The sub-data row is equivalent to a data block containing M data. For example, the row sliding register group divides the data row F0 into B sub-data rows, wherein the first sub-data row includes the data F0(0, 0), F0(0, 1), …, F0(0, M-1) transmitted in the first clock cycle; the second sub-data row includes the data F0(1, 0), F0(1, 1), …, F0(1, M-1) transmitted in the second clock cycle; …, and the Bth sub-data row includes the data F0(B-1, 0), F0(B-1, 1), …, F0(B-1, M-1) transmitted in the Bth clock cycle. Both B and M are integers greater than 1.

[0093] Still referring to FIG. 7, the row sliding register group divides each first data row into B sub-data rows, and one sub-data row contains M data. Then, the calculation unit can obtain B sub-data rows in continuous B clock cycles, and ensure that the multiplication and addition operation of the B sub-data rows is performed using the weight corresponding to the first data row to which the B sub-data rows belong in the continuous B clock cycles. In this way, one weight can be repeatedly used B x M times. Accordingly, the reading bandwidth and power consumption of the weight cache can be reduced by B x M times, and the flip rate of the input signal in the calculation unit can be greatly reduced, thereby greatly reducing the power consumption of the calculation unit.

[0094] In addition, the row sliding register group can transmit one sub-data row to the calculation unit in each clock cycle, where the transmission can be that the row sliding register group writes the sub-data row to the calculation unit, or the calculation unit can read the sub-data row from the row sliding register group. The subsequent data acquisition mode is similar to the transmission, and the difference lies in the acquired data. The specific data acquisition mode is not limited in the embodiments of the present application, and can be set according to application requirements.

[0095] In the embodiments of the present application, the row sliding register group divides each first data row corresponding to the current data row into multiple sub-data rows, which can ensure that the calculation unit performs multiplication and addition operation on multiple sub-data rows using the weight corresponding to one first data row in multiple clock cycles, that is, one weight can remain unchanged in continuous multiple clock cycles for corresponding multiplication and addition operation. In this way, one weight can be repeatedly used multiple times, and accordingly, the reading bandwidth and power consumption of the weight cache can be greatly reduced, and the weight change frequency can be reduced, which can greatly reduce the flip rate of the input signal in the calculation unit, thereby greatly reducing the power consumption of the matrix multiplier.

[0096] Exemplarily, still referring to FIG. 6, the computing unit in the convolution calculation chip can specifically include a matrix multiplier coupled to the row sliding register group and the weight cache respectively, and an accumulator coupled to the matrix multiplier and the row accumulation register group respectively, specifically:

[0097] the matrix multiplier is configured to obtain a plurality of first data rows from the row sliding register group, and obtain the weights corresponding to the sub-feature map to which each first data row belongs through the weight cache; perform matrix multiplication operation on each first data row in the plurality of first data rows and the weights corresponding to the sub-feature map to which the first data row belongs, to obtain a matrix calculation result of the first data row;

[0098] the accumulator is configured to obtain the matrix calculation result of the current first data row from the matrix multiplier, and obtain the last accumulation result from the row accumulation register group, and perform accumulation on the matrix calculation result of the current first data row and the last accumulation result to obtain a current accumulation result; and update the current accumulation result to the row accumulation register group;

[0099] the row accumulation register group is configured to obtain the current accumulation result from the accumulator, and update the last accumulation result using the current accumulation result until the update of the last accumulation result corresponding to the last data row of the feature map is completed, to obtain the convolution calculation result of the feature map.

[0100] Still referring to FIG. 7, the current first data row is data row F0, the matrix multiplier performs matrix multiplication operation on data row F0 and weight (0, 0) to obtain the matrix calculation result of data row F0, and transmits the matrix calculation result of data row F0 to the accumulator. The accumulator accumulates the matrix calculation result of data row F0 and the last accumulation result 0 (which is empty or the initial value 0) to obtain the current accumulation result, i.e., the 0th accumulation result: the accumulation result of the matrix calculation result of data row F0 and the initial value, and transmits the current accumulation result to the row accumulation register group. The row accumulation register group updates the last accumulation result 0 using the current accumulation result. The current accumulation result is the accumulation result of the current first data row (the low 0th shift result of data row F0 in the feature map: data row F0), which is also the current accumulation result of the data row corresponding to the current first data row (data row F0 in the feature map), and is also the current calculation result of the sub-feature map (sub-feature map 1) to which the current first data row belongs.

[0101] Similar to the processing of the data row F0, when the first-time shift result of the current first data row F0 is processed, that is, when the data row F0 is shifted by one bit to obtain a data row, the last-time accumulation result is the above-mentioned 0-time accumulation result: the accumulation result of the matrix calculation result of the data row F0 and the initial value. The accumulator accumulates the matrix calculation result of the first-time shift result of the data row F0 and the 0-time accumulation result to obtain the current accumulation result, that is, the 1-time accumulation result: the accumulation result of the matrix calculation result of the data row F0 and the matrix calculation result of the first-time shift result of the data row F0, and transmits the current accumulation result to the row accumulation register group. The row accumulation register group updates the last-time accumulation result 0 with the current accumulation result. The current accumulation result is the accumulation result of the current first data row (the first-time shift result of the data row F0 in the feature map), and is also the current accumulation result of the data row corresponding to the current first data row (the data row F0 in the feature map), and is also the current calculation result of the sub-feature map (the sub-feature map 2) to which the current first data row belongs.

[0102] The convolution calculation chip processes at least two first data rows corresponding to each data row of the feature map in turn to obtain the accumulation result of each data row, that is, the calculation result of each sub-feature map in the plurality of sub-feature maps, so as to realize the matrix calculation of the feature map in the row direction. That is, the calculation result of a sub-feature map is an intermediate calculation result used for accumulating the convolution calculation result of the sub-feature map. Moreover, the accumulation of the accumulation result of each data row in the plurality of data rows of the feature map by the convolution calculation chip is equivalent to the accumulation of the intermediate calculation result of each sub-feature map in the plurality of sub-feature maps, so as to obtain the convolution calculation result of each sub-feature map, and the accumulation of the convolution calculation result of each sub-feature map in the plurality of sub-feature maps, so as to realize the matrix calculation of the feature map in the column direction.

[0103] In this way, the row sliding register group updates the last-time accumulation result corresponding to the last data row of the feature map, so as to obtain the accumulation result of the convolution calculation result of each sub-feature map in the plurality of sub-feature maps (the accumulation shown in FIG. 2), that is, the convolution calculation result of the feature map. That is, the plurality of calculation results of a sub-feature map are used for accumulating the convolution calculation result of the sub-feature map, and the convolution calculation result of each sub-feature map in the plurality of sub-feature maps is used for accumulating the convolution calculation result of the feature map.

[0104] In addition, the convolution calculation chip can further include an output bus coupled with the row accumulation register group, used for obtaining the convolution calculation result of the feature map in the row accumulation register group and outputting the convolution calculation result of the feature map. The input bus and the output bus in the above-mentioned convolution calculation chip can be the same or different bus systems.

[0105] For the convenience of understanding, the functions of the convolution calculation chip provided in the above embodiments of FIG. 6 and FIG. 7 are specifically described below in combination with FIG. 8 and FIG. 9.

[0106] For example, FIG. 8 is a schematic diagram of a process of 3x3 convolution calculation performed by the convolution calculation chip provided in the embodiments of the present application. As shown in FIG. 8, the feature map buffer is used to store the feature map, the weight buffer is used to store the weights A to I in the 3x3 convolution kernel, the matrix multiplier and the accumulator are used for the multiplication and addition calculation of 3x3 convolution, and the row accumulation register is used to cache and update the multiplication and addition results as intermediate calculation results. For example, FIG. 9 is one of timing diagrams of the row sliding accumulation process in 3x3 convolution provided in the embodiments of the present application. As shown in FIG. 9, the convolution calculation chip provided in the embodiment of FIG. 8 is specifically used for:

[0107] a. The weight buffer transmits the weight in the first row and the first column (i.e., the weight A in the first row and the first column in the 3x3 convolution kernel) to the matrix multiplier.

[0108] b. The feature map buffer transmits the first data row, i.e., the first row data of the feature map, to the row sliding register group buffer. The data row is equivalent to the data row shifted by a step size of 0, and therefore, the first data row is also a first data row.

[0109] c. The row sliding register group divides the first data row into B sub-data rows (each sub-data row contains M data), and respectively outputs the B sub-data rows to the matrix multiplier in continuous B clock periods (the data transmitted in one clock is a sub-data row). The matrix multiplier performs multiplication and addition, i.e., matrix multiplication, on the sub-data row and the weight A, and transmits the multiplication result to the accumulator. The accumulator adds the multiplication result to the last accumulation result (the first accumulation is by default 0, i.e., the accumulation is 0 times) stored in the row accumulation register group at the same storage position as the multiplication result, obtains a new accumulation result, and updates the new accumulation result to the row accumulation register group. The weight A does not change during the entire process of matrix multiplication of the B sub-data rows of the first data row.

[0110] d. The weight buffer updates the weight in the first row and the next column (i.e., the weight B in the first row and the second column in the 3x3 convolution kernel) to the matrix multiplier. The row sliding register group slides one data to the right (i.e., each register transmits the data to the adjacent register on the right), that is, the first data row is shifted for the first time to obtain a first data row corresponding to the first data row.

[0111] e. Perform a process similar to c on the first shift result, except that the specific data processed is different, and weight B is used. Weight B remains unchanged during the execution of c. Perform a process similar to d, except that the updated weight used this time is the weight C at the 1st row, 3rd column position of the 3×3 convolution kernel. Perform a process similar to c on the second shift result obtained from d, except that the specific data processed is different, and weight C is used. Weight C remains unchanged during the execution of c.

[0112] f. The weight buffer transfers the weight of the first column of the next row (i.e., the weight D at the position of the second row and first column in the 3×3 convolution kernel) to the matrix multiplier.

[0113] g. The feature map buffer transfers the second data row, i.e. the next row of the feature map, to the row sliding register group buffer.

[0114] h) Perform a similar process to c-f on the second data row, the difference being the type of data being processed. After processing the second and third data rows, the convolution calculation of the first row of the feature map convolution result is complete. This yields the result of a single-row multiplication of weight A with its corresponding sub-feature map, the result of a single-row multiplication of weight B with its corresponding sub-feature map, and the result of a single-row multiplication of weight C with its corresponding sub-feature map. The final sum of these three single-row multiplication results is then obtained. Therefore, the final sum of a single row corresponding to multiple sub-feature maps (i.e., the sum of the single-row multiplication results of multiple sub-feature maps) is the result of a single-row convolution of the feature map.

[0115] i. Similarly, the convolution calculation for each subsequent row after obtaining the first row's convolution calculation result is similar to the process described above for obtaining the first row's convolution calculation result. The difference lies in the specific data being processed. For the same part, please refer to the process described above for obtaining the first row's convolution calculation result; it will not be repeated here.

[0116] In this embodiment, the first data row entering the matrix multiplier is generated by shifting a row sliding register. Data in the feature map read once (i.e., one row of data) can be reused three times (e.g., for matrix multiplication calculations corresponding to weights A, B, and C). Therefore, the read bandwidth of the feature map buffer is reduced by a factor of three, and the read power consumption is reduced by a factor of three. Furthermore, for any first data row, a weight can be reused B×M times; therefore, the read bandwidth of the weight buffer is reduced by a factor of B×M, and the read power consumption is reduced by a factor of B×M. In the matrix multiplication calculation, the weights can remain unchanged for B consecutive clock cycles, significantly reducing the PPI signal toggle rate in the matrix multiplier; therefore, the computational power consumption of the matrix multiplier is significantly reduced.

[0117] Based on the convolution calculation chip shown in FIG. 5, the embodiment of the present application further provides a convolution calculation method. For example, FIG. 10 is a flowchart of a convolution calculation method provided by the embodiment of the present application. The convolution calculation method can be applied to the convolution calculation chip 500, which can include the feature map cache, the row sliding register group, and the calculation unit shown in FIG. 5. As shown in FIG. 10, the method can include the following steps.

[0118] S1001, the current data row of the feature map in the feature map cache is cached to the row sliding register group.

[0119] S1002, the row sliding register group shifts the data in the current data row to obtain at least two first data rows. The multiple second data rows with the same shift step in the multiple first data rows obtained by shifting the multiple data rows of the feature map are used to form a sub-feature map, and the multiple sub-feature maps correspond to multiple weights respectively.

[0120] S1003, the calculation unit obtains the multiple first data rows from the row sliding register group, and obtains the weight corresponding to the sub-feature map to which each first data row belongs, multiplies and adds each first data row with the weight corresponding to the sub-feature map to which the first data row belongs in the multiple weights respectively, to obtain the current calculation result of the sub-feature map to which the first data row belongs, and the calculation results of the multiple sub-feature maps are used to form the convolution calculation result of the feature map.

[0121] The steps of the embodiment correspond to the functions of the components of the convolution calculation chip in the embodiment of FIG. 6, and the related descriptions in the embodiment of FIG. 6 are referred to here.

[0122] The convolution calculation chip and the convolution calculation method provided by the embodiment of the present application are further described below in combination with FIGS. 11-17.

[0123] In an alternative example, FIG. 11 is a structural block diagram of a convolution calculation chip provided by the embodiment of the present application. As shown in FIG. 11, the convolution calculation chip 500 shown in FIG. 5 further includes multiple row backup register groups 507; the multiple row backup register groups 507 are coupled to the feature map cache respectively, and the multiple row backup register groups 507 are coupled to the row sliding register group respectively.

[0124] The multiple row backup register groups 507 are used to obtain multiple data rows from the feature map cache, wherein each row backup register group is used to cache one data row in the multiple data rows.

[0125] The row sliding register group is coupled to the multiple row backup register groups, and is specifically used to obtain one data row in the multiple data rows from each row backup register group respectively.

[0126] In the embodiment of the present application, the data rows in the feature map are backed up by the plurality of row backup register groups. Thus, the data rows that need to be reused can be directly obtained from the row backup register groups, without the need to repeatedly read from the feature map cache, so that the read bandwidth and power consumption of the feature map cache can be further reduced.

[0127] For example, FIG. 12 is a structural block diagram of a convolution calculation chip provided by an embodiment of the present application. As shown in FIG. 12, the convolution calculation chip further includes a plurality of row backup register groups, for example, n-1 row backup register groups, where n is the number of rows of the convolution kernel, on the basis of the chip shown in FIG. 8. The plurality of row backup register groups can be coupled to the row sliding register group by switches, and the switches are used to control whether the row backup register group coupled to the switch in the plurality of row backup register groups transmits a data row to the row sliding register group. For example, FIG. 13 is a process schematic diagram of a 3x3 convolution calculation of a convolution calculation chip provided by an embodiment of the present application. As shown in FIG. 13, the convolution calculation chip shown in FIG. 12 is illustrated by taking 3x3 convolution as an example. At this time, the row backup register group is 2 and can transmit a data row into the row sliding register group by selecting the switch. The timing diagram of the row sliding accumulation process is the same as that shown in FIG. 9, and reference can be made to the description of the embodiment of FIG. 9, which will not be repeated here.

[0128] In combination with FIG. 13, the convolution calculation chip shown in FIG. 12 is specifically used for:

[0129] a. The weight cache transmits the weight of the first row and the first column (that is, the weight A at the position of the first row and the first column in the 3x3 convolution kernel) to the matrix multiplier.

[0130] b. The feature map cache transmits the first data row, that is, the first row data of the feature map, to the row sliding register group cache.

[0131] c. The row sliding register group divides the first data row into B sub-data rows (each sub-data row contains M data), and outputs the B sub-data rows to the matrix multiplier in the next B clock cycles (the data transmitted in one clock is a sub-data row). The matrix multiplier performs multiplication and addition, that is, matrix multiplication operation, on the sub-data row and the weight A, and transmits the multiplication result to the accumulator. The accumulator adds the multiplication result to the last accumulation result (the first accumulation is 0, that is, the accumulation is 0 times) stored in the row accumulation register group at the same storage position as the multiplication result, obtains a new accumulation result, and updates the new accumulation result to the row accumulation register group. The weight A does not change during the entire process of matrix multiplication operation of the B sub-data rows of the first data row.

[0132] d. The weight buffer updates the first column of the next row of weights (i.e. the weight B in the 3x3 convolution kernel at the position of the first column of the second row) to the matrix multiplier. The row sliding register bank slides one data to the right (i.e. each register passes the data to the right adjacent register), that is, the first data row is shifted for the first time.

[0133] e. The similar process as c is performed, except that the data processed is different, and the weight B is used, during which the weight B is unchanged. The similar process as d is performed, except that the updated weight used this time is the weight C in the 3x3 convolution kernel at the position of the first column of the third row. The similar process as c is performed, except that the data processed is different, and the weight C is used, during which the weight C is unchanged.

[0134] f. The weight buffer transmits the first column of the next row of weights (i.e. the weight D in the 3x3 convolution kernel at the position of the first column of the second row) to the matrix multiplier.

[0135] g. The feature map buffer transmits the second data row, i.e. the next row of data of the feature map, to the row sliding register bank.

[0136] h. The similar process as c-f is performed, except that the data processed is different. After the second and third data rows are processed, the convolution calculation for obtaining the convolution calculation result of the first row of the feature map is completed. That is, the multiplication calculation result of the weight A and the corresponding sub-feature map, the multiplication calculation result of the weight B and the corresponding sub-feature map, the multiplication calculation result of the weight C and the corresponding sub-feature map are obtained, and the accumulation result of these multiplication calculation results, i.e. the final accumulation result of the corresponding sub-feature map, is obtained.

[0137] i. The convolution calculation process for obtaining the convolution calculation result of the first row of the feature map is similar to the above-mentioned convolution calculation process for obtaining the convolution calculation result of the first row of the feature map, except that in the process b, the row backup register bank also obtains the data row not backed up from the feature map buffer for backup, so that the data row already backed up in the row backup register bank is no longer repeatedly obtained from the feature map buffer. For the same part, please refer to the above-mentioned convolution calculation process of the first row of weights, which will not be repeated here.

[0138] In this embodiment, on the basis of the embodiment of FIG. 8, the reused data rows, for example, the second data row and the third data row, are cached in the row backup register group, so as to be used for the matrix multiplication calculation with the second weight and the matrix multiplication calculation with the third row weight, that is, one data row can be used for 3 times in each row weight matrix multiplication calculation, and one data row can be used for 3 row weight matrix multiplication calculations through the row backup register group. Therefore, the data row obtained through one reading of the feature map cache can be reused 9 times, the reading bandwidth is reduced by 9 times, and the reading power consumption is reduced by 9 times.

[0139] In an example, FIG. 14 is one of the structural block diagrams of the convolution calculation chip provided in the embodiments of the present application. As shown in FIG. 14, compared with the convolution calculation chip shown in FIG. 12, the number of the row backup register group in the convolution calculation chip in this embodiment is consistent with the number of rows of the convolution kernel, which is n. At this time, the row sliding register group can no longer obtain the data row of the feature map from the feature map cache.

[0140] In the embodiments of the present application, the number of the row backup register group is the same as the number of rows of the convolution kernel, or is the same as the number of rows of the convolution kernel after being combined with the row sliding register group, so as to ensure that the reduction of the reading and writing times of the feature map cache is more suitable for the convolution calculation, and further improve the performance of the convolution calculation chip. In addition, when the number of the row backup register group is less than the number of rows of the convolution kernel, the data row can be obtained through the row sliding register group, so as to reduce the number of the row backup register group, and realize the effect of reducing the cost and performance of the convolution calculation chip.

[0141] In an optional implementation, the row sliding register group is specifically used for:

[0142] respectively shifting the data in the plurality of data rows by the same shift step to obtain a third data row corresponding to each data row in the plurality of data rows, the third data row corresponding to the data row being one of the at least two first data rows corresponding to the data row;

[0143] Correspondingly, the calculation unit in the convolution calculation chip specifically includes a multiplication-addition unit and a plurality of row accumulation register groups; the multiplication-addition unit and the row sliding register group are coupled, and the plurality of row accumulation register groups are respectively coupled with the convolution unit.

[0144] The multiplication-addition unit is configured to obtain a current third data row of each data row in the plurality of data rows from the row sliding register group, obtain a weight corresponding to a current sub-feature map, and perform multiplication-addition operation on the current third data row, the weight corresponding to the current sub-feature map, and a previous one-row accumulation result corresponding to the plurality of sub-feature maps obtained from the row accumulation register group, to obtain a current one-row accumulation result corresponding to the plurality of sub-feature maps, wherein the current sub-feature map is a sub-feature map to which the current third data row belongs, the one-row accumulation result corresponding to the plurality of sub-feature maps corresponds to one-row convolution calculation result of the feature map, and a final accumulation result corresponding to the plurality of sub-feature maps is a convolution calculation result of the feature map.

[0145] Each row accumulation register group in the plurality of row accumulation register groups is configured to obtain the current one-row accumulation result corresponding to the plurality of sub-feature maps from the multiplication-addition unit, and update a previous one-row accumulation result corresponding to the plurality of sub-feature maps corresponding to the one-row convolution calculation result of the feature map by using the current one-row accumulation result corresponding to the plurality of sub-feature maps corresponding to the one-row convolution calculation result of the feature map.

[0146] For example, FIG. 15 is a structural block diagram of a convolution calculation chip according to an embodiment of the present application. As shown in FIG. 15, the convolution calculation chip can further include a plurality of, for example, n row accumulation register groups, which can cache accumulation results of n data rows, that is, intermediate accumulation results corresponding to sub-feature maps to which respective first data rows corresponding to the n data rows belong, and output the intermediate accumulation results to the accumulator and the new multiplication calculation result output by the matrix multiplier in the next clock cycle, so as to add the intermediate accumulation results to the new multiplication calculation result output by the matrix multiplier, thereby obtaining a new accumulation result. This process is repeated until a final accumulation result is obtained. Here, n is the number of rows of the convolution kernel. The multiplication-addition unit can specifically include a matrix multiplier coupled to the row sliding register group and the weight cache, respectively, and an accumulator coupled to the plurality of row accumulation register groups and the matrix multiplier, respectively.

[0147] For ease of understanding, the convolution calculation chip shown in FIG. 15 is specifically described below by taking 3x3 convolution as an example. For example, FIG. 16 is a process diagram of a convolution calculation chip performing 3x3 convolution calculation according to an embodiment of the present application. As shown in FIG. 16, since the convolution kernel has three rows, that is, three-row convolution needs to be calculated, and there are three rows of convolution calculation results, three row backup register groups and three row accumulation register groups can be added to the embodiment of FIG. 8. For example, FIG. 17 is a timing diagram of a row sliding accumulation process in 3x3 convolution according to an embodiment of the present application. In combination with FIG. 17 and FIG. 16, the convolution calculation chip according to an embodiment of the present application can be specifically used to:

[0148] a. The weight cache transmits the weight of the first row and the first column (i.e., the weight A in the first row and the first column of the 3x3 convolution kernel) to the matrix multiplier.

[0149] b1. The feature map cache transmits the three data rows of the feature map (i.e., the first three rows of data of the feature map, the first data row F0, the second data row F1, and the third data row F2) to the three row backup register banks.

[0150] b2. The row sliding register bank sequentially obtains the three data rows of the feature map from the row backup register bank, caches the obtained data rows, and performs process c on each of the three data rows.

[0151] c. The row sliding register bank outputs B sub-data rows corresponding to the cached data rows (one sub-data row per clock cycle) to the matrix multiplier for multiplication and addition with the weight A (i.e., matrix multiplication operation), and transmits the multiplication result to the accumulator. The accumulator adds the multiplication result to the last accumulation result (the first accumulation is 0 by default, i.e., 0 times of accumulation) stored in the same storage location as the multiplication result in the row accumulation register bank, obtains a new accumulation result, and sequentially updates the new accumulation results of the three data rows to the row accumulation register bank 1, the row accumulation register bank 2, and the row accumulation register bank 3. The matrix multiplier keeps the weight A unchanged during the entire matrix multiplication operation of the first data row F0, the second data row F1, and the third data row F2 with the weight A.

[0152] d. The weight cache updates the weight of the first row and the next column (i.e., the weight B in the first row and the second column of the 3x3 convolution kernel) to the matrix multiplier.

[0153] e1. The three data rows of the feature map are sequentially processed as follows: loaded from the row backup register bank to the row sliding register bank, and the row sliding register bank slides one data to the right (i.e., each register passes the data to the adjacent register on the right), i.e., the first shift of each data row of the three data rows. Similar processing as step c is performed on the shifted result, except that the processed data is different.

[0154] e2. The weight cache updates the next column weight (i.e., the weight C in the first row and the third column of the 3x3 convolution kernel) to the matrix multiplier.

[0155] e3. The three data rows of the feature map are sequentially processed as follows: loaded from the row backup register bank to the row sliding register bank, and the row sliding register bank slides one data to the right (i.e., each register passes the data to the adjacent register on the right), i.e., the second shift of each data row of the three data rows. Similar processing as step c is performed on the shifted result, except that the processed data is different.

[0156] f、the weight buffer transmits the weight of the first column of the next row (i.e. the weight D in the 2nd row and 1st column of the 3x3 convolution kernel) to the matrix multiplier.

[0157] g、after the c-f processes for the second and third data rows are completed, the convolution calculation for obtaining the convolution calculation result of the first row of the feature map is completed. That is, the multiplication calculation result of the weight A and the corresponding sub-feature map, the multiplication calculation result of the weight B and the corresponding sub-feature map, the multiplication calculation result of the weight C and the corresponding sub-feature map are obtained, and the accumulation result of these multiplication calculation results, i.e. the final one row accumulation result corresponding to these sub-feature maps, is obtained. That is, the first row convolution calculation result of the feature map is buffered in the row accumulation register group 1.

[0158] And, since three row accumulation register groups are used, after the c-f processes for the second and third data rows are completed, the intermediate accumulation result for obtaining the second row convolution calculation result of the feature map is buffered in the row accumulation register group 2, and the intermediate accumulation result for obtaining the third row convolution calculation result of the feature map is buffered in the row accumulation register group 3.

[0159] h、the feature map buffer transmits the fourth data row of the feature map, i.e. the fourth row data of the feature map, to the row sliding register group, and updates the row backup register group to cover the first data row of the feature map which is not used subsequently.

[0160] i、the similar processes as c-g are performed for the second to fourth data rows, with the difference being the specific data processed. After the c-f processes for the second to fourth data rows are completed, the convolution calculation for obtaining the second row convolution calculation result of the feature map is completed.

[0161] j、the convolution calculation process for obtaining the convolution calculation result after the first row convolution calculation result of the feature map, e.g. the first convolution calculation result of the feature map, is similar to the convolution calculation process for obtaining the first row convolution calculation result of the feature map described above, with the difference being that the data row already buffered in the row backup register group in the b1 process is not loaded from the feature map buffer to the row backup register group again. For the same part, please refer to the convolution calculation process for obtaining the first row convolution calculation result of the feature map described above, which will not be described here.

[0162] In the embodiments of this application, on the basis of the embodiments of FIG. 12 and FIG. 14, the accumulation results of three data rows are cached by a plurality of row accumulation register groups, for example, three row accumulation register groups, wherein the weight can be reused BxM times when the accumulation result of one data row is obtained, and in the process of obtaining the accumulation results of three data rows, each data row corresponds to a second sub-data row using the same weight, so that one weight can be reused 3xBxM times. In this way, the read bandwidth and power consumption of the weight cache can be reduced by 3xBxM times, one weight can remain unchanged within 3xB consecutive clock cycles, and the signal flipping rate of the PPI in the matrix multiplier is greatly reduced, thereby greatly reducing the calculation power consumption of the matrix multiplier.

[0163] For example, the following takes an 8K MAC (multiply accumulate) as an example, that is, the number of input channels Ci=32, the number of output channels Co=16, the number of data transmitted per clock cycle M=16, and the number of data blocks contained in the to-be-convolved matrix B=8, and compares the AI core applying the convolution calculation chip scheme provided by the embodiments of this application with the traditional AI core applying the traditional convolution calculation chip scheme, as shown in the following Table 1:

[0164] Table 1 Comparison of application effects of the convolution calculation chip provided by the embodiments of this application and the traditional convolution calculation chip

[0165] In combination with the comparison shown in Table 1, compared with the AI core applying the traditional convolution calculation chip scheme, the area of the register group in the AI core applying the convolution calculation chip provided by the embodiments of this application increases by 0.013 mm^2, but the area of the feature map cache decreases by 0.021 mm^2, the total area slightly decreases, and the power consumption is greatly reduced.

[0166] It should be understood that the convolution calculation chip shown in the structural block diagrams of the embodiments of this application is only an example of the convolution calculation chip, and the convolution calculation chip can have more or fewer components than those shown in the figures, can combine two or more components, or can have a different component configuration. The various components shown in the structural block diagrams of the embodiments of this application can be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application specific integrated circuits.

[0167] In addition, the embodiments of this application also provide a device, which can be a chip, a component or a module. The device can include a connected processor and a memory; wherein the memory is used to store computer execution instructions, and when the device is running, the processor can execute the computer execution instructions stored in the memory to enable the chip to perform the convolution calculation method in each method embodiment described above.

[0168] Through the description of the above embodiments, those skilled in the art can understand that, for the convenience and brevity of description, only the division of the above functional modules is exemplified, and in actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above.

[0169] The embodiment of the application further provides a computer storage medium, which stores computer instructions, and when the computer instructions run on an electronic device, the electronic device executes the related method steps to realize the data processing method in the above embodiment.

[0170] The embodiment of the application further provides a computer program product, which, when running on a computer, causes the computer to execute the related steps to realize the data processing method in the above embodiment.

[0171] In several embodiments provided in the application, it should be understood that the disclosed system, i.e., device and method, can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the modules or units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another device, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interface, device or unit, and can be electrical, mechanical or other forms.

[0172] The units described as separate components can or can not be physically separate, and the components shown as units can be one physical unit or multiple physical units, i.e., can be located in one place or can be distributed to multiple different places. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0173] In addition, each functional unit in each embodiment of the application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0174] Any content of each embodiment of the application and any content of the same embodiment can be freely combined. Any combination of the above is within the scope of the application.

[0175] If the integrated unit is implemented in the form of a software function unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on such understanding, the technical solutions of the embodiments of the present application essentially or say the parts that make contributions to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product, which is stored in a storage medium, includes several instructions to make a device (which can be a single-chip microcomputer, a chip, etc.) or a processor execute all or part of the steps of the various embodiments of the method of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various storage media that can store program codes.

[0176] The embodiments of the present application are described above in combination with the drawings, but the present application is not limited to the above-described specific embodiments, and the above-described specific embodiments are only illustrative, not restrictive. Those skilled in the art can make many forms under the inspiration of the present application without departing from the scope of the present application and the scope protected by the claims.

[0177] The steps of the method or algorithm described in combination with the disclosure of the embodiments of the present application can be implemented in hardware or by a processor executing software instructions. The software instructions can include corresponding software modules, which can be stored in a random access memory (RAM), a flash memory, a read-only memory (ROM), an erasable programmable ROM (EPROM), an electrically EPROM (EEPROM), a register, a hard disk, a mobile hard disk, a compact disc (CD-ROM), or any other form of storage medium well known in the art. An exemplary storage medium is coupled to the processor, so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be an integral part of the processor. The processor and the storage medium can be located in an application specific integrated circuit (ASIC). In addition, the ASIC can be located in an electronic device. Of course, the processor and the storage medium can also exist as discrete components in the electronic device.

[0178] Those skilled in the art can understand that the functions described in the embodiments of the present application in the one or more examples above can be implemented in hardware, software, firmware or any combination thereof. When implemented in software, the functions can be stored in a computer readable medium or transmitted as one or more instructions or codes on a computer readable medium. The computer readable medium includes computer storage medium and communication medium, wherein the communication medium includes any medium that facilitates the transfer of computer program from one place to another. The storage medium can be any available medium that can be accessed by a general or special purpose computer.

[0179] The embodiments of the present application are described above in conjunction with the drawings, but the present application is not limited to the specific embodiments described above, which are merely illustrative rather than restrictive, and those of ordinary skill in the art can make many forms under the inspiration of the present application without departing from the purpose of the present application and the scope protected by the claims.

Claims

1. A convolutional computing chip, characterized in that, include: Feature map cache is used to cache feature maps; A row sliding register group, coupled to the feature map cache, is used to obtain the current data row in the feature map, shift the data in the current data row to obtain at least two first data rows, and multiple second data rows with the same shift step size among the multiple first data rows obtained by shifting multiple data rows of the feature map are used to form a sub-feature map, and multiple sub-feature maps correspond to multiple weights respectively. The calculation unit, coupled to the row sliding register group, is used to obtain the plurality of first data rows from the row sliding register group, and to obtain the weight corresponding to the sub-feature map to which each of the plurality of first data rows belongs. The unit performs a multiplication and addition operation on each of the plurality of weights and the weight corresponding to the sub-feature map to which the first data row belongs, respectively, to obtain the current calculation result of the sub-feature map to which the first data row belongs. The calculation results of the plurality of sub-feature maps are used to form the convolution calculation result of the feature map.

2. The chip according to claim 1, characterized in that, The chip also includes: A weight cache, coupled to the computing unit, is used to cache at least one of the plurality of weights; The row sliding register group is specifically used to divide the first data row into multiple sub-data rows for each first data row corresponding to the current data row; The computing unit is specifically used to acquire multiple sub-data rows of the first data row corresponding to the current data row from the row sliding register group in multiple clock cycles, and to acquire weights from the weight cache corresponding to the sub-feature map to which the first data row belongs.

3. The chip according to claim 1 or 2, characterized in that, The chip also includes: Multiple row backup register groups are coupled to the feature map cache respectively, and are used to obtain the multiple data rows from the feature map cache, wherein each row backup register group is used to cache one data row among the multiple data rows; The row sliding register group, coupled to the plurality of row backup register groups, is specifically used to obtain one data row from the plurality of data rows from each of the row backup register groups.

4. The chip according to claim 3, characterized in that, The row sliding register group is specifically used for: Each of the plurality of data rows is shifted by the same step size to obtain a third data row corresponding to each of the plurality of data rows. The third data row corresponding to the data row is one of at least two first data rows corresponding to the data row. The computing unit includes: The multiply-accumulate unit, coupled to the row sliding register group, is used to obtain the current third data row of each data row in the plurality of data rows from the row sliding register group, and obtain the weight corresponding to the current sub-feature map. Based on the current third data row, the weight corresponding to the current sub-feature map, and the existing row accumulation result of the plurality of sub-feature maps obtained from the row accumulation register group, a multiply-accumulate operation is performed to obtain the current row accumulation result of the plurality of sub-feature maps. The current sub-feature map is the sub-feature map to which the current third data row belongs. The row accumulation result of the plurality of sub-feature maps corresponds to the row convolution calculation result of the feature map. The final accumulation result of the plurality of sub-feature maps is the convolution calculation result of the feature map. Multiple row accumulation register groups are coupled to the multiply-accumulate unit, wherein each row accumulation register group is used to obtain the current row accumulation result corresponding to multiple sub-feature maps corresponding to the convolution calculation result of the row of the feature map from the multiply-accumulate unit, and use the current row accumulation result corresponding to the multiple sub-feature maps corresponding to the convolution calculation result of the row of the feature map to update the existing row accumulation result corresponding to the multiple sub-feature maps corresponding to the convolution calculation result of the row of the feature map.

5. A convolution calculation method, characterized in that, The method, applied to a convolutional computing chip, includes: Cache feature map; Obtain the current data row in the feature map, shift the data in the current data row to obtain at least two first data rows, and use multiple second data rows with the same shift step size among the multiple first data rows obtained by shifting multiple data rows in the feature map to form a sub-feature map, and the multiple sub-feature maps correspond to multiple weights respectively; Obtain the plurality of first data rows, and obtain the weight corresponding to the sub-feature map to which each first data row belongs. Perform multiplication and addition operations on each first data row and the weight corresponding to the sub-feature map to which the first data row belongs to obtain the current calculation result of the sub-feature map to which the first data row belongs. The calculation results of the plurality of sub-feature maps are used to form the convolution calculation result of the feature map.

6. The method according to claim 5, characterized in that, Before obtaining the weight corresponding to the sub-feature map to which each of the plurality of first data rows belongs, the method further includes: Cache at least one of the multiple weights; After shifting the data in the current data row to obtain at least two first data rows, the method further includes: For each first data row corresponding to the current data row, the first data row is divided into multiple sub-data rows; The step of obtaining the plurality of first data rows and obtaining the weight corresponding to the sub-feature map to which each of the plurality of first data rows belongs includes: In multiple clock cycles, multiple sub-data rows corresponding to the first data row of the current data row are obtained, and the weights corresponding to the sub-feature maps to which the first data row belongs are obtained.

7. The method according to claim 5 or 6, characterized in that, The step of obtaining the current data row in the feature map includes: Obtain the plurality of data rows from the cached feature map, and cache one data row from each of the plurality of data rows; One data row from the plurality of data rows is obtained to obtain the current data row in the feature map.

8. The method according to claim 7, characterized in that, The step of obtaining the current data row in the feature map and shifting the data in the current data row to obtain at least two first data rows includes: Each of the plurality of data rows is shifted by the same step size to obtain a third data row corresponding to each of the plurality of data rows. The third data row corresponding to the data row is one of at least two first data rows corresponding to the data row. Obtain the current third data row for each of the multiple data rows, and obtain the weight corresponding to the current sub-feature map; Based on the current third data row, the weight corresponding to the current sub-feature map, and the existing row accumulation result of the multiple sub-feature maps obtained from the row accumulation register group, a multiplication and addition operation is performed to obtain the current row accumulation result of the multiple sub-feature maps. The current sub-feature map is the sub-feature map to which the current third data row belongs. The row accumulation result of the multiple sub-feature maps corresponds to the row convolution calculation result of the feature map. The final accumulation result of the multiple sub-feature maps is the convolution calculation result of the feature map. Obtain the current row accumulation result corresponding to the multiple sub-feature maps corresponding to the convolution calculation result of the first row of the feature map, and use the current row accumulation result corresponding to the multiple sub-feature maps corresponding to the convolution calculation result of the first row of the feature map to update the existing row accumulation result corresponding to the multiple sub-feature maps corresponding to the convolution calculation result of the first row of the feature map.

9. A computer-readable storage medium, characterized in that, The method includes a computer program, characterized in that, when the computer program is run on an electronic device, it causes the electronic device to perform the method as described in any one of claims 5 to 8.

10. A computer program product, characterized in that, The method includes a computer program that, when executed by an electronic device, causes the electronic device to perform the method described in any one of claims 5 to 8.

Images