Convolution computing chip and method

Abstract

Embodiments of the present application relates to the technical field of integrated circuits, and provide a convolution computing chip and a method. The convolution computing chip comprises: a feature map cache, used for caching a feature map; a row sliding register group, used for acquiring a current data row in the feature map and shifting data therein to obtain at least two first data rows, a plurality of second data rows having the same shift step among a plurality of first data rows corresponding to the feature map forming a sub-feature map, and a plurality of sub-feature maps respectively corresponding to a plurality of weights; and a computing unit, used for acquiring the plurality of first data rows and a weight corresponding to a sub-feature map to which each first data row belongs, and respectively performing a multiply-add operation on each first data row and the weight corresponding to the sub-feature map to which the first data row belongs among the plurality of weights to obtain a current computing result of the sub-feature map to which the first data row belongs, computing results of the plurality of sub-feature maps forming a convolution computing result of the feature map. The convolution computing chip provided in the present application can improve the performance of convolution computing chips.

Description

Convolutional computing chips and methods

[0001] This application claims priority to Chinese Patent Application No. 202410829682.0, filed on June 24, 2024, entitled "Convolution Computation Chip and Method", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of integrated circuit technology, and in particular to a convolution calculation chip and method. Background Technology

[0003] In neural networks within the field of artificial intelligence (AI), convolution refers to the operation of convolving a feature map (FM) with a feature matrix, i.e., a convolutional kernel. The elements in the convolutional kernel are weights, and the feature map can be either the input or output matrix of the neural network. Specifically, the convolutional kernel slides multiple times across the feature map, with each slide corresponding to a different position. After each slide, the kernel multiplies and sums the data at the corresponding position in the feature map, i.e., a multiplication-addition operation is performed, until all the multiplications and additions corresponding to the multiple slides are completed. The final convolution result forms a matrix, thus achieving feature extraction from the feature map.

[0004] In related technologies, convolutional computation can be implemented using convolutional computation chips such as neural network processing units (NPUs), graphics processing units (GPUs), and convolutional computation modules. Convolutional computation chips typically include a buffer, a cube, an accumulator, and a register set. Taking a 3×3 convolutional kernel as an example: when the convolutional kernel slides multiple times on the feature map, the data corresponding to the nine weights in the kernel can be considered as nine different sub-feature maps. The convolutional computation chip stores these nine sub-feature maps in different regions of the buffer so that it reads data with consecutive storage addresses from each sub-feature map in each iteration. The matrix multiplier performs a multiplication-addition operation (matrix multiplication) on the read sub-feature map data and the corresponding weights in the buffer. The accumulator then adds the multiplication-addition result to the accumulated result stored in the register set from the previous iteration (initially accumulating to 0), and updates the accumulated result in the register set for the next iteration. In this way, by reading the sub-feature maps corresponding to the nine weights nine times and performing the corresponding nine accumulations, the convolution of the feature map can be completed and the convolution result can be output.

[0005] However, when performing convolutional computation, the aforementioned convolutional computing chip needs to perform a large number of read and write operations on the cache to obtain multiple sub-feature maps corresponding to multiple weights. Consequently, the read and write bandwidth of the cache and the power consumption of data transmission are very large, especially in scenarios where the neural network has multiple convolutional kernels and / or performs multiple convolutions, which can easily reduce the performance of the convolutional computing chip. Summary of the Invention

[0006] To address the aforementioned technical problems, this application provides a convolutional computation chip and method. This convolutional computation chip uses a row sliding register group to shift the current data row of a feature map, resulting in at least two first data rows belonging to different sub-feature maps. These rows can participate in multiplication and addition operations with different weights. Thus, a single read result from the feature map cache can be used for multiple computations, significantly reducing the read / write bandwidth and data transmission power consumption of the feature map cache, thereby improving the performance of the convolutional computation chip.

[0007] In a first aspect, embodiments of this application provide a convolution calculation chip, comprising: a feature map cache for caching feature maps; a row sliding register group coupled to the feature map cache for obtaining the current data row in the feature map, shifting the data in the current data row to obtain at least two first data rows, and multiple second data rows with the same shift step size among the multiple first data rows obtained by shifting multiple data rows of the feature map to form a sub-feature map, wherein the multiple sub-feature maps correspond to multiple weights respectively; and a calculation unit coupled to the row sliding register group for obtaining multiple first data rows from the row sliding register group, obtaining the weight corresponding to the sub-feature map to which each first data row belongs, performing multiplication and addition operations on each first data row and the weight corresponding to the sub-feature map to which the first data row belongs to obtain the current calculation result of the sub-feature map to which the first data row belongs, wherein the calculation results of the multiple sub-feature maps are used to form the convolution calculation result of the feature map.

[0008] In this embodiment, the row sliding register group in the convolutional computation chip obtains the current data row of the feature map in the feature map cache, shifts the data in the current data row, and obtains at least two first data rows. Based on this, each data row in the multiple data rows of the feature map can be shifted to obtain at least two first data rows, thus forming multiple first data rows. Multiple second data rows with the same shift step size among these multiple first data rows are used to form a sub-feature map. Therefore, different shifts of multiple data rows in the feature map can form multiple sub-feature maps, and each sub-feature map corresponds to multiple weights. Accordingly, each of the multiple first data rows can be used to perform multiplication and addition operations with the weights corresponding to the sub-feature map to which it belongs to obtain the current calculation result of the sub-feature map. Thus, the calculation results of the multiple sub-feature maps form the convolutional computation result of the feature map. In this way, the convolutional computation chip can perform multiple operations corresponding to multiple weights based on a single read / write result of the feature map in the feature map cache, thereby significantly reducing the number of read / write operations of the feature map cache, which in turn significantly reduces the read / write bandwidth and data transmission power consumption of the feature map cache, improving the performance of the convolutional computation chip.

[0009] According to the first aspect, the chip further includes: a weight cache, coupled to the computing unit, for caching at least one of a plurality of weights; a row sliding register group, specifically for dividing a first data row into a plurality of sub-data rows for each first data row corresponding to the current data row; and a computing unit, specifically for acquiring a plurality of sub-data rows of the first data row corresponding to the current data row from the row sliding register group in a plurality of clock cycles, and acquiring weights from the weight cache corresponding to the sub-feature map to which the first data row belongs.

[0010] In this embodiment, the row sliding register group divides each first data row corresponding to the current data row into multiple sub-data rows. This ensures that the computing unit uses the weight corresponding to one first data row to perform multiplication and addition operations on multiple sub-data rows over multiple clock cycles. In other words, a weight can remain unchanged for multiple consecutive clock cycles for the corresponding multiplication and addition operations. Thus, a weight can be reused multiple times, and consequently, the read bandwidth and power consumption of the weight buffer can be significantly reduced. Furthermore, this reduces the frequency of weight changes, which in turn significantly reduces the inversion rate of the input signal in the computing unit, thereby significantly reducing the power consumption of the matrix multiplier.

[0011] According to the first aspect, or any implementation of the first aspect above, the chip further includes: multiple row backup register groups, each coupled to a feature map cache, for obtaining multiple data rows from the feature map cache, wherein each row backup register group is used to cache one data row from the multiple data rows; and a row sliding register group, coupled to the multiple row backup register groups, specifically for obtaining one data row from each row backup register group from the multiple data rows.

[0012] In this embodiment of the application, data rows in the feature map are backed up by multiple row backup register groups. In this way, data rows that need to be reused can be obtained directly from the row backup register groups without having to be read repeatedly from the feature map cache, thereby further reducing the read bandwidth and power consumption of the feature map cache.

[0013] According to the first aspect, or any implementation of the first aspect above, the row sliding register group is specifically used for: shifting the data in multiple data rows by the same shift step size to obtain the third data row corresponding to each data row in the multiple data rows, wherein the third data row corresponding to the data row is one of at least two first data rows corresponding to the data row; the calculation unit includes: a multiply-accumulate unit coupled to the row sliding register group, used for obtaining the current third data row of each data row in the multiple data rows from the row sliding register group, obtaining the weight corresponding to the current sub-feature map, and performing multiply-accumulate operations based on the current third data row, the weight corresponding to the current sub-feature map, and the existing row accumulation result of the multiple sub-feature maps obtained from the row accumulation register group to obtain multiple sub-feature maps. The corresponding current row accumulation result, where the current sub-feature map is the sub-feature map to which the current third data row belongs, the row accumulation result corresponding to multiple sub-feature maps corresponds to the row convolution calculation result of the feature map, and the final accumulation result corresponding to multiple sub-feature maps is the convolution calculation result of the feature map; multiple row accumulation register groups are coupled to the multiply-accumulate unit, where each row accumulation register group is used to obtain the current row accumulation result corresponding to multiple sub-feature maps corresponding to the row convolution calculation result of the feature map from the multiply-accumulate unit, and use the current row accumulation result corresponding to multiple sub-feature maps corresponding to the row convolution calculation result of the feature map to update the existing row accumulation result corresponding to multiple sub-feature maps corresponding to the row convolution calculation result of the feature map.

[0014] In this embodiment, multiple row backup register groups are combined, and the accumulation results of multiple data rows are cached separately by multiple row accumulation register groups. During the process of obtaining the accumulation results of multiple data rows, each data row corresponds to a second sub-data row using the same weight. Therefore, a weight can be reused multiple times. In this way, the read bandwidth and power consumption of the weight cache can be further reduced. A weight can remain unchanged during the calculation time of multiple data rows, which significantly reduces the signal flip rate of the matrix multiplier, thereby greatly reducing the computational power consumption of the matrix multiplier.

[0015] Secondly, embodiments of this application provide a convolution calculation method applied to a convolution calculation chip. The method includes: caching a feature map; obtaining a current data row in the feature map; shifting the data in the current data row to obtain at least two first data rows; multiple second data rows with the same shift step size among the multiple first data rows obtained by shifting multiple data rows of the feature map are used to form a sub-feature map; the multiple sub-feature maps correspond to multiple weights respectively; obtaining multiple first data rows and obtaining the weight corresponding to the sub-feature map to which each first data row belongs; performing multiplication and addition operations on each first data row and the weight corresponding to the sub-feature map to which the first data row belongs to obtain the current calculation result of the sub-feature map to which the first data row belongs; the calculation results of the multiple sub-feature maps are used to form the convolution calculation result of the feature map.

[0016] According to the second aspect, before obtaining the weight corresponding to the sub-feature map to which each of the multiple first data rows belongs, the method further includes: caching at least one of the multiple weights; after shifting the data in the current data row to obtain at least two first data rows, the method further includes: for each first data row corresponding to the current data row, dividing the first data row into multiple sub-data rows; obtaining the multiple first data rows, and obtaining the weight corresponding to the sub-feature map to which each of the multiple first data rows belongs, including: obtaining multiple sub-data rows of the first data row corresponding to the current data row in multiple clock cycles, and obtaining the weight corresponding to the sub-feature map to which the first data row belongs.

[0017] According to the second aspect, or any implementation of the second aspect above, obtaining the current data row in the feature map includes: obtaining multiple data rows from the cached feature map, and caching one data row from each of the multiple data rows; obtaining one data row from each of the multiple data rows to obtain the current data row in the feature map.

[0018] According to the second aspect, or any implementation of the second aspect above, the current data row in the feature map is obtained, and the data in the current data row is shifted to obtain at least two first data rows, including: shifting the data in multiple data rows by the same shift step size to obtain a third data row corresponding to each data row in the multiple data rows, wherein the third data row corresponding to the data row is one of at least two first data rows corresponding to the data row; obtaining the current third data row of each data row in the multiple data rows, and obtaining the weight corresponding to the current sub-feature map; and accumulating the data based on the current third data row, the weight corresponding to the current sub-feature map, and the existing row corresponding to the multiple sub-feature maps obtained from the row accumulation register group. The results are multiplied and added to obtain the current row's accumulated result corresponding to multiple sub-feature maps. The current sub-feature map is the sub-feature map to which the current third data row belongs. The row's accumulated result corresponding to multiple sub-feature maps corresponds to the row's convolution calculation result of the feature map. The final accumulated result corresponding to multiple sub-feature maps is the convolution calculation result of the feature map. The current row's accumulated result corresponding to multiple sub-feature maps corresponding to the row's convolution calculation result of the feature map is obtained respectively. The current row's accumulated result corresponding to multiple sub-feature maps corresponding to the row's convolution calculation result of the feature map is then used to update the existing row's accumulated result corresponding to multiple sub-feature maps corresponding to the row's convolution calculation result of the feature map.

[0019] The second aspect and any implementation thereof correspond to the first aspect and any implementation thereof, respectively. The technical effects of the second aspect and any implementation thereof are similar to those of the first aspect and any implementation thereof, and will not be repeated here.

[0020] Thirdly, embodiments of this application provide a computer-readable medium for storing a computer program, the computer program including instructions for performing the methods in the second aspect or any possible implementation of the second aspect.

[0021] Fourthly, embodiments of this application provide a computer program including instructions for performing the method in the second aspect or any possible implementation thereof. Attached Figure Description

[0022] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 is a schematic diagram of the 3x3 convolution calculation process with 16 input channels and 16 output channels;

[0024] Figure 2 is a schematic diagram of the convolution calculation process corresponding to a 3x3 convolution kernel;

[0025] Figure 3 is a block diagram of a convolutional computing chip;

[0026] Figure 4 is a timing example diagram of a convolution calculation chip performing convolution calculations;

[0027] Figure 5 is a structural block diagram of an electronic device using a convolutional computing chip according to an embodiment of this application;

[0028] Figure 6 is one of the structural block diagrams of a convolutional computing chip provided in an embodiment of this application;

[0029] Figure 7 is a timing example diagram of row sliding accumulation provided in an embodiment of this application;

[0030] Figure 8 is one of the schematic diagrams of a convolution calculation chip performing 3x3 convolution calculation according to an embodiment of this application;

[0031] Figure 9 is one of the timing examples of the row sliding accumulation process in a 3x3 convolution provided in the embodiments of this application;

[0032] Figure 10 is a flowchart illustrating a convolution calculation method provided in an embodiment of this application;

[0033] Figure 11 is one of the structural block diagrams of a convolutional computing chip provided in an embodiment of this application;

[0034] Figure 12 is one of the structural block diagrams of a convolutional computing chip provided in an embodiment of this application;

[0035] Figure 13 is one of the schematic diagrams of a convolution calculation chip performing 3x3 convolution calculation according to an embodiment of this application;

[0036] Figure 14 is one of the structural block diagrams of a convolutional computing chip provided in an embodiment of this application;

[0037] Figure 15 is one of the structural block diagrams of a convolutional computing chip provided in an embodiment of this application;

[0038] Figure 16 is one of the schematic diagrams of a convolution calculation chip performing 3x3 convolution calculation according to an embodiment of this application;

[0039] Figure 17 is one of the timing examples of the row sliding accumulation process in 3x3 convolution provided in the embodiments of this application. Detailed Implementation

[0040] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0041] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0042] The terms "first" and "second," etc., used in the specification and claims of this application are used to distinguish different objects, not to describe a specific order of objects. For example, "first target object" and "second target object," etc., are used to distinguish different target objects, not to describe a specific order of target objects.

[0043] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0044] In the description of the embodiments in this application, unless otherwise stated, "multiple" means two or more. For example, multiple processing units means two or more processing units; multiple systems means two or more systems.

[0045] To facilitate understanding of this embodiment, some technical terms and background technologies involved in this embodiment will be introduced first:

[0046] Register: A type of memory with limited high-speed access, directly integrated into the chip. It consists of combinations of NAND gates and is divided into general-purpose registers and special-purpose registers. The chip's core has the fastest speed to access registers.

[0047] Static random-access memory (SRAM): A type of random access memory. The term "static" means that as long as this type of memory is powered on, the data stored within it can be permanently retained.

[0048] Input and Output Channels: A feature map has four dimensions: batch size, channels, width, and height. The number of channels can be one or more. When a convolutional computing chip uses a kernel to perform convolution calculations on a feature map, the data is input to the feature map channel by channel. Therefore, the channels of a feature map can be called input channels. In other words, input channels are the independent channels used to input data in a convolutional neural network, and one input channel can correspond to one dimension of the input data. For example, when processing image data, the number of input channels usually corresponds to the number of color channels in the image; an RGB image has three input channels, while a grayscale image has only one. The number of input channels reflects the dimensionality and complexity of the input data. Correspondingly, a convolutional kernel contains multiple input channels and output channels. The input channels of the convolutional kernel correspond to the input channels of the feature map and are used to perform convolution calculations with the data in the feature map of the corresponding input channels. The output channels of the convolutional kernel correspond to the output channels of the feature map and are used to output the convolution calculation results corresponding to the input channels. In other words, output channels are the independent channels used to output data in a convolutional neural network. The number of output channels can reflect the dimensionality and complexity of the output data.

[0049] With the rapid development of AI technology, convolution computation in neural networks has been widely applied. The feature matrix is ​​convolved with the input matrix to extract a specific feature from the input data. The resulting output matrix serves as the input for the next convolution computation. Thus, AI algorithms can be implemented through multiple convolution computations. The feature matrix is ​​the convolution kernel, containing the weights of each pixel, and the input / output matrix is ​​the feature map. For ease of understanding, Figure 1 is used as an example to illustrate convolution computation below.

[0050] For example, Figure 1 is a schematic diagram of the 3x3 convolution calculation process with 16 input channels and 16 output channels. As shown in Figure 1, in the 16 input channels (0-15) of the feature map, each input channel corresponds to a 3x3 convolution kernel for convolution calculation. In the 16 output channels (0-15), each output channel corresponds to a 3x3 convolution kernel to output the result of the convolution calculation corresponding to that kernel. Specifically, W(CHout,Chin,i,j) represents the weight value at position coordinate (i,j) in the 3x3 convolution kernel corresponding to input channel CHin and output channel CHout; F(Chin,x,y) represents the pixel value of the pixel at coordinate (x,y) in the feature map input to input channel Chin.

[0051] The convolution kernel slides to the right and performs a multiplication-addition operation with the pixel in the feature map corresponding to the sliding position of the kernel, thus outputting F'(CHout,x,y), completing the convolution operation on the feature map. Here, F'(CHout,x,y) is the pixel value of the pixel with coordinates (x,y) and output channel CHout in the output feature map, i.e., the convolution result. In other words, the multiplication-addition operation satisfies the following formula:

[0052] The initial value of n is the coordinates of the first row and first column of the convolution kernel in the feature map as the kernel slides across it; the value of n depends on the size of the feature map.

[0053] Convolutional computing chips typically implement convolutional computation based on the im2col method. The following explanation uses a 3×3 convolutional kernel, 16 input channels, and 16 output channels as an example to illustrate convolutional computation based on the im2col method. For example, Figure 2 is a schematic diagram of the convolutional computation process corresponding to a 3x3 convolutional kernel. As shown in Figure 2, the convolutional computing chip arranges the pixels of the feature map in a column from left to right and from top to bottom, and then expands each pixel into a row according to its channel. This results in 16 feature maps represented in columns, i.e., rows, thus obtaining an input matrix with a width of 16 and a height of (FM_W×(FM_H-2)-2), which is the input feature map shown in Figure 2. Here, a row formed by different pixels of each input channel can be considered as a feature map containing only the dimension reflected by that input channel. For example, the first row and first column of the feature map under the first input channel is F(Chin=0,0,0), the first row and first column of the feature map under the first input channel is F(Chin=1,0,0), ..., the first row and first column of the feature map under the 16th input channel is F(Chin=15,0,0), where FM_W refers to the width of the feature map and FM_H refers to the height of the feature map. Each weight in the convolution kernel is unfolded into a matrix of output channel × input channel. When the convolution kernel slides multiple times on the input feature map, the data corresponding to the 9 weights in the convolution kernel can form 9 sub-feature maps: sub-feature map 1 corresponding to the weights in the weight matrix (i=0,j=0), sub-feature map 2 corresponding to the weights in the weight matrix (i=0,j=1), ..., sub-feature map 4 corresponding to the weights in the weight matrix (i=1,j=0), ..., sub-feature map 9 corresponding to the weights in the weight matrix (i=2,j=2). Each weight in the convolution kernel is multiplied by the corresponding sub-feature map, resulting in nine intermediate matrices. These nine intermediate matrices are then summed to obtain the output matrix, which is the convolution result of the input feature map.

[0054] It is understood that after processing according to the embodiment in Figure 2 above, the feature map contains pixels distributed according to the input channels of the feature map. Convolution processing of the feature map is equivalent to performing convolution calculations on the pixels under each input channel of the feature map. The difference in convolution calculations performed on pixels under different input channels lies in the different dimensions reflected by the data in the calculated feature map. For example, for the three input channels of processing a color image, the image data input to the R channel reflects the red dimension of the color image, the image data input to the G channel reflects the green dimension of the color image, and the image data input to the B channel reflects the green dimension of the color image. Similarly, the data output by different output channels reflects different dimensions of the output data. The number of input channels and the number of output channels can be set according to specific application requirements, and this embodiment does not limit this. The data from each input channel can be expanded into rows and concatenated into a feature map for parallel processing, as shown in the embodiment in Figure 2 above. The output results obtained after parallel processing can be output in parallel through the corresponding output channels. For ease of description and understanding, the following explanations will focus on the case of convolution using a feature map from a single input channel. The hardware structure and functions of the convolution chip will be explained using the case of a single input channel, and will not be repeated here.

[0055] For example, Figure 3 is a structural block diagram of a convolutional computation chip. As shown in Figure 3, in related technologies, a convolutional computation chip includes a feature map buffer (SRAM), a weight buffer (SRAM), an accumulator, and a register group, each coupled to a matrix multiplier (CUBE) to implement the convolutional computation provided in the embodiment of Figure 2. When a 3×3 convolutional kernel slides across the input feature map, the data in different regions corresponding to the nine weights in the convolutional kernel can form nine different sub-feature maps. Based on this, the convolutional computation chip reads and writes the input feature map multiple times through an external bus coupled to the feature map buffer to obtain the nine sub-feature maps corresponding to the nine weights: sub-feature maps ① to ⑨, and caches sub-feature maps ① to ⑨ in different regions of the feature map buffer (FM buffer) to continuously read data from the input feature map. Thus, for example, Figure 4 is a timing example diagram of an NPU performing convolutional computation. As shown in Figure 4, each clock cycle, the NPU reads M data points from one of the nine sub-feature maps in the feature map cache and inputs them into the matrix multiplier (CUBE). It also inputs the weights from the nine weights A to I in the weight cache corresponding to the sub-feature map into the matrix multiplier for multiplication and addition. The multiplication and addition result is added to the accumulation result stored in the register group in the previous cycle (the initial accumulation result is 0) through the accumulator, and the current accumulation result is updated in the register group for accumulation in the next cycle. This process continues until the data of the nine sub-feature maps are read in sequence. After nine accumulations, the final convolution result is obtained, and then the result is output through the bus, which is the convolution result of the input feature map.

[0056] However, the NPU provided in the embodiment of Figure 3 above requires multiple read and write operations on the input feature map to obtain nine sub-feature maps stored in the feature map cache: sub-feature maps ① to ⑨. This results in high read / write bandwidth and high data transmission power consumption for the feature map cache. In addition, as shown in Figure 4, the sub-feature maps and weights input to the matrix multiplier change in each clock cycle, i.e., each time frame. For example, within nine clock cycles, sub-feature maps ① to ⑨ change sequentially, and weights A to I change sequentially. This results in a high flip rate for the smallest computational unit in the matrix multiplier, i.e., the partial product integration (PPI), leading to high power consumption of the matrix multiplier.

[0057] Therefore, this application provides a convolution calculation chip to solve the above problems. The row sliding register group in this chip obtains the current data row of the feature map in the feature map cache, shifts the data in the current data row, and obtains at least two first data rows. Based on this, each data row in the multiple data rows of the feature map can be shifted to obtain at least two first data rows, thus forming multiple first data rows. Multiple second data rows with the same shift step size among these multiple first data rows are used to form a sub-feature map. Thus, different shifts of multiple data rows in the feature map can form multiple sub-feature maps, and each sub-feature map corresponds to multiple weights. Accordingly, each of the multiple first data rows can be used to perform multiplication and addition operations with the weights corresponding to the sub-feature map to which the first data row belongs to obtain the current calculation result of the sub-feature map. Therefore, the calculation results of the multiple sub-feature maps form the convolution calculation result of the feature map. In this way, the convolutional computing chip can perform multiple operations corresponding to multiple weights based on the result of a single read and write operation of the feature map in the feature map cache, thereby significantly reducing the number of read and write operations of the feature map cache. This also significantly reduces the read and write bandwidth and data transmission power consumption of the feature map cache, thus improving the performance of the convolutional computing chip.

[0058] For example, in an n×k convolution kernel scenario, a data row of the feature map transmitted to the sliding register group and the result obtained by shifting the data row can be used for matrix calculation of k weights respectively. That is, the result of one read of the feature map cache can be used k times. The read and write bandwidth and power consumption of the feature map cache can be reduced by k times, thereby significantly improving the performance of the convolution calculation chip.

[0059] The convolution calculation chip of this application embodiment will be described below with reference to the accompanying drawings.

[0060] The convolutional computing chip provided in this application embodiment can be applied to electronic devices. For example, the electronic device may specifically be a mobile terminal, computer, server, smart screen, wearable smart device, tablet computer, etc. For ease of description and layout, the structure of the convolutional computing chip and the electronic device will be specifically described below using an electronic device containing a convolutional computing chip as an example. For example, Figure 5 is a structural block diagram of an electronic device using a convolutional computing chip provided in this application embodiment. As shown in Figure 5, the electronic device 100 may include: a convolutional computing chip 500 coupled via a bus 504, a communication interface 101, and an external memory 102.

[0061] The convolution computing chip 500 may include:

[0062] Feature map cache 501, used to cache feature maps;

[0063] The row sliding register group 502, coupled to the feature map buffer 501, is used to obtain the current data row in the feature map, shift the data in the current data row to obtain at least two first data rows, and among the multiple first data rows obtained by shifting multiple data rows of the feature map, multiple second data rows with the same shift step size are used to form a sub-feature map, and the multiple sub-feature maps correspond to multiple weights respectively.

[0064] The calculation unit 503, coupled to the row sliding register group, is used to obtain multiple first data rows from the row sliding register group 502, and obtain the weight corresponding to the sub-feature map to which each first data row belongs. The unit performs multiplication and addition operations on each first data row and the weight corresponding to the first data row among the multiple weights to obtain the current calculation result of the sub-feature map to which the first data row belongs. The calculation results of the multiple sub-feature maps are used to form the convolution calculation result of the feature map.

[0065] The aforementioned coupling can be achieved, for example, through circuit connections or software interaction. In one example, the components in the convolutional computing chip 500 can be coupled via a bus 504. For instance, the bus 504 is a transmission harness including wires. Depending on the type of information transmitted, the bus 504 can be a bus system, specifically including a data bus, an address bus, and a control bus, used to transmit data, data addresses, and control signals, respectively.

[0066] To facilitate understanding and provide a reasonable layout, the structure of the convolutional computing chip 500 and the functions of its various components will be explained in detail below with reference to Figures 6 to 17.

[0067] Communication interface 101 is suitable for any electronic device such as a transceiver for communicating with other devices or communication networks, such as Ethernet, radio access network (RAN), wireless local area network (WLAN), etc.

[0068] External memory 102 is connected to communication interface 101 via bus 504. In one example, external memory 102 can be coupled to either on-chip memory 506 or feature map cache 501 in convolution calculation chip 500 via external storage interface (not shown) or bus 504. External memory 102 is used to store feature maps acquired through communication interface 101 and to transfer feature maps to convolution calculation chip. Exemplarily, external memory 102 can be double data rate synchronous dynamic random access memory (DDR). In one example, external memory 102 can also be used to store convolution calculation results output by calculation unit 503 to reduce storage pressure on on-chip memory 506.

[0069] Figure 5 shows a schematic diagram of the electronic device 100 and the convolutional computing chip 500. It should be understood that the electronic device 100 and the convolutional computing chip 500 shown in Figure 5 are merely examples of an electronic device and a convolutional computing chip, respectively, and that the electronic device 100 and the convolutional computing chip 500 may have more or fewer components than shown in the figure, may combine two or more components, or may have different component configurations. The functions of the various components shown in Figure 5 can be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application-specific integrated circuits.

[0070] For example, Figure 6 is one of the structural block diagrams of a convolutional computation chip provided in an embodiment of this application. As shown in Figure 6, the convolutional computation chip includes an input bus, a feature map buffer, a weight buffer, a row sliding register group, a matrix multiplier, an accumulator, a row accumulator register group, and an output bus. Specifically, in this convolutional computation chip:

[0071] The input bus is used to transfer feature maps to the feature map buffer and convolution kernels to the weight buffer.

[0072] Feature map cache is used to cache feature maps;

[0073] The row sliding register group is used to obtain the current data row of the feature map in the feature map cache; the data in the current data row is cached sequentially into multiple registers of the row sliding register group, the multiple registers including multiple registers connected in pairs except for the registers at the beginning and end.

[0074] Similar to the embodiment in Figure 2, the feature maps in the feature map cache of Figure 6 are stored in a row-like format, and the current data row of the feature map is a row in the matrix-like feature map. In one example, the interval between any two connected registers in the above-mentioned multiple registers, i.e., the minimum interval, can be the same or different. For example, the minimum interval of the multiple registers in Figure 6 is the same, which is 0 registers apart, i.e., two connected registers are adjacent. For example, the data contained in the current data row are F(0,0), F(0,1), ..., F(B-1,M). The two connected registers in the multiple registers are adjacent, and from right to left, they are register 0, register 1, ..., register R. The row sliding register group caches the data in the current data row sequentially starting from the first register on the right: F(0,0) is cached in register 0, F(0,1) is cached in register 1, ..., F(B-1,M-1) is cached in register R. Wherein, (B-1)×M is the total number of data contained in the current data row.

[0075] It is understood that the minimum interval among multiple registers can be zero, one or more registers, for example, register 1 and register 2 are connected with a two-register interval between them; register 3 and register 4 are connected with a two-register interval between them, and so on. The interval between any two connected registers can be set according to application requirements, and this embodiment does not impose any restrictions on this. In addition, data in a data row can be cached sequentially in multiple registers, starting from the first register on the right or the first register on the left. The caching direction can be set according to application requirements, and this embodiment does not impose any restrictions on this.

[0076] In one alternative implementation, the row sliding register group is specifically used for:

[0077] The data in the current data row cached in each of the multiple registers is shifted multiple times according to the shift step size of 0 bits to the adjacent register stored earlier in the 0th shift, and the shift step size of 1 bit to the adjacent register stored earlier in the 0th shift and thereafter. Each shift results in a first data row.

[0078] Multiple first data rows are obtained by shifting multiple data rows of the feature map. Multiple second data rows with the same shift step size are used to form a sub-feature map. The multiple sub-feature maps correspond to multiple weights respectively.

[0079] Similar to the embodiment in Figure 2, in this embodiment, when the convolution kernel slides on the feature map, the data in different regions corresponding to the multiple weights in the convolution kernel in the input feature map can form multiple different sub-feature maps. That is to say, the multiple weights in the convolution kernel correspond one-to-one with the multiple sub-feature maps. For example, in the weight buffer shown in Figure 6, the weight (0,0) of the convolution kernel corresponds to sub-feature map 1, the weight (0,1) corresponds to sub-feature map 2, the weight (0,2) corresponds to sub-feature map 3, ..., the weight (n-1,k-3) corresponds to sub-feature map n×k-2, the weight (n-1,k-2) corresponds to sub-feature map n×k-1, and the weight (n-1,k-1) corresponds to sub-feature map n×k.

[0080] For example, Figure 7 is a timing example diagram of row sliding accumulation provided in an embodiment of this application. As shown in Figure 7, the feature map includes the first data row F0, ..., the nth data row F n-1 Data row F0 includes data F0(0,0), F0(0,1), ..., F0(B-1,M-1); data row F n-1 Including data F n-1 (0,0), F n-1 (0,1), ...,F n-1(B-1, M-1). The row sliding register group retrieves data from the feature map row by row, starting from the first row. Starting from the first register on the right, it caches the data in the current data row sequentially in multiple adjacent registers (excluding the first and last registers). The multiple registers are, from right to left, register 0, register 1, ..., register R.

[0081] Based on the above-mentioned caching function of the row sliding register group, when the current data line is the first data line F0:

[0082] The row sliding register group is cached sequentially as follows: F0(0,0) is cached in register 0, F0(0,1) is cached in register 1, ..., F0(B-1,M-1) is cached in register R. At this time, the shift step size is 0 bits, that is, the 0th shift, and the first data row of sub-feature diagram 1 is obtained.

[0083] The row sliding register performs the first shift on the first data row F0: the data buffered in register 0 is shifted one bit to the right (i.e., discarded), the data buffered in register 1 is shifted one bit to the right (i.e., moved towards the adjacent register 0 that was previously buffered), and so on, the data buffered in register R is shifted one bit to the right (i.e., moved towards the adjacent register R-1 that was previously buffered). Thus, the first shift result of the first data row F0 is: F0(0,1) is buffered in register 0, F0(0,2) is buffered in register 1, and so on, F0(B-1,M-1) is buffered in register R-1. Register R is empty or buffered with 0 data, resulting in the first data row of sub-feature diagram 2.

[0084] The row sliding register performs a second shift on the first data row F0: the data cached in register 0 is shifted one bit to the right (i.e., discarded), the data cached in register 1 is shifted one bit to the right (i.e., moved towards the adjacent register 0 that was previously cached), ..., the data cached in register R is shifted one bit to the right (i.e., moved towards the adjacent register R-1 that was previously cached). Thus, the second shift result for the first data row F0 is: F0(0,2) is cached in register 0, F0(0,2) is cached in register 1, ..., F0(B-1,3) is cached in register R-2, and registers R-1 and R are either empty or have 0 cached data, resulting in the first data row of sub-feature diagram 3.

[0085] Thus, through the aforementioned multiple shifts of the first data row F0, at least two first data rows of that data row F0 are obtained. That is to say, one shift yields one first data row of the current data row in the feature map, and multiple shifts yield at least two first data rows of the current data row. Each first data row of the current data row corresponds to a weight, and different first data rows of the current data row belong to different sub-feature maps. Correspondingly, different first data rows of the current data row correspond to different weights.

[0086] The row sliding register group handles the remaining data rows F1 to F2 in the feature map. n-1 The above-described buffering and shifting processes are performed similarly to those for the first data row F0, except that the data being processed is different. This results in at least two first data rows corresponding to each data row in the feature map. Among the multiple first data rows corresponding to all data rows in the feature map, multiple second data rows with the same shift step size are used to form a sub-feature map, thus obtaining the remaining data rows of sub-feature map 1, i.e., the remaining first data rows, as well as the remaining data rows of each sub-feature map from sub-feature map 2 to sub-feature map n×k. For example, the row sliding register group performs the 0th shift on the data of the buffered data row F1, i.e., the buffered data row F1 becomes the second first data row of sub-feature map 1, and the first shift on data row F1 becomes the second first data row of sub-feature map 2.

[0087] In this diagram, the first data row obtained by shifting data row F1 by 0 bits and the first data row obtained by shifting data row F0 by 0 bits both have a shift step of 0. These two first data rows are used to form the two second data rows of sub-feature diagram 1. Similarly, the second and first first data rows of sub-feature diagram 1 both have a shift step of 0, and are thus used to form the two second data rows of sub-feature diagram 1. Finally, the first data row obtained by shifting data row F1 by 1 bit and the first data row obtained by shifting data row F0 by 1 bit both have a shift step of 1, and are used to form the two second data rows of sub-feature diagram 2.

[0088] In an alternative example, the convolutional computation chip may also include a weight cache coupled to the computation unit;

[0089] Weight cache, used to cache at least one weight in the convolution kernel;

[0090] The row sliding register group is specifically used to divide each first data row corresponding to the current data row into multiple sub-data rows;

[0091] The calculation unit is specifically used to obtain multiple sub-data rows of the first data row corresponding to the current data row from the row sliding register group in multiple clock cycles, and to obtain the weights corresponding to the first data row from the weight cache.

[0092] For example, as shown in Figure 7, if a row sliding register group can transmit M data items in one clock cycle, then each first data row of the current data row, such as data row F0: data row F0, the data row obtained by the first shift of data row F0, and the data row obtained by the second shift of data row F0, can be divided into B sub-data rows, each containing M data items. A sub-data row is equivalent to a data block containing M data items. For example, the row sliding register group divides data row F0 into B sub-data rows, where the first sub-data row includes the data transmitted in the first clock cycle: F0(0,0), F0(0,1), ..., F0(0,M-1); the second sub-data row includes the data transmitted in the second clock cycle: F0(1,0), F0(1,1), ..., F0(1,M-1); ..., the Bth sub-data row includes the data transmitted in the Bth clock cycle: F0(B-1,0), F0(B-1,1), ..., F0(B-1,M-1). Both B and M are integers greater than 1.

[0093] Referring again to Figure 7, the row sliding register group divides each first data row into B sub-data rows. Each sub-data row contains M data items. The computation unit can then acquire B sub-data rows over B consecutive clock cycles, ensuring that the weights corresponding to the first data rows to which these B sub-data rows belong are used for multiplication and addition operations on the B sub-data rows within those B consecutive clock cycles. In this way, a weight can be reused B×M times. Correspondingly, the read bandwidth and power consumption of the weight buffer can be reduced by a factor of B×M, and the toggle rate of the input signal in the computation unit is significantly reduced, thereby significantly reducing the power consumption of the computation unit.

[0094] Additionally, the row sliding register group can transfer a sub-data row to the computing unit each clock cycle. This transfer can be either the row sliding register group writing the sub-data row to the computing unit, or the computing unit reading the sub-data row from the row sliding register group. Subsequent data acquisition methods are similar to this transfer, the difference being the type of data acquired. This application embodiment does not limit the specific data acquisition method and can be set according to application requirements.

[0095] In this embodiment, the row sliding register group divides each first data row corresponding to the current data row into multiple sub-data rows. This ensures that the computing unit uses the weight corresponding to one first data row to perform multiplication and addition operations on multiple sub-data rows over multiple clock cycles. In other words, a weight can remain unchanged for multiple consecutive clock cycles for the corresponding multiplication and addition operations. Thus, a weight can be reused multiple times, and consequently, the read bandwidth and power consumption of the weight buffer can be significantly reduced. Furthermore, this reduces the frequency of weight changes, which in turn significantly reduces the inversion rate of the input signal in the computing unit, thereby significantly reducing the power consumption of the matrix multiplier.

[0096] For example, still referring to Figure 6, the computation unit in the convolution computation chip may specifically include a matrix multiplier coupled to the row sliding register group and the weight cache, respectively, and an accumulator coupled to the matrix multiplier and the row accumulator register group, respectively. Specifically:

[0097] The matrix multiplier is used to obtain multiple first data rows from the row sliding register group and obtain the weights corresponding to the sub-feature map to which each first data row belongs through the weight cache; perform matrix multiplication operation on each first data row and the weights corresponding to the sub-feature map to which the first data row belongs to obtain the matrix calculation result of the first data row;

[0098] The accumulator is used to obtain the matrix calculation result of the current first data row from the matrix multiplier, and obtain the previous accumulation result from the row accumulation register group. It accumulates the matrix calculation result of the current first data row and the previous accumulation result to obtain the current accumulation result; and updates the current accumulation result to the row accumulation register group.

[0099] The row accumulator register group is used to obtain the current accumulation result from the accumulator and update the previous accumulation result with the current accumulation result until the last accumulation result corresponding to the last data row of the feature map is updated, thus obtaining the convolution calculation result of the feature map.

[0100] Referring again to Figure 7, the current first data row is data row F0. The matrix multiplier performs matrix multiplication on data row F0 and weight (0,0) to obtain the current matrix calculation result of data row F0, and transmits the matrix calculation result of data row F0 to the accumulator. The accumulator adds the matrix calculation result of data row F0 to the previous accumulation result 0 (i.e., empty, or initial value 0), to obtain the current accumulation result, which is also the 0th accumulation result: the accumulation result of the matrix calculation result of data row F0 and the initial value, and transmits this current accumulation result to the row accumulation register group. The row accumulation register group uses this current accumulation result to update the previous accumulation result 0. This current accumulation result is the accumulation result of the current first data row (the low 0th shift result of data row F0 in the feature map: data row F0), the current accumulation result of the data row corresponding to the current first data row (data row F0 in the feature map), and the current calculation result of the sub-feature map (sub-feature map 1) to which the current first data row belongs.

[0101] Similar to the processing of data row F0 described above, when the current first data row is the result of the first shift of data row F0, that is, when data row F0 is shifted one position to obtain the data row, the previous accumulation result is the aforementioned 0th accumulation result: the accumulation result of the matrix calculation of data row F0 and the initial value. The accumulator adds the matrix calculation result of the first shift of data row F0 to the result of 0 accumulations, obtaining the current accumulation result, that is, the result of 1 accumulation: the accumulation result of the matrix calculation of data row F0 and the matrix calculation result of the first shift of data row F0, and transmits this current accumulation result to the row accumulation register group. The row accumulation register group uses this current accumulation result to update the previous accumulation result of 0. This current accumulation result is the accumulation result of the current first data row (the first shift result of data row F0 in the feature map), the current accumulation result of the data row corresponding to the current first data row (data row F0 in the feature map), and the current calculation result of the sub-feature map (sub-feature map 2) to which the current first data row belongs.

[0102] The convolutional computation chip processes at least two first data rows corresponding to each data row of the feature map in a similar manner, obtaining the accumulated result of each data row, which is also the calculation result of each sub-feature map in multiple sub-feature maps. This realizes matrix computation of the feature map in the row direction. In other words, the calculation result of a sub-feature map is also the intermediate calculation result used to accumulate and obtain the convolution calculation result of that sub-feature map. Furthermore, the accumulation of the accumulated results of each data row in multiple data rows of the feature map by the convolutional computation chip is equivalent to accumulating the intermediate calculation results of each sub-feature map in multiple sub-feature maps, which can obtain the convolution calculation result of each sub-feature map. The accumulation of the convolution calculation results of each sub-feature map in multiple sub-feature maps also realizes matrix computation of the feature map in the column direction.

[0103] In this way, the row sliding register group completes the update of the last accumulated result corresponding to the last data row of the feature map, thus obtaining the accumulated result of the convolution calculation of each sub-feature map in multiple sub-feature maps (accumulation as shown in Figure 2), which is to say, the convolution calculation result of the feature map. That is to say, multiple calculation results of a sub-feature map are used to accumulate to obtain the convolution calculation result of that sub-feature map, and the convolution results of each sub-feature map in multiple sub-feature maps are used to accumulate to obtain the convolution calculation result of the feature map.

[0104] In addition, the convolution calculation chip may also include an output bus coupled to the row accumulator register group, used to acquire the convolution calculation result of the feature map in the row accumulator register group and output the convolution calculation result of the feature map. The input bus and output bus in the above convolution calculation chip can be the same or different bus systems.

[0105] For ease of understanding, the functions of the convolution calculation chip provided in the embodiments of Figures 6 and 7 will be described in detail below with reference to Figures 8 and 9.

[0106] For example, Figure 8 is one of the schematic diagrams illustrating the process of a convolution calculation chip performing 3x3 convolution calculation according to an embodiment of this application. As shown in Figure 8, the feature map cache is used to store feature maps, the weight cache is used to store the weights A to I in the 3×3 convolution kernel, the matrix multiplier and accumulator are used for the multiplication and addition calculation of the 3×3 convolution, and the row accumulation register is used to cache and update the multiplication and addition results as intermediate calculation results. For example, Figure 9 is one of the timing example diagrams of the row sliding accumulation process in 3x3 convolution according to an embodiment of this application. As shown in Figure 9, the convolution calculation chip provided in the embodiment of Figure 8 of this application is specifically used for:

[0107] a. The weight buffer transfers the weights of the first row and first column (i.e., the weight A at the first row and first column position in the 3×3 convolution kernel) to the matrix multiplier.

[0108] b. The feature map buffer transfers the first data row, i.e., the first row of the feature map, to the row sliding register group buffer. This data row is equivalent to the first data row with a shift step of 0. Therefore, the first data row is also a first data row.

[0109] c. The row sliding register group divides the first data row into B sub-data rows (each sub-data row contains M data items). Each of the B sub-data rows is output to the matrix multiplier within B consecutive clock cycles (one clock cycle of data transmission constitutes one sub-data row). The matrix multiplier performs a matrix multiplication operation (multiplying and adding the sub-data rows and weight A) and transmits the result to the accumulator. The accumulator adds this result to the previous accumulation result stored in the row accumulator register at the same location (the first accumulation is 0 by default, meaning zero accumulations), obtaining a new accumulation result. This new accumulation result is then updated in the row accumulator register group. Throughout the matrix multiplication operation of the B sub-data rows of the first data row, the weight A remains unchanged.

[0110] d. The weight cache updates the weight of the next column in the first row (i.e., the weight B at the position of the 1st row and 2nd column in the 3×3 convolution kernel) to the matrix multiplier. The row sliding register group slides one data point to the right (i.e., each register passes the data to the adjacent register on the right), which is the first shift of the first data row, to obtain the first data row corresponding to the first data row.

[0111] e. Perform a process similar to c on the first shift result, except that the specific data processed is different, and weight B is used. Weight B remains unchanged during the execution of c. Perform a process similar to d, except that the updated weight used this time is the weight C at the 1st row, 3rd column position of the 3×3 convolution kernel. Perform a process similar to c on the second shift result obtained from d, except that the specific data processed is different, and weight C is used. Weight C remains unchanged during the execution of c.

[0112] f. The weight buffer transfers the weight of the first column of the next row (i.e., the weight D at the position of the second row and first column in the 3×3 convolution kernel) to the matrix multiplier.

[0113] g. The feature map buffer transfers the second data row, i.e. the next row of the feature map, to the row sliding register group buffer.

[0114] h) Perform a similar process to c-f on the second data row, the difference being the type of data being processed. After processing the second and third data rows, the convolution calculation of the first row of the feature map convolution result is complete. This yields the result of a single-row multiplication of weight A with its corresponding sub-feature map, the result of a single-row multiplication of weight B with its corresponding sub-feature map, and the result of a single-row multiplication of weight C with its corresponding sub-feature map. The final sum of these three single-row multiplication results is then obtained. Therefore, the final sum of a single row corresponding to multiple sub-feature maps (i.e., the sum of the single-row multiplication results of multiple sub-feature maps) is the result of a single-row convolution of the feature map.

[0115] i. Similarly, the convolution calculation for each subsequent row after obtaining the first row's convolution calculation result is similar to the process described above for obtaining the first row's convolution calculation result. The difference lies in the specific data being processed. For the same part, please refer to the process described above for obtaining the first row's convolution calculation result; it will not be repeated here.

[0116] In this embodiment, the first data row entering the matrix multiplier is generated by shifting a row sliding register. Data in the feature map read once (i.e., one row of data) can be reused three times (e.g., for matrix multiplication calculations corresponding to weights A, B, and C). Therefore, the read bandwidth of the feature map buffer is reduced by a factor of three, and the read power consumption is reduced by a factor of three. Furthermore, for any first data row, a weight can be reused B×M times; therefore, the read bandwidth of the weight buffer is reduced by a factor of B×M, and the read power consumption is reduced by a factor of B×M. In the matrix multiplication calculation, the weights can remain unchanged for B consecutive clock cycles, significantly reducing the PPI signal toggle rate in the matrix multiplier; therefore, the computational power consumption of the matrix multiplier is significantly reduced.

[0117] Based on the convolution calculation chip shown in Figure 5, this application embodiment also provides a convolution calculation method. For example, Figure 10 is a flowchart illustrating a convolution calculation method provided in this application embodiment. This convolution calculation method can be applied to a convolution calculation chip 500, which may include a feature map cache, a row sliding register group, and a calculation unit as shown in Figure 5; as shown in Figure 10, the method may include:

[0118] S1001, cache the current data row of the feature map in the feature map cache to the row sliding register group;

[0119] S1002, the row sliding register group shifts the data in the current data row to obtain at least two first data rows; multiple second data rows with the same shift step size among the multiple first data rows obtained by shifting multiple data rows of the feature map are used to form a sub-feature map, and the multiple sub-feature maps correspond to multiple weights respectively;

[0120] S1003, the calculation unit obtains multiple first data rows from the row sliding register group, and obtains the weight corresponding to the sub-feature map to which each first data row belongs. It performs multiplication and addition operations on each first data row and the weight corresponding to the sub-feature map to which the first data row belongs, respectively, to obtain the current calculation result of the sub-feature map to which the first data row belongs. The calculation results of multiple sub-feature maps are used to form the convolution calculation result of the feature map.

[0121] Each step in this embodiment corresponds to the function implemented by each component of the convolution calculation chip in the embodiment of Figure 6 above. Please refer to the relevant descriptions already provided in the embodiment of Figure 6 above, which will not be repeated here.

[0122] The convolution calculation chip and convolution calculation method provided in the embodiments of this application will be further described below with reference to Figures 11 to 17.

[0123] In an optional example, Figure 11 is one of the structural block diagrams of a convolutional computing chip provided in an embodiment of this application. As shown in Figure 11, the convolutional computing chip 500 shown in Figure 5 also includes multiple row backup register groups 507; the multiple row backup register groups 507 are respectively coupled to feature map cache, and the multiple row backup register groups 507 are respectively coupled to row sliding register groups;

[0124] Multiple row backup register groups 507 are used to retrieve multiple data rows from the feature map cache, wherein each row backup register group is used to cache one of the multiple data rows;

[0125] The row sliding register group, coupled to multiple row backup register groups, is specifically used to obtain one data row from multiple data rows from each row backup register group.

[0126] In this embodiment of the application, data rows in the feature map are backed up by multiple row backup register groups. In this way, data rows that need to be reused can be obtained directly from the row backup register groups without having to be read repeatedly from the feature map cache, thereby further reducing the read bandwidth and power consumption of the feature map cache.

[0127] For example, Figure 12 is one of the structural block diagrams of a convolutional computing chip provided in an embodiment of this application. As shown in Figure 12, based on the chip shown in Figure 8, this convolutional computing chip also includes multiple row backup register groups, for example, n-1 row backup register groups, where n is the number of rows of the convolution kernel. The multiple row backup register groups can be coupled to the row sliding register group respectively through switches. The switches are used to control whether the row backup register group coupled to the switch transmits data rows to the row sliding register group. For example, Figure 13 is one of the schematic diagrams of a convolutional computing chip performing 3x3 convolution calculations provided in an embodiment of this application. As shown in Figure 13, the convolutional computing chip shown in Figure 12 is illustrated using 3×3 convolution as an example. At this time, there are 2 row backup register groups, and data rows can be selected to enter the row sliding register group through switches. The timing diagram of the row sliding accumulation process is the same as shown in Figure 9, and can be referred to the description of the embodiment in Figure 9, which will not be repeated here.

[0128] Referring to Figure 13, the convolution calculation chip shown in Figure 12 is specifically used for:

[0129] a. The weight buffer transfers the weights of the first row and first column (i.e., the weight A at the first row and first column position in the 3×3 convolution kernel) to the matrix multiplier.

[0130] b. The feature map buffer transfers the first data row, which is also the first row of the feature map, to the row sliding register group buffer.

[0131] c. The row sliding register group divides the first data row into B sub-data rows (each sub-data row contains M data items). Each of the B sub-data rows is output to the matrix multiplier within B consecutive clock cycles (one clock cycle of data transmission constitutes one sub-data row). The matrix multiplier performs a matrix multiplication operation (multiplying and adding the sub-data rows and weight A) and transmits the result to the accumulator. The accumulator adds this result to the previous accumulation result stored in the row accumulator register at the same location (the first accumulation is 0 by default, meaning zero accumulations), obtaining a new accumulation result. This new accumulation result is then updated in the row accumulator register group. Throughout the matrix multiplication operation of the B sub-data rows of the first data row, the weight A remains unchanged.

[0132] d. The weight cache updates the weight of the next column in the first row (i.e., the weight B at the position of the 1st row and 2nd column in the 3×3 convolution kernel) to the matrix multiplier. The row sliding register group slides one data point to the right (i.e., each register passes the data to the adjacent register to its right), which is the first shift of the first data row.

[0133] e. Perform a process similar to c on the first shift result, except that the data processed is different, and weight B is used, while weight B remains unchanged during the process. Perform a process similar to d, except that the updated weight used this time is the weight C at the 1st row, 3rd column position of the 3×3 convolution kernel. Perform a process similar to c on the second shift result, except that the data processed is different, and weight C is used, while weight C remains unchanged during the process.

[0134] f. The weight buffer transfers the weight of the first column of the next row (i.e., the weight D at the position of the second row and first column in the 3×3 convolution kernel) to the matrix multiplier.

[0135] g. The feature map buffer transfers the second data row, i.e. the next row of the feature map, to the row sliding register group buffer.

[0136] h) Perform a process similar to c-f on the second data row, the difference being the type of data being processed. After processing the second and third data rows, complete the convolution calculation to obtain the first row convolution result of the feature map. That is, obtain the one-row multiplication result of weight A with the corresponding sub-feature map, the one-row multiplication result of weight B with the corresponding sub-feature map, and the one-row multiplication result of weight C with the corresponding sub-feature map, and obtain the cumulative result of these multiplication results, which is the final one-row cumulative result corresponding to these sub-feature maps.

[0137] i. The convolution calculation process for subsequent convolution results after obtaining the first row convolution result of the feature map, such as the convolution result of the first row convolution, is similar to the process described above for obtaining the first row convolution result of the feature map. The difference is that in process b, the row backup register group also retrieves unbacked-up data rows from the feature map cache for caching. In this way, data rows already cached in the row backup register group will not be retrieved again from the feature map cache. For the same part, please refer to the convolution calculation process corresponding to the first row weights above, which will not be repeated here.

[0138] In this embodiment, based on the embodiment in Figure 8, the reused data rows, such as the second and third data rows, are cached in the row backup register group. This allows them to be used for matrix multiplication calculations with the second weight and the third row weight. In other words, a data row can be used 3 times in the matrix multiplication calculation of each row weight. Through the row backup register group, a data row can be used for matrix multiplication calculations of 3 rows weight. Therefore, the data row obtained from one read of the feature map cache can be reused 9 times, reducing the read bandwidth by 9 times and the read power consumption by 9 times.

[0139] In one example, Figure 14 is a structural block diagram of a convolutional computing chip provided in an embodiment of this application. As shown in Figure 14, compared with the convolutional computing chip shown in Figure 12, the number of row backup register groups in the convolutional computing chip of this embodiment is the same as the number of rows of the convolutional kernel, which is n. At this time, the row sliding register group no longer needs to obtain the data rows of the feature map from the feature map cache.

[0140] In this embodiment, the number of row backup register groups is the same as the number of rows of the convolution kernel, or the number of rows after combining with the row slider group is the same as the number of rows of the convolution kernel. This ensures that the number of read / write operations on the feature map cache is reduced and better adapted to convolution calculation, further improving the performance of the convolution calculation chip. Furthermore, when the number of row backup register groups is less than the number of rows of the convolution kernel, data rows are also obtained through the row slider register group, which can reduce the number of row backup register groups, achieving a balance between reducing the cost and performance of the convolution calculation chip.

[0141] In one optional implementation, the aforementioned row sliding register group is specifically used for:

[0142] Each of the multiple data rows is shifted by the same step size to obtain a third data row corresponding to each data row. The third data row corresponding to a data row is one of at least two first data rows corresponding to the data row.

[0143] Accordingly, the computation unit in the convolutional computation chip specifically includes a multiply-accumulate unit and multiple row accumulator register groups; the multiply-accumulate unit and the row sliding register groups are coupled, and the multiple row accumulator register groups are coupled to the convolutional unit respectively;

[0144] The multiply-accumulate unit is used to obtain the current third data row of each data row in multiple data rows from the row sliding register group, and obtain the weight corresponding to the current sub-feature map. Based on the current third data row, the weight corresponding to the current sub-feature map, and the existing row accumulation result of multiple sub-feature maps obtained from the row accumulation register group, multiply-accumulate operation is performed to obtain the current row accumulation result of multiple sub-feature maps. Here, the current sub-feature map is the sub-feature map to which the current third data row belongs. The row accumulation result of multiple sub-feature maps corresponds to the row convolution calculation result of the feature map. The final accumulation result of multiple sub-feature maps is the convolution calculation result of the feature map.

[0145] Each of the multiple row accumulator register groups is used to obtain the current row accumulator result corresponding to the multiple sub-feature maps corresponding to the convolution calculation result of the feature map from the multiply-accumulate unit, and to update the existing row accumulator result corresponding to the multiple sub-feature maps corresponding to the convolution calculation result of the feature map using the current row accumulator result corresponding to the multiple sub-feature maps corresponding to the convolution calculation result of the feature map.

[0146] For example, Figure 15 is one of the structural block diagrams of a convolutional computation chip provided in an embodiment of this application. As shown in Figure 15, the convolutional computation chip may further include multiple, for example, n row accumulator register groups, which can cache the accumulation results of a total of n data rows, that is, the intermediate accumulation results corresponding to the sub-feature maps of each first data row corresponding to the n data rows. In the next clock cycle, the intermediate accumulation result is output to the accumulator and added to the new multiplication result output by the matrix multiplier to obtain a new accumulation result, and so on, until the final accumulation result is obtained. Here, n is the number of rows of the convolution kernel. The multiply-accumulate unit may specifically include a matrix multiplier coupled to the row sliding register group and the weight cache respectively, and an accumulator coupled to multiple row accumulator register groups and the matrix multiplier respectively.

[0147] To facilitate understanding, the convolution calculation chip shown in Figure 15 will be specifically explained below using a 3×3 convolution as an example. For example, Figure 16 is one of the schematic diagrams illustrating the process of a convolution calculation chip performing a 3x3 convolution calculation according to an embodiment of this application. As shown in Figure 16, since the convolution kernel has three rows, that is, three rows of convolution need to be calculated, corresponding to three rows of convolution calculation results, therefore, three row backup register groups and a total of three row accumulation register groups can be added based on the embodiment in Figure 8. For example, Figure 17 is one of the timing examples of the row sliding accumulation process in the 3x3 convolution provided in an embodiment of this application. Combining Figures 17 and 16, the convolution calculation chip provided in this embodiment can specifically be used for:

[0148] a. The weight buffer transfers the weights of the first row and first column (i.e., the weight A at the first row and first column position in the 3×3 convolution kernel) to the matrix multiplier.

[0149] b1. The feature map cache transfers the three data rows of the feature map, that is, the first three rows of data of the feature map (the first data row F0, the second data row F1, and the third data row F2), to the three row backup register group cache.

[0150] b2. The row sliding register group sequentially obtains three data rows of the feature map from the row backup register group, caches the obtained data rows, and performs process c for each of the three data rows.

[0151] c. The row sliding register group outputs B sub-data rows corresponding to the cached data rows (one sub-data row per clock cycle) to the matrix multiplier for multiplication and addition with weight A, i.e., matrix multiplication. The resulting multiplication is then transmitted to the accumulator. The accumulator adds this multiplication and addition result to the previous accumulation result stored in the row accumulator register group at the same location (the first accumulation is 0 by default, i.e., 0 accumulations) to obtain a new accumulation result. The new accumulation results of the three data rows are then sequentially updated to row accumulator register group 1, row accumulator register group 2, and row accumulator register group 3. Throughout the matrix multiplication operation of the first data row F0, the second data row F1, and the third data row F2 with weight A, the matrix multiplier maintains the weight A involved in the calculation unchanged.

[0152] d. The weight cache updates the weights of the next column in the first row (i.e., the weights B in the first row and second column of the 3×3 convolution kernel) to the matrix multiplier.

[0153] e1. Perform the following processing sequentially on the three data rows of the above feature map: load from the row backup register group to the row sliding register group, and slide the row sliding register group one data position to the right (i.e., each register transfers data to the adjacent register to its right), which is the first shift of each of the three data rows. Perform similar processing to step c on the shifted result, the difference being that the data being processed is different.

[0154] e2. The weight cache updates the next column weight (i.e., the weight C in the first row and third column of the 3×3 convolution kernel) to the matrix multiplier.

[0155] e3. Perform the following processing sequentially on the three data rows of the above feature map: load from the row backup register group to the row sliding register group, and slide the row sliding register group one data position to the right (i.e., each register transfers data to the adjacent register to its right), which is to perform the second shift for each of the three data rows. Perform similar processing to step c on the shifted result, the difference being that the data being processed is different.

[0156] f. The weight buffer transfers the weight of the first column of the next row (i.e., the weight D at the position of the second row and first column in the 3×3 convolution kernel) to the matrix multiplier.

[0157] g. After processing the second and third data rows c-f, the convolution calculation for obtaining the first row convolution result of the feature map is completed. That is, the one-row multiplication result of weight A with its corresponding sub-feature map, the one-row multiplication result of weight B with its corresponding sub-feature map, and the one-row multiplication result of weight C with its corresponding sub-feature map are obtained, and the cumulative result of these multiplication results is the final one-row cumulative result corresponding to these sub-feature maps. In other words, the first row convolution calculation result of the feature map is cached in row accumulation register group 1.

[0158] Furthermore, since three row accumulator register groups are used, after the c-f processing of the second and third data rows is completed, row accumulator register group 2 caches the intermediate accumulator result used to obtain the second row convolution calculation result of the feature map, and row accumulator register group 3 caches the intermediate accumulator result used to obtain the third row convolution calculation result of the feature map.

[0159] h. The feature map cache transfers the fourth data row of the feature map to the row sliding register group and updates the row backup register group, overwriting the first data row in the feature map that is no longer used.

[0160] i. Perform a process similar to c-g on the second to fourth data rows, the difference being that the specific data being processed is different. After completing c-f processing on the second to fourth data rows, the convolution calculation for obtaining the convolution calculation result of the second row of the feature map is completed.

[0161] j. The convolution calculation process for subsequent convolution results after obtaining the first row convolution result of the feature map, such as the convolution result of the first row convolution of the feature map, is similar to the convolution calculation process for obtaining the first row convolution result of the feature map described above. The difference is that in process b1, the data rows already cached in the row backup register group are not repeatedly loaded from the feature map cache into the row backup register group. For the same parts, please refer to the convolution calculation process for obtaining the first row convolution result of the feature map described above, and it will not be repeated here.

[0162] In this embodiment, based on the embodiments in Figures 12 and 14, the accumulated results of three data rows are cached using multiple row accumulator register groups, such as three row accumulator register groups. When retrieving the accumulated result of a data row, the weight can be reused B×M times. Furthermore, during the process of retrieving the accumulated results of three data rows, each data row corresponds to a second sub-data row using the same weight. Therefore, a weight can be reused 3×B×M times. This reduces the read bandwidth and power consumption of the weight cache by a factor of 3×B×M, and a weight can remain unchanged for 3×B clock cycles, significantly reducing the signal toggle rate of PPI in the matrix multiplier, thereby greatly reducing the computational power consumption of the matrix multiplier.

[0163] For example, the following comparison is made between the AI ​​core of this application using the convolutional computing chip solution provided in the embodiments of this application and a traditional AI core using a traditional convolutional computing chip solution, with an input channel number Ci = 32, an output channel number Co = 16, a data transmission count M = 16 per clock cycle, and a data block count B = 8 in the matrix to be convolved, as shown in Table 1 below:

[0164] Table 1 Comparison of application effects between the convolutional computing chip provided in this application embodiment and a traditional convolutional computing chip.

[0165] Based on the comparison shown in Table 1 above, compared with the AI ​​core using a traditional convolutional computing chip solution, the area of ​​the register group in the AI ​​core using the convolutional computing chip provided in this application embodiment increases by 0.013 mm^2, but the area of ​​the feature map cache decreases by 0.021 mm^2, resulting in a slightly smaller total area and a significant reduction in power consumption.

[0166] It should be understood that the convolutional computing chip shown in the block diagrams of the embodiments of this application is only an example of a convolutional computing chip, and the convolutional computing chip may have more or fewer components than shown in the figures, may combine two or more components, or may have different component configurations. The various components shown in the block diagrams of the embodiments of this application can be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application-specific integrated circuits.

[0167] In addition, this application also provides an apparatus, which may specifically be a chip, component or module. The apparatus may include a connected processor and a memory. The memory is used to store computer execution instructions. When the apparatus is running, the processor can execute the computer execution instructions stored in the memory to cause the chip to execute the convolution calculation method in the above method embodiments.

[0168] Through the above description of the embodiments, those skilled in the art will understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0169] This application also provides a computer storage medium storing computer instructions. When the computer instructions are executed on an electronic device, the electronic device performs the aforementioned method steps to implement the data processing method described above.

[0170] This application also provides a computer program product that, when run on a computer, causes the computer to perform the aforementioned steps to implement the data processing method described in the above embodiments.

[0171] In the embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0172] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0173] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0174] Any content in the various embodiments of this application, as well as any content in the same embodiment, can be freely combined. Any combination of the above content is within the scope of this application.

[0175] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, in essence, or the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0176] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

[0177] The steps of the methods or algorithms described in conjunction with the embodiments of this application can be implemented in hardware or by a processor executing software instructions. The software instructions may include corresponding software modules, which may be stored in random access memory (RAM), flash memory, read-only memory (ROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, portable hard disks, CD-ROMs, or any other form of storage medium well known in the art. An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and storage medium may reside in an application-specific integrated circuit (ASIC). Alternatively, the ASIC may reside within an electronic device. Of course, the processor and storage medium may also exist as discrete components within an electronic device.

[0178] Those skilled in the art will recognize that the functions described in the embodiments of this application in one or more of the above examples can be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or code on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein communication media include any medium that facilitates the transfer of a computer program from one place to another. Storage media can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0179] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A convolutional computing chip, characterized in that, include: Feature map cache is used to cache feature maps; A row sliding register group, coupled to the feature map cache, is used to obtain the current data row in the feature map, shift the data in the current data row to obtain at least two first data rows, and multiple second data rows with the same shift step size among the multiple first data rows obtained by shifting multiple data rows of the feature map are used to form a sub-feature map, and multiple sub-feature maps correspond to multiple weights respectively. The calculation unit, coupled to the row sliding register group, is used to obtain the plurality of first data rows from the row sliding register group, and to obtain the weight corresponding to the sub-feature map to which each of the plurality of first data rows belongs. The unit performs a multiplication and addition operation on each of the plurality of weights and the weight corresponding to the sub-feature map to which the first data row belongs, respectively, to obtain the current calculation result of the sub-feature map to which the first data row belongs. The calculation results of the plurality of sub-feature maps are used to form the convolution calculation result of the feature map.

2. The chip according to claim 1, characterized in that, The chip also includes: A weight cache, coupled to the computing unit, is used to cache at least one of the plurality of weights; The row sliding register group is specifically used to divide the first data row into multiple sub-data rows for each first data row corresponding to the current data row; The computing unit is specifically used to acquire multiple sub-data rows of the first data row corresponding to the current data row from the row sliding register group in multiple clock cycles, and to acquire weights from the weight cache corresponding to the sub-feature map to which the first data row belongs.

3. The chip according to claim 1 or 2, characterized in that, The chip also includes: Multiple row backup register groups are coupled to the feature map cache respectively, and are used to obtain the multiple data rows from the feature map cache, wherein each row backup register group is used to cache one data row among the multiple data rows; The row sliding register group, coupled to the plurality of row backup register groups, is specifically used to obtain one data row from the plurality of data rows from each of the row backup register groups.

4. The chip according to claim 3, characterized in that, The row sliding register group is specifically used for: Each of the plurality of data rows is shifted by the same step size to obtain a third data row corresponding to each of the plurality of data rows. The third data row corresponding to the data row is one of at least two first data rows corresponding to the data row. The computing unit includes: The multiply-accumulate unit, coupled to the row sliding register group, is used to obtain the current third data row of each data row in the plurality of data rows from the row sliding register group, and obtain the weight corresponding to the current sub-feature map. Based on the current third data row, the weight corresponding to the current sub-feature map, and the existing row accumulation result of the plurality of sub-feature maps obtained from the row accumulation register group, a multiply-accumulate operation is performed to obtain the current row accumulation result of the plurality of sub-feature maps. The current sub-feature map is the sub-feature map to which the current third data row belongs. The row accumulation result of the plurality of sub-feature maps corresponds to the row convolution calculation result of the feature map. The final accumulation result of the plurality of sub-feature maps is the convolution calculation result of the feature map. Multiple row accumulation register groups are coupled to the multiply-accumulate unit, wherein each row accumulation register group is used to obtain the current row accumulation result corresponding to multiple sub-feature maps corresponding to the convolution calculation result of the row of the feature map from the multiply-accumulate unit, and use the current row accumulation result corresponding to the multiple sub-feature maps corresponding to the convolution calculation result of the row of the feature map to update the existing row accumulation result corresponding to the multiple sub-feature maps corresponding to the convolution calculation result of the row of the feature map.

5. A convolution calculation method, characterized in that, The method, applied to a convolutional computing chip, includes: Cache feature map; Obtain the current data row in the feature map, shift the data in the current data row to obtain at least two first data rows, and use multiple second data rows with the same shift step size among the multiple first data rows obtained by shifting multiple data rows in the feature map to form a sub-feature map, and the multiple sub-feature maps correspond to multiple weights respectively; Obtain the plurality of first data rows, and obtain the weight corresponding to the sub-feature map to which each first data row belongs. Perform multiplication and addition operations on each first data row and the weight corresponding to the sub-feature map to which the first data row belongs to obtain the current calculation result of the sub-feature map to which the first data row belongs. The calculation results of the plurality of sub-feature maps are used to form the convolution calculation result of the feature map.

6. The method according to claim 5, characterized in that, Before obtaining the weight corresponding to the sub-feature map to which each of the plurality of first data rows belongs, the method further includes: Cache at least one of the multiple weights; After shifting the data in the current data row to obtain at least two first data rows, the method further includes: For each first data row corresponding to the current data row, the first data row is divided into multiple sub-data rows; The step of obtaining the plurality of first data rows and obtaining the weight corresponding to the sub-feature map to which each of the plurality of first data rows belongs includes: In multiple clock cycles, multiple sub-data rows corresponding to the first data row of the current data row are obtained, and the weights corresponding to the sub-feature maps to which the first data row belongs are obtained.

7. The method according to claim 5 or 6, characterized in that, The step of obtaining the current data row in the feature map includes: Obtain the plurality of data rows from the cached feature map, and cache one data row from each of the plurality of data rows; One data row from the plurality of data rows is obtained to obtain the current data row in the feature map.

8. The method according to claim 7, characterized in that, The step of obtaining the current data row in the feature map and shifting the data in the current data row to obtain at least two first data rows includes: Each of the plurality of data rows is shifted by the same step size to obtain a third data row corresponding to each of the plurality of data rows. The third data row corresponding to the data row is one of at least two first data rows corresponding to the data row. Obtain the current third data row for each of the multiple data rows, and obtain the weight corresponding to the current sub-feature map; Based on the current third data row, the weight corresponding to the current sub-feature map, and the existing row accumulation result of the multiple sub-feature maps obtained from the row accumulation register group, a multiplication and addition operation is performed to obtain the current row accumulation result of the multiple sub-feature maps. The current sub-feature map is the sub-feature map to which the current third data row belongs. The row accumulation result of the multiple sub-feature maps corresponds to the row convolution calculation result of the feature map. The final accumulation result of the multiple sub-feature maps is the convolution calculation result of the feature map. Obtain the current row accumulation result corresponding to the multiple sub-feature maps corresponding to the convolution calculation result of the first row of the feature map, and use the current row accumulation result corresponding to the multiple sub-feature maps corresponding to the convolution calculation result of the first row of the feature map to update the existing row accumulation result corresponding to the multiple sub-feature maps corresponding to the convolution calculation result of the first row of the feature map.

9. A computer-readable storage medium, characterized in that, The method includes a computer program, characterized in that, when the computer program is run on an electronic device, it causes the electronic device to perform the method as described in any one of claims 5 to 8.

10. A computer program product, characterized in that, The method includes a computer program that, when executed by an electronic device, causes the electronic device to perform the method described in any one of claims 5 to 8.

Images