Computing device, method of performing convolution operation using computing device, and related products

Abstract

The present disclosure discloses a computing device, a method for performing convolution operation by using the computing device, and related products. The computing device can be included in a combined processing device, which can further include an interface device and other processing devices. The computing device interacts with the other processing devices to jointly complete a user-specified computing operation. The combined processing device can further include a storage device connected to the computing device and the other processing devices respectively, for storing data of the computing device and the other processing devices. The scheme of the present disclosure optimizes the convolution operation and improves the operation processing efficiency.

Description

Technical Field

[0001] This disclosure generally relates to the field of data processing. More specifically, this disclosure relates to a computing device configured to perform convolution operations, a method for performing convolution operations using the computing device, a chip, and a board. Background Technology

[0002] Currently, deep learning has become an important branch of machine learning and is greatly promoting the development of artificial intelligence (AI). The core technology of deep learning—deep neural networks (DNNs)—has been widely applied in many industries.

[0003] Neural networks are among the most critical technologies in artificial intelligence and deep learning, with Convolutional Neural Networks (CNNs) being the most important type. The most crucial computation in CNNs is the convolution operation within the convolutional layers (Conv layers). The function of convolutional layers is to extract features from input data. Through multiple convolutions, complex features can be extracted to ensure the network has sufficient expressive and generalization capabilities. Neural network models contain numerous convolution operations of various types, and the computational performance of these operations significantly impacts the overall computational performance of the neural network model. When neural network models are applied to different fields, such as speech recognition, machine translation, and image processing, the corresponding input feature maps and the dimensions of the weights may vary. To fully utilize the hardware advantages of deep learning processors, optimization is needed for convolution operations of different scales and types to improve the computational performance of the neural network model. Summary of the Invention

[0004] To address at least one or more of the technical problems mentioned above, this disclosure proposes a computing device in several aspects that, by segmenting the input feature map and weights into blocks, enables data of various dimensional dimensions to be adapted to the hardware of convolution operations, thereby improving the computational efficiency of convolution operations. The convolution operations in embodiments of this disclosure can be operations within various neural network models applicable to various fields, such as image processing, speech processing, text processing, etc., and these processes may include, but are not limited to, recognition and classification.

[0005] In a first aspect, embodiments of this disclosure provide a computing device configured to perform deep convolution operations in the reverse training of a neural network model. The computing device includes: a main processing circuit configured to: acquire input neuron data and / or neuron gradient data, wherein the input neuron data and neuron gradient data have been split into multiple split units according to a convolution splitting scheme, each split unit including data of a minimum storage dimension and at least one other storage dimension, and the total data volume of a split unit does not exceed the maximum hardware single-operation capacity, and the data within a split unit is stored consecutively as a data row; and multiple slave processing circuits configured to perform the deep convolution operation on corresponding data rows of the input neuron data and neuron gradient data.

[0006] In a second aspect, embodiments of this disclosure provide a chip that includes a computing device according to any of the embodiments of the first aspect.

[0007] In a third aspect, this disclosure provides a board that includes the chip of any of the embodiments of the second aspect above.

[0008] In a fourth aspect, embodiments of this disclosure provide a method for performing convolution operations using a computing device according to any of the embodiments of the first aspect.

[0009] By utilizing the computing device, chip, board, and method for performing convolution operations as described above, the solutions of this disclosure embodiment apply a convolution splitting scheme, enabling the input neuron data to adapt to the processing capabilities of the hardware computing device. This fully leverages the parallel processing capabilities of multiple processing circuits, effectively improving the computational efficiency of convolution operations. Furthermore, in some embodiments, the input feature map and weights can be transmitted through different data paths, supporting multiple reuse methods for the input feature map and weights, further optimizing convolution operations and reducing data access overhead. Attached Figure Description

[0010] The above and other objects, features, and advantages of exemplary embodiments of the present disclosure will become readily apparent upon reading the following detailed description with reference to the accompanying drawings. In the drawings, several embodiments of the present disclosure are illustrated by way of example and not limitation, and like or corresponding reference numerals denote like or corresponding parts, wherein:

[0011] Figure 1 This diagram shows the structure of the board card according to an embodiment of this disclosure;

[0012] Figure 2 This diagram illustrates the structure of the combined processing apparatus according to an embodiment of the present disclosure.

[0013] Figure 3A schematic diagram illustrating the internal structure of a processor core in a single-core or multi-core computing device according to embodiments of the present disclosure;

[0014] Figures 4a-4c This document illustrates several exemplary examples of convolution operation principles that can be applied to embodiments of this disclosure;

[0015] Figure 5 A schematic structural block diagram of a computing device according to an embodiment of the present disclosure is shown;

[0016] Figure 6 An exemplary data storage order according to an embodiment of this disclosure is shown;

[0017] Figures 7a-7d Several exemplary grouping patterns according to embodiments of this disclosure are shown;

[0018] Figure 8 An exemplary splitting diagram of an input feature map according to an embodiment of this disclosure is shown;

[0019] Figures 9a-9d A schematic diagram of data storage in a second storage circuit according to an embodiment of this disclosure is shown;

[0020] Figures 10a-10b A schematic diagram showing the output point division of an arithmetic circuit according to an embodiment of this disclosure is shown;

[0021] Figure 11 A schematic diagram illustrating the splitting and storage of the Updated1 scheme according to an embodiment of this disclosure is shown;

[0022] Figure 12 This diagram illustrates data storage in the second storage circuit of the Update1 scheme according to an embodiment of this disclosure.

[0023] Figure 13 This diagram illustrates a single operation in the Update1 scheme according to an embodiment of this disclosure.

[0024] Figure 14 A schematic diagram of sliding convolution in the Update1 scheme according to an embodiment of this disclosure is shown; and

[0025] Figure 15 A schematic diagram of the output data format of the Update1 scheme according to an embodiment of this disclosure is shown. Detailed Implementation

[0026] The technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, not all of them. Based on the embodiments in this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0027] It should be understood that the terms "first," "second," "third," and "fourth," etc., that may appear in the claims, specification, and drawings of this disclosure are used to distinguish different objects, rather than to describe a specific order. The terms "comprising" and "including" as used in the specification and claims of this disclosure indicate the presence of the described features, integrals, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or collections thereof.

[0028] It should also be understood that the terminology used in this disclosure is for the purpose of describing particular embodiments only and is not intended to limit the disclosure. As used in this disclosure and claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this disclosure and claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes such combinations.

[0029] As used in this specification and claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection."

[0030] Exemplary hardware environment

[0031] Figure 1 A schematic diagram of the structure of a board 10 according to an embodiment of this disclosure is shown. Figure 1 As shown, board 10 includes chip 101, which is a system-on-chip (SoC) integrating one or more combined processing units. These combined processing units are artificial intelligence computing units used to support various deep learning and machine learning algorithms, meeting the intelligent processing needs of complex scenarios in fields such as computer vision, speech, natural language processing, and data mining. In particular, deep learning technology is widely used in cloud intelligence. A significant characteristic of cloud intelligence applications is the large volume of input data, placing high demands on the platform's storage and computing capabilities. Board 10 in this embodiment is suitable for cloud intelligence applications, possessing massive off-chip storage, on-chip storage, and powerful computing capabilities.

[0032] Chip 101 is connected to external device 103 via external interface device 102. External device 103 may be, for example, a server, computer, camera, monitor, mouse, keyboard, network card, or Wi-Fi interface. Data to be processed can be transmitted from external device 103 to chip 101 via external interface device 102. The calculation results from chip 101 can be transmitted back to external device 103 via external interface device 102. Depending on the application scenario, external interface device 102 may have different interface forms, such as a PCIe interface.

[0033] The board 10 also includes a storage device 104 for storing data, which includes one or more memory cells 105. The storage device 104 is connected to and transmits data with the controller 106 and the chip 101 via a bus. The controller 106 in the board 10 is configured to regulate the state of the chip 101. Therefore, in one application scenario, the controller 106 may include a microcontroller (MCU).

[0034] Figure 2 This is a structural diagram illustrating the combined processing device in chip 101 of this embodiment. (As shown) Figure 2 As shown, the combined processing device 20 includes a computing device 201, an interface device 202, a processing device 203, and a storage device 204.

[0035] The computing device 201 is configured to perform user-specified operations. It is mainly implemented as a single-core intelligent processor or a multi-core intelligent processor to perform deep learning or machine learning calculations. It can interact with the processing device 203 through the interface device 202 to jointly complete the user-specified operations.

[0036] Interface device 202 is used to transmit data and control commands between computing device 201 and processing device 203. For example, computing device 201 can obtain input data from processing device 203 via interface device 202 and write it to on-chip storage device of computing device 201. Further, computing device 201 can obtain control commands from processing device 203 via interface device 202 and write them to on-chip control cache of computing device 201. Alternatively or optionally, interface device 202 can also read data from storage device of computing device 201 and transmit it to processing device 203.

[0037] Processing device 203, as a general-purpose processing device, performs basic control including but not limited to data transfer, and starting and / or stopping computing device 201. Depending on the implementation, processing device 203 may be one or more types of processors, including but not limited to digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., and their number can be determined according to actual needs. As mentioned above, computing device 201 disclosed herein can be considered as having a single-core structure or a homogeneous multi-core structure. However, when computing device 201 and processing device 203 are considered together, they are considered to form a heterogeneous multi-core structure.

[0038] Storage device 204 is used to store data to be processed. It may be DRAM or DDR memory, typically 16G or larger in size, and is used to store data of computing device 201 and / or processing device 203.

[0039] Figure 3 The diagram shows the internal structure of the processing core when the computing device 201 is a single-core or multi-core device. The computing device 301 is used to process input data such as computer vision, speech, natural language, and data mining. The computing device 301 includes three main modules: a control module 31, a processing module 32, and a storage module 33.

[0040] The control module 31 coordinates and controls the operation of the computation module 32 and the storage module 33 to complete the deep learning task. It includes an instruction fetch unit (IFU) 311 and an instruction decode unit (IDU) 312. The instruction fetch unit 311 fetches instructions from the processing device 203, and the instruction decode unit 312 decodes the fetched instructions and sends the decoding result as control information to the computation module 32 and the storage module 33.

[0041] The computation module 32 includes a vector operation unit 321 and a matrix operation unit 322. The vector operation unit 321 is used to perform vector operations and can support complex operations such as vector multiplication, addition, and nonlinear transformations; the matrix operation unit 322 is responsible for the core computations of deep learning algorithms, namely matrix multiplication and convolution.

[0042] The storage module 33 is used to store or move relevant data, including a neuron RAM (NRAM) 331, a weight RAM (WRAM) 332, and a direct memory access (DMA) module 333. The NRAM 331 stores the input neurons, output neurons, and intermediate results after computation; the WRAM 332 stores the convolution kernels of the deep learning network, i.e., the weights; the DMA 333 is connected to the DRAM 204 via bus 34 and is responsible for data transfer between the computing device 301 and the DRAM 204.

[0043] Exemplary convolution operation types

[0044] Based on the aforementioned hardware environment, in one aspect, this disclosure provides a computing device configured to perform convolution operations, thereby optimizing convolution operations, for example, in neural network models. Convolutional layers in a neural network model can perform convolution operations by applying convolution kernels (also called filters, weights, etc.) to an input feature map (also called input data, neurons, or input neurons) to extract features. A convolutional layer may contain multiple convolution kernels, each element of which corresponds to a weight coefficient and a bias.

[0045] Neural network models may contain various convolutional layers, such as convolutional layers that perform forward, regular 3D convolutional operations, and deconvolutional layers that perform depthwise convolutional operations. In reverse training, it may be necessary to perform inverse depthwise convolutional operations or cross-product convolutional operations. The embodiments disclosed herein can be optimized for these different types of convolutional operations.

[0046] In conventional 3D convolution operations, assuming the input feature map tensor shape in the convolutional layer is represented by X[N Hi Wi Ci], the tensor shape of the convolutional kernel is represented by K[Co Kh Kw Ci], and the output is Y[N HoWo Co], then the simplified mathematical formula for convolution operations can be expressed as follows:

[0047] Y in,jc,jh,jw =∑ 0≤ic≤ci,0≤ih≤kh,0≤iw≤kw X in,ic,jh×sh+ih,jw×sw+iw ×K jc,ic,ih,iw (1)

[0048] In the above formula, X is the input data, Y is the output data, K is the convolution kernel, Kh and Kw are the length and width of K, and sh and sw are the strides in the length and width directions. The formula ignores the bias, padding, and dilation, and assumes that the input data X has been padded and the convolution kernel has been dilated. The formula ignores the N and C dimensions. The forward computation of the neural network model is independent in the N dimension, but fully connected in the C dimension. When the convolution kernel works, it scans the input features with a certain stride, performs matrix element-wise multiplication and sums the bias within the convolution window. In conventional 3D convolution operations, the results of the alignment multiplications in the H, W, and Ci directions are accumulated, hence the name 3D convolution. However, this 3D convolution has a constraint: the Ci dimension of the convolution kernel is equal to the Ci dimension of the input feature map, so the convolution kernel does not slide in the Ci direction, making it a pseudo-3D convolution. To distinguish it from other convolution operations in this paper, the above convolution operation is referred to as 3D convolution operation.

[0049] Figure 4a An example of a conventional 3D convolution operation principle that can be applied to embodiments of this disclosure is shown.

[0050] The figure exemplifies a four-dimensional input data X of size [N Hi Wi Ci], which can be represented as N Hi×Wi×Ci 3D rectangles 410a. The figure also exemplifies a four-dimensional convolution kernel K of size [Co Kh Kw Ci], which can be represented as Co Kh×Kw×Ci 3D convolution kernels 420a. The convolution result of the input data X and the convolution kernel K yields the output data Y, which is a four-dimensional data of size [N Ho Wo Co], which can be represented as N Ho×Wo×Co 3D rectangles 430a.

[0051] The figure also shows a specific example of a convolution operation, where the input data is a 6×6×3 input feature map 440a, omitting the N dimension; the convolution kernel is a 3×3×3 stereo convolution kernel 450a, targeting a single convolution kernel Co; and the output data is a 4×4 output feature map 460a. The specific operation process is as follows:

[0052] The convolution kernel 450a scans across the input feature map 440a with a certain stride, performs element-wise matrix multiplication on the input features within the convolution window 470a, and then sums the bias values. That is, the value at each position in the output feature map 460a is obtained by performing a two-dimensional convolution operation between the corresponding block of each input feature map and the corresponding convolution kernel, and then summing the results. For example, the figure shows that the value at position (0,0) on the output feature map 460a (i.e., the convolution output point) is obtained by performing a two-dimensional convolution operation between the convolution window 470a (framed by the black cube in the input feature map) and the 3D convolution kernel 450a, resulting in three values, which are then summed to obtain the final value.

[0053] To obtain outputs at other locations, the position of the convolution kernel 450a can be moved on the input feature map 440a, which means moving the convolution window of the output point. In the example in the figure, the convolution stride (Sx, Sy) is (1,1). When the convolution operation is performed after moving one grid to the right horizontally (width direction) or downward vertically (height direction), the values ​​at positions (0,1) or (1,0) on the output feature map 460a can be obtained respectively.

[0054] As described above, a convolutional layer in a neural network contains N sets of input feature maps, each containing Hi × Wi × Ci information, where Hi and Wi are the height and width of the input feature map, respectively, and Ci is the number of input feature maps, also known as the number of input channels. The convolutional layer has Ci × Co convolutional kernels of size Kh × Kw, where Ci is the number of input channels, Co is the number of output feature maps (or output channels), and Kh and Kw are the height and width of the convolutional kernel, respectively. The output feature map contains Ho × Wo × Co information, where Ho and Wo are the height and width of the output feature map, respectively, and Co is the number of output channels. Furthermore, the convolutional operation also involves the convolution stride (Sx, Sy), the size of which affects the size of the output feature map.

[0055] Figure 4b An exemplary example of the principle of depthwise convolution operations that can be applied to embodiments of this disclosure is shown.

[0056] The difference between depthwise convolution and regular 3D convolution is that the latter does not accumulate along the depth direction, which refers to the input channels Ci. In regular 3D convolution, each convolutional kernel needs to be calculated and accumulated with all layers (input channels) of the input feature map, so the number of input channels for each convolutional kernel is equal to the number of input channels in the input feature map. In depthwise convolution, however, each convolutional kernel is single-channel; one kernel is responsible for one channel, and one channel is convolved by only one convolutional kernel. Therefore, depthwise convolution is sometimes called 2D convolution, meaning it only slides and accumulates along the H and W dimensions.

[0057] As shown in the figure, the input feature map 410b has dimensions of 12×12×3, meaning it includes three channels, each containing a 12×12 image. This depthwise convolution uses three convolutional kernels 420b, each a single-channel kernel with dimensions, for example, 5×5×1. Each kernel convolves only one channel of the input feature map 410b, producing an 8×8×1 output each time. These outputs are then stacked to create an 8×8×3 image, resulting in an 8×8×3 output feature map 430b. As can be seen from the figure, the depth (number of channels) of the output feature map remains consistent with the input feature map. Figure 1 To.

[0058] Since the input channels are not accumulated in depthwise convolution, the dimensions of the input feature map, convolution kernel, and output feature map can be simplified to three dimensions: C (channels), H (height), and W (width).

[0059] In the backpropagation of neural network model training, the calculation of neuron gradients and weight gradients is involved, as shown below:

[0060]

[0061]

[0062] Where top_diff and bottom_diff are the neuron gradients, W is the weight in this iteration, and ΔW is the weight gradient calculated in this iteration. This is the computation in backpropagation, similar to convolution. Relative to the backpropagation direction, the bottom_diff of the previous layer is the top_diff of the current layer, and the bottom_diff of the current layer is the top_diff of the next layer, thus allowing the error to be propagated layer by layer in reverse.

[0063] In the calculation of formula (2), the operation between top_diff and W is similar to the operation between the input neuron and the weight W, where top_diff is equivalent to the input feature map.

[0064] In the calculation of formula (3), the operation between top_diff and bottom_data is similar to the depthwise convolution operation, where top_diff is equivalent to the convolution kernel, which slides and accumulates in the XY direction of bottom_data. The operation principle can be found in [reference]. Figure 4b In this computational scenario, the sizes of top_diff and bottom_data are typically quite large. Therefore, this disclosed embodiment also provides an optimization scheme for convolution operations (referred to as inverse depthwise convolution) in this scenario.

[0065] In backpropagation, for a convolutional layer performing conventional 3D convolution operations, the operation in the reverse process can be called a cross-product convolution operation. The embodiments disclosed herein can also provide optimization schemes for this type of convolution operation.

[0066] Figure 4c An example of the cross-product convolution operation principle that can be applied to the embodiments of this disclosure is shown.

[0067] The figure exemplifies the 3D data `top_diff` of size [Ho Wo Co], which can be represented as a 3D rectangle 410c of size Ho × Wo × Co. The figure also shows 3D data `bottom_data` of size [Hi Wi Ci], which can be represented as a 3D rectangle 420c of size Hi × Wi × Ci. Performing a cross product convolution operation between `top_diff` and `bottom_data` yields output data 430c, which is 4D data of size [Co Kh Kw Ci], and can be represented as Co Kh × Kw × Ci 3D rectangles 430c. Figure 4a The comparison shows that Figure 4c The cross-product convolution is equivalent to the reverse operation of a regular 3D convolution, that is, the convolution kernel is calculated using the output feature map (top_diff) and the input feature map (bottom_data). Figure 4c The N dimension has been omitted.

[0068] Specifically, for each HoWo facet in top_diff, i.e., for each HoWo facet with a Co value, Ci copies are made, resulting in a Ho×Wo×Ci data set 440c. This 440c data set is then subjected to a depthwise convolution operation with bottom_data (see reference). Figure 4b (See the diagram). This means that the Ci direction is not accumulated, resulting in an output of 460c, which is a 3D data of size Kh×Kw×Ci. For each HoWo surface, this copying and depthwise convolution operation is repeated to obtain Co 3D data of size Kh×Kw×Ci, which is equivalent to obtaining a four-dimensional convolution kernel 430c, with a size of Co×Kh×Kw×Ci.

[0069] In this paper, the terms input feature map, input data, neuron, or input neuron are interchangeable; convolution kernel, filter, or weights are interchangeable. Furthermore, the H (height) and Y dimensions are interchangeable, as are the W (width) and X dimensions. Accordingly, the H dimension of the input feature map can be represented as Hi or Yi, the H dimension of the output feature map can be represented as Ho or Yo, and the W dimension is represented similarly. In the disclosed embodiment, each convolution output point has a corresponding convolution window, the shape of which is equal to the shape of the convolution kernel. The value of each convolution output point corresponds to the positional multiplication and accumulation result of the input feature map and weights within its convolution window. Furthermore, regardless of the type of convolution operation involved, the data involved can be divided into input feature map, convolution kernel, and output feature map. For example, in the inverse operation, top_diff corresponds to the convolution kernel, bottom_data corresponds to the input feature map, and ΔW corresponds to the output feature map.

[0070] Exemplary computing device

[0071] In this disclosed embodiment, a master-slave computing device can be used to perform the above-described convolution operation. Furthermore, different data paths can be configured for the input feature map and the convolution kernel to improve memory access efficiency.

[0072] Figure 5 A schematic structural block diagram of a computing device 500 according to an embodiment of this disclosure is shown. It can be understood that this structure can be considered as... Figure 3 The detailed internal structure of the processing module of a single processing core can also be viewed as a refinement of multiple processing cores. Figure 3 The diagram shows a functional partitioning based on the processing core's computational modules. Figure 5 As shown, the computing device 500 of this disclosure embodiment can be configured to perform various types of convolution operations, and may include a main processing circuit (MA) 510 and a plurality of slave processing circuits (SL) 520. Sixteen slave processing circuits SL0 to SL15 are shown in the figure. Those skilled in the art will understand that the number of slave processing circuits may be more or less, depending on the specific hardware configuration, and this disclosure embodiment is not limited in this respect.

[0073] The master processing circuit and slave processing circuits, as well as multiple slave processing circuits, can communicate with each other through various connections. In different application scenarios, the connection between multiple slave processing circuits can be either a hard connection arranged by hardwired lines or a logical connection configured according to, for example, microinstructions, to form a topology of various slave processing circuit arrays. The embodiments disclosed herein are not limited in this respect. The master processing circuit and slave processing circuits can cooperate with each other to achieve parallel processing.

[0074] To support computational functions, the main processing circuit and the slave processing circuit can include various computing circuits, such as vector operation units and matrix operation units. The vector operation unit is used to perform vector operations and can support complex operations such as vector multiplication, addition, and nonlinear transformations; the matrix operation unit is responsible for the core computations of deep learning algorithms, such as matrix multiplication and convolution.

[0075] The processing circuit can, for example, perform intermediate operations on the corresponding data in parallel according to the operation instructions to obtain multiple intermediate results, and then transmit the multiple intermediate results back to the main processing circuit.

[0076] By configuring the computing device 500 into a master-slave structure (e.g., a master-multiple-slave structure, or a multi-master-multiple-slave structure, which is not limited in this disclosure), for forward computation instructions, the data can be split according to the computation instructions, thereby enabling multiple slave processing circuits to perform parallel computation on the computationally intensive parts to improve computation speed, save computation time, and thus reduce power consumption.

[0077] In some embodiments disclosed herein, by utilizing different data paths to transmit input feature maps and weights, multiple reuse methods for input feature maps and weights can be supported, thereby reducing data access during computation and improving processing efficiency.

[0078] Specifically, the computing device 500 may further include a first storage circuit 530 and a second storage circuit 540 for storing data transmitted via different data channels, respectively. Optionally, the first storage circuit 530 and the second storage circuit 540 may be two storage blocks formed by partitioning the same memory, or they may be two independent memories; no specific limitation is made here.

[0079] The first storage circuit 530 can be used to store multicast data, meaning that the data in the first storage circuit will be transmitted to multiple slave processing circuits via a broadcast bus, and these slave processing circuits will receive the same data. It can be understood that broadcasting and multicasting can be implemented via a broadcast bus. Multicast refers to a communication method that transmits a single data set to multiple slave processing circuits; while broadcasting is a communication method that transmits a single data set to all slave processing circuits, and is a special case of multicast. Since both multicast and broadcasting correspond to one-to-many transmission methods, this document does not specifically distinguish between the two; broadcasting and multicast can be collectively referred to as multicast, and those skilled in the art can understand their meaning from the context.

[0080] The second storage circuit 540 can be used to store and distribute data, that is, the data in the second storage circuit will be transmitted to different slave processing circuits respectively, and each slave processing circuit receives different data.

[0081] By providing a first storage circuit and a second storage circuit respectively, it is possible to support the transmission of data to be processed in different transmission methods, thereby reducing the amount of data access by multiplexing multicast data among multiple slave processing circuits.

[0082] In some embodiments, one of the input feature map and the convolution kernel can be determined as multicast data and stored in a first storage circuit to transmit the data to multiple scheduled slave processing circuits via broadcast during computation. Correspondingly, the other of the input feature map and the convolution kernel can be determined as distribution data and stored in a second storage circuit. This distribution data can be distributed to the corresponding slave processing circuits before computation.

[0083] Figure 5 A schematic diagram of the internal structure of the slave processing circuit SL according to an embodiment of this disclosure is also shown. As shown, each slave processing circuit 520 may include multiple arithmetic circuits CU 521, a first buffer circuit 522, and a second buffer circuit 523. Four arithmetic circuits CU0 to CU3 are shown in the figure. Those skilled in the art will understand that the number of arithmetic circuits may be more or less, depending on the specific hardware configuration, and the embodiments of this disclosure are not limited in this respect.

[0084] In some embodiments, the first buffer circuit 522 can be used to buffer the weights or input feature maps allocated to the slave processing circuit. Correspondingly, the second buffer circuit 523 can be used to buffer the input feature maps or weights allocated to the slave processing circuit. Both buffer circuits are used to select data for computation. The data in the first buffer circuit 522 can be multiple data rows from, for example, the first storage circuit 530 or the second storage circuit 540; correspondingly, the data in the second buffer circuit 523 can be multiple data rows from, for example, the second storage circuit 540 or the first storage circuit 530. Depending on the specific multiplexing method, these data rows can be distributed to the corresponding computation circuit CU 521 or broadcast to all CUs 521 within the slave processing circuit 520 during computation.

[0085] Each arithmetic circuit CU 521 is used to perform bitwise multiplication and accumulation operations on data rows selected from the first buffer circuit and data rows selected from the second buffer circuit, respectively, in each arithmetic cycle.

[0086] By providing a first buffer circuit and a second buffer circuit respectively, it is possible to support the transmission of data to be processed in different transmission methods, thereby reducing the amount of data access by reusing data as much as possible among multiple processing circuits within a single slave processing circuit.

[0087] The processing circuit 520 may also include a third buffer circuit 524 for buffering the calculation results of each arithmetic circuit CU 521.

[0088] Understandable, although Figure 5 The various processing circuits and storage circuits are shown as separate modules, but depending on the configuration, the storage circuits and processing circuits can also be combined into a single module. For example, the first storage circuit 530 can be combined with the main processing circuit 510, while the second storage circuit 540 can be shared by multiple slave processing circuits 520, with each slave processing circuit allocated an independent storage area to accelerate access. This disclosure does not limit the embodiments in this respect. Furthermore, in this computing device, the main processing circuit and slave processing circuits can belong to different modules of the same processor or chip, or they can belong to different processors; this disclosure also does not limit this in this respect.

[0089] Exemplary data splitting and storage

[0090] In this disclosed embodiment, the dimensions of the multidimensional data are represented as (N, H, W, C) or (Co, H, W, Ci), which represent the storage order of the data in memory. It is understood that although multidimensional data has multiple dimensions, because the layout of memory is always one-dimensional, there is a correspondence between the multidimensional data and the storage order in memory. Multidimensional data is usually allocated in contiguous storage space, meaning that multidimensional data can be unfolded one-dimensionally and stored sequentially in memory. For example, in this disclosed embodiment, the initial input feature maps can be stored sequentially in a low-dimensional priority manner (here, C / Ci is the lowest dimension); and to optimize convolution operations, the storage order of the input feature maps can be adjusted during or before the operation, as will be described in detail later. Adjacent dimensions refer to dimensions that are immediately adjacent in the dimensional information representation of multidimensional data; for example, W and Ci are adjacent. Adjacent dimensions can also be called continuous dimensions.

[0091] In intelligent processors, due to computational power requirements and considerations of area and power consumption, the main hardware operation unit is the vector multiply-accumulate unit. Implementing support for various convolution algorithms in hardware design essentially involves maximizing the extraction of multiply-accumulate operations from the algorithms and implementing them in on-chip RAM (such as...) via data paths. Figure 3 It efficiently exchanges input and output data for multiplication and addition operations between NRAM, WRAM, etc. and the arithmetic unit.

[0092] Hardware stores data line by line (cache line). Read, write, and computation operations are most efficient when aligned to the entire line. Therefore, to fully utilize bandwidth and accommodate the memory access demands of the arithmetic unit array, data is typically vectorized and aligned. Artificial intelligence chips are usually designed with the Ci dimension as the lowest dimension, i.e., the NHWC arrangement mentioned above, where data along the Ci dimension is continuous. Therefore, vectorization alignment requires the Ci dimension to be aligned to a specified value, such as alignment value M, so that data can be accessed in units of this alignment value M. M can also be called the maximum number of operations per hardware operation. Depending on the hardware design, M can have different values, such as 64 bits, 128 bits, 256 bits, 512 bits, etc. Typically, the size of the arithmetic unit array's input ports is also related to M. For example, in the case of symmetrical input data bit width, the input port size of the arithmetic unit array is usually twice M, meaning it can process input feature map data and weight data of alignment value M in one operation. When the Ci dimension of the input feature map is large, it is easier to meet the above alignment requirements.

[0093] When the Ci dimension of the input feature map is small, such as less than the size of a cache line, the Ci dimension needs to be padded to a single line of data (e.g., 512 bits), i.e., filled with invalid data 0. This padding causes a lot of redundant computation, resulting in wasted resources and reduced computational efficiency.

[0094] In this disclosed embodiment, a convolution operation scheme is proposed, which can be, for example, by... Figure 5 The computational device performs this operation. The main processing circuitry acquires the input feature map and / or convolutional kernel. The input feature map and convolutional kernel have been split into multiple split units according to a convolution splitting scheme, and their dimensionality storage order has been transformed so that data within a split unit is stored consecutively as a single data row. Depending on different hardware configurations and / or other considerations, the aforementioned splitting and dimensionality transformation of the input feature map and convolutional kernel can be performed at different locations and times. During the backpropagation of neuron gradient updates, top_diff can be considered as the input feature map.

[0095] In some embodiments, the main processing circuit may include a segmentation circuit, i.e., the segmentation circuit is integrated into the main processing circuit, for splitting and dimensionality-transforming the input feature map and convolutional kernel separately for storage. For example, the main processing circuit can read the input feature map and convolutional kernel in their original storage format from an external storage circuit (e.g., DDR), and then use the segmentation circuit to split and dimensionalize the input feature map and convolutional kernel separately. Afterward, one of the input feature map and convolutional kernel is stored in a first storage circuit, while the other is stored in a second storage circuit. The above-described splitting process can be performed during computation or before computation to prepare the data.

[0096] In other embodiments, the main processing circuit may include a partial segmentation circuitry used only for splitting and dimensionality-transforming data identified as multicast data in the input feature map and convolutional kernels. Data identified as distribution data can be split and dimensionality-transformed via an external segmentation circuitry. For example, in one example, convolutional kernels identified as distribution data can be split and dimensionality-transformed by an external circuitry and pre-stored in a second storage circuitry. This storage can be directly transferred from an off-chip storage circuitry to the second storage circuitry, or it can be transferred via a first storage circuitry to the second storage circuitry.

[0097] In some other embodiments, the main processing circuit may not include the block circuitry at all or may not perform the functions of the block circuitry. In these embodiments, the input feature map and convolutional kernel are split and dimensionally transformed and stored by the block circuitry, which is independent of the main processing circuitry. One of the split and dimensionally transformed input feature map and convolutional kernel may be stored in a first storage circuitry, while the other may be stored in a second storage circuitry.

[0098] The corresponding convolution splitting scheme can be determined based on the minimum storage dimension (e.g., Ci) of the input feature map, where the convolution splitting scheme at least indicates the shape of the splitting unit of the data to be processed. The amount of data contained in a splitting unit does not exceed the maximum hardware computation in a single operation.

[0099] In some embodiments, the amount of data contained in a split unit can be set to the hardware's one-time processing alignment value M, so that computational processing can be performed on a unit basis, which can give full play to the hardware's computing power and avoid or reduce invalid computation.

[0100] In the exemplary description disclosed herein, it is assumed that M = 512 bits = 64 bytes, the data type can be Int8, Int16, Float16, or Float32, and the data type of the input feature map is consistent with that of the convolution kernel. Since the data type requires a width of at least 1 byte, and the smallest unit of computation is a single data point, various calculations in the following examples are performed in bytes, such as M = 64 bytes, Ci = 28 bytes, etc. Sometimes, units are omitted for the sake of brevity.

[0101] When the data size of a split unit is equal to M, the shape of each data block in the split unit is blockC*blockY*blockX, which may have several forms. Table 1 lists some of them:

[0102]

[0103] Table 1. Data Block Shapes

[0104] As can be seen from Table 1, some data block shapes have equal X and Y dimensions (as shown in the darker rows), and this shape can simplify subsequent calculations. Therefore, in this disclosed embodiment, this data block shape can be preferably used to split the data to be calculated.

[0105] For simplicity, the 64B×1×1 partitioning scheme is called Forward64, the 16B×2×2 partitioning scheme is called Forward16, the 4B×4×4 partitioning scheme is called Forward4, the 4B×4×4 partitioning scheme applied to depthwise convolution is called Forward1, the 4B×4×4 partitioning scheme applied to inverse depthwise convolution is called Update1, and the 4B×4×4 partitioning scheme applied to cross-product convolution is called Update4. Except for Forward64, these partitioning schemes are suitable for scenarios where the channel C in convolution computation is relatively small, and therefore can be collectively referred to as small convolutions. In these small convolution partitioning schemes, a partitioning unit includes data from the lowest storage dimension and at least one other storage dimension, and the total data volume of a partitioning unit does not exceed the maximum hardware computation time in a single operation.

[0106] Different convolution splitting schemes can be applied to different computational scenarios, thereby achieving varying degrees of performance optimization. Specifically, in some embodiments, the corresponding convolution splitting scheme can be determined according to at least one of the following rules:

[0107] Align the lowest storage dimension Ci of the input feature map before splitting to the nearest M / 4. n A multiple of M, where M is the maximum number of operations performed by the hardware in a single operation. And the size Uci (i.e., block C) of the split unit in the lowest storage dimension is determined to be M / 4. n ;

[0108] There are multiple nearest M / 4 n When the value is a multiple of M, take M / 4. n The maximum value is taken as Uci, or the minimum alignment padding value of M / 4 is taken. n As Uci; and

[0109] Determine the size of the split unit in the X and Y storage dimensions, Ux (i.e., blockX) and Uy (blockY), such that Uci × Uy × Ux = M, where preferably Ux = Uy.

[0110] The following examples illustrate the application of the above rules. In all examples, it is assumed that M = 64, then M / 4 n It can be 64, 16, or 4.

[0111] In one example, assuming Ci = 28, then it is aligned to the nearest M / 4. n The multiple is 4*7, so the size Uc (i.e., block C) of the split unit in the lowest storage dimension is determined to be 4. When Ux = Uy is preferred, the shape of the split unit can be determined to be 4B×4×4, which is the Forward4 scheme.

[0112] In another example, assuming Ci = 112, if aligned to 64 * 2 = 128, 16 zeros are added; if aligned to 16 * 7 = 112, no zeros are added; if aligned to 4 * 28 = 112, no zeros are added either. In this case, the nearest M / 4... n The multiples are 16 * 7 = 4 * 28 = 112. According to the rules, we can take M / 4. n The maximum value of 16 is taken as Uc. When Ux = Uy is preferred, the shape of the splitting unit can be determined as 16B×2×2, which is the Forward16 scheme.

[0113] After determining the splitting scheme, the input feature map and convolution kernel can be split into multiple corresponding splitting units according to the determined convolution splitting scheme, and their dimensional storage order can be transformed so that the data in a splitting unit is stored continuously as a data row, which facilitates subsequent reading and processing in units of splitting units (data rows).

[0114] In some embodiments, for three-dimensional or four-dimensional neuron or weight data, it is divided into data blocks of size blockC*blockY*blockX (Uc×Uy×Ux), and each data block is stored consecutively on a row, for example, M=64B, so that when reading a row of data, the data of one data block is actually retrieved.

[0115] Specifically, one or more split units can be read from the data to be processed stored in the first dimension storage order, in the order of the first reading, and the read split units can be stored on the corresponding storage circuit. The data in each split unit is stored in the second dimension storage order, and the split units are stored in the third dimension storage order.

[0116] Figure 6 An exemplary data storage order according to an embodiment of this disclosure is shown.

[0117] As shown in the figure, 610 represents the storage method of the four-dimensional tensor to be computed, which contains N three-dimensional sub-tensors, with N in the highest dimension. Therefore, the storage order of the first dimension of the four-dimensional tensor is NHWC. Note that H and Y, W and X are interchangeable in this paper. Each sub-tensor is divided into smaller data blocks or split units, with the number of data blocks in each dimension being C / Y / X, respectively.

[0118] Figure 620 in the middle illustrates the storage method for each sub-tensor. Each data block is stored as a contiguous 64-byte row. The order in which data blocks are read changes the order of the rows. In the example shown, data blocks are read in the order of C, then X, and finally Y (the first read order is YXC). Therefore, the rows are stored in the order Y*X*C, meaning the third-dimensional storage order is either YXC or HWC. In this example, the third-dimensional storage order is the same as the first-dimensional storage order. It is understandable that other read orders can be used, resulting in a different third-dimensional storage order than the first-dimensional storage order; these will not be listed here.

[0119] Figure 630 on the right shows the order within each row, that is, the data order within each data block, with a shape of blockC*blockY*blockX. In this case, the storage order of the second dimension is CYX or CHW. The specific splitting schemes will be described in detail later with examples of various exemplary convolution splitting schemes.

[0120] Exemplary grouping operations and data reuse

[0121] The preceding description of the hardware structure of the computing device and exemplary data partitioning and storage methods of the embodiments disclosed herein provides different data paths for the input feature maps and weights involved in the computation, thereby reducing data access during computation and improving computational efficiency by utilizing different data transmission methods (e.g., broadcasting, multicasting, distribution, etc.). Convolution computation requires each input feature map to be multiplied and added with each Co convolution kernel to output Co output feature maps. However, on-chip space cannot necessarily store convolution kernels and input feature maps of all sizes simultaneously. Therefore, for the hardware, there is a series of operations involving repeatedly loading input feature data or weight data. How to balance repeatedly loading input feature data or weight data will have a certain impact on computational efficiency. In actual computation, to reduce frequent off-chip memory access, different reuse methods can be adopted according to the scale characteristics of the data involved in the computation. In convolution operations, there are two main data reuse methods: convolution kernel reuse and input feature map reuse.

[0122] Depending on the scenario, convolutional kernel reuse can be divided into intra-channel convolutional kernel reuse and inter-batch convolutional kernel reuse. Intra-channel convolutional kernel reuse is for a single output channel, i.e., a single output feature map, in which case there is only one set of convolutional kernels. For each input feature map, multiple convolutional windows can reuse the same convolutional kernels. Inter-batch convolutional kernel reuse is for batch processing, i.e., processing multiple input images simultaneously. Multiple input images are processed using the same set of convolutional kernels, thus allowing kernel reuse.

[0123] Similarly, depending on the reuse scenario, input feature map reuse can be divided into intra-channel input feature map reuse and inter-channel input feature map reuse. Intra-channel input feature map reuse is for a single output channel. For each input feature map, its adjacent convolutional windows can reuse part of the input feature map data. Inter-channel input feature map reuse is for multiple output channels, that is, when there are multiple output feature maps (i.e., multiple sets of convolutional kernels). In this case, the input feature map within a convolutional window can perform convolution operations with multiple sets of convolutional kernels.

[0124] As described above regarding the principles of convolution, the results of operations on the Co dimension (C dimension for depthwise convolution) do not need to be accumulated. Therefore, operations on different Co dimensions can be performed relatively independently on different processing circuits. In scenarios with a small number of input channels, the convolution kernels are generally also small; for example, Kh and Kw are usually single digits, and the sizes of Co and Ci are similar. In these embodiments, the size of the output channel Co dimension of the convolution kernel in a single round of operation typically does not exceed the number of scheduled slave processing circuits. Therefore, the operation of a single Co dimension needs to be completed by one or more slave processing circuits. More generally, even when the Co dimension is large, it can be achieved by splitting the operation into multiple rounds, where the Co dimension processed in each round does not exceed the number of scheduled slave processing circuits. Thus, in one example, the number of rounds required to complete the convolution operation and the number of Co dimensions processed in each round, or the corresponding grouping pattern, can be determined first based on the output channel Co dimension of the convolution kernel and the number of scheduled slave processing circuits Ns.

[0125] When determining the number of operation rounds required to complete the convolution operation, the number of Co values ​​processed in each round may be different, so even for the same Co dimension, there may be multiple allocation methods.

[0126] For example, with Figure 5Taking a computing device with 16 slave processing circuits (SLs) as an example, assuming all slave processing circuits are schedulable, i.e., Ns = 16. When Co = 40, the operation can be divided into three rounds: the first round processes the first 16 Co values, with each SL processing one different Co value; the second round processes the next 16 Co values, with each SL processing one different Co value; and the last round processes the remaining 8 Co values, with every two SLs processing one different Co value. Alternatively, the operation can be divided into two rounds: the first round processes the first 32 Co values, with each SL processing two different Co values; and the last round processes the remaining 8 Co values, with every two SLs processing one different Co value. For example, when Co = 12, the operation can be divided into a single round, with each SL processing one different Co value, while four SLs are idle or performing invalid operations. In another allocation method, the operation can be divided into three rounds, processing four consecutive Co values ​​each time, with every four SLs processing one different Co value, thus utilizing all schedulable slave processing circuits in each round. It is understandable that those skilled in the art can conceive of many other allocation schemes.

[0127] Therefore, regardless of the allocation method, Co may have two allocation scenarios in a single round of computation: multiple slave processing circuits process a single Co value, or a single slave processing circuit processes one or more Co values. Specifically, in a single round of computation processing Nco output channels, each Rs SL constitutes a slave processing circuit group SLB, processing the convolution kernel corresponding to the same output Co value. Rs = [Ns / Nco], meaning the same convolution kernel is reused on Rs SL within the same SLB, where Rs represents the number of times the convolution kernel is reused among slave processing circuits. Correspondingly, the input feature map can be reused among different slave processing circuit groups SLB, Rn = [Ns / Rs], representing the number of times the input feature map is reused among slave processing circuits.

[0128] Optionally or additionally, when each slave processing circuit processes convolutional kernels corresponding to rn Co values, rn = [Nco / Ns], the input feature map processed by each slave processing circuit can be reused for rn convolutional kernels, where rn represents the number of times the input feature map is reused within a single slave processing circuit. Factors such as hardware buffer space limitations (e.g.) can be considered. Figure 5 The size of the first and second buffer circuits in the circuit determines the maximum number of convolution kernel reuses rs and the maximum number of input feature map reuses rn that can be applied within a single processing circuit.

[0129] Considering the limitations of cache size and the benefits of reuse in hardware circuits, in some embodiments disclosed herein, the case where a slave processing circuit processes multiple Co values ​​in a single round of operation is temporarily not considered, but only the case where one or more slave processing circuits process only one Co value in a single round of operation is considered.

[0130] Different grouping patterns can be adopted depending on the number of slave processing circuits (SLs) that process the same Co value in a single round of computation. It is understood that it is preferable to distribute the callable slave processing circuits (SLs) evenly to balance computing power; for example, grouping them into groups of two, so that 16 SLs can process 8 Co values ​​simultaneously; or grouping them into groups of four, so that 16 SLs can process 4 Co values ​​simultaneously; and so on. In some embodiments, for Figure 5 The computing device shown includes Ns = 16 SLs and can select the following grouping modes: Group1 mode, Group4 mode, and Group16 mode. Those skilled in the art will understand that different grouping modes are possible depending on the value of Ns, and each grouping mode can be processed accordingly by referring to the three representative grouping modes given above in this document.

[0131] In some embodiments, the above grouping pattern can be uniformly represented as GroupN, which means that all slave processing circuits (SLs) scheduled in the current round of operation are divided into N groups, each slave processing circuit group (SLB) processes the same Co value, and different slave processing circuit groups (SLBs) process different Co values. In the case of a total of 16 schedulable SLs, N can be 1, 4, or 16, corresponding to Group1, Group4, and Group16 above, respectively.

[0132] Figures 7a-7d Several exemplary grouping patterns according to embodiments of this disclosure are shown. Figure 7a The Group1 pattern is shown. Figure 7b The Group16 pattern is shown. Figure 7c A Group4 pattern is shown, and Figure 7d Another Group4 pattern is shown.

[0133] like Figure 7a As shown, Group1 mode means that all 16 schedulable SLs belong to one group and jointly process a Co value. For example, SL0 to SL15 belong to group G0. Thus, the operation for this output channel is distributed across the 16 SLs. In this mode, it is preferable to broadcast the convolution kernel 720 of the output channel to each SL, while the input feature map 710 is split and distributed to each SL, thereby improving memory access efficiency.

[0134] In one embodiment, the convolution kernel can be stored in Figure 5The input feature map is stored in the first storage circuit 530 for transmission via a broadcast channel. The input feature map can be partitioned according to the XY direction of the output feature map and stored in the second storage circuit 540 for allocation to different SLs. Thus, all SLs jointly compute an output feature map of Co. The partitioning and storage of the input feature map will be described in detail later with reference to the accompanying drawings.

[0135] like Figure 7b As shown, the Group16 mode means that all 16 schedulable input feature maps (SLs) are divided into 16 groups, that is, one SL per group, and each SL processes a different Co value. For example, SL0 belongs to group G0, SL1 belongs to group G1, and so on, until SL15 belongs to group G15. In this mode, the same input feature map 730 can be reused among the 16 SLs, so it is preferable to broadcast the input feature map 730 to each SL, while the convolutional kernels 740 corresponding to different Co values ​​are distributed to the corresponding SLs.

[0136] In one embodiment, the input feature map can be stored in Figure 5 The first storage circuit 530 is used for transmission via a broadcast channel. The convolution kernels are divided according to Co and stored in the second storage circuit 540 for allocation to different SLs. Thus, all SLs compute output feature maps with different Cos for the same input feature map.

[0137] Group 4 mode divides all 16 schedulable SLs into 4 groups, with each group processing one Co value. Each SL group (SLB) includes SLs equal to Rs = Ns / 4 = 4. For example, SL0–SL3 belong to group G0, SL4–SL7 to group G1, SL8–SL11 to group G2, and SL12–SL15 to group G3. This mode is between Group 1 and Group 16, allowing either the convolutional kernel or the input feature map to be designated as multicast data, while the other is designated as distributed data.

[0138] In one embodiment, the convolutional kernels can be divided into 4 groups according to Co and stored in... Figure 5 The first storage circuit 530 transmits the data using a broadcast channel. The input feature map can be divided into four parts along the XY direction of the output feature map and copied four times, stored in the second storage circuit 540, and then distributed to the four SLBs. Each SLB receives the same input feature map, which is then further divided into four parts and distributed to its four SLBs. Thus, all SLBs in each SLB jointly compute an output feature map of Co, while the four SLBs each process a different Co.

[0139] In another embodiment, the convolution kernel can be stored in Figure 5The second storage circuit 540 stores the input feature map, while the first storage circuit 530 stores the feature map. The division method is similar to that in the previous embodiment.

[0140] In this mode, there are multiple ways to divide the convolution kernel between SLBs.

[0141] Figure 7c A method 770 for allocating the Co dimension of a convolutional kernel is illustrated. In this method, the convolutional kernel is divided into four groups, with each group assigned a Co dimension in increments of 1. For example, when Co = 12, the four groups Co are {0, 4, 8}, {1, 5, 9}, {2, 6, 10}, and {3, 7, 11}. Each time, one Co dimension from each group is sent. For example, the first time Co = 0 to 3 is sent, with one Co corresponding to one SLB, and the four SLBs within one SLB sharing the same weights; the second time Co = 4 to 7 is sent, and so on. Therefore, after each round of computation, the Co dimension of the computation results output by each SLB is continuous.

[0142] Figure 7d Another Co allocation method 780 for the convolutional kernel is shown. In this method, the convolutional kernel is continuously and evenly divided into 4 groups according to Co. For example, when Co = 12, the 4 groups Co are {0, 1, 2}, {3, 4, 5}, {6, 7, 8}, and {9, 10, 11}. Each time, one Co of each group is sent. For example, the first time Co = 0, 3, 6, 9 is sent. One Co corresponds to one SLB, and the 4 SLBs within one SLB share the same weights. The second time Co = 1, 4, 7, 10 is sent, and so on. Thus, the Co dimension of the output result of each SLB in multiple rounds of operation is continuous.

[0143] Exemplary splitting of input feature map

[0144] As can be seen from the preceding description, when multiple SLs process a single Co value, the input feature map needs to be split among these multiple SLs. For example, the Group1 grouping mode requires splitting the input feature map into 16 parts, while the Group4 grouping mode requires splitting the input feature map into 4 parts.

[0145] To ensure that the split input feature maps can share the same convolution kernel, they can be partitioned according to the Ho / Wo direction of the output feature map, thus mapping back to the partitioning of the input feature map. In some embodiments, the input feature maps can be partitioned among the Rs slave processing circuits SL included in each slave processing circuit group as follows: based on the size of the corresponding output feature map, the output feature map is evenly divided into Rs output feature blocks of the same shape in the XY dimension (i.e., the Ho / Wo dimension); and based on the input feature map region required to compute each output feature block, the input feature map is divided into Rs input feature blocks in the XY dimension (i.e., the Hi / Wi dimension) for allocation to the Rs slave processing circuits. It is understood that, depending on the convolution kernel size and convolution stride, the input feature maps corresponding to adjacent output points on the output feature map may overlap.

[0146] Figure 8 An exemplary splitting diagram of an input feature map according to an embodiment of this disclosure is shown. In this example, the input feature map is divided into 16 parts and assigned to 16 SLs, corresponding to the Group1 pattern.

[0147] In the diagram, 810 represents the output feature map of a single Co, which is divided into 16 identically shaped output feature blocks in a 4×4 pattern along the XY direction, and assigned to SL0 to SL15 respectively. These 16 output feature blocks can then be mapped onto the input feature map 820 to obtain 16 input feature map regions required for calculating these 16 output feature blocks, which are also divided along the XY direction. These 16 input feature map regions can then be correspondingly assigned to 16 slave processing circuits SL.

[0148] As described above, the input feature map is split into units according to a determined convolutional splitting scheme. Therefore, in the above embodiments, the segmentation of the input feature map must ensure that each segmented input feature map block is a multiple of the XY dimension of the splitting unit, meaning it can be aligned with the splitting unit in the XY direction. For example, when choosing a 4×4×4 convolutional splitting scheme, each input feature map block is aligned 4×4; while when choosing a 16×2×2 convolutional splitting scheme, each input feature map block is aligned 2×2.

[0149] If the output feature map is not aligned by the splitting unit (e.g., 4×4 or 2×2), it is necessary to pad the input feature map accordingly (e.g., pad with 0s) so that the actual calculated output XY is aligned by the splitting unit (e.g., 4×4 or 2×2) and the input XY is also aligned by the splitting unit (e.g., 4×4 or 2×2).

[0150] Those skilled in the art will understand that the output feature map can also be split according to other rules in the XY direction, for example, splitting it into 16 identical output feature blocks in a 1×16 manner and assigning them to SL0 to SL15 respectively. The embodiments disclosed herein are not limited in this respect. Furthermore, it is understood that although the foregoing description is in conjunction with the splitting between processing circuits, this splitting method can also be applied to splitting in other scenarios, such as the splitting between arithmetic circuits CU within a single processing circuit SL. The embodiments disclosed herein are not limited in this respect.

[0151] Data storage example on the second storage circuit

[0152] As mentioned earlier, one of the input feature maps or convolutional kernels can be stored in... Figure 5 The data is stored in the first storage circuit 530, and the other of the two can be stored in the second storage circuit 540. Data in the first storage circuit can be multicast via a broadcast path, while data in the second storage circuit is typically distributed. By rationally allocating the storage methods of each data item, the data access speed can be accelerated. In some embodiments, the second storage circuit can allocate a storage area for each slave processing circuit SL, so that the data required for the operation of each slave processing circuit only needs to be read from its corresponding storage area.

[0153] Figures 9a-9d A schematic diagram of data storage in a second storage circuit according to an embodiment of this disclosure is shown. The diagram exemplarily illustrates 16 storage regions 900-915 allocated to, for example, Ns = 16 slave processing circuits SL0-SL15. Each storage region stores the convolution kernel or input feature map to be processed by that slave processing circuit. It will be understood that the content stored in each storage region may vary depending on the different grouping patterns.

[0154] Figure 9a The diagram illustrates that in Group 1 mode, the input feature map is divided into 16 parts, FB0 to FB15, and stored in various storage regions of the second storage circuit. Each storage region corresponding to a SL stores a continuous two-dimensional region, and these two-dimensional regions are arranged according to, for example... Figure 8 The data is split in a specific way. Within each two-dimensional region, the splitting units described above are stored row by row, meaning one row corresponds to one splitting unit of the input feature map. For example, assuming each split input feature block includes 4 splitting units, or 4 rows of data, then in the storage area 1100 allocated to SL0, the first row (Line01), the second row (Line02), the third row (Line03), and the fourth row (Line04) of the input feature map are stored sequentially. Each row can also be called an input feature row.

[0155] Figure 9bThe diagram illustrates that in Group16 mode, convolutional kernels are partitioned according to Co and stored in various storage regions of the second storage circuit to be allocated to corresponding SLs. Each SL's storage region stores the convolutional kernel assigned to it with a different Co value. For example, as described earlier, there are two Co allocation methods, and correspondingly, two storage methods are also available. Figure 9b One approach is illustrated here, where consecutive Co values ​​are sequentially assigned to each SL in each round of computation. This ensures that the Co dimension of the output results from each SL is continuous after each round. For example, the figure shows that convolutional kernels with Co = 0–15 in the first round are stored sequentially in 16 storage regions 900–915; convolutional kernels with Co = 16–31 in the second round are also stored sequentially in 16 storage regions 900–915, and so on. It can be understood that in Group16 mode, the input feature map may also be stored on a second storage circuit (not shown). In this case, the input feature map does not need to be split; it is directly copied 16 times and stored in different storage regions of the second storage circuit to be assigned to the corresponding SL. Thus, each SL can perform convolution operations on the same input feature map but with different Co values.

[0156] Figure 9c This illustration shows one possible storage configuration in Group4 mode. In this example, the input feature map is split into four parts and copied four times, stored in different storage areas of the second storage circuit. Specifically, each processing circuit group (SLB) processes the same input feature map with convolutional kernels of different Co values; and the four processing lines (SLs) within each SLB process one of the split input feature map blocks. Therefore, the storage content of the storage areas used for the four SLBs is the same; for example, the content in 900–903 is the same as the content in 912–915. Furthermore, within each SLB, the storage areas for different SLs store different split input feature blocks; for example, input feature block FB0 is stored in 900, input feature block FB1 is stored in 901, and so on. The same storage allocation is performed in the storage areas of the other SLBs, which will not be described further.

[0157] Figure 9dThis illustrates another possible storage configuration in the Group4 pattern. In this illustrated example, the convolutional kernels are divided into four groups according to Co, and stored in various storage regions of the second storage circuit. Specifically, the convolutional kernels are divided into groups with an interval of 1 according to Co. For example, when Co = 16, they are sequentially allocated to four SLBs in multiple rounds, where Co = 0 is allocated to G0 {SL0~SL3}, Co = 1 is allocated to G1 {SL4~SL7}, Co = 2 is allocated to G2 {SL8~SL11}, and Co = 3 is allocated to G3 {SL12~SL15}; then, starting from Co = 4, they are sequentially allocated to the four SLBs again. The four SLs within each SLB share the same weights. For example, the same weights are stored in storage regions 900, 901, 902, and 903. Similarly, Co can also be used in a contiguous manner within a single SLB; those skilled in the art can deduce its storage method by referring to the foregoing description, which will not be detailed here.

[0158] A single exemplary convolution operation process within the processing circuit

[0159] After the data to be processed is split and stored accordingly, multiple slave processing circuits can be scheduled to perform convolution operations on the corresponding data rows of the input feature map and the convolution kernel. Then, according to the convolution splitting scheme, the operation results returned by the multiple slave processing circuits can be concatenated to obtain the output feature map of the convolution operation between the input feature map and the convolution kernel. Specifically, multiple computational circuits (CUs) and various buffer circuits in the slave processing circuits can be utilized (see...). Figure 5 The specific convolution operation is performed by a buffer circuit. Depending on the size of the buffer circuit inside the processing circuit and the computing power limitations of the arithmetic circuit, multiple calculations are usually required in each round of operation to complete the required operation.

[0160] In some embodiments, the first buffer circuit can be used to cache the input feature map, which may come from a first storage circuit or a second storage circuit; correspondingly, the second buffer circuit can be used to cache the convolution kernel, which may come from a second storage circuit or a first storage circuit. As mentioned above, performing convolution operations on a unit-by-unit basis (a row of data) can fully utilize the hardware's computing power and avoid or reduce invalid computations. Therefore, each arithmetic circuit CU can perform bitwise multiplication and accumulation operations on data rows selected from the first buffer circuit (e.g., input feature rows) and data rows selected from the second buffer circuit (e.g., weight rows) respectively during each computation. For simplicity, the following description refers to the processing within a single processing circuit SL; it can be understood that similar processing is performed within other SLs.

[0161] As described above, in typical 3D convolution operations, a single slave processing circuit computes one or a portion of the output feature map corresponding to the same output channel Co across all its processing circuits. Depending on the buffer size of the first and second buffer circuits within the slave processing circuit SL, and the processing capability of the processing circuit CU (e.g., internal registers), the slave processing circuit may not be able to compute all the output feature maps allocated to it in one operation. Therefore, the output feature blocks can be divided based on the single-operation capability of the processing circuit (e.g., computing Nop output points or partial sums in a single operation), with each output feature block corresponding to all schedulable N output points within a single SL. CU The single-operation capability (N) of each arithmetic circuit CU *Nop output points). For example, as mentioned earlier. Figure 5 Taking a single SL comprising 4 CUs as an example, assuming each CU can calculate Nop = 4 output points or a partial sum of output points in a single operation, then a single SL can calculate 4*4 = 16 output points (or partial sums) in a single operation. Therefore, the output feature map can be divided into output feature blocks aligned with 16 output points along the XoYo dimension, and each output feature block can be calculated individually. It is understood that these 16 output points can be in a 4*4 format or a 1*16 format; this disclosed embodiment is not limited in this respect.

[0162] When calculating the output feature block for each partition, further calculations can be performed on these N blocks. CU The output points of the output feature block are divided among the arithmetic circuits to determine the processing objects of each arithmetic circuit. Then, based on the division of output points, N values ​​can be selected from the first buffer circuit using the splitting unit as a sliding window. CU The input feature data rows are distributed to N. CU Each arithmetic circuit selects the corresponding weight data from the second buffer circuit and broadcasts it to N. CU A computational circuit is used to achieve parallel computation of the output points corresponding to multiple sliding windows by reusing weight data. Nk sliding selections are performed, where Nk is determined by the smaller of the convolution kernel size in the X and Y dimensions and the maximum convolution kernel size supported by a single operation of the processing circuit.

[0163] In some embodiments, when performing a three-dimensional convolution operation, the corresponding weight data can be selected as follows: 1 / Nop weight rows are selected from the second buffer circuit according to the sliding method corresponding to that in the first buffer circuit, Nop-1 copies are made to expand them into an expanded weight row, and broadcast to N in the processing circuit. CU An operational circuit.

[0164] At this time, each arithmetic circuit can perform bitwise multiplication and accumulation on a unit of 1 / Nop data rows for an input feature row from the first buffer circuit and an extended weight data row from the second buffer circuit during each sliding number selection period to obtain Nop partial sums; and accumulate the Nk*Nop partial sums calculated during Nk sliding number selection periods according to the corresponding convolution output points to obtain and output Nop operation results.

[0165] When the processing circuit outputs the output points of its internal arithmetic circuits, it can output the output points calculated by multiple arithmetic circuits in a specific order according to the division of the output points, so that the continuously output points are continuous in the X and / or Y dimensions, which facilitates subsequent processing. In some embodiments, the aforementioned block circuit can further store the calculation results returned from each processing circuit in a fourth-dimensional storage order. Depending on the situation, the block circuit can also convert the calculation results into the desired dimensional storage order.

[0166] There are multiple ways to divide the output points between operational circuits, and the sliding selection convolution process and the output order of the output points will also be different accordingly.

[0167] Figures 10a-10b This diagram illustrates two different output point divisions between operational circuits.

[0168] Figure 10a A schematic diagram illustrating the allocation of consecutive output points for each arithmetic circuit according to some embodiments of this disclosure is shown. In these embodiments, N CU The output feature block is divided equally among the arithmetic circuits into N. CU Each output feature sub-block has an identical shape, and each sub-block contains Nop output points. Each arithmetic circuit is responsible for calculating one of these sub-blocks. For example, using the example above, the figure shows output feature block 1010a containing 4*4 output points, and each of the equally divided output feature sub-blocks 1011a to 1011d contains 2*2 output points. Each arithmetic circuit calculates 2*2 consecutive output points (or a partial sum) at a time. The figure uses different backgrounds to show the output points assigned to the four different arithmetic circuits CU0 to CU3.

[0169] Based on the above output point partitioning, when performing convolution operations using sliding number selection, the data required to calculate the output feature sub-blocks can be combined with these N... CU Correspondingly, N output feature sub-blocks are selected from the first buffer circuit. CU The calculation is performed on each row of data.

[0170] For example, when selecting the first number of input feature data, the first input data row can be selected from the corresponding input feature blocks according to the four input feature blocks required to calculate the four output feature sub-blocks 1011a to 1014a, and distributed to the four arithmetic circuits.

[0171] When selecting weight data, the corresponding weight data can be selected from the second buffer circuit and broadcast to N. CU Multiple arithmetic circuits are used to achieve parallel computation of the output points of multiple arithmetic circuits by reusing weight data.

[0172] Furthermore, in some embodiments, in order to fully utilize the computing power inside the computing unit CU (e.g., multiply-accumulate unit), such as calculating Nop output points or partial sums in a single operation, weights can be reused within a single input data line to simultaneously calculate Nop output points or partial sums.

[0173] For example, when selecting weight data, you can take only 1 / Nop weight rows, copy them Nop-1 times to expand them into one weight row. This expanded weight row includes Nop identical 1 / Nop weight rows. The expanded weight row can also be broadcast to N. CU This allows for the reuse of weights across multiple operational circuits, while simultaneously reusing weights at a smaller granularity (e.g., 1 / Nop rows) across the computation of Nop output points of a single operational circuit.

[0174] Therefore, by taking N correspondingly each time CU Take one input feature data row, take 1 / Nop weight rows, copy and expand them into one weight row, and calculate N values ​​each time. CU *Nop output points or partial sums. When the calculation result is a partial sum, the partial sum can be calculated multiple times by sliding the slider. The partial sums of each calculation are accumulated according to their respective output points to obtain the final result.

[0175] Based on the way the output points are divided, the number of slides and the step size of the convolution operation can be determined. According to... Figure 10a The partitioning method uses a sliding step of Nk = Kx * Ky, where Kx and Ky are the smaller of the kernel size in the X and Y dimensions and the maximum kernel size supported by the processing circuit in a single operation, respectively, with a sliding step size of 1. The maximum kernel size supported by the processing circuit in a single operation is determined, for example, by the spatial size of the first and second buffer circuits. It can be understood that when the kernel size exceeds the maximum kernel size, it needs to be split in the Kx and Ky directions according to that maximum kernel size.

[0176] according to Figure 10aThis partitioning method allows for the output of each arithmetic circuit to be continuously calculated in the X and / or Y dimensions, thus enabling the output of each circuit's results sequentially. For example, following the order of the arithmetic circuits, the output of one circuit can be completed at a time, such as 2*2 output points, resulting in a 4*4 output feature block after four consecutive iterations.

[0177] Figure 10b A schematic diagram illustrating the allocation of spaced output points for each operational circuit according to other embodiments of this disclosure is shown. In these embodiments, N CU The output feature block is divided equally among the arithmetic circuits into Nop output feature sub-blocks of the same shape. Each output feature sub-block includes N... CU There are N output points, which are respectively assigned to N... CU Each operation circuit. For example, using the example above, the figure shows that output feature block 1010b includes 4*4 output points, and each of the equally divided output feature sub-blocks 1011b to 1011b includes 2*2 output points. In each output feature sub-block, these 2*2 output points are allocated to 4 operation circuits. Thus, each operation circuit calculates one output point in each of the Nop output feature sub-blocks. The figure uses different backgrounds to show the output points allocated to the 4 different operation circuits CU0 to CU3.

[0178] Based on the above output point division, when performing convolution operations through sliding selection, N can be selected from the first buffer circuit according to the data required to calculate the output feature sub-blocks, corresponding to the output point positions of each output feature sub-block. CU The calculation is performed on each row of data.

[0179] For example, during the initial selection of input feature data, four input data rows can be selected from the four input feature blocks required to calculate the four output points within the first output feature sub-block 1011b, and distributed to the four arithmetic circuits. It can be understood that since these four output points are continuous in the X and / or Y directions, the interval or step size of the four simultaneously selected input data rows in the X and / or Y directions is 1.

[0180] When selecting weight data, the corresponding weight data can be selected from the second buffer circuit and broadcast to N. CU Multiple arithmetic circuits are used to achieve parallel computation of the output points of multiple arithmetic circuits by reusing weight data.

[0181] Furthermore, in some embodiments, in order to fully utilize the computing power inside the computing unit CU (e.g., multiply-accumulate unit), such as calculating Nop output points or partial sums in a single operation, weights can be reused within a single input data line to simultaneously calculate Nop output points or partial sums.

[0182] For example, when selecting weight data, you can take only 1 / Nop weight rows, copy them Nop-1 times to expand them into one weight row. This expanded weight row includes Nop identical 1 / Nop weight rows. The expanded weight row can also be broadcast to N. CU This allows for the reuse of weights across multiple operational circuits, while simultaneously reusing weights at a smaller granularity (e.g., 1 / Nop rows) across the computation of Nop output points of a single operational circuit.

[0183] Therefore, by taking N correspondingly each time CU Take one input feature data row, take 1 / Nop weight rows, copy and expand them into one weight row, and calculate N values ​​each time. CU *Nop output points or partial sums. When the calculation result is a partial sum, the partial sum can be calculated multiple times by sliding the slider. The partial sums of each calculation are accumulated according to their respective output points to obtain the final result.

[0184] Based on the way the output points are divided, the number of slides and the step size of the convolution operation can be determined. According to... Figure 10b The partitioning method uses a sliding step of Nk = ceil(Kx / 2) * ceil(Ky / 2), where Kx and Ky are the smaller of the kernel size in the X and Y dimensions and the maximum kernel size supported by the processing circuit in a single operation, respectively, with a sliding step size of 2. Similarly, the maximum kernel size supported by the processing circuit in a single operation is determined, for example, by the spatial size of the first and second buffer circuits. It can be understood that when the kernel size exceeds the maximum kernel size, it needs to be split in the Kx and Ky directions according to that maximum kernel size.

[0185] according to Figure 10b The partitioning method, since the output points calculated by each arithmetic circuit are spaced out in the X and / or Y dimensions (i.e., discontinuous), requires selecting a portion of the computational results from a portion of the circuits for output each time to ensure the output points are continuous in the X and / or Y dimensions. For example, one line can output 1*4 computational results each time, returning a 4*4 output feature block four times consecutively. In this example, the first line needs to output two results from CU0 and two results from CU1, the second line needs to output two results from CU2 and two results from CU3, and so on. In another example, 2*2 computational results can still be output each time, returning a 4*4 output feature block four times consecutively. In this example, the first output is the first computational result of each of CU0 to CU3, the second output is the second computational result of each of CU0 to CU3, and so on. In another example, computational results can also be output column-wise, which will not be elaborated here.

[0186] Furthermore, considering the use of registers within the arithmetic circuit CU, a single processing circuit can calculate multiple 4x4 regions in the Xo / Yo direction, for example, up to 16 4x4 regions. In this case, weights or neurons can be reused based on the stored content in the second storage circuit, reducing the reading frequency of the second storage circuit. If the calculated result is a partial sum, it is stored in a register within the arithmetic circuit.

[0187] In these embodiments, each slave processing circuit can control the reading method of the weight data row and the input feature map data row according to the weight reuse and / or input feature map reuse method, so as to perform bitwise multiplication and accumulation operations on the entire convolution window of the convolution output point simultaneously through multiple operations to obtain multiple partial sums and results, and accumulate them to obtain the convolution output at the corresponding convolution output point.

[0188] The following describes the detailed computation process when different convolution splitting schemes are used and applied to different types of convolution operations, with reference to specific embodiments.

[0189] Example: Update 1

[0190] In Update 1, the splitting unit has a shape of 4B×4×4, and its operation can be applied to similar convolutional splitting schemes. The size of the splitting unit indicated by these convolutional splitting schemes can be expressed as Uci×Uy×Ux=M, where Uci is the size of the splitting unit in the lowest storage dimension (e.g., Ci dimension) of the input feature map and the initial convolutional kernel, Ux and Uy are the sizes of the splitting unit in the X and Y storage dimensions of the input feature map and the initial convolutional kernel, respectively, and M is the maximum number of operations per hardware operation. In these convolutional splitting schemes, Ux=Uy≥Uci>1, Uci=M / 4 n ,

[0191] For example, assuming M = 64, then M / 4 n The possible values ​​are 64, 16, 4, and 1. Following the rule Ux = Uy ≥ Uci > 1, the splitting unit can be a 4B×4×4 shape. When using this convolutional splitting scheme, the Ci dimension of the input feature map and the convolutional kernel needs to be aligned to 4B. For example, when Ci = 10, it can be aligned to 3*4 = 12 by padding with zeros, thus splitting into 4B×4×4 units with 3 splitting units in the Ci dimension.

[0192] For example, assuming M = 128, then M / 4 nThe possible values ​​are 128, 32, 8, and 2. Following the rule Ux = Uy ≥ Uci > 1, the splitting unit can be a 2B × 8 × 8 shape. When using this convolutional splitting scheme, the Ci dimension of the input feature map and the convolutional kernel needs to be aligned to 2B. For example, when Ci = 3, it can be aligned to 2*2 = 4 by padding with zeros, thus splitting according to a 2B × 8 × 8 shape, with two splitting units in the Ci dimension.

[0193] Update 1 applies deep convolution operations to the reverse training of neural network models; it is a 2D convolution operation. The principle of reverse deep convolution operations can be found in the previous sections. Figure 4b The description is as follows. In reverse depthwise convolution operations, the sizes of top_diff and bottom_data are usually quite large, thus requiring different optimization schemes.

[0194] In the following description, although `top_diff` and `bottom_data` will refer to the data to be processed, the description of the convolution kernel can be similarly applied to `top_diff`, and the description of the input feature map can be similarly applied to `bottom_data`; that is, the two are interchangeable. The following description can be applied to convolution splitting schemes similar to Update 1.

[0195] Since the input channels are not accumulated in depthwise convolution, the dimensions of top_diff and bottom_data can be simplified to three dimensions: C (channels), H (height), and W (width). The shape of the split units indicated by these convolutional splitting schemes also satisfies: Uc × Uy × Ux = M, where Uc is the size of the split unit in the lowest initial storage dimension (e.g., C dimension) of bottom_data and top_diff, Ux and Uy are the sizes of the split unit in the initial X and Y storage dimensions of bottom_data and top_diff, respectively, and M is the maximum number of operations per hardware cycle. In these convolutional splitting schemes, Ux = Uy ≥ Uc > 1, Uc = M / 4 n ,

[0196] Because the product results in the C-channel dimension are not accumulated in depthwise convolution operations, the processor, which would normally obtain 1 number after multiplying 64 numbers in the C-dimensional dimension, will now obtain 64 numbers instead of 1. This wastes the processor's computational power and results in a performance penalty. To fully utilize the processor's computational power, the above-mentioned splitting method moves the data from the dimension requiring accumulation (e.g., the HW dimension) to the C-dimensional dimension, thereby improving processor utilization. For example, when using a 4B×4×4 splitting unit, assuming the data type is int8, the accumulation result of multiplying 64 numbers will be 4 numbers instead of the original 64 numbers.

[0197] Figure 11 A schematic diagram illustrating the splitting and storage of an Update1 scheme according to an embodiment of this disclosure is shown. For simplicity, the example in the diagram assumes the data type is Int8.

[0198] Figure 1110 shows the original data to be processed (which can be neurons or weights), stored in the order HWC. The figure also shows the original data to be processed divided into four data blocks 1111-1114 according to the splitting unit, each data block including 4×4×4=64 data.

[0199] Figure 1120 illustrates the format of the split data for easy reading. It can be seen that the original data blocks (e.g., 1111-1114) are arranged into rows along dimension C (e.g., 1121-1124). Within each row, the data is stored in CHW order. For example, for data row 1121, the 16 data points with C=0 are stored first, followed by the 16 with C=1, then the 16 with C=2, and finally the 16 with C=3.

[0200] Specifically, for bottom_data, the data needs to be arranged from [Hi Wi C] as follows:

[0201] [Hi / 4*Wi / 4*C / 4*(4×4×4)], the shape of this six-dimensional tensor, omitting the N dimension.

[0202] For top_diff, the data needs to be arranged from [Ho Wo C] as follows:

[0203] [Ho / 4*Wo / 4*C / 4*(4×4×4)], the shape of this six-dimensional tensor, omitting the N dimension.

[0204] When using Figure 5When the computing device shown executes the Update1 convolution splitting scheme, it can split bottom_data and top_diff into multiple splitting units by a block circuit integrated within the main processing circuit, or a block circuit that is completely or partially independent of the main processing circuit, according to the Update1 convolution splitting scheme. The block circuit can also transform the dimensional storage order of bottom_data and top_diff so that the data within each splitting unit is stored consecutively as a data row. The split and transformed bottom_data and / or top_diff can be provided to the main processing circuit or slave processing circuits. The main processing circuit can then distribute the obtained data to multiple slave processing circuits for performing convolution operations; and, according to the convolution splitting scheme, concatenate the operation results returned by the multiple slave processing circuits to obtain the output ΔW (or weight_diff) of the depthwise convolution operation of bottom_data and top_diff. The multiple slave processing circuits can then perform convolution operations based on the data they obtain and return the operation results to the main processing circuit.

[0205] To fully utilize the schedulable slave processing circuits, corresponding computational tasks can be allocated among the slave processing circuits to improve parallel processing efficiency. Considering that in the Update1 deep convolution operation scenario, the results of operations on the C dimension do not need to be accumulated, operations on different C dimensions can be performed relatively independently on different processing circuits. It should be noted that in the Update1 splitting scheme, the C dimension is aligned to 4B. Therefore, when processing is performed in units of splitting, the C dimension will be aligned to 4B (i.e., Uc) before splitting. In other words, the processing on different processing circuits is split in units of Uc along the C dimension.

[0206] In inverse depthwise convolution scenarios, the C dimension is typically large, for example, greater than 64, and bottom_data and top_diff are generally also large. In these embodiments, the channel C dimension Nc of bottom_data and top_diff in a single round of operation can usually be a multiple of 64. Therefore, the operation of a single channel, calculated in units of Uc, can be assigned to a slave processing circuit. Thus, in some embodiments, the convolution splitting scheme also indicates a grouping method for performing depthwise convolution operations. This grouping method can divide the bottom_data and top_diff data sequentially into Ns schedulable slave processing circuits according to the channel C dimension, in units of Uc. Each slave processing circuit processes different consecutive Uc C values ​​of bottom_data and top_diff data. In other words, a single slave processing circuit can be a group, processing operations of different C (in units of Uc), which corresponds to the Group16 grouping mode mentioned above.

[0207] In this embodiment where groups are divided according to dimension C, the top_diff can be split according to the aforementioned convolution splitting scheme, and after dimension transformation, it can be stored in the first storage circuit. Since each slave processing circuit processes a different Uc, the top_diff corresponding to different Uc C values ​​can be unicast / transmitted to the scheduled Ns slave processing circuits via the broadcast bus during the operation.

[0208] In these embodiments, bottom_data can be identified as the distribution data, and the distribution data after splitting and converting the dimension storage order is stored in the storage areas corresponding to Ns slave processing circuits in the second storage circuit according to the channel C dimension and in Uc units, so as to be distributed to the corresponding slave processing circuits.

[0209] Figure 12 The schematic storage method of bottom_data in a second storage circuit according to some embodiments of this disclosure is shown.

[0210] As shown in the figure, the second storage circuit can allocate a storage area for each slave processing circuit, so that the bottom_data data required for the operation of each slave processing circuit only needs to be read from its corresponding storage area. The figure exemplifies that 16 storage areas 1200-1215 are allocated for 16 slave processing circuits, and each storage area stores the bottom_data data block to be processed by the slave processing circuit.

[0211] As mentioned earlier, the C dimension is divided into units of Uc. In the example in the diagram, assuming Uc = 4B and the data type is int8, then one Uc includes four C values. When the size of the C dimension exceeds Uc times the number of schedulable slave processing circuits, multiple rounds of computation are required to perform the operation.

[0212] Taking the example in the diagram, assuming that all 16 slave processing circuits are schedulable, and further assuming that the C-dimensional size of the bottom_data is 128, which exceeds the number of schedulable slave processing circuits by a factor of Uc (16*4=64), then the entire computation can be completed in two rounds. The bottom_data can be divided into 32 bottom_data data blocks according to the C-dimensional size and in units of Uc. The first 16 data blocks are calculated in the first round of computation, and the last 16 data blocks are calculated in the second round of computation.

[0213] As shown in the figure, in the data of the first round of calculation, the bottom_data block containing C=0,1,2,3 is allocated to the first slave processing circuit; the bottom_data block containing C=4,5,6,7 is allocated to the second slave processing circuit; and so on. In the data of the second round of calculation, the bottom_data block is similarly divided and stored accordingly, which will not be repeated here.

[0214] Accordingly, the first buffer circuit can buffer multiple bottom_data data lines from the second storage circuit that are distributed to the slave processing circuit; while the second buffer circuit can buffer multiple top_diff data lines from the first storage circuit that are unicast to the corresponding UC of the slave processing circuit. Depending on the specific splitting and / or multiplexing method, these data lines can be distributed to the corresponding arithmetic circuit CU or broadcast to all CUs within the slave processing circuit during the operation. Subsequently, each arithmetic circuit CU can perform bitwise multiplication and accumulation operations in each operation for the bottom_data data lines selected from the first buffer circuit and the top_diff data lines selected from the second buffer circuit, respectively.

[0215] When multiple computational circuits (CUs) within a single processing circuit SL jointly process a single Uc, the output points need to be split among these CUs. In Update1, this is done by allocating output points at intervals to each computational circuit (e.g., ...). Figure 10b In Update1, the convolution kernel top_diff is split into 4×4 units, while the bottom_data only uses 2×2 64B bytes from the first buffer circuit each time. Therefore, after multiple sliding data retrieval calculations on the first buffer circuit, a maximum of 4×4 output points can be calculated.

[0216] Specifically, in one embodiment, during each calculation, each computational circuit calculates one output point on the XY plane of the Uc channel C values ​​of the output ΔW, adjacent in the X and / or Y dimensions; and in different calculations, each computational circuit calculates different output points on the output ΔW in the X and / or Y dimensions. The number of slides Nk = ceil(Kx / 2) * ceil(Ky / 2), where Kx and Ky are the smaller of the dimensions of the output ΔW in the X and Y dimensions or the maximum output size supported by the processing circuit in a single operation under the current convolution splitting mode. For example, for the case where Kx = Ky = 4, Nk = 2 * 2 = 4 times, that is, slide 4 times, calculate 2 × 2 output points each time, and calculate a total of 4 × 4 output points.

[0217] Figure 13A schematic diagram of a single operation process in the Update1 scheme according to an embodiment of this disclosure is shown. In this example, the size of the first buffer circuit 1310 is 3×3×64B, meaning it can cache up to 9 rows of data, and the size of the second buffer circuit 1320 is 2×2×64B, meaning it can cache up to 4 rows of data. For consistency with the splitting unit, the storage within the buffer circuits in the figure is also shown in units of splitting units.

[0218] The diagram illustrates the calculation process of the first sliding selection. Following the method corresponding to the division of output points, N is selected from the first buffer circuit using the splitting unit as the sliding window. CU Each of the input feature lines is sent to N input features in the processing circuit. CU One arithmetic circuit is used for calculation; one weight data line is read from the second buffer circuit and broadcast to N in the processing circuit. CU An operational circuit.

[0219] Specifically, in Figure 5 In the computing device shown, N CU =4, Nop=4.

[0220] As shown in the figure, an input feature data row is selected from the first buffer circuit 1310 at the starting position and at a position moved 1 in each of the X and / or Y directions, for a total of 4 input feature data rows, which are then sent to the 4 arithmetic circuits 1340 in the processing circuit SL. A weight data row, i.e., a 4×4 data row 1330, is selected from the second buffer circuit 1320 at the starting position and broadcast to the 4 arithmetic circuits 1340 in the SL.

[0221] In each calculation, for an input feature row from the first buffer circuit and a weight row from the second buffer circuit, the feature data and weight data corresponding to the same channel value are multiplied and accumulated bitwise in units of 1 / Uc data rows to obtain Uc output points.

[0222] As shown in the figure, four arithmetic circuits 1340 perform bitwise multiplication and accumulation operations on the distributed input feature data rows and the broadcast weight data rows according to 1 / Uc (Uc=4), resulting in a result 1350. Different background colors in 1350 represent results obtained by different arithmetic circuits 1340. It can be seen that in each operation, one CU calculates one output point on each XoYo plane of Uc, and the four CUs obtain a total of Uc×2×2 output points. It can also be seen that the output points calculated by the four CUs are adjacent in the XoYo dimension of the output feature map.

[0223] Next, the data is selected by sliding in the first buffer circuit, while no sliding is required in the second buffer circuit; the same row weight is still used for the next calculation. Nk sliding selections are performed in the first buffer circuit, where Nk = Kx * Ky, and Kx and Ky are the smaller of the convolution kernel size in the X and Y dimensions, or the maximum convolution kernel size supported by the processing circuit in a single operation under the current convolution splitting mode. Correspondingly, the computation circuit concatenates the Nk * Uc output points calculated during the Nk sliding calculations according to the output point partitioning method, obtaining Nk * N values ​​across Uc channels. CU The result of the operation.

[0224] In some embodiments, in Update1 mode, the maximum convolution kernel size supported by a single operation from the processing circuit is 4×4.

[0225] Figure 14 A schematic diagram of the sliding convolution process in the Update1 scheme according to an embodiment of this disclosure is shown. In this example, the first buffer circuit buffers 2*2 = 4 bottom_data rows, shown as 1410 in the figure, where the C dimension is omitted; the second buffer circuit buffers 1 top_diff data row, shown as 1420 in the figure, also omitting the C dimension. Each data row is a 4×4×4 (C×H×W) block. The dimensions of ΔW in the X and Y dimensions are Kx = Ky = 4. During each calculation, a 4×4 top_diff block, corresponding exactly to the 4×4 bottom_data block, is selected from the second buffer circuit and broadcast to the four computation circuits.

[0226] Specifically, following the method corresponding to the division of output points, N is selected from the first buffer circuit using the splitting unit as a sliding window. CU Each bottom_data line is sent to N from the processing circuit. CU A computational circuit is used for calculation. Further, one top_diff data line is read from the second buffer circuit and broadcast to N in the processing circuit. CU A computational circuit. Nk sliding selections are performed on the first buffer circuit, where Nk = ceil(Kx / 2) * ceil(Ky / 2), and Kx and Ky are the smaller of the dimensions of the weight gradient data ΔW in the X and Y dimensions, or the maximum output size supported by the processing circuit in a single operation under the current convolution splitting mode.

[0227] The selection ranges of bottom_data and top_diff in the first and second buffer circuits for each slide are as follows: Figure 14As shown, there are four diagrams representing a total of four slides. In the diagrams, block 1410 represents the bottom_data in the first buffer circuit, and the four dashed boxes indicate the regions selected for distribution to the four CUs. Block 1420 represents the top_diff in the second buffer circuit, and the dashed box represents the selected top_diff data row, which is broadcast to the four CUs and does not require reselection during the slide. The number of slides Nk = 4, and the slide step size = 2. In the Update1 convolution operation mode, the maximum ΔW size supported by a single operation from the processing circuit is 4×4. It can be understood that when ΔW exceeds the maximum supported size, it needs to be split according to this maximum supported size in the XY directions.

[0228] In each calculation, each CU, for a bottom_data data row from the first buffer circuit and a top_diff data row from the second buffer circuit, performs bitwise multiplication and accumulation on a unit of 1 / Uc data rows, on the bottom_data and top_diff data corresponding to the same channel value, to obtain Uc output points, which are also 1 output point of ΔW on each KxKy plane of Uc, thus N CU Each arithmetic circuit obtains N values ​​from Uc on the KxKy plane each time. CU There are Nk output points. This can be understood as follows: after the sliding motion has undergone Nk computation cycles, each computational circuit calculates Uc output points spaced Nk intervals along the X and / or Y dimensions of the KxKy plane. CU The total number of Nk sliding operations of the arithmetic circuits can obtain Nk*N values ​​on Uc KxKy surfaces. CU There are 4 output points. These output points can be spliced ​​together to form a maximum of 4×4 (Kx*Ky) output points on the Uc planes in dimension C, that is, Uc×4×4.

[0229] Specifically, for Figure 14 In each image, the number of convolutional units (CUs) is Ncu = 4. Each CU calculates one output point on Uc planes along dimension C in a single pass. This partial sum is the result of 1 / Uc (1 / 4) rows of data, where each output point is a 4×4 (Y×X) 2D convolution. After sliding Nk = 4 times, the maximum output point is calculated, resulting in a 4×4 (Y×X) output in one sliding loop (SL). Figure 10b (As shown).

[0230] It is understandable that when the calculated Kx / Ky value for each CU is greater than 4, it is necessary to slide along the Kx / Ky direction and read different bottom_data and top_diff values. Those skilled in the art can similarly deduce the calculation process based on the foregoing description, and it will not be elaborated here.

[0231] As seen from the preceding sliding convolution process, the output of the sliding mode is not in the normal order of traditional convolution output data. Therefore, during the output process, each slave processing circuit SL can convert the operation results of its internal computation circuit CU into a specified format. In some embodiments, each slave processing circuit can output one output point at the same position on the XY plane from Uc output points obtained by one of its internal computation circuits each time. Ns slave processing circuits simultaneously output one output point at the same position on the XY plane from Ns*Uc output points each time. With this output method, the Ns*Uc output points are continuous in the C dimension. The block circuit can further store the operation results returned from each slave processing circuit in a fourth-dimensional storage order, for example, by concatenating and storing them in the Ky*Kx*(Ns*Uc) dimensional order. Depending on the situation, the block circuit can also convert the operation results into the desired dimensional storage order for storage.

[0232] Figure 15 A schematic diagram of the output data format of an Update1 scheme according to an embodiment of this disclosure is shown. In this embodiment, a grouping method based on dimension C is adopted, that is, each slave processing circuit SL processes different Uc operations.

[0233] Figure 1510 shows the raw output of one SL. As can be seen from the figure, each SL outputs a region of Uc×1×1 (C×Y×X) at a time, that is, it outputs Uc operation results of one operation circuit within it each time. For example, CU0 outputs four operation results, which are continuous in the C dimension of the output data. Since different SLs process operations of different Uc values, 16 SLs can simultaneously output one output point at the same position on different Uc values ​​in the XY plane. These can be pieced together in the C dimension to form 16*Uc output points, which are continuous in the C dimension.

[0234] Figure 1520 illustrates the data structure for 16 SLs. As shown, the outputs of the 16 SLs are concatenated into a single, continuous row of data along dimension C. For example, the first time, all 16 SLs output the point at position Ky=0, Kx=0 (marked "1"); the second time, all 16 SLs output the point at position Ky=0, Kx=1 (marked "2"); and so on. The final output data, after being written to the storage circuit (e.g., the first storage circuit), becomes the format Kh*Kw*(16*Uc), where 16 represents the division across the 16 SLs. In some implementations, this concatenation operation can be performed again to transform the data into other desired formats as needed.

[0235] In some embodiments, considering the storage space of the internal registers of the arithmetic circuit, for example, a single slave processing circuit including four arithmetic circuits can calculate up to 16 4×4 output point regions, so bottom_data can be reused, thereby reducing the reading frequency of the second storage circuit. That is, the reading frequencies of the first storage circuit and the second storage circuit can be different. If the result calculated by the arithmetic circuit is a partial sum, it is stored in the internal register.

[0236] In these embodiments, the processing circuit can be further configured to: determine the number of times `bottom_data` is reused (rn) within the processing circuit based on storage space limitations within the arithmetic circuit; and control the loading frequency of `top_diff` data in the second buffer circuit, such that the `bottom_data` loaded each time in the first buffer circuit is reused rn times, and a convolution operation is performed with the corresponding `top_diff` data loaded rn times in the second buffer circuit. In some examples, `rn` can take a value no greater than 16.

[0237] The above description and explanation of the convolution optimization scheme provided in this disclosure embodiment, in conjunction with the specific convolution splitting scheme of Update 1, serve as an example. Based on the teachings of this disclosure, those skilled in the art can conceive of other convolution splitting schemes according to specific hardware circuit configurations (such as the number of processing circuits, the number of arithmetic circuits within the processing circuits, the hardware's single-processing capability, etc.), all of which fall within the scope of this disclosure and will not be listed here.

[0238] This disclosure also provides a method for performing convolution operations using the aforementioned computing device. Those skilled in the art will understand that the steps of performing convolution operations correspond to the various circuits of the computing device described above in conjunction with the accompanying drawings; therefore, the features described above also apply to the method steps and will not be repeated here.

[0239] This disclosure also provides a chip that may include the computing device of any of the embodiments described above in conjunction with the accompanying drawings. Furthermore, this disclosure also provides a board that may include the aforementioned chip.

[0240] Depending on the application scenario, the electronic devices or apparatus disclosed herein may include servers, cloud servers, server clusters, data processing devices, robots, computers, printers, scanners, tablets, smart terminals, PC devices, IoT terminals, mobile terminals, mobile phones, dashcams, navigators, sensors, cameras, video cameras, projectors, watches, headphones, mobile storage, wearable devices, visual terminals, autonomous driving terminals, vehicles, home appliances, and / or medical devices. The vehicles include airplanes, ships, and / or vehicles; the home appliances include televisions, air conditioners, microwave ovens, refrigerators, rice cookers, humidifiers, washing machines, lights, gas stoves, and range hoods; the medical devices include MRI scanners, ultrasound machines, and / or electrocardiographs. The electronic devices or apparatus disclosed herein can also be applied in fields such as the Internet, IoT, data centers, energy, transportation, public management, manufacturing, education, power grids, telecommunications, finance, retail, construction sites, and healthcare. Furthermore, the electronic devices or apparatus disclosed herein can also be used in application scenarios related to artificial intelligence, big data, and / or cloud computing, such as cloud computing, edge computing, and terminal applications. In one or more embodiments, the high-computing-power electronic devices or apparatuses according to the present disclosure can be applied to cloud devices (e.g., cloud servers), while the low-power electronic devices or apparatuses can be applied to terminal devices and / or edge devices (e.g., smartphones or cameras). In one or more embodiments, the hardware information of the cloud devices and the hardware information of the terminal devices and / or edge devices are compatible with each other, so that suitable hardware resources can be matched from the hardware resources of the cloud devices to simulate the hardware resources of the terminal devices and / or edge devices based on the hardware information of the terminal devices and / or edge devices, so as to complete the unified management, scheduling and collaborative work of end-to-cloud or cloud-edge-end integration.

[0241] It should be noted that, for the sake of brevity, this disclosure describes some methods and their embodiments as a series of actions and combinations thereof. However, those skilled in the art will understand that the solutions disclosed herein are not limited by the order of the described actions. Therefore, based on the disclosure or teachings of this document, those skilled in the art will understand that some steps can be performed in a different order or simultaneously. Furthermore, those skilled in the art will understand that the embodiments described in this disclosure can be considered optional embodiments, that is, the actions or modules involved are not necessarily essential for the implementation of one or more solutions disclosed herein. In addition, depending on the solution, the description of some embodiments in this disclosure may have different emphases. In view of this, those skilled in the art will understand that parts not described in detail in a certain embodiment of this disclosure can also be referred to the relevant descriptions of other embodiments.

[0242] In terms of specific implementation, based on the disclosure and teachings of this document, those skilled in the art will understand that several embodiments disclosed herein can also be implemented in other ways not disclosed herein. For example, regarding the various units in the electronic device or apparatus embodiments described above, this document has divided them based on logical functions, but in actual implementation, there may be other ways of division. As another example, multiple units or components can be combined or integrated into another system, or some features or functions in a unit or component can be selectively disabled. Regarding the connection relationships between different units or components, the connections discussed above in conjunction with the accompanying drawings can be direct or indirect couplings between units or components. In some scenarios, the aforementioned direct or indirect couplings involve communication connections utilizing interfaces, where the communication interface can support electrical, optical, acoustic, magnetic, or other forms of signal transmission.

[0243] In this disclosure, the units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units. The aforementioned components or units may be located in the same location or distributed across multiple network units. Furthermore, depending on actual needs, some or all of the units can be selected to achieve the purpose of the solution described in the embodiments of this disclosure. Additionally, in some scenarios, multiple units in the embodiments of this disclosure may be integrated into one unit or each unit may exist physically independently.

[0244] In other implementation scenarios, the integrated units described above can also be implemented in hardware, i.e., as specific hardware circuits, which may include digital circuits and / or analog circuits. The physical implementation of the circuit's hardware structure may include, but is not limited to, physical devices, which may include, but are not limited to, transistors or memristors. Therefore, the various devices described herein (e.g., computing devices or other processing devices) can be implemented using appropriate hardware processors, such as central processing units, GPUs, FPGAs, DSPs, and ASICs. Furthermore, the aforementioned storage units or storage devices can be any suitable storage medium (including magnetic storage media or magneto-optical storage media), such as resistive random access memory (RRAM), dynamic random access memory (DRAM), static random access memory (SRAM), enhanced dynamic random access memory (EDRAM), high-bandwidth memory (HBM), hybrid memory cube (HMC), ROM, and RAM.

[0245] The embodiments of this disclosure have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this disclosure. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this disclosure. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this disclosure. Therefore, the content of this specification should not be construed as a limitation of this disclosure.

Claims

1. A computing device configured to perform depthwise convolution operations in the reverse training of a neural network model, the computing device comprising: The main processing circuit is used to: acquire input neuron data and neuron gradient data, wherein the input neuron data and neuron gradient data have been split into multiple splitting units according to the convolution splitting scheme and the dimension storage order has been converted. One splitting unit includes data of the lowest storage dimension and at least one other storage dimension. The data on the lowest storage dimension is continuous, and the total data volume of a splitting unit does not exceed the maximum single hardware operation volume. The data in a splitting unit is stored continuously as a data row. as well as Multiple slave processing circuits are used to perform the depthwise convolution operation on corresponding data rows of the input neuron data and neuron gradient data.

2. The computing device according to claim 1, wherein the convolution splitting scheme indicates that the shape of the splitting unit is Uc×Uy×Ux=M, Uc is the size of the splitting unit in the initial lowest storage dimension and channel C of the input neuron data and neuron gradient data, Ux and Uy are the sizes of the splitting unit in the initial X and Y storage dimensions of the input neuron data and neuron gradient data, respectively, M is the maximum number of operations per hardware operation, Ux=Uy≥Uc>1, Uc=M / 4 n n=1,2,... .

3. The computing device according to claim 2, wherein the convolution splitting scheme further indicates a grouping method for performing the depthwise convolution operation, wherein the grouping method divides the input neuron data and neuron gradient data sequentially into schedulable Ns slave processing circuits according to the channel C dimension and in units of Uc, wherein each slave processing circuit processes input neuron data and neuron gradient data with different consecutive Uc C values.

4. The computing device according to claim 3, wherein the computing device further comprises a first storage circuit and a second storage circuit. The neuron gradient data is determined to be unicast data. The unicast data, after being split and its dimensional storage order transformed, is stored in the first storage circuit. During computation, the neuron gradient data corresponding to different Uc C values ​​are transmitted to the scheduled Ns slave processing circuits via a broadcast bus. The input neuron data is determined as distribution data. After splitting and converting the dimensional storage order, the distribution data is stored in the storage area corresponding to the Ns slave processing circuits in the second storage circuit according to the channel C dimension and in Uc units, so as to distribute it to the corresponding slave processing circuits.

5. The computing device according to claim 4, wherein each of the slave processing circuits includes a first buffer circuit, a second buffer circuit, and a plurality of arithmetic circuits, wherein: The first buffer circuit is used to buffer multiple lines of input neuron data from the second storage circuit that are distributed to the slave processing circuit; The second buffer circuit is used to buffer multiple neuron gradient data lines unicast from the first storage circuit to the slave processing circuit; and Each arithmetic circuit is used to perform positional multiplication and accumulation operations in each operation on the input neuron data line selected from the first buffer circuit and the neuron gradient data line selected from the second buffer circuit, respectively.

6. The computing device of claim 5, wherein the slave processing circuitry is further configured to operate according to the following in its schedulable N... CU Divide the output points among the operational circuits: During each computation, each computational circuit calculates the Uc channel C values ​​of the weight gradient data, and one output point adjacent in the X and / or Y dimensions on the XY plane; and In different calculations, each computational circuit calculates different output points of the weight gradient data in the X and / or Y dimensions.

7. The computing device of claim 6, wherein each of the slave processing circuits is further configured to: According to the method corresponding to the division of the output points, using the splitting unit as a sliding window, N is slidably selected from the first buffer circuit. CU Each input neuron data line is sent to the N input neuron in the processing circuit. CU A computational circuit is used for calculation; One neuron gradient data line is read from the second buffer circuit and broadcast to N in the processing circuit. CU Arithmetic circuit; and Perform Nk sliding selections on the first buffer circuit, where Nk = ceil(Kx / 2). ceil(Ky / 2), Kx and Ky are the smaller of the weight gradient data in the X and Y dimensions, or the maximum weight gradient size supported by the processing circuit in a single operation in the convolution split mode.

8. The computing device of claim 7, wherein each of the arithmetic circuits is further configured to: In each calculation, for one input neuron data row from the first buffer circuit and one neuron gradient data row from the second buffer circuit, the input neuron data and neuron gradient data corresponding to the same channel value are multiplied and accumulated in units of 1 / Uc data rows to obtain Uc output points at the same position on the XY plane; and In Nk sliding calculations, Uc output points with Nk intervals in the X and / or Y dimensions on the XY plane are calculated.

9. The computing device of claim 8, wherein each of the slave processing circuits is further configured to: Each time, it outputs one of the Uc output points at the same position on the XY plane, obtained from the operation of one of its internal arithmetic circuits.

10. The computing device of claim 9, wherein the main processing circuitry is further configured to: The calculation results output from the processing circuit will be processed according to Ky. Kx (Ns) The dimensions of Uc are concatenated and stored in order.

11. The computing device according to any one of claims 1-10, wherein M=64 bytes, Uc=4 bytes, Ux=Uy=4.

12. The computing device according to any one of claims 6-10, wherein N CU =4, Ns=16.

13. The computing device of claim 7, wherein the maximum weight gradient size supported by the slave processing circuit in a single operation in the convolution splitting mode is 4×4.

14. A chip comprising a computing device according to any one of claims 1-13.

15. A circuit board comprising the chip according to claim 14.

16. A method for performing convolution operations using a computing device according to any one of claims 1-13.

Images