Computing circuits, chips and boards

Abstract

The present disclosure discloses an operation circuit, a chip and a board. The operation circuit can be implemented as a calculation device included in a combined processing device, and the combined processing device can also include an interface device and other processing devices. The calculation device interacts with other processing devices to jointly complete the calculation operation specified by the user. The combined processing device can also include a storage device, which is connected to the calculation device and other processing devices respectively, and is used to store data of the calculation device and other processing devices. The scheme disclosed in the present disclosure optimizes the operation circuit and realizes a multi-dimensional operation circuit array, so as to facilitate the operation of multi-dimensional data such as convolution operation, and help to improve the operation speed of neural networks.

Description

Technical Field

[0001] The present disclosure generally relates to the field of circuits. More specifically, the present disclosure relates to a computing circuit, a chip and a board. Background Art

[0002] At present, deep learning has become an important branch of machine learning and has greatly promoted the development of artificial intelligence (AI). The core technology of deep learning, deep neural network (DNN), has been widely used in many industries.

[0003] There are a large number of multiplication, addition, and mixed operations of multiplication and addition in the neural network model. The convolution layer is one of the commonly used hidden layers in the neural network model, which extracts features from the input data through convolution operations. The neural network model contains a large number of convolution operations, and the computational performance of the convolution operation greatly affects the computational performance of the entire neural network model. In addition, the neural network model usually operates on multidimensional data, and the existing operation methods often use the method of flattening the operation. This operation process is decoupled from the meaning of the original neural network model, resulting in the loss of some usable information and ineffective use. Summary of the invention

[0004] In order to at least solve one or more of the technical problems mentioned above, the present disclosure proposes an operation circuit, which can directly map operations of multidimensional data such as neural network models onto the multidimensional operation circuit array by constructing a multidimensional operation circuit array with at least three dimensions, thereby effectively fitting the multidimensional data structure of the neural network model, helping to improve the efficiency of programmers and further improve the operation speed of the neural network model.

[0005] In a first aspect, an embodiment of the present disclosure provides an operational circuit, comprising an operational circuit array having at least three dimensions, wherein: each dimension of the operational circuit array comprises at least one operational element, each operational element in the operational circuit array is electrically connected to at least one adjacent operational element in at least one dimension, and the operational element comprises a multiplier and / or an adder.

[0006] In some embodiments, the operation circuit array is a three-dimensional operation circuit array, and each dimension includes multiple operation elements, adjacent operation elements are electrically connected, and each operation element includes a multiplication and accumulation unit containing a multiplier and an addition unit to realize multiplication and accumulation operations.

[0007] In some embodiments, the operation circuit array is used to perform a convolution operation as follows: enable an operation element array in the operation circuit array with the same dimension as the convolution kernel of the convolution operation, wherein the convolution kernel is a three-dimensional array; load the convolution kernel elements corresponding to the same output channel Co value into each operation element in the operation element array one by one; transfer the input data elements in the operation element array in a specified direction; and each operation element in the operation element array multiplies the assigned convolution kernel element with the transferred input data element, accumulates the result with the operation result transferred from its adjacent operation element, and transfers it in the specified direction.

[0008] In a second aspect, the present disclosure provides a chip comprising the computing circuit of the first aspect.

[0009] In a third aspect, the present disclosure provides a board comprising the chip of the second aspect.

[0010] Through the operation circuit, chip and board provided above, the solution of the embodiment of the present disclosure can be mapped with at least three-dimensional data such as in a neural network model by providing at least three-dimensional operation circuit array, so that the operation process is coupled with the meaning in the neural network model, and effective information can be fully utilized. In addition, in some embodiments, the convolution operation based on such a three-dimensional operation circuit array can make full use of the reusability of data, reduce data access requirements by improving data reusability, and thus improve the overall operation efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] By reading the detailed description below with reference to the accompanying drawings, the above and other purposes, features and advantages of the exemplary embodiments of the present disclosure will become readily understood. In the accompanying drawings, several embodiments of the present disclosure are shown in an exemplary and non-limiting manner, and the same or corresponding reference numerals represent the same or corresponding parts, wherein:

[0012] Figure 1 A structural diagram of a board according to an embodiment of the present disclosure is shown;

[0013] Figure 2 A structural diagram showing a combined processing device according to an embodiment of the present disclosure;

[0014] Figure 3 A schematic diagram showing the internal structure of a processor core of a single-core or multi-core computing device according to an embodiment of the present disclosure;

[0015] Figure 4 A schematic structural diagram of a multi-dimensional operation circuit array according to an embodiment of the present disclosure is shown;

[0016] Figure 5An exemplary convolution operation principle example to which the disclosed embodiments can be applied is shown; and

[0017] Figure 6 The convolution operation process according to one embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0018] The following will be combined with the drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present disclosure.

[0019] It should be understood that the terms "first", "second", "third", and "fourth" etc. that may appear in the claims, specifications, and drawings of the present disclosure are used to distinguish different objects rather than to describe a specific order. The terms "include" and "comprise" used in the specifications and claims of the present disclosure indicate the presence of the described features, wholes, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their collections.

[0020] It should also be understood that the terms used in this disclosure are only for the purpose of describing specific embodiments and are not intended to limit the disclosure. As used in this disclosure and claims, the singular forms of "a", "an", and "the" are intended to include the plural forms unless the context clearly indicates otherwise. It should also be further understood that the term "and / or" used in this disclosure and claims refers to any combination of one or more of the associated listed items and all possible combinations, including these combinations.

[0021] As used in this specification and claims, the term "if" may be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context.

[0022] The specific implementation of the present disclosure is described in detail below with reference to the accompanying drawings.

[0023] Figure 1 FIG. 1 is a schematic diagram showing the structure of a board 10 according to an embodiment of the present disclosure. Figure 1As shown, the board 10 includes a chip 101, which is a system-on-chip (SoC), or system on chip, integrated with one or more combined processing devices, which are artificial intelligence computing units that support various deep learning and machine learning algorithms to meet the intelligent processing needs in complex scenarios in the fields of computer vision, speech, natural language processing, data mining, etc. In particular, deep learning technology is widely used in the field of cloud intelligence. A notable feature of cloud intelligence applications is the large amount of input data, which places high demands on the storage capacity and computing power of the platform. The board 10 of this embodiment is suitable for cloud intelligence applications and has huge off-chip storage, on-chip storage, and powerful computing power.

[0024] The chip 101 is connected to an external device 103 via an external interface device 102. The external device 103 is, for example, a server, a computer, a camera, a display, a mouse, a keyboard, a network card, or a wifi interface. The data to be processed can be transmitted from the external device 103 to the chip 101 via the external interface device 102. The calculation result of the chip 101 can be transmitted back to the external device 103 via the external interface device 102. According to different application scenarios, the external interface device 102 can have different interface forms, such as a PCIe interface.

[0025] The board 10 also includes a storage device 104 for storing data, which includes one or more storage units 105. The storage device 104 is connected to the control device 106 and the chip 101 through a bus and transmits data. The control device 106 in the board 10 is configured to control the state of the chip 101. To this end, in an application scenario, the control device 106 may include a microcontroller (MCU).

[0026] Figure 2 1 is a block diagram showing the combined processing device in the chip 101 of this embodiment. Figure 2 As shown in , the combined processing device 20 includes a computing device 201 , an interface device 202 , a processing device 203 and a storage device 204 .

[0027] The computing device 201 is configured to execute user-specified operations, and is mainly implemented as a single-core intelligent processor or a multi-core intelligent processor to perform deep learning or machine learning calculations. It can interact with the processing device 203 through the interface device 202 to jointly complete the user-specified operations.

[0028] The interface device 202 is used to transmit data and control instructions between the computing device 201 and the processing device 203. For example, the computing device 201 can obtain input data from the processing device 203 via the interface device 202 and write it into the storage device on the computing device 201 chip. Further, the computing device 201 can obtain control instructions from the processing device 203 via the interface device 202 and write them into the control cache on the computing device 201 chip. Alternatively or optionally, the interface device 202 can also read data in the storage device of the computing device 201 and transmit it to the processing device 203.

[0029] The processing device 203, as a general processing device, performs basic controls including but not limited to data handling, starting and / or stopping the computing device 201. Depending on the implementation, the processing device 203 can be a central processing unit (CPU), a graphics processing unit (GPU) or one or more types of processors in other general and / or special processors, these processors include but are not limited to digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc., and their number can be determined according to actual needs. As mentioned above, only with respect to the computing device 201 disclosed in the present invention, it can be regarded as having a single-core structure or a homogeneous multi-core structure. However, when the computing device 201 and the processing device 203 are integrated and considered together, the two are regarded as forming a heterogeneous multi-core structure.

[0030] The storage device 204 is used to store data to be processed, which may be DRAM or DDR memory, and is usually 16G or larger in size, and is used to store data of the computing device 201 and / or the processing device 203 .

[0031] Figure 3 The internal structure diagram of the processing core when the computing device 201 is a single-core or multi-core device is shown. The computing device 301 is used to process input data such as computer vision, speech, natural language, and data mining. The computing device 301 includes three modules: a control module 31, a computing module 32, and a storage module 33.

[0032] The control module 31 is used to coordinate and control the operation of the operation module 32 and the storage module 33 to complete the deep learning task, and includes an instruction fetch unit (IFU) 311 and an instruction decode unit (IDU) 312. The instruction fetch unit 311 is used to obtain instructions from the processing device 203, and the instruction decode unit 312 decodes the obtained instructions and sends the decoding results to the operation module 32 and the storage module 33 as control information.

[0033] The operation module 32 includes a vector operation unit 321 and a matrix operation unit 322. The vector operation unit 321 is used to perform vector operations and can support complex operations such as vector multiplication, addition, and nonlinear transformation; the matrix operation unit 322 is responsible for the core calculation of the deep learning algorithm, namely matrix multiplication and convolution.

[0034] The storage module 33 is used to store or transfer relevant data, including a neuron RAM (NRAM) 331, a weight RAM (WRAM) 332, and a direct memory access module (DMA) 333. NRAM 331 is used to store input neurons, output neurons, and intermediate results after calculation; WRAM 332 is used to store the convolution kernel of the deep learning network, that is, the weight; DMA 333 is connected to DRAM 204 through bus 34, and is responsible for data transfer between the computing device 301 and DRAM 204.

[0035] Combination of the above Figure 1-Figure 3 The hardware architecture and internal structure of the present disclosure are described in detail. It is understandable that the above description is only exemplary and not restrictive. According to different application scenarios and hardware specifications, those skilled in the art may also make changes to the board and its internal structure of the present disclosure, and these changes still fall within the scope of protection of the present disclosure. The embodiments of the present disclosure are based on the aforementioned hardware environment and provide a computing circuit that can be used to implement, for example, Figure 3 Various circuits in the operation module 32 include but are not limited to a vector operation unit 321 and a matrix operation unit 322.

[0036] As mentioned in the background technology, the existing operation method for multi-dimensional data is usually to spread the operation flat and use a two-dimensional structure or even a one-dimensional structure for operation. For example, taking a neuron with a three-dimensional structure of WHC (width W, height H, channel C) as an example, a certain dimension (such as W) is often prioritized to perform traversal loop expansion, thereby merging the W / H dimensions into one dimension. It is even possible to define a dimensional priority order, such as C->W->H, so that the three dimensions are merged into one dimension for operation. Such an operation process is decoupled from the meaning of the original neural network, resulting in the loss of some usable information and inability to effectively use it.

[0037] The disclosed embodiment provides a multi-dimensional operation circuit array, which can effectively fit the multi-dimensional data structure of the neural network, help improve the efficiency of programmers, and further improve the operation speed of the neural network. Furthermore, the disclosed embodiment also provides a solution for performing convolution operations using a multi-dimensional operation circuit array, which can fully utilize the data reuse characteristics in the convolution operation and the mapping relationship between the multi-dimensional operation circuit array and the multi-dimensional data, reduce the data access bandwidth requirements, and improve the overall operation performance.

[0038] Figure 4 A schematic structural diagram of a multidimensional operation circuit array according to an embodiment of the present disclosure is shown. In the embodiment of the present disclosure, the multidimensional operation circuit array has at least three dimensions to map with the three-dimensional data commonly used in the neural network model. It can be understood that when mapping with higher-dimensional data is required, a higher-dimensional operation circuit array can be constructed accordingly. For the purpose of example description, Figure 4 A three-dimensional computing circuit array is used as an example for illustration and explanation.

[0039] As shown in the figure, the three-dimensional operation circuit array 400 includes three dimensions, which can be respectively referred to as width W, height H and depth D. Each dimension includes at least one operation element (PE) 401, which can include a multiplier and / or an adder. Further, each operation element in the operation circuit array is electrically connected to at least one adjacent operation element in at least one dimension.

[0040] Here, "adjacent" means adjacent in the same dimension, such as the same width, height or depth. For example, if the X, Y, and Z coordinates are used to indicate the computing elements in the three-dimensional space, then in some embodiments, PE(X,Y,Z) may be electrically connected to PE(X-1,Y,Z), PE(X+1,Y,Z), PE(X,Y-1,Z), PE(X,Y+1,Z), PE(X,Y,Z-1) and PE(X,Y,Z+1).

[0041] exist Figure 4In the illustrated embodiment, each dimension of the three-dimensional computing circuit array includes a plurality of computing elements, and adjacent computing elements are electrically connected.

[0042] In some embodiments, in order to support common operations in neural networks, such as multiplication-accumulation operations, each operation element may include a multiplication-accumulation unit including a multiplier and an addition unit to implement the multiplication-accumulation operation.

[0043] Figure 5 An example of an exemplary convolution operation principle to which the disclosed embodiment can be applied is shown to better understand the scheme of performing convolution operations on the multi-dimensional operation circuit array of the disclosed embodiment. As shown in the figure, for example, a convolution layer in a neural network model can perform a convolution operation by applying a convolution kernel (also called a filter, weight, etc.) to an input feature map (also called input data, neuron, or input neuron) to perform convolution processing, thereby performing feature extraction.

[0044] The figure shows an example of input data of size 6×6×3, which can represent three input feature maps of size 6×6 (i.e., a three-dimensional matrix of 6×6×3), each representing three different features. In this example, the width W of the feature map is 6, and the height H is also 6. The number of input feature maps can also be called the number of input channels Ci. For example, the example input in the figure has 3 feature maps, also called 3 feature channels or 3 input channels.

[0045] The figure also shows a convolution kernel of size 2×3×3×3, which can represent two 3×3×3 stereo convolution kernels (i.e., two 3×3×3 three-dimensional matrices). Each stereo convolution kernel (also called a filter) has three different 3×3 two-dimensional convolution kernels, corresponding to three different input feature maps. The number of stereo convolution kernels can be called the output channel number Co, which is 2 in this example. In each stereo convolution kernel, the number of two-dimensional convolution kernels can be called the input channel number Ci, which is consistent with the number of channels of the input feature map. Each two-dimensional convolution kernel has a corresponding width Kw and height Kh. In this example, Kw and Kh are both 3.

[0046] The convolution result of the input feature map and the filter outputs two 4×4 feature maps. Among them, the convolution result of the input feature map and the upper stereo convolution kernel obtains a 4×4 output feature map at the top, and the convolution result of the input feature map and the lower stereo convolution kernel obtains a 4×4 output feature map at the bottom. The value at each position in the output feature map is obtained by performing a two-dimensional convolution operation on the corresponding block of each input feature map and the corresponding convolution kernel, and then adding them together. For example, the figure shows that the value of the (0,0) position on the upper output feature map (that is, the convolution output point) is obtained by performing a two-dimensional convolution operation on the block framed by the black cube in the input feature map and the upper stereo convolution kernel to obtain 3 values, and then adding them together to obtain the final value.

[0047] In the disclosed embodiment, each convolution output point has a corresponding convolution window, or called a receptive field, and the shape of the receptive field is equal to the shape of the convolution kernel. For example, the receptive field of the convolution output point at position (0,0) on the output feature map in the figure is a 3×3×3 black cube box in the figure. The value of each convolution output point corresponds to the positional multiplication and accumulation result of the input feature map and the weight within its receptive field. It can be understood that in the disclosed embodiment, the receptive field is relative to a single convolution layer, and the feature vector of a certain position in the input feature map of the current layer is calculated by the input of a fixed area of ​​the previous layer, and this area is the receptive field of this position.

[0048] In order to obtain outputs at other positions, the position of the convolution kernel can be moved on the input feature map, that is, the receptive field of the convolution output point can be moved. In the example in the figure, the convolution step size (Sx, Sy) is (1,1). When the convolution operation is performed after moving one grid to the right horizontally (width direction) or downward vertically (height direction), the value of the position (0,1) or (1,0) on the output feature map above can be obtained respectively.

[0049] From the above description, we can know that in a convolution layer of a neural network, there is a set of input feature maps, which contains a total of H×W×Ci information, where H and W are the height and width of the input feature map, respectively, and Ci is the number of input feature maps, also known as the number of input channels. The convolution layer has Ci×Co convolution kernels of size Kh×Kw, where Ci is the number of input channels, Co is the number of output feature maps (or the number of output channels), and Kh and Kw are the height and width of the convolution kernel, respectively. The output feature map contains Ho×Wo×Co information, where Ho and Wo are the height and width of the output feature map, respectively, and Co is the number of output channels. In addition, in the convolution operation, the convolution step size (Sx, Sy) is also involved, and the size of the convolution step size will affect the size of the output feature map.

[0050] From the description of the convolution operation above, it can be seen that since the operation results of the output channel Co dimension do not need to be accumulated, when a three-dimensional operation circuit array is used to perform the convolution operation, the convolution operation can be split according to the output channel Co dimension.

[0051] Therefore, in some embodiments of the present disclosure, when the multi-dimensional operation circuit array is used to perform convolution operation, only the operation element array in the operation circuit array with the same dimension as the convolution kernel of the convolution operation can be enabled. The convolution kernel here refers to the convolution kernel corresponding to the same Co value. For example, in Figure 5 In the example, the dimension of the convolution kernel at this time refers to the dimension of a 3×3×3 stereo convolution kernel, so only an array of computing elements of 3×3×3 size can be enabled.

[0052] Furthermore, in the operation of each output channel Co value, since the dimension size of the enabled operation element array matches the dimension size of the convolution kernel, in some embodiments, the convolution kernel elements corresponding to the same output channel Co value can be loaded one-to-one to each operation element in the operation element array. In other words, in the operation for each Co value, the convolution kernel elements in each operation element are fixed, so that the same convolution kernel is reused between different convolution windows of the same Co value.

[0053] In one implementation, the computing element array can allocate convolution kernel elements of the same Co value according to the following mapping relationship: the width dimension of the convolution kernel corresponds to the width dimension of the computing element array; the height dimension of the convolution kernel corresponds to the height dimension of the computing element array; and the input channel dimension of the convolution kernel corresponds to the depth dimension of the computing element array. This mapping facilitates programmers' understanding of the relationship between data meaning and operation, thereby improving programmers' usage efficiency.

[0054] Furthermore, part of the input data can be reused between adjacent convolution windows, thereby reducing frequent off-chip memory accesses. Therefore, when the convolution kernel elements are fixed in each computing element, the effect of moving the convolution window in the convolution operation can be achieved by flowing the input data elements on each computing element.

[0055] Specifically, in some embodiments, the input data elements can be transferred in a specified direction in the computing element array. The required data can be obtained by transferring data between computing elements without obtaining data from on-chip RAM or off-chip DDR, which can effectively reduce the off-chip memory access bandwidth and relieve the memory access pressure.

[0056] Furthermore, in some embodiments, considering the accumulation operation in the convolution operation, each operation element in the operation element array can multiply the assigned convolution kernel element with the transmitted input data element, and accumulate it with the operation result transmitted from its adjacent operation element and then transmit it in the specified direction.

[0057] Optionally or additionally, the operation circuit of the disclosed embodiment may further include an accumulation circuit, which may be electrically connected to each operation element on the last side of the enabled operation element array in the depth dimension, so as to accumulate the operation results of each operation element on the last side to obtain the corresponding output data element. It can be understood that this accumulation circuit may be included as a part of the multidimensional operation circuit array, or may be located outside the multidimensional operation circuit array, and the disclosed embodiment is not limited in this respect.

[0058] After the traversal calculation for the current output channel Co value is completed, the convolution kernel elements on each computing element can be updated, and the convolution kernel elements corresponding to the next output channel Co value can be loaded to perform the convolution operation for the next input channel Co value.

[0059] Considering the operation of the same Co value, in some embodiments, the input data elements flow sequentially on the operation elements, so the operation of each operation element is also completed sequentially. Therefore, in these embodiments, the convolution kernel elements of each operation element can be updated sequentially. For example, when the input data elements are transmitted in the width and / or height direction, and the operation results are transmitted in the depth direction, the convolution kernel elements of each operation element can be updated sequentially in the depth direction. Specifically, for each operation element on the same depth dimension value in the operation element array, after the traversal calculation for the current output channel Co value is completed, the convolution kernel elements on each operation element are updated, and the convolution kernel elements of the input channel Ci value corresponding to the depth dimension value corresponding to the next output channel Co value are loaded.

[0060] The above describes the convolution operation scheme based on the multi-dimensional operation array of the embodiment of the present disclosure. In order to better understand the convolution operation process on the multi-dimensional operation array, the data loading / transfer and calculation of each operation element in each step are described in detail below with reference to a specific example.

[0061] In this example, it is assumed that the dimension size of each convolution kernel on Co is 3×3×3, and the step size of the convolution operation is 1. n(0,0,0) represents the input data or neuron, where the first 0 represents the 0th in the direction of the input channel Ci, the second 0 represents the 0th in the direction of width W, and the third 0 represents the 0th in the direction of height H. w(0,0,0,0) represents the convolution kernel or weight, and its representation is similar to the input data. The fourth 0 represents the 0th group of weights (i.e., the corresponding output channel value Co=0). Although the coordinates of the input data and the convolution kernel are shown in a specific order in this example, the disclosed embodiments are not limited in this regard, and the order can be exchanged, as long as the agreement is consistent.

[0062] Based on the above symbolic representation, the way to calculate the output data or output neurons through convolution operation can be expressed as follows:

[0063]

[0064] Among them, out(0,0,0) represents the output data or output neuron, where the first 0 indicates the 0th in the direction of output channel Co, the second 0 indicates the 0th in the direction of width W, and the third 0 indicates the 0th in the direction of height H.

[0065] First, an array of computing elements of size 3×3×3 may be initialized.

[0066] Specifically, the convolution kernel with Co=0 can be read from the storage circuit and sent to 3×3×3 computing elements. In one implementation, each convolution kernel element can be assigned to each computing element at the same time. In another implementation, considering the directional transmission order of the operation results, that is, the operation of the next computing element needs to obtain the operation result of the previous computing element, the convolution kernel elements can be loaded sequentially based on the directional transmission order of the operation results. This loading method can reduce the memory access bandwidth. For example, the 3×3 convolution elements with Ci=0 in the convolution kernel can be first loaded to the 3×3 computing elements (also called the computing elements on the 0th surface) at the depth D=0 in the three-dimensional computing element array, so that the computing elements on the 0th surface can perform the corresponding operation. In the next clock cycle, the 3×3 convolution elements with Ci=1 in the convolution kernel can be loaded correspondingly to the 3×3 operational elements at depth D=1 (also called the operational elements on the first surface); and in the next clock cycle, the 3×3 convolution elements with Ci=2 in the convolution kernel can be loaded correspondingly to the 3×3 operational elements at depth D=2 (also called the operational elements on the second surface).

[0067] For the input data elements, initialization can be performed similarly, that is, the input data elements covered by the convolution window of the first output data element can be read from the storage circuit and loaded one by one into the 3×3×3 computing elements. Similarly, in one implementation, each input data element can be assigned to each computing element at the same time. In another implementation, considering the directional transmission order of the operation results, that is, the operation of the next computing element needs to obtain the operation result of the previous computing element, the input data elements can be loaded sequentially based on the directional transmission order of the operation results, thereby reducing the memory access bandwidth.

[0068] Next, each computing element multiplies the convolution kernel element and the corresponding input data element, and then passes the operation result backward, and the last computing element passes the operation result to the accumulation circuit. Specifically, in the width or height dimension of the computing element array, according to the step size of the convolution operation, the input data elements are passed to the adjacent computing elements from right to left or from bottom to top, where the input data elements of the rightmost and bottommost computing elements come from the storage circuit; in the depth dimension of the computing element array, the operation results of each computing element are passed from front to back to the adjacent computing element. The operation results of each computing element on the last side are passed to the accumulation circuit for accumulation to obtain the corresponding output data elements.

[0069] Figure 6 The convolution operation process according to one embodiment of the present disclosure is shown in the figure. The operation of each operation element in each step is shown in the figure.

[0070] As shown in the figure, in step S1, the 0th plane operation element (that is, the 3×3 operation array plane with D=0) performs a multiplication operation to obtain the product of n(0,i,j)*w(0,i,j), where i,j=0,1,2, and transmits it in the direction of Ci=1, that is, in the direction of D=1 in the three-dimensional operation array.

[0071] In step S2, the first surface operation element (that is, the 3×3 operation array surface with D=1) performs a multiplication operation to obtain the product of n(1,i,j)*w(1,i,j), where i,j=0,1,2, and adds the product to the product received from the 0th surface to obtain a partial sum, which is transmitted in the direction of Ci=2, that is, in the direction of D=2 in the three-dimensional operation array.

[0072] At this time, the input data elements of the 0th surface operation element can be moved to the left, that is, the 0th column operation element accepts the input data elements of the 1st column operation element, the 1st column accepts the 2nd column, and the 2nd column obtains the new input data element neuron from the storage unit. The 0th surface operation element performs a product operation to obtain the product of n(0,i,j)*w(0,i,j), where i=1,2,3; j=0,1,2, and transmits it in the direction of Ci=1. It can be understood that since the input data elements have moved, the operation result of the 0th surface operation element at this time corresponds to the convolution window in which the convolution kernel moves to the right in the width direction of the input data. It can also be understood that depending on the calculation order of the convolution window, such as width first or height first, the input data elements on each surface can move horizontally (for example, from right to left) or vertically (for example, from bottom to top), and the disclosed embodiment has no limitation in this regard.

[0073] In step S3, the second side operation element performs a multiplication operation to obtain the product of n(2,i,j)*w(2,i,j), where i,j=0,1,2, and adds the product to the operation result received from the first side to obtain a new partial sum. At this time, the operation result of each input channel (Ci) is obtained, so it can be passed to the accumulation circuit for final accumulation. The accumulation circuit adds the operation results of each operation element on the second side to obtain the output data element out(0,0,0).

[0074] At this time, the input data elements of the operation element on the first side are moved to the left, and then the product operation is performed to obtain the product of n(1,i,j)*w(1,i,j), where i=1,2,3; j=0,1,2, and the product is added to the product received from the 0th side to obtain a partial sum and transmitted in the direction of Ci=2.

[0075] At the same time, the input data elements of the 0th face operation element also move to the left, that is, the 0th column operation element receives the neurons of the 1st column operation element, the 1st column receives the 2nd column, and the 2nd column obtains new input data elements from the storage circuit. The 0th face operation element performs a product operation to obtain the product of n(0,i,j)*w(0,i,j), where i=1,2,3; j=0,1,2, and transmits it to the direction of Ci=1.

[0076] By analogy, when the operation in the width W direction is completed, the corresponding output data element can be output, so that the next row of output data can be calculated. Figure 6 Assuming that the output data in the table has 3 data in one row, in step S4, a new input data element is input.

[0077] from Figure 6 It can also be seen from the table that operations with the same background color represent partial sums corresponding to the same output data element, so the above operation process also completes the operation of each output data element in a pipeline manner.

[0078] The present disclosure also provides a chip, which may include the computing circuit of any of the embodiments described above in conjunction with the accompanying drawings. Further, the present disclosure also provides a board, which may include the aforementioned chip.

[0079] According to different application scenarios, the electronic equipment or device disclosed herein may include servers, cloud servers, server clusters, data processing devices, robots, computers, printers, scanners, tablet computers, smart terminals, PC devices, IoT terminals, mobile terminals, mobile phones, driving recorders, navigators, sensors, cameras, cameras, video cameras, projectors, watches, headphones, mobile storage, wearable devices, visual terminals, automatic driving terminals, transportation, household appliances, and / or medical equipment. The transportation includes airplanes, ships and / or vehicles; the household appliances include televisions, air conditioners, microwave ovens, refrigerators, rice cookers, humidifiers, washing machines, electric lights, gas stoves, and range hoods; the medical equipment includes magnetic resonance imaging, ultrasound machines and / or electrocardiographs. The electronic equipment or device disclosed herein may also be applied to the Internet, IoT, data centers, energy, transportation, public administration, manufacturing, education, power grids, telecommunications, finance, retail, construction sites, and medical fields. Further, the electronic equipment or device disclosed herein may also be used in cloud, edge, and terminal applications related to artificial intelligence, big data, and / or cloud computing. In one or more embodiments, electronic devices or devices with high computing power according to the disclosed solution can be applied to cloud devices (such as cloud servers), while electronic devices or devices with low power consumption can be applied to terminal devices and / or edge devices (such as smart phones or cameras). In one or more embodiments, the hardware information of the cloud device and the hardware information of the terminal device and / or edge device are compatible with each other, so that according to the hardware information of the terminal device and / or edge device, appropriate hardware resources can be matched from the hardware resources of the cloud device to simulate the hardware resources of the terminal device and / or edge device, so as to complete the unified management, scheduling and collaborative work of end-to-end or cloud-edge-to-end.

[0080] It should be noted that, for the purpose of simplicity, the present disclosure describes some methods and embodiments thereof as a series of actions and combinations thereof, but those skilled in the art will appreciate that the scheme of the present disclosure is not limited by the order of the actions described. Therefore, based on the disclosure or teaching of the present disclosure, those skilled in the art will appreciate that some of the steps therein may be performed in other orders or simultaneously. Further, those skilled in the art will appreciate that the embodiments described in the present disclosure may be regarded as optional embodiments, i.e., the actions or modules involved therein may not necessarily be necessary for the implementation of one or some of the schemes of the present disclosure. In addition, depending on the different schemes, the present disclosure also has different focuses on the description of some embodiments. In view of this, those skilled in the art may appreciate the parts that are not described in detail in a certain embodiment of the present disclosure, and may also refer to the relevant descriptions of other embodiments.

[0081] In terms of specific implementation, based on the disclosure and teaching of this disclosure, those skilled in the art can understand that several embodiments disclosed in this disclosure can also be implemented by other methods not disclosed herein. For example, with respect to the various units in the electronic device or device embodiments described above, this article splits them on the basis of considering the logical functions, and there may be other splitting methods in actual implementation. For another example, multiple units or components can be combined or integrated into another system, or some features or functions in the units or components can be selectively disabled. In terms of the connection relationship between different units or components, the connection discussed in the above text in conjunction with the accompanying drawings can be a direct or indirect coupling between units or components. In some scenarios, the aforementioned direct or indirect coupling involves a communication connection using an interface, wherein the communication interface can support electrical, optical, acoustic, magnetic or other forms of signal transmission.

[0082] In the present disclosure, the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units. The aforementioned components or units may be located in the same location or distributed on multiple network units. In addition, according to actual needs, some or all of the units may be selected to achieve the purpose of the scheme described in the embodiments of the present disclosure. In addition, in some scenarios, multiple units in the embodiments of the present disclosure may be integrated into one unit or each unit may exist physically separately.

[0083] In some other implementation scenarios, the above-mentioned integrated unit can also be implemented in the form of hardware, that is, a specific hardware circuit, which may include digital circuits and / or analog circuits, etc. The physical implementation of the hardware structure of the circuit may include but is not limited to physical devices, and the physical devices may include but are not limited to devices such as transistors or memristors. In view of this, the various devices described herein (such as computing devices or other processing devices) can be implemented by appropriate hardware processors, such as central processing units, GPUs, FPGAs, DSPs, and ASICs, etc. Further, the aforementioned storage unit or storage device may be any appropriate storage medium (including magnetic storage media or magneto-optical storage media, etc.), which may be, for example, a variable resistive memory (Resistive Random Access Memory, RRAM), a dynamic random access memory (Dynamic Random Access Memory, DRAM), a static random access memory (Static Random Access Memory, SRAM), an enhanced dynamic random access memory (Enhanced Dynamic Random Access Memory, EDRAM), a high bandwidth memory (High Bandwidth Memory, HBM), a hybrid memory cube (Hybrid Memory Cube, HMC), ROM and RAM, etc.

[0084] The foregoing content can be better understood in accordance with the following terms:

[0085] Clause 1. An arithmetic circuit comprising an arithmetic circuit array having at least three dimensions, wherein:

[0086] Each dimension of the operation circuit array includes at least one operation element.

[0087] Each operational element in the operational circuit array is electrically connected to at least one adjacent operational element in at least one dimension, and

[0088] The computing element includes a multiplier and / or an adder.

[0089] Item 2. An operational circuit according to Item 1, wherein the operational circuit array is a three-dimensional operational circuit array, and each dimension includes multiple operational elements, adjacent operational elements are electrically connected, and each operational element includes a multiplier-accumulator containing a multiplier and an adder to realize multiplication and accumulation operations.

[0090] Clause 3. The operation circuit according to clause 2, wherein the operation circuit array is used to perform a convolution operation as follows:

[0091] Enable an operation element array in the operation circuit array having the same dimension as a convolution kernel of the convolution operation, wherein the convolution kernel is a three-dimensional array;

[0092] Loading the convolution kernel elements corresponding to the same output channel Co value to each computing element in the computing element array one by one;

[0093] Passing input data elements in the array of computing elements in a specified direction; and

[0094] Each operation element in the operation element array performs a multiplication operation on the assigned convolution kernel element and the transmitted input data element, and accumulates the operation result transmitted from its adjacent operation element and transmits it in a specified direction.

[0095] Item 4. The operation circuit according to Item 3, wherein the operation element array further allocates convolution kernel elements of the same Co value according to the following mapping relationship:

[0096] The width dimension of the convolution kernel corresponds to the width dimension of the computing element array;

[0097] The height dimension of the convolution kernel corresponds to the height dimension of the computing element array; and

[0098] The input channel dimension of the convolution kernel corresponds to the depth dimension of the computing element array.

[0099] Clause 5. The arithmetic circuit according to clause 4, wherein the arithmetic element array is further configured to transfer input data elements in the arithmetic element array in a specified direction as follows:

[0100] In the width or height dimension of the computing element array, the input data elements are transferred to adjacent computing elements from right to left or from bottom to top according to the step size of the convolution operation, wherein the input data elements of the rightmost and bottommost computing elements come from the storage circuit.

[0101] Clause 6. The computing circuit according to clause 5, wherein the computing element array is further configured to be initialized as follows:

[0102] The input data elements within the convolution window corresponding to the first output data element are loaded one by one into each computing element in the computing element array.

[0103] Item 7. The operation circuit according to Item 6, wherein the operation element array is further configured to transfer the operation result in a specified direction as follows:

[0104] In the depth dimension of the computing element array, the computing results of each computing element are transmitted to the adjacent computing elements from front to back.

[0105] Clause 8. The computing circuit according to clause 7, further comprising:

[0106] The accumulation circuit is electrically connected to each computing element on the last side of the computing element array in the depth dimension, so as to accumulate the operation results of each computing element on the last side to obtain corresponding output data elements.

[0107] Clause 9. The computing circuit according to any one of clauses 3 to 8, wherein the computing element array is further configured to:

[0108] For each computing element on the same depth dimension value in the computing element array, after the traversal calculation of the current output channel Co value is completed, the convolution kernel elements on each computing element are updated, and the convolution kernel elements of the input channel Ci value corresponding to the next output channel Co value and the depth dimension value are loaded.

[0109] Item 10. A chip comprising the computing circuit according to any one of Items 1-9.

[0110] Clause 11. A board comprising the chip according to clause 10.

[0111] The embodiments of the present disclosure are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present disclosure. The description of the above embodiments is only used to help understand the method of the present disclosure and its core idea. At the same time, for those skilled in the art, according to the ideas of the present disclosure, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on the present disclosure.

Claims

1. An operation circuit, comprising an operation circuit array having at least three dimensions, wherein: Each dimension of the operation circuit array includes at least one operation element. Each operational element in the operational circuit array is electrically connected to at least one adjacent operational element in at least one dimension, and The computing element includes a multiplier and / or an adder; The operation circuit array is used to perform the convolution operation as follows: Enable an operation element array in the operation circuit array having the same dimension as a convolution kernel of the convolution operation, wherein the convolution kernel is a three-dimensional array, and the operation element array is further used for: For each computing element on the same depth dimension value in the computing element array, after the traversal calculation of the current output channel Co value is completed, the convolution kernel elements on each computing element are updated, and the convolution kernel elements of the input channel Ci value corresponding to the next output channel Co value and the depth dimension value are loaded, where Ci is the number of input channels and Co is the number of output channels.

2. The operational circuit according to claim 1, wherein the operational circuit array is a three-dimensional operational circuit array, and each dimension includes multiple operational elements, adjacent operational elements are electrically connected, and each operational element includes a multiplier-accumulator containing a multiplier and an adder to realize multiplication and accumulation operations.

3. The operation circuit according to claim 2, wherein the operation circuit array is used to perform a convolution operation as follows: Loading the convolution kernel elements corresponding to the same output channel Co value to each computing element in the computing element array one by one; Passing input data elements in the array of computing elements in a specified direction; and Each operation element in the operation element array performs a multiplication operation on the assigned convolution kernel element and the transmitted input data element, and accumulates the operation result transmitted from its adjacent operation element and transmits it in a specified direction.

4. The operation circuit according to claim 3, wherein the operation element array further allocates convolution kernel elements of the same Co value according to the following mapping relationship: The width dimension of the convolution kernel corresponds to the width dimension of the computing element array; The height dimension of the convolution kernel corresponds to the height dimension of the computing element array; and The input channel dimension of the convolution kernel corresponds to the depth dimension of the computing element array.

5. The arithmetic circuit according to claim 4, wherein the arithmetic element array is further configured to transfer input data elements in the arithmetic element array in a specified direction as follows: In the width or height dimension of the computing element array, the input data elements are transferred to adjacent computing elements from right to left or from bottom to top according to the step size of the convolution operation, wherein the input data elements of the rightmost and bottommost computing elements come from the storage circuit.

6. The operation circuit according to claim 5, wherein the operation element array is further configured to be initialized as follows: The input data elements within the convolution window corresponding to the first output data element are loaded one by one into each computing element in the computing element array.

7. The operation circuit according to claim 6, wherein the operation element array is further used to transfer the operation result in a specified direction as follows: In the depth dimension of the computing element array, the computing results of each computing element are transmitted to the adjacent computing elements from front to back.

8. The operation circuit according to claim 7, further comprising: The accumulation circuit is electrically connected to each computing element on the last side of the computing element array in the depth dimension, so as to accumulate the operation results of each computing element on the last side to obtain corresponding output data elements.

9. A chip, comprising the computing circuit according to any one of claims 1-8.

10. A board comprising the chip according to claim 9.

Images