Methods and apparatus for deep learning

Abstract

A method and apparatus for deep learning are disclosed. The apparatus for deep learning includes a processor configured to support a plurality of different operation modes, the processor including a systolic array having a plurality of multiplier accumulator (MAC) units, and a control circuit configured to control a selection operation of the plurality of MAC units and a data movement between the plurality of MAC units, respectively, for each of the plurality of different operation modes.

Description

[0001] This application claims the benefit of Korean Patent Application No. 10-2020-0096333, filed with the Korean Intellectual Property Office on July 31, 2020, the entire disclosure of which is incorporated herein by reference for all purposes. Technical Field

[0002] The following description relates to a method and apparatus for deep learning. Background Technology

[0003] Many artificial intelligence applications provide complex but slow-responding services via network connections between servers and clients. One example is that artificial neural networks (a type of deep learning algorithm) typically require 1,000 times or more of computational effort than non-AI processing power on a general-purpose processor (AP). Summary of the Invention

[0004] This summary is provided to introduce, in a simplified form, the selection of concepts that will be further described in the detailed embodiments below. This summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to help determine the scope of the claimed subject matter.

[0005] In one general aspect, a deep learning device includes: a processor configured to support multiple different operating modes, the processor including: a systolic array including multiple multiplier accumulator (ACC) units; and control circuitry configured to control the selection operation of the multiple MAC units and the data movement between the multiple MAC units for each of the multiple different operating modes.

[0006] The control circuit can be configured to control the operation of the accumulators of multiple MAC units based on which of the various operating modes is being implemented by the control circuit.

[0007] The various operating modes may include any combination of two or more of the following: adder tree mode, single instruction multiple data (SIMD) mode, systolic adder tree mode, and systolic mode.

[0008] The processor can be configured to perform depthwise convolution operations in SIMD mode.

[0009] The various operating modes may include at least one of adder tree mode and pulsating adder tree mode, and the control circuit may be configured to operate the accumulator of only one MAC unit among a series of MAC units in response to the control circuit implementing adder tree mode or pulsating adder tree mode.

[0010] The various operating modes may also include at least one of SIMD mode and pulse mode, and the control circuit may be configured to control the adder in the MAC unit to not operate in response to the SIMD mode or pulse mode being implemented by the control circuit.

[0011] One of the adders in the MAC unit can be configured to take into account the adder outputs of adjacent MAC units in response to the control circuit implementing adder tree mode or pulsating adder tree mode.

[0012] Each MAC unit in the series of MAC units, except for the aforementioned MAC unit, may have only one adder, and the aforementioned MAC unit may include multiple adders.

[0013] The various operating modes may include at least one of SIMD mode and pulse mode, and the control circuit may be configured to: in response to the SIMD mode or pulse mode being implemented by the control circuit, control the adder of one MAC unit in a series of MAC units to not operate, and control the other adder of the one MAC unit and all corresponding adders of the other MAC units in the series of MAC units to operate.

[0014] The various operating modes may include another mode, and the adder in one MAC may be configured to: in response to the other mode being implemented by the control circuitry, operate on the adder outputs of adjacent MAC units in the series of MAC units and take into account the adder outputs of adjacent MAC units in the series of MAC units.

[0015] The various operating modes may include at least one of SIMD mode or pulse mode, and the control circuit may be configured to operate the accumulator of all the plurality of MAC units in response to the SIMD mode or pulse mode being implemented by the control circuit.

[0016] The various operating modes may include at least one of adder tree mode and pulsating adder tree mode, and the control circuit may be configured to control the data movement of the operation result from one MAC unit to the adjacent MAC unit for each of the plurality of adjacent MAC units in response to the control circuit implementing adder tree mode or pulsating adder tree mode.

[0017] Data movement can occur in two directions from the corresponding edge MAC unit toward the inward MAC unit of the edge MAC unit, wherein the inward MAC unit can be configured to take into account the corresponding computation results from the two directions.

[0018] The various operating modes may also include another mode, and the control circuit is configured to not perform data movement in response to the other mode being implemented by the control circuit.

[0019] The pulse array is configured to arrange the plurality of MAC cells in the following manner: at least two rows of MAC cells and at least one column of the at least two rows of MAC cells, at least two columns of MAC cells and at least one row of the at least two columns of MAC cells, or two or more rows of MAC cells and two or more columns of MAC cells.

[0020] The various operating modes may include at least one of a pulsating mode and a pulsating adder tree mode, and the control circuit may be configured to control the selection data movement among the plurality of MAC units in response to the implementation of the pulsating mode or the pulsating adder tree mode by the control circuit, so as to send input data from the first MAC unit of the row to the second MAC unit of the row adjacent MAC unit for each row of one or more rows of the pulsating array, and / or send other input data from the third MAC unit to the fourth MAC unit of the column adjacent MAC unit for each column of one or more columns of the pulsating array.

[0021] The control circuit may include a multiplexer disposed in at least one of the plurality of MAC units, wherein the multiplexer may be configured to determine the input of the adder of the at least one MAC unit.

[0022] The control circuit may include an adder tree circuit configured to receive the corresponding outputs of the multipliers of the plurality of MAC units.

[0023] An adder tree circuit may include flip-flops inserted between the adders in the adder tree circuit.

[0024] Each of the plurality of MAC units may be configured in a corresponding processing element (PE) arranged in an array, wherein the first PE of the array may include: a first corresponding portion of the control circuit and a first MAC unit, the first MAC unit having a first adder, a first multiplier and a first accumulator; the second PE of the array may include: a second corresponding portion of the control circuit and a second MAC unit, the second MAC unit having a second adder, a second multiplier and a second accumulator; and the third PE of the array may include: a third corresponding portion of the control circuit and a third MAC unit, the third MAC unit having a third adder, a third multiplier and a third accumulator.

[0025] The various operating modes may include any combination of two or more of the following: adder tree mode, single instruction multiple data (SIMD) mode, pulsating adder tree mode, and pulsating mode. The control circuit may be configured to: control the selection operation of the first MAC unit, the second MAC unit, and the third MAC unit respectively according to which of the various operating modes is indicated by the mode signal provided to the first PE, the second PE, and the third PE; and control the first corresponding part, the second corresponding part, and the third corresponding part to control the pipelined selection of the inputs and / or outputs of the first MAC unit, the second MAC unit, and the third MAC unit.

[0026] A first corresponding portion of the control circuit may include: a first multiplexer configured to determine one of the output of a first accumulator or the output of an adjacent processing element as the input of a first adder; a second corresponding portion of the control circuit may include: a second multiplexer configured to determine one of a predetermined value or the output of a second multiplier as the input of a second adder; and a third corresponding portion of the control circuit may include: a third multiplexer configured to determine one of the output of a third adder or the sum of the output of the third adder and the output of an adjacent processing element as the input of a third accumulator.

[0027] In one general aspect, an apparatus includes: an array of a plurality of processing elements, wherein the plurality of processing elements may include any one or any combination of: a first processing element of the array, the first processing element including a first multiplexer configured to: determine one of the output of an accumulator of the first processing element or the output of an adjacent processing element as the input of an adder of the first processing element; a second processing element of the array, the second processing element including a second multiplexer configured to: determine one of the predetermined value or the output of a multiplier of the second processing element as the input of an adder of the second processing element; and a third processing element, the third processing element including a third multiplexer configured to: determine one of the output of an adder of the third processing element or the sum of the output of the adder of the third processing element and the output of an adjacent processing element as the input of an accumulator of the third processing element.

[0028] The device may include: a first processing element, a second processing element, and a third processing element. The first processing element may include a first multiplexer and a first multiplier-accumulator (MAC) unit. The first multiplexer serves as a first part of the control circuit of the device. The first multiplier-accumulator (MAC) unit has an adder, a multiplier, and an accumulator of the first processing element. The second processing element may include a second multiplexer and a second MAC unit. The second multiplexer serves as a second part of the control circuit. The second MAC unit has an adder, a multiplier, and an accumulator of the second processing element. The third processing element may include a third multiplexer and a third MAC unit. The third multiplexer serves as a third part of the control circuit. The third MAC unit has an adder, a multiplier, and an accumulator of the third processing element.

[0029] Based on the specific operation of the corresponding operating modes of the first multiplexer, the second multiplexer, and the third multiplexer, the device can be configured to enter different selective pipelined operating modes of the inputs and / or outputs of the MAC unit, wherein the different operating modes may include at least two of the following: adder tree mode, single instruction multiple data (SIMD) mode, systolic adder tree mode, and systolic mode.

[0030] In response to the adder tree operation mode of the device, a first multiplexer of a first processing element may be configured to determine the output of an adjacent processing element as the input of the adder of the first processing element, a second multiplexer of a second processing element may be configured to determine the output of the multiplier of the second processing element as the input of the adder of the second processing element, and a third multiplexer of a third processing element may be configured to determine the sum of the output of the adder of the third processing element and the output of an adjacent processing element as the input of the accumulator of the third processing element.

[0031] In response to the adder tree operation mode of the device, the accumulators of the second processing element and the third processing element may not operate.

[0032] In response to the single instruction multiple data (SIMD) operation mode of the device, a first multiplexer of the first processing element may be configured to determine the output of the accumulator of the first processing element as the input of the adder of the first processing element, a second multiplexer of the second processing element may be configured to determine a predetermined value as the input of the adder of the second processing element, and a third multiplexer of the third processing element may be configured to determine the output of the adder of the third processing element as the input of the accumulator of the third processing element.

[0033] The third processing element may further include: a fourth adder configured to add the outputs of adjacent processing elements; and a fifth adder configured to add the output of the adder of the third processing element to the output of the fourth adder.

[0034] In response to the SIMD operating mode of the device, the fourth and fifth adders of the third processing element may not operate.

[0035] In one general aspect, a deep learning method includes: receiving input data; receiving an indication of an operating mode; and, in response to the indicated operating mode, controlling the operation of a plurality of multiplier-accumulator (MAC) units arranged in a systolic array and data movement between the plurality of MAC units.

[0036] The steps of controlling the operation of the plurality of MAC units and the data movement between the plurality of MAC units include: in response to applying an indication of a received operating mode to control circuitry arranged in a pulse array for at least one of two or more of the plurality of MAC units, controlling the operation of the two or more MAC units and / or the data movement between the two or more MAC units.

[0037] The indicated operating mode can be one of a variety of different operating modes, and any two or more of adder tree mode, single instruction multiple data (SIMD) mode, systolic adder tree mode, and systolic mode. For each of the different operating mode indications used in the variety of different operating modes, each operating mode indication can be configured differently for selective use of all components of each MAC unit in one or more of the plurality of MAC units and / or for selective data movement between at least one pair of adjacent MAC units in the plurality of MAC units. Each of the different operating mode indications is applied to one or more MAC units in the plurality of MAC units and / or to the corresponding control circuitry of each of at least one MAC unit in the one or more of the plurality of MAC units.

[0038] The steps of controlling the operation of the plurality of MAC units and the data movement between the plurality of MAC units may include: controlling the use of the accumulators of the plurality of MAC units in response to an indicated operating mode.

[0039] In response to the indicated operating mode, which is either Single Instruction Multiple Data (SIMD) or systolic mode, the control steps can control the operation of MAC units in a series of adjacent MAC units in the systolic array, so as to operate the accumulator of all MAC units in the series of adjacent MAC units.

[0040] In response to the indicated operating mode, which is either adder tree mode or systolic adder tree mode, the control steps can control the operation of MAC units in a series of adjacent MAC units in the systolic array so that only one accumulator of one MAC unit among all the accumulators of all MAC units in the series of adjacent MAC units is used.

[0041] In response to the indicated operating mode, which is either Single Instruction Multiple Data (SIMD) or Pulse mode, the control steps can control the operation of MAC units in the series of adjacent MAC units, such that one MAC unit does not operate on the adder of that MAC unit, wherein the operation of the adder can realize the data movement from one or more other MAC units in the series of adjacent MAC units to the one MAC unit.

[0042] In response to the indicated operating mode being either SIMD mode or pulse mode, the control steps can control the operation of the MAC units in the series of adjacent MAC units to operate the accumulators of all MAC units in the series of adjacent MAC units.

[0043] In response to the indicated operating mode, which is either adder tree mode or pulsating adder tree mode, the controlled steps can control the data movement between MAC units in a series of adjacent MAC units in the pulsating array, so that one of the series of adjacent MAC units receives the operation result from one or more other MAC units in the series of adjacent MAC units.

[0044] In response to the indicated operating mode being either systolic mode or systolic adder tree mode, the controlled steps may control the data movement between MAC units along a row of the systolic array to send first input data from one MAC unit along the row to one or more other MAC units along the row, and / or control the data movement between MAC units along a column of the systolic array to send second input data from one MAC unit along the column to one or more other MAC units along the column.

[0045] In one general aspect, one or more embodiments include a non-transitory computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to perform one or more operations or methods, or all operations or methods, described herein.

[0046] In one general aspect, a deep learning device includes: a processor configured to support multiple different operating modes, the processor comprising: a systolic array including a plurality of multiplier-accumulator (MAC) units arranged in two dimensions of rows and columns, and configurable row input and / or output data communication paths between the plurality of MAC units for each of the plurality of rows, and configurable column input and / or output data communication paths between the plurality of MAC units for each of the plurality of columns; and control circuitry for the systolic array configured to reconfigure the systolic array to operate differently for input data in at least two of the following modes: adder tree mode, single instruction multiple data (SIMD) mode, systolic adder tree mode, and systolic mode.

[0047] The pulsating array may also include multiple MAC units arranged in the third dimension.

[0048] Each of the plurality of MAC units may be included in a respective processing element (PE) arranged in two dimensions of columns and rows, and each of the plurality of PEs may include a control circuit section that, when an operation mode signal is applied, controls the selection and use of components of each of the respective MAC units in the plurality of PEs, and / or controls which of the respective row input data communication paths and / or the respective row output data communication paths are configured to be implemented and not implemented, and / or controls which of the respective column input data communication paths and / or the respective column output data communication paths are configured to be implemented and not implemented.

[0049] When the operating mode is SIMD mode, the input data may include activation feature data, which is input into multiple rows of the plurality of rows and transmitted along each of the plurality of rows using the corresponding implemented row input data communication paths. The input data may include neural network filter weights, which are input into multiple columns of the plurality of columns and transmitted along each of the plurality of columns using the corresponding implemented column input data communication paths. The output data of the systolic array may be provided by the corresponding sum of each of the plurality of columns of the systolic array, each corresponding sum being the sum of the results of multipliers of one or more row processing elements for the corresponding column through the corresponding implemented column output data communication paths, the sum being the result of a depthwise convolution of the activation feature data and the neural network filter weights.

[0050] Other features and aspects will become clear from the following detailed description, drawings, and claims. Attached Figure Description

[0051] Figure 1A An example of an artificial neural network is shown.

[0052] Figure 1B This section shows an example of a method for performing deep learning operations using an adder tree structure.

[0053] Figure 1C An example of a method for performing deep learning operations using a single instruction multiple data (SIMD) structure that includes multiple multiplier accumulator (MAC) units is shown.

[0054] Figure 2 An example of a method for performing deep learning operations is shown.

[0055] Figure 3 An example of the structure of a device configured to perform deep learning operations is shown.

[0056] Figures 4A to 4C Examples of devices configured to perform deep learning operations in adder tree mode and SIMD mode are shown.

[0057] Figures 5A to 5C An example of a device configured to perform deep learning operations is shown.

[0058] Figures 6A to 6B An example of a device configured to perform deep learning operations in systolic adder tree mode and SIMD mode is shown.

[0059] Figure 7A and Figure 7B An example of a device configured to perform deep learning operations in systolic adder tree mode and systolic mode is shown.

[0060] Figure 8 Examples of devices configured to perform deep learning operations in systolic adder tree mode, SIMD mode, and systolic mode are shown.

[0061] Figure 9A and Figure 9B An example of a device configured to perform deep learning operations in systolic adder tree mode and multiple SIMD modes is shown.

[0062] Figure 10 An example of a device configured to perform deep learning operations in systolic adder tree mode, multiple SIMD modes, and systolic mode is shown.

[0063] Figure 11AAn example of a pulsating array arranged in the form of a three-dimensional (3D) array is shown.

[0064] Figure 11B Examples of devices configured to perform deep learning operations, such as convolution, matrix-vector multiplication, and matrix-matrix multiplication, are shown.

[0065] Figure 11C This is a diagram illustrating an example electronic device.

[0066] Throughout the accompanying drawings and detailed embodiments, unless otherwise described or provided, the same reference numerals will be understood to denote the same elements, features, and structures. The drawings may not be to scale, and for clarity, illustration, and convenience, the relative sizes, proportions, and depictions of elements in the drawings may be exaggerated. Detailed Implementation

[0067] The following detailed embodiments are provided to aid the reader in gaining a comprehensive understanding of the methods, apparatus, and / or systems described herein. However, various changes, modifications, and equivalents of the methods, apparatus, and / or systems described herein will become apparent upon understanding this disclosure. For example, the order of operations described herein is merely illustrative and is not limited to the order set forth herein, but may be changed as will become clear upon understanding this disclosure, except for operations that must occur in a specific order. Furthermore, for clarity and conciseness, descriptions of features known upon understanding this disclosure may be omitted.

[0068] The features described herein may be implemented in different forms and should not be construed as limited to the examples described herein. Rather, the examples provided herein are merely to illustrate some of the many feasible ways of implementing the methods, apparatus, and / or systems described herein, which will be clear upon understanding the disclosure of this application.

[0069] The terminology used herein is for the purpose of describing particular examples only and is not intended to limit disclosure. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. As used herein, the term “and / or” includes any one of the associated listed items and any combination of any two or more. As used herein, the terms “comprising,” “including,” and “having” indicate the presence of the stated features, quantities, operations, elements, components, and / or combinations thereof, but do not preclude the presence or addition of one or more other features, quantities, operations, elements, components, and / or combinations thereof. The term “may” (e.g., regarding what an example or embodiment may include or implement) used herein with respect to examples or embodiments indicates the presence of at least one example or embodiment that includes or implements such features, and all examples are not limited thereto.

[0070] Furthermore, terms such as first, second, A, B, (a), (b), etc., may be used herein to describe components. Each of these terms is not intended to define the nature, order, or sequence of the corresponding component, but only to distinguish the corresponding component from one or more other components.

[0071] Throughout this specification, when an element (such as a layer, region, or substrate) is described as being "on" another element, "connected to," or "bonded to" another element, it may be directly "on" said other element, directly "connected to," or directly "bonded to" said other element, or there may be one or more other elements in between. Conversely, when an element is described as being "directly on" another element, directly connected to, or directly "bonded to" another element, there may be no other elements in between. Similarly, expressions such as "between," "directly in direct contact with," and "immediately between," as well as "adjacent to" and "closely adjacent to," can also be interpreted as described above.

[0072] Unless otherwise defined, all terms used herein (including technical and scientific terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains and as understood in this disclosure. Unless expressly defined herein, terms (such as those defined in general dictionaries) shall be interpreted as having the same meaning as they have in the context of the relevant field and in this disclosure, and shall not be interpreted in an idealized or overly formalized sense.

[0073] Furthermore, in the description of the exemplary embodiments, descriptions will be omitted where a detailed description of a structure or function known therefrom after understanding the disclosure of this application would lead to a vague interpretation of the exemplary embodiments. The examples will be described in detail with reference to the accompanying drawings, in which the same reference numerals always denote the same elements.

[0074] Example devices include various types of products or electronic devices (such as data centers, servers, personal computers, laptops, tablets, smartphones, televisions, smart home appliances, smart vehicles, self-service terminals, and wearable devices) configured to perform deep learning operations.

[0075] Figure 1A An example of an artificial neural network is shown. Figure 1B This example demonstrates a method for performing deep learning operations using an adder tree structure. Figure 1C An example of a method for performing deep learning operations using a Single Instruction Multiple Data (SIMD) architecture that includes multiple multiplier-accumulator (MAC) units is shown. The term "unit" described herein refers to a hardware component or a combination of hardware components and instructions.

[0076] Artificial intelligence (AI) algorithms implementing deep learning techniques input training data into an artificial neural network (ANN) to train the ANN using output data and latent labeling information to perform ANN operations (such as convolution). The trained ANN can then be used, for example, to implement such convolutions or one or more other operations to extract features from the input information. In an ANN, nodes are interconnected, for example, through weighted connections and operate together to process the input data. Various types of ANNs exist (e.g., feedforward ANNs, convolutional neural networks (CNNs), recurrent neural networks (RNNs), deep belief networks (DBNs), restricted Boltzmann machines (RBMs), etc.), and as a non-limiting example, any combination of any two or more of these types of ANNs exists. However, the examples are not limited to this. In a feedforward ANN, for example, nodes of the ANN have weighted connections or links to other nodes in the ANN. Such links can extend in one direction (e.g., in the forward direction) through multiple layers of the ANN, where each layer includes multiple nodes, and the weighted connections or links are located between nodes in different layers. In an RNN example, the artificial neural network may also include such weighted connections or links to the same node in the same layer at different times.

[0077] Among the various types of artificial neural networks, CNNs can be used to extract features from input data. For example, a CNN can extract visual features (such as edges, lines, colors, etc.) from an input image. A CNN can include multiple layers, and each layer can receive and process corresponding input data to generate data to be output. For example, the data output from a layer can be a feature map generated by performing a convolution operation between an image or input feature map and the weights (also called kernels) trained on a filter. In one example, the initial layer of a CNN can extract simple features (such as edges or gradients) from the input, and subsequent layers of the CNN can progressively extract more complex features (such as eyes, nose, etc.) from the image.

[0078] Reference Figure 1A The convolution operation 110 may include processing such that a 6×6 single-channel output feature map 115 is generated by performing multiplication and addition operations between an 8×8 three-channel input feature map and a 3×3 three-channel filter 113. The size of the data can be defined by the width, height, and number of channels. For example, this size of the output feature map 115 can also be referred to as the volume.

[0079] The depthwise convolution operation 120 can perform convolution operations only within the same channel, thereby extracting spatial features for each channel. The depthwise convolution operation 120 may include the following process: generating a 6×6 three-channel output feature map 130, comprising output feature maps 127, 128, and 129 corresponding to each input channel, by performing convolution operations for each input channel between an 8×8 three-channel input feature map 111 and each of the three 3×3 filters 124, 125, and 126. Specifically, the output feature map 127 of the first output channel can be generated by performing multiplication and addition operations between the input feature map 121 of the first input channel and the first filter 124. Similarly, the output feature map 128 of the second output channel can be generated by performing multiplication and addition operations between the input feature map 122 of the second input channel and the second filter 125, and the output feature map 129 of the third output channel can be generated by performing multiplication and addition operations between the input feature map 123 of the third input channel and the third filter 126.

[0080] Reference Figure 1B Devices configured to perform deep learning operations can use adder tree structures that can be driven with low power when performing convolution operations. An adder tree structure can include multiple multipliers, adders, and accumulators. The multipliers are configured to compute the product of two data points, the adders are configured to compute the sum of the outputs of two adjacent multipliers or the sum of two adjacent adders, and the accumulators are configured to compute the cumulative sum of the final output data. Adder tree structures using only a small number of accumulators (e.g., a single accumulator) can perform convolution operations with low power.

[0081] For example, the adder tree structure can perform convolution operations between a 5×5 four-channel input feature map 140 and four 3×3 filters 150 (e.g., specifically, between the nine data points 0, 1, 2, 5, 6, 7, 10, 11, and 12 of the input feature map 141 and the weights 0 to 8 of the first filter 151). The multiplier of the adder tree structure can compute the product of the data of the input feature map 141 of the first input channel and the weights of the first filter 151, and the adder of the adder tree structure can accumulate and sum the output values ​​of the multiplier (i.e., 0×0, 1×1, 2×2, 5×3, 6×4, 7×5, 10×6, 11×7, and 12×8).

[0082] Since a typical convolution operation accumulates and sums the output values ​​of the input feature maps for each input channel, multipliers 160, which are not used for the convolution operation between input feature map 141 and filter 151, can be used to perform convolution operations on the input feature map of another input channel. However, depthwise convolution operations perform convolution operations on each input feature map for each input channel. Therefore, when performing depthwise convolution operations using an adder tree structure, multipliers 160 may not be fully utilized, resulting in reduced resource utilization.

[0083] Furthermore, because convolution operations using adder tree structures have long data paths, the device can operate at low clock frequencies. Therefore, adder tree structures are suitable for performing general convolution operations, but may generally not be suitable for operations involving parallel data processing (such as depthwise convolution). Additionally, resource utilization efficiency can be further reduced when adder tree structures are used to perform asymmetric convolution operations using asymmetric filters (such as 7×1, 1×7, 3×1, or 1×3 filters).

[0084] Reference Figure 1C Devices configured to perform deep learning operations can use a SIMD structure that includes multiple MAC units to perform deep convolution operations.

[0085] The SIMD structure includes, for example, a number of processing elements (PEs) 191, 192, 193, and 194 configured to perform the same operations, so that many operations can be performed simultaneously by inputting data into each PE. Each PE in the SIMD structure can be configured as a MAC unit to perform the operation ACC = ACC + (A × B).

[0086] In a SIMD structure, each MAC unit includes an ACC accumulator. Therefore, the SIMD structure is suitable for performing deep convolution operations that perform convolution operations on a per-channel basis. For example, PEs 191, 192, 193, and 194 in the SIMD structure can perform deep convolution operations between 5×5 four-channel input feature maps 171, 172, 173, and 174 and 3×3 filters 181, 182, 183, and 184, respectively. Specifically, PE 191 performs a convolution operation between input feature map 171 and filter 181, PE 192 performs a convolution operation between input feature map 172 and filter 182, PE 193 performs a convolution operation between input feature map 173 and filter 183, and PE 194 performs a convolution operation between input feature map 174 and filter 184. Furthermore, the SIMD structure can have short data paths for computation; therefore, the SIMD structure can operate at high clock frequencies.

[0087] However, such a SIMD structure may require an accumulator for each MAC unit, thus leading to higher power consumption. For example, while a single accumulator is sufficient to perform a general convolution operation using an adder tree structure, when performing a general convolution operation using a SIMD structure, all accumulators are operational, resulting in reduced power efficiency compared to an adder tree structure. Therefore, while SIMD structures are suitable for operations involving parallel data processing (such as depthwise convolution), they may not be suitable for performing general convolution operations.

[0088] As mentioned above, adder tree structures are better suited for performing general convolution operations, but may not be suitable for asymmetric convolution operations using asymmetric filters and operations for parallel data processing (such as depthwise convolution operations). Conversely, SIMD structures are better suited for operations for parallel data processing (such as depthwise convolution operations), but may not be suitable for performing general convolution operations.

[0089] Figure 2 Examples of methods for performing deep learning operations according to one or more embodiments are shown.

[0090] Reference Figure 2 Operations 210 and 220 can be performed by a device configured to perform deep learning operations as described above and below. This device can be implemented by one or more hardware modules, but examples also include the device using various combinations of hardware and instructions implemented or executed by the hardware to implement deep learning operations.

[0091] In operation 210, the device receives an operating mode and input data. The device can be implemented to select an operating mode, wherein the selectable operating modes may include adder tree mode, pulsating adder tree mode, SIMD mode, and / or pulsating mode.

[0092] In operation 220, the device can control the operation of MAC units included in the systolic array and the data movement between MAC units in response to a selected operating mode. The device can perform operations corresponding to the received / selected operating mode. For example, the device can be selected to operate in adder tree mode to perform general convolution operations, and optionally, the device can be selected to operate in SIMD mode to perform depthwise convolution operations. In one example, data movement can occur from individual edge MAC units (e.g., included in...) Figure 4AThe MAC units in PE 401 and 408) move in two directions toward the inward MAC units of the edge MAC units (e.g., from PE 401 to PE 404 and from PE 408 to PE 404), wherein the inward MAC units are configured to consider the respective computation results from the two directions. That is, data movement can occur from the leftmost MAC unit (e.g., included in...) Figure 4A MAC units in PE 401) to any intermediate MAC units (e.g., included in Figure 4A The direction of the MAC unit in PE 404 and from the rightmost MAC unit (e.g., included in Figure 4A It occurs in the direction from the MAC unit in PE 408 to any intermediate MAC unit.

[0093] Specifically, the device can control the operation of MAC units included in the systolic array to operate in a mode determined to be best suited for a predetermined operation. For example, in adder tree mode, the device can control the operation of the MAC units to use only the accumulator of any one of the MAC units included in the systolic array and perform a general convolution operation. In SIMD mode, the device can control the operation of the MAC units to use the accumulators of all MAC units included in the systolic array individually and perform operations for parallel processing of the data (such as depthwise convolution operations).

[0094] Furthermore, the device can control the movement of data between MAC units included in the pulsating array. For example, in adder tree mode, the device can control the movement of data between MAC units to receive computation results from adjacent MAC units.

[0095] Although described in more detail below, in systolic adder tree mode, the device can perform adder tree mode operations while controlling data movement between MAC units, so that the input data of the systolic array is piped to the MAC units arranged along the columns. In one example, the MAC units are arranged at positions corresponding to one or more rows and one or more columns. Furthermore, in systolic mode, the device can control data movement between MAC units to deliver the input data of the systolic array to the MAC units along rows or columns.

[0096] As described herein, the device can support multiple operating modes and perform operations that are determined or predetermined to be suitable for the corresponding operating mode.

[0097] For example, as will be described in more detail below, the device can use a systolic array with a pipelined structure of inputs and / or outputs of adjacent MAC units to support multiple operating modes. Therefore, such a device can include techniques for achieving both high processing speed and high power efficiency.

[0098] Figure 3 An example of the structure of a device configured to perform deep learning operations is shown.

[0099] Reference Figure 3 The device configured to perform deep learning operations includes a processor 300, which is configured to support multiple operating modes. The processor 300 may include multiple physical exciters (PEs), and each PE may be configured as a multiple MAC units 310. The processor 300 may have a cell array structure in which the cells corresponding to the MAC units 310 are arranged in an array structure; specifically, the processor 300 may have a pulsating array structure in which adjacent MAC units 310 are interconnected.

[0100] The processor 300 may include control circuitry 320, configured to control the operation of MAC units 310 included in a pulse array and data movement between MAC units 310 in response to various operating modes. Control circuitry 320 can control MAC units 310 via control paths. However, Figure 3 The device shown is described as an example only, and specific structures (such as the number of MAC units 310 and control paths) may vary in different examples.

[0101] The device can support multiple operating modes using control circuitry 320. Control circuitry 320 can control the operation of the accumulators included in the MAC unit 310 within the pulse array in response to these multiple operating modes. For example, control circuitry 320 can control the operation of the MAC units in response to an adder tree mode so that only the accumulator of one of the MAC units 310 is operated. The following will refer to... Figure 4A The device 410 is described in detail below in its operation in adder tree mode. Furthermore, the control circuit 320 can control the operation of the MAC units in response to SIMD mode, so as to operate the accumulators of all MAC units 310 included in the pulsation array. The following will refer to... Figure 4A The device 420 describes in detail the operation of the device in SIMD mode.

[0102] Figures 4A to 4C Examples of devices configured to perform deep learning operations in adder tree mode and SIMD mode are shown.

[0103] Reference Figure 4AThe device 400, configured to perform deep learning operations, can operate in either adder tree mode or SIMD mode. For example, the device 400 can operate in adder tree mode when a control signal "0" is input, and in SIMD mode when a control signal "1" is input.

[0104] The control circuitry may include a multiplexer (MUX) disposed within the MAC unit to determine the input of the adder. Device 400 may use the multiplexer to control data movement between MAC units. For example, the multiplexer may be disposed individually within each MAC unit, or it may be disposed only within a portion of the MAC units. In the example where the multiplexer is disposed only within a portion of the MAC units, the predetermined multiplexer may control multiple MAC units together.

[0105] Figure 4A The device 400 may include multiple PEs 401 to 408. Each of PEs 401 to 408 may include a MAC unit and control circuitry, and may have different control circuitry depending on the type of PE. (Refer to...) Figure 4B Description can constitute Figure 4A The type of array to process.

[0106] Reference Figure 4B The device 400 may include a combination of a first PE 430, a second PE 440 and / or a third PE 450.

[0107] The first PE 430 may include a first multiplier 431, a first adder 432 and a first accumulator 435 constituting a MAC unit, and a first multiplexer 433 and a first AND gate 434 constituting a control circuit.

[0108] The first multiplexer 433 can determine one of the outputs of the first accumulator 435 and the outputs of the adjacent PEs as the input of the first adder 432. In adder tree mode (e.g., when the control signal "0" is input), the first multiplexer 433 can determine the output of the adjacent PE as the input of the first adder 432. In SIMD mode (e.g., when the control signal "1" is input), the first multiplexer 433 can determine the output of the first accumulator 435 as the input of the first adder 432.

[0109] The second PE 440 may be a PE located at an edge (e.g., the edge of an example array of PEs) and may include a second multiplier 441, a second adder 442 and a second accumulator 445 constituting a MAC unit, as well as a second multiplexer 443 and a second AND gate 444 constituting a control circuit.

[0110] The second multiplexer 443 can determine one of a predetermined value (e.g., "0") and the output of the second multiplier 441 as the input of the second adder 442. In adder tree mode (e.g., when the control signal "0" is input), the second multiplexer 443 can determine the predetermined value (e.g., "0") as the input of the second adder 442. In SIMD mode (e.g., when the control signal "1" is input), the second multiplexer 443 can determine the output of the second multiplier 441 as the input of the second adder 442.

[0111] The third PE 450 is a PE configured to output the final output value in adder tree mode, and may include a third multiplier 451, a third adder 457 and a third accumulator 456 constituting a MAC unit, and a fourth adder 452, a third multiplexer 453, a fourth multiplexer 454, a third AND gate 455, a fifth adder 458 and a fifth multiplexer 459 constituting a control circuit.

[0112] The fifth multiplexer 459 can determine one of the output of the third adder 457 and the sum of the output of the third adder 457 and the output of the adjacent PE as the input of the third accumulator 456.

[0113] In adder tree mode (e.g., when control signal "0" is input), the outputs of adjacent PEs can be input via third multiplexer 453 and fourth multiplexer 454, and fourth adder 452 can add the outputs of adjacent PEs. Furthermore, third adder 457 can add the output of third multiplier 451 and the output of third accumulator 456. Fifth multiplexer 459 can determine the output of fifth adder 458, which adds the outputs of third adder 457 and fourth adder 452, as the input of third accumulator 456. In SIMD mode (e.g., when control signal "1" is input), fifth multiplexer 459 can determine the output of third adder 457 as the input of third accumulator 456. In one embodiment, each of the plurality of PEs 401 to 408 may each include a control circuit.

[0114] Return to reference Figure 4A Equipment 400, PE 401 and 408 may have Figure 4B The second PE 440 structure, PE 402, 403, 405, 406 and 407 may have Figure 4B The first PE 430 structure, and PE 404 may have Figure 4B The structure of the third PE 450.

[0115] Device 410 illustrates an example of operation in adder tree mode when a control signal "0" is input. As described above, when a control signal "0" is input, only the accumulator of the MAC unit included in PE 404 is operational, and the accumulators of the MAC units included in the remaining PEs are not operational. The adders of the MAC units included in PEs 401 and 408 are also not operational. By limiting the operation of the accumulators and adders as described above, adder tree mode reduces power consumption. Furthermore, the multiplexer included in the control circuitry of PEs 402 to 407 can receive data from adjacent MAC units and perform operations as an adder tree. For example, in adder tree mode, device 410 can perform operations similar to those of the MAC units in the control circuitry of PEs 402 to 407. Figure 4C The operations represented by the adder tree are the same as the operations.

[0116] Figure 4A Device 420 illustrates an example of operation in SIMD mode when a control signal "1" is input. As described above, when the control signal "1" is input, the accumulators included in each MAC unit of all PEs 401 to 408 are operational. Simultaneously, only one adder among the adders included in PE 404 is operational, and the other adders included in PE 404 are inoperable. For example, the operation of unused adders can be limited by controlling the output value of the multiplexer connected to the input of the unused adder to "0", thereby reducing power consumption.

[0117] Figures 5A to 5C An example of a device configured to perform deep learning operations is shown.

[0118] When reference Figures 4A to 4C When the described device operates in adder tree mode, the distance from PE 404 to the edge-located PEs 401 and 408 can be relatively large, thus the data path can traverse many adders and multiplexers. Therefore, achieving high timing may be difficult.

[0119] Figures 5A to 5C The device shown can perform adder tree mode operation at high speed without going through a multiplexer by designing the adder tree separately.

[0120] For example, Figure 5A The device may include a systolic array and an adder tree 520, the adder tree 520 being configured to share a multiplier included in a MAC unit in the systolic array.

[0121] For ease of description, the operation of PE 510 among multiple PEs will be described. When the control signal "0" is input, the output of multiplier 511 is transmitted to adder 515 of adder tree 520 via second multiplexer 514, instead of being transmitted to adder 513 of MAC unit via first multiplexer 512. Thus, when the control signal "0" is input, the device can operate in adder tree mode.

[0122] If the control signal "1" is input, the output of multiplier 511 can be transmitted to adder 513 of the MAC unit via the first multiplexer 512, and can be transmitted to adder 515 of adder tree 520 without passing through the second multiplexer 514. Thus, when the control signal "1" is input, the device can operate in SIMD mode.

[0123] Reference Figure 5B The device may include a systolic array and an adder tree 550, the adder tree 550 being configured to share a multiplier and an accumulator of a MAC unit included in the systolic array.

[0124] For ease of description, the operation of PE 530 among multiple PEs will be described. When the control signal "0" is input, the output of multiplier 531 is transmitted to adder 535 of adder tree 550 via second multiplexer 534, instead of being transmitted to adder 533 of MAC unit via first multiplexer 532. The final output of adder tree 550 is transmitted to accumulator 542 of MAC unit via first multiplexer 541 of PE 540. Thus, the device can operate in adder tree mode when the control signal "0" is input.

[0125] If the control signal "1" is input, the output of multiplier 531 can be transmitted to adder 533 of the MAC unit via the first multiplexer 532, and can be transmitted to adder 535 of adder tree 550 without passing through the second multiplexer 534. Thus, when the control signal "1" is input, the device can operate in SIMD mode.

[0126] Reference Figure 5CThe device may have a structure that also includes adder trees with flip-flops (F / F) 560 and 565. Flip-flops 560 and 565 may be positioned between the multipliers included in the systolic array and the accumulators of the adder tree. The device can be configured with synchronous circuitry by adding flip-flops 560 and 565, thereby increasing the operating frequency in adder tree mode. The flip-flops continuously output information input at the edge of a clock signal (e.g., rising or falling edge) until the edge of a subsequent clock signal. When the same clock signal is applied to the flip-flops, the device can operate synchronously with the clock signal. Synchronous circuitry is generally robust to propagation delays or circuit delays compared to asynchronous circuitry; therefore, synchronous circuitry can operate at higher operating frequencies.

[0127] Figure 6A and Figure 6B Examples of devices configured to perform deep learning operations in systolic adder tree mode and SIMD mode are shown. Figure 1 to Figure 5C The description also applies to Figure 6A and Figure 6B Therefore, repeated descriptions will be omitted.

[0128] Reference Figure 6A The device can selectively operate in either pulsating adder tree mode or SIMD mode. For example, the device can operate in pulsating adder tree mode when a control signal "0" is input, and in SIMD mode when a control signal "1" is input.

[0129] Figure 6A The device shown also includes those based on Figure 4A The device shown is configured in a predetermined manner and can be configured to provide a systolic adder tree mode. Therefore, the control circuitry may also include multiple multiplexers 611 and 621 and a data path 612, which, when operating in systolic adder tree mode, transmit input data from the upper systolic array 600 to the lower systolic array 650. The control circuitry can control the operation of the accumulators in the MAC units included in the systolic array and the data movement between MAC units in response to various operating modes. For example, the control circuitry can control the operation of the MAC units included in the systolic array in response to systolic adder tree mode so that only the accumulator of one of the MAC units is operated. Furthermore, the device can multiplex the outputs of the accumulators and output the multiplexed outputs via a shift register (SFT). With this structure, the number of output ports can be reduced.

[0130] The systolic adder tree mode performs the same operations as the adder tree mode, but differs in the data input method. In the systolic adder tree mode, the systolic arrays are arranged in a 2D array, such that the input data (e.g., weights) of the upper systolic array can be transmitted to the lower systolic array, for example, using the corresponding data path 612, along the corresponding column of the arranged PE cells. For ease of description, PEs 610 and 620 among the multiple PEs will be described. Figure 4A Compared to PE 401, PE 620 may further include a multiplexer 621. When a control signal "0" is input, data input to PE 610 via input terminal B is shared to or transmitted to PE 620 via data path 612, and multiplexer 621 can apply the data received via data path 612 and input via input terminal B to the multiplier of PE 620. PEs other than PE 610 and 620 may also operate as described above.

[0131] For example, it can be understood that in the pulsating adder tree mode, the device can, as in Figure 6B Operate as described in the structure. Refer to... Figure 6B It can be seen that when the control signal "0" is input, the input data of the upper pulsation array 600 is transmitted to the lower pulsation array 650.

[0132] Conversely, when the control signal "1" is input, the multiplexer 621 of PE 620 can select a separate input data (or other weights) as input instead of the input data (with the same weights) received through data path 612. Therefore, data input to PE 610 via input terminal B is not transmitted to PE 620. Thus, when the control signal "1" is input, the device can operate in SIMD mode, where data does not move between pulse arrays. PEs other than PEs 610 and 620 can also operate as described above.

[0133] Figure 7A and Figure 7B Examples of devices configured to perform deep learning operations in systolic adder tree mode and systolic mode are shown. Figure 1 to Figure 6B The description also applies to Figure 7A and Figure 7B Therefore, repeated descriptions will be omitted.

[0134] Figure 7A The device can selectively operate in either a pulsating adder tree mode or a pulsating mode. For example, the device can operate in pulsating adder tree mode when a control signal "0" is input, and in pulsating mode when a control signal "1" is input.

[0135] Figure 7AThe device shown also includes devices based on Figure 6A The device shown is in a pre-defined configuration, and a pulse mode can be provided via additional configuration (e.g., instead of...). Figure 6A (SIMD mode). For example, the control circuit may also include multiple multiplexers and data paths that, when operating in pulse mode, transmit the input data of the pulse array in the row direction shown.

[0136] The control circuitry can control the operation of the accumulators in the MAC units included in the pulsating array and the data movement between MAC units in response to a selected operating mode from a variety of operating modes. For example, the control circuitry can control the operation of the MAC units in response to the pulsating mode to operate the accumulators in all MAC units included in the pulsating array. Furthermore, the device can multiplex the outputs of the accumulators and output the multiplexed outputs via SFT. With this structure, the number of output ports can be reduced.

[0137] Figure 7A The pulsation pattern differs from the data input method. Figure 6A The SIMD mode. In systolic mode, input data for the systolic array can be transmitted to MAC units along rows and / or columns. For example, when the systolic array is arranged in a 2D array, first input data (e.g., weights) can be transmitted along columns from the upper systolic array to the lower systolic array, and second input data (e.g., data values ​​of the input feature map) can be transmitted along rows to the adjacent MAC units of each of the upper and lower systolic arrays. For ease of description, PEs 710 and 720 among a plurality of PEs will be described. Figure 6A Compared to PE610, PE 720 may further include a multiplexer 712. When a control signal "1" is input, data input to PE 710 via input terminal A is shared or transmitted to PE 720 via data path 711. The multiplexer 712 can apply the data received via data path 711 and input to input terminal A to the multiplier of PE 720. PEs other than PE 710 and 720 may also operate as described above.

[0138] Reference Figure 7B The device can perform matrix-vector multiplication operations using a pulse mode. For example, in pulse mode, the MAC unit can receive matrix data 730 via input terminal B at the indicated timings (e.g., t1 to t8). Furthermore, the control circuit can, in response to the pulse mode, control the transmission of vector data 740 along their respective rows at the indicated timings (e.g., t1 to t8) to the adjacent MAC units of each of the upper and lower pulse arrays. Thus, the MAC unit can receive vector data via input terminal A at the indicated timings (e.g., t1 to t8).

[0139] The control circuit can control the operation of the MAC units in response to the pulsation pattern, so as to operate the accumulators of all MAC units included in the pulsation array. Thus, the MAC units can perform matrix-vector multiplication operations by accumulating the product of the corresponding matrix data 730 and vector data 740 each time.

[0140] Figure 8 Examples of devices configured to perform deep learning operations in systolic adder tree mode, SIMD mode, and systolic mode are shown. Figure 1 to Figure 7B The description also applies to Figure 8 Therefore, repeated descriptions will be omitted.

[0141] Figure 8 The device can operate in one of the following modes: pulse adder tree mode, SIMD mode, and pulse mode. For example, the device can operate in pulse adder tree mode when a control signal "0" is input, in SIMD mode when a control signal "1" is input, and in pulse mode when a control signal "2" is input.

[0142] For ease of description, PEs 810, 820, and 830 will be described among multiple PEs. For example, as... Figure 8 As shown, control signals can be input to multiplexers 811 and 816 of PE 810, multiplexer 813 of PE 820, and multiplexer 815 of PE 830.

[0143] When control signal "0" is input, if multiplexer 815 of PE 830 selects the input data received via data path 814 as the input to input terminal B of PE 830 according to control signal "0", the input data entering through input terminal B of PE 810 can be shared or transmitted to PE 830 via data path 814. Furthermore, when control signal "0" is input, multiplexer 816 of PE 810 may not share or transmit the output of the multiplier of PE 810 to the adder of PE 810; therefore, multiplexer 816 of PE 810 may not operate in pulsating adder tree mode. Thus, when control signal "0" is input, the device can operate in pulsating adder tree mode, in which the input data (e.g., weights) of the upper pulsating array is transmitted along the columns to the lower pulsating array. PEs other than PE 810 and 830 can also operate as described above.

[0144] When control signal "2" is input, if the multiplexer 813 of PE 820 selects the input data received through data path 812 as the input to input terminal A of PE 820 according to control signal "2", the input data entering through input terminal A of PE 810 is shared or transmitted to PE 820 through data path 812. Therefore, when control signal "2" is input, the device can operate in pulse mode.

[0145] Conversely, when the control signal "1" is input, the multiplexer 813 of PE 820 can select individual input data as input for input terminal A of PE 820, instead of selecting the input data received through data path 812. Therefore, the input data of input terminal A of PE 810 may not be transmitted as input data of input terminal A of PE 820. Thus, when the control signal "1" is input, the device can operate in SIMD mode, in which data does not move between pulse arrays. PEs other than PE 810 and 820 can also operate as described above.

[0146] Figure 9A and Figure 9B Examples of devices configured to perform deep learning operations in systolic adder tree mode and multiple SIMD modes are shown. Figure 1 to Figure 8 The description also applies to Figure 9A and Figure 9B Therefore, repeated descriptions will be omitted.

[0147] Figure 9A The device can operate in one of several SIMD modes and a systolic adder tree mode. For example, the device can operate in systolic adder tree mode when control signal "0" is input, in SIMD mode for performing element-wise addition when control signal "1" is input, in SIMD mode for performing matrix-vector multiplication when control signal "2" is input, and in SIMD mode for performing depthwise convolution when control signal "3" is input.

[0148] Figure 9A The device shown also includes devices based on Figure 6A The device shown is configured as intended, and multiple SIMD modes can be provided instead of a single SIMD mode through additional configurations.

[0149] When control signal "2" is input, the device can perform the following actions: Figure 9B The matrix-vector multiplication operation is shown in the diagram. This device is also applicable to recurrent neural networks (RNNs), where matrix-vector operations are frequently used, and can process data while minimizing time latency, even as batch sizes increase.

[0150] Reference Figure 9A Multiplexer 911 can output a weight selected from the weights received from the pulsating array of another row, based on a control signal. In response to the control signal, multiplexer 912 can selectively output "0", the output of the multiplier of PE 910, and one of the input data received through input terminal B of PE 910. Multiplexer 912 can be operated to perform element-wise addition of input data by outputting the input data received through input terminal B of PE 910 to the adder of PE 910. In response to the control signal, multiplexer 913 can selectively output one of the input data received through input terminal A of PE 910, "0", and the output of the accumulator (ACC) of PE 910. In response to the control signal, multiplexer 914 can selectively output one of the output of the ACC of PE 910, the output of the adder of PE 910, and the output of the multiplier of PE 910.

[0151] Figure 10 Examples of devices configured to perform deep learning operations in systolic adder tree mode, multiple SIMD modes, and systolic mode are shown. Figure 1 to Figure 9B The description also applies to Figure 10 Therefore, repeated descriptions will be omitted.

[0152] Figure 10 The device can operate in one of the following modes: systolic adder tree mode, multiple SIMD modes, and systolic mode. For example, the device can operate in systolic adder tree mode when control signal "0" is input, in SIMD mode for performing element-wise addition when control signal "1" is input, in SIMD mode for performing matrix-vector multiplication when control signal "2" is input, in SIMD mode for performing depthwise convolution when control signal "3" is input, and in systolic mode when control signal "4" is input.

[0153] Figure 10 The device shown also includes devices based on Figure 9A The predetermined configuration of the device shown, and Figure 10 The device shown can also provide a pulse mode via additional configuration.

[0154] This device can use multiplexers to reduce the number of shift registers at the output. For example, in element-wise addition and matrix-vector multiplication modes, an output may need to be fetched within one cycle. If the clock frequency applied to the shift register is increased to N times the frequency applied to the MAC unit, and N:1 multiplexers are used, an output can be shifted within one cycle.

[0155] The systolic mode can be used for applications that require input-stationary, weight-stationary, output-stationary, or various combinations thereof. High MAC utilization can also be achieved when processing depthwise convolution operations in systolic mode.

[0156] Compared to the systolic mode, the systolic adder tree mode can be driven with lower power. Furthermore, compared to the systolic mode, the SIMD mode has higher MAC utilization and can achieve relatively high MAC utilization in depthwise convolution operations.

[0157] When the size of the filter (e.g., width (w) × height (h) × number of channels (c)) is greater than the horizontal / vertical length of the MAC unit, operation in systolic mode may be efficient. On the other hand, it can take h × w × c × 2 times to move the output to the shift register, so it may not be efficient when the size of the MAC unit is relatively large. Therefore, Figure 10 The device shown may be efficient operating in systolic mode in the first layer of the neural network. However, the power efficiency in the remaining layers may be slightly reduced due to the relatively high power consumption of the MAC array.

[0158] Figure 9A The device shown may have a higher capacity than Figure 10 The device shown has high power efficiency.

[0159] Because the MAC unit can handle element-wise addition and matrix-vector multiplication, Figure 9A The device shown can have a relatively high overall processing rate. However, since element-wise addition is expected to have high bandwidth, a high clock frequency is also expected to fetch the output; therefore, the example may be configured with an additional multiplexer inserted for element-wise addition. Thus, in such an example, Figure 9A The device shown may therefore have increased power consumption due to the insertion of a multiplexer.

[0160] Because Figure 9A The device shown is configured to provide multiple SIMD modes, so Figure 7A The device shown may have a higher capacity than Figure 9A The device shown has high power efficiency. Figure 7A The device shown can receive input data differently for each MAC unit, therefore, Figure 7A The device shown can achieve relatively higher MAC utilization compared to a pure adder tree structure. Furthermore, compared to pulsating mode, Figure 7AThe device shown can reduce the time spent filling the MAC unit with data, therefore, Figure 7A The device shown can have high MAC utilization.

[0161] Figure 11A An example of a pulsating array arranged in the form of a three-dimensional (3D) array is shown. Figure 11B Examples of devices configured to perform deep learning operations, such as convolution, matrix-vector multiplication, and matrix-matrix multiplication, are shown. Figure 11A and Figure 11B In different examples, the locations of the input feature map (IFM) (or activation (ACT)) memory and weight (WGT) memory can vary.

[0162] Figure 11A The device can improve the processing speed of convolution operations, matrix-vector multiplication operations, and matrix-matrix multiplication operations by arranging systolic arrays in an example 3D form.

[0163] For example, Figure 11B The device may also include a direct memory access (DMA) unit 1110, a controller 1120, an SRAM cluster 1130, and a normalized lattice filter (NLF) 1140. The device can use a means capable of moving data (such as DMA 1110 or a central processing unit (CPU)) to apply data to desired rows or columns and read the output results. The following... Figure 11C The processor can be, for example, a CPU. Furthermore, the device can connect the outputs of two or more rows or columns to add or accumulate the resulting values ​​of some rows or columns.

[0164] Figure 11C This is a diagram illustrating an example electronic device. Here, Figures 1A to 11C Any of the devices may also be referred to as a deep learning device or apparatus with a corresponding configuration having deep learning capabilities for training and / or inference operations.

[0165] Electronic device 1100 can represent the above-mentioned... Figures 1A to 11B The described devices are any one, any combination, or all of those configured for deep learning computations. In another example, Figure 11C The neural processor 1150 can represent the above-mentioned... Figures 1A to 11B Any one, any combination, or all of the devices described. As a non-limiting example, in the various corresponding examples, electronic device 1100 can be any of a data center, server, personal computer, laptop computer, tablet computer, smartphone, television, smart home appliance, smart vehicle, self-service terminal, and wearable device.

[0166] Reference Figure 11C The electronic device 1100 may include a processor 1160, a neural processing unit (NPU) 1150, a memory 1165, a communication device 1170, a storage device 1175, a communication bus 1180, an input device 1185, and an output device 1190.

[0167] Processor 1160 can control the overall operation of electronic device 1100 and execute functions and instructions within electronic device 1100. For example, processor 1160 may be or include a CPU. For example, processor 1160 may be configured to interact with NPU 1150 to execute the functions and instructions described above with reference to Figure 1 to... Figure 11B One or more operations or methods are described. In one example, as a non-limiting example, the NPU 1150 may be configured to perform the operations described above, based on input / activation information from the processor 1160 and core weights from the memory 1165. (Ref. 1 to 1) Figure 11B One or more of the operations or methods described herein. Another example exists without NPU1150, and processor 1160 may correspond to the processor or device described herein, and processor 1160 is configured to perform, for example, the operations described above with reference to Figure 1 to... Figure 11B One or more operations or methods described.

[0168] Memory 1165 may store information for the processor 1160 and / or NPU 1150 to perform various training or training operation objectives, i.e., examples of inference operations using trained weights and / or examples of training operations that generate one or more or all of the trained weights through iterative operations, as described herein, representing deep learning operations. Memory 1165 may also store instructions to be executed by the processor 1160 and / or NPU 1150, and related information during the execution of software or applications in electronic device 1100. Memory 1165 may include, for example, random access memory (RAM), dynamic RAM (DRAM), static RAM (SRAM), or other types of non-volatile memory known in the art.

[0169] Storage device 1175 may include a computer-readable storage medium or a computer-readable storage device. Compared to memory 1165, storage device 1175 can store a larger amount of information over a longer period of time. Storage device 1175 may include, for example, magnetic hard disks, optical disks, flash memory, floppy disks, electrically erasable programmable read-only memory (EEPROM), and other types of non-volatile memory known in the related art.

[0170] For example, input device 1185 may receive input from a user. Input device 1185 may include, for example, a keyboard, mouse, touchscreen, camera, microphone, and other devices that can detect input from the user.

[0171] Output device 1190 can be based on this for Figures 1A to 11B The output of any device in the described apparatus is provided to, for example, a user via a visual, auditory, or tactile channel. Output device 1190 may include a display, touchscreen, speaker, and other devices that can provide output to a user.

[0172] Communication device 1170 can communicate with external devices via wired or wireless networks. Communication device 1170 can receive data or information from external devices and send data or information to external devices. Communication bus 1180 provides communication between these components of electronic device 1100.

[0173] In this regard Figures 1A to 11CThe described DMA 1110, controller 1120, SRAM cluster 1130, NLF 1140, processor, PE unit, MAC unit, accumulator, multiplexer, adder, multiplier, flip-flop, gate, shift register, timing clock, two-dimensional array, three-dimensional array, electronic device 1100, processor 1160, NPU 1150, memory 1165, storage device 1175, communication device 1170, communication bus 1180, input device 1185, and output device 1190, as well as other devices, apparatuses, units, modules, and components, are implemented through hardware components. Examples of hardware components that can be used to perform the operations described in this application include, where appropriate, controllers, sensors, generators, drivers, memory, comparators, arithmetic logic units, adders, subtractors, multipliers, dividers, integrators, and any other electronic components configured to perform the operations described in this application. In other examples, one or more hardware components performing the operations described herein are implemented using computing hardware (e.g., one or more processors or computers). A processor or computer may be implemented using one or more processing elements (such as logic gate arrays, controllers and arithmetic logic units, digital signal processors, microcomputers, programmable logic controllers or units, field-programmable gate arrays, programmable logic arrays, microprocessors, or any other means or combination of means configured to respond to and execute instructions in a defined manner to achieve a desired result). In one example, the processor or computer includes or is connected to one or more memories storing instructions or software executed by the processor or computer. The hardware components implemented by the processor or computer can execute instructions or software (such as an operating system (OS) and one or more software applications running on the OS) for performing the operations described herein. The hardware components may also access, manipulate, process, create, and store data in response to the execution of instructions or software. For brevity, the singular terms “processor” or “computer” may be used in the description of the examples described herein; however, in other examples, multiple processors or computers may be used, or a processor or computer may include multiple processing elements or multiple types of processing elements or both. For example, a single hardware component or two or more hardware components may be implemented using a single processor, or two or more processors, or a processor and a controller. One or more hardware components may be implemented using one or more processors, or a processor and a controller, and one or more other hardware components may be implemented using one or more other processors, or additional processors and additional controllers. One or more processors, or processors and controllers, may implement a single hardware component or two or more hardware components.The hardware components can have any one or more different processing configurations, examples of which include: a single processor, a discrete processor, a parallel processor, a single instruction single data (SISD) multiprocessing, a single instruction multiple data (SIMD) multiprocessing, multiple instruction single data (MISD) multiprocessing, and multiple instruction multiple data (MIMD) multiprocessing.

[0174] Figures 1A to 11C The methods for performing the operations described in this application, as shown, can be executed by computing hardware (e.g., by one or more processors or a computer), which is implemented to execute instructions or software as described above to perform the operations performed by the methods described in this application. For example, a single operation or two or more operations can be executed by a single processor or two or more processors, or a processor and a controller. One or more operations can be executed by one or more processors, or a processor and a controller, and one or more other operations can be executed by one or more other processors, or additional processors and additional controllers. One or more processors, or a processor and a controller, can execute a single operation or two or more operations.

[0175] Instructions or software for controlling computing hardware (e.g., one or more processors or computers) to implement hardware components and perform the methods described above can be written as computer programs, code segments, instructions, or any combination thereof to individually or collectively instruct or configure one or more processors or computers, such as machines or special-purpose computers, to perform the operations performed by the hardware components and methods described above. In one example, the instructions or software include machine code (such as machine code generated by a compiler) that is directly executed by one or more processors or computers. In another example, the instructions or software include high-level code that is executed by one or more processors or computers using an interpreter. The instructions or software can be written using any programming language based on the block diagrams and flowcharts shown in the accompanying drawings and the corresponding description used herein, which disclose algorithms for performing the operations performed by the hardware components and methods described above.

[0176] Instructions or software for controlling computing hardware (e.g., one or more processors or computers) to implement hardware components and perform the methods described above, along with any associated data, data files, and data structures, may be recorded, stored, or fixed in, or on, one or more non-transitory computer-readable storage media. Examples of non-transitory computer-readable storage media include: read-only memory (ROM), random access programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, non-volatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blu-ray or optical disc storage, hard disk drive (HDD), solid-state drive (SSD), card-type storage (such as multimedia cards or microcards (e.g., Secure Digital (SD) or Extreme Digital (XD))), magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid-state drive, and any other device configured to store instructions or software and any associated data, data files, and data structures in a non-transitory manner and to provide said instructions or software and any associated data, data files, and data structures to one or more processors or computers, such that one or more processors or computers can execute the instructions. In one example, the instructions or software and any associated data, data files, and data structures are distributed across a networked computer system, such that the instructions and software and any associated data, data files, and data structures are stored, accessed, and executed by one or more processors or computers in a distributed manner.

[0177] While this disclosure includes specific examples, it will be clear upon understanding this disclosure that various changes in form and detail may be made in these examples without departing from the spirit and scope of the claims and their equivalents. The examples described herein should be considered descriptive only and not for limiting purposes. The description of features or aspects in each example will be considered applicable to similar features or aspects in other examples. Suitable results may be achieved if the described techniques are performed in a different order, and / or if components in the described system, architecture, apparatus, or circuit are combined in a different manner, and / or replaced or supplemented by other components or their equivalents.

Claims

1. A device for deep learning, the device comprising: The processor is configured to support multiple different operating modes, the processor comprising: A pulsating array, comprising multiple multiplier-accumulator units; and The control circuit is configured to control the operation of the plurality of multiplier-accumulator (MAC) units and the data movement between the plurality of MAC units for each of the various operating modes. Each of the plurality of MAC units is configured in a corresponding processing element arranged in a pulse array. The first processing element of the pulsating array includes a first corresponding portion of a control circuit and a first MAC unit. The first corresponding portion of the control circuit includes a first multiplexer configured to determine one of the output of a first accumulator of the first processing element and the output of a processing element adjacent to the first processing element as the input of a first adder of the first processing element. The second processing element of the pulsating array includes a second corresponding portion of a control circuit and a second MAC unit. The second corresponding portion of the control circuit includes a second multiplexer configured to determine one of a predetermined value and the output of a second multiplier of the second processing element as the input of a second adder of the second processing element. The third processing element of the pulsating array includes a third corresponding part of the control circuit and a third MAC unit. The third corresponding part of the control circuit includes a third multiplexer configured to determine one of the output of the third adder of the third processing element and the sum of the output of the third adder of the third processing element and the output of the processing element adjacent to the third processing element as the input of the third accumulator of the third processing element.

2. The device according to claim 1, wherein, The control circuit is configured to control the operation of the accumulator of at least two of the plurality of MAC units, depending on which of the various operating modes is being implemented by the control circuit.

3. The device according to claim 1, wherein, The various operating modes include any combination of two or more of the following: adder tree mode, single instruction multiple data mode, systolic adder tree mode, and systolic mode.

4. The device according to claim 3, wherein, The processor is configured to perform depthwise convolution operations in single instruction multiple data mode.

5. The device according to claim 1, wherein, The various operating modes include at least one of adder tree mode and pulsating adder tree mode, and the control circuit is configured to operate the accumulator of only one MAC unit among a series of MAC units in response to the control circuit implementing adder tree mode or pulsating adder tree mode.

6. The device according to claim 5, wherein, The various operating modes also include at least one of Single Instruction Multiple Data (SMD) mode and Pulse mode, and the control circuit is configured to control one adder in the MAC unit to operate in response to the implementation of SMD or Pulse mode by the control circuit, and to control the other adders in the MAC unit not to operate.

7. The device according to claim 6, wherein, The adder of at least one of the series of MAC units is configured to take into account the adder output of the MAC unit adjacent to the at least one MAC unit in response to the control circuit implementing adder tree mode or pulsating adder tree mode.

8. The device according to claim 5, wherein, All MAC units in the series except for the aforementioned MAC unit have only one adder, and the aforementioned MAC unit includes multiple adders.

9. The device according to claim 1, wherein, The various operating modes include at least one of Single Instruction Multiple Data (SMD) mode and Pulse mode, and the control circuit is configured to: in response to the implementation of SMD or Pulse mode by the control circuit, control one adder of one MAC unit in a series of MAC units to operate, control the other adders of the one MAC unit to not operate, and control all the respective adders of the other MAC units in the series of MAC units to operate.

10. The device according to claim 9, wherein, The various operating modes also include another mode, and The adder in one of the MAC units is configured to: in response to the other mode being implemented by the control circuit, operate on the adder outputs of the MAC units adjacent to the one MAC unit in the series of MAC units and take into account the adder outputs of the MAC units adjacent to the one MAC unit in the series of MAC units.

11. The device according to claim 1, wherein, The various operating modes include at least one of Single Instruction Multiple Data (SMD) mode and Pulse mode, and the control circuit is configured to operate the accumulator of all the plurality of MAC units in response to the implementation of SMD or Pulse mode by the control circuit.

12. The device according to claim 1, wherein, The various operating modes include at least one of: adder tree mode and pulsating adder tree mode, and The control circuit is configured to: in response to an adder tree mode or a pulsed adder tree mode implemented by the control circuit, control the data movement of the operation result from one MAC unit to the MAC unit adjacent to the one MAC unit for each of a plurality of adjacent MAC units.

13. The device according to claim 12, wherein, Data movement occurs in two directions from each edge MAC unit toward the inward MAC unit of the edge MAC unit, wherein the inward MAC unit is configured to consider the respective computation results from the two directions.

14. The device according to claim 12, wherein, The various operating modes also include another mode, and the control circuit is configured to not perform data movement in response to the other mode being implemented by the control circuit.

15. The device according to claim 1, wherein, The pulsating array is configured to arrange the plurality of MAC units at positions corresponding to one or more rows and one or more columns.

16. The device according to claim 15, wherein, The various operating modes include at least one of: pulsating mode and pulsating adder tree mode, and The control circuit is configured to: in response to a pulsation mode or a pulsation adder tree mode being implemented by the control circuit, control the data movement between the plurality of MAC units to send input data from a first MAC unit in one or more rows of the pulsation array to a second MAC unit adjacent to the first MAC unit in the row for each row, and / or send additional input data from a third MAC unit in one or more columns of the pulsation array to a fourth MAC unit adjacent to the third MAC unit in the column for each column.

17. The device according to claim 1, wherein, The control circuit includes a multiplexer disposed in at least one of the plurality of MAC units, wherein the multiplexer is configured to determine the input of the adder of the at least one MAC unit.

18. The device according to claim 1, wherein, The control circuit includes an adder tree circuit configured to receive the corresponding outputs of the multipliers of the plurality of MAC units.

19. The device according to claim 18, wherein, An adder tree circuit includes flip-flops inserted between the adders in the adder tree circuit.

20. The device according to claim 1, wherein, The first MAC unit has a first adder, a first multiplier, and a first accumulator. The second MAC unit has a second adder, a second multiplier, and a second accumulator. The third MAC unit has a third adder, a third multiplier, and a third accumulator.

21. The device according to claim 20, wherein, The various operating modes include: adder tree mode, single instruction multiple data mode, systolic adder tree mode, and any combination of two or more of the systolic modes, and The control circuit is configured to: control the operation of the first MAC unit, the second MAC unit, and the third MAC unit respectively according to which of the various different operating modes is indicated by the mode signal provided to the first processing element, the second processing element, and the third processing element; and control the first corresponding part, the second corresponding part, and the third corresponding part to control the pipelined input and / or output of the first MAC unit, the second MAC unit, and the third MAC unit.

22. An apparatus for deep learning, comprising: An array of multiple processing elements The plurality of processing elements includes one or more of the following: The array has a first processing element, which includes a first multiplexer configured to determine one of the output of an accumulator of the first processing element and the output of a processing element adjacent to the first processing element as the input of a first adder of the first processing element. The array has a second processing element, which includes a second multiplexer configured to: determine one of the predetermined value and the output of the multiplier of the second processing element as the input of the second adder of the second processing element; and The third processing element includes a third multiplexer configured to determine one of the output of the third adder of the third processing element and the sum of the output of the third adder of the third processing element and the output of the processing element adjacent to the third processing element as the input of the accumulator of the third processing element.

23. The apparatus according to claim 22, wherein, The device includes: a first processing element, a second processing element, and a third processing element. The first processing element includes a first multiplexer and a first multiplier-accumulator (MAC) unit. The first multiplexer serves as a first part of the control circuit of the device. The first MAC unit includes a first adder, a multiplier, and an accumulator of the first processing element. The second processing element includes a second multiplexer and a second MAC unit. The second multiplexer serves as the second part of the control circuit, and the second MAC unit includes a second adder, a multiplier, and an accumulator of the second processing element. The third processing element includes a third multiplexer and a third MAC unit. The third multiplexer serves as the third part of the control circuit, and the third MAC unit has a third adder, a third multiplier, and a third accumulator of the third processing element.

24. The apparatus according to claim 23, wherein, Based on the specific operating modes of the first, second, and third multiplexers, the device is configured to enter different pipelined operating modes via the inputs and / or outputs of adjacent MAC units, and... The different operating modes include at least two of the following: adder tree mode, single instruction multiple data mode, systolic adder tree mode, and systolic mode.

25. The apparatus according to claim 22, wherein, In response to the adder tree operation mode of the device, The first multiplexer of the first processing element is configured to determine the output of the processing element adjacent to the first processing element as the input of the first adder of the first processing element. The second multiplexer of the second processing element is configured to: determine a predetermined value as the input of the second adder of the second processing element, and The third multiplexer of the third processing element is configured to determine the sum of the output of the third adder of the third processing element and the output of the processing element adjacent to the third processing element as the input of the accumulator of the third processing element.

26. The apparatus according to claim 25, wherein, In response to the adder tree operation mode of the device, The accumulators of the second processing element and the first processing element do not operate.

27. The apparatus according to claim 22, wherein, In response to the single instruction multiple data operation mode of the device, The first multiplexer of the first processing element is configured to: determine the output of the accumulator of the first processing element as the input of the first adder of the first processing element. The second multiplexer of the second processing element is configured to determine the output of the multiplier of the second processing element as the input of the second adder of the second processing element. The third multiplexer of the third processing element is configured to determine the output of the third adder of the third processing element as the input of the accumulator of the third processing element.

28. The apparatus according to claim 22, wherein, The third processing element also includes: A fourth adder is configured to add the outputs of the processing element adjacent to the third processing element; and The fifth adder is configured to add the output of the third adder of the third processing element to the output of the fourth adder.

29. The apparatus according to claim 28, wherein, In response to the single instruction multiple data operation mode of the device, The fourth and fifth adders of the third processing element do not operate.

30. A method for deep learning, the method comprising: Receive input data; Receive instructions on the operating mode; as well as In response to the indicated operating mode, the operation of multiple multiplier-accumulator (MAC) units arranged in a pulsating array and the data movement between the multiple MAC units are controlled. In response to the indicated operating mode being either adder tree mode or systolic adder tree mode, the control steps include: controlling the operation of MAC units in a series of adjacent MAC units in the systolic array so as to use only one accumulator of one MAC unit among all accumulators of all MAC units in the series of adjacent MAC units, wherein, when the indicated operating mode is adder tree mode or systolic adder tree mode, the operation of the adder will realize the data movement from one or more other MAC units in the series of adjacent MAC units to the one MAC unit. The operation mode in response to the instruction is either Single Instruction Multiple Data (SID) or Pulse mode. The control steps include: controlling the operation of the MAC units in the series of adjacent MAC units, such that one adder of the MAC unit operates, the other adders of the MAC unit do not operate, and the accumulators of all MAC units in the series of adjacent MAC units operate.

31. The method according to claim 30, wherein, The steps of controlling the operation of the plurality of MAC units and the data movement between the plurality of MAC units include: in response to applying an indication of a received operating mode to control circuitry arranged in a pulse array for at least one of two or more of the plurality of MAC units, controlling the operation of the two or more MAC units and / or the data movement between the two or more MAC units.

32. The method according to claim 31, wherein, The indicated operating mode is one of several different operating modes, including: adder tree mode, single instruction multiple data mode, systolic adder tree mode, and any combination of two or more of the systolic modes. Specifically, for different operation mode indications used for the various different operation modes, each of the different operation mode indications is configured differently for the use of all components of each of the one or more MAC units in the plurality of MAC units and / or for data movement between at least one pair of adjacent MAC units in the plurality of MAC units, wherein each of the different operation mode indications is applied to one or more MAC units in the plurality of MAC units and / or to each corresponding control circuit of at least one of the one or more MAC units.

33. The method according to claim 30, wherein, The steps of controlling the operation of the plurality of MAC units and the data movement between the plurality of MAC units include: controlling the use of the accumulators of the plurality of MAC units in response to an indicated operating mode.

34. The method according to claim 30, wherein, In response to the indicated operating mode being either adder tree mode or systolic adder tree mode, the control steps further include: controlling data movement between MAC units in a series of adjacent MAC units in the systolic array, so that one of the series of adjacent MAC units receives the operation result from one or more other MAC units in the series of adjacent MAC units.

35. The method according to claim 30, wherein, In response to the indicated operating mode, which is either pulse mode or pulse adder tree mode, the control steps also include: Control the data movement between MAC units along a row of a pulsating array so as to send first input data from one MAC unit along the row to one or more other MAC units along the row, and / or Control the movement of data between MAC cells along a column of a pulsating array so as to send second input data from one MAC cell along the column to one or more other MAC cells along the column.

36. A non-transitory computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to perform the method of any one of claims 30 to 35.

37. An apparatus for deep learning, the apparatus comprising: The processor is configured to support multiple different operating modes, the processor comprising: A pulsating array includes multiple multiplier-accumulator (MAC) units arranged in two dimensions, rows and columns; configurable row input and / or output data communication paths between the MAC units for each row; and configurable column input and / or output data communication paths between the MAC units for each column. The control circuitry of the systolic array is configured to reconfigure the systolic array to operate differently in at least two of the following modes: adder tree mode, single instruction multiple data mode, systolic adder tree mode, and systolic mode.

38. The device according to claim 37, wherein, The pulsating array also includes multiple MAC units arranged in the third dimension.

39. The device according to claim 37, wherein, Each of the plurality of MAC units is included in a corresponding processing element arranged in two dimensions, columns and rows, and each of the plurality of processing elements includes a control circuit section, which, when an operation mode signal is applied, Control the use of components in each MAC unit of the respective MAC unit in the plurality of processing elements, and / or The corresponding row input data communication path and / or the corresponding row output data communication path are configured to be implemented or not implemented, respectively, and / or the corresponding column input data communication path and / or the corresponding column output data communication path are configured to be implemented or not implemented.

40. The device according to claim 39, wherein, When the operating mode is Single Instruction Multiple Data mode The input data includes activation feature data, which is input into at least two of the plurality of rows, and transmitted along each of the at least two rows using the corresponding implemented row input data communication paths. The input data includes neural network filter weights, which are respectively input into at least two of the plurality of columns and transmitted along each of the at least two columns using the corresponding implemented column input data communication paths. The output data of the systolic array is provided by the corresponding sum of each of the at least two columns of the systolic array, each corresponding sum being the sum of the results of one or more multipliers in the processing elements of the corresponding column through the implemented corresponding column output data communication path, the sum being the result of a deep convolution of activation feature data and neural network filter weights.

Images